RePO MAN – Remote Pilotage Using Mixed Reality
The challenge
Nearly 1 in 5 pilot transfers in European waters fail to meet safety standards, with non-compliant pilot ladder arrangements contributing to serious and sometimes fatal accidents. Traditional pilot boarding methods expose maritime pilots to significant physical risks, especially in poor weather. These challenges can delay operations and compromise safety for both pilots and crew.
The innovation
The RePO MAN (Remote Pilotage – Operational, innovative and Manageable Alternatives for Navigation routines) system offers a safer, smarter alternative. Developed by the Fraunhofer Center for Maritime Logistics, it uses a mixed reality solution combining Augmented Reality (AR) and Virtual Reality (VR) to enable shore-based pilots to guide vessels remotely - without boarding.
- Onboard AR: Crew members use Microsoft HoloLens 2 to view real-time navigational data overlaid in their field of vision, with intuitive hand-tracking for interaction.
- Shore-based VR: Pilots operate in a virtual environment with 360° video feeds, AIS data, radar, and electronic charts, maintaining full situational awareness.
- Integrated communication: Visual markers and VoIP tools allow pilots to guide crews with precision, enhancing coordination and safety.
How it works
The system uses secure, real-time data transmission via MQTT protocol. Onboard, 360° cameras feed live visuals to the pilot’s VR interface. The AR application, built with Unity and MRTK3, anchors holograms spatially to ensure accurate positioning. Pilots can interact with the vessel environment remotely, providing guidance as if they were physically present.
Case study
Testing confirmed stable data transmission, accurate hologram positioning, and effective real-time communication. Pilots reported high situational awareness and successful navigation support during trials. The AR interface performed reliably in varying light conditions, and gesture recognition enabled seamless interaction. The system proved especially effective in controlled environments, laying the groundwork for broader deployment.
Potential impact
RePO MAN has the potential to transform maritime pilotage by:
- Eliminating physical transfer risks
- Maintaining pilotage services in adverse weather
- Improving safety and operational continuity
- Reducing delays and costs associated with traditional pilot boarding
As global shipping demands grow, RePO MAN offers a scalable, future-ready solution that enhances safety and efficiency in maritime navigation.
- Date
- 22/07/2025
