Mixing realities for lunar exploration

Anyone who has tried virtual reality (VR) applications knows about its many possibilities: from immersive video games and 360-degree movies, to virtual museum tours, meetings, or journeys to space – all from the comfort of your home.

ESA’s extended reality (XR) lab at the European Astronaut Centre goes one step further and uses virtual, augmented and mixed reality to train astronauts for missions to the International Space Station and beyond. This includes virtual tours of the Space Station to familiarise astronauts with its modules and learning how to operate a robotic arm.

Lora-Line Faure applied her background in cognitive sciences to prepare training for the Moon. During her internship at the XR lab, she developed MIRAGE, or Mixed Interactive Rendering with Gesture recognition for Enhanced hand-tracked LUNA augmented experience.

This application uses a hand-tracking system to visualise and interact with objects in mixed reality. Mixed reality allows users to interact with both digital and physical objects. Users wear a virtual reality headset and see a menu displayed on their hand, allowing them to place and interact with various objects, such as a lunar lander or rover projected directly in front of them.

Landing on and exploring our Moon, as planned with the ESA-powered Artemis missions and the construction of the Gateway station in lunar orbit, will help us understand the formation of life on Earth and the Universe as a whole. Discovering resources like water or minerals on the Moon can also provide valuable insights for sustainable and long-distance space missions to Mars and beyond in the future.

Currently, MIRAGE aims to be used on Earth for visualisation and training in the ESA-DLR LUNA analogue facility, which is currently being built next to ESA’s Astronaut Centre in Cologne, Germany.

LUNA will be a testbed for rovers and astronauts to determine the best systems, tools, and operations for working and living on the Moon. The facility recreates the lunar surface and will be filled with simulated Moon dust, called regolith, and can recreate lunar day and night cycles, as well as reduced gravity through a suspension system. The level of gravity on the Moon is about one sixth of Earth's.

In the future, Lora-Line’s interactive system could also be applied to hand tracking for surface operations using mixed reality displays for astronauts on the Moon.

Engineers and scientists could use it to visualise models or run simulations without the need for building real size models, making quick and low-cost adjustments possible.

Astronauts could benefit from more immersive training and simulations, combining a real environment with virtual elements. It is also valuable for showing 3D models to help users get a better sense of proportion, as with the Argonaut lunar lander.

For Lora-Line, a challenge of this new approach was to accurately track the real world with the headset and to introduce hand tracking to people unfamiliar with the technology.

“I'm happy to have had the chance to learn, experiment, and develop such an innovative and stimulating application,” she reflected. For her, the internship was a full success, and she plans to continue the path of turning virtual concepts into reality.

Click here to see a video demonstration of MIRAGE.

Credits: ESA - L.-L. Faure

Taken in: Germany / Northrhine-Westphalia / Rhein-Sieg-Kreis / Altenrath (show map)

Taken on: June 14, 2024

Tags:

ESA

European Space Agency

Universe

Cosmos

Space Science

Science

Space Technology

Tech

Technology

Virtual Reality

Mixing reality

Extended Reality

MIRAGE

EAC

European Astronaut Centre

Apulia

Puglia

Italy

Italia

Basilicata

Aleppo pine

Storna Nature Reserve

Taranto

Matera

Bari

Brindisi

Ionian Sea

Adriatic Sea

Lecce « less

Sets appears in: • Human Spaceflight and Robotic Explorations Image of the Week

Stats: 1,429 views