With Hololens, Microsoft has designed a headset which enables interaction with virtual images and objects which are projected as holograms into the user’s field of vision. The developers describe the concept as mixed reality, which borrows from both augmented reality and virtual reality. What does it really involve, and what kind of uses could it lead towards?
As a reminder, augmented reality consists of displaying a virtual object in the user’s real environment and enables interactions based upon their position within the space. It underlies the well-known game Pokémon Go, or the recent application Ikea Place, which allows iPhones running iOS 11 to display a sofa in the middle of your living room so that you can judge the final result.
Virtual reality pursues a different goal: transporting the user to another world thanks to complex 3D representations and an advanced level of interactivity. The downside is that once the user is wearing a VR headset like the Oculus Rift or the HTC Vive, they’re cut off from the world around them.
Mixed reality is looking to bring together the best of both worlds: an ‘augmented’ environment thanks to digital information displays and sophisticated interactivity. To achieve this, imagine a device capable of detecting the user’s environment in order to project interactive virtual objects onto it: that is what Microsoft is offering with its Hololens headset.
Holoportation: the first example of mixed reality
Hololens is based upon the process of holographic projection, which is displayed on the lenses adorning the front of the headset. It calls upon an array of intelligent cameras and depth sensors which are responsible for calculating how the image is to be shown, oriented and sized, so that it appears to be virtually anchored in the user’s immediate environment.
The main difference from augmented reality is that this image is not static. It’s a truly virtual object, the representation of which changes in real time based on the user’s interactions. In practice, it’s easy to imagine displaying a friend’s avatar and talking to them as if they were at your side in the same room.
In order to carry out this complex scenario, specific equipment is required to capture and model the body of the person calling. Microsoft has dubbed this experience Holoportation, though it remains confined to the research labs for the time being.
Crossing the boundary between the real and the virtual
While we might still be waiting for true telepresence, this ability to bring together the real and the virtual, unconstrained by distance, is already offering tangible possibilities. A customer at a car dealership, for example, might modify the car displayed before them however they like, so that they can judge the effect of adding rims or a bodywork option. On the assembly line, a worker will be able to study a virtual model of a component before working on the real object itself. The German group ThyssenKrupp, an elevator specialist, envisions equipping its employees with mixed reality headsets. Using this headset, a technician will be able to visualise the results of a diagnostic test performed remotely in real time, effortlessly receiving the precise procedures and instructions required to deal with the specific machine to be repaired; the perfect complement to predictive maintenance algorithms.
Collaboration: how to advance towards ‘teleportation’
Alongside the Hololens hardware and the headsets devised by the developer’s partners, Microsoft already has programming interfaces available which enable the designing of mixed reality applications. The success of these interfaces lies in their ability to represent a real variety of uses, such as supporting collaboration.
Using mixed reality, a team of designers can now work on a 3D model in real time, even if its members are located thousands of kilometres from one another. Tomorrow, it will open the door to online courses during which each student will feel as if their teacher is by their side. Remote meetings, currently restricted to meeting rooms, will be able to take on another dimension. Every collaborator can find themselves at the most relevant site, sharing understanding of the physical problem via a 3D representation. Taking part in a cooking course, standing directly by the stove of top chefs, participating in an experiment on the international space station by the side of the astronaut in charge… Collaboration using mixed reality will finally integrate the essential component of all communication; the accurate and instantaneous perception of everyone participating.
From providing training to enabling participation, the possibilities offered by this ‘virtual teleportation’ are countless.
Latest posts by Yoann YVON (see all)
- Hololens, or ‘teleportation’ at the service of business - 4 October 2017