How to create user-friendly Mixed Reality applications
We are now faced with a shift in the technological world, from traditional square screens, to immersive realities that mix digital and physical together, creating mixed reality experiences.
Major players such as Facebook, Apple, Microsoft and Google are focusing heavily on this type of experience and the pace of the market is high. This inspires trust and we can see that several of our customers want to be the first to use this kind of technology. The goal is to achieve a competitive advantage and to figure out what the future will look like in their industry.
Mixed Reality is defined as experiences where the digital and physical are mixed together. These experiences are for instance provided through closed VR glasses like HTC Vive, open glasses like HoloLens and ARKit for iPhone. At the innovation lab in Sopra Steria, we have started looking at how we can make simple and good user experiences for HoloLens. In this regard, I have formulated five questions that would be useful to consider when designing user-friendly applications in mixed reality.
1) Why is this particular technology best suited to solving our problem?
This question actually applies to every solution we create, whether it’s a web application, mobile app or mixed reality application. Before we start designing anything, we must be sure that we have chosen the right tool for the job. Which features will help you solve your problem in the best possible way and what technology has these properties?
Closed glasses like HTC Vive will for instance provide you with the ability to transport users to a fully digital location, and could therefore be suitable for simulations or games. Open glasses like HoloLens allow you to display both 2-dimensional and 3-dimensional digital objects placed in your real-world environment, and could for instance be very suitable for training purposes by displaying contextual information directly in the real world. What properties would be useful to address your needs? Be sure you can defend your choice of technology in a good way.
2) What design expertise do we need for mixed reality?
As in other projects, we need people with different skills to design good user experiences. We need service designers who can think about the entire customer journey, such as how it feels like when putting on the glasses for the first time, and getting started with a task. We need interaction designers who can find ergonomic and intuitive ways to interact with the system. We need graphic designers who can solve visual problems and make sure the interface is pleasant to look at. But these designers must also learn to understand the limitations and capabilities of the technology, and this makes it necessary to invest in developing the expertise of our designers who are going to work with these types of applications.
In addition, further skills are needed from other industries, such as those that can be found in gaming development and the film industry. We need designers with expertise in 3D, animation, sound, interior design etc. Since the technology is new, it means that the probability of finding someone who already has the perfect skill-set will be very small. We therefore need to avoid looking for someone who does not exist yet, and instead focus on finding someone who has transferable skills, and who may eventually become the designer we wish we had now.
3) How many different design proposals have we come up with?
When we create a website or a mobile application, we have finished libraries and style guides that we can use to design systems. They tell us how a button should look and behave, or how to send and display alerts. But mixed reality is still at an early stage, and so we do not yet have many established guidelines to follow. In other words, we don’t have many constraints when producing designs.
I therefore believe that it is a crucial step that we develop several different design proposals early in the design process, experimenting with both creative and revolutionary design, as well as testing out how traditional and more recognisable patterns work in mixed reality. We cannot take for granted that the designs we are familiar with from traditional screen-based solutions can be transferred in a good way to mixed reality, and it is difficult to come up with something completely new. We therefore need to make sure we constantly think a little outside the box, while also benefiting from our knowledge from previous experience.
4) Have we tested how the design works on users?
This question is closely linked to the previous one, because we need to test which of the design proposals actually work best for the task. User testing is one of the most important tools we have for creating good, user-friendly systems, and it may be even more important for this kind of applications, since we have probably not created anything similar before.
So, we must always test the various design proposals that we encounter on real users. Remember that some testing will always be better than no testing, and that early testing will prevent costly changes later in the development process.
5) How can we make users feel secure with the technology?
One of the arguments for using this type of technology is that it is more intuitive and natural to use than a keyboard, mouse and monitor. But today, these types of glasses are regarded as futuristic and a little advanced. Working with HoloLens at the DigiLab, I have observed that users tend to blame themselves if they cannot quite get things to work. This can make them feel insecure and clumsy. It is therefore vital that we help users quickly achieve a sense of security and control.
Our experience tells us that providing training in using the glasses and their control mechanisms is an important step for achieving this sense of security, and for ensuring users’ joy in future use. So make sure that you have thought about the entire customer journey, from the first encounter with the technology until the users have solved their task.
It’s an exciting time to explore what opportunities this type of technology can offer, and we hope these questions may be useful to others who are already working with mixed reality or who are considering doing so.