The fundamentals of mixed reality (MR) for engineers

Combining both augmented and virtual reality, MR provides a convenient spatial computing experience for engineers.

Sometimes straight reality is hard to take. Mixing can help.

Mixed reality (MR) is a type of spatial computing that incorporates both augmented reality (AR) and virtual reality (VR). MR is useful as a descriptor for certain head-mounted displays (HMDs) that can smoothly transition between both AR and VR experiences.

Like AR and VR, MR offers exciting new ways for engineers to work and collaborate. Here’s what you need to know.


What is mixed reality (MR)?

If you already understand the concepts of augmented and virtual reality, mixed reality is a natural complement. (And if you don’t understand AR and VR, make sure to read The fundamentals of augmented reality for engineering and The fundamentals of virtual reality for engineering for a primer.)

While AR presents an overlay of virtual elements onto the real world, and VR presents an entirely virtual world, MR allows users to choose the level of reality that works for them. MR quite literally offers a slider between the real world and a virtual world.

MR is enabled by head-mounted displays with outward facing cameras that record the real world. Unlike a standard VR headset, which is like a heavy blindfold, a user wearing an MR headset can still see what’s in front of them and interact in real time with the real world. The HMD is effectively see-through.

The Apple Vision Pro headset isn’t actually see-through, but it does its best to fool the user and the outside world. (Image: Apple.)

Of course, an expensive and sensor-laden headset isn’t much use if all you want to do is see what’s right in front of you. The real magic of MR is that it can use all or none of the real world in its output, depending on the user’s preference and application.

For example, an MR user could pull up a virtual computer monitor in their real office, typing on it with their physical keyboard that they can still see. At lunchtime they could turn off the real world entirely and enter a virtual movie theater to catch the latest episode of End of the Line. And they could switch back to full reality when their spouse knocks on the door to share the cool news they just learned on This Week in Engineering. (Some MR headsets even display fake eyes on the outside for when you’re talking to real people, though whether this is helpful or creepy is a matter of debate.)

How are engineers using MR?

Engineers can use MR in the same ways that they use AR and VR.

For instance, design engineers can pull up virtual 3D models to examine their designs at life size and in a real (or virtual) environment. Engineers planning a new factory can use MR to visualize different configurations of equipment and pick the best option for efficiency, worker ergonomics or other factors. Engineers and architects can use MR to walk through virtual buildings to better understand and optimize the layout before construction.

MR doesn’t offer any fundamentally new capabilities over AR and VR, but the ability to switch between the two modes of spatial computing adds convenience and flexibility. Why choose one or the other when you could have both at the nudge of a slider?

Written by

Michael Alba

Michael is a senior editor at engineering.com. He covers computer hardware, design software, electronics, and more. Michael holds a degree in Engineering Physics from the University of Alberta.