How Unreal Engine is Fueling the HMI Revolution

Cars are evolving, and so too must HMI development.

Epic Games has sponsored this post.

A 2005 Honda S2000. (Source: Wikimedia/RichN.)

A 2005 Honda S2000. (Source: Wikimedia/RichN.)

The Honda S2000 was a sleek sports car that first hit the roads in 1999. Production lasted for a decade, and during that time the S2000 was praised by car enthusiasts for its exceptional performance and handling. By all accounts, it was damn fine drive.

But that’s not why Joe Andresen remembers the S2000. For him, the most iconic part of the roadster was its dashboard:

The iconic instrument cluster of the Honda S2000. (Source: Car Throttle.)

The iconic instrument cluster of the Honda S2000. (Source: Car Throttle.)

It may look retro now, but the Honda S2000’s instrument cluster was one of the first steps to the digital human machine interfaces (HMIs) of modern vehicles. The giant touchscreen displays in Tesla EVs, for example, provide not just controls for the vehicle, but entertainment for the passengers. Tesla is not alone in reimaging the interface between car and driver. Nearly every automaker is upping its HMI game.

As the technical product manager for HMI and embedded systems at Epic Games, Andresen appreciates just how far automotive HMIs have come—and how quickly.

“HMIs in cars have gone through a revolution in the past five years or so,” he said.

The HMI Revolution

At its most basic, Andresen explains, an HMI is “the screens and buttons and different things that you use to control your car.” One of the most important jobs of an HMI is to convey information to the driver. This could take the form of an analog speedometer, a flashy digital speed gauge like the Honda S2000’s, or a full touchscreen display like those in Teslas.

To get an idea of how far HMIs have come (and how far they can yet go), look no further than your pocket. A phone used to be a machine with a few buttons that you pressed to talk to someone outside of shouting range. Now, after the smartphone revolution of the last 15 years, a phone functions as the command hub of most of our lives. We can check emails, stream videos, play games, and hop on a video call with our families.

Imagine all that with a car around it, and that’s where HMIs are headed.

“The hardware for HMIs is doing what smartphones did,” Andresen pointed out. “We went from having basically no hardware in the car to the same hardware that you have in your mobile phone.”

Perhaps you’re wondering: If it’s the same hardware, why bother? It’s true that both Androids and iPhones can connect to your car via Android Auto and Apple CarPlay, respectively, offering navigation, music, voice control and more. But this is a bit like casting your phone to a TV. Most TV manufacturers want to make smart TVs with custom interfaces, and so too do most automotive manufacturers want to make smart cars with custom HMIs. And there’s a good reason for OEMs to try to outdo Apple and Google.

“These OEMs have a brand, and the moment you give up that HMI to a phone, you lose your brand and your identity to that device. OEMs want to give you an experience that’s authentic to their brand,” Andresen explained.

Fortunately, it’s never been easier to develop custom HMIs. As automotive HMIs have gone through a revolution, so too has HMI development. That’s where Andresen and his team at Epic Games come in.

The Three Layers of HMI Development

According to Andresen, all automotive HMIs have essentially the same architecture, split into three domains: the platform layer, the service layer, and the UI layer.

An HMI developed in Unreal Engine. (Source: Epic Games.)

An HMI developed in Unreal Engine. (Source: Epic Games.)

The platform layer is like the operating system of a smartphone. In fact, in some cases it is the operating system of a smartphone. Android is a popular choice for automotive HMIs, as it’s a relatively easy operating system to implement and customize (note the difference between Android, which is an operating system, and Android Auto, which is an application).

But Android was made for phones, not HMIs, so it’s not a perfect solution. Some OEMs prefer to spend the effort to develop a custom platform, usually based on Linux, to have finer control of the HMI experience. There are other out-of-the-box options as well, like Blackberry QNX. Andresen anticipates the HMI platform market to heat up substantially in the coming years.

The service layer contains the things an HMI can actually do, such as analyze GPS coordinates for a navigation service, process audio input for a voice assistant service, or connect to a music streaming service. Think of the service layer as apps that run in the dark. To see the information from the service layer, we need to go up to the final layer: the UI.

“The UI layer is quite simply what you see when you get in your car,” Andresen said. “This is where Unreal Engine lives.”

Using Unreal Engine for HMI Development

Unreal Engine is Epic Games’ real-time 3D software platform. The titular Engine has nothing to do with cars, but with the fact that Unreal began as a game engine. However, the real-time 3D software has expanded its territory significantly, providing visualization solutions across a range of industries. Automotive HMIs is but the latest example.

An HMI developed in Unreal Engine. (Source: Epic Games.)

An HMI developed in Unreal Engine. (Source: Epic Games.)

Why use Unreal Engine for HMI development? Andresen can answer that firsthand—he’s developed HMI systems for the automotive industry for 10 years. When he started, he had to make HMIs the old-fashioned way: code, code, and more code. Anything he wanted to see in the UI layer had to be programmed by hand.

After work, Andresen and his friends would relax by developing indie games. Like many game developers, they used Unreal Engine. It was an easy way to get started on a project, and it combined both code and visuals in an integrated workspace. A question began to gnaw at Andresen.

“Me and one other person were making real-time applications—games—way more efficiently and faster than I was at the big OEM. Why is that?” he wondered.

One of the biggest reasons, Andresen realized, was the direct feedback provided by Unreal Engine’s visual editor. Another was that Unreal Engine allowed Andresen to deploy his applications directly to a device, such as an Android smartphone. It became clear that if Andresen could use Unreal Engine at his day job, there would be no blind coding of visual interfaces, and no waiting weeks for an engineer to implement the code in the vehicle.

Andresen couldn’t stop thinking about how much better HMI development could be. Eventually, he joined the Unreal Engine team to get them thinking about it too.

That was over a decade ago, and today Unreal Engine has doubled down on being easy to use and quick to deploy.

“In Unreal Engine, the artists and designers work on exactly the project that is going into the vehicle. They’re able to import all of their assets and build their materials and their visualizations in the editor. And then that editor can deploy a project that goes directly into the vehicle,” Andresen explained.

Using Unreal Engine for HMI development. (Source: Epic Games.)

Using Unreal Engine for HMI development. (Source: Epic Games.)

Unreal Engine is being used for HMI development by several automotive OEMs, including General Motors and Rivian, but not all automakers have come around. The automotive industry doesn’t switch gears quickly (ironic, isn’t it?), but Andresen is confident that OEMs will see the light.

“This shift is not an easy one for big OEMs to make,” Andresen admitted. “They have millions of dollars invested in the teams that they’ve created to do the traditional workflow. But once the shift is made, they never want to go back, because it gives them an unprecedented amount of control over HMI design.”

Enter the HMI Design Challenge

In an effort to inspire the automotive industry, Epic Games is calling on its community of artists, architects, and gameplay designers to envision the future of HMI systems via a friendly competition. The HMI Design Challenge, which kicks off later this month, invites Unreal Engine users to design their own ideal HMI. Epic Games will supply a project template with a 3D car buck (the interior of the car only) and configurable screens as a starting point. Judges from automotive OEMs will be weighing in on the entries, and winners will be highlighted in the annual Car Design Review book from Car Design News.

“It’ll be great to see what OEMs pick, what they’re inspired by, and it’ll be really cool to see what the community creates,” Andresen said.

To learn how to use the template and submit your concept for the HMI Design Challenge, register for the upcoming webinar Exploring the New HMI Design Template in Unreal Engine

You can learn more about using Unreal Engine for HMI development at UnrealEngine.com.

Written by

Michael Alba

Michael is a senior editor at engineering.com. He covers computer hardware, design software, electronics, and more. Michael holds a degree in Engineering Physics from the University of Alberta.