Eye-tracking and foveated rendering will become integral to all future virtual reality experiences.
At CES 2019, HTC Vive announced the addition of new virtual reality products. The Vive Pro Eye incorporates foveated rendering developed by Sweden-based Tobii, a company that creates an array of eye-tracking technology primarily for domains like special education, researchers, businesses, OEMs, device makers and the gaming community.
Tobii was started in 2001, and they’ve been successful by focusing on three areas of expertise: The Tobii Dynavox division makes specially designed computers and touch screens that are controlled by eye-movement for people with special needs due to medical conditions. Tobii Pro creates a different set of eye-tracking products for research institutions and universities. And Tobii Tech is the third division of the company that makes eye-tracking tech for computer games, PCs, smartphones and virtual reality.
Before this new integration from Tobii, HTC offered users the ability to add eye-tracking to the Vive by means of a kit. The Vive Pro costs USD 799, which is already pretty expensive, and the integration of Tobii’s eye-tracking technology will only drive the price higher. For their mass market consumers, the HTC Vive Cosmos was presented at CES 2019.
Why the Partnership between HTC Vive and Tobii?
Virtual reality is a natural fit for the Tobii’s foveated rendering technology, especially since they offer customers eye-tracking analytics and the subsequent ability to analyze and influence user behaviors and decision-making. This is the concept behind Tobii Pro VR Analytics, a new analysis tool for training individuals in complex domains and high-risk occupations like emergency response, industrial and medical.
Manufacturing companies could use the new tool to train workers on operations that require different responses in a variety of scenarios. The eye-tracking analytics data could help employers zero-in on problematic behaviors and perhaps help workers identify helpful signs for maintenance of machinery and notice early-warning signs of potentially dangerous accidents.
A construction company may be interested in having eye-tracking analytics to track people’s focus when shown a completed building design on a VR headset and then use the data to make design decisions like which locations to place signage on, and where to place emergency exits.
An aviation company could use Tobii Pro VR Analytics during virtual reality flight simulations to refine training modules, which could help teach pilots about best practices when it comes to reading instrumentation and executing proper maneuvers given a certain situation.
Why is Foveated Rendering Important for Virtual Reality?
Eye-tracking analytics in virtual reality could prove to be useful in multiple areas if integrated into VR design reviews and control group testing. Yielding useful maps of areas where people’s eyes naturally focus could make certain design decisions easier to implement based on hard evidence.
When the HTC Vive Pro Eye was demonstrated at CES 2019, the audience was treated to a video presentation of how Vive partners BMW, NVIDIA and ZeroLight are using the new eye-tracking and foveated rendering features to enhance immersive product experiences.
The BMW M Virtual experience takes advantage of foveated rendering by integrating it into this virtual experience for current and prospective BMW customers, allowing them to track what areas of new cars customers are looking at most, giving them real-time analytics data to incorporate back into product design. NVIDIA’s RTX GPU’s power improved graphics in the visualization experience and eye-tracking is presented as a way to increase the efficiency of both the GPU and the CPU.
The BMW M Drive Tour gives existing and prospective customers a way to digitally experience the company’s signature M vehicles as they race around famous tracks across Europe. BMW M customers can preset their preferences by selecting interior options, wheel and paint preferences.
The visualization experience also provides the customer with access to a microsite featuring their own personalized customizations. The customer could see a full 360-degree view of their car, footage of them driving down a racetrack along with shareable assets to post on social media.
More efficient distribution of rendering power
Foveated rendering caters to the way the human eye actually works. When we focus on a particular area or object, that area is rendered clear while the peripheral area is not as sharp. By tracking a user’s eyes in virtual reality, a foveated display moves with the user’s eyes to concentrate a clear rendering in the area of focus while peripheral details are less detailed.
Instead of rendering the whole display as clear as possible, foveated rendering with eye-tracking parses out high-resolution as needed, following the user’s eyes. Though it’s possible that foveated rendering could lead to a higher field-of-view, developers making content for VR may have a new challenge that could prove costly if they want to incorporate eye-tracking and foveated rendering into their projects.
Eye-tracking and foveated rendering will likely become more commonplace as virtual reality continues to evolve in enterprise and mass-consumer markets.