Research team tracks real ball in virtual reality to provide dynamic real-world feedback.

If I were to strap a set of virtual reality (VR) goggles onto your head right now and throw a ball at you, would you be able to catch it? It’s unlikely. The ball would probably bounce harmlessly off you and hit the ground because you wouldn’t see it coming.
Now what if I tracked that ball and fed its motion into your VR world. Would you catch it? Disney Research thinks that you would. And by employing predictive trajectory algorithms, it reckons you could probably catch it 95 percent of the time.
A team at Disney Research recently decided to up its feedback game significantly by doing away with the simulated haptic touch experience of old and replacing it with an actual touch experience, allowing users to both see objects in VR and experience that very same object with their hands. That is to say, the VR system is tracking a real-world event (throwing a ball), capturing that event data, transforming it into a 3D model and feeding it back into the user’s eyes via the headset.
The user then reacts as anyone else would when they see a ball coming towards them in real life—they catch it, both in VR, and in reality. This provides a far more realistic feedback experience, largely because it is real. Unlike haptic devices that tend to provide a one-feeling-fits-all vibration experience for every event, when you catch a ball in the Disney Research system, it actually feels like a ball. Because it is a ball.
So how does it work? Disney Research has developed three methods for enabling this hybrid VR-flesh interaction.
The first visualization method involves motion capture of the ball in real time and feeding that motion data through the computer before rendering it as a ball in VR. In this case, the VR catcher relies on good old hand/eye coordination to catch the ball.
The second method plots the predicted trajectory path in the VR environment as a white arc, allowing the user to position their hand before the ball lands at its destination.
The final visualization involves plotting the intended target based on the trajectory.
In all three scenarios, the user was able to catch the ball 95 percent of the time, although, interestingly, the catching strategy changed somewhat in the third method, with the catcher’s hand zooming to the predicted target zone far in advance of it landing.
The point of this project is to enhance the user experience. “Catching and feeling the real ball in your hand makes VR much richer, more believable, more exciting, more interactive, more dynamic, more real,” said Disney Research lab associate Günter Niemeyer.
According to the Disney Research paper, the act of displaying the trajectory or target in advance of the ball landing is somewhat akin to precognition. It enhances the senses.
And who knows, maybe it can boost your ball-catching skills in real life too.
For more news from Disney research, check out this one-legged hopping robot.