Do We Need a Parkouring Robot?

Boston Dynamics uses an obstacle course to teach its robot how to navigate the world and shift behaviors on the fly.

Videos of Boston Dynamics’ robots doing parkour have taken the Internet by storm. But how and why—beyond just that it looks cool—is the company getting its robots to tackle obstacle courses?

The robot in question is Atlas, which the company deploys as a research platform rather than a commercial product (so, no, you can’t buy your own parkouring robot). Parkour is used to teach Atlas how to perceive, analyze and navigate a difficult and dynamic environment—in real time. Boston Dynamics uses this obstacle course to develop and refine a robot that can handle these environments and the software needed to help it perform its tasks successfully.

Why Parkour?

While a backflipping robot might not be practical, a robot that can replicate human motion would have many uses across industries and sectors. If that robot can respond to its environment with the same flexibility and reaction speed as a human, it could potentially perform its tasks significantly better. While a humanoid form may not be the best suited for any particular task, it is an ideal platform to execute a wide variety of physical tasks due to its flexibility and adaptability. But while a humanoid form may be ideal, it’s challenging to program a robot to move like a human.

“From a technical perspective, humanoids present several challenges that we welcome as a research team,” said Atlas team lead Scott Kuindersma in a blog post. “Their combination of size and complexity creates hardware design tradeoffs related to strength to weight ratio, runtime, range of motion, and physical robustness. At the same time, our control team has to create algorithms that can reason about the physical complexity of these machines to create a broad set of high energy and coordinated behavior.”

That’s where parkour comes in. It gives Atlas’ engineers a highly useful sandbox in which to play and experiment with new behaviors. Tasks include the robot maintaining balance using its entire body while moving seamlessly from one behavior to another. The team uses parkour to study rapid behavior creation, dynamic locomotion, and how the robot makes connections between what it perceives and how it responds—in real time. “It’s really about creating behaviors at the limits of the robot’s capabilities and getting them all to work together in a flexible control system,” said Kuindersma.

The company’s work on parkour has enabled Boston Dynamics engineers to develop a precise understanding of how to create, deploy and control a robot with a wide range of dynamic behaviors. Not only has the team created a robot that is physically capable of those behaviors—it has also developed a robust software platform that empowers Atlas and will continue to grow as the robot gains new abilities to perceive, navigate and change its environment.

Boston Dynamics demonstrates how Atlas works.

How Atlas Navigates a Parkour Course

Atlas uses sensors such as cameras and LiDAR to create a map of its environment. It also uses joint position indicators, force sensors and inertial measurement units—which are a set of sensors including gyroscopes, accelerometers and magnetometers used to calculate useful information such as position, orientation and acceleration—to control the robot’s motion and balance.

In particular, Atlas uses a time-of-flight depth camera to create point clouds of its surroundings at a rate of 15 frames per second. The result looks like this:

A rotating point cloud generated by the depth camera of one Atlas that captures another Atlas jumping over an obstacle. (GIF courtesy of Boston Dynamics.)

A rotating point cloud generated by the depth camera of one Atlas that captures another Atlas jumping over an obstacle. (GIF courtesy of Boston Dynamics.)

Atlas uses robotic perception to convert the sensor data into information that it uses to make decisions and perform physical actions. The robot’s software uses a multiplane segmentation algorithm to identify surfaces within the point cloud and then feeds the output into the robot’s mapping system. That system then builds models of the objects Atlas sees with its camera.

Boston Dynamics’ engineers give the robot a high-level map of the parkour course and indicate where they want Atlas to go and what stunts it should perform as it navigates the course. The map is an approximate description of the course that contains obstacle navigation templates and annotated actions. While the map may not be geometrically precise, it still provides Atlas with enough basic information to navigate the course.

Atlas’ perception software plans the robot’s next steps on the parkour course. (Image courtesy of Boston Dynamics.)

Atlas’ perception software plans the robot’s next steps on the parkour course. (Image courtesy of Boston Dynamics.)

Atlas uses live perception data to fill in the details that the map doesn’t include. For example, Atlas knows to look for a platform to jump on—but if the platform has been moved a half meter to the side, Atlas will adjust its movements to jump on it in its new position. (And if the platform has been moved too far for Atlas to find it, the robot will stop moving.)

The image below demonstrates Atlas using live perception data to update its map as it moves through the parkour course. Atlas highlights actively track objects in green, which fade into purple as they leave the robot’s line of sight. The tracking system also continuously estimates the poses of objects. Atlas’ navigation system then plans the route, marking with green footsteps where the robot should step.

Atlas software plans the route and the robot executes the stunts. (GIF courtesy of Boston Dynamics.)

Atlas software plans the route and the robot executes the stunts. (GIF courtesy of Boston Dynamics.)

Atlas analyzes an obstacle and create targets, pulls a routine from its library that most closely matches the targets, and executes the movements. The moves are all derived from templates created ahead of time via trajectory optimization software. These are stored in Atlas’ memory, where new trajectories are being added constantly. Creating templates offline allows Atlas’ engineers to test ahead of time the robot’s design and mechanical limits, such as actuation limits and how the robot coordinates its limb movements to perform a back flip. Resolving those limits reduces the computational load the robot must bear while parkouring, streamlining its decisions by using a single controller.

That controller is a model predictive controller (MPC) that uses a model of Atlas’ dynamics to predict how the robot’s motion will evolve as it executes its parkouring plan. The MPC computes the best action for the robot to take in any individual step to produce the best possible motion over the course of time.

The controller takes a template from the library that informs it what an ideal solution would be. The MPC then adjusts the template with details such as force, positioning and behavior time as well as environmental differences, foot slips and other real-time variables. For example, if Atlas has a template for jumping onto a box that is 0.5 meters high and it is approaching a box that is 0.65 meters high with momentum from having completed another jump, the MPC will adjust the template to match the scenario before it.

Atlas doesn’t need to reinvent the wheel for every new obstacle: its engineers entrust the MPC to find the right template and adjust accordingly. The MPC’s predictive ability also enables Atlas to make smoother transitions from one behavior to another—such as knowing that a jump will be followed by a spinning jump to change direction.

As a result, Atlas can perform a variety of moves in response to a range of obstacles. However, one movement proved to be particularly challenging: vaulting over a low balance beam. For humans, this is a relatively straightforward move—but not for Atlas. This is partly due to the robot’s design differences compared to humans. The robot doesn’t have a spine or shoulder blades and can’t rely on the same range of motion a human would use to perform a vault. Atlas also has a heavier torso than a human, as well as relatively weak arm joints.

Atlas and its engineers eventually found a solution: the robot puts its arm on the beam and then pulls itself over it—accomplishing the task but not doing it nearly as efficiently or as gracefully as a human would. It took a lot of trial and error—and plenty of bumps and scrapes. Even at the time of filming the parkour video, Atlas was able to overcome the vault only about 50 percent of the time—and sometimes even if it cleared the vault, the robot would still lose its balance and fall backward. Performing a back flip actually seems to be easier for the robot than the vault. The engineers welcomed the challenge, though, since it helped them to extend their tools and push the robot’s limits.

From Parkour to the Real World?

Boston Dynamics aims to use the lessons learned from Atlas’ adventures in parkouring, such as the vault challenge, to help robots navigate the real world more effectively, reliably and safely.

“I find it hard to imagine a world 20 years from now where there aren’t capable mobile robots that move with grace, reliability, and work alongside humans to enrich our lives,” said Kuindersma. “But we’re still in the early days of creating that future. I hope that demonstrations like this provide a small glimpse of what’s possible.”

Boston Dynamics continues to push the envelope in robotics technology. In fact, the company recently partnered with automaker Hyundai to launch the Boston Dynamics AI Institute and advance artificial intelligence, robotics and intelligent machines.

Here’s hoping the institute’s work involves more videos of parkouring robots.