Gestures and Pens: The Perfect Combination for 3D Sketching?
Michael Alba posted on August 10, 2018 |
Creating air scaffolds (left) and sketching details with a pen (right). (Image courtesy of Kim et. al.)
Creating air scaffolds (left) and sketching details with a pen (right). (Image courtesy of Kim et. al.)

Researchers at the Korea Advanced Institute of Science and Technology (KAIST) have developed a novel way for industrial designers and concept artists to sketch out their ideas in 3D. The method is dubbed “Agile 3D Sketching with Air Scaffolding” (at least until somebody comes up with a sexier name), and it uses a combination of hand gestures and tablet pen sketching to create the final product.

Users start with their hands to develop a rough shape of their 3D sketch, using what the researchers call air scaffolds. The idea relies on everybody’s natural inclinations for describing a 3D object with hand gestures. For example, to trace out the form of a basketball in midair, you would most likely gesture such that your palms move along the surface of an imaginary ball. Ditto for a box, or a pyramid, or any other shape you’re trying to convey with your hands.

The KAIST researchers used a Leap Motion hand-tracking sensor to capture these gestures, developing an algorithm to separate intentional movements from extraneous ones. Essentially, the algorithm detects whether your palm is moving tangential to the curve it describes. If so, 3D scaffolds are created; if not, the movement is ignored. Repeated motion over the same area boosts the scaffold, exploiting our natural tendency to repeat gestures when describing a shape.

Visualization of how the air-scaffolding algorithm differentiates between intentional shape information and extraneous hand movement. The vector n, located at the center of the middle finger’s proximal phalanx, defines the direction normal to the curve. If the velocity vector v, describing the palm’s movement, is within 27.5 degrees of t (the vector tangent to n), air scaffolds are created. (Image courtesy of Kim et. al.)
Visualization of how the air-scaffolding algorithm differentiates between intentional shape information and extraneous hand movement. The vector n, located at the center of the middle finger’s proximal phalanx, defines the direction normal to the curve. If the velocity vector v, describing the palm’s movement, is within 27.5 degrees of t (the vector tangent to n), air scaffolds are created. (Image courtesy of Kim et. al.)

Users can then use their air scaffolds as a reference to create sketch planes, on which they can draw with a pen (the researchers used a Wacom Cintiq 21UX graphics tablet). Users can easily create new sketch planes with reference to pen sketches, air scaffolds or both. New air scaffolds can be added at any time, and users can erase scaffolds they no longer need.

According to the researchers, the new technique combines the best of pen sketching and hand gesture techniques, allowing for an intuitive, easy-to-learn user experience. They sharpened this claim with a trial of 12 industrial design students who quickly took to the method and expressed positive comments about the experience. One of the biggest advantages of using air scaffolds, as reported by the students, was the time efficiency and accuracy of defining scale and proportion in their 3D sketches.

To learn more about the air scaffolding method, you can read the research paper here.


Recommended For You