Articles‎ > ‎

Kinect 3D Hand Modeling, Picasso, and Cubism

posted Jan 30, 2013, 9:51 PM by Ellen Pearlman

Kinect 3D Hand Modeling, Picasso, and Cubism

In scientific parlance its called “Tracking of a hand interacting with an object by modeling occlusions and physical constraints.”

This is what it looks like:

Tracking modeled occlusions and physical constraints (2011)

However, in 1937 this was already being done but in a completely different way.

1937 Picasso - Notice that hand!

It’s pretty amazing that the tracking of occlusions and physical constrains was a foregone conclusion by the master.

Picasso and his hand

Even though his hands seemed the size of meat cleavers.

What is the sound of two hands not clapping?

I am not implying it is easy to track hands so clearly with a Kinect. It’s not. It derives from some pretty serious research.  The scientists say they are working on “an optimization problem, seeking for the hand model parameters that minimize the discrepancy between the 3D structure and appearance of hypothesized instances of a hand model and actual hand observations. This optimization problem is effectively solved using a variant of Particle Swarm Optimization (PSO).”

The hand is broken down into 25 dots or spheres.

37 geometric primitives (left) and the 25 spheres (right)

The European Union is so into this they are supporting a project called “Grasp” to render it correctly. Their stated aim “is the design of a cognitive system capable of performing grasping and manipulation tasks in open-ended environments, dealing with novelty, uncertainty and unforeseen situations. “

They maintain an “Open Grasp” site that has “an open source simulation toolkit for grasping and dexterous manipulation hosted on SourceForge. ” And then they have “Grasp the Robot Editor” that can be used with Blender.

Grasp the Robot Editor has their own Youtube Channel

They even have special summer schools on grasping!