Articles‎ > ‎

FACE OSC - What It Means For You And Me

posted Jan 30, 2013, 8:40 PM by Ellen Pearlman

FACE OSC - What It Means For You And Me

What in heaven’s name is FACE OSC, and why should I care? Its a Face Tracker program Kyle McDonald made as a “tool for prototyping face-based interaction. It’s built on non-commercial open source FaceTracker code from Jason Saragih.”

There is a “failure mode” of traditional algorithms to track people’s faces via a camera and software. That’s because its a lot like painting in that it tracks subtle lighting and color differences a la Rembrandt - except its mathematical. However, it needs tweaking. Bad lighting makes it go crazy and even the striations of grey hair throws it off.

One aspect of basic face algorithms are drawn from thousands of comparisons of faces in a huge database called Multi-Pie . Multi-Pie has thousands and thousands of pictures of faces that marks up and averages features and makes calculations about where everything is and in what type of lighting. Here are some typical mappings. In scientific parlance it is referred to as “Deformable model fitting.”

Some basic face mapping

Its really how things are registered and abstracted. And when I am talking about abstract, this is a 3D model of a face that is used that would make Picasso proud. So was Picasso seeing in 3D Maya before it was fashionable? And that brings us back to African sculpture, where he originally got a lot of his Cubism. Hmm….

3D Computer Modeling of Face Recognition

19th Century African Fang Sculpture

But then it gets even more abstracted to a bunch of triangles in space.

Triangles in Space

This is called a “deformable model” probably because the triangles can be rearranged. This is all only 2D, with an “X” and “Y” height/width axis. When you add depth, or the forward backward “Z” parameter, then it become really complicated.

But what about areas of light and dark? That is when the Haar Algorithm comes in, but that has to be broken down into a histogram. A histogram is how your camera reads levels of dark and light in the image. Wikipedia calls a histogram “a graphical representation showing a visual impression of the distribution of data.” 

It looks like a basic class in mask making with a lot of p(1n|x) formulas stuck in.  I think what this means is all the squares and locations are all tied in together eventually and are not independent from one another.

What happens if you mess the face up or only show parts of it? That has to be figured in as well.

Mapping for Obscured Parts

Now in a strange way all of this is related to skeleton tracking in 3D that the Kinect and other devices use.

It actually comes down to tracking and computing these blobs of dark and light.

The Face As Blobs Of Dark And Light

Chuck Close would be proud, since he was all over this years ago.

Chuck Close With His Own Version Of Face Mapping

Apparently Close suffers from Prosopagnosia a condition that messes up facial perception in humans. Hmmm… So African Art and messed up facial perceptions = computer algorithms.

FaceOSC comes as an example app with the ofxFaceTracker addon for openFrameworks. You can download an OSX 10.5+ binary
OSCulator is an excellent program for routing OSC and MIDI. Download it

Original Source For This Article