Articles‎ > ‎

Raise Your Eyebrow - Control Michael Jackson

posted Feb 24, 2013, 8:18 PM by Ellen Pearlman   [ updated Feb 24, 2013, 8:56 PM ]


Make Michael Dance

  A new face mapping technology called FaceShift was mapped so a simple gesture, such as the raising of an eyebrow, controlled the movements of a prerecorded simulated Michael Jackson dancer.


Face calibrating

First a user’s face is mapped in a Kinect. Then it is calibrated to make a smoothed grid. The face in the Kinect is on the right, and the smoothed calibration, which is actually rendering on screen in this shot is on the left.


Original face, Kinect face, modeled face

In this view you can see the original subject, who is wearing glasses on the upper right, the morphed face in the Kinect in the lower right and the end result modeled face on the left.


Original face as Green Creature

A face can then be morphed to an avatar or creature with lifelike gestures. However, the program was hacked by two guys at Eyebeam, James George and Kyle McDonald. Using UDP to OSC (User Datagram Protocol to Open Sound Control )commands they were able to control the movements of the Michael Jackson character to make it dance or move by doing things as simple as moving an eyebrow. From what I understood during the demonstration the most difficult but exciting part was remotely creating in a motion capture studio a Michael Jackson imitator dancing a complex set of Michael’s moves, and then importing that information into a usable scenario that could be mapped. I know proprietary code was written to do that by Kyle McDonald but I don’t know specifically how it was done.


Using facial gestures to make Michael dance

Here is subject is sitting in front of the Kinect, watching himself, his contoured mapped face, and the video of a Michael Jackson-like figure dancing, which he is controlling through facial gestures. 

The implications of this are enormous, especially for stroke victims and paraplegics. In art, the boundaries are truly crossed. If the puckering of one’s lips, or raising of one’s eyebrows can control a choreographed work what does that say about the aesthetics and cross modalities of producing the work? If my intention is to squint my eyes and it results in a dancer doing a backflip, am I creating a new modality, or am I imitating someone with brain damage? When the purpose for which the connectivity was intended by being born human becomes completely remapped, how does this play out? As a fad or as a new reality?