Articles‎ > ‎

I Want To Be Inside Of You - Latching Onto Total Immersion

posted Jul 4, 2014, 10:47 PM by Ellen Pearlman


Oliver Kreylos wearing an Oculus acting out in front of a Kinect

The Kinect has penetrated the Oculus Rift. And facial recognition technology has made it into a number one Top Of the Pops hit.

Previously when playing inside Virtual Reality worlds in the Rift, you looked down and could only see someone else’s hands and feet, meaning whoever or whatever the developer or animator placed in there. Now you can look down and see your own hands and feet, a huge difference for VR world enthusiasts. 

Oliver Kreylos merged three Kinects and fed the stream into the Oculus. He also explores the “Uncanny Vally" principal of aesthetics that suggests when something moves that looks almost, but not quite like a human, it causes feelings of revulsion. 


Kreylos tracking experimental eye focal points

Why this is so important is the Oculus Rift is going to eventually use special eye tracking algorithms. Germany based SensorMotoric Instruments, or “SMI” prototype eye tracker follows a user’s pupil position and gaze direction in all three dimension. It has little cameras not so different from a Kinect embedded inside. It even drills down enough to follow the shape of the cornea and the position of the pupil, as well as aspects of the retina. The issues this type of set up has to overcome are accuracy and latency, which can induce dizziness and nausea. The company already sells eye tracking glasses, so the technology it now hawks is just a step up to merge inside the Rift.


SMI Eye tracking glasses at $11,000 a pop


Two tiny prototype eyetrackers embedded in an Oculus Rift

What I like is how fast this fizzes into pop culture and film. In Sam Smith’s song “Latch,” facial recognition technology that looks to me like OpenFrameworks puts a rather poetic spin on a rather creepy, invasive experience.


Sam Smith’s “Latch” using facial recognition technology

The song croons on about love, “I”m latching on babe, now I know what I have found.” No “Uncanny Valley” syndrome present.