Articles‎ > ‎

Magic Leap Gets It Right

posted Dec 12, 2015, 10:45 PM by Ellen Pearlman


Whale breeching in high school gymnasium

Magic Leap has combined holography and augmented reality producing life-like computer generated optical illusions. In this jaw dropping video a school basketball court becomes the stage for a full sized whale to breech from underneath the floorboards before flopping down into the non-existent ocean and disappearing. Its illusions are so life like they fool the brain, just like they are doing in this video clip in front of a real audience.

Most augmented reality displays have been 2D, and are used on individual’s mobile phones to superimposed one image over another. Mixed reality is putting an object into the real world just like you would normally interact with it

Magic Leap was founded by Rony Abovitz, who previously founded the world’s first robotic haptic surgical arm company, Mako Surgical. He sold that for 1.3 billion dollars to a company called Stryker. With so much money, he dressed up as a spaceman and danced around with two human furballs called “Shaggles” selling ‘space fudge’  during TEDxSarasota. He called his presentation “The Synthesis of Imagination.”


Abovitz as spaceman hawking space fudge after his 1.3 billion dollar windfall

However, there was a deeper motive for the shenanigans. He briefly spoke to the audience saying “A few awkward steps for me, a magic leap for mankind” Later in his presentation he flashed an overhead sign saying “Creativity Matters: 12/12/12″ 

Abovitz has basically made a Google hololens with one difference - its image generation capabilities. It uses light field signal technology. Lightfields collect all rays of light that come from an event. Normally a photograph takes the rays and stores color and intensity onto a sensor plate (or in the old days, film). A lightfield does not store the pixels but the entire ray. When you have all the rays emanating from a scene, the image can be refocused at any position. When you do this over a period of time it is called a dynamic lightfield. With the correct lens the saved ray information can be refocused so our eyes see depth perception, or a type of 3D microdisplay. It is a specialized way of performing advanced signal processing. Magic Leap is perfecting its own lightfield chip to project images onto the user’s retina using a specialized projector.

The big issue here is safety for both the human eye and human brain. Abovitz won’t work with devices that use traditional stereoscopic 3D tech. A lot of his initial insight about the technology came from mechanical engineer researcherEric Seibel at the University of Washington who has made true 3D displays investigations feasible. 


Sparkydog and Frieds, illustration by Andy Lanning and Anthony Williams, DC Comics

Abovitz has a virtual rock and roll band he calls “Sparkydog & Friends.”  They were the inspiration for the Magic Leap, because he wanted to take them on a virtual tour where they would appear in as many places as possible. Preliminary reports about Magic Leap suggest the viewer would wear special sunglasses with the equivalent of a small battery pack attached. It also needs a special software to track the viewer’s eyes and fingers, and configure all of this through deep learning AI research.

The company already hired Neal Stephenson as ‘chief futurist’. Stephenson wrote the 1992 sci-fi blockbuster Snowcrash.


A recent U.S.  patent filed by Magic Leap of the type of glasses they are envisioning


The belt pack you would need to wear

Google has invested a half billion dollars in Magic Leap, and the company just raised 827 million in a third round of funding. This means they will have raised 1.4 billion in startup funding, the most in the world. Now they can hire the staff they need to make this into a compelling reality.