Articles‎ > ‎

Memory, Dreams and Visions - New Interpretations

posted Aug 5, 2013, 3:20 PM by Ellen Pearlman


Inspector Clousseau (aka Steve Martin)  and his brain algorhythm artistic reinterpretation

Professor Jack Gallant has developed a way to reconstruct the eye’s visual imagery in the brain by running an fMRI of visual interpretation that evokes theImpressionistic era of European art. Three subjects were placed inside the machine for a number of hours. They watched two different groups of Hollywood trailers. I have always thought the trailer is an art form in and of itself, sort of like a Twitter of the cinema. The subjects fMRIs were recorded as they watched. How they stuck a TV screen inside an fMRI machine and did not screw up the signals is beyond me, but well, never mind, Gallant pulled it off, and the blood flow through the brain was recorded. 

The data went into a software program that divided the information into voxels (volumetric pixels.) With some sort of wacked out but effective algorithm that examined 935 different object and action categories, the brain signals were decoded from the movie trailers as shape and motion. Next a computer analyzed eighteen seconds of thousands of random YouTube videos sorted most importantly by their color palette. The software picked stimuli that matched the brains patterns.  It then combined those patterns into a clip of what the person was likely seeing, but NOT WHAT THEY ACTUALLY SAW.image

 Brains especially like faces. The real face is on the left and the “reconstructed" image is on the right. People like to see, and are hardwired to understand faces. 


Sampling of faces off YouTube, including a dog and legs just to shake things up

Above is a sampling of the list of faces the algorithm went through to match the reconstructions. The readings were fed into a computer program that divided them into three-dimensional pixels units called voxels (volumetric pixels). This process effectively decodes the brain signals generated by moving pictures, connecting the shape and motion information from the movies to specific brain actions.  Notice the cartoon face on the top right of this sample. Brains like cartoon faces as well. Maybe that’s why advertisers use so many cartoons to get our attention, appealing to our lowest common denominator.  

Gallant’s lab is the same lab I referred to some blog posts back about the semantic brain research Alex Huth did. He published the results in the journalNature Neuroscience and this is a representation of how different images are perceived inside your brain.


Image from Nature Neuroscience 16763–770 (2013)

 The brain is also pretty good at figuring out simple things, like big blobby dots.


Blobby dot on screen equals fMRI reconstruction of blobby dot in brain

It didn’t do so well with marching elephants and a kid wearing a stethoscope, though it was pretty good with a flying parrot. It got screwed up mistaking a frontal shot of a landing plane for a distant horizon.


Landing airplane does not equal distant horizon of whatever that distant horizon really is

Certain schools of Buddhist philosophy have understood this perceptual conundrum inherently. 

According to the Particularist (Vaibhashika) School of Buddhism:

When you look at the continuum of consciousness, you may think it is one thing, but really it breaks down into its individual moments of the past, the present and the future. These [individual] moments can be divided even further. This whole that you believe to exist is, in fact, just a conceptual construct.

That applies to both material things and the mind.

Anything that can be broken down into parts is only apparently real, not genuinely real.

That leads me to my next little exposition, on the shifting sands of memory.


Scientists use a pipette to go into one area of a mouse brain

 MIT professor Ed Boyden and Stanford professor Karl Deisseroth invented a technique called optogenetics, which means manipulating individual cells with light. Recently optogenetics was used to manipulate a mouse’s memory.

In 2012 scientists put a mouse in a chamber and shocked it. Then they altered the genes of the brain cells in the region of the brain where the shock took place. They moved the mouse to a new chamber. The mouse behaved normally. The scientists then shone a special blue light on the mouse. The light activated the genetically manipulated memory cells from the fearful experience and the mouse, though in a different chamber freaked out, thinking it was back in the little mouse house of horrors. The normal stimuli that activated the fear response was recalled by shining the special light. 


Using the pipette to somehow optogenetically change the brain cells

The Optogenetics Innovation Laboratory (OIL) actually holds three-day workshops where one can learn this technique. 

What it means in essence is you can implant a memory. This makes "The Manchurian Candidate," a movie starring Frank Sinatra that is full of cold war innuendo seem plausible. Artists, who often work with the issues evoked by memory could really run with this. Get an injection in your brain. Wipe out your memory. Reboot. Start all over again. Just like the movie "Eternal Sunshine Of the Spotless Mind." Of course that is an exaggeration, but by how much?