Articles‎ > ‎

What Do Computers Dream Of When They Dream?

posted Sep 10, 2015, 3:11 AM by Ellen Pearlman   [ updated Sep 10, 2015, 4:21 AM ]

image

Calling Agnes Martin, with a sprinkle of Sol Le Witt and a dab of Vincent Van Gogh - computer image processed by Günther Noack, Software Engineer

image

Agnes Martin - Cover of the Book “The Nineties And Beyond”

image

Sol Le Witt - Color Bands, 2000

image

Vincent Van Gogh, Madhouse Garden Painting

image

I dream of Spiral Horned Antelopes - Original photo by Zachi Evenor

Google research engineers  Alexander Mordvintsev, Christopher Olah, and Intern Mike Tyka are deep into artificial neural networks, image classification andspeech recognition. According to the authors, the way this works for computer generated images is they use “10-30 stacked layers of artificial neurons. Each image is fed into the input layer, which then talks to the next layer, until eventually the “output” layer is reached. The network’s “answer” comes from this final output layer.” So stacking and layering with links between the layers that are mathematically determined and tweaked is how they build it. What one layer looks for in another, and the amount of ‘noise’ involved is the ‘dreaming’ part of the equation for computers, if you want to draw an analogy. Aren’t dreams collections of random image noise in humans as well? 

The authors say they just pick an image, and then pick a layer and let the network algorithm go to work. Each layer has a difference in complexity. For example lower layers make strokes or “simple ornament-like patterns” because they deal mostly with edges and orientation.

image

Plain old photo of the simple sky and clouds. Not much going on here.

Using higher level layers, images begin to emerge, manifesting an ‘over interpretation’.

image

This is what you might see if you were on LSD or peyote or mescaline. Not these exact images of course, but depending on what you took, how high the dosage, what your life was like, and what environment you took the drugs in, something like this could happen to you - or your machine learning computer.

The engineers call this technique “inceptionism” and it is part of a neural network architecture

Some of the more layered images from the engineers work in inceptionism look like they came straight out of the 1960s and 1970s psychedelic movement.

image

Tutti Frutti ? Neural net “dreams”— generated purely from random noise, using a network trained on places by MIT Computer Science and AI Laboratory

image

Very radicle Timothy Leary on LSD-type images generated by inceptionism. Neural net “dreams”— generated purely from random noise, using a network trained on places by MIT Computer Science and AI Laboratory

image

This is your cat on drugs, you on drugs - or inceptionism according to your computer. Take your pick.

I used it on this picture, one from ‘environments’.

image

I Love NY parking lot photo

This is what the neural network saw when I tried it, and what it commented. The image is awful, but it did recognise it was “outdoor” “Alley” “Parking Lot” “natural light” “brick” “dirty” (hey that’s not fair!). 



This is the way, in a visualisation of the algorithm, they figure it out. They use something called ‘drawNet’ that combines images, though how it determines what to use and why is a bit too technical for my understanding at the moment, but its clear they run and compare lots of images in a database and get a composite image.

Comments