"Psychedelic Filter"
Boy, Asia, John
How can we create a filter that
makes any image look psychedelic?
More generally, how can
we apply the style of one photo
to the content of another?
Several years ago,
this problem would have been
considered very difficult
or even impossible.
Recent research
in neural networks
has made it possible!
Google DeepDream - July 2015
"Neural Algorithm of Artistic Style" - Sept 2015
PIKAZO - March 2016
PRISMA - June 2016
Artist: Tyler Barth + PRISMA
How does it work?
"Neural Algorithm of Artistic Style"
-
Select a neural network trained for visual classification.
-
Compute the output at each layer for the style and content photos.
- Keep trying different inputs to the network to match the output at each layer.
- Start with a random image, then iterate with updated images (using backpropagation).
-
Select a neural network trained for visual classification.
- Compute the output at each layer for the style and content photos.
- Keep trying different inputs to the network to match the output at each layer.
- Start with a random image, then iterate with updated images (using backpropagation).
"VGG" Convolutional Neural Network
trained for visual classification
Visual Geometry Group, Oxford University (2014)
Put the photos into the neural network and compare the layer outputs.
Sum up the mismatch with the style and content photos. (For comparing style, need to apply a covariance computation.)
Optimization
- Define the error function as a weighted sum of the mismatch with the style and content photos.
- Use gradient descent on this function to update the (initially random) input image.
- In this case, gradient descent turns out to be backpropagation through the neural network.
- Define the error function as a weighted sum of the mismatch with the style and content photos.
- Use gradient descent on this function to update the (initially random) input image.
- In this case, gradient descent turns out to be backpropagation through the neural network.
Use the error to update the original input image
(back propagation)
After many iterations (e.g. 1000), the input image converges to something that combines the style and content!
DEMO
https://github.com/jcjohnson/neural-style
Python / Tensorflow
Set up on server (AWS EC2)
DEMO result
DEMO result
Compare with PIKAZO
BIG CHALLENGE:
How to speed up computations?
Our current implementation
(using the "neuralstyle" code)
takes ~40 minutes
PIKAZO takes ~10 minutes
PRISMA takes ~10 seconds
HOW?!?!
Stanford paper, March 2016
Perceptual Losses for Real-time Style Transfer
Run training images through
image transformation network (W)
Use the error to update W
(back propagation)
Keep putting LOTS of training images
Keep updating the image transform.
Real-time style transfer!!!
Fast (no iteration) and similar to neural style.
Content Neural Style "Real-time"
BIGGER CHALLENGE:
All known implementations are
running on server side.
Can this be done in the mobile app?
GAME TIME
Who is this?
Who is this?
Who is this?
Who is this?
Who is this?
Who is this?
Who is this?
Can you tell who these people are?
Who is this?
What is this?
Our original goal ---
Psychedelic filter
Psychedelic Filters
By piccollage
