J. Horacsek
M. Zeiler and R. Fergus
In general, neural networks are hard to decipher.
Can we "look" into networks to inspect their behaviour?
We can easily visualize the first convolutional layer. The first conv layer is often a collection of N x N (by 3) kernels.
Higher layers are trickier, they're often made up of a collection (say 64) M x M filters.
The model used in the paper is different than VGG16
What does the network see?
We can easily push data through the network and inspect the activations at each layer.
This isn't super helpful...
Visualize filters in the first layer. This is fairly easy to do since they're all just 3 x 3 x 3 filters.
Deeper layers need something more sophisticated
Go down to a specific layer, choose a feature map, take max activation
Basically, keep track of what activated a pool through the forward pass
Pass unpooled map through ReLU
Deconvolution using the transpose of the original filters
For a given layer, pick some random features. Find images that maximally activate that feature.
For a given layer, pick some random features. Find images that maximally activate that feature.
As a diagnostic tool, it allows you to pick out problematic filters.