# t-SNE visualization of CNN codes

Description
I took 50,000 ILSVRC 2012 validation images, extracted the 4096-dimensional fc7 CNN (Convolutional Neural Network) features using Caffe and then used Barnes-Hut t-SNE to compute a 2-dimensional embedding that respects the high-dimensional (L2) distances. In other words, t-SNE arranges images that have a similar CNN (fc7) code nearby in the embedding.
Results
Embeddings where images are displayed exactly at their embedded location:
1000x1000 (240KB)
4000x4000 (3.2MB)
6000x6000 (6.6MB)
And below, embeddings where every position is filled with its nearest neighbor. Note that since the actual embedding is roughly circular, this leads to a visualization where the corners are a little "stretched" out and over-represented:
1000x1000 (307KB)
4000x4000 (4.9MB)
6000x6000 (10.8MB)
It's impossible to precisely embed 4096-dimensional space in 2 dimensions so in this final visualization, I take the 4000x4000 image and also draw the "seams", which measure the actual (L2) distance between the full 4096-dimensional codes of neighboring images in the grid. A bright red edge means the distance is high, and a black edge means the distance is low, in the original space.
Code, features, embedding
- Here's a link to the 50,000 image filenames, the 2-dimensional embeddings and my Matlab visualization code if you'd like to produce your own images. (1MB)
- And here are the raw 4096-dimensional CNN codes for the 50,000 images (as .mat file) if you'd like to re-run your own t-SNE or something else. (261MB)
Feel free to use any of the images/code anywhere. Ping me at @karpathy for questions.