\n
![\"\"](\"https://media.giphy.com/media/g04LCahxOOt22Q0UDj/giphy.gif\")
https://www.kaggle.com/davidwagnerkc/visualizing-33k-images-with-google-facets
\nKernel if you are interested in how to make it happen, inside of a notebook or hosted.
\nAnd a few pictures of how it allows you to interact with the data.
\nScattering across x and y axes (random values)
\n
Images in this demo are 60 x 30. Enough resolution to study the images and catch corner cases.
\n![\"\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/463588/11146/Screen%20Shot%202019-01-30%20at%202.43.44%20AM.png\")
Grouping vertically by number of samples per identity.
\n![\"\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/463588/11144/Screen%20Shot%202019-01-30%20at%202.01.15%20AM.png\")
Binning by image ratio. Blue indicates train set and red test set.
\n![\"\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/463588/11145/Screen%20Shot%202019-01-30%20at%202.41.41%20AM.png\")
https://www.kaggle.com/davidwagnerkc/visualizing-33k-images-with-google-facets
\nKernel if you are interested in how to make it happen, inside of a notebook or hosted.
\nAnd a few pictures of how it allows you to interact with the data.
\nScattering across x and y axes (random values)
\n
Images in this demo are 60 x 30. Enough resolution to study the images and catch corner cases.
\n
Grouping vertically by number of samples per identity.
\n
Binning by image ratio. Blue indicates train set and red test set.
\n
Nice visualization!.
\n\nThank for your ideas, I tried to visualize my embedding of validation by model which gets 0.76 local CV and 0.60 LB.
\nMy model was trained by Online Triplet Loss. It is a good way to debug what is wrong inside my models.
https://ngxbac.github.io/TSNE-Embedding-Visualisation/
\n\nBy running T-SNE :
\n* Perplexity: 30
\n* Learning rate: 0.1 \n* Steps: 2000
\nI can see some pairs are correct and uncorrect.
You can refer this repo for more information:
\nhttps://github.com/harveyslash/TSNE-Embedding-Visualisation
Oh, I like that. Probably helpful to have that third dimension to scatter on.Thanks for sharing.
\n\n@melgor for visualizing embeddings you might want to look at above repo.
", "rawMarkdown": "Oh, I like that. Probably helpful to have that third dimension to scatter on.Thanks for sharing.\n\n @melgor for visualizing embeddings you might want to look at above repo. " } ] }, { "id": 466227, "postDate": "2019-02-04T22:16:32.693Z", "content": "As a side note about data set visualization and tSNE type visualization particularly. Was watching the new fastai videos (https://www.youtube.com/watch?v=hkBa9pU-H48&t=) and they made a new tool for data labelling (and model training it looks like) that uses visualizations like the one @backaggle pointed out. Looks like a useful tool. https://platform.ai/
", "rawMarkdown": "As a side note about data set visualization and tSNE type visualization particularly. Was watching the new fastai videos (https://www.youtube.com/watch?v=hkBa9pU-H48&t=) and they made a new tool for data labelling (and model training it looks like) that uses visualizations like the one @backaggle pointed out. Looks like a useful tool. https://platform.ai/" }, { "id": 464446, "postDate": "2019-01-31T22:14:16.780Z", "content": "Very nice visualization, thansk for the kernel!\nCould you elaborate more about this idea:\nIf doing metric learning reduce dimensionality with PCA / tSNE and plot the images in that space
How do you think do approach this problem? From my investigation look like the Facets does not support such visualiation. We could create the embedding column in csv, but I'm not sure abouyt idea of displaing it.
", "rawMarkdown": "Very nice visualization, thansk for the kernel!\nCould you elaborate more about this idea:\n`If doing metric learning reduce dimensionality with PCA / tSNE and plot the images in that space`\n\nHow do you think do approach this problem? From my investigation look like the Facets does not support such visualiation. We could create the embedding column in csv, but I'm not sure abouyt idea of displaing it. ", "replies": [ { "id": 464450, "postDate": "2019-01-31T22:39:04.677Z", "content": "If you look at top controls of the Facet visualization there are Scatter X and Scatter Y drop downs. So if you took 128D embeddings down to 2D with some dimensionality reduction technique you could create two columns for for those dimensions and scatter them across x and y.
\n\nThat make sense?
\n\nUpdate: I did a quick check to make sure this actually works like I thought. Added GIF to the post showing scatter with random x and y.
", "rawMarkdown": "If you look at top controls of the Facet visualization there are Scatter X and Scatter Y drop downs. So if you took 128D embeddings down to 2D with some dimensionality reduction technique you could create two columns for for those dimensions and scatter them across x and y. \n\nThat make sense? \n\nUpdate: I did a quick check to make sure this actually works like I thought. Added GIF to the post showing scatter with random x and y. ", "votes": 1 }, { "id": 464642, "postDate": "2019-02-01T07:45:43.900Z", "content": "Thanks for the explenation. So you are thinking about doing PCA/t-SNE outside facets. That make sense.
", "rawMarkdown": "Thanks for the explenation. So you are thinking about doing PCA/t-SNE outside facets. That make sense. " } ] }, { "id": 464979, "postDate": "2019-02-01T23:44:48.967Z", "rawMarkdown": "", "isDeleted": true }, { "id": 617498, "postDate": "2019-09-04T07:31:57.437Z", "content": "Thanks!
", "rawMarkdown": "Thanks!", "votes": 1 } ], "comments": [ { "id": 464672, "author_name": "cab", "author_url": "", "post_date": "2019-02-01T09:14:28.450000", "content": "Nice visualization!.
\n\nThank for your ideas, I tried to visualize my embedding of validation by model which gets 0.76 local CV and 0.60 LB.
\nMy model was trained by Online Triplet Loss. It is a good way to debug what is wrong inside my models.
https://ngxbac.github.io/TSNE-Embedding-Visualisation/
\n\nBy running T-SNE :
\n* Perplexity: 30
\n* Learning rate: 0.1 \n* Steps: 2000
\nI can see some pairs are correct and uncorrect.
You can refer this repo for more information:
\nhttps://github.com/harveyslash/TSNE-Embedding-Visualisation
Oh, I like that. Probably helpful to have that third dimension to scatter on.Thanks for sharing.
\n\n@melgor for visualizing embeddings you might want to look at above repo.
", "votes": 0, "replies": [] } ] }, { "id": 466227, "author_name": "David Wagner", "author_url": "", "post_date": "2019-02-04T22:16:32.693000", "content": "As a side note about data set visualization and tSNE type visualization particularly. Was watching the new fastai videos (https://www.youtube.com/watch?v=hkBa9pU-H48&t=) and they made a new tool for data labelling (and model training it looks like) that uses visualizations like the one @backaggle pointed out. Looks like a useful tool. https://platform.ai/
", "votes": 0, "replies": [] }, { "id": 464446, "author_name": "Bartek", "author_url": "", "post_date": "2019-01-31T22:14:16.780000", "content": "Very nice visualization, thansk for the kernel!\nCould you elaborate more about this idea:\nIf doing metric learning reduce dimensionality with PCA / tSNE and plot the images in that space
How do you think do approach this problem? From my investigation look like the Facets does not support such visualiation. We could create the embedding column in csv, but I'm not sure abouyt idea of displaing it.
", "votes": 0, "replies": [ { "id": 464450, "author_name": "David Wagner", "author_url": "", "post_date": "2019-01-31T22:39:04.677000", "content": "If you look at top controls of the Facet visualization there are Scatter X and Scatter Y drop downs. So if you took 128D embeddings down to 2D with some dimensionality reduction technique you could create two columns for for those dimensions and scatter them across x and y.
\n\nThat make sense?
\n\nUpdate: I did a quick check to make sure this actually works like I thought. Added GIF to the post showing scatter with random x and y.
", "votes": 1, "replies": [] }, { "id": 464642, "author_name": "Bartek", "author_url": "", "post_date": "2019-02-01T07:45:43.900000", "content": "Thanks for the explenation. So you are thinking about doing PCA/t-SNE outside facets. That make sense.
", "votes": 0, "replies": [] } ] }, { "id": 464979, "author_name": "", "author_url": "", "post_date": "2019-02-01T23:44:48.967000", "content": "", "votes": 0, "replies": [] }, { "id": 617498, "author_name": "Alexandr Krasovskiy", "author_url": "", "post_date": "2019-09-04T07:31:57.437000", "content": "Thanks!
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "463588": "~~https://davidwagnerkc.github.io/~~\n\n~~Live full screen demo with this competition's dataset. (Takes a minute to load, about ~100MB)~~\n\nhttps://www.kaggle.com/davidwagnerkc/visualizing-33k-images-with-google-facets\n\nKernel if you are interested in how to make it happen, inside of a notebook or hosted. \n\nAnd a few pictures of how it allows you to interact with the data. \n\nScattering across x and y axes (random values)\n\n\nImages in this demo are 60 x 30. Enough resolution to study the images and catch corner cases.\n![][3]\n\nGrouping vertically by number of samples per identity. \n![][1]\n\nBinning by image ratio. Blue indicates train set and red test set.\n![][2]\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/463588/11144/Screen%20Shot%202019-01-30%20at%202.01.15%20AM.png\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/463588/11145/Screen%20Shot%202019-01-30%20at%202.41.41%20AM.png\n [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/463588/11146/Screen%20Shot%202019-01-30%20at%202.43.44%20AM.png", "464672": "Nice visualization!. \n\nThank for your ideas, I tried to visualize my embedding of validation by model which gets `0.76` local CV and `0.60` LB. \nMy model was trained by Online Triplet Loss. It is a good way to debug what is wrong inside my models. \n\nhttps://ngxbac.github.io/TSNE-Embedding-Visualisation/ \n\nBy running `T-SNE ` : \n* Perplexity: 30 \n* Learning rate: 0.1 \n* Steps: 2000 \nI can see some pairs are correct and uncorrect. \n\nYou can refer this repo for more information: \nhttps://github.com/harveyslash/TSNE-Embedding-Visualisation", "466227": "As a side note about data set visualization and tSNE type visualization particularly. Was watching the new fastai videos (https://www.youtube.com/watch?v=hkBa9pU-H48&t=) and they made a new tool for data labelling (and model training it looks like) that uses visualizations like the one @backaggle pointed out. Looks like a useful tool. https://platform.ai/", "464446": "Very nice visualization, thansk for the kernel!\nCould you elaborate more about this idea:\n`If doing metric learning reduce dimensionality with PCA / tSNE and plot the images in that space`\n\nHow do you think do approach this problem? From my investigation look like the Facets does not support such visualiation. We could create the embedding column in csv, but I'm not sure abouyt idea of displaing it. ", "464979": "", "617498": "Thanks!" } }