![\"\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/933196/16355/umap_projection_.jpg\")
Normally after I set my validation, I go iterative starting from a basic model. I continuously check in what kind of examples model makes mistakes and try to improve on that. But in this competition, I can't visually see the difference between cover and stego images. When the model makes a mistake on validation set, I can only have some empathy, nothing else:) Do you experience the same?
", "messages": [ { "id": "932972", "postDate": "07/17/2020 12:00:52", "content": "Normally after I set my validation, I go iterative starting from a basic model. I continuously check in what kind of examples model makes mistakes and try to improve on that. But in this competition, I can't visually see the difference between cover and stego images. When the model makes a mistake on validation set, I can only have some empathy, nothing else:) Do you experience the same?
", "rawMarkdown": "Normally after I set my validation, I go iterative starting from a basic model. I continuously check in what kind of examples model makes mistakes and try to improve on that. But in this competition, I can't visually see the difference between cover and stego images. When the model makes a mistake on validation set, I can only have some empathy, nothing else:) Do you experience the same?", "votes": null }, { "id": "933046", "postDate": "07/17/2020 12:58:53", "content": "I experienced the same. This dataset is unique.
", "rawMarkdown": "I experienced the same. This dataset is unique.", "votes": null }, { "id": "933111", "postDate": "07/17/2020 13:50:59", "content": "Same here. \n.. can't visually see the difference between cover and stego images - damn true. ๐ฏ \nI think the problem is kind of, the payload is hidden in DCT, and we're using pre-trained model that generalizes on pixel values.
\"I think the problem is kind of, the payload is hidden in DCT, and we're using pre-trained model that generalizes on pixel values.\"
\n\nI've been wondering if any of teams trained imagenet from scratch with dct as input, on efficientnet. And used that as pretraining for this task. So no pixelwise generalization involved. Alternatively, train imagenet from scratch on ycbcr, I was surprised to find little publically in this area of alternate imagenet colorspaces. The rules allow it. Not long to find out ...
", "rawMarkdown": "\"I think the problem is kind of, the payload is hidden in DCT, and we're using pre-trained model that generalizes on pixel values.\"\n\nI've been wondering if any of teams trained imagenet from scratch with dct as input, on efficientnet. And used that as pretraining for this task. So no pixelwise generalization involved. Alternatively, train imagenet from scratch on ycbcr, I was surprised to find little publically in this area of alternate imagenet colorspaces. The rules allow it. Not long to find out ...", "votes": null }, { "id": "933147", "postDate": "07/17/2020 14:24:39", "content": "I experienced the same. I look forward to hearing after the competition how others are doing it.
", "rawMarkdown": "I experienced the same. I look forward to hearing after the competition how others are doing it.", "votes": null }, { "id": "933192", "postDate": "07/17/2020 14:57:27", "content": "That would be great. But I believe that would be also a pure research work.
", "rawMarkdown": "That would be great. But I believe that would be also a pure research work.", "votes": null }, { "id": "933196", "postDate": "07/17/2020 15:00:35", "content": "You may not be able to visualize the differences very well in the image space, but one way to gain insight is to project the embeddings from the last layer of your network using UMAP or tSNE and look for where your models are having issues discriminating along class boundaries. It should give some clues as to what's going on. Here's an example from a high performing model.
\n\n
\n\n\nI've been wondering if any of teams trained imagenet from scratch on efficientnet using dct or ycbcr as input.
\n
Now that's an idea.
", "rawMarkdown": "> I've been wondering if any of teams trained imagenet from scratch on efficientnet using dct or ycbcr as input. \n\nNow that's an idea.", "votes": null }, { "id": "933206", "postDate": "07/17/2020 15:06:02", "content": "Great visualization and this give a new perspective to others.\nthanks for sharing it.\nAwesome! ๐
", "rawMarkdown": "Great visualization and this give a new perspective to others.\nthanks for sharing it.\nAwesome! ๐", "votes": null }, { "id": "935351", "postDate": "07/19/2020 10:03:39", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F113660%2F120121c004d2c3fd630aab652635db27%2FSelection_037.png?generation=1595152988715693&alt=media\")
you can make difference images like this to inspect what kinds of micro-pattern are detected or missed
", "rawMarkdown": "\n\n\nyou can make difference images like this to inspect what kinds of micro-pattern are detected or missed", "votes": null }, { "id": "935931", "postDate": "07/19/2020 19:33:53", "content": "If you can visually see the difference between cover and stego images - in a few hours you can manually label all test images๐
", "rawMarkdown": "If you can visually see the difference between cover and stego images - in a few hours you can manually label all test images๐", "votes": null }, { "id": "937323", "postDate": "07/20/2020 23:44:08", "content": "Nice trick. I did the same thing in Melanoma Comp to compare the training images and test images distribution. Also when using external data, you can compare how well external image data overlaps train and test https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/168028
\nUsing UMAP or t-SNE is a powerful tool in image comps that often goes overlooked. Now with RAPIDS cuML, you can perform UMAP and t-SNE in seconds on large datasets!
", "rawMarkdown": "Nice trick. I did the same thing in Melanoma Comp to compare the training images and test images distribution. Also when using external data, you can compare how well external image data overlaps train and test https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/168028\n\nUsing UMAP or t-SNE is a powerful tool in image comps that often goes overlooked. Now with RAPIDS cuML, you can perform UMAP and t-SNE in seconds on large datasets!", "votes": null } ], "comments": [ { "id": 935931, "author_name": "golubev", "author_url": "", "post_date": "07/19/2020 19:33:53", "content": "If you can visually see the difference between cover and stego images - in a few hours you can manually label all test images๐
", "votes": null, "replies": [] }, { "id": 933046, "author_name": "titericz", "author_url": "", "post_date": "07/17/2020 12:58:53", "content": "I experienced the same. This dataset is unique.
", "votes": null, "replies": [] }, { "id": 933111, "author_name": "ipythonx", "author_url": "", "post_date": "07/17/2020 13:50:59", "content": "Same here. \n.. can't visually see the difference between cover and stego images - damn true. ๐ฏ \nI think the problem is kind of, the payload is hidden in DCT, and we're using pre-trained model that generalizes on pixel values.
\"I think the problem is kind of, the payload is hidden in DCT, and we're using pre-trained model that generalizes on pixel values.\"
\n\nI've been wondering if any of teams trained imagenet from scratch with dct as input, on efficientnet. And used that as pretraining for this task. So no pixelwise generalization involved. Alternatively, train imagenet from scratch on ycbcr, I was surprised to find little publically in this area of alternate imagenet colorspaces. The rules allow it. Not long to find out ...
", "votes": null, "replies": [] }, { "id": 933192, "author_name": "ipythonx", "author_url": "", "post_date": "07/17/2020 14:57:27", "content": "That would be great. But I believe that would be also a pure research work.
", "votes": null, "replies": [] }, { "id": 933199, "author_name": "authman", "author_url": "", "post_date": "07/17/2020 15:01:04", "content": "\n\n\nI've been wondering if any of teams trained imagenet from scratch on efficientnet using dct or ycbcr as input.
\n
Now that's an idea.
", "votes": null, "replies": [] } ] }, { "id": 933147, "author_name": "zaburo", "author_url": "", "post_date": "07/17/2020 14:24:39", "content": "I experienced the same. I look forward to hearing after the competition how others are doing it.
", "votes": null, "replies": [] }, { "id": 933196, "author_name": "tivfrvqhs5", "author_url": "", "post_date": "07/17/2020 15:00:35", "content": "You may not be able to visualize the differences very well in the image space, but one way to gain insight is to project the embeddings from the last layer of your network using UMAP or tSNE and look for where your models are having issues discriminating along class boundaries. It should give some clues as to what's going on. Here's an example from a high performing model.
\n\n
Great visualization and this give a new perspective to others.\nthanks for sharing it.\nAwesome! ๐
", "votes": null, "replies": [] }, { "id": 937323, "author_name": "cdeotte", "author_url": "", "post_date": "07/20/2020 23:44:08", "content": "Nice trick. I did the same thing in Melanoma Comp to compare the training images and test images distribution. Also when using external data, you can compare how well external image data overlaps train and test https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/168028
\nUsing UMAP or t-SNE is a powerful tool in image comps that often goes overlooked. Now with RAPIDS cuML, you can perform UMAP and t-SNE in seconds on large datasets!
", "votes": null, "replies": [] } ] }, { "id": 935351, "author_name": "hengck23", "author_url": "", "post_date": "07/19/2020 10:03:39", "content": "
you can make difference images like this to inspect what kinds of micro-pattern are detected or missed
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "932972": "Normally after I set my validation, I go iterative starting from a basic model. I continuously check in what kind of examples model makes mistakes and try to improve on that. But in this competition, I can't visually see the difference between cover and stego images. When the model makes a mistake on validation set, I can only have some empathy, nothing else:) Do you experience the same?", "933046": "I experienced the same. This dataset is unique.", "933111": "Same here. \n`.. can't visually see the difference between cover and stego images` - damn true. ๐ฏ \nI think the problem is kind of, the payload is hidden in DCT, and we're using pre-trained model that generalizes on pixel values.", "933122": "\"I think the problem is kind of, the payload is hidden in DCT, and we're using pre-trained model that generalizes on pixel values.\"\n\nI've been wondering if any of teams trained imagenet from scratch with dct as input, on efficientnet. And used that as pretraining for this task. So no pixelwise generalization involved. Alternatively, train imagenet from scratch on ycbcr, I was surprised to find little publically in this area of alternate imagenet colorspaces. The rules allow it. Not long to find out ...", "933147": "I experienced the same. I look forward to hearing after the competition how others are doing it.", "933192": "That would be great. But I believe that would be also a pure research work.", "933196": "You may not be able to visualize the differences very well in the image space, but one way to gain insight is to project the embeddings from the last layer of your network using UMAP or tSNE and look for where your models are having issues discriminating along class boundaries. It should give some clues as to what's going on. Here's an example from a high performing model.\n\n", "933199": "> I've been wondering if any of teams trained imagenet from scratch on efficientnet using dct or ycbcr as input. \n\nNow that's an idea.", "933206": "Great visualization and this give a new perspective to others.\nthanks for sharing it.\nAwesome! ๐", "935351": "\n\n\nyou can make difference images like this to inspect what kinds of micro-pattern are detected or missed", "935931": "If you can visually see the difference between cover and stego images - in a few hours you can manually label all test images๐", "937323": "Nice trick. I did the same thing in Melanoma Comp to compare the training images and test images distribution. Also when using external data, you can compare how well external image data overlaps train and test https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/168028\n\nUsing UMAP or t-SNE is a powerful tool in image comps that often goes overlooked. Now with RAPIDS cuML, you can perform UMAP and t-SNE in seconds on large datasets!" }, "source": "meta" }