![\"confusion](\"https://www.kaggleusercontent.com/kf/36014948/eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..p-9FbbYVC9NroRNvmJmdqg.SgXJzsnNYbUF_VcE1Fz2yKCv0P2fco5KvJJ6P7Xa9ulyNxZ_n_O-XNAaPHE2Dcmu3ATBU1nWV3N0iMPbBcCkpAuOiw5AXX3jaXFPXZb73zRxN1TgYcdaWCf7-gbU2t_dcMBaJ9j9hqd_t_MNSiGHAUjCfLTKBmhH7nWErMUBO4YzE2EL1WIp7mVQ71Puwk1G9MIFsMY78uFx-SRwyyKE2e2gw4OLWLQuz4vrcbCk1-ryujQxbnMz01c3ZN8fBnuiUUlT7ZJnZpEbYp2hGHTo4ydaz7q0zr_Sd4198ZTSudd0wKd6O8UI_XobR3jAC62TBZfc4DAFZpJ4-Epjbe8jZmsKiAWJwfa3Scj9odoQHq6aBzBVGJGugt9XosWS6azWOfe4vAcESKD4dhfDazv-8kfAe_WTEM5TvCJaErb6IFBUxDeCzcH1tW4-4rMwMqCoFcqvuqIz7aC8Wg6dALkMFH_vj7IzUSAcCGcNpE18W6uDdm5MFLqxakD1aWqFZRefvAtSEYmbW9NTODfpgE53V2ydrEim93Ld0Tf81Owo5TPqYwq8nSIHLYCP43uoLWvquP0ZrCMSn_RsszzxH4R81orzg-SgcXyStUf5v0YHARWPwsEMkro7wDX6zuWLvfp79tkLQfMyjrtFp2RdABFl0R2qIeZKkinDVwAYWD05yXOxUPzwk5P4N1JrPkwlhhY8.ZnB_Le_j3oFo3cskfI7JqQ/__results___files/__results___15_1.png\")
\nYou can find the code to reproduce the figure here.In this competition we are just supposed to classify an image as stego or not. Binary classification seems to make intuitive sense. But it seems that the classification performance depends a lot on the quality of image and algorithm used to embed data (UERD, JUNIWARD etc). So, i wanted the network to be explicitly aware of this distinction. Having 9 classes (based on image quality and embedding scheme) seemed to be a good way to make the network aware that the learnt features need to be different .
\n\n\nYou can find the code to reproduce the figure here.
The idea of having granular classes has been investigated before. The paper, Understanding the Impact of Label Granularity on CNN-based Image Classification concludes that training CNNs with fine-grain labels improves both network’s optimization and generalization capabilities, as intuitively it encourages the network to learn more features, and hence increases classification accuracy on coarse-grain classes under all datasets considered. \n> In recent years, supervised learning using Convolutional Neural Networks (CNNs) has achieved great success in image classification tasks, and large scale labeled datasetshave contributed significantly to this achievement. However, the definition of al abelis often application dependent. For example,an image of a cat can be labeled as “cat” or perhaps more specifically “Persian cat.” We refer to this as label granularity.In this paper, we conduct extensive experiments using various datasets to demonstrate and analyze how and why training based on fine-grain labeling, such as “Persian cat” can improve CNN accuracy on classifying coarse-grain classes, in this case “cat.”
", "messages": [ { "id": "872370", "postDate": "06/03/2020 06:11:26", "content": "In this competition we are just supposed to classify an image as stego or not. Binary classification seems to make intuitive sense. But it seems that the classification performance depends a lot on the quality of image and algorithm used to embed data (UERD, JUNIWARD etc). So, i wanted the network to be explicitly aware of this distinction. Having 9 classes (based on image quality and embedding scheme) seemed to be a good way to make the network aware that the learnt features need to be different .
\n\n\nYou can find the code to reproduce the figure here.
The idea of having granular classes has been investigated before. The paper, Understanding the Impact of Label Granularity on CNN-based Image Classification concludes that training CNNs with fine-grain labels improves both network’s optimization and generalization capabilities, as intuitively it encourages the network to learn more features, and hence increases classification accuracy on coarse-grain classes under all datasets considered. \n> In recent years, supervised learning using Convolutional Neural Networks (CNNs) has achieved great success in image classification tasks, and large scale labeled datasetshave contributed significantly to this achievement. However, the definition of al abelis often application dependent. For example,an image of a cat can be labeled as “cat” or perhaps more specifically “Persian cat.” We refer to this as label granularity.In this paper, we conduct extensive experiments using various datasets to demonstrate and analyze how and why training based on fine-grain labeling, such as “Persian cat” can improve CNN accuracy on classifying coarse-grain classes, in this case “cat.”
", "rawMarkdown": "In this competition we are just supposed to classify an image as stego or not. Binary classification seems to make intuitive sense. But it seems that the classification performance depends a lot on the quality of image and algorithm used to embed data (UERD, JUNIWARD etc). So, i wanted the network to be explicitly aware of this distinction. Having 9 classes (based on image quality and embedding scheme) seemed to be a good way to make the network aware that the learnt features need to be different .\n\n\n\nYou can find the code to reproduce the figure [here](https://www.kaggle.com/meaninglesslives/alaska2-analyzing-model-predictions).\n\nThe idea of having granular classes has been investigated before. The paper, [Understanding the Impact of Label Granularity on CNN-based Image Classification](https://arxiv.org/pdf/1901.07012.pdf) concludes that training CNNs with fine-grain labels improves both network’s optimization and generalization capabilities, as intuitively it encourages the network to learn more features, and hence increases classification accuracy on coarse-grain classes under all datasets considered. \n> In recent years, supervised learning using Convolutional Neural Networks (CNNs) has achieved great success in image classification tasks, and large scale labeled datasetshave contributed significantly to this achievement. However, the definition of al abelis often application dependent. For example,an image of a cat can be labeled as “cat” or perhaps more specifically “Persian cat.” We refer to this as label granularity.In this paper, we conduct extensive experiments using various datasets to demonstrate and analyze how and why training based on fine-grain labeling, such as “Persian cat” can improve CNN accuracy on classifying coarse-grain classes, in this case “cat.”", "votes": null }, { "id": "872452", "postDate": "06/03/2020 07:42:51", "content": "Hello There,
\n\nI totally agree that multi-class seem to me to be a good idea.\nHowever, those 3 x 3 classes are quite different because on the one hand the embedding scheme is not known when you inspect an image, typically from the testing set while on the other hand the JPEG quality factor is publicly know (it is part of the JPEG header and is needed for decompression, so whatever the embedding, if you want the image to remain readable you need to give this information).
\n\nTherefore you can split the testing set to use different models for different JPEG quality factor but you have to use a multi-class (or several binary classifier + a merging function) for embedding scheme
\n\nRémi
", "rawMarkdown": "Hello There,\n\nI totally agree that multi-class seem to me to be a good idea.\nHowever, those 3 x 3 classes are quite different because on the one hand the embedding scheme is not known when you inspect an image, typically from the testing set while on the other hand the JPEG quality factor is publicly know (it is part of the JPEG header and is needed for decompression, so whatever the embedding, if you want the image to remain readable you need to give this information).\n\nTherefore you can split the testing set to use different models for different JPEG quality factor but you have to use a multi-class (or several binary classifier + a merging function) for embedding scheme\n\nRémi", "votes": null }, { "id": "872657", "postDate": "06/03/2020 11:57:54", "content": "Thanks for sharing, I am curious to know why you don't also split Cover into 3 quality factors?
", "rawMarkdown": "Thanks for sharing, I am curious to know why you don't also split Cover into 3 quality factors?", "votes": null }, { "id": "872794", "postDate": "06/03/2020 14:05:14", "content": "Interesting.\nLooks like the quality factor can be 100% accurately learned by the model.\nThere are confusions between cover and embeddings or between different embedding types, but never between embeddings of different quality.
", "rawMarkdown": "Interesting.\nLooks like the quality factor can be 100% accurately learned by the model.\nThere are confusions between cover and embeddings or between different embedding types, but never between embeddings of different quality.", "votes": null }, { "id": "873178", "postDate": "06/03/2020 21:56:47", "content": "In my early experiments, the results were better using 10 classes rather than 12 classes. Holding all other factors (batch size, learning rate, network, epoch count, augmentations) constant:
\n\n12 classes = LB 0.744\n10 classes = LB 0.870
\n\nI have made many changes since then, but based on this test I abandoned 12 class models.
", "rawMarkdown": "In my early experiments, the results were better using 10 classes rather than 12 classes. Holding all other factors (batch size, learning rate, network, epoch count, augmentations) constant:\n\n12 classes = LB 0.744\n10 classes = LB 0.870\n\nI have made many changes since then, but based on this test I abandoned 12 class models.", "votes": null }, { "id": "873444", "postDate": "06/04/2020 06:43:35", "content": "Have you calculated ACR as described in the paper?
", "rawMarkdown": "Have you calculated ACR as described in the paper?", "votes": null }, { "id": "873912", "postDate": "06/04/2020 14:13:18", "content": "Could you please give the code of your confusion matrix ?
", "rawMarkdown": "Could you please give the code of your confusion matrix ?", "votes": null } ], "comments": [ { "id": 872452, "author_name": "remicogranne", "author_url": "", "post_date": "06/03/2020 07:42:51", "content": "Hello There,
\n\nI totally agree that multi-class seem to me to be a good idea.\nHowever, those 3 x 3 classes are quite different because on the one hand the embedding scheme is not known when you inspect an image, typically from the testing set while on the other hand the JPEG quality factor is publicly know (it is part of the JPEG header and is needed for decompression, so whatever the embedding, if you want the image to remain readable you need to give this information).
\n\nTherefore you can split the testing set to use different models for different JPEG quality factor but you have to use a multi-class (or several binary classifier + a merging function) for embedding scheme
\n\nRémi
", "votes": null, "replies": [] }, { "id": 872657, "author_name": "vincentpoont2", "author_url": "", "post_date": "06/03/2020 11:57:54", "content": "Thanks for sharing, I am curious to know why you don't also split Cover into 3 quality factors?
", "votes": null, "replies": [ { "id": 873178, "author_name": "jbfarrar", "author_url": "", "post_date": "06/03/2020 21:56:47", "content": "In my early experiments, the results were better using 10 classes rather than 12 classes. Holding all other factors (batch size, learning rate, network, epoch count, augmentations) constant:
\n\n12 classes = LB 0.744\n10 classes = LB 0.870
\n\nI have made many changes since then, but based on this test I abandoned 12 class models.
", "votes": null, "replies": [] } ] }, { "id": 872794, "author_name": "vzaguskin", "author_url": "", "post_date": "06/03/2020 14:05:14", "content": "Interesting.\nLooks like the quality factor can be 100% accurately learned by the model.\nThere are confusions between cover and embeddings or between different embedding types, but never between embeddings of different quality.
", "votes": null, "replies": [] }, { "id": 873444, "author_name": "magauiya", "author_url": "", "post_date": "06/04/2020 06:43:35", "content": "Have you calculated ACR as described in the paper?
", "votes": null, "replies": [] }, { "id": 873912, "author_name": "raphaelensae", "author_url": "", "post_date": "06/04/2020 14:13:18", "content": "Could you please give the code of your confusion matrix ?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "872370": "In this competition we are just supposed to classify an image as stego or not. Binary classification seems to make intuitive sense. But it seems that the classification performance depends a lot on the quality of image and algorithm used to embed data (UERD, JUNIWARD etc). So, i wanted the network to be explicitly aware of this distinction. Having 9 classes (based on image quality and embedding scheme) seemed to be a good way to make the network aware that the learnt features need to be different .\n\n\n\nYou can find the code to reproduce the figure [here](https://www.kaggle.com/meaninglesslives/alaska2-analyzing-model-predictions).\n\nThe idea of having granular classes has been investigated before. The paper, [Understanding the Impact of Label Granularity on CNN-based Image Classification](https://arxiv.org/pdf/1901.07012.pdf) concludes that training CNNs with fine-grain labels improves both network’s optimization and generalization capabilities, as intuitively it encourages the network to learn more features, and hence increases classification accuracy on coarse-grain classes under all datasets considered. \n> In recent years, supervised learning using Convolutional Neural Networks (CNNs) has achieved great success in image classification tasks, and large scale labeled datasetshave contributed significantly to this achievement. However, the definition of al abelis often application dependent. For example,an image of a cat can be labeled as “cat” or perhaps more specifically “Persian cat.” We refer to this as label granularity.In this paper, we conduct extensive experiments using various datasets to demonstrate and analyze how and why training based on fine-grain labeling, such as “Persian cat” can improve CNN accuracy on classifying coarse-grain classes, in this case “cat.”", "872452": "Hello There,\n\nI totally agree that multi-class seem to me to be a good idea.\nHowever, those 3 x 3 classes are quite different because on the one hand the embedding scheme is not known when you inspect an image, typically from the testing set while on the other hand the JPEG quality factor is publicly know (it is part of the JPEG header and is needed for decompression, so whatever the embedding, if you want the image to remain readable you need to give this information).\n\nTherefore you can split the testing set to use different models for different JPEG quality factor but you have to use a multi-class (or several binary classifier + a merging function) for embedding scheme\n\nRémi", "872657": "Thanks for sharing, I am curious to know why you don't also split Cover into 3 quality factors?", "872794": "Interesting.\nLooks like the quality factor can be 100% accurately learned by the model.\nThere are confusions between cover and embeddings or between different embedding types, but never between embeddings of different quality.", "873178": "In my early experiments, the results were better using 10 classes rather than 12 classes. Holding all other factors (batch size, learning rate, network, epoch count, augmentations) constant:\n\n12 classes = LB 0.744\n10 classes = LB 0.870\n\nI have made many changes since then, but based on this test I abandoned 12 class models.", "873444": "Have you calculated ACR as described in the paper?", "873912": "Could you please give the code of your confusion matrix ?" }, "source": "meta" }