Hi everyone,
\n\nFirst of all I want to congratulate everyone for the hard work and the effort put in this challenge. So, It is a fact that when we do research there are tons of approaches that we try and do not work and there are only a few that actually improves the State of the Art. That’s why I have always thought that it is important to share our achievements but also our failures. That’s why I want to share with you my failed ideas and what I have learnt during these months. So, let’s discuss my approaches and what I learnt during this journey.
\n\nI highly recommend checking this post as it gives the first notions to tackle this challenge.
\n\nWhat I have learnt:\n1. Do not modify the frequencies of the image. That means: do not resize, do not crop, do not perform any augmentation that might affect the frequencies of the image. Rotations and flips are ok. Why? Basically some steganography uses high frequencies to hide their message.\n2. Steganalysis seems to perform better in RGB space in pre-trained NN. YCbCr and DCT do not perform as well with the same pretrained network. Maybe training the same network with ImageNet but with YCbCr/DCT spaces would improve?\n3. Big batch size is important in order to allow batch normalization work correctly. If the network is too big to fit in the GPU/TPU we need to find other approaches.\n4. TTA always helped me, giving more importance to the original image than the other transformation. Isometry improved my score a bit.\n5. Using KFold worked for me. The most important part is separating the data not randomly but including the same image (Cover + 3 algorithms) improves a lot. Not all folds were useful though.\n6. Balancing the batch with BalanceClassSampler was positive\n7. MSE with a binary approach did not really work. It seems that pixel difference is not enough. I would have liked to check which algorithms were more prone to be detected with pixel difference and which not.\n8. Multiclass using 10 classes did not provide a good baseline result. It seems that it is an extra effort to separate the features by JPEG quality factor. It seems that using only 4 classes is enough to separate better the features.
\n\nWhat I implemented and did not worked for me:\n1. My idea was that attention might focus on regions of the image where possible hidden data might be in it. For instance, the data might be hidden in high frequency regions but it seems that is not always like this. I tried implementing it, it seemed good at the beginning but it ended up overfitting. Selecting the layers to extract the attention was the worst part. The paper did not mention anything so I tried to tune it manually but of course did not work. This is the paper.
\n\nI’ll leave here my GitHub in case anyone wants to give it a look!
\n\nCheers!
", "messages": [ { "id": "937335", "postDate": "07/21/2020 00:02:38", "content": "Hi everyone,
\n\nFirst of all I want to congratulate everyone for the hard work and the effort put in this challenge. So, It is a fact that when we do research there are tons of approaches that we try and do not work and there are only a few that actually improves the State of the Art. That’s why I have always thought that it is important to share our achievements but also our failures. That’s why I want to share with you my failed ideas and what I have learnt during these months. So, let’s discuss my approaches and what I learnt during this journey.
\n\nI highly recommend checking this post as it gives the first notions to tackle this challenge.
\n\nWhat I have learnt:\n1. Do not modify the frequencies of the image. That means: do not resize, do not crop, do not perform any augmentation that might affect the frequencies of the image. Rotations and flips are ok. Why? Basically some steganography uses high frequencies to hide their message.\n2. Steganalysis seems to perform better in RGB space in pre-trained NN. YCbCr and DCT do not perform as well with the same pretrained network. Maybe training the same network with ImageNet but with YCbCr/DCT spaces would improve?\n3. Big batch size is important in order to allow batch normalization work correctly. If the network is too big to fit in the GPU/TPU we need to find other approaches.\n4. TTA always helped me, giving more importance to the original image than the other transformation. Isometry improved my score a bit.\n5. Using KFold worked for me. The most important part is separating the data not randomly but including the same image (Cover + 3 algorithms) improves a lot. Not all folds were useful though.\n6. Balancing the batch with BalanceClassSampler was positive\n7. MSE with a binary approach did not really work. It seems that pixel difference is not enough. I would have liked to check which algorithms were more prone to be detected with pixel difference and which not.\n8. Multiclass using 10 classes did not provide a good baseline result. It seems that it is an extra effort to separate the features by JPEG quality factor. It seems that using only 4 classes is enough to separate better the features.
\n\nWhat I implemented and did not worked for me:\n1. My idea was that attention might focus on regions of the image where possible hidden data might be in it. For instance, the data might be hidden in high frequency regions but it seems that is not always like this. I tried implementing it, it seemed good at the beginning but it ended up overfitting. Selecting the layers to extract the attention was the worst part. The paper did not mention anything so I tried to tune it manually but of course did not work. This is the paper.
\n\nI’ll leave here my GitHub in case anyone wants to give it a look!
\n\nCheers!
", "rawMarkdown": "Hi everyone,\n\nFirst of all I want to congratulate everyone for the hard work and the effort put in this challenge. So, It is a fact that when we do research there are tons of approaches that we try and do not work and there are only a few that actually improves the State of the Art. That’s why I have always thought that it is important to share our achievements but also our failures. That’s why I want to share with you my failed ideas and what I have learnt during these months. So, let’s discuss my approaches and what I learnt during this journey. \n\nI highly recommend checking this [post](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/155392) as it gives the first notions to tackle this challenge.\n\n**What I have learnt:**\n1. Do not modify the frequencies of the image. That means: do not resize, do not crop, do not perform any augmentation that might affect the frequencies of the image. Rotations and flips are ok. Why? Basically some steganography uses high frequencies to hide their message.\n2. Steganalysis seems to perform better in RGB space in pre-trained NN. YCbCr and DCT do not perform as well with the same pretrained network. Maybe training the same network with ImageNet but with YCbCr/DCT spaces would improve?\n3. Big batch size is important in order to allow batch normalization work correctly. If the network is too big to fit in the GPU/TPU we need to find other approaches.\n4. TTA always helped me, giving more importance to the original image than the other transformation. Isometry improved my score a bit.\n5. Using KFold worked for me. The most important part is separating the data not randomly but including the same image (Cover + 3 algorithms) improves a lot. Not all folds were useful though.\n6. Balancing the batch with BalanceClassSampler was positive\n7. MSE with a binary approach did not really work. It seems that pixel difference is not enough. I would have liked to check which algorithms were more prone to be detected with pixel difference and which not.\n8. Multiclass using 10 classes did not provide a good baseline result. It seems that it is an extra effort to separate the features by JPEG quality factor. It seems that using only 4 classes is enough to separate better the features.\n\n**What I implemented and did not worked for me:**\n1. My idea was that attention might focus on regions of the image where possible hidden data might be in it. For instance, the data might be hidden in high frequency regions but it seems that is not always like this. I tried implementing it, it seemed good at the beginning but it ended up overfitting. Selecting the layers to extract the attention was the worst part. The paper did not mention anything so I tried to tune it manually but of course did not work. This is the [paper](https://arxiv.org/abs/1804.02391). \n\nI’ll leave here my [GitHub](https://github.com/guillemdelgado/ALASKA2-Image-Steganalysis) in case anyone wants to give it a look!\n\nCheers!", "votes": null }, { "id": "937842", "postDate": "07/21/2020 07:31:37", "content": "Thanks for sharing your learning @guillemdelgado
", "rawMarkdown": "Thanks for sharing your learning @guillemdelgado", "votes": null } ], "comments": [ { "id": 937842, "author_name": "vishnurapps", "author_url": "", "post_date": "07/21/2020 07:31:37", "content": "Thanks for sharing your learning @guillemdelgado
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "937335": "Hi everyone,\n\nFirst of all I want to congratulate everyone for the hard work and the effort put in this challenge. So, It is a fact that when we do research there are tons of approaches that we try and do not work and there are only a few that actually improves the State of the Art. That’s why I have always thought that it is important to share our achievements but also our failures. That’s why I want to share with you my failed ideas and what I have learnt during these months. So, let’s discuss my approaches and what I learnt during this journey. \n\nI highly recommend checking this [post](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/155392) as it gives the first notions to tackle this challenge.\n\n**What I have learnt:**\n1. Do not modify the frequencies of the image. That means: do not resize, do not crop, do not perform any augmentation that might affect the frequencies of the image. Rotations and flips are ok. Why? Basically some steganography uses high frequencies to hide their message.\n2. Steganalysis seems to perform better in RGB space in pre-trained NN. YCbCr and DCT do not perform as well with the same pretrained network. Maybe training the same network with ImageNet but with YCbCr/DCT spaces would improve?\n3. Big batch size is important in order to allow batch normalization work correctly. If the network is too big to fit in the GPU/TPU we need to find other approaches.\n4. TTA always helped me, giving more importance to the original image than the other transformation. Isometry improved my score a bit.\n5. Using KFold worked for me. The most important part is separating the data not randomly but including the same image (Cover + 3 algorithms) improves a lot. Not all folds were useful though.\n6. Balancing the batch with BalanceClassSampler was positive\n7. MSE with a binary approach did not really work. It seems that pixel difference is not enough. I would have liked to check which algorithms were more prone to be detected with pixel difference and which not.\n8. Multiclass using 10 classes did not provide a good baseline result. It seems that it is an extra effort to separate the features by JPEG quality factor. It seems that using only 4 classes is enough to separate better the features.\n\n**What I implemented and did not worked for me:**\n1. My idea was that attention might focus on regions of the image where possible hidden data might be in it. For instance, the data might be hidden in high frequency regions but it seems that is not always like this. I tried implementing it, it seemed good at the beginning but it ended up overfitting. Selecting the layers to extract the attention was the worst part. The paper did not mention anything so I tried to tune it manually but of course did not work. This is the [paper](https://arxiv.org/abs/1804.02391). \n\nI’ll leave here my [GitHub](https://github.com/guillemdelgado/ALASKA2-Image-Steganalysis) in case anyone wants to give it a look!\n\nCheers!", "937842": "Thanks for sharing your learning @guillemdelgado" }, "source": "meta" }