![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2006644%2F86c070756b810c74293e6ada0707b630%2Fshuffle.png?generation=1595300985030345&alt=media\")
Definitely it's overfitting to the public LB. But a solo gold is good enough to me. Maybe the best to me, because no need to submit my mess source code :) \nCongrats to everyone in this competition. We have learned a lot from it.
\n\nHere is a brief of my solution.
\n\n■ Using Tensorflow and TPU
\n\n■ Preparation of the data\n Save DCT(512x512x16) as png into .tfrec file. (lossless)\n Save quality factor into .tfrec file.\n Save draft payload into .tfrec file. (If there any change in 8x8 area, count as 1)
\n\nTransfer DCT into YCbCr in TF way at training time. By using this method, I can train one epoch in 15~20 mins for Effnet B0/B1, and 30~45 mins for B6/B7.\n The reason of using YCbCr is here.
\n\n■ Augmentation \n All of the augmentations are implemented in the TF way. (maybe there is something wrong here)
\n\nFlip LR/UD\n Flip +/- (IMO it's available by using YCbCr)\n Multiply random number (0.98~1.02). (We are detecting the change of wavelets)\n Random shuffle the 24x24 / 32x32 / 40x40 blocks. IMO this can make model focus on wavelets of block but the contents of image. Keeping the boarder(16 pixels) because UERD always change this area.\n
No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)
\n\n■ Models\n Changing one pixel by +/-1 in DCT will cause a change of 8x8 block in YCbCr, There are a lot of patterns to learn, so I tried B6/B7/B8 and the final result is the ensemble of them. The best single model is B6(Local/public/private: 0.940/0.940/0.929)
\n\nMulticlass(4) + Aux loss (payload, mae)\n Quality factor as input. Not target.\n Using small learning rate at final stage.\n TTA: LR/UD/+-, 8 per image. (see Augmentation)
\n\n■ Training\n I just focused on one fold and refined it on all data. Regrettably, it is overfitting to the public LB. By observing the public/private LB, I think many of others are like me except the winner. Congratulations again.
\n\nIf I trained more folds maybe I could ...... :) (It will take a long time and forget it now)
\n\n■Update \n7/22 Add a link of best single model.
", "messages": [ { "id": "937458", "postDate": "07/21/2020 03:24:01", "content": "Definitely it's overfitting to the public LB. But a solo gold is good enough to me. Maybe the best to me, because no need to submit my mess source code :) \nCongrats to everyone in this competition. We have learned a lot from it.
\n\nHere is a brief of my solution.
\n\n■ Using Tensorflow and TPU
\n\n■ Preparation of the data\n Save DCT(512x512x16) as png into .tfrec file. (lossless)\n Save quality factor into .tfrec file.\n Save draft payload into .tfrec file. (If there any change in 8x8 area, count as 1)
\n\nTransfer DCT into YCbCr in TF way at training time. By using this method, I can train one epoch in 15~20 mins for Effnet B0/B1, and 30~45 mins for B6/B7.\n The reason of using YCbCr is here.
\n\n■ Augmentation \n All of the augmentations are implemented in the TF way. (maybe there is something wrong here)
\n\nFlip LR/UD\n Flip +/- (IMO it's available by using YCbCr)\n Multiply random number (0.98~1.02). (We are detecting the change of wavelets)\n Random shuffle the 24x24 / 32x32 / 40x40 blocks. IMO this can make model focus on wavelets of block but the contents of image. Keeping the boarder(16 pixels) because UERD always change this area.\n
No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)
\n\n■ Models\n Changing one pixel by +/-1 in DCT will cause a change of 8x8 block in YCbCr, There are a lot of patterns to learn, so I tried B6/B7/B8 and the final result is the ensemble of them. The best single model is B6(Local/public/private: 0.940/0.940/0.929)
\n\nMulticlass(4) + Aux loss (payload, mae)\n Quality factor as input. Not target.\n Using small learning rate at final stage.\n TTA: LR/UD/+-, 8 per image. (see Augmentation)
\n\n■ Training\n I just focused on one fold and refined it on all data. Regrettably, it is overfitting to the public LB. By observing the public/private LB, I think many of others are like me except the winner. Congratulations again.
\n\nIf I trained more folds maybe I could ...... :) (It will take a long time and forget it now)
\n\n■Update \n7/22 Add a link of best single model.
", "rawMarkdown": "Definitely it's overfitting to the public LB. But a solo gold is good enough to me. Maybe the best to me, because no need to submit my mess source code :) \nCongrats to everyone in this competition. We have learned a lot from it.\n\nHere is a brief of my solution.\n\n■ Using Tensorflow and TPU\n\n■ Preparation of the data\n Save DCT(512x512x16) as png into .tfrec file. (lossless)\n Save quality factor into .tfrec file.\n Save draft payload into .tfrec file. (If there any change in 8x8 area, count as 1)\n\n Transfer DCT into YCbCr in TF way at training time. By using this method, I can train one epoch in 15~20 mins for Effnet B0/B1, and 30~45 mins for B6/B7.\n The reason of using YCbCr is [here](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/150359#845167).\n\n\n■ Augmentation \n All of the augmentations are implemented in the TF way. (maybe there is something wrong here)\n\n Flip LR/UD\n Flip +/- (IMO it's available by using YCbCr)\n Multiply random number (0.98~1.02). (We are detecting the change of wavelets)\n Random shuffle the 24x24 / 32x32 / 40x40 blocks. IMO this can make model focus on wavelets of block but the contents of image. Keeping the boarder(16 pixels) because UERD always change this area.\n\n\n No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)\n\n\n■ Models\n Changing one pixel by +/-1 in DCT will cause a change of 8x8 block in YCbCr, There are a lot of patterns to learn, so I tried B6/B7/B8 and the final result is the ensemble of them. The [best single model](https://www.kaggle.com/wuliaokaola/alaska2-best-b6-inference) is B6(Local/public/private: 0.940/0.940/0.929)\n\n Multiclass(4) + Aux loss (payload, mae)\n Quality factor as input. Not target.\n Using small learning rate at final stage.\n TTA: LR/UD/+-, 8 per image. (see Augmentation)\n\n■ Training\n I just focused on one fold and refined it on all data. Regrettably, it is overfitting to the public LB. By observing the public/private LB, I think many of others are like me except the winner. Congratulations again.\n \n If I trained more folds maybe I could ...... :) (It will take a long time and forget it now) \n\n■Update \n7/22 Add a link of [best single model](https://www.kaggle.com/wuliaokaola/alaska2-best-b6-inference).", "votes": null }, { "id": "937471", "postDate": "07/21/2020 03:38:12", "content": "Thank you very much for sharing! I was wondering if you could publish your code. I also used TF/Keras on TPU, but I found the score much lower than same models with PyTorch on GPU. Also, I saw that you mentioned using efficientnet-b8. I tried to implement that with TF/Keras, but didn't work out for me. Thanks again for the great write up.
", "rawMarkdown": "Thank you very much for sharing! I was wondering if you could publish your code. I also used TF/Keras on TPU, but I found the score much lower than same models with PyTorch on GPU. Also, I saw that you mentioned using efficientnet-b8. I tried to implement that with TF/Keras, but didn't work out for me. Thanks again for the great write up.", "votes": null }, { "id": "937508", "postDate": "07/21/2020 04:08:19", "content": "\n\n\"No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)\"
\n\nyou can rotate the quantization table too. you can refer to
\n\nhttps://linux.die.net/man/1/jpegtran\nhttps://github.com/fhanau/Efficient-Compression-Tool/blob/8cc69fac9f26e3878298580222b890cc88d5e080/src/mozjpeg/transupp.c
", "rawMarkdown": "wuliaokaola \n\n\"No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)\"\n\nyou can rotate the quantization table too. you can refer to\n\nhttps://linux.die.net/man/1/jpegtran\nhttps://github.com/fhanau/Efficient-Compression-Tool/blob/8cc69fac9f26e3878298580222b890cc88d5e080/src/mozjpeg/transupp.c", "votes": null }, { "id": "937510", "postDate": "07/21/2020 04:09:34", "content": "Sorry, I call it B8 but It's not real B8. I just change B7 more deeper and use larger dropout.
\n\nefn.EfficientNet(\n 2.0, 3.8, 600, 0.55, 0.25,\n input_tensor=x,\n weights=None,\n include_top=False\n )
", "rawMarkdown": "Sorry, I call it B8 but It's not real B8. I just change B7 more deeper and use larger dropout. \n\nefn.EfficientNet(\n 2.0, 3.8, 600, 0.55, 0.25,\n input_tensor=x,\n weights=None,\n include_top=False\n )", "votes": null }, { "id": "937523", "postDate": "07/21/2020 04:19:12", "content": "I'm not sure it is a single rotation.
", "rawMarkdown": "I'm not sure it is a single rotation.", "votes": null }, { "id": "937550", "postDate": "07/21/2020 04:36:31", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2006644%2Fb51702978100832b974499462e179c8d%2Fq.png?generation=1595306042130772&alt=media\")
\nI know what you mean, but maybe the real stego is something different between vertical and horizon.
Congrats @wuliaokaola
\nSorry for my naive question. But what are the wavelets ? We are detecting the change of wavelets
Thanks for sharing!\nI always wondered how you overfit. 👍
", "rawMarkdown": "Thanks for sharing!\nI always wondered how you overfit. 👍", "votes": null }, { "id": "937652", "postDate": "07/21/2020 05:43:51", "content": "Congrats on gold medal and thanks for sharing your solution @wuliaokaola
", "rawMarkdown": "Congrats on gold medal and thanks for sharing your solution @wuliaokaola", "votes": null }, { "id": "937695", "postDate": "07/21/2020 06:06:20", "content": "Thank you!
", "rawMarkdown": "Thank you!", "votes": null }, { "id": "937716", "postDate": "07/21/2020 06:16:37", "content": "I'm not familiar to it, too. But refer to this paper, UNIWARD make it difficult to be detected. And in this competition UNIWARD is the most difficult one.
", "rawMarkdown": "I'm not familiar to it, too. But refer to this [paper](https://www.researchgate.net/publication/259639875_Universal_Distortion_Function_for_Steganography_in_an_Arbitrary_Domain), UNIWARD make it difficult to be detected. And in this competition UNIWARD is the [most difficult one](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/155814).", "votes": null }, { "id": "937804", "postDate": "07/21/2020 07:05:43", "content": "k.π/2 rotations are fine as long as you rotate in the \"spatial\" domain, i.e. YCbCr or RGB. \nIf you want to rotate in the DCT domain, jpegtran has some implementation of \"lossless\" rotations, or you can refer to [http://www.ws.binghamton.edu/fridrich/Research/OneHot_Revised.pdf] Section VI-A Data augmentation in DCT domain.
", "rawMarkdown": "k.π/2 rotations are fine as long as you rotate in the \"spatial\" domain, i.e. YCbCr or RGB. \nIf you want to rotate in the DCT domain, jpegtran has some implementation of \"lossless\" rotations, or you can refer to [http://www.ws.binghamton.edu/fridrich/Research/OneHot_Revised.pdf] Section VI-A Data augmentation in DCT domain.", "votes": null }, { "id": "937840", "postDate": "07/21/2020 07:30:31", "content": "@yousfi Thank you very much. But I still think the real stego is something different between vertical and horizon. In my experience, rot90 helps under 0.93(public LB), but makes the result worse above 0.93.
", "rawMarkdown": "yousfi Thank you very much. But I still think the real stego is something different between vertical and horizon. In my experience, rot90 helps under 0.93(public LB), but makes the result worse above 0.93.", "votes": null }, { "id": "937847", "postDate": "07/21/2020 07:38:00", "content": "I see what you mean, if the stego scheme has anisotropic cost assignments, then the stego image can be slightly different if rotated, but for the sake of augmentation, we can go away with that... For our models, D4 TTA always helped, as long as the net is trained with D4 augs.
", "rawMarkdown": "I see what you mean, if the stego scheme has anisotropic cost assignments, then the stego image can be slightly different if rotated, but for the sake of augmentation, we can go away with that... For our models, D4 TTA always helped, as long as the net is trained with D4 augs.", "votes": null }, { "id": "937851", "postDate": "07/21/2020 07:44:06", "content": "Thank you for your explanation. Maybe you are right.
", "rawMarkdown": "Thank you for your explanation. Maybe you are right.", "votes": null }, { "id": "938161", "postDate": "07/21/2020 11:29:57", "content": "Congrats! The method keeps the border and shuffles grid tiles was great. For UERD, the border information is important. I will try this type of shuffling to our models. Thanks for sharing.
", "rawMarkdown": "Congrats! The method keeps the border and shuffles grid tiles was great. For UERD, the border information is important. I will try this type of shuffling to our models. Thanks for sharing.", "votes": null }, { "id": "938300", "postDate": "07/21/2020 12:48:18", "content": "@wuliaokaola
\nAlmost from the beginning to the end, you were the king on public LB. However, I'm curious to know 20 min per epoch, that's a lot of leverage to experiment. I think you were using kaggle TPU, it would be kind if you make a vanilla notebook of your work; not need to burn the TPU but simple and precise. Anyway, congratulation =)
How did you incorporate the quality factor as input to efficient net?
", "rawMarkdown": "How did you incorporate the quality factor as input to efficient net?", "votes": null }, { "id": "938995", "postDate": "07/22/2020 00:04:58", "content": "Thank you.
", "rawMarkdown": "Thank you.", "votes": null }, { "id": "938999", "postDate": "07/22/2020 00:16:51", "content": "Thank you.\n If you use .tfrec file, 20 min per epoch is easy to achieve. For TPU the bottleneck is I/O.
", "rawMarkdown": "Thank you.\n If you use .tfrec file, 20 min per epoch is easy to achieve. For TPU the bottleneck is I/O.", "votes": null }, { "id": "939001", "postDate": "07/22/2020 00:20:48", "content": "Treat quality factor as 4th channel. The input shape is 512x512x4.
", "rawMarkdown": "Treat quality factor as 4th channel. The input shape is 512x512x4.", "votes": null }, { "id": "939012", "postDate": "07/22/2020 00:57:52", "content": "Ah, I see thank you. Are you still able to use pretrained networks when you change the input channel dimension like this?
", "rawMarkdown": "Ah, I see thank you. Are you still able to use pretrained networks when you change the input channel dimension like this?", "votes": null }, { "id": "939727", "postDate": "07/22/2020 12:13:44", "content": "@ipythonx Here is an example of inference. If you want to use TPU, you should change GCS_DS_PATH_Test to \"KaggleDatasets()...\".
", "rawMarkdown": "ipythonx Here is an example of [inference](https://www.kaggle.com/wuliaokaola/alaska2-best-b6-inference). If you want to use TPU, you should change GCS_DS_PATH_Test to \"KaggleDatasets()...\".", "votes": null }, { "id": "939742", "postDate": "07/22/2020 12:33:12", "content": "@wuliaokaola
\nAwesome, thank you. =)
@mr3543 Yes. By using load_weights and set skip_mismatch to True.
", "rawMarkdown": "mr3543 Yes. By using load_weights and set skip_mismatch to True.", "votes": null } ], "comments": [ { "id": 937637, "author_name": "vishnurapps", "author_url": "", "post_date": "07/21/2020 05:39:30", "content": "Congrats @wuliaokaola
\nSorry for my naive question. But what are the wavelets ? We are detecting the change of wavelets
I'm not familiar to it, too. But refer to this paper, UNIWARD make it difficult to be detected. And in this competition UNIWARD is the most difficult one.
", "votes": null, "replies": [] } ] }, { "id": 937652, "author_name": "duykhanh99", "author_url": "", "post_date": "07/21/2020 05:43:51", "content": "Congrats on gold medal and thanks for sharing your solution @wuliaokaola
", "votes": null, "replies": [] }, { "id": 938300, "author_name": "ipythonx", "author_url": "", "post_date": "07/21/2020 12:48:18", "content": "@wuliaokaola
\nAlmost from the beginning to the end, you were the king on public LB. However, I'm curious to know 20 min per epoch, that's a lot of leverage to experiment. I think you were using kaggle TPU, it would be kind if you make a vanilla notebook of your work; not need to burn the TPU but simple and precise. Anyway, congratulation =)
Thank you.\n If you use .tfrec file, 20 min per epoch is easy to achieve. For TPU the bottleneck is I/O.
", "votes": null, "replies": [] }, { "id": 939727, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/22/2020 12:13:44", "content": "@ipythonx Here is an example of inference. If you want to use TPU, you should change GCS_DS_PATH_Test to \"KaggleDatasets()...\".
", "votes": null, "replies": [] }, { "id": 939742, "author_name": "ipythonx", "author_url": "", "post_date": "07/22/2020 12:33:12", "content": "@wuliaokaola
\nAwesome, thank you. =)
How did you incorporate the quality factor as input to efficient net?
", "votes": null, "replies": [ { "id": 939001, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/22/2020 00:20:48", "content": "Treat quality factor as 4th channel. The input shape is 512x512x4.
", "votes": null, "replies": [] }, { "id": 939012, "author_name": "mr3543", "author_url": "", "post_date": "07/22/2020 00:57:52", "content": "Ah, I see thank you. Are you still able to use pretrained networks when you change the input channel dimension like this?
", "votes": null, "replies": [] }, { "id": 941998, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/23/2020 14:35:56", "content": "@mr3543 Yes. By using load_weights and set skip_mismatch to True.
", "votes": null, "replies": [] } ] }, { "id": 937471, "author_name": "tonychenxyz", "author_url": "", "post_date": "07/21/2020 03:38:12", "content": "Thank you very much for sharing! I was wondering if you could publish your code. I also used TF/Keras on TPU, but I found the score much lower than same models with PyTorch on GPU. Also, I saw that you mentioned using efficientnet-b8. I tried to implement that with TF/Keras, but didn't work out for me. Thanks again for the great write up.
", "votes": null, "replies": [ { "id": 937510, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/21/2020 04:09:34", "content": "Sorry, I call it B8 but It's not real B8. I just change B7 more deeper and use larger dropout.
\n\nefn.EfficientNet(\n 2.0, 3.8, 600, 0.55, 0.25,\n input_tensor=x,\n weights=None,\n include_top=False\n )
", "votes": null, "replies": [] }, { "id": 937695, "author_name": "tonychenxyz", "author_url": "", "post_date": "07/21/2020 06:06:20", "content": "Thank you!
", "votes": null, "replies": [] } ] }, { "id": 937508, "author_name": "hengck23", "author_url": "", "post_date": "07/21/2020 04:08:19", "content": "\n\n\"No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)\"
\n\nyou can rotate the quantization table too. you can refer to
\n\nhttps://linux.die.net/man/1/jpegtran\nhttps://github.com/fhanau/Efficient-Compression-Tool/blob/8cc69fac9f26e3878298580222b890cc88d5e080/src/mozjpeg/transupp.c
", "votes": null, "replies": [ { "id": 937523, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/21/2020 04:19:12", "content": "I'm not sure it is a single rotation.
", "votes": null, "replies": [] }, { "id": 937550, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/21/2020 04:36:31", "content": "\nI know what you mean, but maybe the real stego is something different between vertical and horizon.
Thanks for sharing!\nI always wondered how you overfit. 👍
", "votes": null, "replies": [] }, { "id": 937804, "author_name": "yousfi", "author_url": "", "post_date": "07/21/2020 07:05:43", "content": "k.π/2 rotations are fine as long as you rotate in the \"spatial\" domain, i.e. YCbCr or RGB. \nIf you want to rotate in the DCT domain, jpegtran has some implementation of \"lossless\" rotations, or you can refer to [http://www.ws.binghamton.edu/fridrich/Research/OneHot_Revised.pdf] Section VI-A Data augmentation in DCT domain.
", "votes": null, "replies": [ { "id": 937840, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/21/2020 07:30:31", "content": "@yousfi Thank you very much. But I still think the real stego is something different between vertical and horizon. In my experience, rot90 helps under 0.93(public LB), but makes the result worse above 0.93.
", "votes": null, "replies": [] }, { "id": 937847, "author_name": "yousfi", "author_url": "", "post_date": "07/21/2020 07:38:00", "content": "I see what you mean, if the stego scheme has anisotropic cost assignments, then the stego image can be slightly different if rotated, but for the sake of augmentation, we can go away with that... For our models, D4 TTA always helped, as long as the net is trained with D4 augs.
", "votes": null, "replies": [] }, { "id": 937851, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/21/2020 07:44:06", "content": "Thank you for your explanation. Maybe you are right.
", "votes": null, "replies": [] } ] }, { "id": 938161, "author_name": "songwonho", "author_url": "", "post_date": "07/21/2020 11:29:57", "content": "Congrats! The method keeps the border and shuffles grid tiles was great. For UERD, the border information is important. I will try this type of shuffling to our models. Thanks for sharing.
", "votes": null, "replies": [ { "id": 938995, "author_name": "wuliaokaola", "author_url": "", "post_date": "07/22/2020 00:04:58", "content": "Thank you.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "937458": "Definitely it's overfitting to the public LB. But a solo gold is good enough to me. Maybe the best to me, because no need to submit my mess source code :) \nCongrats to everyone in this competition. We have learned a lot from it.\n\nHere is a brief of my solution.\n\n■ Using Tensorflow and TPU\n\n■ Preparation of the data\n Save DCT(512x512x16) as png into .tfrec file. (lossless)\n Save quality factor into .tfrec file.\n Save draft payload into .tfrec file. (If there any change in 8x8 area, count as 1)\n\n Transfer DCT into YCbCr in TF way at training time. By using this method, I can train one epoch in 15~20 mins for Effnet B0/B1, and 30~45 mins for B6/B7.\n The reason of using YCbCr is [here](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/150359#845167).\n\n\n■ Augmentation \n All of the augmentations are implemented in the TF way. (maybe there is something wrong here)\n\n Flip LR/UD\n Flip +/- (IMO it's available by using YCbCr)\n Multiply random number (0.98~1.02). (We are detecting the change of wavelets)\n Random shuffle the 24x24 / 32x32 / 40x40 blocks. IMO this can make model focus on wavelets of block but the contents of image. Keeping the boarder(16 pixels) because UERD always change this area.\n\n\n No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)\n\n\n■ Models\n Changing one pixel by +/-1 in DCT will cause a change of 8x8 block in YCbCr, There are a lot of patterns to learn, so I tried B6/B7/B8 and the final result is the ensemble of them. The [best single model](https://www.kaggle.com/wuliaokaola/alaska2-best-b6-inference) is B6(Local/public/private: 0.940/0.940/0.929)\n\n Multiclass(4) + Aux loss (payload, mae)\n Quality factor as input. Not target.\n Using small learning rate at final stage.\n TTA: LR/UD/+-, 8 per image. (see Augmentation)\n\n■ Training\n I just focused on one fold and refined it on all data. Regrettably, it is overfitting to the public LB. By observing the public/private LB, I think many of others are like me except the winner. Congratulations again.\n \n If I trained more folds maybe I could ...... :) (It will take a long time and forget it now) \n\n■Update \n7/22 Add a link of [best single model](https://www.kaggle.com/wuliaokaola/alaska2-best-b6-inference).", "937471": "Thank you very much for sharing! I was wondering if you could publish your code. I also used TF/Keras on TPU, but I found the score much lower than same models with PyTorch on GPU. Also, I saw that you mentioned using efficientnet-b8. I tried to implement that with TF/Keras, but didn't work out for me. Thanks again for the great write up.", "937508": "wuliaokaola \n\n\"No Rot90, because the quantization table is not diagonal symmetry. (maybe I was wrong)\"\n\nyou can rotate the quantization table too. you can refer to\n\nhttps://linux.die.net/man/1/jpegtran\nhttps://github.com/fhanau/Efficient-Compression-Tool/blob/8cc69fac9f26e3878298580222b890cc88d5e080/src/mozjpeg/transupp.c", "937510": "Sorry, I call it B8 but It's not real B8. I just change B7 more deeper and use larger dropout. \n\nefn.EfficientNet(\n 2.0, 3.8, 600, 0.55, 0.25,\n input_tensor=x,\n weights=None,\n include_top=False\n )", "937523": "I'm not sure it is a single rotation.", "937550": "\nI know what you mean, but maybe the real stego is something different between vertical and horizon.", "937637": "Congrats @wuliaokaola \n\nSorry for my naive question. But what are the wavelets ? `We are detecting the change of wavelets`", "937650": "Thanks for sharing!\nI always wondered how you overfit. 👍", "937652": "Congrats on gold medal and thanks for sharing your solution @wuliaokaola", "937695": "Thank you!", "937716": "I'm not familiar to it, too. But refer to this [paper](https://www.researchgate.net/publication/259639875_Universal_Distortion_Function_for_Steganography_in_an_Arbitrary_Domain), UNIWARD make it difficult to be detected. And in this competition UNIWARD is the [most difficult one](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/155814).", "937804": "k.π/2 rotations are fine as long as you rotate in the \"spatial\" domain, i.e. YCbCr or RGB. \nIf you want to rotate in the DCT domain, jpegtran has some implementation of \"lossless\" rotations, or you can refer to [http://www.ws.binghamton.edu/fridrich/Research/OneHot_Revised.pdf] Section VI-A Data augmentation in DCT domain.", "937840": "yousfi Thank you very much. But I still think the real stego is something different between vertical and horizon. In my experience, rot90 helps under 0.93(public LB), but makes the result worse above 0.93.", "937847": "I see what you mean, if the stego scheme has anisotropic cost assignments, then the stego image can be slightly different if rotated, but for the sake of augmentation, we can go away with that... For our models, D4 TTA always helped, as long as the net is trained with D4 augs.", "937851": "Thank you for your explanation. Maybe you are right.", "938161": "Congrats! The method keeps the border and shuffles grid tiles was great. For UERD, the border information is important. I will try this type of shuffling to our models. Thanks for sharing.", "938300": "wuliaokaola\nAlmost from the beginning to the end, you were the king on public LB. However, I'm curious to know 20 min per epoch, that's a lot of leverage to experiment. I think you were using kaggle TPU, it would be kind if you make a vanilla notebook of your work; not need to burn the TPU but simple and precise. Anyway, congratulation =)", "938975": "How did you incorporate the quality factor as input to efficient net?", "938995": "Thank you.", "938999": "Thank you.\n If you use .tfrec file, 20 min per epoch is easy to achieve. For TPU the bottleneck is I/O.", "939001": "Treat quality factor as 4th channel. The input shape is 512x512x4.", "939012": "Ah, I see thank you. Are you still able to use pretrained networks when you change the input channel dimension like this?", "939727": "ipythonx Here is an example of [inference](https://www.kaggle.com/wuliaokaola/alaska2-best-b6-inference). If you want to use TPU, you should change GCS_DS_PATH_Test to \"KaggleDatasets()...\".", "939742": "wuliaokaola \nAwesome, thank you. =)", "941998": "mr3543 Yes. By using load_weights and set skip_mismatch to True." }, "source": "meta" }