{ "id": 169448, "title": "0.927 Private LB single model (1-fold) B0-inspired architecture (late sub)", "url": "/competitions/alaska2-image-steganalysis/discussion/169448", "author_name": "Andrés Miguel Torrubia Sáez", "post_date": "2020-07-23T22:28:43.191000", "votes": 15, "comment_count": 6, "views": 0, "content": "

Still testing a potential improvement, but want to report a B0 model that scores:

\n\n

\"\"

\n\n

How:

\n\n\n\n

Trained on 3 GPUs, FP16, Flips/rotations, and (for performance) DCT->YCbCr->RGB done on GPU, 1-cycle policy; just a cross-entropy loss on 4-classes (adding payload, i.e. % of nzAC coeff changed or segmentation-like auxiliary tasks did not really help (to our surprise).

", "messages": [ { "id": 942637, "postDate": "2020-07-23T22:28:43.190Z", "content": "

Still testing a potential improvement, but want to report a B0 model that scores:

\n\n

\"\"

\n\n

How:

\n\n\n\n

Trained on 3 GPUs, FP16, Flips/rotations, and (for performance) DCT->YCbCr->RGB done on GPU, 1-cycle policy; just a cross-entropy loss on 4-classes (adding payload, i.e. % of nzAC coeff changed or segmentation-like auxiliary tasks did not really help (to our surprise).

", "rawMarkdown": "Still testing a potential improvement, but want to report a B0 model that scores:\n\n![](https://i.imgur.com/twasXs3.png)\n\nHow:\n\n- B0 w/ 1st stride moved. Testing now removing latest blocks (will update if it improves)\n- 8 input channels (instead of 3):\n* RGB (decoded YCbCr in fp32 and then to RGB fp32 also w/o truncation)\n* Q as a channel (quality)\n* Chessboard pattern with borders (why? because some stego images there's 8x8 blocks that change are in border; this way conv net can see where it's looking at, à la coordconv) \n* **3 JUNIWARD cost maps**: JUNIWARD uses a per-DCT coeff cost map of how much does it cost to add +1 or -1 to a block. I modified JUNIWARD C++ code to output the cost maps as 3x64x64 arrays and computed all cost maps for all train and test images (took a while). This significantly improved performance.\n\nTrained on 3 GPUs, FP16, Flips/rotations, and (for performance) DCT->YCbCr->RGB done on GPU, 1-cycle policy; just a cross-entropy loss on 4-classes (adding payload, i.e. % of nzAC coeff changed or segmentation-like auxiliary tasks did not really help (to our surprise).\n", "votes": 15 }, { "id": 949721, "postDate": "2020-07-28T21:02:15.680Z", "content": "

@antorsae @hengck23 hat off to you guys for keep digging! 👍

", "rawMarkdown": "@antorsae @hengck23 hat off to you guys for keep digging! 👍 ", "votes": 1 }, { "id": 944790, "postDate": "2020-07-25T10:51:58.917Z", "content": "

i did a similar thing and report my results/code at at https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/168542.

\n\n

modified efficientnet-B0 gives private/public LB 0.925/0.936 for me.

", "rawMarkdown": "i did a similar thing and report my results/code at at https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/168542.\n\nmodified efficientnet-B0 gives private/public LB 0.925/0.936 for me.", "votes": 1, "replies": [ { "id": 945387, "postDate": "2020-07-25T19:15:52.263Z", "content": "

I think JUNIWARD cost maps are key here (0.927 private was with 48 epochs, and like in your case it showed no signs of saturation).

", "rawMarkdown": "I think JUNIWARD cost maps are key here (0.927 private was with 48 epochs, and like in your case it showed no signs of saturation)." }, { "id": 945398, "postDate": "2020-07-25T19:25:38.303Z", "content": "

i wonder if the following will work?\n- input = 9 channels, of which \n - 3 channels = original ycbcr (or rgb)\n - 3 channels = ycbcr for increase all DCT by +1\n - 3 channels = ycbcr for decrase all DCT by -1,

", "rawMarkdown": "i wonder if the following will work?\n- input = 9 channels, of which \n - 3 channels = original ycbcr (or rgb)\n - 3 channels = ycbcr for increase all DCT by +1\n - 3 channels = ycbcr for decrase all DCT by -1, " }, { "id": 945909, "postDate": "2020-07-26T07:57:07.433Z", "content": "

This is interesting. One thing I noticed was that (JUNIWARD) cost maps are computed on +1 -1 differences on each DCT coeff, however stegoimages had some coeffs modified by more than +-1... at any rate I'm wrapping efforts on this competition already...

", "rawMarkdown": "This is interesting. One thing I noticed was that (JUNIWARD) cost maps are computed on +1 -1 differences on each DCT coeff, however stegoimages had some coeffs modified by more than +-1... at any rate I'm wrapping efforts on this competition already..." } ] }, { "id": 942653, "postDate": "2020-07-23T22:55:21.247Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 949721, "author_name": "Yifan Xie", "author_url": "", "post_date": "2020-07-28T21:02:15.680000", "content": "

@antorsae @hengck23 hat off to you guys for keep digging! 👍

", "votes": 1, "replies": [] }, { "id": 944790, "author_name": "hengck23", "author_url": "", "post_date": "2020-07-25T10:51:58.917000", "content": "

i did a similar thing and report my results/code at at https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/168542.

\n\n

modified efficientnet-B0 gives private/public LB 0.925/0.936 for me.

", "votes": 1, "replies": [ { "id": 945387, "author_name": "Andrés Miguel Torrubia Sáez", "author_url": "", "post_date": "2020-07-25T19:15:52.263000", "content": "

I think JUNIWARD cost maps are key here (0.927 private was with 48 epochs, and like in your case it showed no signs of saturation).

", "votes": 0, "replies": [] }, { "id": 945398, "author_name": "hengck23", "author_url": "", "post_date": "2020-07-25T19:25:38.303000", "content": "

i wonder if the following will work?\n- input = 9 channels, of which \n - 3 channels = original ycbcr (or rgb)\n - 3 channels = ycbcr for increase all DCT by +1\n - 3 channels = ycbcr for decrase all DCT by -1,

", "votes": 0, "replies": [] }, { "id": 945909, "author_name": "Andrés Miguel Torrubia Sáez", "author_url": "", "post_date": "2020-07-26T07:57:07.433000", "content": "

This is interesting. One thing I noticed was that (JUNIWARD) cost maps are computed on +1 -1 differences on each DCT coeff, however stegoimages had some coeffs modified by more than +-1... at any rate I'm wrapping efforts on this competition already...

", "votes": 0, "replies": [] } ] }, { "id": 942653, "author_name": "", "author_url": "", "post_date": "2020-07-23T22:55:21.247000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "942637": "Still testing a potential improvement, but want to report a B0 model that scores:\n\n![](https://i.imgur.com/twasXs3.png)\n\nHow:\n\n- B0 w/ 1st stride moved. Testing now removing latest blocks (will update if it improves)\n- 8 input channels (instead of 3):\n* RGB (decoded YCbCr in fp32 and then to RGB fp32 also w/o truncation)\n* Q as a channel (quality)\n* Chessboard pattern with borders (why? because some stego images there's 8x8 blocks that change are in border; this way conv net can see where it's looking at, à la coordconv) \n* **3 JUNIWARD cost maps**: JUNIWARD uses a per-DCT coeff cost map of how much does it cost to add +1 or -1 to a block. I modified JUNIWARD C++ code to output the cost maps as 3x64x64 arrays and computed all cost maps for all train and test images (took a while). This significantly improved performance.\n\nTrained on 3 GPUs, FP16, Flips/rotations, and (for performance) DCT->YCbCr->RGB done on GPU, 1-cycle policy; just a cross-entropy loss on 4-classes (adding payload, i.e. % of nzAC coeff changed or segmentation-like auxiliary tasks did not really help (to our surprise).\n", "949721": "@antorsae @hengck23 hat off to you guys for keep digging! 👍 ", "944790": "i did a similar thing and report my results/code at at https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/168542.\n\nmodified efficientnet-B0 gives private/public LB 0.925/0.936 for me.", "942653": "" } }