![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1723677%2F83f7e91ddee2858286f0e5bc537ff2f4%2FScreen%20Shot%202020-03-20%20at%204.26.04%20PM.png?generation=1584746782612826&alt=media\")
Congrats all the winners and thanks for sharing solutions!\nI was shock down -1356th places, which means my model was good for seen classes but totally failed to predict unseen combinations.\nBy reference to winners solution I fixed my models, finally it achieved public 0.9925 / private 0.9552.\nAnd this is trained only on TPU and using keras, so it may be useful for someone who want to use TPU and keras user.\nhttps://www.kaggle.com/bamps53/private0-9552-tpu-keras-metric-learning
\n\n| No | model | cv | public | private | rank |\n|----|--------------------------------------|-------|--------|---------|--------|\n| 1 | decode 1292 unique labels (original) | 0.999 | 0.9870 | 0.9035 | 1416th |\n| 2 | 3 heads for each r, v, c | | 0.9891 | 0.9228 | 639th |\n| 3 | 3 models for each r, v, c | | 0.9665 | 0.9383 | 56th |\n| 4 | seen: No.1 unseen: No.3 | | 0.9925 | 0.9552 | 8th |\n| 5 | No.4 with postprocess | | 0.9941 | 0.9704 | 2nd |
", "messages": [ { "id": "779521", "postDate": "03/19/2020 12:48:32", "content": "Congrats all the winners and thanks for sharing solutions!\nI was shock down -1356th places, which means my model was good for seen classes but totally failed to predict unseen combinations.\nBy reference to winners solution I fixed my models, finally it achieved public 0.9925 / private 0.9552.\nAnd this is trained only on TPU and using keras, so it may be useful for someone who want to use TPU and keras user.\nhttps://www.kaggle.com/bamps53/private0-9552-tpu-keras-metric-learning
\n\n| No | model | cv | public | private | rank |\n|----|--------------------------------------|-------|--------|---------|--------|\n| 1 | decode 1292 unique labels (original) | 0.999 | 0.9870 | 0.9035 | 1416th |\n| 2 | 3 heads for each r, v, c | | 0.9891 | 0.9228 | 639th |\n| 3 | 3 models for each r, v, c | | 0.9665 | 0.9383 | 56th |\n| 4 | seen: No.1 unseen: No.3 | | 0.9925 | 0.9552 | 8th |\n| 5 | No.4 with postprocess | | 0.9941 | 0.9704 | 2nd |
", "rawMarkdown": "Congrats all the winners and thanks for sharing solutions!\nI was shock down -1356th places, which means my model was good for seen classes but totally failed to predict unseen combinations.\nBy reference to winners solution I fixed my models, finally it achieved public 0.9925 / private 0.9552.\nAnd this is trained only on TPU and using keras, so it may be useful for someone who want to use TPU and keras user.\nhttps://www.kaggle.com/bamps53/private0-9552-tpu-keras-metric-learning\n\n| No | model | cv | public | private | rank |\n|----|--------------------------------------|-------|--------|---------|--------|\n| 1 | decode 1292 unique labels (original) | 0.999 | 0.9870 | 0.9035 | 1416th |\n| 2 | 3 heads for each r, v, c | | 0.9891 | 0.9228 | 639th |\n| 3 | 3 models for each r, v, c | | 0.9665 | 0.9383 | 56th |\n| 4 | seen: No.1 unseen: No.3 | | 0.9925 | 0.9552 | 8th |\n| 5 | No.4 with postprocess | | 0.9941 | 0.9704 | 2nd |", "votes": null }, { "id": "779547", "postDate": "03/19/2020 13:16:31", "content": "Thank you for sharing! \nThis post is very helpful for us to understand how unseen graphemes shook LB💥
", "rawMarkdown": "Thank you for sharing! \nThis post is very helpful for us to understand how unseen graphemes shook LB💥", "votes": null }, { "id": "779573", "postDate": "03/19/2020 13:40:40", "content": "Thanks, I was too naive to suspect public/private is not random...\nI'm trying to reproduce your result too! I assume sSE is the key👀
", "rawMarkdown": "Thanks, I was too naive to suspect public/private is not random...\nI'm trying to reproduce your result too! I assume sSE is the key👀", "votes": null }, { "id": "779583", "postDate": "03/19/2020 13:53:41", "content": "Thank you! \nThanks to uratatsu( @d1348k ), It is becoming clear that not only RandomErasing but also sSE-Pooling plays an important role. (see this discussion: https://www.kaggle.com/c/bengaliai-cv19/discussion/137029#779153)
\n\nI'm looking forward to your reproduction result 😃
", "rawMarkdown": "Thank you! \nThanks to uratatsu( @d1348k ), It is becoming clear that not only RandomErasing but also sSE-Pooling plays an important role. (see this discussion: https://www.kaggle.com/c/bengaliai-cv19/discussion/137029#779153)\n\nI'm looking forward to your reproduction result 😃", "votes": null }, { "id": "779602", "postDate": "03/19/2020 14:12:12", "content": "@ttahara For me, cutmix didn't work only for consonant model. I assume if the key object is too small, cutmix doesn't make sense. But it's a different subject from how to deal with unseen classes.
", "rawMarkdown": "ttahara For me, cutmix didn't work only for consonant model. I assume if the key object is too small, cutmix doesn't make sense. But it's a different subject from how to deal with unseen classes.", "votes": null }, { "id": "779604", "postDate": "03/19/2020 14:15:23", "content": "lovely, thanks for sharing 💙
", "rawMarkdown": "lovely, thanks for sharing 💙", "votes": null }, { "id": "779608", "postDate": "03/19/2020 14:18:55", "content": "@ipythonx Thanks, hope it's useful:)
", "rawMarkdown": "ipythonx Thanks, hope it's useful:)", "votes": null }, { "id": "779634", "postDate": "03/19/2020 14:50:35", "content": "I see. This post explains a risk of using CutMix roughly. I think CAM CutMix is one of effective ways.
", "rawMarkdown": "I see. [This post](https://www.kaggle.com/c/bengaliai-cv19/discussion/126557) explains a risk of using CutMix roughly. I think [CAM CutMix](https://www.kaggle.com/c/bengaliai-cv19/discussion/136021) is one of effective ways.", "votes": null }, { "id": "780047", "postDate": "03/19/2020 23:43:12", "content": "Sad story, i can relate my friend.. I dropped 1247 positions, however my best 3 head model that i didn't select is 0.9389 with a resnet34, so i try to console myself with that haha.. It sucks that the public score doesnt reflect how the model will perform in the end, i wouldve spent way more effort to get a better score with a 3 head model if only i knew it would perform better..
", "rawMarkdown": "Sad story, i can relate my friend.. I dropped 1247 positions, however my best 3 head model that i didn't select is 0.9389 with a resnet34, so i try to console myself with that haha.. It sucks that the public score doesnt reflect how the model will perform in the end, i wouldve spent way more effort to get a better score with a 3 head model if only i knew it would perform better..", "votes": null }, { "id": "780577", "postDate": "03/20/2020 12:16:47", "content": "@yannmajewski > It sucks that the public score doesnt reflect how the model will perform in the end\nI agree, I could have done much better if public/private split was random... Anyway, in this competition I learned many things, so in total it was good experience:)
", "rawMarkdown": "yannmajewski > It sucks that the public score doesnt reflect how the model will perform in the end\nI agree, I could have done much better if public/private split was random... Anyway, in this competition I learned many things, so in total it was good experience:)", "votes": null }, { "id": "781071", "postDate": "03/20/2020 22:11:30", "content": "Awesome model thanks for sharing. With post process your solution would be 2nd place:\n
You need to lower the EXP on consonant diacritic. Our final ensemble needed to also. Use
\n\n r_preds = postprocess(r_preds, 168, -1.2)\n v_preds = postprocess(v_preds, 11, -1.2)\n c_preds = postprocess(c_preds, 7, -0.5)\nwow, that's great.
", "rawMarkdown": "wow, that's great.", "votes": null }, { "id": "781085", "postDate": "03/20/2020 22:41:46", "content": "Great, it's a magic!
", "rawMarkdown": "Great, it's a magic!", "votes": null }, { "id": "781164", "postDate": "03/21/2020 01:39:23", "content": "@cdeotte Wow, Really amazing!! I though your post process saturate if model itself predict well, but actually not. I learned how important to face with metric seriously!
", "rawMarkdown": "cdeotte Wow, Really amazing!! I though your post process saturate if model itself predict well, but actually not. I learned how important to face with metric seriously!", "votes": null } ], "comments": [ { "id": 779547, "author_name": "ttahara", "author_url": "", "post_date": "03/19/2020 13:16:31", "content": "Thank you for sharing! \nThis post is very helpful for us to understand how unseen graphemes shook LB💥
", "votes": null, "replies": [ { "id": 779573, "author_name": "bamps53", "author_url": "", "post_date": "03/19/2020 13:40:40", "content": "Thanks, I was too naive to suspect public/private is not random...\nI'm trying to reproduce your result too! I assume sSE is the key👀
", "votes": null, "replies": [] }, { "id": 779583, "author_name": "ttahara", "author_url": "", "post_date": "03/19/2020 13:53:41", "content": "Thank you! \nThanks to uratatsu( @d1348k ), It is becoming clear that not only RandomErasing but also sSE-Pooling plays an important role. (see this discussion: https://www.kaggle.com/c/bengaliai-cv19/discussion/137029#779153)
\n\nI'm looking forward to your reproduction result 😃
", "votes": null, "replies": [] }, { "id": 779602, "author_name": "bamps53", "author_url": "", "post_date": "03/19/2020 14:12:12", "content": "@ttahara For me, cutmix didn't work only for consonant model. I assume if the key object is too small, cutmix doesn't make sense. But it's a different subject from how to deal with unseen classes.
", "votes": null, "replies": [] }, { "id": 779634, "author_name": "ttahara", "author_url": "", "post_date": "03/19/2020 14:50:35", "content": "I see. This post explains a risk of using CutMix roughly. I think CAM CutMix is one of effective ways.
", "votes": null, "replies": [] } ] }, { "id": 779604, "author_name": "ipythonx", "author_url": "", "post_date": "03/19/2020 14:15:23", "content": "lovely, thanks for sharing 💙
", "votes": null, "replies": [ { "id": 779608, "author_name": "bamps53", "author_url": "", "post_date": "03/19/2020 14:18:55", "content": "@ipythonx Thanks, hope it's useful:)
", "votes": null, "replies": [] } ] }, { "id": 780047, "author_name": "yannmajewski", "author_url": "", "post_date": "03/19/2020 23:43:12", "content": "Sad story, i can relate my friend.. I dropped 1247 positions, however my best 3 head model that i didn't select is 0.9389 with a resnet34, so i try to console myself with that haha.. It sucks that the public score doesnt reflect how the model will perform in the end, i wouldve spent way more effort to get a better score with a 3 head model if only i knew it would perform better..
", "votes": null, "replies": [ { "id": 780577, "author_name": "bamps53", "author_url": "", "post_date": "03/20/2020 12:16:47", "content": "@yannmajewski > It sucks that the public score doesnt reflect how the model will perform in the end\nI agree, I could have done much better if public/private split was random... Anyway, in this competition I learned many things, so in total it was good experience:)
", "votes": null, "replies": [] } ] }, { "id": 781071, "author_name": "cdeotte", "author_url": "", "post_date": "03/20/2020 22:11:30", "content": "Awesome model thanks for sharing. With post process your solution would be 2nd place:\n
You need to lower the EXP on consonant diacritic. Our final ensemble needed to also. Use
\n\n r_preds = postprocess(r_preds, 168, -1.2)\n v_preds = postprocess(v_preds, 11, -1.2)\n c_preds = postprocess(c_preds, 7, -0.5)\nwow, that's great.
", "votes": null, "replies": [] }, { "id": 781085, "author_name": "ttahara", "author_url": "", "post_date": "03/20/2020 22:41:46", "content": "Great, it's a magic!
", "votes": null, "replies": [] }, { "id": 781164, "author_name": "bamps53", "author_url": "", "post_date": "03/21/2020 01:39:23", "content": "@cdeotte Wow, Really amazing!! I though your post process saturate if model itself predict well, but actually not. I learned how important to face with metric seriously!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "779521": "Congrats all the winners and thanks for sharing solutions!\nI was shock down -1356th places, which means my model was good for seen classes but totally failed to predict unseen combinations.\nBy reference to winners solution I fixed my models, finally it achieved public 0.9925 / private 0.9552.\nAnd this is trained only on TPU and using keras, so it may be useful for someone who want to use TPU and keras user.\nhttps://www.kaggle.com/bamps53/private0-9552-tpu-keras-metric-learning\n\n| No | model | cv | public | private | rank |\n|----|--------------------------------------|-------|--------|---------|--------|\n| 1 | decode 1292 unique labels (original) | 0.999 | 0.9870 | 0.9035 | 1416th |\n| 2 | 3 heads for each r, v, c | | 0.9891 | 0.9228 | 639th |\n| 3 | 3 models for each r, v, c | | 0.9665 | 0.9383 | 56th |\n| 4 | seen: No.1 unseen: No.3 | | 0.9925 | 0.9552 | 8th |\n| 5 | No.4 with postprocess | | 0.9941 | 0.9704 | 2nd |", "779547": "Thank you for sharing! \nThis post is very helpful for us to understand how unseen graphemes shook LB💥", "779573": "Thanks, I was too naive to suspect public/private is not random...\nI'm trying to reproduce your result too! I assume sSE is the key👀", "779583": "Thank you! \nThanks to uratatsu( @d1348k ), It is becoming clear that not only RandomErasing but also sSE-Pooling plays an important role. (see this discussion: https://www.kaggle.com/c/bengaliai-cv19/discussion/137029#779153)\n\nI'm looking forward to your reproduction result 😃", "779602": "ttahara For me, cutmix didn't work only for consonant model. I assume if the key object is too small, cutmix doesn't make sense. But it's a different subject from how to deal with unseen classes.", "779604": "lovely, thanks for sharing 💙", "779608": "ipythonx Thanks, hope it's useful:)", "779634": "I see. [This post](https://www.kaggle.com/c/bengaliai-cv19/discussion/126557) explains a risk of using CutMix roughly. I think [CAM CutMix](https://www.kaggle.com/c/bengaliai-cv19/discussion/136021) is one of effective ways.", "780047": "Sad story, i can relate my friend.. I dropped 1247 positions, however my best 3 head model that i didn't select is 0.9389 with a resnet34, so i try to console myself with that haha.. It sucks that the public score doesnt reflect how the model will perform in the end, i wouldve spent way more effort to get a better score with a 3 head model if only i knew it would perform better..", "780577": "yannmajewski > It sucks that the public score doesnt reflect how the model will perform in the end\nI agree, I could have done much better if public/private split was random... Anyway, in this competition I learned many things, so in total it was good experience:)", "781071": "Awesome model thanks for sharing. With post process your solution would be 2nd place:\n\n\nYou need to lower the EXP on consonant diacritic. Our final ensemble needed to also. Use\n\n r_preds = postprocess(r_preds, 168, -1.2)\n v_preds = postprocess(v_preds, 11, -1.2)\n c_preds = postprocess(c_preds, 7, -0.5)", "781077": "wow, that's great.", "781085": "Great, it's a magic!", "781164": "cdeotte Wow, Really amazing!! I though your post process saturate if model itself predict well, but actually not. I learned how important to face with metric seriously!" }, "source": "meta" }