![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1490082%2Fbf2a7752d2089e41e8ba37f1ffd0f5c6%2FBengali.png?generation=1584438905462146&alt=media\")
Go down 447! 😭 What just happened!!!
", "messages": [ { "id": 775691, "postDate": "2020-03-17T00:06:07.240Z", "content": "More shakeup than expected, but as anticipated more unseen graphemes on private. So memorizing graphemes works worse.
", "rawMarkdown": "More shakeup than expected, but as anticipated more unseen graphemes on private. So memorizing graphemes works worse.", "votes": 12, "replies": [ { "id": 775713, "postDate": "2020-03-17T00:19:27.373Z", "content": "I think so too.. I see so many teams have 1000+ boost, I guess I need to do experiments more carefully next time. \nCongrats to you guys!💯
", "rawMarkdown": "I think so too.. I see so many teams have 1000+ boost, I guess I need to do experiments more carefully next time. \nCongrats to you guys!💯 ", "votes": 1 } ] }, { "id": 775689, "postDate": "2020-03-17T00:05:12.123Z", "content": "Go down 447! 😭 What just happened!!!
", "rawMarkdown": "Go down 447! 😭 What just happened!!!", "votes": 9 }, { "id": 776324, "postDate": "2020-03-17T10:01:13.860Z", "content": "
First 3 submissions trained with 30% cutout, 30% Gridamsk with mode 0 and 30% Gridmask with mode 2\nLast 3 submissions trained with 40% cutmix, 20% mixup and 20% cutout rest everything was same.
", "rawMarkdown": "\n\nFirst 3 submissions trained with 30% cutout, 30% Gridamsk with mode 0 and 30% Gridmask with mode 2\nLast 3 submissions trained with 40% cutmix, 20% mixup and 20% cutout rest everything was same.", "votes": 1, "replies": [ { "id": 776691, "postDate": "2020-03-17T15:05:51.400Z", "content": "Sorry for your wrong selection. Guessing at this point, we all know mixup and cutmix are truly important! My best private LB using them as well, but I wasn't able to get good accuracy for both training set and the validation set. So I chose wrongly as well. I need to retrain it ASAP. Good luck next time!
", "rawMarkdown": "Sorry for your wrong selection. Guessing at this point, we all know mixup and cutmix are truly important! My best private LB using them as well, but I wasn't able to get good accuracy for both training set and the validation set. So I chose wrongly as well. I need to retrain it ASAP. Good luck next time!", "votes": 1 } ] }, { "id": 775703, "postDate": "2020-03-17T00:13:56.117Z", "content": "Truly horrible. Clearly densenet works way better than seresnext.
", "rawMarkdown": "Truly horrible. Clearly densenet works way better than seresnext.", "votes": 1, "replies": [ { "id": 775767, "postDate": "2020-03-17T00:46:54.260Z", "content": "I never try densenet. I couldn't wait to see the top solutions.
", "rawMarkdown": "I never try densenet. I couldn't wait to see the top solutions." } ] }, { "id": 775701, "postDate": "2020-03-17T00:10:26.890Z", "content": "Horrible shake-up!
", "rawMarkdown": "Horrible shake-up! ", "votes": 1, "replies": [ { "id": 775769, "postDate": "2020-03-17T00:47:42.793Z", "content": "Sure. I see some familiar faces get shake out as well. 🤝
", "rawMarkdown": "Sure. I see some familiar faces get shake out as well. 🤝 " } ] }, { "id": 775807, "postDate": "2020-03-17T01:14:14.120Z", "content": "Amazing contest, I wrote a bug on mixup and cutmix, but it works better. Kaggle is fantastic！
", "rawMarkdown": "Amazing contest, I wrote a bug on mixup and cutmix, but it works better. Kaggle is fantastic！", "votes": 2, "replies": [ { "id": 775822, "postDate": "2020-03-17T01:26:06.807Z", "content": "Wow, good for you! Congrats!
", "rawMarkdown": "Wow, good for you! Congrats!" } ] }, { "id": 775770, "postDate": "2020-03-17T00:49:26.797Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1212661%2F691c5c980dfc8a788f4867361ea4daeb%2F1.png?generation=1584406110066207&alt=media\")
\nIt happens( Even my public kernel (0.9340) performs better than the selected sub(
oh no. Who would have thought so?
", "rawMarkdown": "oh no. Who would have thought so? " }, { "id": 775801, "postDate": "2020-03-17T01:11:05.173Z", "content": "Essentially the difference is using 3 head approach, like in my public kernel, vs just 3 linear layers after pooling... like in most of other public kernels. The first one generalizes muuuch better for unseen graphems than the second one, while is working a little bit worse on seen graphems. \nSo, the result is expected since the private LB is just composed by a large number of unseen examples, in contrast to public one, having just a few(
", "rawMarkdown": "Essentially the difference is using 3 head approach, like in my public kernel, vs just 3 linear layers after pooling... like in most of other public kernels. The first one generalizes muuuch better for unseen graphems than the second one, while is working a little bit worse on seen graphems. \nSo, the result is expected since the private LB is just composed by a large number of unseen examples, in contrast to public one, having just a few(", "votes": 5 }, { "id": 775840, "postDate": "2020-03-17T01:35:48.100Z", "content": "oh, I see. BTW, I first get the multi heads idea from your kernel, really learned a lot from you!👍 \nMy problem is that I used four heads, including the graphems. I guess that's the main reason why I went down so much.
", "rawMarkdown": "oh, I see. BTW, I first get the multi heads idea from your kernel, really learned a lot from you!👍 \nMy problem is that I used four heads, including the graphems. I guess that's the main reason why I went down so much.", "votes": 1 }, { "id": 775863, "postDate": "2020-03-17T02:00:51.223Z", "content": "Actually for our best private LB sub we also used 4 heads... with weight 0.1 on the last one. The thing is that those heads should be quite complicated to isolate features needed to produce a particular output. Though 3 heads would work quite better on private LB.\n0.9438 - 0.9880 (224x224 resized original images with optimized setup and 4 heads)\n0.9418 - 0.9714 (128x128 crops from my public kernel with a simple setup trained 2 month ago)
", "rawMarkdown": "Actually for our best private LB sub we also used 4 heads... with weight 0.1 on the last one. The thing is that those heads should be quite complicated to isolate features needed to produce a particular output. Though 3 heads would work quite better on private LB.\n**0.9438 - 0.9880** (224x224 resized original images with optimized setup and 4 heads)\n**0.9418 - 0.9714** (128x128 crops from my public kernel with a simple setup trained 2 month ago)", "votes": 1 }, { "id": 775881, "postDate": "2020-03-17T02:15:48.197Z", "content": "oh, I see. Thanks for your detailed reply. Your 4heads is doing so much better than the 3heads on public LB! I couldn't make such high score.. In this case, I think my model is basically not good enough. So much need to be learned! \nWould you mind talking a little bit about the optimized setup you mentioned?
", "rawMarkdown": "oh, I see. Thanks for your detailed reply. Your 4heads is doing so much better than the 3heads on public LB! I couldn't make such high score.. In this case, I think my model is basically not good enough. So much need to be learned! \nWould you mind talking a little bit about the optimized setup you mentioned? " }, { "id": 776110, "postDate": "2020-03-17T06:02:42.687Z", "content": "I think the next competition, your public kernel will gain more forks😂
", "rawMarkdown": "I think the next competition, your public kernel will gain more forks😂 " }, { "id": 777537, "postDate": "2020-03-17T18:47:38.480Z", "content": "@iafoss, ditto and thanks for all that you have shared. My public kernel based on your kernel is my 2nd best score on the private LB and I did not select either of those. So yeah it happens. In a real real world ML I may not have chosen my 2 final submissions because they are both ensembles.
", "rawMarkdown": "@iafoss, ditto and thanks for all that you have shared. My public kernel based on your kernel is my 2nd best score on the private LB and I did not select either of those. So yeah it happens. In a real real world ML I may not have chosen my 2 final submissions because they are both ensembles.", "votes": 1 }, { "id": 777580, "postDate": "2020-03-17T19:37:31.907Z", "content": "@iafoss, sorry to see your team been shaken up. Your kernels were great starters for many of us!
", "rawMarkdown": "@iafoss, sorry to see your team been shaken up. Your kernels were great starters for many of us!" }, { "id": 777665, "postDate": "2020-03-17T20:58:54.953Z", "content": "Thank you
", "rawMarkdown": "Thank you" }, { "id": 779066, "postDate": "2020-03-19T02:13:51.460Z", "content": "And someone used @iafoss public kernel and got a silver medal
", "rawMarkdown": "And someone used @iafoss public kernel and got a silver medal" } ] }, { "id": 775705, "postDate": "2020-03-17T00:15:11.873Z", "content": "Horrible shakeup went down by 1176 places
", "rawMarkdown": "Horrible shakeup went down by 1176 places", "votes": 2, "replies": [ { "id": 775726, "postDate": "2020-03-17T00:29:53.597Z", "content": "Sorry to hear that. I try to balance the final two submission but never thought that would happen.
", "rawMarkdown": "Sorry to hear that. I try to balance the final two submission but never thought that would happen.", "votes": 1 } ] }, { "id": 775828, "postDate": "2020-03-17T01:29:31.673Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 775691, "author_name": "Psi", "author_url": "", "post_date": "2020-03-17T00:06:07.240000", "content": "More shakeup than expected, but as anticipated more unseen graphemes on private. So memorizing graphemes works worse.
", "votes": 12, "replies": [ { "id": 775713, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T00:19:27.373000", "content": "I think so too.. I see so many teams have 1000+ boost, I guess I need to do experiments more carefully next time. \nCongrats to you guys!💯
", "votes": 1, "replies": [] } ] }, { "id": 776324, "author_name": "Raghawendra Singh", "author_url": "", "post_date": "2020-03-17T10:01:13.860000", "content": "
First 3 submissions trained with 30% cutout, 30% Gridamsk with mode 0 and 30% Gridmask with mode 2\nLast 3 submissions trained with 40% cutmix, 20% mixup and 20% cutout rest everything was same.
", "votes": 1, "replies": [ { "id": 776691, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T15:05:51.400000", "content": "Sorry for your wrong selection. Guessing at this point, we all know mixup and cutmix are truly important! My best private LB using them as well, but I wasn't able to get good accuracy for both training set and the validation set. So I chose wrongly as well. I need to retrain it ASAP. Good luck next time!
", "votes": 1, "replies": [] } ] }, { "id": 775703, "author_name": "Gold Retriever", "author_url": "", "post_date": "2020-03-17T00:13:56.117000", "content": "Truly horrible. Clearly densenet works way better than seresnext.
", "votes": 1, "replies": [ { "id": 775767, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T00:46:54.260000", "content": "I never try densenet. I couldn't wait to see the top solutions.
", "votes": 0, "replies": [] } ] }, { "id": 775701, "author_name": "Innat", "author_url": "", "post_date": "2020-03-17T00:10:26.890000", "content": "Horrible shake-up!
", "votes": 1, "replies": [ { "id": 775769, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T00:47:42.793000", "content": "Sure. I see some familiar faces get shake out as well. 🤝
", "votes": 0, "replies": [] } ] }, { "id": 775807, "author_name": "HOHOYAO", "author_url": "", "post_date": "2020-03-17T01:14:14.120000", "content": "Amazing contest, I wrote a bug on mixup and cutmix, but it works better. Kaggle is fantastic！
", "votes": 2, "replies": [ { "id": 775822, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T01:26:06.807000", "content": "Wow, good for you! Congrats!
", "votes": 0, "replies": [] } ] }, { "id": 775770, "author_name": "Iafoss", "author_url": "", "post_date": "2020-03-17T00:49:26.797000", "content": "\nIt happens( Even my public kernel (0.9340) performs better than the selected sub(
oh no. Who would have thought so?
", "votes": 0, "replies": [] }, { "id": 775801, "author_name": "Iafoss", "author_url": "", "post_date": "2020-03-17T01:11:05.173000", "content": "Essentially the difference is using 3 head approach, like in my public kernel, vs just 3 linear layers after pooling... like in most of other public kernels. The first one generalizes muuuch better for unseen graphems than the second one, while is working a little bit worse on seen graphems. \nSo, the result is expected since the private LB is just composed by a large number of unseen examples, in contrast to public one, having just a few(
", "votes": 5, "replies": [] }, { "id": 775840, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T01:35:48.100000", "content": "oh, I see. BTW, I first get the multi heads idea from your kernel, really learned a lot from you!👍 \nMy problem is that I used four heads, including the graphems. I guess that's the main reason why I went down so much.
", "votes": 1, "replies": [] }, { "id": 775863, "author_name": "Iafoss", "author_url": "", "post_date": "2020-03-17T02:00:51.223000", "content": "Actually for our best private LB sub we also used 4 heads... with weight 0.1 on the last one. The thing is that those heads should be quite complicated to isolate features needed to produce a particular output. Though 3 heads would work quite better on private LB.\n0.9438 - 0.9880 (224x224 resized original images with optimized setup and 4 heads)\n0.9418 - 0.9714 (128x128 crops from my public kernel with a simple setup trained 2 month ago)
", "votes": 1, "replies": [] }, { "id": 775881, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T02:15:48.197000", "content": "oh, I see. Thanks for your detailed reply. Your 4heads is doing so much better than the 3heads on public LB! I couldn't make such high score.. In this case, I think my model is basically not good enough. So much need to be learned! \nWould you mind talking a little bit about the optimized setup you mentioned?
", "votes": 0, "replies": [] }, { "id": 776110, "author_name": "Shiyuan Zeng", "author_url": "", "post_date": "2020-03-17T06:02:42.687000", "content": "I think the next competition, your public kernel will gain more forks😂
", "votes": 0, "replies": [] }, { "id": 777537, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-03-17T18:47:38.480000", "content": "@iafoss, ditto and thanks for all that you have shared. My public kernel based on your kernel is my 2nd best score on the private LB and I did not select either of those. So yeah it happens. In a real real world ML I may not have chosen my 2 final submissions because they are both ensembles.
", "votes": 1, "replies": [] }, { "id": 777580, "author_name": "Jo Tom", "author_url": "", "post_date": "2020-03-17T19:37:31.907000", "content": "@iafoss, sorry to see your team been shaken up. Your kernels were great starters for many of us!
", "votes": 0, "replies": [] }, { "id": 777665, "author_name": "Iafoss", "author_url": "", "post_date": "2020-03-17T20:58:54.953000", "content": "Thank you
", "votes": 0, "replies": [] }, { "id": 779066, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-03-19T02:13:51.460000", "content": "And someone used @iafoss public kernel and got a silver medal
", "votes": 0, "replies": [] } ] }, { "id": 775705, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-03-17T00:15:11.873000", "content": "Horrible shakeup went down by 1176 places
", "votes": 2, "replies": [ { "id": 775726, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T00:29:53.597000", "content": "Sorry to hear that. I try to balance the final two submission but never thought that would happen.
", "votes": 1, "replies": [] } ] }, { "id": 775828, "author_name": "", "author_url": "", "post_date": "2020-03-17T01:29:31.673000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "775691": "More shakeup than expected, but as anticipated more unseen graphemes on private. So memorizing graphemes works worse.", "775689": "Go down 447! 😭 What just happened!!!", "776324": "\n\nFirst 3 submissions trained with 30% cutout, 30% Gridamsk with mode 0 and 30% Gridmask with mode 2\nLast 3 submissions trained with 40% cutmix, 20% mixup and 20% cutout rest everything was same.", "775703": "Truly horrible. Clearly densenet works way better than seresnext.", "775701": "Horrible shake-up! ", "775807": "Amazing contest, I wrote a bug on mixup and cutmix, but it works better. Kaggle is fantastic！", "775770": "\nIt happens( Even my public kernel (0.9340) performs better than the selected sub(", "775705": "Horrible shakeup went down by 1176 places", "775828": "" } }