![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3543139%2F6435e40486536115e06b8b5263bdc6c0%2F676e909e1c0d1ca891bc9afbe3e91d9c.png?generation=1584403463405091&alt=media\")
\nIM LITERALLY DEAD\nIM LITERALLY DEAD
\nIM LITERALLY DEAD
You are not in a bad shape, we dropped about 250 because of a wrong selection:\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1212661%2Fef8d8ae085af1d25b7b0ba787e8dc831%2F1.png?generation=1584405720919875&alt=media\")
![\"\"](\"https://wompampsupport.azureedge.net/fetchimage?siteId=7575&v=2&jpgQuality=100&width=700&url=https%3A%2F%2Fi.kym-cdn.com%2Fentries%2Ficons%2Ffacebook%2F000%2F027%2F763%2F07B89120-B48D-45FB-AF1D-49AF6CD16790.jpg\")
It's okay man, this was a great experience! My first time to meet the shake up. My best private score is ~1100 places better than my final submissions. lol![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3617078%2Fcdf1686512daaa5d1c3fb982e64ab35c%2F2020-03-17%208.23.11.png?generation=1584404647358865&alt=media\")
oh no š§
", "rawMarkdown": "oh no š§ " }, { "id": 775744, "postDate": "2020-03-17T00:35:52.967Z", "content": "I feel sad and exciting in the same time, this just confusing.
", "rawMarkdown": "I feel sad and exciting in the same time, this just confusing.", "votes": 4 }, { "id": 775785, "postDate": "2020-03-17T00:58:52.093Z", "content": "What you said is exactly how I feel! š¤
", "rawMarkdown": "What you said is exactly how I feel! š¤ ", "votes": 1 }, { "id": 776213, "postDate": "2020-03-17T07:53:52.417Z", "content": "Can you share what that solution was @xiejialun? If possible with code also?
\n\nSome of the augmentations you implemented in Flower Classification TPU contest were really good and it helped to learn a lot of tricks to be used in this competition also.
\n\nEven though I also faced a huge drop, I learned a lot from folks like you, @ipythonx , @seesee etc
", "rawMarkdown": "Can you share what that solution was @xiejialun? If possible with code also?\n\nSome of the augmentations you implemented in Flower Classification TPU contest were really good and it helped to learn a lot of tricks to be used in this competition also.\n\nEven though I also faced a huge drop, I learned a lot from folks like you, @ipythonx , @seesee etc" }, { "id": 776253, "postDate": "2020-03-17T08:36:34.870Z", "content": "@kurianbenoy Sure, but to be honest, my solution might have very few value to reference.\nI think if you start from See--`s TPU kernel, then huge drop is unavoidable unless you
\n\nSince the original 1 head tail will limit the model to predict the results on classes only showed in training dataset, which brings your model to overfitting.
\n\nOnce you change the tail from 1 head to 3 heads, then you should can easily get 0.92+ private score with original kernel from See--.
\n\nAnd here are something I tried to and workable:
\n\nThe augmentation can only perform 1 type in each time, like this\n```\nselect_prob = self.random_prob()
\n\nif select_prob <= 0.35:\n image, g_labels, v_labels, c_labels = self.MixCut(image, g_labels, v_labels, c_labels)\nelif select_prob > 0.35 and select_prob <= 0.7:\n image, g_labels, v_labels, c_labels = self.MixUp(image, g_labels, v_labels, c_labels)\nelse:\n image = self.CutOut(image)\n```
\n\nAfter the competition, I change the 1 head tail to 3 heads tail and rest of the settings are as I listed above. Then I got 0.9348 private score and 0.9797 public score. \nActually, I had noticed that 1 head tail might cause me overfitting problem, but since the 1 head tail gave me higher LB and I was too lazy to change the architecture, so here I go..LOL
", "rawMarkdown": "@kurianbenoy Sure, but to be honest, my solution might have very few value to reference.\nI think if you start from See--`s TPU kernel, then huge drop is unavoidable unless you \n\n1. Change the prediction head from 1 head to 3 head\n2. applying weights to the class prediction like this [solution](https://www.kaggle.com/c/bengaliai-cv19/discussion/136056)\n\nSince the original 1 head tail will limit the model to predict the results on classes only showed in training dataset, which brings your model to overfitting.\n\nOnce you change the tail from 1 head to 3 heads, then you should can easily get 0.92+ private score with original kernel from See--.\n\nAnd here are something I tried to and workable:\n\n1. ReduceLROnPlateau training \n2. CutMix(prob 0.35) MixUp(prob 0.35) and Cutout(prob 0.3) augmentation\n3. Change backbone to EfficientNetB4-B5\n4. Image resolution to 224x224\n5. Batch size 1024 : 128 for each core, Kaggle TPU has 8 cores\n6. Training epochs : 70, Adam optimizer with learning rate 1e-3\n7. Regular categorical cross-entropy loss\n\nThe augmentation can only perform 1 type in each time, like this\n```\nselect_prob = self.random_prob()\n \nif select_prob <= 0.35:\n image, g_labels, v_labels, c_labels = self.MixCut(image, g_labels, v_labels, c_labels)\nelif select_prob > 0.35 and select_prob <= 0.7:\n image, g_labels, v_labels, c_labels = self.MixUp(image, g_labels, v_labels, c_labels)\nelse:\n image = self.CutOut(image)\n```\n\nAfter the competition, I change the 1 head tail to 3 heads tail and rest of the settings are as I listed above. Then I got 0.9348 private score and 0.9797 public score. \nActually, I had noticed that 1 head tail might cause me overfitting problem, but since the 1 head tail gave me higher LB and I was too lazy to change the architecture, so here I go..LOL", "votes": 1 }, { "id": 776258, "postDate": "2020-03-17T08:44:00.717Z", "rawMarkdown": "", "isDeleted": true }, { "id": 776261, "postDate": "2020-03-17T08:47:15.320Z", "content": "I already set my training notebook public to you, so you should be able to see it.\nBut the code is quite messy though.
", "rawMarkdown": "I already set my training notebook public to you, so you should be able to see it.\nBut the code is quite messy though." }, { "id": 776787, "postDate": "2020-03-17T16:28:07.253Z", "content": "Thanks for the detailed solution @xiejialun. It great to see how well you used the power of TPUs with changing batch size to 1024. Learned a lot from your solution and thanks for sharing the notebook
", "rawMarkdown": "Thanks for the detailed solution @xiejialun. It great to see how well you used the power of TPUs with changing batch size to 1024. Learned a lot from your solution and thanks for sharing the notebook", "votes": 1 } ] }, { "id": 776222, "postDate": "2020-03-17T08:02:33.800Z", "content": "My reaction after checking private LB: š \n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F97455%2F0f89b1f545344b34009a1bd11ae3a173%2FScreenshot%202020-03-17%20at%201.58.22%20PM.png?generation=1584431953141542&alt=media\")
But few minutes later I felt: š \n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F97455%2F6bc0ed3e7deb0cd68ed362f203158bd5%2FScreenshot%202020-03-17%20at%201.52.59%20PM.png?generation=1584431789009610&alt=media\")
That's 17th position on private leaderboard!
", "rawMarkdown": "My reaction after checking private LB: š \n\n\nBut few minutes later I felt: š \n\n\nThat's 17th position on private leaderboard!", "votes": 3 }, { "id": 775694, "postDate": "2020-03-17T00:08:33.737Z", "content": "My best sub ? The first ! I would have been 29th. Now I am 266th.
", "rawMarkdown": "My best sub ? The first ! I would have been 29th. Now I am 266th.", "votes": 3 }, { "id": 776200, "postDate": "2020-03-17T07:37:30.417Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2722541%2F7e340f89107aca3c8abae150fada1d0e%2Fsad.PNG?generation=1584430500387546&alt=media\")
Dude, @greatgamedota u atleast got bronze; we slid down 235 places due to wrong submission. We would have got 87th position if the above sub was submitted and would have secured my first silver (first medal as well). I am crying as hell š š
", "rawMarkdown": "\n\n\nDude, @greatgamedota u atleast got bronze; we slid down 235 places due to wrong submission. We would have got 87th position if the above sub was submitted and would have secured my first silver (first medal as well). I am crying as hell š š ", "votes": 1 }, { "id": 775730, "postDate": "2020-03-17T00:32:21.300Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1984321%2F9b7e88f776e4d221d0e8c7723ef975d9%2FF.png?generation=1584405053029372&alt=media\")
damn! -_-
", "rawMarkdown": "\n\ndamn! -_- ", "votes": 1 }, { "id": 776292, "postDate": "2020-03-17T09:15:31.100Z", "content": "I think, it requires to trust in own intuition while selecting two final submissions.
", "rawMarkdown": "I think, it requires to trust in own intuition while selecting two final submissions." }, { "id": 776173, "postDate": "2020-03-17T07:06:55.623Z", "content": "IM SCREAMING
\n\nI could have got silver š¤
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1718514%2F15679425f3ee4184875539cb23e1258a%2FScreenshot%20from%202020-03-17%2011-31-52.png?generation=1584428789694572&alt=media\")
Same :(\nI dropped ~350 places:
\n\n.png?generation=1584422466690495&alt=media)
Main too. Is there any way to get recover from that? I am new and really thought it would pick highest score.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1466783%2F58bb761c189e08684a1f9216a69112e2%2FScreenshot%202020-03-17%20at%2011.03.58%20AM.png?generation=1584421595693590&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1494907%2F3fccd90394e87eaabe30f5372fa6f2af%2FSnipaste_2020-03-17_10-17-00.png?generation=1584411448203163&alt=media\")
Could have got a silver... š
", "rawMarkdown": "\n\nCould have got a silver... š " }, { "id": 775812, "postDate": "2020-03-17T01:18:42.037Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2261136%2F9ecb69baa3d217e14d35c720bb90b2d7%2FCapture_1.PNG?generation=1584407864227382&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2261136%2F0595e4d3d34138a740277690ff12b895%2FCapture_2.PNG?generation=1584407863045974&alt=media\")
\nš
Oh man same thing for me, my best model is 0.9389 but i selected a 0.90.... first time experiencing a shake up like this\ni dropped 1200 positions.. im sad :(
", "rawMarkdown": "Oh man same thing for me, my best model is 0.9389 but i selected a 0.90.... first time experiencing a shake up like this\ni dropped 1200 positions.. im sad :(" }, { "id": 775779, "postDate": "2020-03-17T00:54:15.797Z", "content": "Who knows this guy will win.. Could get a medal ..\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F4008529%2F09abda0eb9c688e5289aba9353d03ecc%2FScreen%20Shot%202020-03-16%20at%205.51.26%20PM.png?generation=1584406435550458&alt=media\")
You are not in a bad shape, we dropped about 250 because of a wrong selection:\n
It's okay man, this was a great experience! My first time to meet the shake up. My best private score is ~1100 places better than my final submissions. lol
oh no š§
", "votes": 0, "replies": [] }, { "id": 775744, "author_name": "Tsai29", "author_url": "", "post_date": "2020-03-17T00:35:52.967000", "content": "I feel sad and exciting in the same time, this just confusing.
", "votes": 4, "replies": [] }, { "id": 775785, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T00:58:52.093000", "content": "What you said is exactly how I feel! š¤
", "votes": 1, "replies": [] }, { "id": 776213, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-03-17T07:53:52.417000", "content": "Can you share what that solution was @xiejialun? If possible with code also?
\n\nSome of the augmentations you implemented in Flower Classification TPU contest were really good and it helped to learn a lot of tricks to be used in this competition also.
\n\nEven though I also faced a huge drop, I learned a lot from folks like you, @ipythonx , @seesee etc
", "votes": 0, "replies": [] }, { "id": 776253, "author_name": "Tsai29", "author_url": "", "post_date": "2020-03-17T08:36:34.870000", "content": "@kurianbenoy Sure, but to be honest, my solution might have very few value to reference.\nI think if you start from See--`s TPU kernel, then huge drop is unavoidable unless you
\n\nSince the original 1 head tail will limit the model to predict the results on classes only showed in training dataset, which brings your model to overfitting.
\n\nOnce you change the tail from 1 head to 3 heads, then you should can easily get 0.92+ private score with original kernel from See--.
\n\nAnd here are something I tried to and workable:
\n\nThe augmentation can only perform 1 type in each time, like this\n```\nselect_prob = self.random_prob()
\n\nif select_prob <= 0.35:\n image, g_labels, v_labels, c_labels = self.MixCut(image, g_labels, v_labels, c_labels)\nelif select_prob > 0.35 and select_prob <= 0.7:\n image, g_labels, v_labels, c_labels = self.MixUp(image, g_labels, v_labels, c_labels)\nelse:\n image = self.CutOut(image)\n```
\n\nAfter the competition, I change the 1 head tail to 3 heads tail and rest of the settings are as I listed above. Then I got 0.9348 private score and 0.9797 public score. \nActually, I had noticed that 1 head tail might cause me overfitting problem, but since the 1 head tail gave me higher LB and I was too lazy to change the architecture, so here I go..LOL
", "votes": 1, "replies": [] }, { "id": 776258, "author_name": "", "author_url": "", "post_date": "2020-03-17T08:44:00.717000", "content": "", "votes": 0, "replies": [] }, { "id": 776261, "author_name": "Tsai29", "author_url": "", "post_date": "2020-03-17T08:47:15.320000", "content": "I already set my training notebook public to you, so you should be able to see it.\nBut the code is quite messy though.
", "votes": 0, "replies": [] }, { "id": 776787, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-03-17T16:28:07.253000", "content": "Thanks for the detailed solution @xiejialun. It great to see how well you used the power of TPUs with changing batch size to 1024. Learned a lot from your solution and thanks for sharing the notebook
", "votes": 1, "replies": [] } ] }, { "id": 776222, "author_name": "Atikur Rahman", "author_url": "", "post_date": "2020-03-17T08:02:33.800000", "content": "My reaction after checking private LB: š \n
But few minutes later I felt: š \n
That's 17th position on private leaderboard!
", "votes": 3, "replies": [] }, { "id": 775694, "author_name": "Ppo", "author_url": "", "post_date": "2020-03-17T00:08:33.737000", "content": "My best sub ? The first ! I would have been 29th. Now I am 266th.
", "votes": 3, "replies": [] }, { "id": 776200, "author_name": "Viraj Bagal", "author_url": "", "post_date": "2020-03-17T07:37:30.417000", "content": "
Dude, @greatgamedota u atleast got bronze; we slid down 235 places due to wrong submission. We would have got 87th position if the above sub was submitted and would have secured my first silver (first medal as well). I am crying as hell š š
", "votes": 1, "replies": [] }, { "id": 775730, "author_name": "Innat", "author_url": "", "post_date": "2020-03-17T00:32:21.300000", "content": "
damn! -_-
", "votes": 1, "replies": [] }, { "id": 776292, "author_name": "Shayekh Islam", "author_url": "", "post_date": "2020-03-17T09:15:31.100000", "content": "I think, it requires to trust in own intuition while selecting two final submissions.
", "votes": 0, "replies": [] }, { "id": 776173, "author_name": "Tanyapohn", "author_url": "", "post_date": "2020-03-17T07:06:55.623000", "content": "IM SCREAMING
\n\nI could have got silver š¤
\n\n
Same :(\nI dropped ~350 places:
\n\n.png?generation=1584422466690495&alt=media)
Main too. Is there any way to get recover from that? I am new and really thought it would pick highest score.
\n\n
Could have got a silver... š
", "votes": 0, "replies": [] }, { "id": 775812, "author_name": "Utsav Nandi", "author_url": "", "post_date": "2020-03-17T01:18:42.037000", "content": "\n\nš
Oh man same thing for me, my best model is 0.9389 but i selected a 0.90.... first time experiencing a shake up like this\ni dropped 1200 positions.. im sad :(
", "votes": 0, "replies": [] }, { "id": 775779, "author_name": "Helen", "author_url": "", "post_date": "2020-03-17T00:54:15.797000", "content": "Who knows this guy will win.. Could get a medal ..\n