\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2390050%2F04cf106b9e12f0a94528a809714f1c9e%2Fgive_that_man_a_medal.jpg?generation=1610353178133602&alt=media\")
Here are some tips that can make your start in this challenge a little bit easier.
\n(I just consider public lb score.)
Larger image size helps public lb score. 384 x 384 ~ 512 x 512 are enough.
\nAugmentation with flips and rotations are good for starter. Things like cutmix helps with CV score, but there is no significant change in public lb for me.
\nUsing mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores.
\nA small model like resnext50 or efficientnet-b3 is doable for medals. However, keep the possibilities open for larger models.
\nThe CV strategy is important. 5 folds is enough. 10 folds had an advantage in cv, but had no significant impact in public lb. For each specific fold, the difference in scores for both cv and lb is very large. Because of this, changing the seed may can help. Not using all folds helps public lb.
\n(CV and public lb seem to be related to some extent.)
Adam is a good starting point. RAdam+Lookahead is also good. Other than that, testing was required, but there was no significant change in public lb score.
\nCV improved a lot, but public lb did not change much.
\n(The CV was very bad when only training the dataset that my model thought was noise.)
Label smoothing is good in public lb. Bi-Tempered Logistic Loss and Focal Cosine Loss can be a good alternative.
\nChoosing commonly used TTA, public lb score may get worse. This is heavily influenced by the CV Strategy. A little rotation or simple augmentation can help. However, it is sensitive to the number of TTAs.
\n(In fact, you can reach the current gold medal area with no TTA.)
It helps with public lb. However, single model can get public lb score for current gold medal area.
\nIf you have any other insights, it will be helpful for other starter.
", "messages": [ { "id": 1131463, "postDate": "2020-12-29T18:26:54.910Z", "content": "Here are some tips that can make your start in this challenge a little bit easier.
\n(I just consider public lb score.)
Larger image size helps public lb score. 384 x 384 ~ 512 x 512 are enough.
\nAugmentation with flips and rotations are good for starter. Things like cutmix helps with CV score, but there is no significant change in public lb for me.
\nUsing mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores.
\nA small model like resnext50 or efficientnet-b3 is doable for medals. However, keep the possibilities open for larger models.
\nThe CV strategy is important. 5 folds is enough. 10 folds had an advantage in cv, but had no significant impact in public lb. For each specific fold, the difference in scores for both cv and lb is very large. Because of this, changing the seed may can help. Not using all folds helps public lb.
\n(CV and public lb seem to be related to some extent.)
Adam is a good starting point. RAdam+Lookahead is also good. Other than that, testing was required, but there was no significant change in public lb score.
\nCV improved a lot, but public lb did not change much.
\n(The CV was very bad when only training the dataset that my model thought was noise.)
Label smoothing is good in public lb. Bi-Tempered Logistic Loss and Focal Cosine Loss can be a good alternative.
\nChoosing commonly used TTA, public lb score may get worse. This is heavily influenced by the CV Strategy. A little rotation or simple augmentation can help. However, it is sensitive to the number of TTAs.
\n(In fact, you can reach the current gold medal area with no TTA.)
It helps with public lb. However, single model can get public lb score for current gold medal area.
\nIf you have any other insights, it will be helpful for other starter.
", "rawMarkdown": "# 1. Introduction\n\nHere are some tips that can make your start in this challenge a little bit easier.\n(I just consider public lb score.)\n\n# 2. Things may help public lb score:\n\n - Image Size\n\n**Larger image size helps public lb score.** 384 x 384 ~ 512 x 512 are enough.\n\n- Augmentation\n\n**Augmentation with flips and rotations are good for starter.** Things like cutmix helps with CV score, but there is no significant change in public lb for me.\n\n- Normalization\n\nUsing mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores.\n\n- Model size\n\n**A small model** like resnext50 or efficientnet-b3 **is doable for medals**. However, keep the possibilities open for larger models.\n\n- Cross-Validation Strategy\n\n**The CV strategy is important**. 5 folds is enough. 10 folds had an advantage in cv, but had no significant impact in public lb. For each specific fold, the difference in scores for both cv and lb is very large. Because of this, changing the seed may can help. Not using all folds helps public lb.\n(CV and public lb seem to be related to some extent.)\n\n- Optimizer\n\n**Adam is a good starting point**. RAdam+Lookahead is also good. Other than that, testing was required, but there was no significant change in public lb score.\n\n- Relabeling and Finetuning\n\nCV improved a lot, but **public lb did not change much**. \n(The CV was very bad when only training the dataset that my model thought was noise.)\n\n- Criterion (loss function)\n\n**Label smoothing is good in public lb**. Bi-Tempered Logistic Loss and Focal Cosine Loss can be a good alternative.\n\n- TTA\n\n**Choosing commonly used TTA, public lb score may get worse.** This is heavily influenced by the CV Strategy. **A little rotation or simple augmentation can help.** However, it is sensitive to the number of TTAs.\n(In fact, you can reach the current gold medal area with no TTA.)\n\n- Ensemble\n\n**It helps with public lb.** However, single model can get public lb score for current gold medal area.\n\n\n# 3. End\n\nIf you have any other insights, it will be helpful for other starter.\n\n", "votes": 225 }, { "id": 2185588, "postDate": "2023-03-17T07:15:54.590Z", "content": "Your observation is beautiful
", "rawMarkdown": "Your observation is beautiful ", "votes": 4 }, { "id": 1132524, "postDate": "2020-12-30T13:18:40.427Z", "content": "Good advices @piantic . With a good decent knowledge in ml/cv you can easily reach top 10% in this competition
\nI would add to this:
Warm restarts didn't seem to help much for me. How many epochs are you using per restart?
", "rawMarkdown": "Warm restarts didn't seem to help much for me. How many epochs are you using per restart?" }, { "id": 1142014, "postDate": "2021-01-07T04:35:46.170Z", "content": "What does cleaning the dataset look in this case? Sorry, I'm a beginner, and it's not apparent to me how you go about cleaning a dataset of images.
", "rawMarkdown": "What does cleaning the dataset look in this case? Sorry, I'm a beginner, and it's not apparent to me how you go about cleaning a dataset of images.", "votes": 1 } ] }, { "id": 1131783, "postDate": "2020-12-30T00:46:44.187Z", "content": "BTW \"RAdam+Lookahead\" is called Ranger, so if you see Ranger (like in my kernel), that's the same as what the OP mentioned.
", "rawMarkdown": "BTW \"RAdam+Lookahead\" is called Ranger, so if you see Ranger (like in [my kernel](https://www.kaggle.com/tanlikesmath/cassava-classification-eda-fastai-starter)), that's the same as what the OP mentioned.", "votes": 5, "replies": [ { "id": 1133339, "postDate": "2020-12-31T06:02:58.533Z", "content": "Thanks. I was looking for this. BTW had anyone experimented with AdamW and is that helping?
", "rawMarkdown": "Thanks. I was looking for this. BTW had anyone experimented with AdamW and is that helping?", "votes": 1 } ] }, { "id": 1148577, "postDate": "2021-01-11T08:21:13.697Z", "content": "I've been thinking of how to improve my Lb score, Thanks for the tips.
\n
thank you for sharing . i am new to this and i’m learning great stuff here
", "rawMarkdown": "thank you for sharing . i am new to this and i’m learning great stuff here ", "votes": 1 }, { "id": 1185190, "postDate": "2021-02-04T02:56:41.293Z", "content": "Hi,
\nI am new to this field. How one can came up these ideas. Will they do trial and error methods or is there any strategy that by seeing datasets like these will works finne or these won't.
\nLoss function like we need to use label smoothing or we need to use other loss functions.
Similar competition experience and trial and error.
", "rawMarkdown": "Similar competition experience and trial and error. ", "votes": 1 } ] }, { "id": 1179384, "postDate": "2021-01-31T14:30:28.567Z", "content": "Thanks for sharing your gold points, so kind of you! One question, will training on merged dataset (2019 + 2020) help improve lb score compared to original competition dataset (only 2020)? Thank you again!
", "rawMarkdown": "Thanks for sharing your gold points, so kind of you! One question, will training on merged dataset (2019 + 2020) help improve lb score compared to original competition dataset (only 2020)? Thank you again!", "votes": 1, "replies": [ { "id": 1179566, "postDate": "2021-01-31T16:28:56.187Z", "content": "Other Kaglers say the merged dataset is helpful. @youjiawang
", "rawMarkdown": "Other Kaglers say the merged dataset is helpful. @youjiawang ", "votes": 1 } ] }, { "id": 1178024, "postDate": "2021-01-30T15:55:08.453Z", "content": "Nice topic to go, especially after having enough things to got a clear idea on what it is mentioned here. I am starting training some models with Label-smoothing, I did not managed to implement Bi-Tempered Logistic Loss.
\nRegarding this, I guess there is no obvious way to find out how much which label-smoothing to use except for trial and error ?
", "rawMarkdown": "Nice topic to go, especially after having enough things to got a clear idea on what it is mentioned here. I am starting training some models with Label-smoothing, I did not managed to implement Bi-Tempered Logistic Loss. \n\nRegarding this, I guess there is no obvious way to find out how much which label-smoothing to use except for trial and error ?", "votes": 1, "replies": [ { "id": 1178048, "postDate": "2021-01-30T16:14:13.207Z", "content": "I have implemented loss functions that other kagglers found useful in this competition. I hope it helps.
\n\nAnd I think it will save you trial and error by referring to the notebooks I wrote.
\n\n", "rawMarkdown": "I have implemented loss functions that other kagglers found useful in this competition. I hope it helps.\n\n- https://www.kaggle.com/piantic/train-cassava-starter-using-various-loss-funcs\n\n\nAnd I think it will save you trial and error by referring to the notebooks I wrote.\n\n- https://www.kaggle.com/c/cassava-leaf-disease-classification/notebooks?competitionId=13836&sortBy=voteCount&tab=profile\n\n- https://www.kaggle.com/piantic/how-to-finetuning-models-pytorch-xla-tpu\n\n@cdk292 ", "votes": 1 }, { "id": 1178078, "postDate": "2021-01-30T16:32:37.897Z", "content": "Haha thanks. I am using R, which I guess explain more why I did not manage to use the bi-tempered loss. Reticulate did not did the tricks.
\nI guess you meant this : https://www.kaggle.com/c/cassava-leaf-disease-classification/notebooks?competitionId=13836&searchQuery=Heroseo :p
\nWhich smoothing value did you used in public kernel ?
", "rawMarkdown": "Haha thanks. I am using R, which I guess explain more why I did not manage to use the bi-tempered loss. Reticulate did not did the tricks.\n\nI guess you meant this : https://www.kaggle.com/c/cassava-leaf-disease-classification/notebooks?competitionId=13836&searchQuery=Heroseo :p\n\nWhich smoothing value did you used in public kernel ?", "votes": 1 }, { "id": 1178084, "postDate": "2021-01-30T16:35:43.517Z", "content": "Most people try and use very small values(e.g. 0.01) and 0.2, 0.3 and even 0.5 for smoothing value in this competition.
\n", "rawMarkdown": "Most people try and use very small values(e.g. 0.01) and 0.2, 0.3 and even 0.5 for smoothing value in this competition.\n\n@cdk292 ", "votes": 2 }, { "id": 1178091, "postDate": "2021-01-30T16:38:47.797Z", "content": "Aha, thanks. I was thinking \"let's go crazy and use 0.2\". I did not though 0.2 to be a small value. Thanks.
", "rawMarkdown": "Aha, thanks. I was thinking \"let's go crazy and use 0.2\". I did not though 0.2 to be a small value. Thanks." }, { "id": 1178117, "postDate": "2021-01-30T16:56:53.497Z", "content": "There was a typo. Hope it helps anyway. @cdk292
", "rawMarkdown": "There was a typo. Hope it helps anyway. @cdk292 ", "votes": 1 } ] }, { "id": 1177650, "postDate": "2021-01-30T12:33:16.790Z", "content": "Thanks for sharing . I am feeling that I am learning a new things today.
", "rawMarkdown": "Thanks for sharing . I am feeling that I am learning a new things today.", "votes": 1 }, { "id": 1167698, "postDate": "2021-01-24T12:39:39.807Z", "content": "thank you for sharing! how about pretrained?
", "rawMarkdown": "thank you for sharing! how about pretrained?", "votes": 1, "replies": [ { "id": 1167730, "postDate": "2021-01-24T13:02:01.857Z", "content": "I don't know exactly what means pretrained.
\nBut it it weights like imagenet pretrained, it is helpful.
Thankyou, Helpful!
", "rawMarkdown": "Thankyou, Helpful!", "votes": 1 }, { "id": 1145930, "postDate": "2021-01-09T12:32:24.783Z", "content": "Thankyou, Helpful!
", "rawMarkdown": "Thankyou, Helpful!", "votes": 1 }, { "id": 1145470, "postDate": "2021-01-09T07:02:20.053Z", "content": "@piantic , Thanks for Sharing this are great insights . :)
", "rawMarkdown": "@piantic , Thanks for Sharing this are great insights . :) ", "votes": 1 }, { "id": 1144226, "postDate": "2021-01-08T10:31:21.230Z", "content": "@piantic Good stuff. I think I can add some things from here to my list too 😊.
\nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/208402
Thanks for updating. I was out of touch from this competition from 20 days. These insights will surely help me :)
", "rawMarkdown": "Thanks for updating. I was out of touch from this competition from 20 days. These insights will surely help me :)", "votes": 1 }, { "id": 1133679, "postDate": "2020-12-31T12:35:35.303Z", "content": "Thank you for your advice! It helps me a lot.I think more dataset in 2019 competition will be helpful! But it is less important than other things you mentioned! Thank you very much your insights!
", "rawMarkdown": "Thank you for your advice! It helps me a lot.I think more dataset in 2019 competition will be helpful! But it is less important than other things you mentioned! Thank you very much your insights! ", "votes": 1 }, { "id": 1160565, "postDate": "2021-01-20T02:39:45.253Z", "content": "Thanks for the suggestions!
", "rawMarkdown": "Thanks for the suggestions!", "votes": 2, "replies": [ { "id": 1160680, "postDate": "2021-01-20T04:27:48.797Z", "content": "I am always learning a lot from you. Thank you! @cdeotte
", "rawMarkdown": "I am always learning a lot from you. Thank you! @cdeotte ", "votes": 1 } ] }, { "id": 1140910, "postDate": "2021-01-06T11:05:40.213Z", "content": "Thank you for your insight!! It really helped!!!
", "rawMarkdown": "Thank you for your insight!! It really helped!!!", "votes": 2 }, { "id": 1131635, "postDate": "2020-12-29T21:31:33.803Z", "content": "How do you Perform normalization? In particular using Keras?
\n\n", "rawMarkdown": "How do you Perform normalization? In particular using Keras?\n\n>Normalization\n>Using mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores", "votes": 2, "replies": [ { "id": 1133267, "postDate": "2020-12-31T04:15:15.023Z", "content": "Normalization
\n
\n Using mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores
@zzy990106 calculated the statistics for the competition to be mean=[0.4580, 0.5274, 0.3245], std=[0.2267, 0.2285, 0.2170]. Based on this information I tried normalizing the input in tensorflow using the function below. But, I do not know if this is the right implementation or not.
\ndef decode_image(path, label=None, target_size=(512, 512)):\n img = tf.image.decode_jpeg(tf.io.read_file(path), channels=3)\n img = tf.cast(img, tf.float32) / 255.0\n ch_1 = (tf.cast(img[:, :, 0], tf.float32) - 0.4590)/ 0.2267\n ch_2 = (tf.cast(img[:, :, 1], tf.float32) - 0.5274)/ 0.2285\n ch_3 = (tf.cast(img[:, :, 2], tf.float32) - 0.3245)/ 0.2170\n img = tf.stack([ch_1, ch_2, ch_3], axis = 2)\n img = tf.image.resize(img, target_size)\n return img if label is None else img, label\nThanks. With above code, you rescaled the range of the image to [0,1]. If using Effnet, it requires input to in in range of [0,255]. Do you need to rescale back to [0,255] before feed the image to training?
", "rawMarkdown": "Thanks. With above code, you rescaled the range of the image to [0,1]. If using Effnet, it requires input to in in range of [0,255]. Do you need to rescale back to [0,255] before feed the image to training?" }, { "id": 1137131, "postDate": "2021-01-03T17:45:56.423Z", "content": "\n\nIf using Effnet, it requires input to in in range of [0,255].
\n
@luqing2 Is it written somewhere in the Effnet documentation? In my case, Effnet is working just fine if you feed it an input from 0 to 1.
", "rawMarkdown": "> If using Effnet, it requires input to in in range of [0,255].\n\n@luqing2 Is it written somewhere in the Effnet documentation? In my case, Effnet is working just fine if you feed it an input from 0 to 1. ", "votes": 2 }, { "id": 1138449, "postDate": "2021-01-04T17:38:43.723Z", "content": "@luqing2 I'm not sure. So far I am getting good results when just dividing input channels by 255 (i.e. rescaling to between 0 and 1). I am getting mediocre results when normalizing as shown in the code above. All notebooks I've come across with respectable scores just divide by 255.
", "rawMarkdown": "@luqing2 I'm not sure. So far I am getting good results when just dividing input channels by 255 (i.e. rescaling to between 0 and 1). I am getting mediocre results when normalizing as shown in the code above. All notebooks I've come across with respectable scores just divide by 255. " }, { "id": 1138739, "postDate": "2021-01-04T23:22:10.040Z", "content": "I followed the tutorial from https://keras.io/examples/vision/image_classification_efficientnet_fine_tuning/
\nit stated stated that Efficientnet has normalization build in, it pixel range should be [0,255]. It is a surprise that range [0,1) worked out to be fine too.
See my reply below
", "rawMarkdown": "See my reply below" }, { "id": 1154268, "postDate": "2021-01-15T14:28:43.293Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1192274, "postDate": "2021-02-09T04:22:20.750Z", "content": "Check the first layers of your EfficientNet. You may see the rescaling layer. For Tensorflow, use:
\n>model.summary()\nModel: \"model\"\n___________________________________________________________\nLayer (type) Output Shape Param # \n============================================================\ninput_1 (InputLayer) [(None, 512, 512, 3) 0 \n____________________________________________________________\nrescaling (Rescaling) (None, 512, 512, 3) 0 \n____________________________________________________________\nnormalization (Normalization) (None, 512, 512, 3) 7 \nThe scaling factor is
\n>1/model.layers[1].scale\n255.0\nCan you share with us your training strategy, how to fine tuning… I thinks that is most important key for get high LB score. Thanks for your tips btw!
", "rawMarkdown": "Can you share with us your training strategy, how to fine tuning... I thinks that is most important key for get high LB score. Thanks for your tips btw!" }, { "id": 1152717, "postDate": "2021-01-14T11:49:02.960Z", "content": "\n\nNormalization
\n
Using mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores.
\nHow is this implemented
", "rawMarkdown": "> Normalization\n\nUsing mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores.\n\nHow is this implemented" }, { "id": 1152675, "postDate": "2021-01-14T11:01:56.427Z", "content": "Hi, @piantic! Thanks a lot for the useful insights! Could you please share on how you calculated mean and std for the dataset? My calculations lead to small numbers and as a result bad score, while mean and std calculated on Imagenet show quite descent results.
", "rawMarkdown": "Hi, @piantic! Thanks a lot for the useful insights! Could you please share on how you calculated mean and std for the dataset? My calculations lead to small numbers and as a result bad score, while mean and std calculated on Imagenet show quite descent results. " }, { "id": 1136723, "postDate": "2021-01-03T11:20:28.137Z", "content": "How many iterations of the model is better?
", "rawMarkdown": "How many iterations of the model is better?" }, { "id": 1134608, "postDate": "2021-01-01T12:27:18.023Z", "content": "How long it takes when we use image size as 384 on Efficicinet-b3 for 5 folds?
\nIf it exceeds more than 9 hrs then, need to think other sources to train the model
For me, it took 8 hours to train my model for 10 epochs with size 384 x 384 on Efficicinet-b3
", "rawMarkdown": "For me, it took **8 hours** to train my model for **10 epochs** with size **384 x 384** on **Efficicinet-b3**" } ] }, { "id": 1132757, "postDate": "2020-12-30T17:00:17.603Z", "content": "I'm sorry, but what does public lb mean?
", "rawMarkdown": "I'm sorry, but what does public lb mean? ", "replies": [ { "id": 1133309, "postDate": "2020-12-31T05:21:44.657Z", "content": "public leaderboard score
", "rawMarkdown": "public leaderboard score" } ] }, { "id": 1132031, "postDate": "2020-12-30T06:10:11.700Z", "content": "Hi,what do you use lr_scheduler?Thanks!I am stoped in 0.898,and cant improve this score.Thanks!
", "rawMarkdown": "Hi,what do you use lr_scheduler?Thanks!I am stoped in 0.898,and cant improve this score.Thanks!" }, { "id": 1131770, "postDate": "2020-12-30T00:21:01.747Z", "content": "Are you using keras or pytorch? I've been having trouble with trying to run the Keras implementation of Bi-Tempered Logistic Loss on TPUs.
", "rawMarkdown": "Are you using keras or pytorch? I've been having trouble with trying to run the Keras implementation of Bi-Tempered Logistic Loss on TPUs." }, { "id": 1131482, "postDate": "2020-12-29T18:44:02.100Z", "content": "What about using the previous dataset?
\nAlso here snapmix helped both in CV and LB.
\nMy current problem is the time of convergence… my models are taking more than 50 epochs.
\nHow many epochs do you need to train a small model?
did swa can contribute in both cv & public lb?
", "rawMarkdown": "did swa can contribute in both cv & public lb?", "votes": 1 }, { "id": 1160212, "postDate": "2021-01-19T18:30:45.550Z", "content": "It's a little lower than my best model score in public lb.
\nIf others want, I would like to update the SWA to my training code.
\n@projdev
Thanks for sharing！！！
", "rawMarkdown": "Thanks for sharing！！！", "votes": 1 }, { "id": 1153220, "postDate": "2021-01-14T17:35:04.283Z", "content": "Thank you! This helped a lot.
", "rawMarkdown": "Thank you! This helped a lot.", "votes": 1 }, { "id": 1151850, "postDate": "2021-01-13T15:55:59.613Z", "content": "Very helpful! Thanks
", "rawMarkdown": "Very helpful! Thanks", "votes": 1 }, { "id": 1147562, "postDate": "2021-01-10T15:24:02.380Z", "content": "Thanks you man.
", "rawMarkdown": "Thanks you man.", "votes": 1 }, { "id": 1147086, "postDate": "2021-01-10T09:26:48.450Z", "content": "Thanks for sharing @piantic
", "rawMarkdown": "Thanks for sharing @piantic ", "votes": 1 }, { "id": 1145100, "postDate": "2021-01-08T22:14:24.587Z", "content": "Ok good ,Thank you. Great stuff!
", "rawMarkdown": "Ok good ,Thank you. Great stuff!", "votes": 1 }, { "id": 1144196, "postDate": "2021-01-08T09:55:16.097Z", "content": "Thank you. Great Job!
", "rawMarkdown": "Thank you. Great Job!", "votes": 1 }, { "id": 1143417, "postDate": "2021-01-07T22:32:06.040Z", "content": "Good work 👍, thanks a lot
", "rawMarkdown": "Good work 👍, thanks a lot ", "votes": 1 }, { "id": 1141586, "postDate": "2021-01-06T19:18:01.523Z", "content": "Thank you. Great stuff!
", "rawMarkdown": "Thank you. Great stuff!", "votes": 1 }, { "id": 1141548, "postDate": "2021-01-06T18:55:19.050Z", "content": "Thanks a lot, very useful!👍
", "rawMarkdown": "Thanks a lot, very useful!👍", "votes": 1 } ], "comments": [ { "id": 2185588, "author_name": "Yeakub Sadlil", "author_url": "", "post_date": "2023-03-17T07:15:54.590000", "content": "Your observation is beautiful
", "votes": 4, "replies": [] }, { "id": 1132524, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-12-30T13:18:40.427000", "content": "Good advices @piantic . With a good decent knowledge in ml/cv you can easily reach top 10% in this competition
\nI would add to this:
Warm restarts didn't seem to help much for me. How many epochs are you using per restart?
", "votes": 0, "replies": [] }, { "id": 1142014, "author_name": "Shourya Mukherjee", "author_url": "", "post_date": "2021-01-07T04:35:46.170000", "content": "What does cleaning the dataset look in this case? Sorry, I'm a beginner, and it's not apparent to me how you go about cleaning a dataset of images.
", "votes": 1, "replies": [] } ] }, { "id": 1131783, "author_name": "ilovescience", "author_url": "", "post_date": "2020-12-30T00:46:44.187000", "content": "BTW \"RAdam+Lookahead\" is called Ranger, so if you see Ranger (like in my kernel), that's the same as what the OP mentioned.
", "votes": 5, "replies": [ { "id": 1133339, "author_name": "tamilselvan eswaramoorthy", "author_url": "", "post_date": "2020-12-31T06:02:58.533000", "content": "Thanks. I was looking for this. BTW had anyone experimented with AdamW and is that helping?
", "votes": 1, "replies": [] } ] }, { "id": 1148577, "author_name": "Sripaad Srinivasan", "author_url": "", "post_date": "2021-01-11T08:21:13.697000", "content": "I've been thinking of how to improve my Lb score, Thanks for the tips.
\n
thank you for sharing . i am new to this and i’m learning great stuff here
", "votes": 1, "replies": [] }, { "id": 1185190, "author_name": "sai1919", "author_url": "", "post_date": "2021-02-04T02:56:41.293000", "content": "Hi,
\nI am new to this field. How one can came up these ideas. Will they do trial and error methods or is there any strategy that by seeing datasets like these will works finne or these won't.
\nLoss function like we need to use label smoothing or we need to use other loss functions.
Similar competition experience and trial and error.
", "votes": 1, "replies": [] } ] }, { "id": 1179384, "author_name": "YokkaBear", "author_url": "", "post_date": "2021-01-31T14:30:28.567000", "content": "Thanks for sharing your gold points, so kind of you! One question, will training on merged dataset (2019 + 2020) help improve lb score compared to original competition dataset (only 2020)? Thank you again!
", "votes": 1, "replies": [ { "id": 1179566, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-31T16:28:56.187000", "content": "Other Kaglers say the merged dataset is helpful. @youjiawang
", "votes": 1, "replies": [] } ] }, { "id": 1178024, "author_name": "Etienne R", "author_url": "", "post_date": "2021-01-30T15:55:08.453000", "content": "Nice topic to go, especially after having enough things to got a clear idea on what it is mentioned here. I am starting training some models with Label-smoothing, I did not managed to implement Bi-Tempered Logistic Loss.
\nRegarding this, I guess there is no obvious way to find out how much which label-smoothing to use except for trial and error ?
", "votes": 1, "replies": [ { "id": 1178048, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-30T16:14:13.207000", "content": "I have implemented loss functions that other kagglers found useful in this competition. I hope it helps.
\n\nAnd I think it will save you trial and error by referring to the notebooks I wrote.
\n\n", "votes": 1, "replies": [] }, { "id": 1178078, "author_name": "Etienne R", "author_url": "", "post_date": "2021-01-30T16:32:37.897000", "content": "Haha thanks. I am using R, which I guess explain more why I did not manage to use the bi-tempered loss. Reticulate did not did the tricks.
\nI guess you meant this : https://www.kaggle.com/c/cassava-leaf-disease-classification/notebooks?competitionId=13836&searchQuery=Heroseo :p
\nWhich smoothing value did you used in public kernel ?
", "votes": 1, "replies": [] }, { "id": 1178084, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-30T16:35:43.517000", "content": "Most people try and use very small values(e.g. 0.01) and 0.2, 0.3 and even 0.5 for smoothing value in this competition.
\n", "votes": 2, "replies": [] }, { "id": 1178091, "author_name": "Etienne R", "author_url": "", "post_date": "2021-01-30T16:38:47.797000", "content": "Aha, thanks. I was thinking \"let's go crazy and use 0.2\". I did not though 0.2 to be a small value. Thanks.
", "votes": 0, "replies": [] }, { "id": 1178117, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-30T16:56:53.497000", "content": "There was a typo. Hope it helps anyway. @cdk292
", "votes": 1, "replies": [] } ] }, { "id": 1177650, "author_name": "sai1919", "author_url": "", "post_date": "2021-01-30T12:33:16.790000", "content": "Thanks for sharing . I am feeling that I am learning a new things today.
", "votes": 1, "replies": [] }, { "id": 1167698, "author_name": "SeHwanJoo", "author_url": "", "post_date": "2021-01-24T12:39:39.807000", "content": "thank you for sharing! how about pretrained?
", "votes": 1, "replies": [ { "id": 1167730, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-24T13:02:01.857000", "content": "I don't know exactly what means pretrained.
\nBut it it weights like imagenet pretrained, it is helpful.
Thankyou, Helpful!
", "votes": 1, "replies": [] }, { "id": 1145930, "author_name": "RUPESH POOJARY", "author_url": "", "post_date": "2021-01-09T12:32:24.783000", "content": "Thankyou, Helpful!
", "votes": 1, "replies": [] }, { "id": 1145470, "author_name": "Athar Sayed", "author_url": "", "post_date": "2021-01-09T07:02:20.053000", "content": "@piantic , Thanks for Sharing this are great insights . :)
", "votes": 1, "replies": [] }, { "id": 1144226, "author_name": "Izzy Adesanya", "author_url": "", "post_date": "2021-01-08T10:31:21.230000", "content": "@piantic Good stuff. I think I can add some things from here to my list too 😊.
\nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/208402
Thanks for updating. I was out of touch from this competition from 20 days. These insights will surely help me :)
", "votes": 1, "replies": [] }, { "id": 1133679, "author_name": "Dongkyu Kim", "author_url": "", "post_date": "2020-12-31T12:35:35.303000", "content": "Thank you for your advice! It helps me a lot.I think more dataset in 2019 competition will be helpful! But it is less important than other things you mentioned! Thank you very much your insights!
", "votes": 1, "replies": [] }, { "id": 1160565, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-01-20T02:39:45.253000", "content": "Thanks for the suggestions!
", "votes": 2, "replies": [ { "id": 1160680, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-20T04:27:48.797000", "content": "I am always learning a lot from you. Thank you! @cdeotte
", "votes": 1, "replies": [] } ] }, { "id": 1140910, "author_name": "Jihan Chae", "author_url": "", "post_date": "2021-01-06T11:05:40.213000", "content": "Thank you for your insight!! It really helped!!!
", "votes": 2, "replies": [] }, { "id": 1131635, "author_name": "luqing Wang", "author_url": "", "post_date": "2020-12-29T21:31:33.803000", "content": "How do you Perform normalization? In particular using Keras?
\n\n", "votes": 2, "replies": [ { "id": 1133267, "author_name": "Matthew Thomas", "author_url": "", "post_date": "2020-12-31T04:15:15.023000", "content": "Normalization
\n
\n Using mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores
@zzy990106 calculated the statistics for the competition to be mean=[0.4580, 0.5274, 0.3245], std=[0.2267, 0.2285, 0.2170]. Based on this information I tried normalizing the input in tensorflow using the function below. But, I do not know if this is the right implementation or not.
\ndef decode_image(path, label=None, target_size=(512, 512)):\n img = tf.image.decode_jpeg(tf.io.read_file(path), channels=3)\n img = tf.cast(img, tf.float32) / 255.0\n ch_1 = (tf.cast(img[:, :, 0], tf.float32) - 0.4590)/ 0.2267\n ch_2 = (tf.cast(img[:, :, 1], tf.float32) - 0.5274)/ 0.2285\n ch_3 = (tf.cast(img[:, :, 2], tf.float32) - 0.3245)/ 0.2170\n img = tf.stack([ch_1, ch_2, ch_3], axis = 2)\n img = tf.image.resize(img, target_size)\n return img if label is None else img, label\nThanks. With above code, you rescaled the range of the image to [0,1]. If using Effnet, it requires input to in in range of [0,255]. Do you need to rescale back to [0,255] before feed the image to training?
", "votes": 0, "replies": [] }, { "id": 1137131, "author_name": "Alexey Pronin", "author_url": "", "post_date": "2021-01-03T17:45:56.423000", "content": "\n\nIf using Effnet, it requires input to in in range of [0,255].
\n
@luqing2 Is it written somewhere in the Effnet documentation? In my case, Effnet is working just fine if you feed it an input from 0 to 1.
", "votes": 2, "replies": [] }, { "id": 1138449, "author_name": "Matthew Thomas", "author_url": "", "post_date": "2021-01-04T17:38:43.723000", "content": "@luqing2 I'm not sure. So far I am getting good results when just dividing input channels by 255 (i.e. rescaling to between 0 and 1). I am getting mediocre results when normalizing as shown in the code above. All notebooks I've come across with respectable scores just divide by 255.
", "votes": 0, "replies": [] }, { "id": 1138739, "author_name": "luqing Wang", "author_url": "", "post_date": "2021-01-04T23:22:10.040000", "content": "I followed the tutorial from https://keras.io/examples/vision/image_classification_efficientnet_fine_tuning/
\nit stated stated that Efficientnet has normalization build in, it pixel range should be [0,255]. It is a surprise that range [0,1) worked out to be fine too.
See my reply below
", "votes": 0, "replies": [] }, { "id": 1154268, "author_name": "", "author_url": "", "post_date": "2021-01-15T14:28:43.293000", "content": "", "votes": 0, "replies": [] }, { "id": 1192274, "author_name": "KWK", "author_url": "", "post_date": "2021-02-09T04:22:20.750000", "content": "Check the first layers of your EfficientNet. You may see the rescaling layer. For Tensorflow, use:
\n>model.summary()\nModel: \"model\"\n___________________________________________________________\nLayer (type) Output Shape Param # \n============================================================\ninput_1 (InputLayer) [(None, 512, 512, 3) 0 \n____________________________________________________________\nrescaling (Rescaling) (None, 512, 512, 3) 0 \n____________________________________________________________\nnormalization (Normalization) (None, 512, 512, 3) 7 \nThe scaling factor is
\n>1/model.layers[1].scale\n255.0\nCan you share with us your training strategy, how to fine tuning… I thinks that is most important key for get high LB score. Thanks for your tips btw!
", "votes": 0, "replies": [] }, { "id": 1152717, "author_name": "norendas", "author_url": "", "post_date": "2021-01-14T11:49:02.960000", "content": "\n\nNormalization
\n
Using mean, std for this competition helps a little boost in CV and public lb, instead of mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] for ImageNet. But the mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] is enough for public lb scores.
\nHow is this implemented
", "votes": 0, "replies": [] }, { "id": 1152675, "author_name": "Emin Tagiev", "author_url": "", "post_date": "2021-01-14T11:01:56.427000", "content": "Hi, @piantic! Thanks a lot for the useful insights! Could you please share on how you calculated mean and std for the dataset? My calculations lead to small numbers and as a result bad score, while mean and std calculated on Imagenet show quite descent results.
", "votes": 0, "replies": [] }, { "id": 1136723, "author_name": "兜里有辣条", "author_url": "", "post_date": "2021-01-03T11:20:28.137000", "content": "How many iterations of the model is better?
", "votes": 0, "replies": [] }, { "id": 1134608, "author_name": "V.Prasanna Kumar", "author_url": "", "post_date": "2021-01-01T12:27:18.023000", "content": "How long it takes when we use image size as 384 on Efficicinet-b3 for 5 folds?
\nIf it exceeds more than 9 hrs then, need to think other sources to train the model
For me, it took 8 hours to train my model for 10 epochs with size 384 x 384 on Efficicinet-b3
", "votes": 0, "replies": [] } ] }, { "id": 1132757, "author_name": "Sander Jyhne", "author_url": "", "post_date": "2020-12-30T17:00:17.603000", "content": "I'm sorry, but what does public lb mean?
", "votes": 0, "replies": [ { "id": 1133309, "author_name": "newbiejailer", "author_url": "", "post_date": "2020-12-31T05:21:44.657000", "content": "public leaderboard score
", "votes": 0, "replies": [] } ] }, { "id": 1132031, "author_name": "Bcw93", "author_url": "", "post_date": "2020-12-30T06:10:11.700000", "content": "Hi,what do you use lr_scheduler?Thanks!I am stoped in 0.898,and cant improve this score.Thanks!
", "votes": 0, "replies": [] }, { "id": 1131770, "author_name": "Gabriel Prado", "author_url": "", "post_date": "2020-12-30T00:21:01.747000", "content": "Are you using keras or pytorch? I've been having trouble with trying to run the Keras implementation of Bi-Tempered Logistic Loss on TPUs.
", "votes": 0, "replies": [] }, { "id": 1131482, "author_name": "IgorMuniz", "author_url": "", "post_date": "2020-12-29T18:44:02.100000", "content": "What about using the previous dataset?
\nAlso here snapmix helped both in CV and LB.
\nMy current problem is the time of convergence… my models are taking more than 50 epochs.
\nHow many epochs do you need to train a small model?
did swa can contribute in both cv & public lb?
", "votes": 1, "replies": [] }, { "id": 1160212, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-19T18:30:45.550000", "content": "It's a little lower than my best model score in public lb.
\nIf others want, I would like to update the SWA to my training code.
\n@projdev