{ "id": 221074, "title": "Looking back at my approach 1284 pos [0.8952 private leaderboard and 0.8950 public leaderboard]", "url": "/competitions/cassava-leaf-disease-classification/discussion/221074", "author_name": "Krut Patel", "post_date": "2021-02-21T01:24:26.916000", "votes": 9, "comment_count": 0, "views": 0, "content": "

I got around 0.8952 acc on private leaderboard and 0.8952 on public leaderboard. My final position on private lb is 1284. Surprisingly I went up by 556 positions on private board despite no major change in accuracy. What a shakeup!!

\n

My Approach

\n

1. Models

\n\n

2. k-fold Cross Validation

\n\n

3. Metric Used

\n\n

4. Loss Function

\n\n

5. Image Augmentations

\n\n

6. Training

\n\n

7. Test Time Augmentations (TTA)

\n\n

8. Soft-Voting Ensemble

\n\n

Glance at other approaches

\n\n

Here are links to some top submission notebooks and discussion threads that I enjoyed reading:

\n\n

Conclusion

\n\n

My kernel links

\n

Below are my kernel links. I used separate kernels. One for k-fold cv, one for full training and the other for submission.

\n", "messages": [ { "id": 1212186, "postDate": "2021-02-21T01:24:26.917Z", "content": "

I got around 0.8952 acc on private leaderboard and 0.8952 on public leaderboard. My final position on private lb is 1284. Surprisingly I went up by 556 positions on private board despite no major change in accuracy. What a shakeup!!

\n

My Approach

\n

1. Models

\n\n

2. k-fold Cross Validation

\n\n

3. Metric Used

\n\n

4. Loss Function

\n\n

5. Image Augmentations

\n\n

6. Training

\n\n

7. Test Time Augmentations (TTA)

\n\n

8. Soft-Voting Ensemble

\n\n

Glance at other approaches

\n\n

Here are links to some top submission notebooks and discussion threads that I enjoyed reading:

\n\n

Conclusion

\n\n

My kernel links

\n

Below are my kernel links. I used separate kernels. One for k-fold cv, one for full training and the other for submission.

\n", "rawMarkdown": "I got around 0.8952 acc on private leaderboard and 0.8952 on public leaderboard. My final position on private lb is 1284. Surprisingly I went up by 556 positions on private board despite no major change in accuracy. What a shakeup!!\n\n## My Approach\n### 1. Models\n- Since, I had the option to select my two best performing submissions for the competition, I decided to train two separate models. One being ResNet50_32x4 and the other EfficientNet-B4. Both were initialized with pretrained ImageNet weights.\n\n### 2. k-fold Cross Validation\n- Since no validation set was made available, all the train data had to be used for creating the final model. I decided to use k-fold for model and hyperparameter selection and then train that model on the full set of images.\n- Since the dataset is imbalanced and to maintain the same class distribution across the folds, I used stratified K-fold that makes sure that the percentage of samples for each class are preserved across the folds.\n- I used 5-folds to estimate the best performing model, data augmentation schemes and hyper-parameters, by averaging the predictions on the hold out across all k-folds.\n\n### 3. Metric Used\n- Though, the competition scored submissions based on accuracy, I used F1 score during validation, since the classes are imbalanced and hence accuracy would not have been the best option. But for the final model selection while training on the full trainset I used accuracy as my selection metric.\n\n### 4. Loss Function\n- I tried using a weighted cross entropy loss to overcome the class imbalance, but it did not show any f1-score improvement over cross entropy loss. So I went with the plain cross entropy loss.\n\n### 5. Image Augmentations\n- I started off with the basic image augmentations like random resized cropping, transposing the image, horizontal and vertical flipping, random hue, saturation, value,brightness and contrast adjustments. Yet the model struggled with classifying 0,1, 2, and 4 class labelled images\n- I got a significant boost in the f1 scores across all the classes after adding CoarseDropout, CutOut and CLAHE. Yet still the model was struggling with class 0 and 1.\nFinally using CutMix boosted class 0's and 1's f1-scores.\n\n### 6. Training\n- Both the models were fully trained on the entire training set with model selection based on the model accuracy.\n- I used SGD optimizer with Cosine Annealing learning rate scheduler, decaying the lr every epoch. I also tried cosine annealing with restarts but I saw no reduction in convergence time.\n\n### 7. Test Time Augmentations (TTA)\n- I saw an increase in my public score accuracy after using test time augmentations. I used 10 rounds of predictions for each image, where every time random augmentations are applied to it and then the predictions are averaged.\n- I skipped CoarseDropout and CutOut for test time augmentations since they did more harm then help with predictions. I thought CLAHE would help get a better score when used along TTA. But, I was wrong, so I discarded it from TTA as well.\n\n### 8. Soft-Voting Ensemble\n- To boost my accuracy I used a soft voting ensemble on my best performing Resnet50_32x4 and EfficientNet-B4 model. I tried a number of weights for combining their predictions. The best score I got with was Resnext50 weighted 0.6 and EfficientNet-B4 weighted 0.4\n\n## Glance at other approaches\n- I noticed a few approaches using more models in the ensemble in the numbers of 8 and more and also used previous competitions data as supplement to the given dataset. Here is one discussion I found interesting that used both of them - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220682\n- I saw people combining image augmentation techniques like Fmix, Mixup and CutMix\nThere is a huge variety of models and techniques used across the participants.\n\n## Here are links to some top submission notebooks and discussion threads that I enjoyed reading:\n- 2nd Place - Summary of Approach - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220898 [Surprisingly, this submission uses nothing fancy]\n- 375th place private LB (278th place public) - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220879\n- Private 7th | Public 71th Solution - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220735\n- 8th place solution - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220994\n- 10th place solution (23rd public) - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220788\n\n## Conclusion\n- This was my first kaggle competition and I learned a lot from it. It was fun and exciting reading other participant's kernels, going through interesting ideas on the discussion board and implementing techniques I had learned from papers and school onto a real world problem.\n- I signed up late for the competition, almost 15 days before the deadline and this definitely prevented me from trying different models and augmentation techniques. I can't hold this against anyone but myself.\n- One of things I wanted to do was add train more models to add to the ensemble. Another was picking my ensemble weights, which I could have learned instead of manually trying a few different submission on the public leaderboard.\n- Other things I wanted to try to tackle the imbalance problem was experimenting with focal loss and Remix: rebalanced mixup\n\n## My kernel links\nBelow are my kernel links. I used separate kernels. One for k-fold cv, one for full training and the other for submission.\n- k-fold cv kernel - https://www.kaggle.com/krutpatel2257/cassava-leaf-disease-classification-k-fold\n- full train kernel - https://www.kaggle.com/krutpatel2257/cassava-leaf-disease-classification\n- Ensemble submission - https://www.kaggle.com/krutpatel2257/cassava-leaf-disease-classification-ensemble-sub", "votes": 9 } ], "comments": [], "raw_markdown_by_id": { "1212186": "I got around 0.8952 acc on private leaderboard and 0.8952 on public leaderboard. My final position on private lb is 1284. Surprisingly I went up by 556 positions on private board despite no major change in accuracy. What a shakeup!!\n\n## My Approach\n### 1. Models\n- Since, I had the option to select my two best performing submissions for the competition, I decided to train two separate models. One being ResNet50_32x4 and the other EfficientNet-B4. Both were initialized with pretrained ImageNet weights.\n\n### 2. k-fold Cross Validation\n- Since no validation set was made available, all the train data had to be used for creating the final model. I decided to use k-fold for model and hyperparameter selection and then train that model on the full set of images.\n- Since the dataset is imbalanced and to maintain the same class distribution across the folds, I used stratified K-fold that makes sure that the percentage of samples for each class are preserved across the folds.\n- I used 5-folds to estimate the best performing model, data augmentation schemes and hyper-parameters, by averaging the predictions on the hold out across all k-folds.\n\n### 3. Metric Used\n- Though, the competition scored submissions based on accuracy, I used F1 score during validation, since the classes are imbalanced and hence accuracy would not have been the best option. But for the final model selection while training on the full trainset I used accuracy as my selection metric.\n\n### 4. Loss Function\n- I tried using a weighted cross entropy loss to overcome the class imbalance, but it did not show any f1-score improvement over cross entropy loss. So I went with the plain cross entropy loss.\n\n### 5. Image Augmentations\n- I started off with the basic image augmentations like random resized cropping, transposing the image, horizontal and vertical flipping, random hue, saturation, value,brightness and contrast adjustments. Yet the model struggled with classifying 0,1, 2, and 4 class labelled images\n- I got a significant boost in the f1 scores across all the classes after adding CoarseDropout, CutOut and CLAHE. Yet still the model was struggling with class 0 and 1.\nFinally using CutMix boosted class 0's and 1's f1-scores.\n\n### 6. Training\n- Both the models were fully trained on the entire training set with model selection based on the model accuracy.\n- I used SGD optimizer with Cosine Annealing learning rate scheduler, decaying the lr every epoch. I also tried cosine annealing with restarts but I saw no reduction in convergence time.\n\n### 7. Test Time Augmentations (TTA)\n- I saw an increase in my public score accuracy after using test time augmentations. I used 10 rounds of predictions for each image, where every time random augmentations are applied to it and then the predictions are averaged.\n- I skipped CoarseDropout and CutOut for test time augmentations since they did more harm then help with predictions. I thought CLAHE would help get a better score when used along TTA. But, I was wrong, so I discarded it from TTA as well.\n\n### 8. Soft-Voting Ensemble\n- To boost my accuracy I used a soft voting ensemble on my best performing Resnet50_32x4 and EfficientNet-B4 model. I tried a number of weights for combining their predictions. The best score I got with was Resnext50 weighted 0.6 and EfficientNet-B4 weighted 0.4\n\n## Glance at other approaches\n- I noticed a few approaches using more models in the ensemble in the numbers of 8 and more and also used previous competitions data as supplement to the given dataset. Here is one discussion I found interesting that used both of them - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220682\n- I saw people combining image augmentation techniques like Fmix, Mixup and CutMix\nThere is a huge variety of models and techniques used across the participants.\n\n## Here are links to some top submission notebooks and discussion threads that I enjoyed reading:\n- 2nd Place - Summary of Approach - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220898 [Surprisingly, this submission uses nothing fancy]\n- 375th place private LB (278th place public) - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220879\n- Private 7th | Public 71th Solution - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220735\n- 8th place solution - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220994\n- 10th place solution (23rd public) - https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220788\n\n## Conclusion\n- This was my first kaggle competition and I learned a lot from it. It was fun and exciting reading other participant's kernels, going through interesting ideas on the discussion board and implementing techniques I had learned from papers and school onto a real world problem.\n- I signed up late for the competition, almost 15 days before the deadline and this definitely prevented me from trying different models and augmentation techniques. I can't hold this against anyone but myself.\n- One of things I wanted to do was add train more models to add to the ensemble. Another was picking my ensemble weights, which I could have learned instead of manually trying a few different submission on the public leaderboard.\n- Other things I wanted to try to tackle the imbalance problem was experimenting with focal loss and Remix: rebalanced mixup\n\n## My kernel links\nBelow are my kernel links. I used separate kernels. One for k-fold cv, one for full training and the other for submission.\n- k-fold cv kernel - https://www.kaggle.com/krutpatel2257/cassava-leaf-disease-classification-k-fold\n- full train kernel - https://www.kaggle.com/krutpatel2257/cassava-leaf-disease-classification\n- Ensemble submission - https://www.kaggle.com/krutpatel2257/cassava-leaf-disease-classification-ensemble-sub" } }