![\"img1\"](\"https://github.com/riron1206/kaggle_Cassava/blob/master/Cassava_Leaf_Disease_Classification_Model_Pipeline.jpg?raw=true\")
Thank you to all my teammates and participants. I would like to share a summary of our solution.
\n※Some teams were removed. I changed Topic Title because our LB changed (private LB 38→34th, public LB 31→30th).
\n
Private LB 34th / Public LB 30th Inference Notebook:
\nhttps://www.kaggle.com/anonamename/cassava-emsemble-v2-stacking?scriptVersionId=54065022
We increased number of TTA for the 224*224 Vit model because accuracy of TTA using RandomResizedCrop with small image size was not stable.
\nSince public LB did not increase with stronger augmentation such as RandomBrightnessContrast, we only did flip.
\nDepending on random number of TTA, public LB changed by about 0.001-0.003, but this problem could not be solved.
We blended ViT-B/16, EfficientnetB4, ResNeSt101e, and SE-Resnext50_32x4d model.
\nWe used Unsupervised Data Augmentation (Semi-Supervised Learning) or BYOL (Self-Supervised Learning) on 2019 data, but it did not contribute much to accuracy improvement.
\nBy averaging k-fold confidence and blending five or more models, we were able to achieve a stable public LB over 0.900.
\nWe blended based on confusion matrix to get a higher cv, but private LB became worse.
We were able to increase public LB from 0.905 to 0.907 by stacking using Conv2d+MLP.
\nWe used predictive label of blended model as Pseudo label.
\nTo avoid overfiting, we trained the model by adding gaussian noise to the feature.
For simple average blending without stacking, there was a submit that was 0.902 for both public and private LB.
\n![\"img2\"](\"https://github.com/riron1206/kaggle_Cassava/blob/master/sub_img.png?raw=true\")
As discussed in other discussions, simple average blending was better for noisy test data sets.
", "messages": [ { "id": "1210384", "postDate": "02/19/2021 11:43:17", "content": "Thank you to all my teammates and participants. I would like to share a summary of our solution.
\n※Some teams were removed. I changed Topic Title because our LB changed (private LB 38→34th, public LB 31→30th).
\n
Private LB 34th / Public LB 30th Inference Notebook:
\nhttps://www.kaggle.com/anonamename/cassava-emsemble-v2-stacking?scriptVersionId=54065022
We increased number of TTA for the 224*224 Vit model because accuracy of TTA using RandomResizedCrop with small image size was not stable.
\nSince public LB did not increase with stronger augmentation such as RandomBrightnessContrast, we only did flip.
\nDepending on random number of TTA, public LB changed by about 0.001-0.003, but this problem could not be solved.
We blended ViT-B/16, EfficientnetB4, ResNeSt101e, and SE-Resnext50_32x4d model.
\nWe used Unsupervised Data Augmentation (Semi-Supervised Learning) or BYOL (Self-Supervised Learning) on 2019 data, but it did not contribute much to accuracy improvement.
\nBy averaging k-fold confidence and blending five or more models, we were able to achieve a stable public LB over 0.900.
\nWe blended based on confusion matrix to get a higher cv, but private LB became worse.
We were able to increase public LB from 0.905 to 0.907 by stacking using Conv2d+MLP.
\nWe used predictive label of blended model as Pseudo label.
\nTo avoid overfiting, we trained the model by adding gaussian noise to the feature.
For simple average blending without stacking, there was a submit that was 0.902 for both public and private LB.
\n
As discussed in other discussions, simple average blending was better for noisy test data sets.
", "rawMarkdown": "Thank you to all my teammates and participants. I would like to share a summary of our solution.\n\n※Some teams were removed. I changed Topic Title because our LB changed (private LB 38→34th, public LB 31→30th).\n\n# Summary\n\n\n\nPrivate LB 34th / Public LB 30th Inference Notebook: \nhttps://www.kaggle.com/anonamename/cassava-emsemble-v2-stacking?scriptVersionId=54065022\n\n### TTA\nWe increased number of TTA for the 224*224 Vit model because accuracy of TTA using RandomResizedCrop with small image size was not stable.\nSince public LB did not increase with stronger augmentation such as RandomBrightnessContrast, we only did flip.\nDepending on random number of TTA, public LB changed by about 0.001-0.003, but this problem could not be solved.\n\n### Blending\nWe blended ViT-B/16, EfficientnetB4, ResNeSt101e, and SE-Resnext50_32x4d model.\nWe used Unsupervised Data Augmentation (Semi-Supervised Learning) or BYOL (Self-Supervised Learning) on 2019 data, but it did not contribute much to accuracy improvement.\nBy averaging k-fold confidence and blending five or more models, we were able to achieve a stable public LB over 0.900.\nWe blended based on confusion matrix to get a higher cv, but private LB became worse.\n\n### Stacking\nWe were able to increase public LB from 0.905 to 0.907 by stacking using Conv2d+MLP.\nWe used predictive label of blended model as Pseudo label.\nTo avoid overfiting, we trained the model by adding gaussian noise to the feature.\n\n-----------------------------------------------------------------------------------------\n\n### Comment\nFor simple average blending without stacking, there was a submit that was 0.902 for both public and private LB.\n\n\nAs discussed in other discussions, simple average blending was better for noisy test data sets.", "votes": null }, { "id": "1210405", "postDate": "02/19/2021 12:08:24", "content": "The explanation of pipeline is clear and well done!
\nI think your team will be able to win the gold medal next time. @anonamename
", "rawMarkdown": "The explanation of pipeline is clear and well done!\n\nI think your team will be able to win the gold medal next time. @anonamename", "votes": null }, { "id": "1212507", "postDate": "02/21/2021 09:34:13", "content": "Thanks @anonamename for sharing your solution. It'll be great learning for beginners like me
", "rawMarkdown": "Thanks @anonamename for sharing your solution. It'll be great learning for beginners like me", "votes": null }, { "id": "1224791", "postDate": "03/03/2021 04:27:34", "content": "How did you use \"Stacking\"?
\nWhat is the input? Does it consist of Pseudo labels?
\nConv2D means you apply 2D Convolusional layer on 2D space. What are the column and the row of the input?
\nI could see your notebook, but just want to hear from you.
The procedure for stacking is as follows.
\n1.Prepare out of fold (oof) for the number of models
\n※column is 5class confidence score.
\n2.Predict test set using average of 5fold for each model
\n3.Concatenate 1 and 2 data (Pseudo labels)
\n4.Combining data into one. shape=(36167, 1, 5, 5)
\n5.Divide combining data into 5fold by StratifiedKfold.
\n6.Train 5fold with torch.nn.Conv2d→mlp 2-layer model
\n7.Use average of 5fold models as final prediction
", "rawMarkdown": "The procedure for stacking is as follows.\n\n1.Prepare out of fold (oof) for the number of models\n- EfficientnetB4_1 5fold oof. shape=(21397,5)\n- ResNeSt101e 5fold oof. shape=(21397,5)\n- EfficientnetB4_2 5fold oof. shape=(21397,5)\n- SE-Resnext50_32x4d 5fold oof. shape=(21397,5)\n- Vit 10fold oof. shape=(21397,5)\n\n※column is 5class confidence score.\n\n\n2.Predict test set using average of 5fold for each model\n- EfficientnetB4_1 5fold test set prediction. shape=(15000,5)\n- ResNeSt101e 5fold test set prediction. shape=(15000,5)\n- EfficientnetB4_2 5fold test set prediction. shape=(15000,5)\n- SE-Resnext50_32x4d 5fold test set prediction. shape=(15000,5)\n- Vit 10fold test set prediction. shape=(15000,5)\n\n3.Concatenate 1 and 2 data (Pseudo labels)\n- EfficientnetB4_1 5fold oof + test set prediction. shape=(36197,5)\n- ResNeSt101e 5fold oof + test set prediction. shape=(36197,5)\n- EfficientnetB4_2 5fold oof + test set prediction. shape=(36197,5)\n- SE-Resnext50_32x4d 5fold oof + test set prediction. shape=(36197,5)\n- Vit 10fold oof + test set prediction. shape=(36197,5)\n\n4.Combining data into one. shape=(36167, 1, 5, 5)\n\n5.Divide combining data into 5fold by StratifiedKfold.\n\n6.Train 5fold with torch.nn.Conv2d→mlp 2-layer model\n\n7.Use average of 5fold models as final prediction", "votes": null } ], "comments": [ { "id": 1210405, "author_name": "piantic", "author_url": "", "post_date": "02/19/2021 12:08:24", "content": "The explanation of pipeline is clear and well done!
\nI think your team will be able to win the gold medal next time. @anonamename
", "votes": null, "replies": [] }, { "id": 1212507, "author_name": "suryajrrafl", "author_url": "", "post_date": "02/21/2021 09:34:13", "content": "Thanks @anonamename for sharing your solution. It'll be great learning for beginners like me
", "votes": null, "replies": [] }, { "id": 1224791, "author_name": "hihunjin", "author_url": "", "post_date": "03/03/2021 04:27:34", "content": "How did you use \"Stacking\"?
\nWhat is the input? Does it consist of Pseudo labels?
\nConv2D means you apply 2D Convolusional layer on 2D space. What are the column and the row of the input?
\nI could see your notebook, but just want to hear from you.
The procedure for stacking is as follows.
\n1.Prepare out of fold (oof) for the number of models
\n※column is 5class confidence score.
\n2.Predict test set using average of 5fold for each model
\n3.Concatenate 1 and 2 data (Pseudo labels)
\n4.Combining data into one. shape=(36167, 1, 5, 5)
\n5.Divide combining data into 5fold by StratifiedKfold.
\n6.Train 5fold with torch.nn.Conv2d→mlp 2-layer model
\n7.Use average of 5fold models as final prediction
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1210384": "Thank you to all my teammates and participants. I would like to share a summary of our solution.\n\n※Some teams were removed. I changed Topic Title because our LB changed (private LB 38→34th, public LB 31→30th).\n\n# Summary\n\n\n\nPrivate LB 34th / Public LB 30th Inference Notebook: \nhttps://www.kaggle.com/anonamename/cassava-emsemble-v2-stacking?scriptVersionId=54065022\n\n### TTA\nWe increased number of TTA for the 224*224 Vit model because accuracy of TTA using RandomResizedCrop with small image size was not stable.\nSince public LB did not increase with stronger augmentation such as RandomBrightnessContrast, we only did flip.\nDepending on random number of TTA, public LB changed by about 0.001-0.003, but this problem could not be solved.\n\n### Blending\nWe blended ViT-B/16, EfficientnetB4, ResNeSt101e, and SE-Resnext50_32x4d model.\nWe used Unsupervised Data Augmentation (Semi-Supervised Learning) or BYOL (Self-Supervised Learning) on 2019 data, but it did not contribute much to accuracy improvement.\nBy averaging k-fold confidence and blending five or more models, we were able to achieve a stable public LB over 0.900.\nWe blended based on confusion matrix to get a higher cv, but private LB became worse.\n\n### Stacking\nWe were able to increase public LB from 0.905 to 0.907 by stacking using Conv2d+MLP.\nWe used predictive label of blended model as Pseudo label.\nTo avoid overfiting, we trained the model by adding gaussian noise to the feature.\n\n-----------------------------------------------------------------------------------------\n\n### Comment\nFor simple average blending without stacking, there was a submit that was 0.902 for both public and private LB.\n\n\nAs discussed in other discussions, simple average blending was better for noisy test data sets.", "1210405": "The explanation of pipeline is clear and well done!\n\nI think your team will be able to win the gold medal next time. @anonamename", "1212507": "Thanks @anonamename for sharing your solution. It'll be great learning for beginners like me", "1224791": "How did you use \"Stacking\"?\nWhat is the input? Does it consist of Pseudo labels?\nConv2D means you apply 2D Convolusional layer on 2D space. What are the column and the row of the input?\nI could see your notebook, but just want to hear from you.", "1225160": "The procedure for stacking is as follows.\n\n1.Prepare out of fold (oof) for the number of models\n- EfficientnetB4_1 5fold oof. shape=(21397,5)\n- ResNeSt101e 5fold oof. shape=(21397,5)\n- EfficientnetB4_2 5fold oof. shape=(21397,5)\n- SE-Resnext50_32x4d 5fold oof. shape=(21397,5)\n- Vit 10fold oof. shape=(21397,5)\n\n※column is 5class confidence score.\n\n\n2.Predict test set using average of 5fold for each model\n- EfficientnetB4_1 5fold test set prediction. shape=(15000,5)\n- ResNeSt101e 5fold test set prediction. shape=(15000,5)\n- EfficientnetB4_2 5fold test set prediction. shape=(15000,5)\n- SE-Resnext50_32x4d 5fold test set prediction. shape=(15000,5)\n- Vit 10fold test set prediction. shape=(15000,5)\n\n3.Concatenate 1 and 2 data (Pseudo labels)\n- EfficientnetB4_1 5fold oof + test set prediction. shape=(36197,5)\n- ResNeSt101e 5fold oof + test set prediction. shape=(36197,5)\n- EfficientnetB4_2 5fold oof + test set prediction. shape=(36197,5)\n- SE-Resnext50_32x4d 5fold oof + test set prediction. shape=(36197,5)\n- Vit 10fold oof + test set prediction. shape=(36197,5)\n\n4.Combining data into one. shape=(36167, 1, 5, 5)\n\n5.Divide combining data into 5fold by StratifiedKfold.\n\n6.Train 5fold with torch.nn.Conv2d→mlp 2-layer model\n\n7.Use average of 5fold models as final prediction" }, "source": "meta" }