{ "id": 220969, "title": "[Public 272nd / Private 37th] 1 week solution", "url": "/competitions/cassava-leaf-disease-classification/writeups/1-week-tea-break-public-272nd-private-37th-1-week-", "author_name": "", "post_date": "2021-02-20T13:55:53.310Z", "votes": 30, "comment_count": 1, "views": 0, "content": "

First of all, congrats to all the participants who got medals, and thanks to the host and kaggle team.

\n

There is little difference in the scores of most participants. I'm really looking forward to the 1st place solution :)

\n

Brief Summary

\n\n

Details

\n

Data

\n

I used publicly shared merged training dataset(2019 + 2020). Thank you @tahsin for sharing it.

\n

Image Classification Models

\n

I used the following 10 models provided in timm.

\n\n

Training

\n
cross validation
\n\n
data augmentation
\n

Transpose -> HorizontalFlip -> VerticalFlip ->
\nShiftScaleRotate -> RandomResizedCrop ->
\nHueSaturationValue -> RandomBrightnessContrast ->
\nOneOf(RandomErase, CoarseDropout, Cutout) ->
\nNormalize -> ToTensorV2

\n
other settings
\n\n
Ensemble
\n\n

Unfortunately, however, simple averaging has better private score(0.9014) than this stacking(0.9008). I missed a gold medal 😇

\n


\n

That's all. Thanks you reading 😉

", "messages": [ { "id": "1211525", "postDate": "02/20/2021 09:57:52", "content": "

First of all, congrats to all the participants who got medals, and thanks to the host and kaggle team.

\n

There is little difference in the scores of most participants. I'm really looking forward to the 1st place solution :)

\n

Brief Summary

\n\n

Details

\n

Data

\n

I used publicly shared merged training dataset(2019 + 2020). Thank you @tahsin for sharing it.

\n

Image Classification Models

\n

I used the following 10 models provided in timm.

\n\n

Training

\n
cross validation
\n\n
data augmentation
\n

Transpose -> HorizontalFlip -> VerticalFlip ->
\nShiftScaleRotate -> RandomResizedCrop ->
\nHueSaturationValue -> RandomBrightnessContrast ->
\nOneOf(RandomErase, CoarseDropout, Cutout) ->
\nNormalize -> ToTensorV2

\n
other settings
\n\n
Ensemble
\n\n

Unfortunately, however, simple averaging has better private score(0.9014) than this stacking(0.9008). I missed a gold medal 😇

\n


\n

That's all. Thanks you reading 😉

", "rawMarkdown": "First of all, congrats to all the participants who got medals, and thanks to the host and kaggle team.\n\nThere is little difference in the scores of most participants. I'm really looking forward to the 1st place solution :)\n\n\n### Brief Summary\n\n* training 10 image classification models and ensembling them by 1D-CNN, 2D-CNN, and weight optimization\n* using merged dataset(2019 + 2020) for training but only 2020 for validation\n* using Cross Entropy Loss with label Smoothing (alpha=0.3)\n* as the result, simple averaging(I didn't select as final submission) is better than stacking 😇\n * stacking(Public: 0.9014, Private: 0.9008): https://www.kaggle.com/ttahara/infer-cassava-ens10\n * averaging(Public: 0.9018, Private 0.9014): https://www.kaggle.com/ttahara/infer-cassava-ens09\n\n### Details\n\n#### Data\nI used [publicly shared merged training dataset(2019 + 2020)](https://www.kaggle.com/tahsin/cassava-leaf-disease-merged). Thank you @tahsin for sharing it.\n\n#### Image Classification Models\n I used the following 10 models provided in [timm](https://github.com/rwightman/pytorch-image-models). \n\n* small models: ResNet50D, ResNeXt50_32x4d, ECA-ResNet50D, ResNeSt50_fast_1s4x24d, RegNetY032\n\n* big models: ResNet100D, ResNeXt101_32x4d, ECA-ResNet101D, ResNeSt101, RegNetY080 \n\n#### Training\n##### cross validation\n* spliting merged training dataset by Stratified K-fold manner(K=5)\n* using 2019 and 2020 data for training but only 2020 for validation\n\n##### data augmentation\n\nTranspose -> HorizontalFlip -> VerticalFlip -> \nShiftScaleRotate -> RandomResizedCrop ->\nHueSaturationValue -> RandomBrightnessContrast -> \nOneOf(RandomErase, CoarseDropout, Cutout) ->\nNormalize -> ToTensorV2\n\n##### other settings\n* image size: 512x512x3\n* max epoch : 10\n* batch size: 64(small models), 32(big models)\n* loss: CrossEntropy with LabelSmoothing\n * smoothing alpha: **0.3**\n* optimizer: AdamW\n * weight decay: 1.0e-02\n * learning rate: 5e-04(small models), 2.5e-04(big models)\n* scheduler: CosineAnnealingWarmRestarts\n * T_0: 10\n * T_mult: 1\n\n##### Ensemble\n* training 1D-CNN and 2D-CNN using image classification models' outputs as inputs\n * for more details, see my past topic in MoA competition: \nhttps://www.kaggle.com/c/lish-moa/discussion/204685\n* applying weight optimization to image classification models' outputs\n* finally averaging these three models' outputs\n\nUnfortunately, however, simple averaging has better private score(0.9014) than this stacking(0.9008). I missed a gold medal 😇\n\n
\n\nThat's all. Thanks you reading 😉", "votes": null }, { "id": "1213897", "postDate": "02/22/2021 12:25:11", "content": "

Thanks for sharing and good work! @ttahara

", "rawMarkdown": "Thanks for sharing and good work! @ttahara", "votes": null } ], "comments": [ { "id": 1213897, "author_name": "piantic", "author_url": "", "post_date": "02/22/2021 12:25:11", "content": "

Thanks for sharing and good work! @ttahara

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1211525": "First of all, congrats to all the participants who got medals, and thanks to the host and kaggle team.\n\nThere is little difference in the scores of most participants. I'm really looking forward to the 1st place solution :)\n\n\n### Brief Summary\n\n* training 10 image classification models and ensembling them by 1D-CNN, 2D-CNN, and weight optimization\n* using merged dataset(2019 + 2020) for training but only 2020 for validation\n* using Cross Entropy Loss with label Smoothing (alpha=0.3)\n* as the result, simple averaging(I didn't select as final submission) is better than stacking 😇\n * stacking(Public: 0.9014, Private: 0.9008): https://www.kaggle.com/ttahara/infer-cassava-ens10\n * averaging(Public: 0.9018, Private 0.9014): https://www.kaggle.com/ttahara/infer-cassava-ens09\n\n### Details\n\n#### Data\nI used [publicly shared merged training dataset(2019 + 2020)](https://www.kaggle.com/tahsin/cassava-leaf-disease-merged). Thank you @tahsin for sharing it.\n\n#### Image Classification Models\n I used the following 10 models provided in [timm](https://github.com/rwightman/pytorch-image-models). \n\n* small models: ResNet50D, ResNeXt50_32x4d, ECA-ResNet50D, ResNeSt50_fast_1s4x24d, RegNetY032\n\n* big models: ResNet100D, ResNeXt101_32x4d, ECA-ResNet101D, ResNeSt101, RegNetY080 \n\n#### Training\n##### cross validation\n* spliting merged training dataset by Stratified K-fold manner(K=5)\n* using 2019 and 2020 data for training but only 2020 for validation\n\n##### data augmentation\n\nTranspose -> HorizontalFlip -> VerticalFlip -> \nShiftScaleRotate -> RandomResizedCrop ->\nHueSaturationValue -> RandomBrightnessContrast -> \nOneOf(RandomErase, CoarseDropout, Cutout) ->\nNormalize -> ToTensorV2\n\n##### other settings\n* image size: 512x512x3\n* max epoch : 10\n* batch size: 64(small models), 32(big models)\n* loss: CrossEntropy with LabelSmoothing\n * smoothing alpha: **0.3**\n* optimizer: AdamW\n * weight decay: 1.0e-02\n * learning rate: 5e-04(small models), 2.5e-04(big models)\n* scheduler: CosineAnnealingWarmRestarts\n * T_0: 10\n * T_mult: 1\n\n##### Ensemble\n* training 1D-CNN and 2D-CNN using image classification models' outputs as inputs\n * for more details, see my past topic in MoA competition: \nhttps://www.kaggle.com/c/lish-moa/discussion/204685\n* applying weight optimization to image classification models' outputs\n* finally averaging these three models' outputs\n\nUnfortunately, however, simple averaging has better private score(0.9014) than this stacking(0.9008). I missed a gold medal 😇\n\n
\n\nThat's all. Thanks you reading 😉", "1213897": "Thanks for sharing and good work! @ttahara" }, "source": "meta" }