{ "id": 329299, "title": "1st place solution", "url": "/competitions/herbarium-2022-fgvc9/discussion/329299", "author_name": "Shuhao", "post_date": "2022-06-06T02:45:52.742000", "votes": 5, "comment_count": 1, "views": 0, "content": "", "messages": [ { "id": 1812566, "postDate": "2022-06-06T02:45:52.743Z", "content": "", "rawMarkdown": "- Optimization on single model:\n|change|public|private|\n|:--|:--|:--|\n|swin-B 224 baseline|0.77223|0.78442|\n|add cross-entropy losses on genera,species, family|0.7829|0.79544|\n|learning rate from 2e-4 to 5e-4|0.79444|0.80501|\n|+5crop (resize 256, 288, 320, 384, 448 and center crop 224)|0.80088|0.80981|\n|replacing cross-entropy by subcenter-arcface with [dynamic](https://www.kaggle.com/c/landmark-recognition-2020/discussion/187757) margins|0.81532|0.82267|\n|+ extra fc by cross-entropy, add to subcenter-arcface output after softmax|0.82201|0.82929|\n|+ data augmentation the same with Swin-Transformer|0.8315|0.83554|\n|from swin-B224 to SwinB384|0.8468|0.85245|\n|+5crop (resize 400, 416, 448, 480, 512 and center crop 384)|0.84895|0.85654|\n|+Freeze layers (not update parameters) from 100 to 0|0.85499|0.86055|\n|+square resize and test crop from 384|0.85563|0.86201|\n|+from swin base to swin V2|0.85851|0.86282|\n\n\n- Backbones:\n|id|backbone|public|private|\n|:--|:--|:--|:--|\n|0|swin-B|0.85499|0.86055|\n|1|convnext-B|0.85464|0.85956|\n|2|deit-iii B|0.84423|0.8495|\n|3|resnest-101|0.83208|0.83838|\n|4|efficient-B6|0.83841|0.84321|\n|5|cswin-L|0.84373|0.8501|\n|6|swin-L|0.85057|0.85607|\n|7|swinv2-B|0.85851|0.86282|\n\n\n- Fusion：\nwe utilize the scores on public dataset as the weight. \n|fusion approach|public|private|\n|:--|:--|:--|\n|fusion models (0,1,2)|0.86366|0.8679|\n|fusion models (0,1,2,3,4)|0.86619|0.87185|\n|fusion models (0,1,2,3,4,5,6,7)|0.87193|0.87662|\n\n\n- Other settings:\nBatch size is always 512 (256 for big models), trained with 100epochs.\ntorch.amp is adopted.\nFollowing [solutions](https://www.kaggle.com/competitions/herbarium-2021-fgvc8/discussion/242233)， we adopt similar Post-Process and obtain 0.001 improvement.\nWe use Swin-B pretrained on Imagenet22k for initialization, for other backbones, we try to use Imagenet22k pretrained weights if possible.\nWe freeze partial layers to enlarge the batchsize during the training process, We then reduce the batch size and train the full parameters. \n\n\n- Something less effective:\nclass-aware sampling\ndata cleaning \n\n\n\n", "votes": 5 }, { "id": 1815325, "postDate": "2022-06-08T23:19:20.103Z", "content": "

Hi @shuhaocui, congratulations and thank you very much for the detailed explanation of your methods. Would you please fill out this form as well? https://forms.gle/n8Bsr5ZfkK4VLP1f7 Thank you.

", "rawMarkdown": "Hi @shuhaocui, congratulations and thank you very much for the detailed explanation of your methods. Would you please fill out this form as well? https://forms.gle/n8Bsr5ZfkK4VLP1f7 Thank you." } ], "comments": [ { "id": 1815325, "author_name": "John Park", "author_url": "", "post_date": "2022-06-08T23:19:20.103000", "content": "

Hi @shuhaocui, congratulations and thank you very much for the detailed explanation of your methods. Would you please fill out this form as well? https://forms.gle/n8Bsr5ZfkK4VLP1f7 Thank you.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1812566": "- Optimization on single model:\n|change|public|private|\n|:--|:--|:--|\n|swin-B 224 baseline|0.77223|0.78442|\n|add cross-entropy losses on genera,species, family|0.7829|0.79544|\n|learning rate from 2e-4 to 5e-4|0.79444|0.80501|\n|+5crop (resize 256, 288, 320, 384, 448 and center crop 224)|0.80088|0.80981|\n|replacing cross-entropy by subcenter-arcface with [dynamic](https://www.kaggle.com/c/landmark-recognition-2020/discussion/187757) margins|0.81532|0.82267|\n|+ extra fc by cross-entropy, add to subcenter-arcface output after softmax|0.82201|0.82929|\n|+ data augmentation the same with Swin-Transformer|0.8315|0.83554|\n|from swin-B224 to SwinB384|0.8468|0.85245|\n|+5crop (resize 400, 416, 448, 480, 512 and center crop 384)|0.84895|0.85654|\n|+Freeze layers (not update parameters) from 100 to 0|0.85499|0.86055|\n|+square resize and test crop from 384|0.85563|0.86201|\n|+from swin base to swin V2|0.85851|0.86282|\n\n\n- Backbones:\n|id|backbone|public|private|\n|:--|:--|:--|:--|\n|0|swin-B|0.85499|0.86055|\n|1|convnext-B|0.85464|0.85956|\n|2|deit-iii B|0.84423|0.8495|\n|3|resnest-101|0.83208|0.83838|\n|4|efficient-B6|0.83841|0.84321|\n|5|cswin-L|0.84373|0.8501|\n|6|swin-L|0.85057|0.85607|\n|7|swinv2-B|0.85851|0.86282|\n\n\n- Fusion：\nwe utilize the scores on public dataset as the weight. \n|fusion approach|public|private|\n|:--|:--|:--|\n|fusion models (0,1,2)|0.86366|0.8679|\n|fusion models (0,1,2,3,4)|0.86619|0.87185|\n|fusion models (0,1,2,3,4,5,6,7)|0.87193|0.87662|\n\n\n- Other settings:\nBatch size is always 512 (256 for big models), trained with 100epochs.\ntorch.amp is adopted.\nFollowing [solutions](https://www.kaggle.com/competitions/herbarium-2021-fgvc8/discussion/242233)， we adopt similar Post-Process and obtain 0.001 improvement.\nWe use Swin-B pretrained on Imagenet22k for initialization, for other backbones, we try to use Imagenet22k pretrained weights if possible.\nWe freeze partial layers to enlarge the batchsize during the training process, We then reduce the batch size and train the full parameters. \n\n\n- Something less effective:\nclass-aware sampling\ndata cleaning \n\n\n\n", "1815325": "Hi @shuhaocui, congratulations and thank you very much for the detailed explanation of your methods. Would you please fill out this form as well? https://forms.gle/n8Bsr5ZfkK4VLP1f7 Thank you." } }