{ "id": 83885, "title": "2nd place code, end to end whale Identification model ", "url": "/competitions/humpback-whale-identification/discussion/83885", "author_name": "SeuTao", "post_date": "2019-03-13T18:17:08.164000", "votes": 60, "comment_count": 17, "views": 0, "content": "

Here is the code of my solution\n[https://github.com/SeuTao/Kaggle_Whale2019_2nd_palce_solution]

", "messages": [ { "id": 489285, "postDate": "2019-03-13T18:17:08.163Z", "content": "

Here is the code of my solution\n[https://github.com/SeuTao/Kaggle_Whale2019_2nd_palce_solution]

", "rawMarkdown": "Here is the code of my solution\n[https://github.com/SeuTao/Kaggle_Whale2019_2nd_palce_solution]", "votes": 60 }, { "id": 492226, "postDate": "2019-03-16T23:54:19.660Z", "content": "

@shentao Thank you for sharing, and congratulations! Your solution is quite simple enough and strong.

\n\n

And I'd like to share with people who is using a single GPU,\nLB 0.96+ is reproducible with single GPU, by decreasing batch size to 32. It took two days.

\n\n

Thanks again SeuTao, it's great work...

", "rawMarkdown": "@shentao Thank you for sharing, and congratulations! Your solution is quite simple enough and strong.\n\nAnd I'd like to share with people who is using a single GPU,\nLB 0.96+ is reproducible with single GPU, by decreasing batch size to 32. It took two days.\n\nThanks again SeuTao, it's great work...", "votes": 6, "replies": [ { "id": 492259, "postDate": "2019-03-17T01:41:16.980Z", "content": "

Wow, Thanks for your information!

", "rawMarkdown": "Wow, Thanks for your information! ", "votes": 1 } ] }, { "id": 489904, "postDate": "2019-03-14T06:49:39.763Z", "content": "

Thanks for sharing your solution. In fact, look like your idea is very similar to the Winner (Classification + TripletLoss).

", "rawMarkdown": "Thanks for sharing your solution. In fact, look like your idea is very similar to the Winner (Classification + TripletLoss). ", "votes": 3, "replies": [ { "id": 490038, "postDate": "2019-03-14T08:49:33.503Z", "content": "

Yes, I also use clf based method, whereas I think my network is simpler and lighter, only 3 channels' input as well as one-stage lr schedule are used. Actually, margin based softmax loss is the key in my pipeline, removing triplet loss has no harm on the final score.

", "rawMarkdown": "Yes, I also use clf based method, whereas I think my network is simpler and lighter, only 3 channels' input as well as one-stage lr schedule are used. Actually, margin based softmax loss is the key in my pipeline, removing triplet loss has no harm on the final score. ", "votes": 8 }, { "id": 490208, "postDate": "2019-03-14T11:42:31.767Z", "content": "

Thanks for the information. For my margin based softmax was also the key-point, but I achived much smaller score (this is why I'm searching for difference between our approach). You did not calcualte any center-of-class, right?\nAlso, do you think that Focal-Loss was crucial for accuracy of final model? I was trying to apply the Focal-Loss on Margin-SoftMax without any sucess. But applying it parallel is greate idea.

", "rawMarkdown": "Thanks for the information. For my margin based softmax was also the key-point, but I achived much smaller score (this is why I'm searching for difference between our approach). You did not calcualte any center-of-class, right?\nAlso, do you think that Focal-Loss was crucial for accuracy of final model? I was trying to apply the Focal-Loss on Margin-SoftMax without any sucess. But applying it parallel is greate idea.\n\n", "votes": 1 }, { "id": 490234, "postDate": "2019-03-14T12:23:47.617Z", "content": "

My baseline model is trained on arcface only (non-new_whale images). For inference, I remove the margin m and directly use the softmax result of last fc. The L2-normed weight matrix of last fc is close to the center-of-class feature. My arcface only model can achieve around 0.930~940 on public LB. The further improvement comes from the added binary head with focal loss (both on non-new_whale and new_whale images ). The ID flipping trick is important which gaves me the final 0.01 boost.

", "rawMarkdown": "My baseline model is trained on arcface only (non-new_whale images). For inference, I remove the margin m and directly use the softmax result of last fc. The L2-normed weight matrix of last fc is close to the center-of-class feature. My arcface only model can achieve around 0.930~940 on public LB. The further improvement comes from the added binary head with focal loss (both on non-new_whale and new_whale images ). The ID flipping trick is important which gaves me the final 0.01 boost.", "votes": 7 }, { "id": 490312, "postDate": "2019-03-14T13:22:58.747Z", "content": "

About new-whale, look like I have similar idea, but I does not work well for me (I was forcing the logits to have zero values). But I was using the same Linear payer like for Margin-Softmax and BCE. You have just two independent layers and losses. Also for final eval you used BCE output.

\n\n

In general, look like you have some very nice ideas for learning model. Again, thanks for sharing.

", "rawMarkdown": "About new-whale, look like I have similar idea, but I does not work well for me (I was forcing the logits to have zero values). But I was using the same Linear payer like for Margin-Softmax and BCE. You have just two independent layers and losses. Also for final eval you used BCE output. \n\nIn general, look like you have some very nice ideas for learning model. Again, thanks for sharing.", "votes": 3 }, { "id": 493184, "postDate": "2019-03-18T11:44:35.703Z", "content": "

Indeed, the flipping trick to double ids is really important. After using it, I managed to get 0.95864/0.95815 private/public LB using a single DenseNet121 and CosFace (512x512 images). I guess the rest comes with the binary head and ensembles.

\n\n

By the way, the flipping trick also works well for other datasets. I've tested it in a private dog identification dataset and it also helped considerably!

", "rawMarkdown": "Indeed, the flipping trick to double ids is really important. After using it, I managed to get 0.95864/0.95815 private/public LB using a single DenseNet121 and CosFace (512x512 images). I guess the rest comes with the binary head and ensembles.\n\nBy the way, the flipping trick also works well for other datasets. I've tested it in a private dog identification dataset and it also helped considerably!", "votes": 5 }, { "id": 495561, "postDate": "2019-03-21T08:58:20.103Z", "content": "

hi seu,congrats for your position..\nhi seu,congrats for your position..

\n\n

1) What you mean by margin based softmxloss..? Which loss do you refer to ?\n2) What is image size used to achieve score above 90 using Arcface . Using 356 size Arcface,resnet50 i m getting so far 90.9 on PLB. \n3) Could you please elaborate on Flipping trick

", "rawMarkdown": "hi seu,congrats for your position..\nhi seu,congrats for your position..\n\n\n1) What you mean by margin based softmxloss..? Which loss do you refer to ?\n2) What is image size used to achieve score above 90 using Arcface . Using 356 size Arcface,resnet50 i m getting so far 90.9 on PLB. \n3) Could you please elaborate on Flipping trick" } ] }, { "id": 489439, "postDate": "2019-03-13T23:06:26.967Z", "content": "

Congratulations on the second place and for the grandmaster rank! =]

", "rawMarkdown": "Congratulations on the second place and for the grandmaster rank! =]", "votes": 2 }, { "id": 2371589, "postDate": "2023-08-03T07:27:34.890Z", "content": "

Thanks for attaching pptx on GitHub, really insightful, helped me learn a lot.

", "rawMarkdown": "Thanks for attaching pptx on GitHub, really insightful, helped me learn a lot." }, { "id": 1693997, "postDate": "2022-02-17T05:29:49.877Z", "content": "

congratulations @shentao 😎Thanks for sharing!🤗 new follower 🙋‍♀️

", "rawMarkdown": "congratulations @shentao 😎Thanks for sharing!🤗 new follower 🙋‍♀️\n" }, { "id": 577761, "postDate": "2019-07-17T03:45:52.250Z", "content": "

Thanks @shentao for sharing ..and congrats for the good work

", "rawMarkdown": "Thanks @shentao for sharing ..and congrats for the good work" }, { "id": 489934, "postDate": "2019-03-14T07:21:24.793Z", "content": "

great job，congratulations

", "rawMarkdown": "great job，congratulations" }, { "id": 577658, "postDate": "2019-07-16T22:58:27.367Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1693596, "postDate": "2022-02-16T19:20:06.223Z", "content": "

thanks for sharing

", "rawMarkdown": "thanks for sharing" }, { "id": 577743, "postDate": "2019-07-17T03:13:23.200Z", "content": "

thanks for sharing

", "rawMarkdown": "thanks for sharing" } ], "comments": [ { "id": 492226, "author_name": "daisukelab", "author_url": "", "post_date": "2019-03-16T23:54:19.660000", "content": "

@shentao Thank you for sharing, and congratulations! Your solution is quite simple enough and strong.

\n\n

And I'd like to share with people who is using a single GPU,\nLB 0.96+ is reproducible with single GPU, by decreasing batch size to 32. It took two days.

\n\n

Thanks again SeuTao, it's great work...

", "votes": 6, "replies": [ { "id": 492259, "author_name": "SeuTao", "author_url": "", "post_date": "2019-03-17T01:41:16.980000", "content": "

Wow, Thanks for your information!

", "votes": 1, "replies": [] } ] }, { "id": 489904, "author_name": "Bartek", "author_url": "", "post_date": "2019-03-14T06:49:39.763000", "content": "

Thanks for sharing your solution. In fact, look like your idea is very similar to the Winner (Classification + TripletLoss).

", "votes": 3, "replies": [ { "id": 490038, "author_name": "SeuTao", "author_url": "", "post_date": "2019-03-14T08:49:33.503000", "content": "

Yes, I also use clf based method, whereas I think my network is simpler and lighter, only 3 channels' input as well as one-stage lr schedule are used. Actually, margin based softmax loss is the key in my pipeline, removing triplet loss has no harm on the final score.

", "votes": 8, "replies": [] }, { "id": 490208, "author_name": "Bartek", "author_url": "", "post_date": "2019-03-14T11:42:31.767000", "content": "

Thanks for the information. For my margin based softmax was also the key-point, but I achived much smaller score (this is why I'm searching for difference between our approach). You did not calcualte any center-of-class, right?\nAlso, do you think that Focal-Loss was crucial for accuracy of final model? I was trying to apply the Focal-Loss on Margin-SoftMax without any sucess. But applying it parallel is greate idea.

", "votes": 1, "replies": [] }, { "id": 490234, "author_name": "SeuTao", "author_url": "", "post_date": "2019-03-14T12:23:47.617000", "content": "

My baseline model is trained on arcface only (non-new_whale images). For inference, I remove the margin m and directly use the softmax result of last fc. The L2-normed weight matrix of last fc is close to the center-of-class feature. My arcface only model can achieve around 0.930~940 on public LB. The further improvement comes from the added binary head with focal loss (both on non-new_whale and new_whale images ). The ID flipping trick is important which gaves me the final 0.01 boost.

", "votes": 7, "replies": [] }, { "id": 490312, "author_name": "Bartek", "author_url": "", "post_date": "2019-03-14T13:22:58.747000", "content": "

About new-whale, look like I have similar idea, but I does not work well for me (I was forcing the logits to have zero values). But I was using the same Linear payer like for Margin-Softmax and BCE. You have just two independent layers and losses. Also for final eval you used BCE output.

\n\n

In general, look like you have some very nice ideas for learning model. Again, thanks for sharing.

", "votes": 3, "replies": [] }, { "id": 493184, "author_name": "Eduardo Rocha de Andrade", "author_url": "", "post_date": "2019-03-18T11:44:35.703000", "content": "

Indeed, the flipping trick to double ids is really important. After using it, I managed to get 0.95864/0.95815 private/public LB using a single DenseNet121 and CosFace (512x512 images). I guess the rest comes with the binary head and ensembles.

\n\n

By the way, the flipping trick also works well for other datasets. I've tested it in a private dog identification dataset and it also helped considerably!

", "votes": 5, "replies": [] }, { "id": 495561, "author_name": "Jaideep", "author_url": "", "post_date": "2019-03-21T08:58:20.103000", "content": "

hi seu,congrats for your position..\nhi seu,congrats for your position..

\n\n

1) What you mean by margin based softmxloss..? Which loss do you refer to ?\n2) What is image size used to achieve score above 90 using Arcface . Using 356 size Arcface,resnet50 i m getting so far 90.9 on PLB. \n3) Could you please elaborate on Flipping trick

", "votes": 0, "replies": [] } ] }, { "id": 489439, "author_name": "Eduardo Rocha de Andrade", "author_url": "", "post_date": "2019-03-13T23:06:26.967000", "content": "

Congratulations on the second place and for the grandmaster rank! =]

", "votes": 2, "replies": [] }, { "id": 2371589, "author_name": "nikhilsos", "author_url": "", "post_date": "2023-08-03T07:27:34.890000", "content": "

Thanks for attaching pptx on GitHub, really insightful, helped me learn a lot.

", "votes": 0, "replies": [] }, { "id": 1693997, "author_name": "Aruna S", "author_url": "", "post_date": "2022-02-17T05:29:49.877000", "content": "

congratulations @shentao 😎Thanks for sharing!🤗 new follower 🙋‍♀️

", "votes": 0, "replies": [] }, { "id": 577761, "author_name": "P_R_Mohanty", "author_url": "", "post_date": "2019-07-17T03:45:52.250000", "content": "

Thanks @shentao for sharing ..and congrats for the good work

", "votes": 0, "replies": [] }, { "id": 489934, "author_name": "QiangBing Peng", "author_url": "", "post_date": "2019-03-14T07:21:24.793000", "content": "

great job，congratulations

", "votes": 0, "replies": [] }, { "id": 577658, "author_name": "", "author_url": "", "post_date": "2019-07-16T22:58:27.367000", "content": "", "votes": 0, "replies": [] }, { "id": 1693596, "author_name": "Dieter", "author_url": "", "post_date": "2022-02-16T19:20:06.223000", "content": "

thanks for sharing

", "votes": 0, "replies": [] }, { "id": 577743, "author_name": "", "author_url": "", "post_date": "2019-07-17T03:13:23.200000", "content": "

thanks for sharing

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "489285": "Here is the code of my solution\n[https://github.com/SeuTao/Kaggle_Whale2019_2nd_palce_solution]", "492226": "@shentao Thank you for sharing, and congratulations! Your solution is quite simple enough and strong.\n\nAnd I'd like to share with people who is using a single GPU,\nLB 0.96+ is reproducible with single GPU, by decreasing batch size to 32. It took two days.\n\nThanks again SeuTao, it's great work...", "489904": "Thanks for sharing your solution. In fact, look like your idea is very similar to the Winner (Classification + TripletLoss). ", "489439": "Congratulations on the second place and for the grandmaster rank! =]", "2371589": "Thanks for attaching pptx on GitHub, really insightful, helped me learn a lot.", "1693997": "congratulations @shentao 😎Thanks for sharing!🤗 new follower 🙋‍♀️\n", "577761": "Thanks @shentao for sharing ..and congrats for the good work", "489934": "great job，congratulations", "577658": "", "1693596": "thanks for sharing", "577743": "thanks for sharing" } }