So before jumping into some massive models networks i wanted to reimplement in pytorch some of the solutions there are so far (efficient net + arc face tpu notebook for instance) but on a smaller scale. to easily train on one rtx3090.
\nIm bit suprised bc using b0-b1 + Arcface and a similar scheme for choosing the thresholds as the TPU notebooks, the most im getting is ~0.3LB. Im a bit suprised that model size itself has such a big influence. Or maybe i have some bugs, anyone trained something similar and wants to share their scores and findings?
", "messages": [ { "id": "1685723", "postDate": "02/11/2022 13:56:57", "content": "So before jumping into some massive models networks i wanted to reimplement in pytorch some of the solutions there are so far (efficient net + arc face tpu notebook for instance) but on a smaller scale. to easily train on one rtx3090.
\nIm bit suprised bc using b0-b1 + Arcface and a similar scheme for choosing the thresholds as the TPU notebooks, the most im getting is ~0.3LB. Im a bit suprised that model size itself has such a big influence. Or maybe i have some bugs, anyone trained something similar and wants to share their scores and findings?
", "rawMarkdown": "So before jumping into some massive models networks i wanted to reimplement in pytorch some of the solutions there are so far (efficient net + arc face tpu notebook for instance) but on a smaller scale. to easily train on one rtx3090.\n\nIm bit suprised bc using b0-b1 + Arcface and a similar scheme for choosing the thresholds as the TPU notebooks, the most im getting is ~0.3LB. Im a bit suprised that model size itself has such a big influence. Or maybe i have some bugs, anyone trained something similar and wants to share their scores and findings?", "votes": null }, { "id": "1685963", "postDate": "02/11/2022 16:49:24", "content": "I guess one of your problem is image size, if you have look at data, you will some images like 00b773df4a863d.jpg, where the object is only 5-10% of image, it has around 0 info at 224 scale
Well I have similar results ~0.38LB with 224x224 images and efficientnet B0
", "rawMarkdown": "Well I have similar results ~0.38LB with 224x224 images and efficientnet B0", "votes": null }, { "id": "1686193", "postDate": "02/11/2022 20:52:39", "content": "0.38 vs 0.30 is a huge diff;) and a result i would feel ok with, given small model and small image sizes. Theo did you do anything unusual to achieve that? @wolfy73
", "rawMarkdown": "0.38 vs 0.30 is a huge diff;) and a result i would feel ok with, given small model and small image sizes. Theo did you do anything unusual to achieve that? @wolfy73", "votes": null }, { "id": "1690715", "postDate": "02/15/2022 04:25:55", "content": "Thanks for sharing your insights. If I may ask, are you directly tried to cluster individual ids with arcface?
", "rawMarkdown": "Thanks for sharing your insights. If I may ask, are you directly tried to cluster individual ids with arcface?", "votes": null }, { "id": "1693250", "postDate": "02/16/2022 14:47:55", "content": "I got similar score (0.300 LB), effnet_b0, image size 256, Arcface, no augmentations other than normalising, and the model is overfitting badly :/
", "rawMarkdown": "I got similar score (0.300 LB), effnet_b0, image size 256, Arcface, no augmentations other than normalising, and the model is overfitting badly :/", "votes": null } ], "comments": [ { "id": 1685963, "author_name": "kwentar", "author_url": "", "post_date": "02/11/2022 16:49:24", "content": "I guess one of your problem is image size, if you have look at data, you will some images like 00b773df4a863d.jpg, where the object is only 5-10% of image, it has around 0 info at 224 scale
Well I have similar results ~0.38LB with 224x224 images and efficientnet B0
", "votes": null, "replies": [] }, { "id": 1686193, "author_name": "crimzoid", "author_url": "", "post_date": "02/11/2022 20:52:39", "content": "0.38 vs 0.30 is a huge diff;) and a result i would feel ok with, given small model and small image sizes. Theo did you do anything unusual to achieve that? @wolfy73
", "votes": null, "replies": [] }, { "id": 1690715, "author_name": "bsridatta", "author_url": "", "post_date": "02/15/2022 04:25:55", "content": "Thanks for sharing your insights. If I may ask, are you directly tried to cluster individual ids with arcface?
", "votes": null, "replies": [] }, { "id": 1693250, "author_name": "atharvaingle", "author_url": "", "post_date": "02/16/2022 14:47:55", "content": "I got similar score (0.300 LB), effnet_b0, image size 256, Arcface, no augmentations other than normalising, and the model is overfitting badly :/
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1685723": "So before jumping into some massive models networks i wanted to reimplement in pytorch some of the solutions there are so far (efficient net + arc face tpu notebook for instance) but on a smaller scale. to easily train on one rtx3090.\n\nIm bit suprised bc using b0-b1 + Arcface and a similar scheme for choosing the thresholds as the TPU notebooks, the most im getting is ~0.3LB. Im a bit suprised that model size itself has such a big influence. Or maybe i have some bugs, anyone trained something similar and wants to share their scores and findings?", "1685963": "I guess one of your problem is image size, if you have look at data, you will some images like `00b773df4a863d.jpg`, where the object is only 5-10% of image, it has around 0 info at 224 scale", "1686128": "Well I have similar results ~0.38LB with 224x224 images and efficientnet B0", "1686193": "0.38 vs 0.30 is a huge diff;) and a result i would feel ok with, given small model and small image sizes. Theo did you do anything unusual to achieve that? @wolfy73", "1690715": "Thanks for sharing your insights. If I may ask, are you directly tried to cluster individual ids with arcface?", "1693250": "I got similar score (0.300 LB), effnet_b0, image size 256, Arcface, no augmentations other than normalising, and the model is overfitting badly :/" }, "source": "meta" }