{ "id": 319900, "title": "8th place solution [My part]", "url": "/competitions/happy-whale-and-dolphin/discussion/319900", "author_name": "", "post_date": "2022-04-19T10:27:56.041780200Z", "votes": 17, "comment_count": 7, "views": 0, "content": "

We decided to choose minimal sharing inside our team, so our solutions are different.

\n

Please check topics from my teammates:

\n
    \n
  1. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319868
    2. \n
  2. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319894
    3. \n
\n

All my models are effnet-b7, I use CosineAnnealingLR with SGD and AMSoftmax with scale=35 and margin=0.35. I've train on whole data without folds.

\n

I exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images.

\n

I made several datasets:

\n
    \n
  1. body and fin - 806 on lb
    2. \n
  2. body, fin anf fullframe - 811 on lb
    3. \n
  3. body, fin, fullframe, detic box - ~815 on lb (didnt send, trust cv)
    4. \n
\n

After merge this 3 models I got 838 on lb.

\n

After team merge we got pseudo from our ensenmble and I retrained my models with pseudo (70%).

\n

After concat 3 models without pseudo and 3 models with pseudo I have 862 on lb.

\n

Our final ensemble - 884 on lb.

\n

For new_individual I used fixed threshold 19%.

\n

Thank you @kwentar, @lenny27, @ilyadobrynin for this competition!

", "messages": [ { "id": "1760492", "postDate": "04/19/2022 10:27:56", "content": "

We decided to choose minimal sharing inside our team, so our solutions are different.

\n

Please check topics from my teammates:

\n
    \n
  1. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319868
    2. \n
  2. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319894
    3. \n
\n

All my models are effnet-b7, I use CosineAnnealingLR with SGD and AMSoftmax with scale=35 and margin=0.35. I've train on whole data without folds.

\n

I exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images.

\n

I made several datasets:

\n
    \n
  1. body and fin - 806 on lb
    2. \n
  2. body, fin anf fullframe - 811 on lb
    3. \n
  3. body, fin, fullframe, detic box - ~815 on lb (didnt send, trust cv)
    4. \n
\n

After merge this 3 models I got 838 on lb.

\n

After team merge we got pseudo from our ensenmble and I retrained my models with pseudo (70%).

\n

After concat 3 models without pseudo and 3 models with pseudo I have 862 on lb.

\n

Our final ensemble - 884 on lb.

\n

For new_individual I used fixed threshold 19%.

\n

Thank you @kwentar, @lenny27, @ilyadobrynin for this competition!

", "rawMarkdown": "We decided to choose minimal sharing inside our team, so our solutions are different.\n\nPlease check topics from my teammates:\n1. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319868\n2. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319894\n\nAll my models are effnet-b7, I use CosineAnnealingLR with SGD and AMSoftmax with scale=35 and margin=0.35. I've train on whole data without folds.\n\nI exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images. \n\nI made several datasets:\n\n1. body and fin - 806 on lb\n2. body, fin anf fullframe - 811 on lb\n3. body, fin, fullframe, detic box - ~815 on lb (didnt send, trust cv)\n\nAfter merge this 3 models I got 838 on lb.\n\nAfter team merge we got pseudo from our ensenmble and I retrained my models with pseudo (70%).\n\nAfter concat 3 models without pseudo and 3 models with pseudo I have 862 on lb.\n\nOur final ensemble - 884 on lb.\n\nFor new_individual I used fixed threshold 19%.\n\nThank you @kwentar, @lenny27, @ilyadobrynin for this competition!", "votes": null }, { "id": "1761124", "postDate": "04/19/2022 17:23:57", "content": "

Nice work! Thank you for sharing! I have a couple of questions if you don't mind.

\n", "rawMarkdown": "Nice work! Thank you for sharing! I have a couple of questions if you don't mind.\n- What was the reason for minimising sharing within the team? Do you think it benefitted you despite the risk of overlap between your models?\n- Did you use a particular method to select your threshold? It seems interestingly low in comparison to other solutions I have seen.", "votes": null }, { "id": "1761296", "postDate": "04/19/2022 19:37:51", "content": "
    \n
  1. At the merged moment we all have the strong models and when we discussed our ideas we found out that some ideas improve score for me but not others. E.g. for my models optimal embedding size was 320, for others - much bigger (4096). Other reason is the fact that ensemble boosts score if models are different, and it`s our case.
    2. \n
  2. I just check it on lb :) 19% means that new_individuals were set in top1 place, but for most of rest images new_individuals were on second place.
    3. \n
", "rawMarkdown": "1. At the merged moment we all have the strong models and when we discussed our ideas we found out that some ideas improve score for me but not others. E.g. for my models optimal embedding size was 320, for others - much bigger (4096). Other reason is the fact that ensemble boosts score if models are different, and it`s our case.\n2. I just check it on lb :) 19% means that new_individuals were set in top1 place, but for most of rest images new_individuals were on second place.", "votes": null }, { "id": "1761380", "postDate": "04/19/2022 21:02:56", "content": "

Ah ok, it makes sense! Thank you for your reply!

", "rawMarkdown": "Ah ok, it makes sense! Thank you for your reply!", "votes": null }, { "id": "1764040", "postDate": "04/22/2022 05:19:35", "content": "

Good Jobs! Thank for sharing.
\nI have a question:

\n
\n

I exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images.

\n
\n

Where do you think out this idea?

", "rawMarkdown": "Good Jobs! Thank for sharing.\nI have a question:\n> I exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images.\n\nWhere do you think out this idea?", "votes": null }, { "id": "1764319", "postDate": "04/22/2022 11:40:39", "content": "

Hi @olegshapovalov, you performed very well during this competition. Congratulations!
\nAs I can see joining datasets was really good choice! It improved score significantly. Then using pseudolabeling made one more improvement - TOP20 at least.

\n
\n

AMSoftmax

\n
\n

ArcFace Softmax?

\n
\n

one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images

\n
\n

I am trying to understand this one. Correct me if I am wrong:

\n\n

What does it mean - one embedding space for all representation of each image? I can't imagine.

", "rawMarkdown": "Hi @olegshapovalov, you performed very well during this competition. Congratulations!\nAs I can see joining datasets was really good choice! It improved score significantly. Then using pseudolabeling made one more improvement - TOP20 at least. \n\n> AMSoftmax \n\nArcFace Softmax?\n\n> one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images\n\nI am trying to understand this one. Correct me if I am wrong:\n- take one photo of individual_id1 \n- crop - how crops looks like? I am thinking about logic behind this.\n- add crops to train as a individual_id1\n\nWhat does it mean - one embedding space for all representation of each image? I can't imagine.", "votes": null }, { "id": "1771779", "postDate": "04/29/2022 15:01:15", "content": "

Just think a lot :)

", "rawMarkdown": "Just think a lot :)", "votes": null }, { "id": "1771797", "postDate": "04/29/2022 15:23:03", "content": "
\n

ArcFace Softmax?

\n
\n

No, this one https://github.com/happynear/AMSoftmax

\n
\n

crop - how crops looks like? I am thinking about logic behind this.

\n
\n

For example, I have image.jpg with individual_id1. I will add to my dataset:

\n
image.jpg, individual_id1 (full frame)\nbody.jpg, individual_id1 (body crop)\nfin.jpg, individual_id1 (fin crop)\ndetc.jpg (detic.crop)\n
", "rawMarkdown": "> ArcFace Softmax?\n\nNo, this one https://github.com/happynear/AMSoftmax\n\n> crop - how crops looks like? I am thinking about logic behind this.\n\nFor example, I have image.jpg with individual_id1. I will add to my dataset: \n```\nimage.jpg, individual_id1 (full frame)\nbody.jpg, individual_id1 (body crop)\nfin.jpg, individual_id1 (fin crop)\ndetc.jpg (detic.crop)\n```", "votes": null } ], "comments": [ { "id": 1761124, "author_name": "frlemarchand", "author_url": "", "post_date": "04/19/2022 17:23:57", "content": "

Nice work! Thank you for sharing! I have a couple of questions if you don't mind.

\n", "votes": null, "replies": [ { "id": 1761296, "author_name": "olegshapovalov", "author_url": "", "post_date": "04/19/2022 19:37:51", "content": "
    \n
  1. At the merged moment we all have the strong models and when we discussed our ideas we found out that some ideas improve score for me but not others. E.g. for my models optimal embedding size was 320, for others - much bigger (4096). Other reason is the fact that ensemble boosts score if models are different, and it`s our case.
    2. \n
  2. I just check it on lb :) 19% means that new_individuals were set in top1 place, but for most of rest images new_individuals were on second place.
    3. \n
", "votes": null, "replies": [] }, { "id": 1761380, "author_name": "frlemarchand", "author_url": "", "post_date": "04/19/2022 21:02:56", "content": "

Ah ok, it makes sense! Thank you for your reply!

", "votes": null, "replies": [] } ] }, { "id": 1764040, "author_name": "phanttan", "author_url": "", "post_date": "04/22/2022 05:19:35", "content": "

Good Jobs! Thank for sharing.
\nI have a question:

\n
\n

I exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images.

\n
\n

Where do you think out this idea?

", "votes": null, "replies": [ { "id": 1771779, "author_name": "olegshapovalov", "author_url": "", "post_date": "04/29/2022 15:01:15", "content": "

Just think a lot :)

", "votes": null, "replies": [] } ] }, { "id": 1764319, "author_name": "remekkinas", "author_url": "", "post_date": "04/22/2022 11:40:39", "content": "

Hi @olegshapovalov, you performed very well during this competition. Congratulations!
\nAs I can see joining datasets was really good choice! It improved score significantly. Then using pseudolabeling made one more improvement - TOP20 at least.

\n
\n

AMSoftmax

\n
\n

ArcFace Softmax?

\n
\n

one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images

\n
\n

I am trying to understand this one. Correct me if I am wrong:

\n\n

What does it mean - one embedding space for all representation of each image? I can't imagine.

", "votes": null, "replies": [ { "id": 1771797, "author_name": "olegshapovalov", "author_url": "", "post_date": "04/29/2022 15:23:03", "content": "
\n

ArcFace Softmax?

\n
\n

No, this one https://github.com/happynear/AMSoftmax

\n
\n

crop - how crops looks like? I am thinking about logic behind this.

\n
\n

For example, I have image.jpg with individual_id1. I will add to my dataset:

\n
image.jpg, individual_id1 (full frame)\nbody.jpg, individual_id1 (body crop)\nfin.jpg, individual_id1 (fin crop)\ndetc.jpg (detic.crop)\n
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1760492": "We decided to choose minimal sharing inside our team, so our solutions are different.\n\nPlease check topics from my teammates:\n1. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319868\n2. https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319894\n\nAll my models are effnet-b7, I use CosineAnnealingLR with SGD and AMSoftmax with scale=35 and margin=0.35. I've train on whole data without folds.\n\nI exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images. \n\nI made several datasets:\n\n1. body and fin - 806 on lb\n2. body, fin anf fullframe - 811 on lb\n3. body, fin, fullframe, detic box - ~815 on lb (didnt send, trust cv)\n\nAfter merge this 3 models I got 838 on lb.\n\nAfter team merge we got pseudo from our ensenmble and I retrained my models with pseudo (70%).\n\nAfter concat 3 models without pseudo and 3 models with pseudo I have 862 on lb.\n\nOur final ensemble - 884 on lb.\n\nFor new_individual I used fixed threshold 19%.\n\nThank you @kwentar, @lenny27, @ilyadobrynin for this competition!", "1761124": "Nice work! Thank you for sharing! I have a couple of questions if you don't mind.\n- What was the reason for minimising sharing within the team? Do you think it benefitted you despite the risk of overlap between your models?\n- Did you use a particular method to select your threshold? It seems interestingly low in comparison to other solutions I have seen.", "1761296": "1. At the merged moment we all have the strong models and when we discussed our ideas we found out that some ideas improve score for me but not others. E.g. for my models optimal embedding size was 320, for others - much bigger (4096). Other reason is the fact that ensemble boosts score if models are different, and it`s our case.\n2. I just check it on lb :) 19% means that new_individuals were set in top1 place, but for most of rest images new_individuals were on second place.", "1761380": "Ah ok, it makes sense! Thank you for your reply!", "1764040": "Good Jobs! Thank for sharing.\nI have a question:\n> I exploit only one key idea: one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images.\n\nWhere do you think out this idea?", "1764319": "Hi @olegshapovalov, you performed very well during this competition. Congratulations!\nAs I can see joining datasets was really good choice! It improved score significantly. Then using pseudolabeling made one more improvement - TOP20 at least. \n\n> AMSoftmax \n\nArcFace Softmax?\n\n> one embedding space for all representaion of each image. It means I take several crops for each image and just add them as new images\n\nI am trying to understand this one. Correct me if I am wrong:\n- take one photo of individual_id1 \n- crop - how crops looks like? I am thinking about logic behind this.\n- add crops to train as a individual_id1\n\nWhat does it mean - one embedding space for all representation of each image? I can't imagine.", "1771779": "Just think a lot :)", "1771797": "> ArcFace Softmax?\n\nNo, this one https://github.com/happynear/AMSoftmax\n\n> crop - how crops looks like? I am thinking about logic behind this.\n\nFor example, I have image.jpg with individual_id1. I will add to my dataset: \n```\nimage.jpg, individual_id1 (full frame)\nbody.jpg, individual_id1 (body crop)\nfin.jpg, individual_id1 (fin crop)\ndetc.jpg (detic.crop)\n```" }, "source": "meta" }