{ "id": 319851, "title": "My solution - 29th rank [part]", "url": "/competitions/happy-whale-and-dolphin/writeups/all-the-best-my-solution-29th-rank-part", "author_name": "", "post_date": "2022-04-20T15:05:42.130Z", "votes": 25, "comment_count": 7, "views": 0, "content": "

Rank: 34

\n

Image Size

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelTrain Image sizeInfer Image size
B59601056
B6768840
B7600660
\n

Augmentations:

\n
    image = tf.image.random_flip_left_right(image)\n    image = tf.image.random_hue(image, 0.1)\n    image = tf.image.random_saturation(image, 0.80, 1.20)\n    image = tf.image.random_contrast(image, 0.80, 1.20)\n    image = tf.image.random_brightness(image, 0.10)\n
\n

Architecture

\n

Encoder {B5/B6/B7} -> GAP {Global Avg pooling} -> Batchnorm -> Multi-SampleDropout

\n

The output of Multi-SampleDropout is fed into two Arcface Classification heads
\n 1. Individual Classification head,
\n 2. Species Classification head

\n
individual_margin = head(n_classes = config.N_CLASSES, s = 30, m = 0.3, name=f'head_individual/{config.head}', dtype='float32')\n        species_margin =  head(n_classes = config.N_SPECIES, s = 30, m = 0.3, name=f'head_species/{config.head}', dtype='float32')\n
\n

Training Settings

\n
    \n
  1. 40 epochs for first stage
    2. \n
  2. 10 epochs with pseudo labels
    \nUse multiple models, find top-1 examples that the majority agree on, ensuring 0.2 separation between confidence of 1st class and 2nd class. Generated 12157 pseudo labels
    3. \n
  3. Embedding dimension: 4096
    4. \n
  4. Freeze Batchnorm
    5. \n
  5. Use 4 different datasets
    \na. Full body
    \nb. Tracer + Full body {Use tracer for most cases. if tracer bbox not present, use fullbody bbox}
    \nc. backfin {all the bbox}
    \nd. backfin {bbox after cleanup as suggested by the @bre}
    6. \n
  6. Lookahead + Adam
    7. \n
\n

Thanks
\nThanks to datasets contributors
\n@adnanpen
\n@jpbremer

\n

Teammates
\n@lextoumbourou
\n@zekunn
\n@poorneshwaran

\n

We were able to get a good ranking due to good support and hard work from my teammates. Thanks a lot. Let's catch up on some other challenges and do better than this.

\n

Notes
\nCheck out the notebook shared by @lextoumbourou
\nhttps://www.kaggle.com/code/lextoumbourou/happywhale-tpu-baseline-to-0-804-elasticface/notebook

", "messages": [ { "id": "1760208", "postDate": "04/19/2022 06:08:49", "content": "

Rank: 34

\n

Image Size

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelTrain Image sizeInfer Image size
B59601056
B6768840
B7600660
\n

Augmentations:

\n
    image = tf.image.random_flip_left_right(image)\n    image = tf.image.random_hue(image, 0.1)\n    image = tf.image.random_saturation(image, 0.80, 1.20)\n    image = tf.image.random_contrast(image, 0.80, 1.20)\n    image = tf.image.random_brightness(image, 0.10)\n
\n

Architecture

\n

Encoder {B5/B6/B7} -> GAP {Global Avg pooling} -> Batchnorm -> Multi-SampleDropout

\n

The output of Multi-SampleDropout is fed into two Arcface Classification heads
\n 1. Individual Classification head,
\n 2. Species Classification head

\n
individual_margin = head(n_classes = config.N_CLASSES, s = 30, m = 0.3, name=f'head_individual/{config.head}', dtype='float32')\n        species_margin =  head(n_classes = config.N_SPECIES, s = 30, m = 0.3, name=f'head_species/{config.head}', dtype='float32')\n
\n

Training Settings

\n
    \n
  1. 40 epochs for first stage
    2. \n
  2. 10 epochs with pseudo labels
    \nUse multiple models, find top-1 examples that the majority agree on, ensuring 0.2 separation between confidence of 1st class and 2nd class. Generated 12157 pseudo labels
    3. \n
  3. Embedding dimension: 4096
    4. \n
  4. Freeze Batchnorm
    5. \n
  5. Use 4 different datasets
    \na. Full body
    \nb. Tracer + Full body {Use tracer for most cases. if tracer bbox not present, use fullbody bbox}
    \nc. backfin {all the bbox}
    \nd. backfin {bbox after cleanup as suggested by the @bre}
    6. \n
  6. Lookahead + Adam
    7. \n
\n

Thanks
\nThanks to datasets contributors
\n@adnanpen
\n@jpbremer

\n

Teammates
\n@lextoumbourou
\n@zekunn
\n@poorneshwaran

\n

We were able to get a good ranking due to good support and hard work from my teammates. Thanks a lot. Let's catch up on some other challenges and do better than this.

\n

Notes
\nCheck out the notebook shared by @lextoumbourou
\nhttps://www.kaggle.com/code/lextoumbourou/happywhale-tpu-baseline-to-0-804-elasticface/notebook

", "rawMarkdown": "Rank: 34 \n\n**Image Size**\n|Model | Train Image size | Infer Image size |\n| --- | --- | --- |\n| B5 | 960 | 1056|\n| B6 | 768 | 840|\n| B7 | 600 | 660|\n\n**Augmentations:**\n\n ```\n image = tf.image.random_flip_left_right(image)\n image = tf.image.random_hue(image, 0.1)\n image = tf.image.random_saturation(image, 0.80, 1.20)\n image = tf.image.random_contrast(image, 0.80, 1.20)\n image = tf.image.random_brightness(image, 0.10)\n```\n\n**Architecture**\n\nEncoder {B5/B6/B7} -> GAP {Global Avg pooling} -> Batchnorm -> Multi-SampleDropout \n\nThe output of Multi-SampleDropout is fed into two Arcface Classification heads \n 1. Individual Classification head,\n 2. Species Classification head\n\n```\nindividual_margin = head(n_classes = config.N_CLASSES, s = 30, m = 0.3, name=f'head_individual/{config.head}', dtype='float32')\n species_margin = head(n_classes = config.N_SPECIES, s = 30, m = 0.3, name=f'head_species/{config.head}', dtype='float32')\n```\n\n**Training Settings**\n1. 40 epochs for first stage\n2. 10 epochs with pseudo labels\n`Use multiple models, find top-1 examples that the majority agree on, ensuring 0.2 separation between confidence of 1st class and 2nd class. Generated 12157 pseudo labels`\n3. Embedding dimension: 4096\n4. Freeze Batchnorm\n5. Use 4 different datasets\n a. Full body\n b. Tracer + Full body {Use tracer for most cases. if tracer bbox not present, use fullbody bbox}\n c. backfin {all the bbox}\n d. backfin {bbox after cleanup as suggested by the @bre}\n6. Lookahead + Adam\n\n**Thanks**\nThanks to datasets contributors\n@adnanpen \n@jpbremer \n\nTeammates\n@lextoumbourou \n@zekunn \n@poorneshwaran \n\nWe were able to get a good ranking due to good support and hard work from my teammates. Thanks a lot. Let's catch up on some other challenges and do better than this.\n\n**Notes**\nCheck out the notebook shared by @lextoumbourou\nhttps://www.kaggle.com/code/lextoumbourou/happywhale-tpu-baseline-to-0-804-elasticface/notebook", "votes": null }, { "id": "1760309", "postDate": "04/19/2022 07:49:55", "content": "

Was great teaming up with you! I learned so much 😃

", "rawMarkdown": "Was great teaming up with you! I learned so much 😃", "votes": null }, { "id": "1760448", "postDate": "04/19/2022 09:55:09", "content": "

Congratulations Balaji for attaining 34th ranks I tried this competition but its not accepting new entries now …

", "rawMarkdown": "Congratulations Balaji for attaining 34th ranks I tried this competition but its not accepting new entries now ...", "votes": null }, { "id": "1761153", "postDate": "04/19/2022 17:49:34", "content": "

Nice work and congratulations! I remember you said \"> After updating model structure and changing augmentation schemes, performance increased a lot\" from around 0.7 --. 0.786. Could you specify what changing did you make? Thanks!

", "rawMarkdown": "Nice work and congratulations! I remember you said \"> After updating model structure and changing augmentation schemes, performance increased a lot\" from around 0.7 --. 0.786. Could you specify what changing did you make? Thanks!", "votes": null }, { "id": "1761869", "postDate": "04/20/2022 08:50:52", "content": "

@leonshangguan

\n

You can find the augmentations that I used. Changing hue was one of the main boosters.
\nIntroduced the Batch norm and Multi-sample drop out before the classification heads.

", "rawMarkdown": "leonshangguan \n\nYou can find the augmentations that I used. Changing hue was one of the main boosters. \nIntroduced the Batch norm and Multi-sample drop out before the classification heads.", "votes": null }, { "id": "1762472", "postDate": "04/20/2022 17:52:05", "content": "

Great job! It's nice to see a solution in between baseline and gold ranked, and the explanation is really clean and understandable. Out of curiosity, could you share what computing resources did you use (i.e. Kaggle, Colab/Pro, GCS, etc.)? Seeing how large your training and inference image sizes are, I'm interested in your pipeline.

", "rawMarkdown": "Great job! It's nice to see a solution in between baseline and gold ranked, and the explanation is really clean and understandable. Out of curiosity, could you share what computing resources did you use (i.e. Kaggle, Colab/Pro, GCS, etc.)? Seeing how large your training and inference image sizes are, I'm interested in your pipeline.", "votes": null }, { "id": "1763047", "postDate": "04/21/2022 08:30:29", "content": "

Colab Pro. I reduced batch size a lot

", "rawMarkdown": "Colab Pro. I reduced batch size a lot", "votes": null }, { "id": "1763251", "postDate": "04/21/2022 11:38:36", "content": "

@dhakshiin1601 thank you for your solution description. I checked BN freeze implementation. We had the same. Oryginal implementation did not work. However multi-dropout you have different implementation. My did not improve score … (so I made something wrong :)) Thank you for providing your implementation. I will retrain my model using your multi-dropout.

", "rawMarkdown": "dhakshiin1601 thank you for your solution description. I checked BN freeze implementation. We had the same. Oryginal implementation did not work. However multi-dropout you have different implementation. My did not improve score ... (so I made something wrong :)) Thank you for providing your implementation. I will retrain my model using your multi-dropout.", "votes": null } ], "comments": [ { "id": 1760309, "author_name": "lextoumbourou", "author_url": "", "post_date": "04/19/2022 07:49:55", "content": "

Was great teaming up with you! I learned so much 😃

", "votes": null, "replies": [] }, { "id": 1760448, "author_name": "vidiqlogy", "author_url": "", "post_date": "04/19/2022 09:55:09", "content": "

Congratulations Balaji for attaining 34th ranks I tried this competition but its not accepting new entries now …

", "votes": null, "replies": [] }, { "id": 1761153, "author_name": "leonshangguan", "author_url": "", "post_date": "04/19/2022 17:49:34", "content": "

Nice work and congratulations! I remember you said \"> After updating model structure and changing augmentation schemes, performance increased a lot\" from around 0.7 --. 0.786. Could you specify what changing did you make? Thanks!

", "votes": null, "replies": [ { "id": 1761869, "author_name": "dhakshiin1601", "author_url": "", "post_date": "04/20/2022 08:50:52", "content": "

@leonshangguan

\n

You can find the augmentations that I used. Changing hue was one of the main boosters.
\nIntroduced the Batch norm and Multi-sample drop out before the classification heads.

", "votes": null, "replies": [] } ] }, { "id": 1762472, "author_name": "anandparthiban", "author_url": "", "post_date": "04/20/2022 17:52:05", "content": "

Great job! It's nice to see a solution in between baseline and gold ranked, and the explanation is really clean and understandable. Out of curiosity, could you share what computing resources did you use (i.e. Kaggle, Colab/Pro, GCS, etc.)? Seeing how large your training and inference image sizes are, I'm interested in your pipeline.

", "votes": null, "replies": [ { "id": 1763047, "author_name": "dhakshiin1601", "author_url": "", "post_date": "04/21/2022 08:30:29", "content": "

Colab Pro. I reduced batch size a lot

", "votes": null, "replies": [] } ] }, { "id": 1763251, "author_name": "remekkinas", "author_url": "", "post_date": "04/21/2022 11:38:36", "content": "

@dhakshiin1601 thank you for your solution description. I checked BN freeze implementation. We had the same. Oryginal implementation did not work. However multi-dropout you have different implementation. My did not improve score … (so I made something wrong :)) Thank you for providing your implementation. I will retrain my model using your multi-dropout.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1760208": "Rank: 34 \n\n**Image Size**\n|Model | Train Image size | Infer Image size |\n| --- | --- | --- |\n| B5 | 960 | 1056|\n| B6 | 768 | 840|\n| B7 | 600 | 660|\n\n**Augmentations:**\n\n ```\n image = tf.image.random_flip_left_right(image)\n image = tf.image.random_hue(image, 0.1)\n image = tf.image.random_saturation(image, 0.80, 1.20)\n image = tf.image.random_contrast(image, 0.80, 1.20)\n image = tf.image.random_brightness(image, 0.10)\n```\n\n**Architecture**\n\nEncoder {B5/B6/B7} -> GAP {Global Avg pooling} -> Batchnorm -> Multi-SampleDropout \n\nThe output of Multi-SampleDropout is fed into two Arcface Classification heads \n 1. Individual Classification head,\n 2. Species Classification head\n\n```\nindividual_margin = head(n_classes = config.N_CLASSES, s = 30, m = 0.3, name=f'head_individual/{config.head}', dtype='float32')\n species_margin = head(n_classes = config.N_SPECIES, s = 30, m = 0.3, name=f'head_species/{config.head}', dtype='float32')\n```\n\n**Training Settings**\n1. 40 epochs for first stage\n2. 10 epochs with pseudo labels\n`Use multiple models, find top-1 examples that the majority agree on, ensuring 0.2 separation between confidence of 1st class and 2nd class. Generated 12157 pseudo labels`\n3. Embedding dimension: 4096\n4. Freeze Batchnorm\n5. Use 4 different datasets\n a. Full body\n b. Tracer + Full body {Use tracer for most cases. if tracer bbox not present, use fullbody bbox}\n c. backfin {all the bbox}\n d. backfin {bbox after cleanup as suggested by the @bre}\n6. Lookahead + Adam\n\n**Thanks**\nThanks to datasets contributors\n@adnanpen \n@jpbremer \n\nTeammates\n@lextoumbourou \n@zekunn \n@poorneshwaran \n\nWe were able to get a good ranking due to good support and hard work from my teammates. Thanks a lot. Let's catch up on some other challenges and do better than this.\n\n**Notes**\nCheck out the notebook shared by @lextoumbourou\nhttps://www.kaggle.com/code/lextoumbourou/happywhale-tpu-baseline-to-0-804-elasticface/notebook", "1760309": "Was great teaming up with you! I learned so much 😃", "1760448": "Congratulations Balaji for attaining 34th ranks I tried this competition but its not accepting new entries now ...", "1761153": "Nice work and congratulations! I remember you said \"> After updating model structure and changing augmentation schemes, performance increased a lot\" from around 0.7 --. 0.786. Could you specify what changing did you make? Thanks!", "1761869": "leonshangguan \n\nYou can find the augmentations that I used. Changing hue was one of the main boosters. \nIntroduced the Batch norm and Multi-sample drop out before the classification heads.", "1762472": "Great job! It's nice to see a solution in between baseline and gold ranked, and the explanation is really clean and understandable. Out of curiosity, could you share what computing resources did you use (i.e. Kaggle, Colab/Pro, GCS, etc.)? Seeing how large your training and inference image sizes are, I'm interested in your pipeline.", "1763047": "Colab Pro. I reduced batch size a lot", "1763251": "dhakshiin1601 thank you for your solution description. I checked BN freeze implementation. We had the same. Oryginal implementation did not work. However multi-dropout you have different implementation. My did not improve score ... (so I made something wrong :)) Thank you for providing your implementation. I will retrain my model using your multi-dropout." }, "source": "meta" }