{
  "id": 311033,
  "title": "Recap of the Top Solutions from the Previous Whale Identification Competition",
  "url": "/competitions/happy-whale-and-dolphin/discussion/311033",
  "author_name": "",
  "post_date": "2022-03-04T13:19:45.684091100Z",
  "votes": 46,
  "comment_count": 4,
  "views": 0,
  "content": "<p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/overview\" target=\"_blank\">https://www.kaggle.com/c/humpback-whale-identification/overview</a></p>\n<ul>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82366\" target=\"_blank\">1st Place Solution</a></p>\n<ul>\n<li>Input size is (512, 256)</li>\n<li>Use 4 channels, RGB + masks (trained by 450 open source labels) as input.</li>\n<li>Step 1: Training within all labels with &gt;10 samples.</li>\n<li>Step 2: Training with all samples, and fixed all of the networks except the last two layers.</li>\n<li>Flip images (+0.006) and consider flipped id-whales as different whales and keep new whales as the same.</li>\n<li>Pseudo labels (+ 0.001). Added around 2000 test images (with confidence &gt; 0.96) into the training set.</li>\n<li>Class balance (+0.001 ~ 0.002). For top 5 predictions class1 to class5, if: conf class1 – conf class 2 &lt; 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, switch class1 and class2’s positions.</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/83885\" target=\"_blank\">2nd Place Solution</a></p>\n<ul>\n<li><a href=\"https://github.com/SeuTao/Humpback-Whale-Identification-Challenge-2019_2nd_palce_solution\" target=\"_blank\">https://github.com/SeuTao/Humpback-Whale-Identification-Challenge-2019_2nd_palce_solution</a></li>\n<li>Input: 256x512 or 512*512 cropped images</li>\n<li>Backbone: resnet101, seresnet101, seresnext101</li>\n<li>Loss function: arcface loss + triplet loss + focal loss</li>\n<li>Optimizer: adam with warm up lr strategy</li>\n<li>Augmentation: blur,grayscale,noise,shear,rotate,perspective transform</li>\n<li>Horizontal flip to create more ids -&gt; 5004*2</li>\n<li>Pseudo Labeling</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82484\" target=\"_blank\">3rd Place Solution</a></p>\n<ul>\n<li>ArcFace - the layers after last convolution were replaced to flattening -&gt; BN -&gt; dropout -&gt; FC -&gt; BN.</li>\n<li>densenet121</li>\n<li>Hyperparameters: m 0.5 (the default value of the paper), weight decay 0.0005, droupout 0.5</li>\n<li>Augmentation: average blur, motion blur, add, multiply, grayscale, scale, translate, shear, rotate, align or no-align</li>\n<li>adam optimizer - learning rate of 0.00025 -&gt; 0.000125 -&gt; 0.0000625</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82356\" target=\"_blank\">4th Place Solution</a></p>\n<ul>\n<li>Keypoint matching (SIFT, ROOTSIFT, and a host of binary descriptors and matchers)<ul>\n<li>Loop through all test/train pairs</li>\n<li>Match keypoints using faiss</li>\n<li>Double homography filtering of keypoints (LMEDS followed by RANSAC)</li>\n<li>xgboost prediction to validate homography matrix</li>\n<li>if # of matches &gt; threshold, then use prediction</li></ul></li>\n<li>Siamese Network<ul>\n<li>train classification on top 200 classes</li>\n<li>fine-tune on all classes where N&gt;8 (~576 classes)</li>\n<li>fine-tune on all classes</li>\n<li>fine-tune on all classes + mixup + image size 384x384</li></ul></li>\n<li>Postprocessing<ul>\n<li>Suppression of the dominant top-5 predictions by transposing the resulting prediction matrix from the Siamese network and determining the threshold along the train axis where n=1.</li></ul></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82369\" target=\"_blank\">5th Place Solution</a></p>\n<ul>\n<li><a href=\"https://github.com/aaxwaz/Humpback-whale-identification-challenge\" target=\"_blank\">https://github.com/aaxwaz/Humpback-whale-identification-challenge</a></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82352\" target=\"_blank\">7th Place Solution</a></p>\n<ul>\n<li>Part 1<ul>\n<li>Resnet50 -&gt; global concat (max, avg) pool -&gt; BN-&gt;Dropout-&gt; Linear(2048) -&gt; ReLU-&gt; BN-&gt;Dropout -&gt;clf(5004).</li>\n<li>Head is trained for X epochs, then all network is fine-tuned on for Y epochs.</li>\n<li>Validation set is 1 photo per whale for all whales with ≥2 photos + 1000 new_whales.</li></ul></li>\n<li>Part 2<ul>\n<li>Training on RGB images: 256x256, 384x384, 448x448, 360x720.</li>\n<li>Augmentations: random erasing, affine transformations(scale, translation, shear), brightness/contrast.</li>\n<li>Resnet34, resnet50, resnet101, densenet121, densenet162, seresnext50 — backbone architectures, followed by GeM pooling layer +L2 + multiplier.</li>\n<li>Loss: hard triplet loss</li></ul></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82427\" target=\"_blank\">9th Place Solution</a></p>\n<ul>\n<li>Train cosface or arcface net. Took the embeddings and calculate cosine similarity between the train and test images. Then average similarities for each class in train and took 5 most similar.</li>\n<li>resnet34, bninception and densenet121</li>\n<li>Final models after initial 64 epochs on 256x256 images, increase image size up to 1024 for resnet34, up to 512 for bninception and up to 640 for densenet121 and train for 64 epochs more.</li>\n<li>Augmentation: HorizontalFlip, rotate with 16 degree limit, ShiftScaleRotate with 16 degree limit, RandomBrightnessContrast, RandomGamma, Blur, Perspective transform: tile left, right and corner, Shear, MotionBlur,. GridDistortion, ElasticTransform, Cutout</li>\n<li>Cosface and Arcface parameters was optimised. Cosface: S = 32.0, M=0.35. Arcface: M1 = 1.0, M2 = 0.4, M3 = 0.15.</li>\n<li>Add test images with pseudo labels into train.</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82430\" target=\"_blank\">10th Place Solution</a></p>\n<ul>\n<li>Part 1<ul>\n<li>Siamese architecture<ul>\n<li>trained progressively 299-&gt;384-&gt;512.</li>\n<li>ResNet-18, ResNet-34, SE-ResNeXt-50, ResNet-50</li>\n<li>a smart flipping strategy, which makes the model differentiate between the left and the right part of the fluke</li></ul></li>\n<li>Classification on features: concatenated all of the features generated by branch models and trained classification model on top of them. The head of classification was two dense layers with a little dropout. </li>\n<li>Large blend for new whale/not new whale binary classification</li></ul></li>\n<li>Part 2<ul>\n<li>metric learning with multiple branches and margin loss, trained on multiple resolution crops using bboxes, grayscale and RGB input images</li></ul></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82361\" target=\"_blank\">15th Place Solution</a></p>\n<ul>\n<li>Sphereface (seresnext-50(multi-layer fusion, 384x384), resnext50(multi-layer fusion, 384x384))</li>\n<li>Oversample images in 5004 class that occur less than 20 to 20. Images are resized to 384 and 512 as input. </li>\n<li>Augmentation: randompadding + randomcrop</li>\n<li>A self-designed keypoint detector on 1000 Hand-Annotated Humpback Whale Fluke Keypoints dataset. Then affine the image to prefined coordinate using the keypoints by learning a transfer matrix</li></ul></li>\n</ul>",
  "messages": [
    {
      "id": "1711917",
      "postDate": "03/04/2022 13:19:45",
      "content": "<p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/overview\" target=\"_blank\">https://www.kaggle.com/c/humpback-whale-identification/overview</a></p>\n<ul>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82366\" target=\"_blank\">1st Place Solution</a></p>\n<ul>\n<li>Input size is (512, 256)</li>\n<li>Use 4 channels, RGB + masks (trained by 450 open source labels) as input.</li>\n<li>Step 1: Training within all labels with &gt;10 samples.</li>\n<li>Step 2: Training with all samples, and fixed all of the networks except the last two layers.</li>\n<li>Flip images (+0.006) and consider flipped id-whales as different whales and keep new whales as the same.</li>\n<li>Pseudo labels (+ 0.001). Added around 2000 test images (with confidence &gt; 0.96) into the training set.</li>\n<li>Class balance (+0.001 ~ 0.002). For top 5 predictions class1 to class5, if: conf class1 – conf class 2 &lt; 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, switch class1 and class2’s positions.</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/83885\" target=\"_blank\">2nd Place Solution</a></p>\n<ul>\n<li><a href=\"https://github.com/SeuTao/Humpback-Whale-Identification-Challenge-2019_2nd_palce_solution\" target=\"_blank\">https://github.com/SeuTao/Humpback-Whale-Identification-Challenge-2019_2nd_palce_solution</a></li>\n<li>Input: 256x512 or 512*512 cropped images</li>\n<li>Backbone: resnet101, seresnet101, seresnext101</li>\n<li>Loss function: arcface loss + triplet loss + focal loss</li>\n<li>Optimizer: adam with warm up lr strategy</li>\n<li>Augmentation: blur,grayscale,noise,shear,rotate,perspective transform</li>\n<li>Horizontal flip to create more ids -&gt; 5004*2</li>\n<li>Pseudo Labeling</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82484\" target=\"_blank\">3rd Place Solution</a></p>\n<ul>\n<li>ArcFace - the layers after last convolution were replaced to flattening -&gt; BN -&gt; dropout -&gt; FC -&gt; BN.</li>\n<li>densenet121</li>\n<li>Hyperparameters: m 0.5 (the default value of the paper), weight decay 0.0005, droupout 0.5</li>\n<li>Augmentation: average blur, motion blur, add, multiply, grayscale, scale, translate, shear, rotate, align or no-align</li>\n<li>adam optimizer - learning rate of 0.00025 -&gt; 0.000125 -&gt; 0.0000625</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82356\" target=\"_blank\">4th Place Solution</a></p>\n<ul>\n<li>Keypoint matching (SIFT, ROOTSIFT, and a host of binary descriptors and matchers)<ul>\n<li>Loop through all test/train pairs</li>\n<li>Match keypoints using faiss</li>\n<li>Double homography filtering of keypoints (LMEDS followed by RANSAC)</li>\n<li>xgboost prediction to validate homography matrix</li>\n<li>if # of matches &gt; threshold, then use prediction</li></ul></li>\n<li>Siamese Network<ul>\n<li>train classification on top 200 classes</li>\n<li>fine-tune on all classes where N&gt;8 (~576 classes)</li>\n<li>fine-tune on all classes</li>\n<li>fine-tune on all classes + mixup + image size 384x384</li></ul></li>\n<li>Postprocessing<ul>\n<li>Suppression of the dominant top-5 predictions by transposing the resulting prediction matrix from the Siamese network and determining the threshold along the train axis where n=1.</li></ul></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82369\" target=\"_blank\">5th Place Solution</a></p>\n<ul>\n<li><a href=\"https://github.com/aaxwaz/Humpback-whale-identification-challenge\" target=\"_blank\">https://github.com/aaxwaz/Humpback-whale-identification-challenge</a></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82352\" target=\"_blank\">7th Place Solution</a></p>\n<ul>\n<li>Part 1<ul>\n<li>Resnet50 -&gt; global concat (max, avg) pool -&gt; BN-&gt;Dropout-&gt; Linear(2048) -&gt; ReLU-&gt; BN-&gt;Dropout -&gt;clf(5004).</li>\n<li>Head is trained for X epochs, then all network is fine-tuned on for Y epochs.</li>\n<li>Validation set is 1 photo per whale for all whales with ≥2 photos + 1000 new_whales.</li></ul></li>\n<li>Part 2<ul>\n<li>Training on RGB images: 256x256, 384x384, 448x448, 360x720.</li>\n<li>Augmentations: random erasing, affine transformations(scale, translation, shear), brightness/contrast.</li>\n<li>Resnet34, resnet50, resnet101, densenet121, densenet162, seresnext50 — backbone architectures, followed by GeM pooling layer +L2 + multiplier.</li>\n<li>Loss: hard triplet loss</li></ul></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82427\" target=\"_blank\">9th Place Solution</a></p>\n<ul>\n<li>Train cosface or arcface net. Took the embeddings and calculate cosine similarity between the train and test images. Then average similarities for each class in train and took 5 most similar.</li>\n<li>resnet34, bninception and densenet121</li>\n<li>Final models after initial 64 epochs on 256x256 images, increase image size up to 1024 for resnet34, up to 512 for bninception and up to 640 for densenet121 and train for 64 epochs more.</li>\n<li>Augmentation: HorizontalFlip, rotate with 16 degree limit, ShiftScaleRotate with 16 degree limit, RandomBrightnessContrast, RandomGamma, Blur, Perspective transform: tile left, right and corner, Shear, MotionBlur,. GridDistortion, ElasticTransform, Cutout</li>\n<li>Cosface and Arcface parameters was optimised. Cosface: S = 32.0, M=0.35. Arcface: M1 = 1.0, M2 = 0.4, M3 = 0.15.</li>\n<li>Add test images with pseudo labels into train.</li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82430\" target=\"_blank\">10th Place Solution</a></p>\n<ul>\n<li>Part 1<ul>\n<li>Siamese architecture<ul>\n<li>trained progressively 299-&gt;384-&gt;512.</li>\n<li>ResNet-18, ResNet-34, SE-ResNeXt-50, ResNet-50</li>\n<li>a smart flipping strategy, which makes the model differentiate between the left and the right part of the fluke</li></ul></li>\n<li>Classification on features: concatenated all of the features generated by branch models and trained classification model on top of them. The head of classification was two dense layers with a little dropout. </li>\n<li>Large blend for new whale/not new whale binary classification</li></ul></li>\n<li>Part 2<ul>\n<li>metric learning with multiple branches and margin loss, trained on multiple resolution crops using bboxes, grayscale and RGB input images</li></ul></li></ul></li>\n<li><p><a href=\"https://www.kaggle.com/c/humpback-whale-identification/discussion/82361\" target=\"_blank\">15th Place Solution</a></p>\n<ul>\n<li>Sphereface (seresnext-50(multi-layer fusion, 384x384), resnext50(multi-layer fusion, 384x384))</li>\n<li>Oversample images in 5004 class that occur less than 20 to 20. Images are resized to 384 and 512 as input. </li>\n<li>Augmentation: randompadding + randomcrop</li>\n<li>A self-designed keypoint detector on 1000 Hand-Annotated Humpback Whale Fluke Keypoints dataset. Then affine the image to prefined coordinate using the keypoints by learning a transfer matrix</li></ul></li>\n</ul>",
      "rawMarkdown": "https://www.kaggle.com/c/humpback-whale-identification/overview\n\n- [1st Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82366)\n    - Input size is (512, 256)\n    - Use 4 channels, RGB + masks (trained by 450 open source labels) as input.\n    - Step 1: Training within all labels with >10 samples.\n    - Step 2: Training with all samples, and fixed all of the networks except the last two layers.\n    - Flip images (+0.006) and consider flipped id-whales as different whales and keep new whales as the same.\n    - Pseudo labels (+ 0.001). Added around 2000 test images (with confidence > 0.96) into the training set.\n    - Class balance (+0.001 ~ 0.002). For top 5 predictions class1 to class5, if: conf class1 – conf class 2 < 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, switch class1 and class2’s positions.\n\n- [2nd Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/83885)\n    - https://github.com/SeuTao/Humpback-Whale-Identification-Challenge-2019_2nd_palce_solution\n    - Input: 256x512 or 512*512 cropped images\n    - Backbone: resnet101, seresnet101, seresnext101\n    - Loss function: arcface loss + triplet loss + focal loss\n    - Optimizer: adam with warm up lr strategy\n    - Augmentation: blur,grayscale,noise,shear,rotate,perspective transform\n    - Horizontal flip to create more ids -> 5004*2\n    - Pseudo Labeling\n\n- [3rd Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82484)\n    - ArcFace - the layers after last convolution were replaced to flattening -> BN -> dropout -> FC -> BN.\n    - densenet121\n    - Hyperparameters: m 0.5 (the default value of the paper), weight decay 0.0005, droupout 0.5\n    - Augmentation: average blur, motion blur, add, multiply, grayscale, scale, translate, shear, rotate, align or no-align\n    - adam optimizer - learning rate of 0.00025 -> 0.000125 -> 0.0000625\n\n- [4th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82356)\n    - Keypoint matching (SIFT, ROOTSIFT, and a host of binary descriptors and matchers)\n        - Loop through all test/train pairs\n        - Match keypoints using faiss\n        - Double homography filtering of keypoints (LMEDS followed by RANSAC)\n        - xgboost prediction to validate homography matrix\n        - if # of matches > threshold, then use prediction\n    - Siamese Network\n        - train classification on top 200 classes\n        - fine-tune on all classes where N>8 (~576 classes)\n        - fine-tune on all classes\n        - fine-tune on all classes + mixup + image size 384x384\n    - Postprocessing\n        - Suppression of the dominant top-5 predictions by transposing the resulting prediction matrix from the Siamese network and determining the threshold along the train axis where n=1.\n\n- [5th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82369)\n    - https://github.com/aaxwaz/Humpback-whale-identification-challenge\n\n- [7th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82352)\n    - Part 1\n        - Resnet50 -> global concat (max, avg) pool -> BN->Dropout-> Linear(2048) -> ReLU-> BN->Dropout ->clf(5004).\n        - Head is trained for X epochs, then all network is fine-tuned on for Y epochs.\n        - Validation set is 1 photo per whale for all whales with ≥2 photos + 1000 new_whales.\n    - Part 2\n        - Training on RGB images: 256x256, 384x384, 448x448, 360x720.\n        - Augmentations: random erasing, affine transformations(scale, translation, shear), brightness/contrast.\n        - Resnet34, resnet50, resnet101, densenet121, densenet162, seresnext50 — backbone architectures, followed by GeM pooling layer +L2 + multiplier.\n        - Loss: hard triplet loss\n\n- [9th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82427)\n    - Train cosface or arcface net. Took the embeddings and calculate cosine similarity between the train and test images. Then average similarities for each class in train and took 5 most similar.\n    - resnet34, bninception and densenet121\n    - Final models after initial 64 epochs on 256x256 images, increase image size up to 1024 for resnet34, up to 512 for bninception and up to 640 for densenet121 and train for 64 epochs more.\n    - Augmentation: HorizontalFlip, rotate with 16 degree limit, ShiftScaleRotate with 16 degree limit, RandomBrightnessContrast, RandomGamma, Blur, Perspective transform: tile left, right and corner, Shear, MotionBlur,. GridDistortion, ElasticTransform, Cutout\n    - Cosface and Arcface parameters was optimised. Cosface: S = 32.0, M=0.35. Arcface: M1 = 1.0, M2 = 0.4, M3 = 0.15.\n    - Add test images with pseudo labels into train.\n    \n- [10th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82430)\n    - Part 1\n        - Siamese architecture\n            - trained progressively 299->384->512.\n            - ResNet-18, ResNet-34, SE-ResNeXt-50, ResNet-50\n            - a smart flipping strategy, which makes the model differentiate between the left and the right part of the fluke\n        - Classification on features: concatenated all of the features generated by branch models and trained classification model on top of them. The head of classification was two dense layers with a little dropout. \n        - Large blend for new whale/not new whale binary classification\n    - Part 2\n        - metric learning with multiple branches and margin loss, trained on multiple resolution crops using bboxes, grayscale and RGB input images\n\n- [15th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82361)\n    - Sphereface (seresnext-50(multi-layer fusion, 384x384), resnext50(multi-layer fusion, 384x384))\n    - Oversample images in 5004 class that occur less than 20 to 20. Images are resized to 384 and 512 as input. \n    - Augmentation: randompadding + randomcrop\n    - A self-designed keypoint detector on 1000 Hand-Annotated Humpback Whale Fluke Keypoints dataset. Then affine the image to prefined coordinate using the keypoints by learning a transfer matrix",
      "votes": null
    },
    {
      "id": "1711925",
      "postDate": "03/04/2022 13:26:25",
      "content": "<p>Super great work!!</p>",
      "rawMarkdown": "Super great work!!",
      "votes": null
    },
    {
      "id": "1711960",
      "postDate": "03/04/2022 13:53:15",
      "content": "<p>Glad you like it. Cheers</p>",
      "rawMarkdown": "Glad you like it. Cheers",
      "votes": null
    },
    {
      "id": "1712483",
      "postDate": "03/05/2022 01:16:37",
      "content": "<p>This is a very nice post!<br>\nI've referred to past competitions, but the summary is very helpful.</p>",
      "rawMarkdown": "This is a very nice post!\nI've referred to past competitions, but the summary is very helpful.",
      "votes": null
    },
    {
      "id": "1719300",
      "postDate": "03/11/2022 16:18:30",
      "content": "<p>Very Nice!</p>",
      "rawMarkdown": "Very Nice!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1711925,
      "author_name": "haruki741",
      "author_url": "",
      "post_date": "03/04/2022 13:26:25",
      "content": "<p>Super great work!!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1711960,
          "author_name": "snnclsr",
          "author_url": "",
          "post_date": "03/04/2022 13:53:15",
          "content": "<p>Glad you like it. Cheers</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1712483,
      "author_name": "tock99",
      "author_url": "",
      "post_date": "03/05/2022 01:16:37",
      "content": "<p>This is a very nice post!<br>\nI've referred to past competitions, but the summary is very helpful.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1719300,
      "author_name": "yongdawoon",
      "author_url": "",
      "post_date": "03/11/2022 16:18:30",
      "content": "<p>Very Nice!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1711917": "https://www.kaggle.com/c/humpback-whale-identification/overview\n\n- [1st Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82366)\n    - Input size is (512, 256)\n    - Use 4 channels, RGB + masks (trained by 450 open source labels) as input.\n    - Step 1: Training within all labels with >10 samples.\n    - Step 2: Training with all samples, and fixed all of the networks except the last two layers.\n    - Flip images (+0.006) and consider flipped id-whales as different whales and keep new whales as the same.\n    - Pseudo labels (+ 0.001). Added around 2000 test images (with confidence > 0.96) into the training set.\n    - Class balance (+0.001 ~ 0.002). For top 5 predictions class1 to class5, if: conf class1 – conf class 2 < 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, switch class1 and class2’s positions.\n\n- [2nd Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/83885)\n    - https://github.com/SeuTao/Humpback-Whale-Identification-Challenge-2019_2nd_palce_solution\n    - Input: 256x512 or 512*512 cropped images\n    - Backbone: resnet101, seresnet101, seresnext101\n    - Loss function: arcface loss + triplet loss + focal loss\n    - Optimizer: adam with warm up lr strategy\n    - Augmentation: blur,grayscale,noise,shear,rotate,perspective transform\n    - Horizontal flip to create more ids -> 5004*2\n    - Pseudo Labeling\n\n- [3rd Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82484)\n    - ArcFace - the layers after last convolution were replaced to flattening -> BN -> dropout -> FC -> BN.\n    - densenet121\n    - Hyperparameters: m 0.5 (the default value of the paper), weight decay 0.0005, droupout 0.5\n    - Augmentation: average blur, motion blur, add, multiply, grayscale, scale, translate, shear, rotate, align or no-align\n    - adam optimizer - learning rate of 0.00025 -> 0.000125 -> 0.0000625\n\n- [4th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82356)\n    - Keypoint matching (SIFT, ROOTSIFT, and a host of binary descriptors and matchers)\n        - Loop through all test/train pairs\n        - Match keypoints using faiss\n        - Double homography filtering of keypoints (LMEDS followed by RANSAC)\n        - xgboost prediction to validate homography matrix\n        - if # of matches > threshold, then use prediction\n    - Siamese Network\n        - train classification on top 200 classes\n        - fine-tune on all classes where N>8 (~576 classes)\n        - fine-tune on all classes\n        - fine-tune on all classes + mixup + image size 384x384\n    - Postprocessing\n        - Suppression of the dominant top-5 predictions by transposing the resulting prediction matrix from the Siamese network and determining the threshold along the train axis where n=1.\n\n- [5th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82369)\n    - https://github.com/aaxwaz/Humpback-whale-identification-challenge\n\n- [7th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82352)\n    - Part 1\n        - Resnet50 -> global concat (max, avg) pool -> BN->Dropout-> Linear(2048) -> ReLU-> BN->Dropout ->clf(5004).\n        - Head is trained for X epochs, then all network is fine-tuned on for Y epochs.\n        - Validation set is 1 photo per whale for all whales with ≥2 photos + 1000 new_whales.\n    - Part 2\n        - Training on RGB images: 256x256, 384x384, 448x448, 360x720.\n        - Augmentations: random erasing, affine transformations(scale, translation, shear), brightness/contrast.\n        - Resnet34, resnet50, resnet101, densenet121, densenet162, seresnext50 — backbone architectures, followed by GeM pooling layer +L2 + multiplier.\n        - Loss: hard triplet loss\n\n- [9th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82427)\n    - Train cosface or arcface net. Took the embeddings and calculate cosine similarity between the train and test images. Then average similarities for each class in train and took 5 most similar.\n    - resnet34, bninception and densenet121\n    - Final models after initial 64 epochs on 256x256 images, increase image size up to 1024 for resnet34, up to 512 for bninception and up to 640 for densenet121 and train for 64 epochs more.\n    - Augmentation: HorizontalFlip, rotate with 16 degree limit, ShiftScaleRotate with 16 degree limit, RandomBrightnessContrast, RandomGamma, Blur, Perspective transform: tile left, right and corner, Shear, MotionBlur,. GridDistortion, ElasticTransform, Cutout\n    - Cosface and Arcface parameters was optimised. Cosface: S = 32.0, M=0.35. Arcface: M1 = 1.0, M2 = 0.4, M3 = 0.15.\n    - Add test images with pseudo labels into train.\n    \n- [10th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82430)\n    - Part 1\n        - Siamese architecture\n            - trained progressively 299->384->512.\n            - ResNet-18, ResNet-34, SE-ResNeXt-50, ResNet-50\n            - a smart flipping strategy, which makes the model differentiate between the left and the right part of the fluke\n        - Classification on features: concatenated all of the features generated by branch models and trained classification model on top of them. The head of classification was two dense layers with a little dropout. \n        - Large blend for new whale/not new whale binary classification\n    - Part 2\n        - metric learning with multiple branches and margin loss, trained on multiple resolution crops using bboxes, grayscale and RGB input images\n\n- [15th Place Solution](https://www.kaggle.com/c/humpback-whale-identification/discussion/82361)\n    - Sphereface (seresnext-50(multi-layer fusion, 384x384), resnext50(multi-layer fusion, 384x384))\n    - Oversample images in 5004 class that occur less than 20 to 20. Images are resized to 384 and 512 as input. \n    - Augmentation: randompadding + randomcrop\n    - A self-designed keypoint detector on 1000 Hand-Annotated Humpback Whale Fluke Keypoints dataset. Then affine the image to prefined coordinate using the keypoints by learning a transfer matrix",
    "1711925": "Super great work!!",
    "1711960": "Glad you like it. Cheers",
    "1712483": "This is a very nice post!\nI've referred to past competitions, but the summary is very helpful.",
    "1719300": "Very Nice!"
  },
  "source": "meta"
}