{
  "id": 82480,
  "title": "[143th place] single model trained with cross entropy / contrastive loss using fastai",
  "url": "/competitions/humpback-whale-identification/writeups/radek-143th-place-single-model-trained-with-cross-",
  "author_name": "",
  "post_date": "2019-03-01T16:39:41.621285500Z",
  "votes": 17,
  "comment_count": 4,
  "views": 0,
  "content": "<p>First of all, extremely big congratulations to the competition winners! And big congrats to everyone who participated in the competition. The level seemed extremely high and the conversations on the forums were great!</p>\n\n<p>Here is the <a href=\"https://github.com/radekosmulski/whale/blob/master/classification_and_metric_learning.ipynb\">final addition</a> to my <a href=\"https://github.com/radekosmulski/whale\">whale repository</a>, a notebook for constructing and training the model that served as the basis for my submission.</p>\n\n<p>There is a lot of text in the notebook so not to duplicate things, here is a very short tldr:</p>\n\n<ul>\n<li>resnet50 cnn pretrained on imagenet</li>\n<li>custom loss (a combination of cross entropy and contrastive loss)</li>\n<li>training with the one cycle policy, progressive image resizing, gradual unfreezing, discriminative lrs, Adam\n<ul><li>some data augmentation but nothing too extreme</li>\n<li>generating progressively harder datasets as training progressed while balancing classes to some extent</li>\n<li>no cleaning up of train data, prediction based on euclidean similarity between feature vectors</li>\n<li>model trained on bounding boxes extracted in one of the earlier notebooks</li>\n<li>all code for the training, predicting and generating submission in this single notebook</li></ul></li>\n</ul>\n\n<p>And last but not least, thanks to the organizers for a really fun competition!</p>",
  "messages": [
    {
      "id": "481636",
      "postDate": "03/01/2019 16:39:41",
      "content": "<p>First of all, extremely big congratulations to the competition winners! And big congrats to everyone who participated in the competition. The level seemed extremely high and the conversations on the forums were great!</p>\n\n<p>Here is the <a href=\"https://github.com/radekosmulski/whale/blob/master/classification_and_metric_learning.ipynb\">final addition</a> to my <a href=\"https://github.com/radekosmulski/whale\">whale repository</a>, a notebook for constructing and training the model that served as the basis for my submission.</p>\n\n<p>There is a lot of text in the notebook so not to duplicate things, here is a very short tldr:</p>\n\n<ul>\n<li>resnet50 cnn pretrained on imagenet</li>\n<li>custom loss (a combination of cross entropy and contrastive loss)</li>\n<li>training with the one cycle policy, progressive image resizing, gradual unfreezing, discriminative lrs, Adam\n<ul><li>some data augmentation but nothing too extreme</li>\n<li>generating progressively harder datasets as training progressed while balancing classes to some extent</li>\n<li>no cleaning up of train data, prediction based on euclidean similarity between feature vectors</li>\n<li>model trained on bounding boxes extracted in one of the earlier notebooks</li>\n<li>all code for the training, predicting and generating submission in this single notebook</li></ul></li>\n</ul>\n\n<p>And last but not least, thanks to the organizers for a really fun competition!</p>",
      "rawMarkdown": "First of all, extremely big congratulations to the competition winners! And big congrats to everyone who participated in the competition. The level seemed extremely high and the conversations on the forums were great!\n\nHere is the [final addition](https://github.com/radekosmulski/whale/blob/master/classification_and_metric_learning.ipynb) to my [whale repository](https://github.com/radekosmulski/whale), a notebook for constructing and training the model that served as the basis for my submission.\n\nThere is a lot of text in the notebook so not to duplicate things, here is a very short tldr:\n\n - resnet50 cnn pretrained on imagenet\n - custom loss (a combination of cross entropy and contrastive loss)\n - training with the one cycle policy, progressive image resizing, gradual unfreezing, discriminative lrs, Adam\n- some data augmentation but nothing too extreme\n- generating progressively harder datasets as training progressed while balancing classes to some extent\n- no cleaning up of train data, prediction based on euclidean similarity between feature vectors\n- model trained on bounding boxes extracted in one of the earlier notebooks\n- all code for the training, predicting and generating submission in this single notebook\n\nAnd last but not least, thanks to the organizers for a really fun competition!",
      "votes": null
    },
    {
      "id": "481825",
      "postDate": "03/01/2019 22:16:47",
      "content": "<p>Thanks for sharing Radek!</p>",
      "rawMarkdown": "Thanks for sharing Radek!",
      "votes": null
    },
    {
      "id": "482345",
      "postDate": "03/02/2019 19:40:19",
      "content": "<p>Thank you Giba! 🙂 </p>",
      "rawMarkdown": "Thank you Giba! 🙂",
      "votes": null
    },
    {
      "id": "482451",
      "postDate": "03/03/2019 02:11:31",
      "content": "<p>Congrats <a href=\"/radek1\">@radek1</a> and thanks for sharing your well written notebook. Since I plan on repeating some top solutions post competition, I find this very useful.</p>",
      "rawMarkdown": "Congrats @radek1 and thanks for sharing your well written notebook. Since I plan on repeating some top solutions post competition, I find this very useful.",
      "votes": null
    },
    {
      "id": "482699",
      "postDate": "03/03/2019 14:35:58",
      "content": "<p>Thanks Radek for all your sharing! It is always informative and written code is very clear..</p>",
      "rawMarkdown": "Thanks Radek for all your sharing! It is always informative and written code is very clear..",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 481825,
      "author_name": "titericz",
      "author_url": "",
      "post_date": "03/01/2019 22:16:47",
      "content": "<p>Thanks for sharing Radek!</p>",
      "votes": null,
      "replies": [
        {
          "id": 482345,
          "author_name": "radek1",
          "author_url": "",
          "post_date": "03/02/2019 19:40:19",
          "content": "<p>Thank you Giba! 🙂 </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 482451,
      "author_name": "sheriytm",
      "author_url": "",
      "post_date": "03/03/2019 02:11:31",
      "content": "<p>Congrats <a href=\"/radek1\">@radek1</a> and thanks for sharing your well written notebook. Since I plan on repeating some top solutions post competition, I find this very useful.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 482699,
      "author_name": "hwasiti",
      "author_url": "",
      "post_date": "03/03/2019 14:35:58",
      "content": "<p>Thanks Radek for all your sharing! It is always informative and written code is very clear..</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "481636": "First of all, extremely big congratulations to the competition winners! And big congrats to everyone who participated in the competition. The level seemed extremely high and the conversations on the forums were great!\n\nHere is the [final addition](https://github.com/radekosmulski/whale/blob/master/classification_and_metric_learning.ipynb) to my [whale repository](https://github.com/radekosmulski/whale), a notebook for constructing and training the model that served as the basis for my submission.\n\nThere is a lot of text in the notebook so not to duplicate things, here is a very short tldr:\n\n - resnet50 cnn pretrained on imagenet\n - custom loss (a combination of cross entropy and contrastive loss)\n - training with the one cycle policy, progressive image resizing, gradual unfreezing, discriminative lrs, Adam\n- some data augmentation but nothing too extreme\n- generating progressively harder datasets as training progressed while balancing classes to some extent\n- no cleaning up of train data, prediction based on euclidean similarity between feature vectors\n- model trained on bounding boxes extracted in one of the earlier notebooks\n- all code for the training, predicting and generating submission in this single notebook\n\nAnd last but not least, thanks to the organizers for a really fun competition!",
    "481825": "Thanks for sharing Radek!",
    "482345": "Thank you Giba! 🙂",
    "482451": "Congrats @radek1 and thanks for sharing your well written notebook. Since I plan on repeating some top solutions post competition, I find this very useful.",
    "482699": "Thanks Radek for all your sharing! It is always informative and written code is very clear.."
  },
  "source": "meta"
}