{
  "id": 308389,
  "title": "Pytorch implementation",
  "url": "/competitions/happy-whale-and-dolphin/discussion/308389",
  "author_name": "Vlad Vaduva",
  "post_date": "2022-02-18T12:56:08.799000",
  "votes": 28,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Due to the fact that most notebooks are design in Tensoflow for TPU usage, I started implemented a Pytorch approach. It does not have the advantage of the TPU large batch size and model scaling but it can be trained locally without any hourly TPU limit.<br>\nThe approach is a basic one can can be extended. I will try to update it periodic with code refactoring and architecture changes. The current design consist in a efficient net B2 backbone with a 512 encoder size layer. On the inference part, a KNN classifier is learning the embeddings from the training data and predicts on the test data. The confidence level for introducing \"new_individual\" is currently a naive one. It is based on the distribution of confidences. There are more smart ways of doing this and it will be improved in the future.</p>\n<p>Training notebook link: <a href=\"https://www.kaggle.com/vladvdv/pytorch-train-notebook-arcface-gem-pooling\" target=\"_blank\">https://www.kaggle.com/vladvdv/pytorch-train-notebook-arcface-gem-pooling</a><br>\nInference notebook link: <a href=\"https://www.kaggle.com/vladvdv/pytorch-inference-notebok-arcface-gem-pooling\" target=\"_blank\">https://www.kaggle.com/vladvdv/pytorch-inference-notebok-arcface-gem-pooling</a></p>\n<p>Training notebook was based on <a href=\"https://www.kaggle.com/debarshichanda/pytorch-arcface-gem-pooling-starter\" target=\"_blank\">https://www.kaggle.com/debarshichanda/pytorch-arcface-gem-pooling-starter</a><br>\n<img src=\"https://i.imgur.com/9Bxhanq.jpeg\" alt=\"https://i.imgur.com/9Bxhanq.jpeg\"></p>",
  "messages": [
    {
      "id": 1695907,
      "postDate": "2022-02-18T12:56:08.800Z",
      "content": "<p>Due to the fact that most notebooks are design in Tensoflow for TPU usage, I started implemented a Pytorch approach. It does not have the advantage of the TPU large batch size and model scaling but it can be trained locally without any hourly TPU limit.<br>\nThe approach is a basic one can can be extended. I will try to update it periodic with code refactoring and architecture changes. The current design consist in a efficient net B2 backbone with a 512 encoder size layer. On the inference part, a KNN classifier is learning the embeddings from the training data and predicts on the test data. The confidence level for introducing \"new_individual\" is currently a naive one. It is based on the distribution of confidences. There are more smart ways of doing this and it will be improved in the future.</p>\n<p>Training notebook link: <a href=\"https://www.kaggle.com/vladvdv/pytorch-train-notebook-arcface-gem-pooling\" target=\"_blank\">https://www.kaggle.com/vladvdv/pytorch-train-notebook-arcface-gem-pooling</a><br>\nInference notebook link: <a href=\"https://www.kaggle.com/vladvdv/pytorch-inference-notebok-arcface-gem-pooling\" target=\"_blank\">https://www.kaggle.com/vladvdv/pytorch-inference-notebok-arcface-gem-pooling</a></p>\n<p>Training notebook was based on <a href=\"https://www.kaggle.com/debarshichanda/pytorch-arcface-gem-pooling-starter\" target=\"_blank\">https://www.kaggle.com/debarshichanda/pytorch-arcface-gem-pooling-starter</a><br>\n<img src=\"https://i.imgur.com/9Bxhanq.jpeg\" alt=\"https://i.imgur.com/9Bxhanq.jpeg\"></p>",
      "rawMarkdown": "Due to the fact that most notebooks are design in Tensoflow for TPU usage, I started implemented a Pytorch approach. It does not have the advantage of the TPU large batch size and model scaling but it can be trained locally without any hourly TPU limit.\nThe approach is a basic one can can be extended. I will try to update it periodic with code refactoring and architecture changes. The current design consist in a efficient net B2 backbone with a 512 encoder size layer. On the inference part, a KNN classifier is learning the embeddings from the training data and predicts on the test data. The confidence level for introducing \"new_individual\" is currently a naive one. It is based on the distribution of confidences. There are more smart ways of doing this and it will be improved in the future.\n\nTraining notebook link: https://www.kaggle.com/vladvdv/pytorch-train-notebook-arcface-gem-pooling\nInference notebook link: https://www.kaggle.com/vladvdv/pytorch-inference-notebok-arcface-gem-pooling\n\nTraining notebook was based on https://www.kaggle.com/debarshichanda/pytorch-arcface-gem-pooling-starter\n![https://i.imgur.com/9Bxhanq.jpeg](https://i.imgur.com/9Bxhanq.jpeg)",
      "votes": 28
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1695907": "Due to the fact that most notebooks are design in Tensoflow for TPU usage, I started implemented a Pytorch approach. It does not have the advantage of the TPU large batch size and model scaling but it can be trained locally without any hourly TPU limit.\nThe approach is a basic one can can be extended. I will try to update it periodic with code refactoring and architecture changes. The current design consist in a efficient net B2 backbone with a 512 encoder size layer. On the inference part, a KNN classifier is learning the embeddings from the training data and predicts on the test data. The confidence level for introducing \"new_individual\" is currently a naive one. It is based on the distribution of confidences. There are more smart ways of doing this and it will be improved in the future.\n\nTraining notebook link: https://www.kaggle.com/vladvdv/pytorch-train-notebook-arcface-gem-pooling\nInference notebook link: https://www.kaggle.com/vladvdv/pytorch-inference-notebok-arcface-gem-pooling\n\nTraining notebook was based on https://www.kaggle.com/debarshichanda/pytorch-arcface-gem-pooling-starter\n![https://i.imgur.com/9Bxhanq.jpeg](https://i.imgur.com/9Bxhanq.jpeg)"
  }
}