{
  "id": 319911,
  "title": "49 place  - Silver Solution",
  "url": "/competitions/happy-whale-and-dolphin/writeups/maksim-markeev-49-place-silver-solution",
  "author_name": "",
  "post_date": "2022-04-20T12:58:46.937Z",
  "votes": 13,
  "comment_count": 3,
  "views": 0,
  "content": "<p>First of all, I want to thank Kaggle for the super platform and super community. Everything I learned about DS I learned on Kaggle. Secondly, thanks to the organizers of the contest for an interesting task, it helped me get distracted from sad thoughts.</p>\n<p><strong>Dataset</strong><br>\nThe key point in the competition was the dataset. I used <strong>Baskfins</strong> and <strong>Fullbody</strong> datasets by JAN BRE<br>\nThere was an idea to use an intermediate dataset, but I didn’t have time.<br>\nI also tried a dataset with water removed, but the result was much worse. I added it in ensemble.</p>\n<p><strong>Model</strong><br>\nI tried Effnet models and Effnet 7 work better.<br>\nArcFace loss.<br>\nImage resolution is 768. I tried 1024, but model does not converge(<br>\nI also tried ConvNext result was worse, but it added in ensemble.<br>\n30-50 epochs with 5 epochs warm-up and then cosine decrease.<br>\nThere was no overfit at all. More epochs - better results )</p>\n<p><strong>CV and Ensemble</strong><br>\nI connected embeddings from 9 models and found the cosine distance between train and test images.<br>\nSince there was a lot of data and very little time at the TPU, I did cross-validation on only 1 fold. <br>\nAnd get 0.72 CV without <strong>new_individual</strong> and CV 0.86 with threshold 0.6<br>\nAnd then trained all models on 100% train data, and use threshold 0.65 since in test dataset less <strong>new_individual</strong></p>\n<p><em>Thanks for reading!</em></p>",
  "messages": [
    {
      "id": "1760552",
      "postDate": "04/19/2022 11:32:35",
      "content": "<p>First of all, I want to thank Kaggle for the super platform and super community. Everything I learned about DS I learned on Kaggle. Secondly, thanks to the organizers of the contest for an interesting task, it helped me get distracted from sad thoughts.</p>\n<p><strong>Dataset</strong><br>\nThe key point in the competition was the dataset. I used <strong>Baskfins</strong> and <strong>Fullbody</strong> datasets by JAN BRE<br>\nThere was an idea to use an intermediate dataset, but I didn’t have time.<br>\nI also tried a dataset with water removed, but the result was much worse. I added it in ensemble.</p>\n<p><strong>Model</strong><br>\nI tried Effnet models and Effnet 7 work better.<br>\nArcFace loss.<br>\nImage resolution is 768. I tried 1024, but model does not converge(<br>\nI also tried ConvNext result was worse, but it added in ensemble.<br>\n30-50 epochs with 5 epochs warm-up and then cosine decrease.<br>\nThere was no overfit at all. More epochs - better results )</p>\n<p><strong>CV and Ensemble</strong><br>\nI connected embeddings from 9 models and found the cosine distance between train and test images.<br>\nSince there was a lot of data and very little time at the TPU, I did cross-validation on only 1 fold. <br>\nAnd get 0.72 CV without <strong>new_individual</strong> and CV 0.86 with threshold 0.6<br>\nAnd then trained all models on 100% train data, and use threshold 0.65 since in test dataset less <strong>new_individual</strong></p>\n<p><em>Thanks for reading!</em></p>",
      "rawMarkdown": "First of all, I want to thank Kaggle for the super platform and super community. Everything I learned about DS I learned on Kaggle. Secondly, thanks to the organizers of the contest for an interesting task, it helped me get distracted from sad thoughts.\n\n**Dataset**\nThe key point in the competition was the dataset. I used **Baskfins** and **Fullbody** datasets by JAN BRE\nThere was an idea to use an intermediate dataset, but I didn’t have time.\nI also tried a dataset with water removed, but the result was much worse. I added it in ensemble.\n\n**Model**\nI tried Effnet models and Effnet 7 work better.\nArcFace loss.\nImage resolution is 768. I tried 1024, but model does not converge(\nI also tried ConvNext result was worse, but it added in ensemble.\n30-50 epochs with 5 epochs warm-up and then cosine decrease.\nThere was no overfit at all. More epochs - better results )\n\n**CV and Ensemble**\nI connected embeddings from 9 models and found the cosine distance between train and test images.\nSince there was a lot of data and very little time at the TPU, I did cross-validation on only 1 fold. \nAnd get 0.72 CV without **new_individual** and CV 0.86 with threshold 0.6\nAnd then trained all models on 100% train data, and use threshold 0.65 since in test dataset less **new_individual**\n\n*Thanks for reading!*",
      "votes": null
    },
    {
      "id": "1760965",
      "postDate": "04/19/2022 16:03:55",
      "content": "<p>Thanks for sharing! It is always great to compare solutions!<br>\nI'm surprised your EffnetB7 did not converge with 1024x1024-pixel images as it was my best model and it trained fairly smoothly on a \"fullbody\" dataset. I did not have enough TPU time to train a \"backfin\" model with these specific settings though. <br>\nDo you know how much gain you got from using 100% of the training data for your final model by any chance?</p>",
      "rawMarkdown": "Thanks for sharing! It is always great to compare solutions!\nI'm surprised your EffnetB7 did not converge with 1024x1024-pixel images as it was my best model and it trained fairly smoothly on a \"fullbody\" dataset. I did not have enough TPU time to train a \"backfin\" model with these specific settings though. \nDo you know how much gain you got from using 100% of the training data for your final model by any chance?",
      "votes": null
    },
    {
      "id": "1760991",
      "postDate": "04/19/2022 16:16:16",
      "content": "<p>A big boost. For example single model B7 - 768 on Backfins DS 1 fold i got 0.779 on Public LB, and 0.733 Private. And same model trained on 100% train data  Backfins DS 0.808 Public and 0.762 Private. With same threshold 0.5, same epochs, same LR.</p>",
      "rawMarkdown": "A big boost. For example single model B7 - 768 on Backfins DS 1 fold i got 0.779 on Public LB, and 0.733 Private. And same model trained on 100% train data  Backfins DS 0.808 Public and 0.762 Private. With same threshold 0.5, same epochs, same LR.",
      "votes": null
    },
    {
      "id": "1761018",
      "postDate": "04/19/2022 16:34:08",
      "content": "<p>Oh, nice one! On my best \"fullbody\" model, I got +0.015 on both private and public LB by concatenating the embeddings from different folds, in comparison to a single fold. Definitely looks like you had a more straightforward and effective solution!</p>",
      "rawMarkdown": "Oh, nice one! On my best \"fullbody\" model, I got +0.015 on both private and public LB by concatenating the embeddings from different folds, in comparison to a single fold. Definitely looks like you had a more straightforward and effective solution!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1760965,
      "author_name": "frlemarchand",
      "author_url": "",
      "post_date": "04/19/2022 16:03:55",
      "content": "<p>Thanks for sharing! It is always great to compare solutions!<br>\nI'm surprised your EffnetB7 did not converge with 1024x1024-pixel images as it was my best model and it trained fairly smoothly on a \"fullbody\" dataset. I did not have enough TPU time to train a \"backfin\" model with these specific settings though. <br>\nDo you know how much gain you got from using 100% of the training data for your final model by any chance?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1760991,
          "author_name": "maxmar",
          "author_url": "",
          "post_date": "04/19/2022 16:16:16",
          "content": "<p>A big boost. For example single model B7 - 768 on Backfins DS 1 fold i got 0.779 on Public LB, and 0.733 Private. And same model trained on 100% train data  Backfins DS 0.808 Public and 0.762 Private. With same threshold 0.5, same epochs, same LR.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1761018,
          "author_name": "frlemarchand",
          "author_url": "",
          "post_date": "04/19/2022 16:34:08",
          "content": "<p>Oh, nice one! On my best \"fullbody\" model, I got +0.015 on both private and public LB by concatenating the embeddings from different folds, in comparison to a single fold. Definitely looks like you had a more straightforward and effective solution!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1760552": "First of all, I want to thank Kaggle for the super platform and super community. Everything I learned about DS I learned on Kaggle. Secondly, thanks to the organizers of the contest for an interesting task, it helped me get distracted from sad thoughts.\n\n**Dataset**\nThe key point in the competition was the dataset. I used **Baskfins** and **Fullbody** datasets by JAN BRE\nThere was an idea to use an intermediate dataset, but I didn’t have time.\nI also tried a dataset with water removed, but the result was much worse. I added it in ensemble.\n\n**Model**\nI tried Effnet models and Effnet 7 work better.\nArcFace loss.\nImage resolution is 768. I tried 1024, but model does not converge(\nI also tried ConvNext result was worse, but it added in ensemble.\n30-50 epochs with 5 epochs warm-up and then cosine decrease.\nThere was no overfit at all. More epochs - better results )\n\n**CV and Ensemble**\nI connected embeddings from 9 models and found the cosine distance between train and test images.\nSince there was a lot of data and very little time at the TPU, I did cross-validation on only 1 fold. \nAnd get 0.72 CV without **new_individual** and CV 0.86 with threshold 0.6\nAnd then trained all models on 100% train data, and use threshold 0.65 since in test dataset less **new_individual**\n\n*Thanks for reading!*",
    "1760965": "Thanks for sharing! It is always great to compare solutions!\nI'm surprised your EffnetB7 did not converge with 1024x1024-pixel images as it was my best model and it trained fairly smoothly on a \"fullbody\" dataset. I did not have enough TPU time to train a \"backfin\" model with these specific settings though. \nDo you know how much gain you got from using 100% of the training data for your final model by any chance?",
    "1760991": "A big boost. For example single model B7 - 768 on Backfins DS 1 fold i got 0.779 on Public LB, and 0.733 Private. And same model trained on 100% train data  Backfins DS 0.808 Public and 0.762 Private. With same threshold 0.5, same epochs, same LR.",
    "1761018": "Oh, nice one! On my best \"fullbody\" model, I got +0.015 on both private and public LB by concatenating the embeddings from different folds, in comparison to a single fold. Definitely looks like you had a more straightforward and effective solution!"
  },
  "source": "meta"
}