{
  "id": 378834,
  "title": "Transformer training tips & tricks",
  "url": "/competitions/nfl-player-contact-detection/discussion/378834",
  "author_name": "",
  "post_date": "2023-01-17T06:19:13.911504500Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>What are the best practices (and tips &amp; tricks) for improve transformer's training ?</p>",
  "messages": [
    {
      "id": "2103320",
      "postDate": "01/17/2023 06:19:13",
      "content": "<p>What are the best practices (and tips &amp; tricks) for improve transformer's training ?</p>",
      "rawMarkdown": "What are the best practices (and tips & tricks) for improve transformer's training ?",
      "votes": null
    },
    {
      "id": "2106032",
      "postDate": "01/18/2023 22:41:36",
      "content": "<p>I'm not sure what type of application you are looking for but, this post by <a href=\"https://www.kaggle.com/thedevastator\" target=\"_blank\">@thedevastator</a> will help you in general terms.<br>\n<a href=\"https://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/330119\" target=\"_blank\">https://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/330119</a></p>",
      "rawMarkdown": "I'm not sure what type of application you are looking for but, this post by @thedevastator will help you in general terms.\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/330119",
      "votes": null
    },
    {
      "id": "2119857",
      "postDate": "01/29/2023 06:47:41",
      "content": "<p>Here are some papers and posts that might be useful.</p>\n<ul>\n<li><a href=\"https://arxiv.org/pdf/1804.00247.pdf\" target=\"_blank\">Training Tips for the Transformer Model (Paper)</a></li>\n<li><a href=\"https://www.reddit.com/r/MachineLearning/comments/z088fo/r_tips_on_training_transformers/\" target=\"_blank\">Tips on training Transformers (Reddit Post)</a></li>\n<li><a href=\"https://towardsdatascience.com/this-is-how-to-train-better-transformer-models-d54191299978\" target=\"_blank\">This is how to train better transformer models (Medium Post)</a></li>\n<li><a href=\"https://towardsdatascience.com/advanced-techniques-for-fine-tuning-transformers-82e4e61e16e\" target=\"_blank\">Advanced Techniques for Fine-tuning Transformers(Medium Post)</a></li>\n</ul>",
      "rawMarkdown": "Here are some papers and posts that might be useful.\n- [Training Tips for the Transformer Model (Paper)](https://arxiv.org/pdf/1804.00247.pdf)\n- [Tips on training Transformers (Reddit Post)](https://www.reddit.com/r/MachineLearning/comments/z088fo/r_tips_on_training_transformers/)\n- [This is how to train better transformer models (Medium Post)](https://towardsdatascience.com/this-is-how-to-train-better-transformer-models-d54191299978)\n- [Advanced Techniques for Fine-tuning Transformers(Medium Post)](https://towardsdatascience.com/advanced-techniques-for-fine-tuning-transformers-82e4e61e16e)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2106032,
      "author_name": "surajdengale",
      "author_url": "",
      "post_date": "01/18/2023 22:41:36",
      "content": "<p>I'm not sure what type of application you are looking for but, this post by <a href=\"https://www.kaggle.com/thedevastator\" target=\"_blank\">@thedevastator</a> will help you in general terms.<br>\n<a href=\"https://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/330119\" target=\"_blank\">https://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/330119</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2119857,
      "author_name": "dariussingh",
      "author_url": "",
      "post_date": "01/29/2023 06:47:41",
      "content": "<p>Here are some papers and posts that might be useful.</p>\n<ul>\n<li><a href=\"https://arxiv.org/pdf/1804.00247.pdf\" target=\"_blank\">Training Tips for the Transformer Model (Paper)</a></li>\n<li><a href=\"https://www.reddit.com/r/MachineLearning/comments/z088fo/r_tips_on_training_transformers/\" target=\"_blank\">Tips on training Transformers (Reddit Post)</a></li>\n<li><a href=\"https://towardsdatascience.com/this-is-how-to-train-better-transformer-models-d54191299978\" target=\"_blank\">This is how to train better transformer models (Medium Post)</a></li>\n<li><a href=\"https://towardsdatascience.com/advanced-techniques-for-fine-tuning-transformers-82e4e61e16e\" target=\"_blank\">Advanced Techniques for Fine-tuning Transformers(Medium Post)</a></li>\n</ul>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2103320": "What are the best practices (and tips & tricks) for improve transformer's training ?",
    "2106032": "I'm not sure what type of application you are looking for but, this post by @thedevastator will help you in general terms.\nhttps://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/330119",
    "2119857": "Here are some papers and posts that might be useful.\n- [Training Tips for the Transformer Model (Paper)](https://arxiv.org/pdf/1804.00247.pdf)\n- [Tips on training Transformers (Reddit Post)](https://www.reddit.com/r/MachineLearning/comments/z088fo/r_tips_on_training_transformers/)\n- [This is how to train better transformer models (Medium Post)](https://towardsdatascience.com/this-is-how-to-train-better-transformer-models-d54191299978)\n- [Advanced Techniques for Fine-tuning Transformers(Medium Post)](https://towardsdatascience.com/advanced-techniques-for-fine-tuning-transformers-82e4e61e16e)"
  },
  "source": "meta"
}