{
  "id": 186770,
  "title": "LearningRate Finder ",
  "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/186770",
  "author_name": "",
  "post_date": "2020-09-25T20:39:12.360832600Z",
  "votes": 6,
  "comment_count": 4,
  "views": 0,
  "content": "<p>![](url to embed)To reduce the amount of guesswork concerning choosing a good initial learning rate, a learning rate finder can be used. As described in this paper a learning rate finder does a small run where the learning rate is increased after each processed batch and the corresponding loss is logged. The result of this is a lr vs. loss plot that can be used as guidance for choosing a optimal initial lr.</p>\n<pre><code>model = MyModelClass(hparams)\ntrainer = Trainer()\n\n# Run learning rate finder\nlr_finder = trainer.tuner.lr_find(model)\n\n# Results can be found in\nlr_finder.results\n\n# Plot with\nfig = lr_finder.plot(suggest=True)\nfig.show()\n\n# Pick point based on plot, or get suggestion\nnew_lr = lr_finder.suggestion()\n\n# update hparams of the model\nmodel.hparams.lr = new_lr\n\n# Fit model\ntrainer.fit(model)\n</code></pre>\n<p><img src=\"https://pytorch-lightning.readthedocs.io/en/latest/_images/lr_finder.png\" alt=\"lr\"></p>",
  "messages": [
    {
      "id": "1027097",
      "postDate": "09/25/2020 20:39:12",
      "content": "<p>![](url to embed)To reduce the amount of guesswork concerning choosing a good initial learning rate, a learning rate finder can be used. As described in this paper a learning rate finder does a small run where the learning rate is increased after each processed batch and the corresponding loss is logged. The result of this is a lr vs. loss plot that can be used as guidance for choosing a optimal initial lr.</p>\n<pre><code>model = MyModelClass(hparams)\ntrainer = Trainer()\n\n# Run learning rate finder\nlr_finder = trainer.tuner.lr_find(model)\n\n# Results can be found in\nlr_finder.results\n\n# Plot with\nfig = lr_finder.plot(suggest=True)\nfig.show()\n\n# Pick point based on plot, or get suggestion\nnew_lr = lr_finder.suggestion()\n\n# update hparams of the model\nmodel.hparams.lr = new_lr\n\n# Fit model\ntrainer.fit(model)\n</code></pre>\n<p><img src=\"https://pytorch-lightning.readthedocs.io/en/latest/_images/lr_finder.png\" alt=\"lr\"></p>",
      "rawMarkdown": "![](url to embed)To reduce the amount of guesswork concerning choosing a good initial learning rate, a learning rate finder can be used. As described in this paper a learning rate finder does a small run where the learning rate is increased after each processed batch and the corresponding loss is logged. The result of this is a lr vs. loss plot that can be used as guidance for choosing a optimal initial lr.\n```\nmodel = MyModelClass(hparams)\ntrainer = Trainer()\n\n# Run learning rate finder\nlr_finder = trainer.tuner.lr_find(model)\n\n# Results can be found in\nlr_finder.results\n\n# Plot with\nfig = lr_finder.plot(suggest=True)\nfig.show()\n\n# Pick point based on plot, or get suggestion\nnew_lr = lr_finder.suggestion()\n\n# update hparams of the model\nmodel.hparams.lr = new_lr\n\n# Fit model\ntrainer.fit(model)\n```\n![lr](https://pytorch-lightning.readthedocs.io/en/latest/_images/lr_finder.png)",
      "votes": null
    },
    {
      "id": "1028396",
      "postDate": "09/26/2020 20:22:34",
      "content": "<blockquote>\n  <p>As described in this paper a learning rate finder […]</p>\n</blockquote>\n<p>Can you link the paper pls?</p>",
      "rawMarkdown": "> As described in this paper a learning rate finder [...]\n\nCan you link the paper pls?",
      "votes": null
    },
    {
      "id": "1029114",
      "postDate": "09/27/2020 13:28:22",
      "content": "<p>Yes, please share the link to this paper or any tutorial code!</p>",
      "rawMarkdown": "Yes, please share the link to this paper or any tutorial code!",
      "votes": null
    },
    {
      "id": "1029157",
      "postDate": "09/27/2020 14:05:04",
      "content": "<p>Link of paper: <a href=\"https://arxiv.org/abs/1506.01186\" target=\"_blank\">https://arxiv.org/abs/1506.01186</a></p>",
      "rawMarkdown": "Link of paper: https://arxiv.org/abs/1506.01186",
      "votes": null
    },
    {
      "id": "1029161",
      "postDate": "09/27/2020 14:07:40",
      "content": "<p>Thank you !</p>",
      "rawMarkdown": "Thank you !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1028396,
      "author_name": "aliabdin1",
      "author_url": "",
      "post_date": "09/26/2020 20:22:34",
      "content": "<blockquote>\n  <p>As described in this paper a learning rate finder […]</p>\n</blockquote>\n<p>Can you link the paper pls?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1029114,
      "author_name": "fireheart7",
      "author_url": "",
      "post_date": "09/27/2020 13:28:22",
      "content": "<p>Yes, please share the link to this paper or any tutorial code!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1029157,
      "author_name": "doanquanvietnamca",
      "author_url": "",
      "post_date": "09/27/2020 14:05:04",
      "content": "<p>Link of paper: <a href=\"https://arxiv.org/abs/1506.01186\" target=\"_blank\">https://arxiv.org/abs/1506.01186</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1029161,
          "author_name": "aliabdin1",
          "author_url": "",
          "post_date": "09/27/2020 14:07:40",
          "content": "<p>Thank you !</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1027097": "![](url to embed)To reduce the amount of guesswork concerning choosing a good initial learning rate, a learning rate finder can be used. As described in this paper a learning rate finder does a small run where the learning rate is increased after each processed batch and the corresponding loss is logged. The result of this is a lr vs. loss plot that can be used as guidance for choosing a optimal initial lr.\n```\nmodel = MyModelClass(hparams)\ntrainer = Trainer()\n\n# Run learning rate finder\nlr_finder = trainer.tuner.lr_find(model)\n\n# Results can be found in\nlr_finder.results\n\n# Plot with\nfig = lr_finder.plot(suggest=True)\nfig.show()\n\n# Pick point based on plot, or get suggestion\nnew_lr = lr_finder.suggestion()\n\n# update hparams of the model\nmodel.hparams.lr = new_lr\n\n# Fit model\ntrainer.fit(model)\n```\n![lr](https://pytorch-lightning.readthedocs.io/en/latest/_images/lr_finder.png)",
    "1028396": "> As described in this paper a learning rate finder [...]\n\nCan you link the paper pls?",
    "1029114": "Yes, please share the link to this paper or any tutorial code!",
    "1029157": "Link of paper: https://arxiv.org/abs/1506.01186",
    "1029161": "Thank you !"
  },
  "source": "meta"
}