{
  "id": 174413,
  "title": "Custom LR Scheduler using Pytorch",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/174413",
  "author_name": "Zaber Ibn Abdul Hakim",
  "post_date": "2020-08-13T12:55:48.158000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Most of the public tf/keras notebooks seem to use one similar learning rate scheduler. It gave satisfactory result so I didn't try to experiment with other LR schedulers.</p>\n<p>The LR scheduler changes learning rate like this:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3273493%2Fb0361b859bd01fc163d8115f09355024%2Flrscheduler.jpg?generation=1597322781557570&amp;alt=media\" alt=\"\"></p>\n<p>I have made a pytorch version of this <a href=\"https://www.kaggle.com/zaber666/custom-lr-scheduler-pytorch\" target=\"_blank\">in this dataset</a><br>\nJust add the dataset to your notebook and use it as following:</p>\n<pre><code>import sys\nsys.path.insert(0, '../input/custom-lr-scheduler-pytorch')\n\nimport scheduler\nsc = scheduler.CustomLRScheduler(optimizer=your_optimizer, batch_size=bs, replicas=REPLICAS)\n\n.......\n.......\n\nloss.backward()\noptimizer.step()\nsc.step()\n</code></pre>",
  "messages": [
    {
      "id": 969085,
      "postDate": "2020-08-13T12:55:48.157Z",
      "content": "<p>Most of the public tf/keras notebooks seem to use one similar learning rate scheduler. It gave satisfactory result so I didn't try to experiment with other LR schedulers.</p>\n<p>The LR scheduler changes learning rate like this:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3273493%2Fb0361b859bd01fc163d8115f09355024%2Flrscheduler.jpg?generation=1597322781557570&amp;alt=media\" alt=\"\"></p>\n<p>I have made a pytorch version of this <a href=\"https://www.kaggle.com/zaber666/custom-lr-scheduler-pytorch\" target=\"_blank\">in this dataset</a><br>\nJust add the dataset to your notebook and use it as following:</p>\n<pre><code>import sys\nsys.path.insert(0, '../input/custom-lr-scheduler-pytorch')\n\nimport scheduler\nsc = scheduler.CustomLRScheduler(optimizer=your_optimizer, batch_size=bs, replicas=REPLICAS)\n\n.......\n.......\n\nloss.backward()\noptimizer.step()\nsc.step()\n</code></pre>",
      "rawMarkdown": "Most of the public tf/keras notebooks seem to use one similar learning rate scheduler. It gave satisfactory result so I didn't try to experiment with other LR schedulers.\n\nThe LR scheduler changes learning rate like this:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3273493%2Fb0361b859bd01fc163d8115f09355024%2Flrscheduler.jpg?generation=1597322781557570&alt=media)\n\nI have made a pytorch version of this [in this dataset](https://www.kaggle.com/zaber666/custom-lr-scheduler-pytorch)\nJust add the dataset to your notebook and use it as following:\n```\nimport sys\nsys.path.insert(0, '../input/custom-lr-scheduler-pytorch')\n\nimport scheduler\nsc = scheduler.CustomLRScheduler(optimizer=your_optimizer, batch_size=bs, replicas=REPLICAS)\n\n.......\n.......\n\nloss.backward()\noptimizer.step()\nsc.step()\n```\n",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "969085": "Most of the public tf/keras notebooks seem to use one similar learning rate scheduler. It gave satisfactory result so I didn't try to experiment with other LR schedulers.\n\nThe LR scheduler changes learning rate like this:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3273493%2Fb0361b859bd01fc163d8115f09355024%2Flrscheduler.jpg?generation=1597322781557570&alt=media)\n\nI have made a pytorch version of this [in this dataset](https://www.kaggle.com/zaber666/custom-lr-scheduler-pytorch)\nJust add the dataset to your notebook and use it as following:\n```\nimport sys\nsys.path.insert(0, '../input/custom-lr-scheduler-pytorch')\n\nimport scheduler\nsc = scheduler.CustomLRScheduler(optimizer=your_optimizer, batch_size=bs, replicas=REPLICAS)\n\n.......\n.......\n\nloss.backward()\noptimizer.step()\nsc.step()\n```\n"
  }
}