{
  "id": 196365,
  "title": "optimizer weights saving?",
  "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/196365",
  "author_name": "",
  "post_date": "2020-11-10T21:33:31.880220500Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hey everybody!</p>\n<p>Do you think it is necessary to save and load the <code>optimizer</code>? I've seen a lot of pipelines that only store the model weights. But it seems to me that the optimizer has some value in it.</p>\n<p>I am using <code>Adam</code>. And i'm using Colab Pro to training (1 session something like 15-20 hours). </p>\n<p>I am using the resnet family of models (<code>18</code>, <code>34</code>, <code>50</code>, <code>101</code>). In past competitions, I did not pay so much attention to this, perhaps this is my mistake.</p>\n<p>I would be glad to any advice.</p>",
  "messages": [
    {
      "id": "1074613",
      "postDate": "11/10/2020 21:33:31",
      "content": "<p>Hey everybody!</p>\n<p>Do you think it is necessary to save and load the <code>optimizer</code>? I've seen a lot of pipelines that only store the model weights. But it seems to me that the optimizer has some value in it.</p>\n<p>I am using <code>Adam</code>. And i'm using Colab Pro to training (1 session something like 15-20 hours). </p>\n<p>I am using the resnet family of models (<code>18</code>, <code>34</code>, <code>50</code>, <code>101</code>). In past competitions, I did not pay so much attention to this, perhaps this is my mistake.</p>\n<p>I would be glad to any advice.</p>",
      "rawMarkdown": "Hey everybody!\n\nDo you think it is necessary to save and load the `optimizer`? I've seen a lot of pipelines that only store the model weights. But it seems to me that the optimizer has some value in it.\n\nI am using `Adam`. And i'm using Colab Pro to training (1 session something like 15-20 hours). \n\nI am using the resnet family of models (`18`, `34`, `50`, `101`). In past competitions, I did not pay so much attention to this, perhaps this is my mistake.\n\nI would be glad to any advice.",
      "votes": null
    },
    {
      "id": "1074626",
      "postDate": "11/10/2020 22:02:10",
      "content": "<p>Good question. In my experience in this competition I did not notice any difference between recreating the optimizer each epoch and reusing the same one. But maybe the difference is just small.</p>",
      "rawMarkdown": "Good question. In my experience in this competition I did not notice any difference between recreating the optimizer each epoch and reusing the same one. But maybe the difference is just small.",
      "votes": null
    },
    {
      "id": "1075666",
      "postDate": "11/11/2020 21:38:17",
      "content": "<p>yeah. i load the  optimizer too. i use pytorch_lightning so it  automatic  load all components for me</p>",
      "rawMarkdown": "yeah. i load the  optimizer too. i use pytorch_lightning so it  automatic  load all components for me",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1074626,
      "author_name": "zaharch",
      "author_url": "",
      "post_date": "11/10/2020 22:02:10",
      "content": "<p>Good question. In my experience in this competition I did not notice any difference between recreating the optimizer each epoch and reusing the same one. But maybe the difference is just small.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1075666,
      "author_name": "doanquanvietnamca",
      "author_url": "",
      "post_date": "11/11/2020 21:38:17",
      "content": "<p>yeah. i load the  optimizer too. i use pytorch_lightning so it  automatic  load all components for me</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1074613": "Hey everybody!\n\nDo you think it is necessary to save and load the `optimizer`? I've seen a lot of pipelines that only store the model weights. But it seems to me that the optimizer has some value in it.\n\nI am using `Adam`. And i'm using Colab Pro to training (1 session something like 15-20 hours). \n\nI am using the resnet family of models (`18`, `34`, `50`, `101`). In past competitions, I did not pay so much attention to this, perhaps this is my mistake.\n\nI would be glad to any advice.",
    "1074626": "Good question. In my experience in this competition I did not notice any difference between recreating the optimizer each epoch and reusing the same one. But maybe the difference is just small.",
    "1075666": "yeah. i load the  optimizer too. i use pytorch_lightning so it  automatic  load all components for me"
  },
  "source": "meta"
}