{
  "id": 322982,
  "title": "Training Loss Fluctuating",
  "url": "/competitions/sorghum-id-fgvc-9/discussion/322982",
  "author_name": "",
  "post_date": "2022-05-04T14:12:21.370832700Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello everypne,<br>\nCan any one suggest why training loss decreases with each batch and then increases again and worsens .<br>\nEach epoch my train loss goes down and then starts to go up as I iterate through each batch.</p>\n<p>I am using, the Adam optimizer , lr=1e-4,batch =16 , model=efficient net,</p>",
  "messages": [
    {
      "id": "1777275",
      "postDate": "05/04/2022 14:12:21",
      "content": "<p>Hello everypne,<br>\nCan any one suggest why training loss decreases with each batch and then increases again and worsens .<br>\nEach epoch my train loss goes down and then starts to go up as I iterate through each batch.</p>\n<p>I am using, the Adam optimizer , lr=1e-4,batch =16 , model=efficient net,</p>",
      "rawMarkdown": "Hello everypne,\nCan any one suggest why training loss decreases with each batch and then increases again and worsens .\nEach epoch my train loss goes down and then starts to go up as I iterate through each batch.\n\nI am using, the Adam optimizer , lr=1e-4,batch =16 , model=efficient net,",
      "votes": null
    },
    {
      "id": "1778271",
      "postDate": "05/05/2022 07:46:12",
      "content": "<p>This could be due to the optimizer getting stuck in a local minimum. Try a different optimizer or change your learning rate.</p>",
      "rawMarkdown": "This could be due to the optimizer getting stuck in a local minimum. Try a different optimizer or change your learning rate.",
      "votes": null
    },
    {
      "id": "1793818",
      "postDate": "05/18/2022 08:49:44",
      "content": "<p>Try SGD with lr=1e-3 or larger</p>",
      "rawMarkdown": "Try SGD with lr=1e-3 or larger",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1778271,
      "author_name": "",
      "author_url": "",
      "post_date": "05/05/2022 07:46:12",
      "content": "<p>This could be due to the optimizer getting stuck in a local minimum. Try a different optimizer or change your learning rate.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1793818,
      "author_name": "aoskdu",
      "author_url": "",
      "post_date": "05/18/2022 08:49:44",
      "content": "<p>Try SGD with lr=1e-3 or larger</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1777275": "Hello everypne,\nCan any one suggest why training loss decreases with each batch and then increases again and worsens .\nEach epoch my train loss goes down and then starts to go up as I iterate through each batch.\n\nI am using, the Adam optimizer , lr=1e-4,batch =16 , model=efficient net,",
    "1778271": "This could be due to the optimizer getting stuck in a local minimum. Try a different optimizer or change your learning rate.",
    "1793818": "Try SGD with lr=1e-3 or larger"
  },
  "source": "meta"
}