{
  "id": 216154,
  "title": "Problems with learning.",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/216154",
  "author_name": "",
  "post_date": "2021-02-01T19:44:39.287470500Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Val accuracy then increases, then falls at the same LR. I.e., let's say the 5th epoch of val accuracy is 0.87, and the next epoch is 0.65. What can this be related to?</p>",
  "messages": [
    {
      "id": "1181336",
      "postDate": "02/01/2021 19:44:39",
      "content": "<p>Val accuracy then increases, then falls at the same LR. I.e., let's say the 5th epoch of val accuracy is 0.87, and the next epoch is 0.65. What can this be related to?</p>",
      "rawMarkdown": "Val accuracy then increases, then falls at the same LR. I.e., let's say the 5th epoch of val accuracy is 0.87, and the next epoch is 0.65. What can this be related to?",
      "votes": null
    },
    {
      "id": "1181358",
      "postDate": "02/01/2021 20:12:00",
      "content": "<p>In short: It means overfitting. </p>\n<p>In more detail: The optimizer is still decreasing the training loss and with it increasing the training accuracy, but at your 5th epoch there is overfitting in place, so while the training accuracy is increasing it is only because the network started memorizing the data and no longer \"learn\" anything really -&gt; that's why the val accuracy started to decrease.</p>",
      "rawMarkdown": "In short: It means overfitting. \n\nIn more detail: The optimizer is still decreasing the training loss and with it increasing the training accuracy, but at your 5th epoch there is overfitting in place, so while the training accuracy is increasing it is only because the network started memorizing the data and no longer \"learn\" anything really -> that's why the val accuracy started to decrease.",
      "votes": null
    },
    {
      "id": "1181367",
      "postDate": "02/01/2021 20:29:40",
      "content": "<p>but at the same time, the accuracy on the next epoch may grow again</p>",
      "rawMarkdown": "but at the same time, the accuracy on the next epoch may grow again",
      "votes": null
    },
    {
      "id": "1181406",
      "postDate": "02/01/2021 21:18:10",
      "content": "<p>Most likely your LR is too high. </p>",
      "rawMarkdown": "Most likely your LR is too high.",
      "votes": null
    },
    {
      "id": "1181532",
      "postDate": "02/02/2021 00:41:39",
      "content": "<p>That's true - but the gap is too much between 0.87 and 0.65, it's not just a little random fluctuation downward, so need to decrease lr before this point.</p>",
      "rawMarkdown": "That's true - but the gap is too much between 0.87 and 0.65, it's not just a little random fluctuation downward, so need to decrease lr before this point.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1181358,
      "author_name": "killimi",
      "author_url": "",
      "post_date": "02/01/2021 20:12:00",
      "content": "<p>In short: It means overfitting. </p>\n<p>In more detail: The optimizer is still decreasing the training loss and with it increasing the training accuracy, but at your 5th epoch there is overfitting in place, so while the training accuracy is increasing it is only because the network started memorizing the data and no longer \"learn\" anything really -&gt; that's why the val accuracy started to decrease.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1181367,
          "author_name": "aristarhbfg",
          "author_url": "",
          "post_date": "02/01/2021 20:29:40",
          "content": "<p>but at the same time, the accuracy on the next epoch may grow again</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1181532,
          "author_name": "killimi",
          "author_url": "",
          "post_date": "02/02/2021 00:41:39",
          "content": "<p>That's true - but the gap is too much between 0.87 and 0.65, it's not just a little random fluctuation downward, so need to decrease lr before this point.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1181406,
      "author_name": "alexxx",
      "author_url": "",
      "post_date": "02/01/2021 21:18:10",
      "content": "<p>Most likely your LR is too high. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1181336": "Val accuracy then increases, then falls at the same LR. I.e., let's say the 5th epoch of val accuracy is 0.87, and the next epoch is 0.65. What can this be related to?",
    "1181358": "In short: It means overfitting. \n\nIn more detail: The optimizer is still decreasing the training loss and with it increasing the training accuracy, but at your 5th epoch there is overfitting in place, so while the training accuracy is increasing it is only because the network started memorizing the data and no longer \"learn\" anything really -> that's why the val accuracy started to decrease.",
    "1181367": "but at the same time, the accuracy on the next epoch may grow again",
    "1181406": "Most likely your LR is too high.",
    "1181532": "That's true - but the gap is too much between 0.87 and 0.65, it's not just a little random fluctuation downward, so need to decrease lr before this point."
  },
  "source": "meta"
}