{
  "id": 105781,
  "title": "weird cross  validation ",
  "url": "/competitions/aptos2019-blindness-detection/discussion/105781",
  "author_name": "",
  "post_date": "2019-08-26T12:02:23.388632Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I split Train data into 4 validation set. (by using statified k fold in sklearn)</p>\n\n<p>On the training for the first 3 validation set, validation loss converges appropriately.\nBut On the training for the final validation set, validation loss diverges on the first epoch.</p>\n\n<p>I can't find any reason for this phenomenon.</p>\n\n<p>I can acheive LB 0.809  for those 3 model, and now i think  i can get higher score with 4 model.</p>\n\n<p>I changed the random state of the split function several times, but the results are always same.</p>\n\n<p>What is the problem? </p>",
  "messages": [
    {
      "id": "608145",
      "postDate": "08/26/2019 12:02:23",
      "content": "<p>I split Train data into 4 validation set. (by using statified k fold in sklearn)</p>\n\n<p>On the training for the first 3 validation set, validation loss converges appropriately.\nBut On the training for the final validation set, validation loss diverges on the first epoch.</p>\n\n<p>I can't find any reason for this phenomenon.</p>\n\n<p>I can acheive LB 0.809  for those 3 model, and now i think  i can get higher score with 4 model.</p>\n\n<p>I changed the random state of the split function several times, but the results are always same.</p>\n\n<p>What is the problem? </p>",
      "rawMarkdown": "I split Train data into 4 validation set. (by using statified k fold in sklearn)\n\nOn the training for the first 3 validation set, validation loss converges appropriately.\nBut On the training for the final validation set, validation loss diverges on the first epoch.\n\nI can't find any reason for this phenomenon.\n \nI can acheive LB 0.809  for those 3 model, and now i think  i can get higher score with 4 model.\n\nI changed the random state of the split function several times, but the results are always same.\n\nWhat is the problem?",
      "votes": null
    },
    {
      "id": "608150",
      "postDate": "08/26/2019 12:06:07",
      "content": "<p>First thing that I would do is train longer than 1 epoch and see how the loss changes</p>",
      "rawMarkdown": "First thing that I would do is train longer than 1 epoch and see how the loss changes",
      "votes": null
    },
    {
      "id": "608278",
      "postDate": "08/26/2019 15:26:39",
      "content": "<p>I've already watched ,but the val loss go upward while train loss does not.</p>\n\n<p>I pretrained with 2015 data(with validation data 2019) and finetuned on 2019 data(4fold cross validation)</p>\n\n<p>Is it possible that above pretrained model can fit on only 3/4 of validation set???? </p>\n\n<p>It's so weird .. </p>",
      "rawMarkdown": "I've already watched ,but the val loss go upward while train loss does not.\n\nI pretrained with 2015 data(with validation data 2019) and finetuned on 2019 data(4fold cross validation)\n\nIs it possible that above pretrained model can fit on only 3/4 of validation set???? \n\nIt's so weird ..",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 608150,
      "author_name": "bibek777",
      "author_url": "",
      "post_date": "08/26/2019 12:06:07",
      "content": "<p>First thing that I would do is train longer than 1 epoch and see how the loss changes</p>",
      "votes": null,
      "replies": [
        {
          "id": 608278,
          "author_name": "woongjo",
          "author_url": "",
          "post_date": "08/26/2019 15:26:39",
          "content": "<p>I've already watched ,but the val loss go upward while train loss does not.</p>\n\n<p>I pretrained with 2015 data(with validation data 2019) and finetuned on 2019 data(4fold cross validation)</p>\n\n<p>Is it possible that above pretrained model can fit on only 3/4 of validation set???? </p>\n\n<p>It's so weird .. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "608145": "I split Train data into 4 validation set. (by using statified k fold in sklearn)\n\nOn the training for the first 3 validation set, validation loss converges appropriately.\nBut On the training for the final validation set, validation loss diverges on the first epoch.\n\nI can't find any reason for this phenomenon.\n \nI can acheive LB 0.809  for those 3 model, and now i think  i can get higher score with 4 model.\n\nI changed the random state of the split function several times, but the results are always same.\n\nWhat is the problem?",
    "608150": "First thing that I would do is train longer than 1 epoch and see how the loss changes",
    "608278": "I've already watched ,but the val loss go upward while train loss does not.\n\nI pretrained with 2015 data(with validation data 2019) and finetuned on 2019 data(4fold cross validation)\n\nIs it possible that above pretrained model can fit on only 3/4 of validation set???? \n\nIt's so weird .."
  },
  "source": "meta"
}