{
  "id": 49095,
  "title": "Anyone else with a model with val_accuracy 99%",
  "url": "/competitions/sp-society-camera-model-identification/discussion/49095",
  "author_name": "",
  "post_date": "2018-02-06T22:41:41.424536300Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Is there anyone else with a model which reaches val accuracy of 99% but LB is less than 95%?</p>",
  "messages": [
    {
      "id": "278757",
      "postDate": "02/06/2018 22:41:41",
      "content": "<p>Is there anyone else with a model which reaches val accuracy of 99% but LB is less than 95%?</p>",
      "rawMarkdown": "Is there anyone else with a model which reaches val accuracy of 99% but LB is less than 95%?",
      "votes": null
    },
    {
      "id": "279221",
      "postDate": "02/07/2018 16:01:54",
      "content": "<p>Well, I guess you have seen this thread (<a href=\"https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49114\">https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49114</a>) but it is about the val_acc - LB differences. Not sure if it helps so close to the end of the competition but here it is anyway :) .</p>",
      "rawMarkdown": "Well, I guess you have seen this thread (https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49114) but it is about the val_acc - LB differences. Not sure if it helps so close to the end of the competition but here it is anyway :) .",
      "votes": null
    },
    {
      "id": "279229",
      "postDate": "02/07/2018 16:22:18",
      "content": "<p>Hi @Vasili, yes I am aware of that but what strikes me as odd is the fact that I have another checkpoint of the same model which reaches val. acc. 94% but on LB gets 95% and the same model trained for longer time can get val. acc. 99% but on LB it reaches 93%.</p>",
      "rawMarkdown": "Hi @Vasili, yes I am aware of that but what strikes me as odd is the fact that I have another checkpoint of the same model which reaches val. acc. 94% but on LB gets 95% and the same model trained for longer time can get val. acc. 99% but on LB it reaches 93%.",
      "votes": null
    },
    {
      "id": "279235",
      "postDate": "02/07/2018 16:30:00",
      "content": "<p>Yeah, tbh different checkpoints of the same model give unexpectedly different ls results for me as well. Could it be that you over-tune your model depending on the specific validation set and in the end this validation set is not that representative (maybe the altered images are responsible for that)?  </p>",
      "rawMarkdown": "Yeah, tbh different checkpoints of the same model give unexpectedly different ls results for me as well. Could it be that you over-tune your model depending on the specific validation set and in the end this validation set is not that representative (maybe the altered images are responsible for that)?",
      "votes": null
    },
    {
      "id": "279250",
      "postDate": "02/07/2018 16:49:29",
      "content": "<p>Indeed, I presume it could be from overtuning since I train for longer times but with smaller rates.</p>",
      "rawMarkdown": "Indeed, I presume it could be from overtuning since I train for longer times but with smaller rates.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 279221,
      "author_name": "vmilias",
      "author_url": "",
      "post_date": "02/07/2018 16:01:54",
      "content": "<p>Well, I guess you have seen this thread (<a href=\"https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49114\">https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49114</a>) but it is about the val_acc - LB differences. Not sure if it helps so close to the end of the competition but here it is anyway :) .</p>",
      "votes": null,
      "replies": [
        {
          "id": 279229,
          "author_name": "kirk86",
          "author_url": "",
          "post_date": "02/07/2018 16:22:18",
          "content": "<p>Hi @Vasili, yes I am aware of that but what strikes me as odd is the fact that I have another checkpoint of the same model which reaches val. acc. 94% but on LB gets 95% and the same model trained for longer time can get val. acc. 99% but on LB it reaches 93%.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279235,
          "author_name": "vmilias",
          "author_url": "",
          "post_date": "02/07/2018 16:30:00",
          "content": "<p>Yeah, tbh different checkpoints of the same model give unexpectedly different ls results for me as well. Could it be that you over-tune your model depending on the specific validation set and in the end this validation set is not that representative (maybe the altered images are responsible for that)?  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 279250,
          "author_name": "kirk86",
          "author_url": "",
          "post_date": "02/07/2018 16:49:29",
          "content": "<p>Indeed, I presume it could be from overtuning since I train for longer times but with smaller rates.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "278757": "Is there anyone else with a model which reaches val accuracy of 99% but LB is less than 95%?",
    "279221": "Well, I guess you have seen this thread (https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49114) but it is about the val_acc - LB differences. Not sure if it helps so close to the end of the competition but here it is anyway :) .",
    "279229": "Hi @Vasili, yes I am aware of that but what strikes me as odd is the fact that I have another checkpoint of the same model which reaches val. acc. 94% but on LB gets 95% and the same model trained for longer time can get val. acc. 99% but on LB it reaches 93%.",
    "279235": "Yeah, tbh different checkpoints of the same model give unexpectedly different ls results for me as well. Could it be that you over-tune your model depending on the specific validation set and in the end this validation set is not that representative (maybe the altered images are responsible for that)?",
    "279250": "Indeed, I presume it could be from overtuning since I train for longer times but with smaller rates."
  },
  "source": "meta"
}