{
  "id": 20740,
  "title": "leaky cross validation in this competition?",
  "url": "/competitions/expedia-hotel-recommendations/discussion/20740",
  "author_name": "",
  "post_date": "2016-05-05T17:06:16.837Z",
  "votes": null,
  "comment_count": 2,
  "views": 623,
  "content": "<p>Hi,\nIt's known that there is a data leak.\nMy question is, given that fact, how predictive is cross validation map@5 to the score with regards to the test.csv?\nTo my understanding, in order for it to be predictive, a data leak needs to be present in the train.csv - between future and past results.\nIs that the case?</p>",
  "messages": [
    {
      "id": "118846",
      "postDate": "05/05/2016 17:06:16",
      "content": "<p>Hi,\nIt's known that there is a data leak.\nMy question is, given that fact, how predictive is cross validation map@5 to the score with regards to the test.csv?\nTo my understanding, in order for it to be predictive, a data leak needs to be present in the train.csv - between future and past results.\nIs that the case?</p>",
      "rawMarkdown": "Hi,\r\nIt's known that there is a data leak.\r\nMy question is, given that fact, how predictive is cross validation map@5 to the score with regards to the test.csv?\r\nTo my understanding, in order for it to be predictive, a data leak needs to be present in the train.csv - between future and past results.\r\nIs that the case?",
      "votes": null
    },
    {
      "id": "119016",
      "postDate": "05/06/2016 18:22:29",
      "content": "<p>From the notebook posted elsewhere, a holdout set with the last 4 months of the training data comes pretty close to the LB result (if you use all the available data that is) even when using the leak, so I don't think there will be anything too wild ala Santander, thank goodness...</p>",
      "rawMarkdown": "From the notebook posted elsewhere, a holdout set with the last 4 months of the training data comes pretty close to the LB result (if you use all the available data that is) even when using the leak, so I don't think there will be anything too wild ala Santander, thank goodness...",
      "votes": null
    },
    {
      "id": "119044",
      "postDate": "05/06/2016 22:15:07",
      "content": "<p>[quote=happycube;119016]</p>\n\n<p>... so I don't think there will be anything too wild ala Santander, thank goodness...</p>\n\n<p>[/quote]</p>\n\n<p>May the god of overfitting hear your prayers... ;)</p>",
      "rawMarkdown": "[quote=happycube;119016]\r\n\r\n... so I don't think there will be anything too wild ala Santander, thank goodness...\r\n\r\n[/quote]\r\n\r\nMay the god of overfitting hear your prayers... ;)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 119016,
      "author_name": "happycube",
      "author_url": "",
      "post_date": "05/06/2016 18:22:29",
      "content": "<p>From the notebook posted elsewhere, a holdout set with the last 4 months of the training data comes pretty close to the LB result (if you use all the available data that is) even when using the leak, so I don't think there will be anything too wild ala Santander, thank goodness...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 119044,
      "author_name": "carrdelling",
      "author_url": "",
      "post_date": "05/06/2016 22:15:07",
      "content": "<p>[quote=happycube;119016]</p>\n\n<p>... so I don't think there will be anything too wild ala Santander, thank goodness...</p>\n\n<p>[/quote]</p>\n\n<p>May the god of overfitting hear your prayers... ;)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "118846": "Hi,\r\nIt's known that there is a data leak.\r\nMy question is, given that fact, how predictive is cross validation map@5 to the score with regards to the test.csv?\r\nTo my understanding, in order for it to be predictive, a data leak needs to be present in the train.csv - between future and past results.\r\nIs that the case?",
    "119016": "From the notebook posted elsewhere, a holdout set with the last 4 months of the training data comes pretty close to the LB result (if you use all the available data that is) even when using the leak, so I don't think there will be anything too wild ala Santander, thank goodness...",
    "119044": "[quote=happycube;119016]\r\n\r\n... so I don't think there will be anything too wild ala Santander, thank goodness...\r\n\r\n[/quote]\r\n\r\nMay the god of overfitting hear your prayers... ;)"
  },
  "source": "meta"
}