{
  "id": 21369,
  "title": "How far apart are the train/test sets sampled?",
  "url": "/competitions/avito-duplicate-ads-detection/discussion/21369",
  "author_name": "",
  "post_date": "2016-06-01T21:16:56.387Z",
  "votes": 1,
  "comment_count": 2,
  "views": 640,
  "content": "<p>On the data page, it states:</p>\n\n<blockquote>\n  <p>To ensure that winning models are robust enough to generalize to new\n  duplicate cases, the train and test datasets are sampled from\n  different time intervals. Hence, you may see different distributions\n  of duplicates in train/test datasets.</p>\n</blockquote>\n\n<p>I was wondering how far apart these sets were sampled - are they days, months or years apart? Or were they sampled at different times during the same time period?</p>\n\n<p>Also, I was wondering whether the test set was sampled before or after the training set in time.</p>\n\n<p>Hopefully these questions are something an Admin can answer.</p>\n\n<p>Thanks,\nanokas.</p>",
  "messages": [
    {
      "id": "122183",
      "postDate": "06/01/2016 21:16:56",
      "content": "<p>On the data page, it states:</p>\n\n<blockquote>\n  <p>To ensure that winning models are robust enough to generalize to new\n  duplicate cases, the train and test datasets are sampled from\n  different time intervals. Hence, you may see different distributions\n  of duplicates in train/test datasets.</p>\n</blockquote>\n\n<p>I was wondering how far apart these sets were sampled - are they days, months or years apart? Or were they sampled at different times during the same time period?</p>\n\n<p>Also, I was wondering whether the test set was sampled before or after the training set in time.</p>\n\n<p>Hopefully these questions are something an Admin can answer.</p>\n\n<p>Thanks,\nanokas.</p>",
      "rawMarkdown": "On the data page, it states:\r\n\r\n> To ensure that winning models are robust enough to generalize to new\r\n> duplicate cases, the train and test datasets are sampled from\r\n> different time intervals. Hence, you may see different distributions\r\n> of duplicates in train/test datasets.\r\n\r\nI was wondering how far apart these sets were sampled - are they days, months or years apart? Or were they sampled at different times during the same time period?\r\n\r\nAlso, I was wondering whether the test set was sampled before or after the training set in time.\r\n\r\nHopefully these questions are something an Admin can answer.\r\n\r\nThanks,\r\nanokas.",
      "votes": null
    },
    {
      "id": "122217",
      "postDate": "06/02/2016 05:05:32",
      "content": "<p>Sampling was done from a 2 consequent periods. Each period is several months. Test goes after train.</p>",
      "rawMarkdown": "Sampling was done from a 2 consequent periods. Each period is several months. Test goes after train.",
      "votes": null
    },
    {
      "id": "122910",
      "postDate": "06/08/2016 08:35:48",
      "content": "<p>and can we assume that inside the train set the items are also sorted from oldest to newest ?</p>",
      "rawMarkdown": "and can we assume that inside the train set the items are also sorted from oldest to newest ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 122217,
      "author_name": "ivanguz",
      "author_url": "",
      "post_date": "06/02/2016 05:05:32",
      "content": "<p>Sampling was done from a 2 consequent periods. Each period is several months. Test goes after train.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 122910,
      "author_name": "samehif",
      "author_url": "",
      "post_date": "06/08/2016 08:35:48",
      "content": "<p>and can we assume that inside the train set the items are also sorted from oldest to newest ?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "122183": "On the data page, it states:\r\n\r\n> To ensure that winning models are robust enough to generalize to new\r\n> duplicate cases, the train and test datasets are sampled from\r\n> different time intervals. Hence, you may see different distributions\r\n> of duplicates in train/test datasets.\r\n\r\nI was wondering how far apart these sets were sampled - are they days, months or years apart? Or were they sampled at different times during the same time period?\r\n\r\nAlso, I was wondering whether the test set was sampled before or after the training set in time.\r\n\r\nHopefully these questions are something an Admin can answer.\r\n\r\nThanks,\r\nanokas.",
    "122217": "Sampling was done from a 2 consequent periods. Each period is several months. Test goes after train.",
    "122910": "and can we assume that inside the train set the items are also sorted from oldest to newest ?"
  },
  "source": "meta"
}