{
  "id": 189516,
  "title": "Time Machine Rule",
  "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/189516",
  "author_name": "",
  "post_date": "2020-10-07T20:37:54.404008800Z",
  "votes": 18,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I started to check the <a href=\"https://www.kaggle.com/gaborfodor/train-valid-test-split\" target=\"_blank\">train-valid-test split</a>. As far as I understood we have the same route for all vehicles so it is natural that we have the same location distribution for train/test split.</p>\n<p>The time split was not that trivial.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2F4c5bc20b14495efc2744b107840d0b4c%2FScreenshot%202020-10-07%20at%2022.06.52.png?generation=1602103124538951&amp;alt=media\" alt=\"\"></p>\n<p>The valid and test datasets were created with <code>create_chopped_dataset</code> <br>\nIt should make sure that for the same scene we don't have any leakage regarding the next 5 seconds.</p>\n<p>As I understood the time machine rule was forced based on scenes/hosts so theoretically if another host is close at the same time we might have some (non trivial) overlap between the train and test set.</p>\n<p>In theory we could use leak future weather/traffic info but I am not sure how much would that help given the short forecast horizon.</p>\n<p>Am I right?</p>",
  "messages": [
    {
      "id": "1041595",
      "postDate": "10/07/2020 20:37:54",
      "content": "<p>I started to check the <a href=\"https://www.kaggle.com/gaborfodor/train-valid-test-split\" target=\"_blank\">train-valid-test split</a>. As far as I understood we have the same route for all vehicles so it is natural that we have the same location distribution for train/test split.</p>\n<p>The time split was not that trivial.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2F4c5bc20b14495efc2744b107840d0b4c%2FScreenshot%202020-10-07%20at%2022.06.52.png?generation=1602103124538951&amp;alt=media\" alt=\"\"></p>\n<p>The valid and test datasets were created with <code>create_chopped_dataset</code> <br>\nIt should make sure that for the same scene we don't have any leakage regarding the next 5 seconds.</p>\n<p>As I understood the time machine rule was forced based on scenes/hosts so theoretically if another host is close at the same time we might have some (non trivial) overlap between the train and test set.</p>\n<p>In theory we could use leak future weather/traffic info but I am not sure how much would that help given the short forecast horizon.</p>\n<p>Am I right?</p>",
      "rawMarkdown": "I started to check the [train-valid-test split](https://www.kaggle.com/gaborfodor/train-valid-test-split). As far as I understood we have the same route for all vehicles so it is natural that we have the same location distribution for train/test split.\n\nThe time split was not that trivial.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2F4c5bc20b14495efc2744b107840d0b4c%2FScreenshot%202020-10-07%20at%2022.06.52.png?generation=1602103124538951&alt=media)\n\nThe valid and test datasets were created with `create_chopped_dataset` \nIt should make sure that for the same scene we don't have any leakage regarding the next 5 seconds.\n\nAs I understood the time machine rule was forced based on scenes/hosts so theoretically if another host is close at the same time we might have some (non trivial) overlap between the train and test set.\n\nIn theory we could use leak future weather/traffic info but I am not sure how much would that help given the short forecast horizon.\n\nAm I right?",
      "votes": null
    },
    {
      "id": "1045654",
      "postDate": "10/10/2020 21:14:53",
      "content": "<p>Updated with <code>train_full.zarr</code> <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2Fe6ecc3d0b6e3ba52dceed2c4a62dbc14%2FScreenshot%202020-10-10%20at%2023.13.54.png?generation=1602364450085971&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "Updated with `train_full.zarr` ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2Fe6ecc3d0b6e3ba52dceed2c4a62dbc14%2FScreenshot%202020-10-10%20at%2023.13.54.png?generation=1602364450085971&alt=media)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1045654,
      "author_name": "gaborfodor",
      "author_url": "",
      "post_date": "10/10/2020 21:14:53",
      "content": "<p>Updated with <code>train_full.zarr</code> <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2Fe6ecc3d0b6e3ba52dceed2c4a62dbc14%2FScreenshot%202020-10-10%20at%2023.13.54.png?generation=1602364450085971&amp;alt=media\" alt=\"\"></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1041595": "I started to check the [train-valid-test split](https://www.kaggle.com/gaborfodor/train-valid-test-split). As far as I understood we have the same route for all vehicles so it is natural that we have the same location distribution for train/test split.\n\nThe time split was not that trivial.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2F4c5bc20b14495efc2744b107840d0b4c%2FScreenshot%202020-10-07%20at%2022.06.52.png?generation=1602103124538951&alt=media)\n\nThe valid and test datasets were created with `create_chopped_dataset` \nIt should make sure that for the same scene we don't have any leakage regarding the next 5 seconds.\n\nAs I understood the time machine rule was forced based on scenes/hosts so theoretically if another host is close at the same time we might have some (non trivial) overlap between the train and test set.\n\nIn theory we could use leak future weather/traffic info but I am not sure how much would that help given the short forecast horizon.\n\nAm I right?",
    "1045654": "Updated with `train_full.zarr` ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18102%2Fe6ecc3d0b6e3ba52dceed2c4a62dbc14%2FScreenshot%202020-10-10%20at%2023.13.54.png?generation=1602364450085971&alt=media)"
  },
  "source": "meta"
}