{
  "id": 22489,
  "title": "Overfitting the test set",
  "url": "/competitions/state-farm-distracted-driver-detection/discussion/22489",
  "author_name": "",
  "post_date": "2016-07-25T18:31:35.047Z",
  "votes": null,
  "comment_count": 2,
  "views": 620,
  "content": "<p>I'm disappointed that the administrators of this contest decided to allow competitors to use semi-supervised methods that include the test set data. The contest now appears to be about using algorithms that are good at overfitting the test set rather than methods that generalize well and I'm guessing that the winning methods won't be useful in a general sense. It would have been much more interesting if the contest administrators had provided the videos used to generate the training data frame grabs and prohibited the use of test data for training. Hopefully, this will be a consideration in future contests.</p>",
  "messages": [
    {
      "id": "128991",
      "postDate": "07/25/2016 18:31:35",
      "content": "<p>I'm disappointed that the administrators of this contest decided to allow competitors to use semi-supervised methods that include the test set data. The contest now appears to be about using algorithms that are good at overfitting the test set rather than methods that generalize well and I'm guessing that the winning methods won't be useful in a general sense. It would have been much more interesting if the contest administrators had provided the videos used to generate the training data frame grabs and prohibited the use of test data for training. Hopefully, this will be a consideration in future contests.</p>",
      "rawMarkdown": "I'm disappointed that the administrators of this contest decided to allow competitors to use semi-supervised methods that include the test set data. The contest now appears to be about using algorithms that are good at overfitting the test set rather than methods that generalize well and I'm guessing that the winning methods won't be useful in a general sense. It would have been much more interesting if the contest administrators had provided the videos used to generate the training data frame grabs and prohibited the use of test data for training. Hopefully, this will be a consideration in future contests.",
      "votes": null
    },
    {
      "id": "129578",
      "postDate": "07/31/2016 11:50:58",
      "content": "<p>Hi SpammySmith,</p>\n\n<p>In my opinion, test image should not be processed in batch mode.<br>\nIn some situation, new-test image may be single image.</p>",
      "rawMarkdown": "Hi SpammySmith,\r\n\r\nIn my opinion, test image should not be processed in batch mode.<br>\r\nIn some situation, new-test image may be single image.",
      "votes": null
    },
    {
      "id": "129604",
      "postDate": "07/31/2016 22:45:28",
      "content": "<p>@SpammySmith</p>\n\n<p>Do you mean add labels to test data manually, and used them for training?</p>",
      "rawMarkdown": "SpammySmith\r\n\r\nDo you mean add labels to test data manually, and used them for training?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 129578,
      "author_name": "toshik",
      "author_url": "",
      "post_date": "07/31/2016 11:50:58",
      "content": "<p>Hi SpammySmith,</p>\n\n<p>In my opinion, test image should not be processed in batch mode.<br>\nIn some situation, new-test image may be single image.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 129604,
      "author_name": "ferris",
      "author_url": "",
      "post_date": "07/31/2016 22:45:28",
      "content": "<p>@SpammySmith</p>\n\n<p>Do you mean add labels to test data manually, and used them for training?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "128991": "I'm disappointed that the administrators of this contest decided to allow competitors to use semi-supervised methods that include the test set data. The contest now appears to be about using algorithms that are good at overfitting the test set rather than methods that generalize well and I'm guessing that the winning methods won't be useful in a general sense. It would have been much more interesting if the contest administrators had provided the videos used to generate the training data frame grabs and prohibited the use of test data for training. Hopefully, this will be a consideration in future contests.",
    "129578": "Hi SpammySmith,\r\n\r\nIn my opinion, test image should not be processed in batch mode.<br>\r\nIn some situation, new-test image may be single image.",
    "129604": "SpammySmith\r\n\r\nDo you mean add labels to test data manually, and used them for training?"
  },
  "source": "meta"
}