{
  "id": 20420,
  "title": "MatConvNet",
  "url": "/competitions/state-farm-distracted-driver-detection/discussion/20420",
  "author_name": "",
  "post_date": "2016-04-25T18:17:01.813Z",
  "votes": null,
  "comment_count": 2,
  "views": 561,
  "content": "<p>Is anybody using MatConvNet for this competition.</p>\n\n<p>I am using this CNN toolbox which works very well.</p>\n\n<p>Using the training images split between 80% training and 20% validation I get very good results. See attached plots. Validation error is less than 0.1.</p>\n\n<p>But when I run the trained network against the test data I get terrible results (7.95484).</p>\n\n<p>Does anybody have any ideas, I'm not sure if I am running the test images correctly?</p>\n\n<p>Lorenzo</p>",
  "messages": [
    {
      "id": "116716",
      "postDate": "04/25/2016 18:17:01",
      "content": "<p>Is anybody using MatConvNet for this competition.</p>\n\n<p>I am using this CNN toolbox which works very well.</p>\n\n<p>Using the training images split between 80% training and 20% validation I get very good results. See attached plots. Validation error is less than 0.1.</p>\n\n<p>But when I run the trained network against the test data I get terrible results (7.95484).</p>\n\n<p>Does anybody have any ideas, I'm not sure if I am running the test images correctly?</p>\n\n<p>Lorenzo</p>",
      "rawMarkdown": "Is anybody using MatConvNet for this competition.\r\n\r\nI am using this CNN toolbox which works very well.\r\n\r\nUsing the training images split between 80% training and 20% validation I get very good results. See attached plots. Validation error is less than 0.1.\r\n\r\nBut when I run the trained network against the test data I get terrible results (7.95484).\r\n\r\nDoes anybody have any ideas, I'm not sure if I am running the test images correctly?\r\n\r\nLorenzo",
      "votes": null
    },
    {
      "id": "116721",
      "postDate": "04/25/2016 19:01:18",
      "content": "<p>Probably you are over-fitting, and you should aim for a higher training error - splitting CV by driver id will give you much closer measure of generalisation to the test data. If you split randomly, you will get what appears to be very low CV, but in fact you have a network that cannot generalise to new drivers. All the drivers in the test set are different from drivers in train.</p>",
      "rawMarkdown": "Probably you are over-fitting, and you should aim for a higher training error - splitting CV by driver id will give you much closer measure of generalisation to the test data. If you split randomly, you will get what appears to be very low CV, but in fact you have a network that cannot generalise to new drivers. All the drivers in the test set are different from drivers in train.",
      "votes": null
    },
    {
      "id": "116909",
      "postDate": "04/26/2016 14:27:51",
      "content": "<p>Thanks that seems to have helped produce more realistic test results.</p>",
      "rawMarkdown": "Thanks that seems to have helped produce more realistic test results.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 116721,
      "author_name": "slobo777",
      "author_url": "",
      "post_date": "04/25/2016 19:01:18",
      "content": "<p>Probably you are over-fitting, and you should aim for a higher training error - splitting CV by driver id will give you much closer measure of generalisation to the test data. If you split randomly, you will get what appears to be very low CV, but in fact you have a network that cannot generalise to new drivers. All the drivers in the test set are different from drivers in train.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 116909,
      "author_name": "lorenzol",
      "author_url": "",
      "post_date": "04/26/2016 14:27:51",
      "content": "<p>Thanks that seems to have helped produce more realistic test results.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "116716": "Is anybody using MatConvNet for this competition.\r\n\r\nI am using this CNN toolbox which works very well.\r\n\r\nUsing the training images split between 80% training and 20% validation I get very good results. See attached plots. Validation error is less than 0.1.\r\n\r\nBut when I run the trained network against the test data I get terrible results (7.95484).\r\n\r\nDoes anybody have any ideas, I'm not sure if I am running the test images correctly?\r\n\r\nLorenzo",
    "116721": "Probably you are over-fitting, and you should aim for a higher training error - splitting CV by driver id will give you much closer measure of generalisation to the test data. If you split randomly, you will get what appears to be very low CV, but in fact you have a network that cannot generalise to new drivers. All the drivers in the test set are different from drivers in train.",
    "116909": "Thanks that seems to have helped produce more realistic test results."
  },
  "source": "meta"
}