{
  "id": 21552,
  "title": "Starter Script for using the pre-trained inception v3 model + tensorflow. LB score of ~ 1.2",
  "url": "/competitions/state-farm-distracted-driver-detection/discussion/21552",
  "author_name": "ckleban",
  "post_date": "2016-06-09T18:21:06.703000",
  "votes": 2,
  "comment_count": 0,
  "views": 2873,
  "content": "<p>Hello,</p>\n\n<p>I wanted to try out transfer learning and tensorflow on this, so I put together these scripts to do so. They are a modification of both tensorflow's sample scripts as well as ZFTurbo's starter scripts. </p>\n\n<p>What they do:</p>\n\n<ul>\n<li>Split the training data up by driver into train and CV data sets so that you can tell if you are overfitting or not</li>\n<li>Has an optional script to create more training images with distortions (meant to improve overfitting but I haven't played with this enough to make a difference)</li>\n<li>Uses the inception v3 pre-trainned model as a base for the transfer learning</li>\n<li>Retrains this model with the distracted driver data</li>\n<li>Make predictions and creates a file ready for submission. </li>\n</ul>\n\n<p>I used this to get a LB score of about 1.2. This works on a machine with a CPU and can be done in less than a day. (GPU is faster of course). Right now, the biggest issue is overfitting. To improve the score of this script, I'd focus on solving the overfitting problem. </p>\n\n<p>To see the scripts, howto and more, check out this repo: <a href=\"https://github.com/ckleban/kaggle-distracted-drivers-inceptionv3\">https://github.com/ckleban/kaggle-distracted-drivers-inceptionv3</a></p>\n\n<p>For a relevant doc on transfer learning in tensorflow: \n<a href=\"https://www.tensorflow.org/versions/r0.9/how_tos/image_retraining/index.html\">https://www.tensorflow.org/versions/r0.9/how_tos/image_retraining/index.html</a></p>\n\n<p>Thanks\nChris</p>",
  "messages": [
    {
      "id": 123104,
      "postDate": "2016-06-09T18:21:06.703Z",
      "content": "<p>Hello,</p>\n\n<p>I wanted to try out transfer learning and tensorflow on this, so I put together these scripts to do so. They are a modification of both tensorflow's sample scripts as well as ZFTurbo's starter scripts. </p>\n\n<p>What they do:</p>\n\n<ul>\n<li>Split the training data up by driver into train and CV data sets so that you can tell if you are overfitting or not</li>\n<li>Has an optional script to create more training images with distortions (meant to improve overfitting but I haven't played with this enough to make a difference)</li>\n<li>Uses the inception v3 pre-trainned model as a base for the transfer learning</li>\n<li>Retrains this model with the distracted driver data</li>\n<li>Make predictions and creates a file ready for submission. </li>\n</ul>\n\n<p>I used this to get a LB score of about 1.2. This works on a machine with a CPU and can be done in less than a day. (GPU is faster of course). Right now, the biggest issue is overfitting. To improve the score of this script, I'd focus on solving the overfitting problem. </p>\n\n<p>To see the scripts, howto and more, check out this repo: <a href=\"https://github.com/ckleban/kaggle-distracted-drivers-inceptionv3\">https://github.com/ckleban/kaggle-distracted-drivers-inceptionv3</a></p>\n\n<p>For a relevant doc on transfer learning in tensorflow: \n<a href=\"https://www.tensorflow.org/versions/r0.9/how_tos/image_retraining/index.html\">https://www.tensorflow.org/versions/r0.9/how_tos/image_retraining/index.html</a></p>\n\n<p>Thanks\nChris</p>",
      "rawMarkdown": "Hello,\r\n\r\nI wanted to try out transfer learning and tensorflow on this, so I put together these scripts to do so. They are a modification of both tensorflow's sample scripts as well as ZFTurbo's starter scripts. \r\n\r\nWhat they do:\r\n\r\n- Split the training data up by driver into train and CV data sets so that you can tell if you are overfitting or not\r\n- Has an optional script to create more training images with distortions (meant to improve overfitting but I haven't played with this enough to make a difference)\r\n- Uses the inception v3 pre-trainned model as a base for the transfer learning\r\n- Retrains this model with the distracted driver data\r\n- Make predictions and creates a file ready for submission. \r\n\r\nI used this to get a LB score of about 1.2. This works on a machine with a CPU and can be done in less than a day. (GPU is faster of course). Right now, the biggest issue is overfitting. To improve the score of this script, I'd focus on solving the overfitting problem. \r\n\r\nTo see the scripts, howto and more, check out this repo: https://github.com/ckleban/kaggle-distracted-drivers-inceptionv3\r\n\r\nFor a relevant doc on transfer learning in tensorflow: \r\nhttps://www.tensorflow.org/versions/r0.9/how_tos/image_retraining/index.html\r\n\r\n\r\nThanks\r\nChris\r\n",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "123104": "Hello,\r\n\r\nI wanted to try out transfer learning and tensorflow on this, so I put together these scripts to do so. They are a modification of both tensorflow's sample scripts as well as ZFTurbo's starter scripts. \r\n\r\nWhat they do:\r\n\r\n- Split the training data up by driver into train and CV data sets so that you can tell if you are overfitting or not\r\n- Has an optional script to create more training images with distortions (meant to improve overfitting but I haven't played with this enough to make a difference)\r\n- Uses the inception v3 pre-trainned model as a base for the transfer learning\r\n- Retrains this model with the distracted driver data\r\n- Make predictions and creates a file ready for submission. \r\n\r\nI used this to get a LB score of about 1.2. This works on a machine with a CPU and can be done in less than a day. (GPU is faster of course). Right now, the biggest issue is overfitting. To improve the score of this script, I'd focus on solving the overfitting problem. \r\n\r\nTo see the scripts, howto and more, check out this repo: https://github.com/ckleban/kaggle-distracted-drivers-inceptionv3\r\n\r\nFor a relevant doc on transfer learning in tensorflow: \r\nhttps://www.tensorflow.org/versions/r0.9/how_tos/image_retraining/index.html\r\n\r\n\r\nThanks\r\nChris\r\n"
  }
}