{
  "id": 68581,
  "title": "Learning to use Tensorflow Object Detection API",
  "url": "/competitions/rsna-pneumonia-detection-challenge/discussion/68581",
  "author_name": "",
  "post_date": "2018-10-14T19:44:44.060325600Z",
  "votes": 5,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Hi All! \nI am trying to learn how to use the Tensorflow Object Detection API.  (tf-API)\n<a href=\"https://github.com/tensorflow/models/tree/master/research/object_detection\">https://github.com/tensorflow/models/tree/master/research/object_detection</a></p>\n\n<p>Full disclosure: <strong>My results so far are terrible with the tf-API</strong> (Mask-RCNN results are better) But it has been a great learning experience. Here is what i had to do to get it to run.</p>\n\n<ol>\n<li>After installing it I run the following bash script (you can run object_detection/model_main.py directly from commandline but i just run it from bash script so i remember the parameters). </li>\n</ol>\n\n<p>!/bin/bash</p>\n\n<p>PIPELINE_CONFIG_PATH='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_pets.config'</p>\n\n<p>MODEL_DIR='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_coco_11_06_2017/'</p>\n\n<p>export NUM_TRAIN_STEPS=20000</p>\n\n<p>python3 object_detection/model_main.py \\\n    --pipeline_config_path=${PIPELINE_CONFIG_PATH} \\\n    --model_dir=${MODEL_DIR} \\\n    --num_train_steps=${NUM_TRAIN_STEPS} \\\n    --sample_1_of_n_eval_examples=$SAMPLE_1_OF_N_EVAL_EXAMPLES \\\n    --alsologtostderr</p>\n\n<ol>\n<li><p>I downloaded the model \" ssd_mobilenet_v1_coco_11_06_2017\" from here: \n<a href=\"https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md\">https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md</a></p></li>\n<li><p>The sample config file is from here: \n<a href=\"https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs\">https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs</a>\nI did not update parameters other than no of classes and \"paths to be configured\"</p></li>\n<li><p>I had to convert the data to tf.Records format; i used this kernel and use 9 shards for training and 1 for evaluation: <a href=\"https://www.kaggle.com/lyonzy/convert-dicom-images-to-tfrecords\">https://www.kaggle.com/lyonzy/convert-dicom-images-to-tfrecords</a></p></li>\n<li>The for prediction on the test data, I modified this script:\n<a href=\"https://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb\">https://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb</a></li>\n</ol>\n\n<p>I will make a git repo soon and commit the code. \nCurious to know if anyone had good results with the tensorflow object detection API. </p>\n\n<p>Happy Kaggling ;-)</p>\n\n<p>Mehul</p>",
  "messages": [
    {
      "id": "403881",
      "postDate": "10/14/2018 19:44:44",
      "content": "<p>Hi All! \nI am trying to learn how to use the Tensorflow Object Detection API.  (tf-API)\n<a href=\"https://github.com/tensorflow/models/tree/master/research/object_detection\">https://github.com/tensorflow/models/tree/master/research/object_detection</a></p>\n\n<p>Full disclosure: <strong>My results so far are terrible with the tf-API</strong> (Mask-RCNN results are better) But it has been a great learning experience. Here is what i had to do to get it to run.</p>\n\n<ol>\n<li>After installing it I run the following bash script (you can run object_detection/model_main.py directly from commandline but i just run it from bash script so i remember the parameters). </li>\n</ol>\n\n<p>!/bin/bash</p>\n\n<p>PIPELINE_CONFIG_PATH='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_pets.config'</p>\n\n<p>MODEL_DIR='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_coco_11_06_2017/'</p>\n\n<p>export NUM_TRAIN_STEPS=20000</p>\n\n<p>python3 object_detection/model_main.py \\\n    --pipeline_config_path=${PIPELINE_CONFIG_PATH} \\\n    --model_dir=${MODEL_DIR} \\\n    --num_train_steps=${NUM_TRAIN_STEPS} \\\n    --sample_1_of_n_eval_examples=$SAMPLE_1_OF_N_EVAL_EXAMPLES \\\n    --alsologtostderr</p>\n\n<ol>\n<li><p>I downloaded the model \" ssd_mobilenet_v1_coco_11_06_2017\" from here: \n<a href=\"https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md\">https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md</a></p></li>\n<li><p>The sample config file is from here: \n<a href=\"https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs\">https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs</a>\nI did not update parameters other than no of classes and \"paths to be configured\"</p></li>\n<li><p>I had to convert the data to tf.Records format; i used this kernel and use 9 shards for training and 1 for evaluation: <a href=\"https://www.kaggle.com/lyonzy/convert-dicom-images-to-tfrecords\">https://www.kaggle.com/lyonzy/convert-dicom-images-to-tfrecords</a></p></li>\n<li>The for prediction on the test data, I modified this script:\n<a href=\"https://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb\">https://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb</a></li>\n</ol>\n\n<p>I will make a git repo soon and commit the code. \nCurious to know if anyone had good results with the tensorflow object detection API. </p>\n\n<p>Happy Kaggling ;-)</p>\n\n<p>Mehul</p>",
      "rawMarkdown": "Hi All! \nI am trying to learn how to use the Tensorflow Object Detection API.  (tf-API)\nhttps://github.com/tensorflow/models/tree/master/research/object_detection\n\nFull disclosure: **My results so far are terrible with the tf-API** (Mask-RCNN results are better) But it has been a great learning experience. Here is what i had to do to get it to run.\n\n1. After installing it I run the following bash script (you can run object_detection/model_main.py directly from commandline but i just run it from bash script so i remember the parameters). \n\n!/bin/bash\n\nPIPELINE_CONFIG_PATH='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_pets.config'\n\nMODEL_DIR='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_coco_11_06_2017/'\n\nexport NUM_TRAIN_STEPS=20000\n\npython3 object_detection/model_main.py \\\n    --pipeline_config_path=${PIPELINE_CONFIG_PATH} \\\n    --model_dir=${MODEL_DIR} \\\n    --num_train_steps=${NUM_TRAIN_STEPS} \\\n    --sample_1_of_n_eval_examples=$SAMPLE_1_OF_N_EVAL_EXAMPLES \\\n    --alsologtostderr\n\n2. I downloaded the model \" ssd_mobilenet_v1_coco_11_06_2017\" from here: \nhttps://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md\n\n3. The sample config file is from here: \nhttps://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs\nI did not update parameters other than no of classes and \"paths to be configured\"\n\n4. I had to convert the data to tf.Records format; i used this kernel and use 9 shards for training and 1 for evaluation: https://www.kaggle.com/lyonzy/convert-dicom-images-to-tfrecords\n5. The for prediction on the test data, I modified this script:\nhttps://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb\n\nI will make a git repo soon and commit the code. \nCurious to know if anyone had good results with the tensorflow object detection API. \n\nHappy Kaggling ;-)\n\nMehul",
      "votes": null
    },
    {
      "id": "403911",
      "postDate": "10/14/2018 22:15:47",
      "content": "<p>I just ran model_main.py and then run tensorboard; you can see real-time plots and results on some images. these screenshots are at 7000 training steps and it is taking about 1hr for 2000 steps on a GTX-1080-TI. </p>",
      "rawMarkdown": "I just ran model_main.py and then run tensorboard; you can see real-time plots and results on some images. these screenshots are at 7000 training steps and it is taking about 1hr for 2000 steps on a GTX-1080-TI.",
      "votes": null
    },
    {
      "id": "403913",
      "postDate": "10/14/2018 22:17:06",
      "content": "<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/5d748662b3fa8a7a63e735fed0ee9c7d/sc3.png\" alt=\"tensorboard visualization of results at various epochs\"></p>",
      "rawMarkdown": "![tensorboard visualization of results at various epochs][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/5d748662b3fa8a7a63e735fed0ee9c7d/sc3.png",
      "votes": null
    },
    {
      "id": "403914",
      "postDate": "10/14/2018 22:18:11",
      "content": "<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/103eae71344041fe7b22004d9739a952/sc2.png\" alt=\"Tensorboard visualization of Detection rate\"></p>",
      "rawMarkdown": "![Tensorboard visualization of Detection rate][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/103eae71344041fe7b22004d9739a952/sc2.png",
      "votes": null
    },
    {
      "id": "404180",
      "postDate": "10/15/2018 11:55:40",
      "content": "<p>@Mehul, Thank you! How did you get the image tab to work in tensorboard? I been trying to get it to work without success. </p>",
      "rawMarkdown": "Mehul, Thank you! How did you get the image tab to work in tensorboard? I been trying to get it to work without success.",
      "votes": null
    },
    {
      "id": "404652",
      "postDate": "10/16/2018 06:23:08",
      "content": "<p>@William, i just used the default config files from tensorflow page; did you divide into train and eval datasets ? there was an option to put the path of eval data file in the config file. it could also be my tensorboard version. i will check it tomorrow and let you know . for tensorflow i am using 1.9. </p>\n\n<p>I used two models ssd_mobilenet and faster_rcnn_<em>atrous</em> and both had it by default ; i did not have to do anything special to get the image tab to work. for faster_rcnn_<em>atrous</em> there is also a projection of PCA components tab; it is cool but i dont understand how to read it yet :) </p>",
      "rawMarkdown": "William, i just used the default config files from tensorflow page; did you divide into train and eval datasets ? there was an option to put the path of eval data file in the config file. it could also be my tensorboard version. i will check it tomorrow and let you know . for tensorflow i am using 1.9. \n\nI used two models ssd_mobilenet and faster_rcnn_*atrous* and both had it by default ; i did not have to do anything special to get the image tab to work. for faster_rcnn_*atrous* there is also a projection of PCA components tab; it is cool but i dont understand how to read it yet :)",
      "votes": null
    },
    {
      "id": "404898",
      "postDate": "10/16/2018 14:37:22",
      "content": "<p>Thank you. </p>",
      "rawMarkdown": "Thank you.",
      "votes": null
    },
    {
      "id": "404937",
      "postDate": "10/16/2018 15:56:47",
      "content": "<p>@William, this is one of the config files from github: <a href=\"https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_resnet_v2_atrous_coco.config\">https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_resnet_v2_atrous_coco.config</a></p>",
      "rawMarkdown": "William, this is one of the config files from github: https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_resnet_v2_atrous_coco.config",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 403911,
      "author_name": "mpsampat",
      "author_url": "",
      "post_date": "10/14/2018 22:15:47",
      "content": "<p>I just ran model_main.py and then run tensorboard; you can see real-time plots and results on some images. these screenshots are at 7000 training steps and it is taking about 1hr for 2000 steps on a GTX-1080-TI. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 403913,
      "author_name": "mpsampat",
      "author_url": "",
      "post_date": "10/14/2018 22:17:06",
      "content": "<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/5d748662b3fa8a7a63e735fed0ee9c7d/sc3.png\" alt=\"tensorboard visualization of results at various epochs\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 403914,
      "author_name": "mpsampat",
      "author_url": "",
      "post_date": "10/14/2018 22:18:11",
      "content": "<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/103eae71344041fe7b22004d9739a952/sc2.png\" alt=\"Tensorboard visualization of Detection rate\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 404180,
      "author_name": "dskswu",
      "author_url": "",
      "post_date": "10/15/2018 11:55:40",
      "content": "<p>@Mehul, Thank you! How did you get the image tab to work in tensorboard? I been trying to get it to work without success. </p>",
      "votes": null,
      "replies": [
        {
          "id": 404652,
          "author_name": "mpsampat",
          "author_url": "",
          "post_date": "10/16/2018 06:23:08",
          "content": "<p>@William, i just used the default config files from tensorflow page; did you divide into train and eval datasets ? there was an option to put the path of eval data file in the config file. it could also be my tensorboard version. i will check it tomorrow and let you know . for tensorflow i am using 1.9. </p>\n\n<p>I used two models ssd_mobilenet and faster_rcnn_<em>atrous</em> and both had it by default ; i did not have to do anything special to get the image tab to work. for faster_rcnn_<em>atrous</em> there is also a projection of PCA components tab; it is cool but i dont understand how to read it yet :) </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 404898,
          "author_name": "dskswu",
          "author_url": "",
          "post_date": "10/16/2018 14:37:22",
          "content": "<p>Thank you. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 404937,
          "author_name": "mpsampat",
          "author_url": "",
          "post_date": "10/16/2018 15:56:47",
          "content": "<p>@William, this is one of the config files from github: <a href=\"https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_resnet_v2_atrous_coco.config\">https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_resnet_v2_atrous_coco.config</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "403881": "Hi All! \nI am trying to learn how to use the Tensorflow Object Detection API.  (tf-API)\nhttps://github.com/tensorflow/models/tree/master/research/object_detection\n\nFull disclosure: **My results so far are terrible with the tf-API** (Mask-RCNN results are better) But it has been a great learning experience. Here is what i had to do to get it to run.\n\n1. After installing it I run the following bash script (you can run object_detection/model_main.py directly from commandline but i just run it from bash script so i remember the parameters). \n\n!/bin/bash\n\nPIPELINE_CONFIG_PATH='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_pets.config'\n\nMODEL_DIR='/home/mehul/kaggle/pneumonia-detection-2018/models/ssd_mobilenet_v1_coco_11_06_2017/'\n\nexport NUM_TRAIN_STEPS=20000\n\npython3 object_detection/model_main.py \\\n    --pipeline_config_path=${PIPELINE_CONFIG_PATH} \\\n    --model_dir=${MODEL_DIR} \\\n    --num_train_steps=${NUM_TRAIN_STEPS} \\\n    --sample_1_of_n_eval_examples=$SAMPLE_1_OF_N_EVAL_EXAMPLES \\\n    --alsologtostderr\n\n2. I downloaded the model \" ssd_mobilenet_v1_coco_11_06_2017\" from here: \nhttps://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md\n\n3. The sample config file is from here: \nhttps://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs\nI did not update parameters other than no of classes and \"paths to be configured\"\n\n4. I had to convert the data to tf.Records format; i used this kernel and use 9 shards for training and 1 for evaluation: https://www.kaggle.com/lyonzy/convert-dicom-images-to-tfrecords\n5. The for prediction on the test data, I modified this script:\nhttps://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb\n\nI will make a git repo soon and commit the code. \nCurious to know if anyone had good results with the tensorflow object detection API. \n\nHappy Kaggling ;-)\n\nMehul",
    "403911": "I just ran model_main.py and then run tensorboard; you can see real-time plots and results on some images. these screenshots are at 7000 training steps and it is taking about 1hr for 2000 steps on a GTX-1080-TI.",
    "403913": "![tensorboard visualization of results at various epochs][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/5d748662b3fa8a7a63e735fed0ee9c7d/sc3.png",
    "403914": "![Tensorboard visualization of Detection rate][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/159759/103eae71344041fe7b22004d9739a952/sc2.png",
    "404180": "Mehul, Thank you! How did you get the image tab to work in tensorboard? I been trying to get it to work without success.",
    "404652": "William, i just used the default config files from tensorflow page; did you divide into train and eval datasets ? there was an option to put the path of eval data file in the config file. it could also be my tensorboard version. i will check it tomorrow and let you know . for tensorflow i am using 1.9. \n\nI used two models ssd_mobilenet and faster_rcnn_*atrous* and both had it by default ; i did not have to do anything special to get the image tab to work. for faster_rcnn_*atrous* there is also a projection of PCA components tab; it is cool but i dont understand how to read it yet :)",
    "404898": "Thank you.",
    "404937": "William, this is one of the config files from github: https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_resnet_v2_atrous_coco.config"
  },
  "source": "meta"
}