{
  "id": 112005,
  "title": "Parallel mAP_evaluation ",
  "url": "/competitions/3d-object-detection-for-autonomous-vehicles/discussion/112005",
  "author_name": "",
  "post_date": "2019-10-10T09:10:23.677120600Z",
  "votes": 16,
  "comment_count": 8,
  "views": 0,
  "content": "<p>Hey guys, </p>\n\n<p>The official script (<a href=\"https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/eval/detection/mAP_evaluation.py#L35\">mAP_evaluation</a>) uses a IoU threshold to calculate mAP, to calculated the overall mAP we have to run the script 10 times for all the thresholds (0.5, 0.55, .. 0.95) which is a time consuming process as it's a single threaded implementation.</p>\n\n<p>Here's how it looks like:</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2F03914421f2f6f9eed5f2ac1539466550%2Fsingle.png?generation=1570698335285231&amp;alt=media\" alt=\"\"></p>\n\n<p>I wrote a script to parallelize this process using Python's inbuilt multiprocessing module, it computes mAP for each of the 10 thresholds parallelly, here's how it looks like:</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2Fbccb4e9acf0fe45ff303e7aba6e062c7%2Fparallel.png?generation=1570698459755760&amp;alt=media\" alt=\"\"></p>\n\n<p>Here's the source code: <a href=\"https://github.com/pyaf/parallel_mAP_evaluation\">parallel_mAP_evaluation</a></p>\n\n<p>Enjoy!</p>",
  "messages": [
    {
      "id": "645595",
      "postDate": "10/10/2019 09:10:23",
      "content": "<p>Hey guys, </p>\n\n<p>The official script (<a href=\"https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/eval/detection/mAP_evaluation.py#L35\">mAP_evaluation</a>) uses a IoU threshold to calculate mAP, to calculated the overall mAP we have to run the script 10 times for all the thresholds (0.5, 0.55, .. 0.95) which is a time consuming process as it's a single threaded implementation.</p>\n\n<p>Here's how it looks like:</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2F03914421f2f6f9eed5f2ac1539466550%2Fsingle.png?generation=1570698335285231&amp;alt=media\" alt=\"\"></p>\n\n<p>I wrote a script to parallelize this process using Python's inbuilt multiprocessing module, it computes mAP for each of the 10 thresholds parallelly, here's how it looks like:</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2Fbccb4e9acf0fe45ff303e7aba6e062c7%2Fparallel.png?generation=1570698459755760&amp;alt=media\" alt=\"\"></p>\n\n<p>Here's the source code: <a href=\"https://github.com/pyaf/parallel_mAP_evaluation\">parallel_mAP_evaluation</a></p>\n\n<p>Enjoy!</p>",
      "rawMarkdown": "Hey guys, \n\nThe official script ([mAP_evaluation](https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/eval/detection/mAP_evaluation.py#L35)) uses a IoU threshold to calculate mAP, to calculated the overall mAP we have to run the script 10 times for all the thresholds (0.5, 0.55, .. 0.95) which is a time consuming process as it's a single threaded implementation.\n\nHere's how it looks like:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2F03914421f2f6f9eed5f2ac1539466550%2Fsingle.png?generation=1570698335285231&amp;alt=media)\n\nI wrote a script to parallelize this process using Python's inbuilt multiprocessing module, it computes mAP for each of the 10 thresholds parallelly, here's how it looks like:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2Fbccb4e9acf0fe45ff303e7aba6e062c7%2Fparallel.png?generation=1570698459755760&amp;alt=media)\n \nHere's the source code: [parallel\\_mAP\\_evaluation](https://github.com/pyaf/parallel_mAP_evaluation)\n\nEnjoy!",
      "votes": null
    },
    {
      "id": "645607",
      "postDate": "10/10/2019 09:27:05",
      "content": "<p>Thank you !</p>",
      "rawMarkdown": "Thank you !",
      "votes": null
    },
    {
      "id": "645613",
      "postDate": "10/10/2019 09:39:52",
      "content": "<p>👍 </p>",
      "rawMarkdown": "👍",
      "votes": null
    },
    {
      "id": "645849",
      "postDate": "10/10/2019 14:43:38",
      "content": "<p>well done. </p>",
      "rawMarkdown": "well done.",
      "votes": null
    },
    {
      "id": "645958",
      "postDate": "10/10/2019 16:55:06",
      "content": "<p>Awesome! Great work!</p>",
      "rawMarkdown": "Awesome! Great work!",
      "votes": null
    },
    {
      "id": "645992",
      "postDate": "10/10/2019 17:47:45",
      "content": "<p>Thanks <a href=\"/jackvial\">@jackvial</a> </p>",
      "rawMarkdown": "Thanks @jackvial",
      "votes": null
    },
    {
      "id": "645995",
      "postDate": "10/10/2019 17:48:02",
      "content": "<p>Thank you <a href=\"/valanm\">@valanm</a> :)</p>",
      "rawMarkdown": "Thank you @valanm :)",
      "votes": null
    },
    {
      "id": "647418",
      "postDate": "10/12/2019 15:37:13",
      "content": "<p>I love your code!</p>\n\n<p>Could you please change the license to the same as the original code <a href=\"https://github.com/lyft/nuscenes-devkit/blob/master/license.txt\">https://github.com/lyft/nuscenes-devkit/blob/master/license.txt</a> ?</p>",
      "rawMarkdown": "I love your code!\n\nCould you please change the license to the same as the original code https://github.com/lyft/nuscenes-devkit/blob/master/license.txt ?",
      "votes": null
    },
    {
      "id": "647433",
      "postDate": "10/12/2019 16:03:45",
      "content": "<p>Thank you, I've updated the license :)</p>",
      "rawMarkdown": "Thank you, I've updated the license :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 645607,
      "author_name": "arthurllau",
      "author_url": "",
      "post_date": "10/10/2019 09:27:05",
      "content": "<p>Thank you !</p>",
      "votes": null,
      "replies": [
        {
          "id": 645613,
          "author_name": "rishabhiitbhu",
          "author_url": "",
          "post_date": "10/10/2019 09:39:52",
          "content": "<p>👍 </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 645849,
      "author_name": "valanm",
      "author_url": "",
      "post_date": "10/10/2019 14:43:38",
      "content": "<p>well done. </p>",
      "votes": null,
      "replies": [
        {
          "id": 645995,
          "author_name": "rishabhiitbhu",
          "author_url": "",
          "post_date": "10/10/2019 17:48:02",
          "content": "<p>Thank you <a href=\"/valanm\">@valanm</a> :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 645958,
      "author_name": "jackvial",
      "author_url": "",
      "post_date": "10/10/2019 16:55:06",
      "content": "<p>Awesome! Great work!</p>",
      "votes": null,
      "replies": [
        {
          "id": 645992,
          "author_name": "rishabhiitbhu",
          "author_url": "",
          "post_date": "10/10/2019 17:47:45",
          "content": "<p>Thanks <a href=\"/jackvial\">@jackvial</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 647418,
      "author_name": "iglovikov",
      "author_url": "",
      "post_date": "10/12/2019 15:37:13",
      "content": "<p>I love your code!</p>\n\n<p>Could you please change the license to the same as the original code <a href=\"https://github.com/lyft/nuscenes-devkit/blob/master/license.txt\">https://github.com/lyft/nuscenes-devkit/blob/master/license.txt</a> ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 647433,
          "author_name": "rishabhiitbhu",
          "author_url": "",
          "post_date": "10/12/2019 16:03:45",
          "content": "<p>Thank you, I've updated the license :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "645595": "Hey guys, \n\nThe official script ([mAP_evaluation](https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/eval/detection/mAP_evaluation.py#L35)) uses a IoU threshold to calculate mAP, to calculated the overall mAP we have to run the script 10 times for all the thresholds (0.5, 0.55, .. 0.95) which is a time consuming process as it's a single threaded implementation.\n\nHere's how it looks like:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2F03914421f2f6f9eed5f2ac1539466550%2Fsingle.png?generation=1570698335285231&amp;alt=media)\n\nI wrote a script to parallelize this process using Python's inbuilt multiprocessing module, it computes mAP for each of the 10 thresholds parallelly, here's how it looks like:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761152%2Fbccb4e9acf0fe45ff303e7aba6e062c7%2Fparallel.png?generation=1570698459755760&amp;alt=media)\n \nHere's the source code: [parallel\\_mAP\\_evaluation](https://github.com/pyaf/parallel_mAP_evaluation)\n\nEnjoy!",
    "645607": "Thank you !",
    "645613": "👍",
    "645849": "well done.",
    "645958": "Awesome! Great work!",
    "645992": "Thanks @jackvial",
    "645995": "Thank you @valanm :)",
    "647418": "I love your code!\n\nCould you please change the license to the same as the original code https://github.com/lyft/nuscenes-devkit/blob/master/license.txt ?",
    "647433": "Thank you, I've updated the license :)"
  },
  "source": "meta"
}