{
  "id": 255135,
  "title": "What is current best classification accuracy?",
  "url": "/competitions/siim-covid19-detection/discussion/255135",
  "author_name": "",
  "post_date": "2021-07-26T00:37:50.139126800Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I think competition metric (mAP) of classification task is really hard to understand.<br>\nUsually classification task is evaluated by accuracy or area under curve of ROC.</p>\n<p>What is current best classification accuracy?<br>\nI can get about 69~70% top-1 accuracy in 5-fold test.</p>",
  "messages": [
    {
      "id": "1400028",
      "postDate": "07/26/2021 00:37:50",
      "content": "<p>I think competition metric (mAP) of classification task is really hard to understand.<br>\nUsually classification task is evaluated by accuracy or area under curve of ROC.</p>\n<p>What is current best classification accuracy?<br>\nI can get about 69~70% top-1 accuracy in 5-fold test.</p>",
      "rawMarkdown": "I think competition metric (mAP) of classification task is really hard to understand.\nUsually classification task is evaluated by accuracy or area under curve of ROC.\n\nWhat is current best classification accuracy?\nI can get about 69~70% top-1 accuracy in 5-fold test.",
      "votes": null
    },
    {
      "id": "1400117",
      "postDate": "07/26/2021 04:20:09",
      "content": "<p>You can use <code>sklearn.metrics.average_precision_score(true, pred)</code> for the targets <code>negative</code>, <code>typical</code>, <code>indeterminate</code>, <code>atypical</code> to compute the Kaggle competition metric for those 4 targets. (It is close enough to coco <code>mAP</code>) Your LB score is the average of 6 targets <code>mAP</code> which is those 4 plus <code>none</code> and <code>opacity</code>. To compute <code>none</code>, you can also use sklearn average precision. To compute <code>opacity</code>, you will need to use bbox <code>mAP</code> code from public notebooks.</p>\n<p>Most people know their CV LB <code>mAP</code> per target and can share that. Most people are not computing their top-1 accuracy.</p>",
      "rawMarkdown": "You can use `sklearn.metrics.average_precision_score(true, pred)` for the targets `negative`, `typical`, `indeterminate`, `atypical` to compute the Kaggle competition metric for those 4 targets. (It is close enough to coco `mAP`) Your LB score is the average of 6 targets `mAP` which is those 4 plus `none` and `opacity`. To compute `none`, you can also use sklearn average precision. To compute `opacity`, you will need to use bbox `mAP` code from public notebooks.\n\nMost people know their CV LB `mAP` per target and can share that. Most people are not computing their top-1 accuracy.",
      "votes": null
    },
    {
      "id": "1401321",
      "postDate": "07/27/2021 08:00:31",
      "content": "<p>Thanks for sharing <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> </p>",
      "rawMarkdown": "Thanks for sharing @cdeotte",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1400117,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "07/26/2021 04:20:09",
      "content": "<p>You can use <code>sklearn.metrics.average_precision_score(true, pred)</code> for the targets <code>negative</code>, <code>typical</code>, <code>indeterminate</code>, <code>atypical</code> to compute the Kaggle competition metric for those 4 targets. (It is close enough to coco <code>mAP</code>) Your LB score is the average of 6 targets <code>mAP</code> which is those 4 plus <code>none</code> and <code>opacity</code>. To compute <code>none</code>, you can also use sklearn average precision. To compute <code>opacity</code>, you will need to use bbox <code>mAP</code> code from public notebooks.</p>\n<p>Most people know their CV LB <code>mAP</code> per target and can share that. Most people are not computing their top-1 accuracy.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1401321,
          "author_name": "amritpal333",
          "author_url": "",
          "post_date": "07/27/2021 08:00:31",
          "content": "<p>Thanks for sharing <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1400028": "I think competition metric (mAP) of classification task is really hard to understand.\nUsually classification task is evaluated by accuracy or area under curve of ROC.\n\nWhat is current best classification accuracy?\nI can get about 69~70% top-1 accuracy in 5-fold test.",
    "1400117": "You can use `sklearn.metrics.average_precision_score(true, pred)` for the targets `negative`, `typical`, `indeterminate`, `atypical` to compute the Kaggle competition metric for those 4 targets. (It is close enough to coco `mAP`) Your LB score is the average of 6 targets `mAP` which is those 4 plus `none` and `opacity`. To compute `none`, you can also use sklearn average precision. To compute `opacity`, you will need to use bbox `mAP` code from public notebooks.\n\nMost people know their CV LB `mAP` per target and can share that. Most people are not computing their top-1 accuracy.",
    "1401321": "Thanks for sharing @cdeotte"
  },
  "source": "meta"
}