{
  "id": 220717,
  "title": "Macro AUROC available in Pytorch Lightning master branch",
  "url": "/competitions/ranzcr-clip-catheter-line-classification/discussion/220717",
  "author_name": "Claudio Verdú Ruiz",
  "post_date": "2021-02-19T09:20:47.228000",
  "votes": 3,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hello guys, I just came in to this competition after finishing Cassava Leaf Desease.</p>\n<p>I realized that it's not easy to find this metric implemented in PyTorch, and before implementing it, I decided to make some research. I found that it is available in Pytorch Lightning master branch (no stable release yet), so I wanted to share with you.</p>\n<p>Some points:</p>\n<ul>\n<li>To install master branch, you can do <code>pip install git+https://github.com/PyTorchLightning/pytorch-lightning</code>.</li>\n<li><code>from pytorch_lightning.metrics import AUROC</code></li>\n<li>It may fail if for a certain column there are only zeroes or ones in a batch, so you will have to instantiate as follows: <code>AUROC(n_classes=11, compute_on_step=False)</code>. This way, forward will call <code>update</code> but not <code>compute</code>. You can also imitate this behaviour by yourself.</li>\n<li>Beware that given the prior point, it will return <code>None</code> in forward pass (calling it).</li>\n<li>Need targets to be integers and preds to be probabilities, so you will have to do something like: <code>auc(torch.sigmoid(pred), target.int())</code>.</li>\n<li>Remember to reset at the end of every epoch!</li>\n</ul>\n<p>Hope this helps, cheers!</p>",
  "messages": [
    {
      "id": 1210224,
      "postDate": "2021-02-19T09:20:47.230Z",
      "content": "<p>Hello guys, I just came in to this competition after finishing Cassava Leaf Desease.</p>\n<p>I realized that it's not easy to find this metric implemented in PyTorch, and before implementing it, I decided to make some research. I found that it is available in Pytorch Lightning master branch (no stable release yet), so I wanted to share with you.</p>\n<p>Some points:</p>\n<ul>\n<li>To install master branch, you can do <code>pip install git+https://github.com/PyTorchLightning/pytorch-lightning</code>.</li>\n<li><code>from pytorch_lightning.metrics import AUROC</code></li>\n<li>It may fail if for a certain column there are only zeroes or ones in a batch, so you will have to instantiate as follows: <code>AUROC(n_classes=11, compute_on_step=False)</code>. This way, forward will call <code>update</code> but not <code>compute</code>. You can also imitate this behaviour by yourself.</li>\n<li>Beware that given the prior point, it will return <code>None</code> in forward pass (calling it).</li>\n<li>Need targets to be integers and preds to be probabilities, so you will have to do something like: <code>auc(torch.sigmoid(pred), target.int())</code>.</li>\n<li>Remember to reset at the end of every epoch!</li>\n</ul>\n<p>Hope this helps, cheers!</p>",
      "rawMarkdown": "Hello guys, I just came in to this competition after finishing Cassava Leaf Desease.\n\nI realized that it's not easy to find this metric implemented in PyTorch, and before implementing it, I decided to make some research. I found that it is available in Pytorch Lightning master branch (no stable release yet), so I wanted to share with you.\n\nSome points:\n\n- To install master branch, you can do `pip install git+https://github.com/PyTorchLightning/pytorch-lightning`.\n- `from pytorch_lightning.metrics import AUROC`\n- It may fail if for a certain column there are only zeroes or ones in a batch, so you will have to instantiate as follows: `AUROC(n_classes=11, compute_on_step=False)`. This way, forward will call `update` but not `compute`. You can also imitate this behaviour by yourself.\n- Beware that given the prior point, it will return `None` in forward pass (calling it).\n- Need targets to be integers and preds to be probabilities, so you will have to do something like: `auc(torch.sigmoid(pred), target.int())`.\n- Remember to reset at the end of every epoch!\n\nHope this helps, cheers!\n\n",
      "votes": 3
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1210224": "Hello guys, I just came in to this competition after finishing Cassava Leaf Desease.\n\nI realized that it's not easy to find this metric implemented in PyTorch, and before implementing it, I decided to make some research. I found that it is available in Pytorch Lightning master branch (no stable release yet), so I wanted to share with you.\n\nSome points:\n\n- To install master branch, you can do `pip install git+https://github.com/PyTorchLightning/pytorch-lightning`.\n- `from pytorch_lightning.metrics import AUROC`\n- It may fail if for a certain column there are only zeroes or ones in a batch, so you will have to instantiate as follows: `AUROC(n_classes=11, compute_on_step=False)`. This way, forward will call `update` but not `compute`. You can also imitate this behaviour by yourself.\n- Beware that given the prior point, it will return `None` in forward pass (calling it).\n- Need targets to be integers and preds to be probabilities, so you will have to do something like: `auc(torch.sigmoid(pred), target.int())`.\n- Remember to reset at the end of every epoch!\n\nHope this helps, cheers!\n\n"
  }
}