{
  "id": 370723,
  "title": "Matthews Correlation Coefficient[MCC]  for Loss function [code]. It's can useful this competition.",
  "url": "/competitions/nfl-player-contact-detection/discussion/370723",
  "author_name": "",
  "post_date": "2022-12-06T06:11:42.521104400Z",
  "votes": 24,
  "comment_count": 10,
  "views": 0,
  "content": "<p>Previously, we looked into the code for measure <a href=\"https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/370722\" target=\"_blank\">Matthews Correlation Coefficient(MCC)</a></p>\n<p>This time, let's find out the loss function to optimize for mcc.<br>\nthis code implement for pytorch.</p>\n<p>I hope this loss function helpful improvement for Model. Good Luck</p>\n<pre><code>class MCC_Loss(nn.Module):\n    \"\"\"\n    Calculates the proposed Matthews Correlation Coefficient-based loss.\n    Args:\n        inputs (torch.Tensor): 1-hot encoded predictions\n        targets (torch.Tensor): 1-hot encoded ground truth\n    \"\"\"\n\n    def __init__(self):\n        super(MCC_Loss, self).__init__()\n\n    def forward(self, inputs, targets):\n        \"\"\"\n        MCC = (TP.TN - FP.FN) / sqrt((TP+FP) . (TP+FN) . (TN+FP) . (TN+FN))\n        where TP, TN, FP, and FN are elements in the confusion matrix.\n        \"\"\"\n        tp = torch.sum(torch.mul(inputs, targets))\n        tn = torch.sum(torch.mul((1 - inputs), (1 - targets)))\n        fp = torch.sum(torch.mul(inputs, (1 - targets)))\n        fn = torch.sum(torch.mul((1 - inputs), targets))\n\n        numerator = torch.mul(tp, tn) - torch.mul(fp, fn)\n        denominator = torch.sqrt(\n            torch.add(tp, 1, fp)\n            * torch.add(tp, 1, fn)\n            * torch.add(tn, 1, fp)\n            * torch.add(tn, 1, fn)\n        )\n\n        # Adding 1 to the denominator to avoid divide-by-zero errors.\n        mcc = torch.div(numerator.sum(), denominator.sum() + 1.0)\n        return 1 - mcc\n</code></pre>",
  "messages": [
    {
      "id": "2056453",
      "postDate": "12/06/2022 06:11:42",
      "content": "<p>Previously, we looked into the code for measure <a href=\"https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/370722\" target=\"_blank\">Matthews Correlation Coefficient(MCC)</a></p>\n<p>This time, let's find out the loss function to optimize for mcc.<br>\nthis code implement for pytorch.</p>\n<p>I hope this loss function helpful improvement for Model. Good Luck</p>\n<pre><code>class MCC_Loss(nn.Module):\n    \"\"\"\n    Calculates the proposed Matthews Correlation Coefficient-based loss.\n    Args:\n        inputs (torch.Tensor): 1-hot encoded predictions\n        targets (torch.Tensor): 1-hot encoded ground truth\n    \"\"\"\n\n    def __init__(self):\n        super(MCC_Loss, self).__init__()\n\n    def forward(self, inputs, targets):\n        \"\"\"\n        MCC = (TP.TN - FP.FN) / sqrt((TP+FP) . (TP+FN) . (TN+FP) . (TN+FN))\n        where TP, TN, FP, and FN are elements in the confusion matrix.\n        \"\"\"\n        tp = torch.sum(torch.mul(inputs, targets))\n        tn = torch.sum(torch.mul((1 - inputs), (1 - targets)))\n        fp = torch.sum(torch.mul(inputs, (1 - targets)))\n        fn = torch.sum(torch.mul((1 - inputs), targets))\n\n        numerator = torch.mul(tp, tn) - torch.mul(fp, fn)\n        denominator = torch.sqrt(\n            torch.add(tp, 1, fp)\n            * torch.add(tp, 1, fn)\n            * torch.add(tn, 1, fp)\n            * torch.add(tn, 1, fn)\n        )\n\n        # Adding 1 to the denominator to avoid divide-by-zero errors.\n        mcc = torch.div(numerator.sum(), denominator.sum() + 1.0)\n        return 1 - mcc\n</code></pre>",
      "rawMarkdown": "Previously, we looked into the code for measure [Matthews Correlation Coefficient(MCC)](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/370722)\n\nThis time, let's find out the loss function to optimize for mcc.\nthis code implement for pytorch.\n\nI hope this loss function helpful improvement for Model. Good Luck\n\n```\nclass MCC_Loss(nn.Module):\n    \"\"\"\n    Calculates the proposed Matthews Correlation Coefficient-based loss.\n    Args:\n        inputs (torch.Tensor): 1-hot encoded predictions\n        targets (torch.Tensor): 1-hot encoded ground truth\n    \"\"\"\n\n    def __init__(self):\n        super(MCC_Loss, self).__init__()\n\n    def forward(self, inputs, targets):\n        \"\"\"\n        MCC = (TP.TN - FP.FN) / sqrt((TP+FP) . (TP+FN) . (TN+FP) . (TN+FN))\n        where TP, TN, FP, and FN are elements in the confusion matrix.\n        \"\"\"\n        tp = torch.sum(torch.mul(inputs, targets))\n        tn = torch.sum(torch.mul((1 - inputs), (1 - targets)))\n        fp = torch.sum(torch.mul(inputs, (1 - targets)))\n        fn = torch.sum(torch.mul((1 - inputs), targets))\n\n        numerator = torch.mul(tp, tn) - torch.mul(fp, fn)\n        denominator = torch.sqrt(\n            torch.add(tp, 1, fp)\n            * torch.add(tp, 1, fn)\n            * torch.add(tn, 1, fp)\n            * torch.add(tn, 1, fn)\n        )\n\n        # Adding 1 to the denominator to avoid divide-by-zero errors.\n        mcc = torch.div(numerator.sum(), denominator.sum() + 1.0)\n        return 1 - mcc\n```",
      "votes": null
    },
    {
      "id": "2056685",
      "postDate": "12/06/2022 11:12:29",
      "content": "<p>Great, helpful tip (MCC for Loss function) W. Park.</p>",
      "rawMarkdown": "Great, helpful tip (MCC for Loss function) W. Park.",
      "votes": null
    },
    {
      "id": "2057415",
      "postDate": "12/07/2022 04:58:49",
      "content": "<p>Thanks for Motivation </p>",
      "rawMarkdown": "Thanks for Motivation",
      "votes": null
    },
    {
      "id": "2058982",
      "postDate": "12/08/2022 11:28:21",
      "content": "<p>Is this (Matthews Correlation Coefficient) differentiable? </p>",
      "rawMarkdown": "Is this (Matthews Correlation Coefficient) differentiable?",
      "votes": null
    },
    {
      "id": "2058994",
      "postDate": "12/08/2022 11:44:00",
      "content": "<p><a href=\"https://www.kaggle.com/Simon\" target=\"_blank\">@Simon</a> it can be as it can be used as as loss function, which in my understanding should be differentiable, May be this paper would help you out to get more information<br>\n Matthews Correlation Coefficient Loss for Deep Convolutional Networks: Application to Skin Lesion Segmentation</p>",
      "rawMarkdown": "Simon it can be as it can be used as as loss function, which in my understanding should be differentiable, May be this paper would help you out to get more information\n Matthews Correlation Coefficient Loss for Deep Convolutional Networks: Application to Skin Lesion Segmentation",
      "votes": null
    },
    {
      "id": "2059033",
      "postDate": "12/08/2022 12:12:08",
      "content": "<p>Hi Simon,</p>\n<p>What I found (Google search)was:</p>\n<p>\"5. CONCLUSION<br>\nWe proposed a novel DIFFERENCIABLE LOSS FUNCTION for BINARY SEGMENTATION based on the Matthews correlation coefficient that, unlike IoU and Dice losses, has the desirable property<br>\nof considering all the entries of a confusion matrix including true negative predictions.\"</p>\n<p>On the paper:  \"<br>\nMatthews Correlation coeficient Loss for Deep Convolutional Networks: application to skin lesion Segmentation\" </p>\n<p>Authors: Kumar Abhishek and Ghassan Hamarneh<br>\nSchool of Computing Science, Simon Fraser University, Canada<br>\n{kabhishe, <a>hamarneh}@sfu.ca</a></p>\n<p><a href=\"https://arxiv.org/pdf/2010.13454.pdf\" target=\"_blank\">https://arxiv.org/pdf/2010.13454.pdf</a></p>\n<p>In fact, I'm a beginner and I'm learning everything on the fly. I don't even know what was a differentiable function.</p>\n<p>Thanks to you, I learned that \"loss functions have two properties: they are globally continuous and differentiable.\" </p>\n<p>Loss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)<br>\n<a href=\"https://sentimllc.com/loss-functions.html\" target=\"_blank\">https://sentimllc.com/loss-functions.html</a></p>",
      "rawMarkdown": "Hi Simon,\n\nWhat I found (Google search)was:\n\n\"5. CONCLUSION\nWe proposed a novel DIFFERENCIABLE LOSS FUNCTION for BINARY SEGMENTATION based on the Matthews correlation coefficient that, unlike IoU and Dice losses, has the desirable property\nof considering all the entries of a confusion matrix including true negative predictions.\"\n\nOn the paper:  \"\nMatthews Correlation coeficient Loss for Deep Convolutional Networks: application to skin lesion Segmentation\" \n\nAuthors: Kumar Abhishek and Ghassan Hamarneh\nSchool of Computing Science, Simon Fraser University, Canada\n{kabhishe, hamarneh}@sfu.ca\n\nhttps://arxiv.org/pdf/2010.13454.pdf\n\nIn fact, I'm a beginner and I'm learning everything on the fly. I don't even know what was a differentiable function.\n\nThanks to you, I learned that \"loss functions have two properties: they are globally continuous and differentiable.\" \n\nLoss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)\nhttps://sentimllc.com/loss-functions.html",
      "votes": null
    },
    {
      "id": "2059044",
      "postDate": "12/08/2022 12:25:16",
      "content": "<p>Thank you Satya,<br>\nI noticed that we read the \"same page\" of the same paper : )<br>\n<a href=\"https://arxiv.org/pdf/2010.13454.pdf\" target=\"_blank\">https://arxiv.org/pdf/2010.13454.pdf</a></p>\n<h1>Do loss functions have to be differentiable?</h1>\n<p>\"Yes and no. If the loss function is only used for evaluating the quality of the prediction (on an evaluation set), then it does not have to be differentiable. However, if the loss function is the objective of a gradient descent algorithm, then yes, it has to be differentiable to do that.\"</p>\n<p><a href=\"https://rentruewang.github.io/learning-machine/basics/gradients/loss-fn-derivative.html\" target=\"_blank\">https://rentruewang.github.io/learning-machine/basics/gradients/loss-fn-derivative.html</a></p>\n<p>Loss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)<br>\n<a href=\"https://sentimllc.com/loss-functions.html\" target=\"_blank\">https://sentimllc.com/loss-functions.html</a></p>\n<p>Thank you</p>",
      "rawMarkdown": "Thank you Satya,\nI noticed that we read the \"same page\" of the same paper : )\nhttps://arxiv.org/pdf/2010.13454.pdf\n\n#Do loss functions have to be differentiable?\n\n\"Yes and no. If the loss function is only used for evaluating the quality of the prediction (on an evaluation set), then it does not have to be differentiable. However, if the loss function is the objective of a gradient descent algorithm, then yes, it has to be differentiable to do that.\"\n\nhttps://rentruewang.github.io/learning-machine/basics/gradients/loss-fn-derivative.html\n\nLoss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)\nhttps://sentimllc.com/loss-functions.html\n\nThank you",
      "votes": null
    },
    {
      "id": "2059074",
      "postDate": "12/08/2022 13:08:01",
      "content": "<p>Thanks for sharing the articles :)</p>",
      "rawMarkdown": "Thanks for sharing the articles :)",
      "votes": null
    },
    {
      "id": "2074175",
      "postDate": "12/23/2022 19:49:44",
      "content": "<p>Thanks, W. Park. I was also reading the same article, you mentioned. Thanks for pointing it out to others too.</p>",
      "rawMarkdown": "Thanks, W. Park. I was also reading the same article, you mentioned. Thanks for pointing it out to others too.",
      "votes": null
    },
    {
      "id": "2079490",
      "postDate": "12/29/2022 11:10:25",
      "content": "<p>Many thanks !!</p>",
      "rawMarkdown": "Many thanks !!",
      "votes": null
    },
    {
      "id": "2079520",
      "postDate": "12/29/2022 11:42:51",
      "content": "<p>You're welcome minhquang99.</p>",
      "rawMarkdown": "You're welcome minhquang99.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2056685,
      "author_name": "mpwolke",
      "author_url": "",
      "post_date": "12/06/2022 11:12:29",
      "content": "<p>Great, helpful tip (MCC for Loss function) W. Park.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2057415,
      "author_name": "satyaprakashshukl",
      "author_url": "",
      "post_date": "12/07/2022 04:58:49",
      "content": "<p>Thanks for Motivation </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2058982,
      "author_name": "simonalerdic",
      "author_url": "",
      "post_date": "12/08/2022 11:28:21",
      "content": "<p>Is this (Matthews Correlation Coefficient) differentiable? </p>",
      "votes": null,
      "replies": [
        {
          "id": 2058994,
          "author_name": "satyaprakashshukl",
          "author_url": "",
          "post_date": "12/08/2022 11:44:00",
          "content": "<p><a href=\"https://www.kaggle.com/Simon\" target=\"_blank\">@Simon</a> it can be as it can be used as as loss function, which in my understanding should be differentiable, May be this paper would help you out to get more information<br>\n Matthews Correlation Coefficient Loss for Deep Convolutional Networks: Application to Skin Lesion Segmentation</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2059044,
          "author_name": "mpwolke",
          "author_url": "",
          "post_date": "12/08/2022 12:25:16",
          "content": "<p>Thank you Satya,<br>\nI noticed that we read the \"same page\" of the same paper : )<br>\n<a href=\"https://arxiv.org/pdf/2010.13454.pdf\" target=\"_blank\">https://arxiv.org/pdf/2010.13454.pdf</a></p>\n<h1>Do loss functions have to be differentiable?</h1>\n<p>\"Yes and no. If the loss function is only used for evaluating the quality of the prediction (on an evaluation set), then it does not have to be differentiable. However, if the loss function is the objective of a gradient descent algorithm, then yes, it has to be differentiable to do that.\"</p>\n<p><a href=\"https://rentruewang.github.io/learning-machine/basics/gradients/loss-fn-derivative.html\" target=\"_blank\">https://rentruewang.github.io/learning-machine/basics/gradients/loss-fn-derivative.html</a></p>\n<p>Loss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)<br>\n<a href=\"https://sentimllc.com/loss-functions.html\" target=\"_blank\">https://sentimllc.com/loss-functions.html</a></p>\n<p>Thank you</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2059074,
          "author_name": "satyaprakashshukl",
          "author_url": "",
          "post_date": "12/08/2022 13:08:01",
          "content": "<p>Thanks for sharing the articles :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2059033,
      "author_name": "mpwolke",
      "author_url": "",
      "post_date": "12/08/2022 12:12:08",
      "content": "<p>Hi Simon,</p>\n<p>What I found (Google search)was:</p>\n<p>\"5. CONCLUSION<br>\nWe proposed a novel DIFFERENCIABLE LOSS FUNCTION for BINARY SEGMENTATION based on the Matthews correlation coefficient that, unlike IoU and Dice losses, has the desirable property<br>\nof considering all the entries of a confusion matrix including true negative predictions.\"</p>\n<p>On the paper:  \"<br>\nMatthews Correlation coeficient Loss for Deep Convolutional Networks: application to skin lesion Segmentation\" </p>\n<p>Authors: Kumar Abhishek and Ghassan Hamarneh<br>\nSchool of Computing Science, Simon Fraser University, Canada<br>\n{kabhishe, <a>hamarneh}@sfu.ca</a></p>\n<p><a href=\"https://arxiv.org/pdf/2010.13454.pdf\" target=\"_blank\">https://arxiv.org/pdf/2010.13454.pdf</a></p>\n<p>In fact, I'm a beginner and I'm learning everything on the fly. I don't even know what was a differentiable function.</p>\n<p>Thanks to you, I learned that \"loss functions have two properties: they are globally continuous and differentiable.\" </p>\n<p>Loss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)<br>\n<a href=\"https://sentimllc.com/loss-functions.html\" target=\"_blank\">https://sentimllc.com/loss-functions.html</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2074175,
      "author_name": "malayvyas",
      "author_url": "",
      "post_date": "12/23/2022 19:49:44",
      "content": "<p>Thanks, W. Park. I was also reading the same article, you mentioned. Thanks for pointing it out to others too.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2079490,
      "author_name": "minhquang99",
      "author_url": "",
      "post_date": "12/29/2022 11:10:25",
      "content": "<p>Many thanks !!</p>",
      "votes": null,
      "replies": [
        {
          "id": 2079520,
          "author_name": "mpwolke",
          "author_url": "",
          "post_date": "12/29/2022 11:42:51",
          "content": "<p>You're welcome minhquang99.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2056453": "Previously, we looked into the code for measure [Matthews Correlation Coefficient(MCC)](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/370722)\n\nThis time, let's find out the loss function to optimize for mcc.\nthis code implement for pytorch.\n\nI hope this loss function helpful improvement for Model. Good Luck\n\n```\nclass MCC_Loss(nn.Module):\n    \"\"\"\n    Calculates the proposed Matthews Correlation Coefficient-based loss.\n    Args:\n        inputs (torch.Tensor): 1-hot encoded predictions\n        targets (torch.Tensor): 1-hot encoded ground truth\n    \"\"\"\n\n    def __init__(self):\n        super(MCC_Loss, self).__init__()\n\n    def forward(self, inputs, targets):\n        \"\"\"\n        MCC = (TP.TN - FP.FN) / sqrt((TP+FP) . (TP+FN) . (TN+FP) . (TN+FN))\n        where TP, TN, FP, and FN are elements in the confusion matrix.\n        \"\"\"\n        tp = torch.sum(torch.mul(inputs, targets))\n        tn = torch.sum(torch.mul((1 - inputs), (1 - targets)))\n        fp = torch.sum(torch.mul(inputs, (1 - targets)))\n        fn = torch.sum(torch.mul((1 - inputs), targets))\n\n        numerator = torch.mul(tp, tn) - torch.mul(fp, fn)\n        denominator = torch.sqrt(\n            torch.add(tp, 1, fp)\n            * torch.add(tp, 1, fn)\n            * torch.add(tn, 1, fp)\n            * torch.add(tn, 1, fn)\n        )\n\n        # Adding 1 to the denominator to avoid divide-by-zero errors.\n        mcc = torch.div(numerator.sum(), denominator.sum() + 1.0)\n        return 1 - mcc\n```",
    "2056685": "Great, helpful tip (MCC for Loss function) W. Park.",
    "2057415": "Thanks for Motivation",
    "2058982": "Is this (Matthews Correlation Coefficient) differentiable?",
    "2058994": "Simon it can be as it can be used as as loss function, which in my understanding should be differentiable, May be this paper would help you out to get more information\n Matthews Correlation Coefficient Loss for Deep Convolutional Networks: Application to Skin Lesion Segmentation",
    "2059033": "Hi Simon,\n\nWhat I found (Google search)was:\n\n\"5. CONCLUSION\nWe proposed a novel DIFFERENCIABLE LOSS FUNCTION for BINARY SEGMENTATION based on the Matthews correlation coefficient that, unlike IoU and Dice losses, has the desirable property\nof considering all the entries of a confusion matrix including true negative predictions.\"\n\nOn the paper:  \"\nMatthews Correlation coeficient Loss for Deep Convolutional Networks: application to skin lesion Segmentation\" \n\nAuthors: Kumar Abhishek and Ghassan Hamarneh\nSchool of Computing Science, Simon Fraser University, Canada\n{kabhishe, hamarneh}@sfu.ca\n\nhttps://arxiv.org/pdf/2010.13454.pdf\n\nIn fact, I'm a beginner and I'm learning everything on the fly. I don't even know what was a differentiable function.\n\nThanks to you, I learned that \"loss functions have two properties: they are globally continuous and differentiable.\" \n\nLoss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)\nhttps://sentimllc.com/loss-functions.html",
    "2059044": "Thank you Satya,\nI noticed that we read the \"same page\" of the same paper : )\nhttps://arxiv.org/pdf/2010.13454.pdf\n\n#Do loss functions have to be differentiable?\n\n\"Yes and no. If the loss function is only used for evaluating the quality of the prediction (on an evaluation set), then it does not have to be differentiable. However, if the loss function is the objective of a gradient descent algorithm, then yes, it has to be differentiable to do that.\"\n\nhttps://rentruewang.github.io/learning-machine/basics/gradients/loss-fn-derivative.html\n\nLoss Functions: What are they and why are they important? By Jake Anderson (Jul 4, 2020)\nhttps://sentimllc.com/loss-functions.html\n\nThank you",
    "2059074": "Thanks for sharing the articles :)",
    "2074175": "Thanks, W. Park. I was also reading the same article, you mentioned. Thanks for pointing it out to others too.",
    "2079490": "Many thanks !!",
    "2079520": "You're welcome minhquang99."
  },
  "source": "meta"
}