{
  "id": 160702,
  "title": "Use cross entropy loss (NOT BCE) for focal loss calculation",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/160702",
  "author_name": "ZHU CHAO",
  "post_date": "2020-06-22T10:45:10.374000",
  "votes": 1,
  "comment_count": 0,
  "views": 0,
  "content": "<p>My current neural network outputs 2 logits/raw scores and uses cross entropy as loss function. I want to evaluate how focal loss affects its performance. Did some searches, it seems that the focal loss is implemented on top binary cross entropy. Without changing my neural network, can i directly calculate focal loss using CE loss? sample code is below. Why and why not?</p>\n\n<pre><code>def FocalLossCE(inputs, targets, alpha = ALPHA, gamma = GAMMA):\n\n       CE = F.cross_entropy(inputs, targets, reduction='mean')\n\n       CE_EXP = torch.exp(-CE)\n\n       focal_loss = alpha * (1-CE_EXP)**gamma * CE\n\n       return focal_loss\n</code></pre>",
  "messages": [
    {
      "id": 896648,
      "postDate": "2020-06-22T10:45:10.373Z",
      "content": "<p>My current neural network outputs 2 logits/raw scores and uses cross entropy as loss function. I want to evaluate how focal loss affects its performance. Did some searches, it seems that the focal loss is implemented on top binary cross entropy. Without changing my neural network, can i directly calculate focal loss using CE loss? sample code is below. Why and why not?</p>\n\n<pre><code>def FocalLossCE(inputs, targets, alpha = ALPHA, gamma = GAMMA):\n\n       CE = F.cross_entropy(inputs, targets, reduction='mean')\n\n       CE_EXP = torch.exp(-CE)\n\n       focal_loss = alpha * (1-CE_EXP)**gamma * CE\n\n       return focal_loss\n</code></pre>",
      "rawMarkdown": "My current neural network outputs 2 logits/raw scores and uses cross entropy as loss function. I want to evaluate how focal loss affects its performance. Did some searches, it seems that the focal loss is implemented on top binary cross entropy. Without changing my neural network, can i directly calculate focal loss using CE loss? sample code is below. Why and why not?\n\n\n\n    def FocalLossCE(inputs, targets, alpha = ALPHA, gamma = GAMMA):\n       \n           CE = F.cross_entropy(inputs, targets, reduction='mean')\n       \n           CE_EXP = torch.exp(-CE)\n       \n           focal_loss = alpha * (1-CE_EXP)**gamma * CE\n       \n           return focal_loss",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "896648": "My current neural network outputs 2 logits/raw scores and uses cross entropy as loss function. I want to evaluate how focal loss affects its performance. Did some searches, it seems that the focal loss is implemented on top binary cross entropy. Without changing my neural network, can i directly calculate focal loss using CE loss? sample code is below. Why and why not?\n\n\n\n    def FocalLossCE(inputs, targets, alpha = ALPHA, gamma = GAMMA):\n       \n           CE = F.cross_entropy(inputs, targets, reduction='mean')\n       \n           CE_EXP = torch.exp(-CE)\n       \n           focal_loss = alpha * (1-CE_EXP)**gamma * CE\n       \n           return focal_loss"
  }
}