{
  "id": 79174,
  "title": "Help with implementing focal loss",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/79174",
  "author_name": "Moshel",
  "post_date": "2019-01-31T22:59:11.831000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p><a href=\"/bestfitting\">@bestfitting</a> detailed in his <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\">post</a> that he used focal loss, so I was determined to try it, just for fun and knowledge.\nI have converted <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109#462058\">his implementation</a> from pytorch to keras, but no matter what I tried, the accuracy never tops 0.63.\n<a href=\"https://www.kaggle.com/moshel/mobilenet-fl\">Here is a little minimal kernel</a> I wrote to show the problem. With FOCAL_LOSS set to True, i get </p>\n\n<pre><code>loss: 1.2022 - acc: 0.4562 - val_loss: 1.4919 - val_acc: 0.4818 \n</code></pre>\n\n<p>with FOCAL_LOSS set to False, I get:</p>\n\n<pre><code>loss: 0.1360 - acc: 0.9526 - val_loss: 0.1623 - val_acc: 0.9473\n</code></pre>\n\n<p>(The kernel is a demo kernel - I know the generator sucks, mobilenet is wrong etc. I did not use the imagenet weights just in case it is somehow not the same, but even with the weights the behavior is about the same, as well as using resnet50 and a good generator)</p>\n\n<p>I understand that the loss is very different between focal loss and BCE, but I expected the accuracy to be good. Anyways even if you keep training this kernel for many epochs it will not pass the 0.63, so something is obviously wrong here but I can't figure out what.</p>\n\n<p>If anyone can take a look I would be most grateful.</p>",
  "messages": [
    {
      "id": 464453,
      "postDate": "2019-01-31T22:59:11.833Z",
      "content": "<p><a href=\"/bestfitting\">@bestfitting</a> detailed in his <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\">post</a> that he used focal loss, so I was determined to try it, just for fun and knowledge.\nI have converted <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109#462058\">his implementation</a> from pytorch to keras, but no matter what I tried, the accuracy never tops 0.63.\n<a href=\"https://www.kaggle.com/moshel/mobilenet-fl\">Here is a little minimal kernel</a> I wrote to show the problem. With FOCAL_LOSS set to True, i get </p>\n\n<pre><code>loss: 1.2022 - acc: 0.4562 - val_loss: 1.4919 - val_acc: 0.4818 \n</code></pre>\n\n<p>with FOCAL_LOSS set to False, I get:</p>\n\n<pre><code>loss: 0.1360 - acc: 0.9526 - val_loss: 0.1623 - val_acc: 0.9473\n</code></pre>\n\n<p>(The kernel is a demo kernel - I know the generator sucks, mobilenet is wrong etc. I did not use the imagenet weights just in case it is somehow not the same, but even with the weights the behavior is about the same, as well as using resnet50 and a good generator)</p>\n\n<p>I understand that the loss is very different between focal loss and BCE, but I expected the accuracy to be good. Anyways even if you keep training this kernel for many epochs it will not pass the 0.63, so something is obviously wrong here but I can't figure out what.</p>\n\n<p>If anyone can take a look I would be most grateful.</p>",
      "rawMarkdown": "@bestfitting detailed in his [post][1] that he used focal loss, so I was determined to try it, just for fun and knowledge.\nI have converted [his implementation][2] from pytorch to keras, but no matter what I tried, the accuracy never tops 0.63.\n[Here is a little minimal kernel][3] I wrote to show the problem. With FOCAL_LOSS set to True, i get \n\n    loss: 1.2022 - acc: 0.4562 - val_loss: 1.4919 - val_acc: 0.4818 \n\nwith FOCAL_LOSS set to False, I get:\n\n    loss: 0.1360 - acc: 0.9526 - val_loss: 0.1623 - val_acc: 0.9473\n\n(The kernel is a demo kernel - I know the generator sucks, mobilenet is wrong etc. I did not use the imagenet weights just in case it is somehow not the same, but even with the weights the behavior is about the same, as well as using resnet50 and a good generator)\n\nI understand that the loss is very different between focal loss and BCE, but I expected the accuracy to be good. Anyways even if you keep training this kernel for many epochs it will not pass the 0.63, so something is obviously wrong here but I can't figure out what.\n\nIf anyone can take a look I would be most grateful.\n\n\n  [1]: https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\n  [2]: https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109#462058\n  [3]: https://www.kaggle.com/moshel/mobilenet-fl",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "464453": "@bestfitting detailed in his [post][1] that he used focal loss, so I was determined to try it, just for fun and knowledge.\nI have converted [his implementation][2] from pytorch to keras, but no matter what I tried, the accuracy never tops 0.63.\n[Here is a little minimal kernel][3] I wrote to show the problem. With FOCAL_LOSS set to True, i get \n\n    loss: 1.2022 - acc: 0.4562 - val_loss: 1.4919 - val_acc: 0.4818 \n\nwith FOCAL_LOSS set to False, I get:\n\n    loss: 0.1360 - acc: 0.9526 - val_loss: 0.1623 - val_acc: 0.9473\n\n(The kernel is a demo kernel - I know the generator sucks, mobilenet is wrong etc. I did not use the imagenet weights just in case it is somehow not the same, but even with the weights the behavior is about the same, as well as using resnet50 and a good generator)\n\nI understand that the loss is very different between focal loss and BCE, but I expected the accuracy to be good. Anyways even if you keep training this kernel for many epochs it will not pass the 0.63, so something is obviously wrong here but I can't figure out what.\n\nIf anyone can take a look I would be most grateful.\n\n\n  [1]: https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\n  [2]: https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109#462058\n  [3]: https://www.kaggle.com/moshel/mobilenet-fl"
  }
}