{
  "id": 161597,
  "title": "Trouble: Focal Loss with FP16 training  ",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/161597",
  "author_name": "Phaedrus",
  "post_date": "2020-06-25T11:07:17.647000",
  "votes": 1,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi All, My baseline models are setup to use BCEwithLogitsLoss,and they work well with FP16 based training. However, when I shifted to Focal loss, the losses started becoming NANs and all sorts of weird stuff started happening. So for the time being I am not able to use FP16 with Focal loss.</p>\n\n<p>Does anyone here know why this could be happening or point me to a potential solution.  </p>\n\n<p>Thanks,</p>",
  "messages": [
    {
      "id": 901274,
      "postDate": "2020-06-25T11:07:17.647Z",
      "content": "<p>Hi All, My baseline models are setup to use BCEwithLogitsLoss,and they work well with FP16 based training. However, when I shifted to Focal loss, the losses started becoming NANs and all sorts of weird stuff started happening. So for the time being I am not able to use FP16 with Focal loss.</p>\n\n<p>Does anyone here know why this could be happening or point me to a potential solution.  </p>\n\n<p>Thanks,</p>",
      "rawMarkdown": "Hi All, My baseline models are setup to use BCEwithLogitsLoss,and they work well with FP16 based training. However, when I shifted to Focal loss, the losses started becoming NANs and all sorts of weird stuff started happening. So for the time being I am not able to use FP16 with Focal loss.\n\nDoes anyone here know why this could be happening or point me to a potential solution.  \n\nThanks,",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "901274": "Hi All, My baseline models are setup to use BCEwithLogitsLoss,and they work well with FP16 based training. However, when I shifted to Focal loss, the losses started becoming NANs and all sorts of weird stuff started happening. So for the time being I am not able to use FP16 with Focal loss.\n\nDoes anyone here know why this could be happening or point me to a potential solution.  \n\nThanks,"
  }
}