{
  "id": 70596,
  "title": "focal loss, lovasz understanding",
  "url": "/competitions/airbus-ship-detection/discussion/70596",
  "author_name": "",
  "post_date": "2018-11-05T15:42:46.628821200Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello,\nI've been training my model with bce + dice loss until convergence. I've seen people saying that we can continue training the model with focal loss or lovasz. \n What I have tried is changing the last layer activation to softmax then train with lovasz or keep the sigmoid and train with focal loss. However when I do that, the bce dice loss and the IoU goes the opposite way when these losses are being optimized. Also sometime I have a loss that is over 1000. </p>\n\n<p>I'm trying to learn this notion but I kind of mixing of many things ( like logits too, = no activation ?)</p>\n\n<p>Can anyone please explain to me how to that with Keras ? and why I'm getting decreasing focal when increasing IoU or bce+dice, or vice versa ?</p>\n\n<p>Many thanks!</p>",
  "messages": [
    {
      "id": "415743",
      "postDate": "11/05/2018 15:42:46",
      "content": "<p>Hello,\nI've been training my model with bce + dice loss until convergence. I've seen people saying that we can continue training the model with focal loss or lovasz. \n What I have tried is changing the last layer activation to softmax then train with lovasz or keep the sigmoid and train with focal loss. However when I do that, the bce dice loss and the IoU goes the opposite way when these losses are being optimized. Also sometime I have a loss that is over 1000. </p>\n\n<p>I'm trying to learn this notion but I kind of mixing of many things ( like logits too, = no activation ?)</p>\n\n<p>Can anyone please explain to me how to that with Keras ? and why I'm getting decreasing focal when increasing IoU or bce+dice, or vice versa ?</p>\n\n<p>Many thanks!</p>",
      "rawMarkdown": "Hello,\nI've been training my model with bce + dice loss until convergence. I've seen people saying that we can continue training the model with focal loss or lovasz. \n What I have tried is changing the last layer activation to softmax then train with lovasz or keep the sigmoid and train with focal loss. However when I do that, the bce dice loss and the IoU goes the opposite way when these losses are being optimized. Also sometime I have a loss that is over 1000. \n\nI'm trying to learn this notion but I kind of mixing of many things ( like logits too, = no activation ?)\n\nCan anyone please explain to me how to that with Keras ? and why I'm getting decreasing focal when increasing IoU or bce+dice, or vice versa ?\n\nMany thanks!",
      "votes": null
    },
    {
      "id": "415752",
      "postDate": "11/05/2018 16:10:20",
      "content": "<p>Logits are outputs of the last layer, unscaled and without activation. For lovasz you need to pass the logits directly, without softmax. If your output shape is HxWx1 than use lovasz-hinge, if it is HxWx2 you need to use lovasz-softmax (it computes the softmax internally).</p>\n\n<p>I can't help you with code because I use Pytorch :/</p>",
      "rawMarkdown": "Logits are outputs of the last layer, unscaled and without activation. For lovasz you need to pass the logits directly, without softmax. If your output shape is HxWx1 than use lovasz-hinge, if it is HxWx2 you need to use lovasz-softmax (it computes the softmax internally).\n\nI can't help you with code because I use Pytorch :/",
      "votes": null
    },
    {
      "id": "416214",
      "postDate": "11/06/2018 12:09:31",
      "content": "<p>Thank you for the help. I might learn Pytorch as well as I see it gives more flexibility than Keras</p>",
      "rawMarkdown": "Thank you for the help. I might learn Pytorch as well as I see it gives more flexibility than Keras",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 415752,
      "author_name": "arc144",
      "author_url": "",
      "post_date": "11/05/2018 16:10:20",
      "content": "<p>Logits are outputs of the last layer, unscaled and without activation. For lovasz you need to pass the logits directly, without softmax. If your output shape is HxWx1 than use lovasz-hinge, if it is HxWx2 you need to use lovasz-softmax (it computes the softmax internally).</p>\n\n<p>I can't help you with code because I use Pytorch :/</p>",
      "votes": null,
      "replies": [
        {
          "id": 416214,
          "author_name": "hdatascientist",
          "author_url": "",
          "post_date": "11/06/2018 12:09:31",
          "content": "<p>Thank you for the help. I might learn Pytorch as well as I see it gives more flexibility than Keras</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "415743": "Hello,\nI've been training my model with bce + dice loss until convergence. I've seen people saying that we can continue training the model with focal loss or lovasz. \n What I have tried is changing the last layer activation to softmax then train with lovasz or keep the sigmoid and train with focal loss. However when I do that, the bce dice loss and the IoU goes the opposite way when these losses are being optimized. Also sometime I have a loss that is over 1000. \n\nI'm trying to learn this notion but I kind of mixing of many things ( like logits too, = no activation ?)\n\nCan anyone please explain to me how to that with Keras ? and why I'm getting decreasing focal when increasing IoU or bce+dice, or vice versa ?\n\nMany thanks!",
    "415752": "Logits are outputs of the last layer, unscaled and without activation. For lovasz you need to pass the logits directly, without softmax. If your output shape is HxWx1 than use lovasz-hinge, if it is HxWx2 you need to use lovasz-softmax (it computes the softmax internally).\n\nI can't help you with code because I use Pytorch :/",
    "416214": "Thank you for the help. I might learn Pytorch as well as I see it gives more flexibility than Keras"
  },
  "source": "meta"
}