{
  "id": 465638,
  "title": "Negative loss function?",
  "url": "/competitions/blood-vessel-segmentation/discussion/465638",
  "author_name": "",
  "post_date": "2024-01-05T03:15:34.092395600Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi Everyone,</p>\n<p>I'm having a little trouble training a segmentation model. I've converted the masks to shape (256,256,1) that I have cast to tf.float32 and I have a U-Net I'm trying to train with images that have the same shape and datatype. I've been careful not to mix up my labels, so the masks are paired with the correct images. However, I keep getting a negative loss function when I train the network (whether I use \"binary_crossentropy\" or the dice loss function shown below). I found this loss function on stack overflow and I've seen a few similar ones on tutorials, so I think it's probably right (binary_crossentropy is native, so that confirms something odd is going on).</p>\n<p>I've not seen something like this before, but this is also my first competition with segmentation. Has anyone else seen this? Any tips to troubleshoot it? I thought maybe it was a batch thing, but I get the same behavior even with a batch size of 1.</p>\n<p>Loss function, in case it does prove helpful: </p>\n<pre><code> ():        \n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = K.(y_true_f * y_pred_f)\n    dice = ( * intersection + smooth) / (K.(y_true_f) + K.(y_pred_f) + smooth)\n     dice\n\n ():\n     -dice_coef(y_true, y_pred)\n</code></pre>",
  "messages": [
    {
      "id": "2587784",
      "postDate": "01/05/2024 03:15:34",
      "content": "<p>Hi Everyone,</p>\n<p>I'm having a little trouble training a segmentation model. I've converted the masks to shape (256,256,1) that I have cast to tf.float32 and I have a U-Net I'm trying to train with images that have the same shape and datatype. I've been careful not to mix up my labels, so the masks are paired with the correct images. However, I keep getting a negative loss function when I train the network (whether I use \"binary_crossentropy\" or the dice loss function shown below). I found this loss function on stack overflow and I've seen a few similar ones on tutorials, so I think it's probably right (binary_crossentropy is native, so that confirms something odd is going on).</p>\n<p>I've not seen something like this before, but this is also my first competition with segmentation. Has anyone else seen this? Any tips to troubleshoot it? I thought maybe it was a batch thing, but I get the same behavior even with a batch size of 1.</p>\n<p>Loss function, in case it does prove helpful: </p>\n<pre><code> ():        \n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = K.(y_true_f * y_pred_f)\n    dice = ( * intersection + smooth) / (K.(y_true_f) + K.(y_pred_f) + smooth)\n     dice\n\n ():\n     -dice_coef(y_true, y_pred)\n</code></pre>",
      "rawMarkdown": "Hi Everyone,\n\nI'm having a little trouble training a segmentation model. I've converted the masks to shape (256,256,1) that I have cast to tf.float32 and I have a U-Net I'm trying to train with images that have the same shape and datatype. I've been careful not to mix up my labels, so the masks are paired with the correct images. However, I keep getting a negative loss function when I train the network (whether I use \"binary_crossentropy\" or the dice loss function shown below). I found this loss function on stack overflow and I've seen a few similar ones on tutorials, so I think it's probably right (binary_crossentropy is native, so that confirms something odd is going on).\n\nI've not seen something like this before, but this is also my first competition with segmentation. Has anyone else seen this? Any tips to troubleshoot it? I thought maybe it was a batch thing, but I get the same behavior even with a batch size of 1.\n\nLoss function, in case it does prove helpful: \n\n```python\ndef dice_coef(y_true, y_pred, smooth=1):        \n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = K.sum(y_true_f * y_pred_f)\n    dice = (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n    return dice\n\ndef dice_loss(y_true, y_pred):\n    return 1-dice_coef(y_true, y_pred)\n```",
      "votes": null
    },
    {
      "id": "2588065",
      "postDate": "01/05/2024 08:41:02",
      "content": "<p>I think smoth is applied only on denominator and I think one is too big.</p>",
      "rawMarkdown": "I think smoth is applied only on denominator and I think one is too big.",
      "votes": null
    },
    {
      "id": "2588122",
      "postDate": "01/05/2024 09:28:39",
      "content": "<p>But I don't think that can lead to negative loss, are your outputs sigmoid between zero and one? What's the shape of your outputs?</p>",
      "rawMarkdown": "But I don't think that can lead to negative loss, are your outputs sigmoid between zero and one? What's the shape of your outputs?",
      "votes": null
    },
    {
      "id": "2588387",
      "postDate": "01/05/2024 13:16:37",
      "content": "<p>Thank you very much for your help. I visually inspected one of the masks and I thought it was constrained to (0,1). I decided to explicitly set it to 1 anywhere it was not 0 and that fixed it!</p>",
      "rawMarkdown": "Thank you very much for your help. I visually inspected one of the masks and I thought it was constrained to (0,1). I decided to explicitly set it to 1 anywhere it was not 0 and that fixed it!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2588065,
      "author_name": "sacuscreed",
      "author_url": "",
      "post_date": "01/05/2024 08:41:02",
      "content": "<p>I think smoth is applied only on denominator and I think one is too big.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2588122,
      "author_name": "sacuscreed",
      "author_url": "",
      "post_date": "01/05/2024 09:28:39",
      "content": "<p>But I don't think that can lead to negative loss, are your outputs sigmoid between zero and one? What's the shape of your outputs?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2588387,
          "author_name": "chemdatafarmer",
          "author_url": "",
          "post_date": "01/05/2024 13:16:37",
          "content": "<p>Thank you very much for your help. I visually inspected one of the masks and I thought it was constrained to (0,1). I decided to explicitly set it to 1 anywhere it was not 0 and that fixed it!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2587784": "Hi Everyone,\n\nI'm having a little trouble training a segmentation model. I've converted the masks to shape (256,256,1) that I have cast to tf.float32 and I have a U-Net I'm trying to train with images that have the same shape and datatype. I've been careful not to mix up my labels, so the masks are paired with the correct images. However, I keep getting a negative loss function when I train the network (whether I use \"binary_crossentropy\" or the dice loss function shown below). I found this loss function on stack overflow and I've seen a few similar ones on tutorials, so I think it's probably right (binary_crossentropy is native, so that confirms something odd is going on).\n\nI've not seen something like this before, but this is also my first competition with segmentation. Has anyone else seen this? Any tips to troubleshoot it? I thought maybe it was a batch thing, but I get the same behavior even with a batch size of 1.\n\nLoss function, in case it does prove helpful: \n\n```python\ndef dice_coef(y_true, y_pred, smooth=1):        \n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = K.sum(y_true_f * y_pred_f)\n    dice = (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n    return dice\n\ndef dice_loss(y_true, y_pred):\n    return 1-dice_coef(y_true, y_pred)\n```",
    "2588065": "I think smoth is applied only on denominator and I think one is too big.",
    "2588122": "But I don't think that can lead to negative loss, are your outputs sigmoid between zero and one? What's the shape of your outputs?",
    "2588387": "Thank you very much for your help. I visually inspected one of the masks and I thought it was constrained to (0,1). I decided to explicitly set it to 1 anywhere it was not 0 and that fixed it!"
  },
  "source": "meta"
}