{
  "id": 37808,
  "title": "Weighted DICE loss",
  "url": "/competitions/carvana-image-masking-challenge/discussion/37808",
  "author_name": "",
  "post_date": "2017-08-09T19:00:33.080264800Z",
  "votes": 4,
  "comment_count": 6,
  "views": 0,
  "content": "<p>There's relatively heavy class imbalance, car vs. background is 0.21 vs. 0.79 (please double-check).</p>\n\n<p>I implemented the following weighted DICE loss as per <a href=\"https://arxiv.org/abs/1707.03237\">https://arxiv.org/abs/1707.03237</a></p>\n\n<pre><code>def weighted_dice_coef(y_true, y_pred):\nmean = 0.21649066\nw_1 = 1/mean**2\nw_0 = 1/(1-mean)**2\ny_true_f_1 = K.flatten(y_true)\ny_pred_f_1 = K.flatten(y_pred)\ny_true_f_0 = K.flatten(1-y_true)\ny_pred_f_0 = K.flatten(1-y_pred)\n\nintersection_0 = K.sum(y_true_f_0 * y_pred_f_0)\nintersection_1 = K.sum(y_true_f_1 * y_pred_f_1)\n\nreturn 2 * (w_0 * intersection_0 + w_1 * intersection_1) / ((w_0 * (K.sum(y_true_f_0) + K.sum(y_pred_f_0))) + (w_1 * (K.sum(y_true_f_1) + K.sum(y_pred_f_1))))\n</code></pre>\n\n<p>Anybody willing to try and report? (my training takes forever, I will report back in ~24 hours or so)</p>",
  "messages": [
    {
      "id": "211706",
      "postDate": "08/09/2017 19:00:33",
      "content": "<p>There's relatively heavy class imbalance, car vs. background is 0.21 vs. 0.79 (please double-check).</p>\n\n<p>I implemented the following weighted DICE loss as per <a href=\"https://arxiv.org/abs/1707.03237\">https://arxiv.org/abs/1707.03237</a></p>\n\n<pre><code>def weighted_dice_coef(y_true, y_pred):\nmean = 0.21649066\nw_1 = 1/mean**2\nw_0 = 1/(1-mean)**2\ny_true_f_1 = K.flatten(y_true)\ny_pred_f_1 = K.flatten(y_pred)\ny_true_f_0 = K.flatten(1-y_true)\ny_pred_f_0 = K.flatten(1-y_pred)\n\nintersection_0 = K.sum(y_true_f_0 * y_pred_f_0)\nintersection_1 = K.sum(y_true_f_1 * y_pred_f_1)\n\nreturn 2 * (w_0 * intersection_0 + w_1 * intersection_1) / ((w_0 * (K.sum(y_true_f_0) + K.sum(y_pred_f_0))) + (w_1 * (K.sum(y_true_f_1) + K.sum(y_pred_f_1))))\n</code></pre>\n\n<p>Anybody willing to try and report? (my training takes forever, I will report back in ~24 hours or so)</p>",
      "rawMarkdown": "There's relatively heavy class imbalance, car vs. background is 0.21 vs. 0.79 (please double-check).\n\nI implemented the following weighted DICE loss as per https://arxiv.org/abs/1707.03237\n\n    def weighted_dice_coef(y_true, y_pred):\n    mean = 0.21649066\n    w_1 = 1/mean**2\n    w_0 = 1/(1-mean)**2\n    y_true_f_1 = K.flatten(y_true)\n    y_pred_f_1 = K.flatten(y_pred)\n    y_true_f_0 = K.flatten(1-y_true)\n    y_pred_f_0 = K.flatten(1-y_pred)\n\n    intersection_0 = K.sum(y_true_f_0 * y_pred_f_0)\n    intersection_1 = K.sum(y_true_f_1 * y_pred_f_1)\n\n    return 2 * (w_0 * intersection_0 + w_1 * intersection_1) / ((w_0 * (K.sum(y_true_f_0) + K.sum(y_pred_f_0))) + (w_1 * (K.sum(y_true_f_1) + K.sum(y_pred_f_1))))\n\nAnybody willing to try and report? (my training takes forever, I will report back in ~24 hours or so)",
      "votes": null
    },
    {
      "id": "211902",
      "postDate": "08/10/2017 07:27:35",
      "content": "<p>I've been using this function from the beginning but without weighting. That is I use the dice loss for both foreground and background. With 320x480, LB at 99.2, same as in my validation result. with 640x960 I got 99.5. Note that I only trained the net for few epochs. I believe that cross-validating the learning rate schedule will benefit better. </p>",
      "rawMarkdown": "I've been using this function from the beginning but without weighting. That is I use the dice loss for both foreground and background. With 320x480, LB at 99.2, same as in my validation result. with 640x960 I got 99.5. Note that I only trained the net for few epochs. I believe that cross-validating the learning rate schedule will benefit better.",
      "votes": null
    },
    {
      "id": "211917",
      "postDate": "08/10/2017 08:05:16",
      "content": "<blockquote>\n  <p>cross-validating the learning rate </p>\n</blockquote>\n\n<p>you mean hyper-parameter search?</p>",
      "rawMarkdown": "&gt; cross-validating the learning rate \n\nyou mean hyper-parameter search?",
      "votes": null
    },
    {
      "id": "212144",
      "postDate": "08/10/2017 19:23:46",
      "content": "<p>I think the problem is may not background and foreground pixel. Rather it is boundary pixel and inner pixel. There is little boundary pixels in one image and this is also where the most errors occurs. So you may want to weigh the boundary pixels.</p>\n\n<p>Boundary pixels can be both background and foreground.</p>\n\n<p>This is just an idea. i haven't try it yet.</p>",
      "rawMarkdown": "I think the problem is may not background and foreground pixel. Rather it is boundary pixel and inner pixel. There is little boundary pixels in one image and this is also where the most errors occurs. So you may want to weigh the boundary pixels.\n\nBoundary pixels can be both background and foreground.\n\nThis is just an idea. i haven't try it yet.",
      "votes": null
    },
    {
      "id": "212372",
      "postDate": "08/11/2017 13:38:37",
      "content": "<p>you can find more discussion at my thread:\n<a href=\"https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208\">https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208</a></p>\n\n<p>experiments for weighing pixels at the boundary. see attachment pictures.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/212369/7044/weighted_dice_1.png\" alt=\"enter image description here\" title=\"\">\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/212369/7045/weighted_dice_2.png\" alt=\"enter image description here\" title=\"\"></p>",
      "rawMarkdown": "you can find more discussion at my thread:\nhttps://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208\n\n\nexperiments for weighing pixels at the boundary. see attachment pictures.\n\n  ![enter image description here][1]\n  ![enter image description here][2]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/212369/7044/weighted_dice_1.png\n  [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/212369/7045/weighted_dice_2.png",
      "votes": null
    },
    {
      "id": "212373",
      "postDate": "08/11/2017 13:39:32",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "212957",
      "postDate": "08/13/2017 08:18:22",
      "content": "<p>I also implemented a similar thing, a bit more soft perhaps. I took the exponential of the distance from each pixel to the boarder pixels, so that the further away from the border pixels were, the less weights it got assigned.</p>\n\n<p>Don't think it makes much difference though in the end :)</p>",
      "rawMarkdown": "I also implemented a similar thing, a bit more soft perhaps. I took the exponential of the distance from each pixel to the boarder pixels, so that the further away from the border pixels were, the less weights it got assigned.\n\nDon't think it makes much difference though in the end :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 211902,
      "author_name": "viebboy",
      "author_url": "",
      "post_date": "08/10/2017 07:27:35",
      "content": "<p>I've been using this function from the beginning but without weighting. That is I use the dice loss for both foreground and background. With 320x480, LB at 99.2, same as in my validation result. with 640x960 I got 99.5. Note that I only trained the net for few epochs. I believe that cross-validating the learning rate schedule will benefit better. </p>",
      "votes": null,
      "replies": [
        {
          "id": 211917,
          "author_name": "antorsae",
          "author_url": "",
          "post_date": "08/10/2017 08:05:16",
          "content": "<blockquote>\n  <p>cross-validating the learning rate </p>\n</blockquote>\n\n<p>you mean hyper-parameter search?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 212144,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "08/10/2017 19:23:46",
      "content": "<p>I think the problem is may not background and foreground pixel. Rather it is boundary pixel and inner pixel. There is little boundary pixels in one image and this is also where the most errors occurs. So you may want to weigh the boundary pixels.</p>\n\n<p>Boundary pixels can be both background and foreground.</p>\n\n<p>This is just an idea. i haven't try it yet.</p>",
      "votes": null,
      "replies": [
        {
          "id": 212372,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "08/11/2017 13:38:37",
          "content": "<p>you can find more discussion at my thread:\n<a href=\"https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208\">https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208</a></p>\n\n<p>experiments for weighing pixels at the boundary. see attachment pictures.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/212369/7044/weighted_dice_1.png\" alt=\"enter image description here\" title=\"\">\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/212369/7045/weighted_dice_2.png\" alt=\"enter image description here\" title=\"\"></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 212957,
          "author_name": "adamhart",
          "author_url": "",
          "post_date": "08/13/2017 08:18:22",
          "content": "<p>I also implemented a similar thing, a bit more soft perhaps. I took the exponential of the distance from each pixel to the boarder pixels, so that the further away from the border pixels were, the less weights it got assigned.</p>\n\n<p>Don't think it makes much difference though in the end :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 212373,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "08/11/2017 13:39:32",
      "content": "",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "211706": "There's relatively heavy class imbalance, car vs. background is 0.21 vs. 0.79 (please double-check).\n\nI implemented the following weighted DICE loss as per https://arxiv.org/abs/1707.03237\n\n    def weighted_dice_coef(y_true, y_pred):\n    mean = 0.21649066\n    w_1 = 1/mean**2\n    w_0 = 1/(1-mean)**2\n    y_true_f_1 = K.flatten(y_true)\n    y_pred_f_1 = K.flatten(y_pred)\n    y_true_f_0 = K.flatten(1-y_true)\n    y_pred_f_0 = K.flatten(1-y_pred)\n\n    intersection_0 = K.sum(y_true_f_0 * y_pred_f_0)\n    intersection_1 = K.sum(y_true_f_1 * y_pred_f_1)\n\n    return 2 * (w_0 * intersection_0 + w_1 * intersection_1) / ((w_0 * (K.sum(y_true_f_0) + K.sum(y_pred_f_0))) + (w_1 * (K.sum(y_true_f_1) + K.sum(y_pred_f_1))))\n\nAnybody willing to try and report? (my training takes forever, I will report back in ~24 hours or so)",
    "211902": "I've been using this function from the beginning but without weighting. That is I use the dice loss for both foreground and background. With 320x480, LB at 99.2, same as in my validation result. with 640x960 I got 99.5. Note that I only trained the net for few epochs. I believe that cross-validating the learning rate schedule will benefit better.",
    "211917": "&gt; cross-validating the learning rate \n\nyou mean hyper-parameter search?",
    "212144": "I think the problem is may not background and foreground pixel. Rather it is boundary pixel and inner pixel. There is little boundary pixels in one image and this is also where the most errors occurs. So you may want to weigh the boundary pixels.\n\nBoundary pixels can be both background and foreground.\n\nThis is just an idea. i haven't try it yet.",
    "212372": "you can find more discussion at my thread:\nhttps://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208\n\n\nexperiments for weighing pixels at the boundary. see attachment pictures.\n\n  ![enter image description here][1]\n  ![enter image description here][2]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/212369/7044/weighted_dice_1.png\n  [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/212369/7045/weighted_dice_2.png",
    "212373": "",
    "212957": "I also implemented a similar thing, a bit more soft perhaps. I took the exponential of the distance from each pixel to the boarder pixels, so that the further away from the border pixels were, the less weights it got assigned.\n\nDon't think it makes much difference though in the end :)"
  },
  "source": "meta"
}