{
  "id": 112937,
  "title": "Lovasz loss",
  "url": "/competitions/understanding_cloud_organization/discussion/112937",
  "author_name": "",
  "post_date": "2019-10-16T07:59:59.623855700Z",
  "votes": 5,
  "comment_count": 9,
  "views": 0,
  "content": "<p>Do I understand correctly that lovasz loss is not suitable in this competion, as some masks can overlap?</p>\n\n<p>LL accepts ground truth labels in [HxW] tensor where all values are 0 to C-1, where C is number of classes. In case of overlapping we have multiple candidates for one position, which is bad. </p>",
  "messages": [
    {
      "id": "650239",
      "postDate": "10/16/2019 07:59:59",
      "content": "<p>Do I understand correctly that lovasz loss is not suitable in this competion, as some masks can overlap?</p>\n\n<p>LL accepts ground truth labels in [HxW] tensor where all values are 0 to C-1, where C is number of classes. In case of overlapping we have multiple candidates for one position, which is bad. </p>",
      "rawMarkdown": "Do I understand correctly that lovasz loss is not suitable in this competion, as some masks can overlap?\n\nLL accepts ground truth labels in [HxW] tensor where all values are 0 to C-1, where C is number of classes. In case of overlapping we have multiple candidates for one position, which is bad.",
      "votes": null
    },
    {
      "id": "651720",
      "postDate": "10/17/2019 21:07:29",
      "content": "<p>With raw target variables we can't use lovasz loss, but if we add one more 'undefined' class, we can use lovasz softmax.</p>",
      "rawMarkdown": "With raw target variables we can't use lovasz loss, but if we add one more 'undefined' class, we can use lovasz softmax.",
      "votes": null
    },
    {
      "id": "652105",
      "postDate": "10/18/2019 11:13:29",
      "content": "<p>How about the sum of lovasz loss for each independent label vs its output channel?</p>",
      "rawMarkdown": "How about the sum of lovasz loss for each independent label vs its output channel?",
      "votes": null
    },
    {
      "id": "653258",
      "postDate": "10/20/2019 05:57:26",
      "content": "<p>That might actually work. Thanks for suggestion.</p>",
      "rawMarkdown": "That might actually work. Thanks for suggestion.",
      "votes": null
    },
    {
      "id": "661138",
      "postDate": "10/30/2019 00:33:01",
      "content": "<p>Hi, did you try Lovasz loss yet? I had tried Lovasz hinge loss on each class(separately calculate the loss), the dice coefficient start to decrease once I switched normal BCE+DICE to Lovasz. Maybe I was doing something wrong..Just want to check the effect from you.</p>",
      "rawMarkdown": "Hi, did you try Lovasz loss yet? I had tried Lovasz hinge loss on each class(separately calculate the loss), the dice coefficient start to decrease once I switched normal BCE+DICE to Lovasz. Maybe I was doing something wrong..Just want to check the effect from you.",
      "votes": null
    },
    {
      "id": "662396",
      "postDate": "10/31/2019 14:10:57",
      "content": "<p>Hi! My implementation of Lovasz loss for this competition is below. I followed the advice in comments and implemented sum of lovasz losses for each label.</p>\n\n<pre><code> class CustomLovaszLoss(_Loss):\n\n    def __init__(self, per_image=False, ignore=None):\n        super().__init__()\n        self.ignore = ignore\n        self.per_image = per_image\n\n    def forward(self, logits, target):\n        #{0: 'Fish', 1: 'Flower', 2: 'Gravel', 3: 'Sugar'}\n        # B x C x H x W\n\n        logits_fish = logits[:, 0, :, :]\n        logits_flower = logits[:, 1, :, :]\n        logits_gravel = logits[:, 2, :, :]\n        logits_sugar = logits[:, 3, :, :]\n\n        target_fish = target[:, 0, :, :]\n        target_flower = target[:, 1, :, :]\n        target_gravel = target[:, 2, :, :]\n        target_sugar = target[:, 3, :, :]\n\n\n        lovasz_fish = _lovasz_hinge(\n            logits_fish, target_fish, per_image=self.per_image, ignore=self.ignore\n        )\n\n        lovasz_flower = _lovasz_hinge(\n            logits_flower, target_flower, per_image=self.per_image, ignore=self.ignore\n        )\n\n        lovasz_gravel = _lovasz_hinge(\n            logits_gravel, target_gravel, per_image=self.per_image, ignore=self.ignore\n        )\n\n        lovasz_sugar = _lovasz_hinge(\n            logits_sugar, target_sugar, per_image=self.per_image, ignore=self.ignore\n        )\n\n        return lovasz_fish + lovasz_flower + lovasz_gravel + lovasz_sugar`\n</code></pre>\n\n<p>I tried to run this code with several architectures, though loss value never goes below 4.0. Probably I misunderstood some concept.</p>",
      "rawMarkdown": "Hi! My implementation of Lovasz loss for this competition is below. I followed the advice in comments and implemented sum of lovasz losses for each label.\n  \n        \n     class CustomLovaszLoss(_Loss):\n        \n        def __init__(self, per_image=False, ignore=None):\n            super().__init__()\n            self.ignore = ignore\n            self.per_image = per_image\n\n        def forward(self, logits, target):\n            #{0: 'Fish', 1: 'Flower', 2: 'Gravel', 3: 'Sugar'}\n            # B x C x H x W\n\n            logits_fish = logits[:, 0, :, :]\n            logits_flower = logits[:, 1, :, :]\n            logits_gravel = logits[:, 2, :, :]\n            logits_sugar = logits[:, 3, :, :]\n\n            target_fish = target[:, 0, :, :]\n            target_flower = target[:, 1, :, :]\n            target_gravel = target[:, 2, :, :]\n            target_sugar = target[:, 3, :, :]\n\n\n            lovasz_fish = _lovasz_hinge(\n                logits_fish, target_fish, per_image=self.per_image, ignore=self.ignore\n            )\n\n            lovasz_flower = _lovasz_hinge(\n                logits_flower, target_flower, per_image=self.per_image, ignore=self.ignore\n            )\n\n            lovasz_gravel = _lovasz_hinge(\n                logits_gravel, target_gravel, per_image=self.per_image, ignore=self.ignore\n            )\n\n            lovasz_sugar = _lovasz_hinge(\n                logits_sugar, target_sugar, per_image=self.per_image, ignore=self.ignore\n            )\n\n            return lovasz_fish + lovasz_flower + lovasz_gravel + lovasz_sugar`\n\nI tried to run this code with several architectures, though loss value never goes below 4.0. Probably I misunderstood some concept.",
      "votes": null
    },
    {
      "id": "662474",
      "postDate": "10/31/2019 15:42:58",
      "content": "<p>Thanks for the information! Normally, Lovasz loss is using for final tuning of model. So maybe you can try this to up train your model after you done training with normal loss function. I had tired to train 4 models on each class by using Lovasz loss. The loss did decreased, but the dice_coef was not so well. Maybe after I get to a certain rank I will continue to try Lovasz loss. But right now, I need to enhance the model on normal loss first..</p>",
      "rawMarkdown": "Thanks for the information! Normally, Lovasz loss is using for final tuning of model. So maybe you can try this to up train your model after you done training with normal loss function. I had tired to train 4 models on each class by using Lovasz loss. The loss did decreased, but the dice_coef was not so well. Maybe after I get to a certain rank I will continue to try Lovasz loss. But right now, I need to enhance the model on normal loss first..",
      "votes": null
    },
    {
      "id": "662993",
      "postDate": "11/01/2019 09:50:49",
      "content": "<p>my experiments so far:\n* lovasz_hinge -&gt; can't converge\n* symmetric_lovasz -&gt; converges fast, but overfits extremely</p>",
      "rawMarkdown": "my experiments so far:\n* lovasz_hinge -&gt; can't converge\n* symmetric_lovasz -&gt; converges fast, but overfits extremely",
      "votes": null
    },
    {
      "id": "662999",
      "postDate": "11/01/2019 10:15:54",
      "content": "<p>If you don't mind, did symmetric_lovasz mean calculating lovasz_hinge on each class separately?</p>",
      "rawMarkdown": "If you don't mind, did symmetric_lovasz mean calculating lovasz_hinge on each class separately?",
      "votes": null
    },
    {
      "id": "663004",
      "postDate": "11/01/2019 10:29:37",
      "content": "<p>on each class:\n<code>\ndef symmetric_lovasz(outputs, targets):\n    return (lovasz_hinge(outputs, targets) + lovasz_hinge(-outputs, 1 - targets)) / 2\n</code></p>",
      "rawMarkdown": "on each class:\n`\ndef symmetric_lovasz(outputs, targets):\n    return (lovasz_hinge(outputs, targets) + lovasz_hinge(-outputs, 1 - targets)) / 2\n`",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 651720,
      "author_name": "miklgr500",
      "author_url": "",
      "post_date": "10/17/2019 21:07:29",
      "content": "<p>With raw target variables we can't use lovasz loss, but if we add one more 'undefined' class, we can use lovasz softmax.</p>",
      "votes": null,
      "replies": [
        {
          "id": 652105,
          "author_name": "virilo",
          "author_url": "",
          "post_date": "10/18/2019 11:13:29",
          "content": "<p>How about the sum of lovasz loss for each independent label vs its output channel?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 653258,
          "author_name": "lightnezzofbeing",
          "author_url": "",
          "post_date": "10/20/2019 05:57:26",
          "content": "<p>That might actually work. Thanks for suggestion.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 661138,
      "author_name": "xiejialun",
      "author_url": "",
      "post_date": "10/30/2019 00:33:01",
      "content": "<p>Hi, did you try Lovasz loss yet? I had tried Lovasz hinge loss on each class(separately calculate the loss), the dice coefficient start to decrease once I switched normal BCE+DICE to Lovasz. Maybe I was doing something wrong..Just want to check the effect from you.</p>",
      "votes": null,
      "replies": [
        {
          "id": 662396,
          "author_name": "lightnezzofbeing",
          "author_url": "",
          "post_date": "10/31/2019 14:10:57",
          "content": "<p>Hi! My implementation of Lovasz loss for this competition is below. I followed the advice in comments and implemented sum of lovasz losses for each label.</p>\n\n<pre><code> class CustomLovaszLoss(_Loss):\n\n    def __init__(self, per_image=False, ignore=None):\n        super().__init__()\n        self.ignore = ignore\n        self.per_image = per_image\n\n    def forward(self, logits, target):\n        #{0: 'Fish', 1: 'Flower', 2: 'Gravel', 3: 'Sugar'}\n        # B x C x H x W\n\n        logits_fish = logits[:, 0, :, :]\n        logits_flower = logits[:, 1, :, :]\n        logits_gravel = logits[:, 2, :, :]\n        logits_sugar = logits[:, 3, :, :]\n\n        target_fish = target[:, 0, :, :]\n        target_flower = target[:, 1, :, :]\n        target_gravel = target[:, 2, :, :]\n        target_sugar = target[:, 3, :, :]\n\n\n        lovasz_fish = _lovasz_hinge(\n            logits_fish, target_fish, per_image=self.per_image, ignore=self.ignore\n        )\n\n        lovasz_flower = _lovasz_hinge(\n            logits_flower, target_flower, per_image=self.per_image, ignore=self.ignore\n        )\n\n        lovasz_gravel = _lovasz_hinge(\n            logits_gravel, target_gravel, per_image=self.per_image, ignore=self.ignore\n        )\n\n        lovasz_sugar = _lovasz_hinge(\n            logits_sugar, target_sugar, per_image=self.per_image, ignore=self.ignore\n        )\n\n        return lovasz_fish + lovasz_flower + lovasz_gravel + lovasz_sugar`\n</code></pre>\n\n<p>I tried to run this code with several architectures, though loss value never goes below 4.0. Probably I misunderstood some concept.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 662474,
          "author_name": "xiejialun",
          "author_url": "",
          "post_date": "10/31/2019 15:42:58",
          "content": "<p>Thanks for the information! Normally, Lovasz loss is using for final tuning of model. So maybe you can try this to up train your model after you done training with normal loss function. I had tired to train 4 models on each class by using Lovasz loss. The loss did decreased, but the dice_coef was not so well. Maybe after I get to a certain rank I will continue to try Lovasz loss. But right now, I need to enhance the model on normal loss first..</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 662993,
      "author_name": "tugstugi",
      "author_url": "",
      "post_date": "11/01/2019 09:50:49",
      "content": "<p>my experiments so far:\n* lovasz_hinge -&gt; can't converge\n* symmetric_lovasz -&gt; converges fast, but overfits extremely</p>",
      "votes": null,
      "replies": [
        {
          "id": 662999,
          "author_name": "xiejialun",
          "author_url": "",
          "post_date": "11/01/2019 10:15:54",
          "content": "<p>If you don't mind, did symmetric_lovasz mean calculating lovasz_hinge on each class separately?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 663004,
          "author_name": "tugstugi",
          "author_url": "",
          "post_date": "11/01/2019 10:29:37",
          "content": "<p>on each class:\n<code>\ndef symmetric_lovasz(outputs, targets):\n    return (lovasz_hinge(outputs, targets) + lovasz_hinge(-outputs, 1 - targets)) / 2\n</code></p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "650239": "Do I understand correctly that lovasz loss is not suitable in this competion, as some masks can overlap?\n\nLL accepts ground truth labels in [HxW] tensor where all values are 0 to C-1, where C is number of classes. In case of overlapping we have multiple candidates for one position, which is bad.",
    "651720": "With raw target variables we can't use lovasz loss, but if we add one more 'undefined' class, we can use lovasz softmax.",
    "652105": "How about the sum of lovasz loss for each independent label vs its output channel?",
    "653258": "That might actually work. Thanks for suggestion.",
    "661138": "Hi, did you try Lovasz loss yet? I had tried Lovasz hinge loss on each class(separately calculate the loss), the dice coefficient start to decrease once I switched normal BCE+DICE to Lovasz. Maybe I was doing something wrong..Just want to check the effect from you.",
    "662396": "Hi! My implementation of Lovasz loss for this competition is below. I followed the advice in comments and implemented sum of lovasz losses for each label.\n  \n        \n     class CustomLovaszLoss(_Loss):\n        \n        def __init__(self, per_image=False, ignore=None):\n            super().__init__()\n            self.ignore = ignore\n            self.per_image = per_image\n\n        def forward(self, logits, target):\n            #{0: 'Fish', 1: 'Flower', 2: 'Gravel', 3: 'Sugar'}\n            # B x C x H x W\n\n            logits_fish = logits[:, 0, :, :]\n            logits_flower = logits[:, 1, :, :]\n            logits_gravel = logits[:, 2, :, :]\n            logits_sugar = logits[:, 3, :, :]\n\n            target_fish = target[:, 0, :, :]\n            target_flower = target[:, 1, :, :]\n            target_gravel = target[:, 2, :, :]\n            target_sugar = target[:, 3, :, :]\n\n\n            lovasz_fish = _lovasz_hinge(\n                logits_fish, target_fish, per_image=self.per_image, ignore=self.ignore\n            )\n\n            lovasz_flower = _lovasz_hinge(\n                logits_flower, target_flower, per_image=self.per_image, ignore=self.ignore\n            )\n\n            lovasz_gravel = _lovasz_hinge(\n                logits_gravel, target_gravel, per_image=self.per_image, ignore=self.ignore\n            )\n\n            lovasz_sugar = _lovasz_hinge(\n                logits_sugar, target_sugar, per_image=self.per_image, ignore=self.ignore\n            )\n\n            return lovasz_fish + lovasz_flower + lovasz_gravel + lovasz_sugar`\n\nI tried to run this code with several architectures, though loss value never goes below 4.0. Probably I misunderstood some concept.",
    "662474": "Thanks for the information! Normally, Lovasz loss is using for final tuning of model. So maybe you can try this to up train your model after you done training with normal loss function. I had tired to train 4 models on each class by using Lovasz loss. The loss did decreased, but the dice_coef was not so well. Maybe after I get to a certain rank I will continue to try Lovasz loss. But right now, I need to enhance the model on normal loss first..",
    "662993": "my experiments so far:\n* lovasz_hinge -&gt; can't converge\n* symmetric_lovasz -&gt; converges fast, but overfits extremely",
    "662999": "If you don't mind, did symmetric_lovasz mean calculating lovasz_hinge on each class separately?",
    "663004": "on each class:\n`\ndef symmetric_lovasz(outputs, targets):\n    return (lovasz_hinge(outputs, targets) + lovasz_hinge(-outputs, 1 - targets)) / 2\n`"
  },
  "source": "meta"
}