{
  "id": 114880,
  "title": "[ICCV:2019]Loss function for Noisy Label",
  "url": "/competitions/understanding_cloud_organization/discussion/114880",
  "author_name": "",
  "post_date": "2019-10-29T20:51:05.067317900Z",
  "votes": 8,
  "comment_count": 9,
  "views": 0,
  "content": "<p>Had the chance to learn about this loss function from authors at ICCV:2019. Thought it might be relevant to this competition. </p>\n\n<p><a href=\"https://arxiv.org/abs/1908.06112\">Symmetric Cross Entropy for Robust Learning with Noisy Labels</a></p>\n\n<p><code>Training accurate deep neural networks (DNNs) in the\npresence of noisy labels is an important and challenging\ntask. Though a number of approaches have been proposed\nfor learning with noisy labels, many open issues remain. In\nthis paper, we show that DNN learning with Cross Entropy\n(CE) exhibits overfitting to noisy labels on some classes\n(“easy” classes), but more surprisingly, it also suffers from\nsignificant under learning on some other classes (“hard”\nclasses). Intuitively, CE requires an extra term to facilitate\nlearning of hard classes, and more importantly, this term\nshould be noise tolerant, so as to avoid overfitting to noisy\nlabels. Inspired by the symmetric KL-divergence, we propose the approach of Symmetric cross entropy Learning\n(SL), boosting CE symmetrically with a noise robust counterpart Reverse Cross Entropy (RCE). Our proposed SL approach simultaneously addresses both the under learning\nand overfitting problem of CE in the presence of noisy labels. We provide a theoretical analysis of SL and also empirically show, on a range of benchmark and real-world\ndatasets, that SL outperforms state-of-the-art methods. We\nalso show that SL can be easily incorporated into existing\nmethods in order to further enhance their performance</code></p>",
  "messages": [
    {
      "id": "660983",
      "postDate": "10/29/2019 20:51:05",
      "content": "<p>Had the chance to learn about this loss function from authors at ICCV:2019. Thought it might be relevant to this competition. </p>\n\n<p><a href=\"https://arxiv.org/abs/1908.06112\">Symmetric Cross Entropy for Robust Learning with Noisy Labels</a></p>\n\n<p><code>Training accurate deep neural networks (DNNs) in the\npresence of noisy labels is an important and challenging\ntask. Though a number of approaches have been proposed\nfor learning with noisy labels, many open issues remain. In\nthis paper, we show that DNN learning with Cross Entropy\n(CE) exhibits overfitting to noisy labels on some classes\n(“easy” classes), but more surprisingly, it also suffers from\nsignificant under learning on some other classes (“hard”\nclasses). Intuitively, CE requires an extra term to facilitate\nlearning of hard classes, and more importantly, this term\nshould be noise tolerant, so as to avoid overfitting to noisy\nlabels. Inspired by the symmetric KL-divergence, we propose the approach of Symmetric cross entropy Learning\n(SL), boosting CE symmetrically with a noise robust counterpart Reverse Cross Entropy (RCE). Our proposed SL approach simultaneously addresses both the under learning\nand overfitting problem of CE in the presence of noisy labels. We provide a theoretical analysis of SL and also empirically show, on a range of benchmark and real-world\ndatasets, that SL outperforms state-of-the-art methods. We\nalso show that SL can be easily incorporated into existing\nmethods in order to further enhance their performance</code></p>",
      "rawMarkdown": "Had the chance to learn about this loss function from authors at ICCV:2019. Thought it might be relevant to this competition. \n\n[Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://arxiv.org/abs/1908.06112)\n\n```Training accurate deep neural networks (DNNs) in the\npresence of noisy labels is an important and challenging\ntask. Though a number of approaches have been proposed\nfor learning with noisy labels, many open issues remain. In\nthis paper, we show that DNN learning with Cross Entropy\n(CE) exhibits overfitting to noisy labels on some classes\n(“easy” classes), but more surprisingly, it also suffers from\nsignificant under learning on some other classes (“hard”\nclasses). Intuitively, CE requires an extra term to facilitate\nlearning of hard classes, and more importantly, this term\nshould be noise tolerant, so as to avoid overfitting to noisy\nlabels. Inspired by the symmetric KL-divergence, we propose the approach of Symmetric cross entropy Learning\n(SL), boosting CE symmetrically with a noise robust counterpart Reverse Cross Entropy (RCE). Our proposed SL approach simultaneously addresses both the under learning\nand overfitting problem of CE in the presence of noisy labels. We provide a theoretical analysis of SL and also empirically show, on a range of benchmark and real-world\ndatasets, that SL outperforms state-of-the-art methods. We\nalso show that SL can be easily incorporated into existing\nmethods in order to further enhance their performance```",
      "votes": null
    },
    {
      "id": "661278",
      "postDate": "10/30/2019 04:34:28",
      "content": "<p>good find my friend but you forgot to attach the link of this loss function implementation,here it is : <a href=\"https://github.com/YisenWang/symmetric_cross_entropy_for_noisy_labels\">https://github.com/YisenWang/symmetric_cross_entropy_for_noisy_labels</a></p>\n\n<p>i might will give it a try,thanks a ton</p>",
      "rawMarkdown": "good find my friend but you forgot to attach the link of this loss function implementation,here it is : https://github.com/YisenWang/symmetric_cross_entropy_for_noisy_labels\n\ni might will give it a try,thanks a ton",
      "votes": null
    },
    {
      "id": "661280",
      "postDate": "10/30/2019 04:35:21",
      "content": "<p>Another one : <a href=\"https://github.com/HanxunHuangLemonBear/SCELoss-Reproduce\">https://github.com/HanxunHuangLemonBear/SCELoss-Reproduce</a></p>",
      "rawMarkdown": "Another one : https://github.com/HanxunHuangLemonBear/SCELoss-Reproduce",
      "votes": null
    },
    {
      "id": "661286",
      "postDate": "10/30/2019 04:51:33",
      "content": "<p>Thanks man!! Appreciate your effort</p>",
      "rawMarkdown": "Thanks man!! Appreciate your effort",
      "votes": null
    },
    {
      "id": "661295",
      "postDate": "10/30/2019 05:08:37",
      "content": "<p>So output activation should be a softmax for this ?</p>",
      "rawMarkdown": "So output activation should be a softmax for this ?",
      "votes": null
    },
    {
      "id": "661368",
      "postDate": "10/30/2019 07:47:54",
      "content": "<p>Thanks.  I'll try this loss function.</p>",
      "rawMarkdown": "Thanks.  I'll try this loss function.",
      "votes": null
    },
    {
      "id": "661946",
      "postDate": "10/30/2019 22:28:07",
      "content": "<blockquote>\n  <p><strong>zhangeng wrote:</strong></p>\n  \n  <p>Thanks.  I'll try this loss function.</p>\n</blockquote>\n\n<p>It seems that this loss function doesn't work very well.</p>",
      "rawMarkdown": "&gt; **zhangeng wrote:**\n&gt; \n&gt; Thanks.  I'll try this loss function.\n\nIt seems that this loss function doesn't work very well.",
      "votes": null
    },
    {
      "id": "664642",
      "postDate": "11/04/2019 02:16:54",
      "content": "<p>Thank you for your share</p>",
      "rawMarkdown": "Thank you for your share",
      "votes": null
    },
    {
      "id": "666415",
      "postDate": "11/06/2019 04:46:33",
      "content": "<p>in most paper, the train data is noisy but test data is clean.</p>\n\n<p>here both train and test are noisy. hence the method may or may not apply here</p>",
      "rawMarkdown": "in most paper, the train data is noisy but test data is clean.\n\nhere both train and test are noisy. hence the method may or may not apply here",
      "votes": null
    },
    {
      "id": "666418",
      "postDate": "11/06/2019 04:50:27",
      "content": "<p>good catch,now it seems like this method too won't work well</p>",
      "rawMarkdown": "good catch,now it seems like this method too won't work well",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 661278,
      "author_name": "mobassir",
      "author_url": "",
      "post_date": "10/30/2019 04:34:28",
      "content": "<p>good find my friend but you forgot to attach the link of this loss function implementation,here it is : <a href=\"https://github.com/YisenWang/symmetric_cross_entropy_for_noisy_labels\">https://github.com/YisenWang/symmetric_cross_entropy_for_noisy_labels</a></p>\n\n<p>i might will give it a try,thanks a ton</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 661280,
      "author_name": "mobassir",
      "author_url": "",
      "post_date": "10/30/2019 04:35:21",
      "content": "<p>Another one : <a href=\"https://github.com/HanxunHuangLemonBear/SCELoss-Reproduce\">https://github.com/HanxunHuangLemonBear/SCELoss-Reproduce</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 661286,
          "author_name": "bibek777",
          "author_url": "",
          "post_date": "10/30/2019 04:51:33",
          "content": "<p>Thanks man!! Appreciate your effort</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 661295,
      "author_name": "phoenix9032",
      "author_url": "",
      "post_date": "10/30/2019 05:08:37",
      "content": "<p>So output activation should be a softmax for this ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 661368,
      "author_name": "zhangeng",
      "author_url": "",
      "post_date": "10/30/2019 07:47:54",
      "content": "<p>Thanks.  I'll try this loss function.</p>",
      "votes": null,
      "replies": [
        {
          "id": 661946,
          "author_name": "zhangeng",
          "author_url": "",
          "post_date": "10/30/2019 22:28:07",
          "content": "<blockquote>\n  <p><strong>zhangeng wrote:</strong></p>\n  \n  <p>Thanks.  I'll try this loss function.</p>\n</blockquote>\n\n<p>It seems that this loss function doesn't work very well.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 664642,
          "author_name": "gustavo219",
          "author_url": "",
          "post_date": "11/04/2019 02:16:54",
          "content": "<p>Thank you for your share</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 666415,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "11/06/2019 04:46:33",
      "content": "<p>in most paper, the train data is noisy but test data is clean.</p>\n\n<p>here both train and test are noisy. hence the method may or may not apply here</p>",
      "votes": null,
      "replies": [
        {
          "id": 666418,
          "author_name": "mobassir",
          "author_url": "",
          "post_date": "11/06/2019 04:50:27",
          "content": "<p>good catch,now it seems like this method too won't work well</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "660983": "Had the chance to learn about this loss function from authors at ICCV:2019. Thought it might be relevant to this competition. \n\n[Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://arxiv.org/abs/1908.06112)\n\n```Training accurate deep neural networks (DNNs) in the\npresence of noisy labels is an important and challenging\ntask. Though a number of approaches have been proposed\nfor learning with noisy labels, many open issues remain. In\nthis paper, we show that DNN learning with Cross Entropy\n(CE) exhibits overfitting to noisy labels on some classes\n(“easy” classes), but more surprisingly, it also suffers from\nsignificant under learning on some other classes (“hard”\nclasses). Intuitively, CE requires an extra term to facilitate\nlearning of hard classes, and more importantly, this term\nshould be noise tolerant, so as to avoid overfitting to noisy\nlabels. Inspired by the symmetric KL-divergence, we propose the approach of Symmetric cross entropy Learning\n(SL), boosting CE symmetrically with a noise robust counterpart Reverse Cross Entropy (RCE). Our proposed SL approach simultaneously addresses both the under learning\nand overfitting problem of CE in the presence of noisy labels. We provide a theoretical analysis of SL and also empirically show, on a range of benchmark and real-world\ndatasets, that SL outperforms state-of-the-art methods. We\nalso show that SL can be easily incorporated into existing\nmethods in order to further enhance their performance```",
    "661278": "good find my friend but you forgot to attach the link of this loss function implementation,here it is : https://github.com/YisenWang/symmetric_cross_entropy_for_noisy_labels\n\ni might will give it a try,thanks a ton",
    "661280": "Another one : https://github.com/HanxunHuangLemonBear/SCELoss-Reproduce",
    "661286": "Thanks man!! Appreciate your effort",
    "661295": "So output activation should be a softmax for this ?",
    "661368": "Thanks.  I'll try this loss function.",
    "661946": "&gt; **zhangeng wrote:**\n&gt; \n&gt; Thanks.  I'll try this loss function.\n\nIt seems that this loss function doesn't work very well.",
    "664642": "Thank you for your share",
    "666415": "in most paper, the train data is noisy but test data is clean.\n\nhere both train and test are noisy. hence the method may or may not apply here",
    "666418": "good catch,now it seems like this method too won't work well"
  },
  "source": "meta"
}