{
  "id": 210398,
  "title": "Weighted Label Smoothing",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/210398",
  "author_name": "Gabriel Prado",
  "post_date": "2021-01-10T16:21:50.229000",
  "votes": 7,
  "comment_count": 2,
  "views": 0,
  "content": "<p>This came up as an idea and wanted to share with the forum to know your thoughts. </p>\n<p>We all know by now that classes are imbalanced, and that most models can learn a few classes really well according to confusion matrixes.</p>\n<p>But what if instead of a constant label smooth in which a function transforms [0 1 0 0 0] into [0.05 0.95 0.05 0.05 0.05] for example. Since we know that our model learns class 3 really well, would a transform that transforms [0 1 0 0 0] into [0.05 0.95 0.05 0 0.05] work better than the previous transform? <br>\nAnd if so, would for instance two often confused classes get a boost by differently weighing label smooths? For example, be label_smooth a function that takes an integer and returns the above one hot smoothed Tensors, we have so that the other classes are different as seen in this example: label_smooth(0) (e.g. [0.95 0.03 0.07 0.0 0.17] != label_smooth(4) [0.11 0.09 0.6 0.3 0.95]</p>\n<p>I haven't tried implementing it yet, but wanted to know what you guys think.</p>",
  "messages": [
    {
      "id": 1147657,
      "postDate": "2021-01-10T16:21:50.230Z",
      "content": "<p>This came up as an idea and wanted to share with the forum to know your thoughts. </p>\n<p>We all know by now that classes are imbalanced, and that most models can learn a few classes really well according to confusion matrixes.</p>\n<p>But what if instead of a constant label smooth in which a function transforms [0 1 0 0 0] into [0.05 0.95 0.05 0.05 0.05] for example. Since we know that our model learns class 3 really well, would a transform that transforms [0 1 0 0 0] into [0.05 0.95 0.05 0 0.05] work better than the previous transform? <br>\nAnd if so, would for instance two often confused classes get a boost by differently weighing label smooths? For example, be label_smooth a function that takes an integer and returns the above one hot smoothed Tensors, we have so that the other classes are different as seen in this example: label_smooth(0) (e.g. [0.95 0.03 0.07 0.0 0.17] != label_smooth(4) [0.11 0.09 0.6 0.3 0.95]</p>\n<p>I haven't tried implementing it yet, but wanted to know what you guys think.</p>",
      "rawMarkdown": "This came up as an idea and wanted to share with the forum to know your thoughts. \n\nWe all know by now that classes are imbalanced, and that most models can learn a few classes really well according to confusion matrixes.\n\nBut what if instead of a constant label smooth in which a function transforms [0 1 0 0 0] into [0.05 0.95 0.05 0.05 0.05] for example. Since we know that our model learns class 3 really well, would a transform that transforms [0 1 0 0 0] into [0.05 0.95 0.05 0 0.05] work better than the previous transform? \nAnd if so, would for instance two often confused classes get a boost by differently weighing label smooths? For example, be label_smooth a function that takes an integer and returns the above one hot smoothed Tensors, we have so that the other classes are different as seen in this example: label_smooth(0) (e.g. [0.95 0.03 0.07 0.0 0.17] != label_smooth(4) [0.11 0.09 0.6 0.3 0.95]\n\nI haven't tried implementing it yet, but wanted to know what you guys think.",
      "votes": 7
    },
    {
      "id": 1150218,
      "postDate": "2021-01-12T13:19:12.330Z",
      "content": "<p>Kinda like LDL: <a href=\"https://arxiv.org/abs/1408.6027\" target=\"_blank\">https://arxiv.org/abs/1408.6027</a></p>",
      "rawMarkdown": "Kinda like LDL: https://arxiv.org/abs/1408.6027",
      "votes": 1
    },
    {
      "id": 1147912,
      "postDate": "2021-01-10T19:53:23.193Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1150218,
      "author_name": "KayWang",
      "author_url": "",
      "post_date": "2021-01-12T13:19:12.330000",
      "content": "<p>Kinda like LDL: <a href=\"https://arxiv.org/abs/1408.6027\" target=\"_blank\">https://arxiv.org/abs/1408.6027</a></p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1147912,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-01-10T19:53:23.193000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1147657": "This came up as an idea and wanted to share with the forum to know your thoughts. \n\nWe all know by now that classes are imbalanced, and that most models can learn a few classes really well according to confusion matrixes.\n\nBut what if instead of a constant label smooth in which a function transforms [0 1 0 0 0] into [0.05 0.95 0.05 0.05 0.05] for example. Since we know that our model learns class 3 really well, would a transform that transforms [0 1 0 0 0] into [0.05 0.95 0.05 0 0.05] work better than the previous transform? \nAnd if so, would for instance two often confused classes get a boost by differently weighing label smooths? For example, be label_smooth a function that takes an integer and returns the above one hot smoothed Tensors, we have so that the other classes are different as seen in this example: label_smooth(0) (e.g. [0.95 0.03 0.07 0.0 0.17] != label_smooth(4) [0.11 0.09 0.6 0.3 0.95]\n\nI haven't tried implementing it yet, but wanted to know what you guys think.",
    "1150218": "Kinda like LDL: https://arxiv.org/abs/1408.6027",
    "1147912": ""
  }
}