{
  "id": 216826,
  "title": "Is distillation learning  useful with this noisy data?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/216826",
  "author_name": "",
  "post_date": "2021-02-04T07:52:26.190702400Z",
  "votes": 5,
  "comment_count": 4,
  "views": 0,
  "content": "<p>My idea is , first training a teacher model to fit the noisy data( including the noisy part).<br>\nSecondly, like distillation learning, training a student model to learn soft label( from teacher model prediction) and hard label( real label).<br>\nIn this way, student model can learn both clean label  and noisy distribution.</p>",
  "messages": [
    {
      "id": "1185546",
      "postDate": "02/04/2021 07:52:26",
      "content": "<p>My idea is , first training a teacher model to fit the noisy data( including the noisy part).<br>\nSecondly, like distillation learning, training a student model to learn soft label( from teacher model prediction) and hard label( real label).<br>\nIn this way, student model can learn both clean label  and noisy distribution.</p>",
      "rawMarkdown": "My idea is , first training a teacher model to fit the noisy data( including the noisy part).\nSecondly, like distillation learning, training a student model to learn soft label( from teacher model prediction) and hard label( real label).\nIn this way, student model can learn both clean label  and noisy distribution.",
      "votes": null
    },
    {
      "id": "1185574",
      "postDate": "02/04/2021 08:10:21",
      "content": "<p>Hello!</p>\n<p>There's a <strong><a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607\" target=\"_blank\">discussion</a></strong> on that, as well as a link to the dataset with 0.900 LB ensemble soft labels for a quick start.</p>\n<p>Long story short, noisy student training indeed helps a lot for data noisy as this. It gave me a +0.003 LB boost and seems still room for improvement.</p>",
      "rawMarkdown": "Hello!\n\nThere's a **[discussion](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607)** on that, as well as a link to the dataset with 0.900 LB ensemble soft labels for a quick start.\n\nLong story short, noisy student training indeed helps a lot for data noisy as this. It gave me a +0.003 LB boost and seems still room for improvement.",
      "votes": null
    },
    {
      "id": "1185603",
      "postDate": "02/04/2021 08:35:15",
      "content": "<p>Thank you very much.</p>",
      "rawMarkdown": "Thank you very much.",
      "votes": null
    },
    {
      "id": "1186070",
      "postDate": "02/04/2021 15:15:16",
      "content": "<p>I had experimented,it can real improve cv but decrease LB.</p>",
      "rawMarkdown": "I had experimented,it can real improve cv but decrease LB.",
      "votes": null
    },
    {
      "id": "1187265",
      "postDate": "02/05/2021 10:22:23",
      "content": "<p>Thank you, i will try it.</p>",
      "rawMarkdown": "Thank you, i will try it.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1185574,
      "author_name": "nickuzmenkov",
      "author_url": "",
      "post_date": "02/04/2021 08:10:21",
      "content": "<p>Hello!</p>\n<p>There's a <strong><a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607\" target=\"_blank\">discussion</a></strong> on that, as well as a link to the dataset with 0.900 LB ensemble soft labels for a quick start.</p>\n<p>Long story short, noisy student training indeed helps a lot for data noisy as this. It gave me a +0.003 LB boost and seems still room for improvement.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1185603,
          "author_name": "kingofdaydream",
          "author_url": "",
          "post_date": "02/04/2021 08:35:15",
          "content": "<p>Thank you very much.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1186070,
      "author_name": "hanson0910",
      "author_url": "",
      "post_date": "02/04/2021 15:15:16",
      "content": "<p>I had experimented,it can real improve cv but decrease LB.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1187265,
          "author_name": "kingofdaydream",
          "author_url": "",
          "post_date": "02/05/2021 10:22:23",
          "content": "<p>Thank you, i will try it.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1185546": "My idea is , first training a teacher model to fit the noisy data( including the noisy part).\nSecondly, like distillation learning, training a student model to learn soft label( from teacher model prediction) and hard label( real label).\nIn this way, student model can learn both clean label  and noisy distribution.",
    "1185574": "Hello!\n\nThere's a **[discussion](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607)** on that, as well as a link to the dataset with 0.900 LB ensemble soft labels for a quick start.\n\nLong story short, noisy student training indeed helps a lot for data noisy as this. It gave me a +0.003 LB boost and seems still room for improvement.",
    "1185603": "Thank you very much.",
    "1186070": "I had experimented,it can real improve cv but decrease LB.",
    "1187265": "Thank you, i will try it."
  },
  "source": "meta"
}