{
  "id": 212187,
  "title": "Thoughts on why you should expect noisy data on similar datasets",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/212187",
  "author_name": "Miroslav Valan",
  "post_date": "2021-01-17T23:01:01.024000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I am coping my comment from the previous competition on why the cassava dataset might be noisy. </p>\n<ol>\n<li>humans assign labels and humans are prone to errors due to tiredness, skill, boredom etc. (too general but always true)</li>\n<li>it is sometimes hard to notice symptoms in the early stage of disease (looks like healthy)</li>\n<li>if two diseases are present on the image - the one with less severe symptoms has a high chance to be overlooked and in these type of challenges people are often asked to assign only one label and they assign a dominant disease</li>\n<li>some leaves might look like they have disease but they are in fact healthy (e.g. mechanical or chemical damage)</li>\n<li>I have seen cases where annotators get instruction to sort images into N diseases and +1 where these diseases are absent. The +1 subset will have all the healthy images but also images with diseases not included in the N. Some of those diseases beyond N may have similarly appearing features</li>\n</ol>\n<p>Please note that I have not confirmed any of this on the current dataset. And let me know if I should add something to the list :)</p>",
  "messages": [
    {
      "id": 1157453,
      "postDate": "2021-01-17T23:01:01.023Z",
      "content": "<p>I am coping my comment from the previous competition on why the cassava dataset might be noisy. </p>\n<ol>\n<li>humans assign labels and humans are prone to errors due to tiredness, skill, boredom etc. (too general but always true)</li>\n<li>it is sometimes hard to notice symptoms in the early stage of disease (looks like healthy)</li>\n<li>if two diseases are present on the image - the one with less severe symptoms has a high chance to be overlooked and in these type of challenges people are often asked to assign only one label and they assign a dominant disease</li>\n<li>some leaves might look like they have disease but they are in fact healthy (e.g. mechanical or chemical damage)</li>\n<li>I have seen cases where annotators get instruction to sort images into N diseases and +1 where these diseases are absent. The +1 subset will have all the healthy images but also images with diseases not included in the N. Some of those diseases beyond N may have similarly appearing features</li>\n</ol>\n<p>Please note that I have not confirmed any of this on the current dataset. And let me know if I should add something to the list :)</p>",
      "rawMarkdown": "I am coping my comment from the previous competition on why the cassava dataset might be noisy. \n\n1.  humans assign labels and humans are prone to errors due to tiredness, skill, boredom etc. (too general but always true)\n2. it is sometimes hard to notice symptoms in the early stage of disease (looks like healthy)\n3. if two diseases are present on the image - the one with less severe symptoms has a high chance to be overlooked and in these type of challenges people are often asked to assign only one label and they assign a dominant disease\n4. some leaves might look like they have disease but they are in fact healthy (e.g. mechanical or chemical damage)\n5. I have seen cases where annotators get instruction to sort images into N diseases and +1 where these diseases are absent. The +1 subset will have all the healthy images but also images with diseases not included in the N. Some of those diseases beyond N may have similarly appearing features\n\nPlease note that I have not confirmed any of this on the current dataset. And let me know if I should add something to the list :)",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1157453": "I am coping my comment from the previous competition on why the cassava dataset might be noisy. \n\n1.  humans assign labels and humans are prone to errors due to tiredness, skill, boredom etc. (too general but always true)\n2. it is sometimes hard to notice symptoms in the early stage of disease (looks like healthy)\n3. if two diseases are present on the image - the one with less severe symptoms has a high chance to be overlooked and in these type of challenges people are often asked to assign only one label and they assign a dominant disease\n4. some leaves might look like they have disease but they are in fact healthy (e.g. mechanical or chemical damage)\n5. I have seen cases where annotators get instruction to sort images into N diseases and +1 where these diseases are absent. The +1 subset will have all the healthy images but also images with diseases not included in the N. Some of those diseases beyond N may have similarly appearing features\n\nPlease note that I have not confirmed any of this on the current dataset. And let me know if I should add something to the list :)"
  }
}