Hello fellow scholars,
\nDuring the training and validation sessions on the Plant Pathology dataset I´ve found something strange. The model was predicting the right classification (as per my judgment) but the actual label on train.csv had the wrong annotation.
\nI´ve decided to take a further look on the dataset and I´ve found several cases that were in this condition. So I´ve created the following notebook to compile some findings.
\nhttps://www.kaggle.com/dandierlean/dataset-evaluation
\nI´m no Botanist or Apple Leaf Disease specialist, but a few inputs from the community would be great.
", "messages": [ { "id": "1315623", "postDate": "05/20/2021 02:45:03", "content": "Hello fellow scholars,
\nDuring the training and validation sessions on the Plant Pathology dataset I´ve found something strange. The model was predicting the right classification (as per my judgment) but the actual label on train.csv had the wrong annotation.
\nI´ve decided to take a further look on the dataset and I´ve found several cases that were in this condition. So I´ve created the following notebook to compile some findings.
\nhttps://www.kaggle.com/dandierlean/dataset-evaluation
\nI´m no Botanist or Apple Leaf Disease specialist, but a few inputs from the community would be great.
", "rawMarkdown": "Hello fellow scholars,\n\nDuring the training and validation sessions on the Plant Pathology dataset I´ve found something strange. The model was predicting the right classification (as per my judgment) but the actual label on train.csv had the wrong annotation.\n\nI´ve decided to take a further look on the dataset and I´ve found several cases that were in this condition. So I´ve created the following notebook to compile some findings.\n\nhttps://www.kaggle.com/dandierlean/dataset-evaluation\n\nI´m no Botanist or Apple Leaf Disease specialist, but a few inputs from the community would be great.", "votes": null }, { "id": "1316626", "postDate": "05/20/2021 18:08:29", "content": "Yep, there are a lot of miss-labeled samples. The label is noisy. I also found duplicates that were labeled differently, so there is quite a bit of noise in the labels in the dateset.
", "rawMarkdown": "Yep, there are a lot of miss-labeled samples. The label is noisy. I also found duplicates that were labeled differently, so there is quite a bit of noise in the labels in the dateset.", "votes": null }, { "id": "1316643", "postDate": "05/20/2021 18:30:13", "content": "Exactly. But the implication is that the hidden Test dataset is also noisy. So a lot of classification errors might be that the label is incorrect instead of the model prediction.
\nThis applies to obvious misclassifications (leaf with rust labelled as healthy) or for the most difficult ones (uncertainty between a healthy leaf and one with a initial scab contamination).
", "rawMarkdown": "Exactly. But the implication is that the hidden Test dataset is also noisy. So a lot of classification errors might be that the label is incorrect instead of the model prediction. \n\nThis applies to obvious misclassifications (leaf with rust labelled as healthy) or for the most difficult ones (uncertainty between a healthy leaf and one with a initial scab contamination).", "votes": null } ], "comments": [ { "id": 1316626, "author_name": "datasciencegeek", "author_url": "", "post_date": "05/20/2021 18:08:29", "content": "Yep, there are a lot of miss-labeled samples. The label is noisy. I also found duplicates that were labeled differently, so there is quite a bit of noise in the labels in the dateset.
", "votes": null, "replies": [ { "id": 1316643, "author_name": "dandierlean", "author_url": "", "post_date": "05/20/2021 18:30:13", "content": "Exactly. But the implication is that the hidden Test dataset is also noisy. So a lot of classification errors might be that the label is incorrect instead of the model prediction.
\nThis applies to obvious misclassifications (leaf with rust labelled as healthy) or for the most difficult ones (uncertainty between a healthy leaf and one with a initial scab contamination).
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1315623": "Hello fellow scholars,\n\nDuring the training and validation sessions on the Plant Pathology dataset I´ve found something strange. The model was predicting the right classification (as per my judgment) but the actual label on train.csv had the wrong annotation.\n\nI´ve decided to take a further look on the dataset and I´ve found several cases that were in this condition. So I´ve created the following notebook to compile some findings.\n\nhttps://www.kaggle.com/dandierlean/dataset-evaluation\n\nI´m no Botanist or Apple Leaf Disease specialist, but a few inputs from the community would be great.", "1316626": "Yep, there are a lot of miss-labeled samples. The label is noisy. I also found duplicates that were labeled differently, so there is quite a bit of noise in the labels in the dateset.", "1316643": "Exactly. But the implication is that the hidden Test dataset is also noisy. So a lot of classification errors might be that the label is incorrect instead of the model prediction. \n\nThis applies to obvious misclassifications (leaf with rust labelled as healthy) or for the most difficult ones (uncertainty between a healthy leaf and one with a initial scab contamination)." }, "source": "meta" }