{ "id": 198202, "title": "Duplicate Images", "url": "/competitions/cassava-leaf-disease-classification/discussion/198202", "author_name": "", "post_date": "2020-11-20T08:14:36.887470500Z", "votes": 45, "comment_count": 6, "views": 0, "content": "

There seems to be duplicate images.
\n'1562043567.jpg' and '3551135685.jpg'
\n'2252529694.jpg' and '911861181.jpg'
\n\"\"

\n

\"\"

\n

1562043567.jpg and 3551135685.jpg seem to be given different labels.

\n

The code is available below.
\nhttps://www.kaggle.com/nakajima/duplicate-train-images?scriptVersionId=47295222

", "messages": [ { "id": "1084630", "postDate": "11/20/2020 08:14:36", "content": "

There seems to be duplicate images.
\n'1562043567.jpg' and '3551135685.jpg'
\n'2252529694.jpg' and '911861181.jpg'
\n\"\"

\n

\"\"

\n

1562043567.jpg and 3551135685.jpg seem to be given different labels.

\n

The code is available below.
\nhttps://www.kaggle.com/nakajima/duplicate-train-images?scriptVersionId=47295222

", "rawMarkdown": "There seems to be duplicate images.\n'1562043567.jpg' and '3551135685.jpg'\n'2252529694.jpg' and '911861181.jpg'\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F468287%2F8ed540241d6752eace502411d4a76b93%2Fduplicate1.png?generation=1605859984077292&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F468287%2F2c629f4d147b738be97f17bee406d04b%2Fduplicate2.png?generation=1605860061245765&alt=media)\n\n1562043567.jpg and 3551135685.jpg seem to be given different labels.\n\n\nThe code is available below.\nhttps://www.kaggle.com/nakajima/duplicate-train-images?scriptVersionId=47295222", "votes": null }, { "id": "1087373", "postDate": "11/22/2020 16:40:03", "content": "

I think there can be more which will affect our cv score; this problem can be solved if we could somehow plot it in a 2 axis plane and used knn to detect which images are similar and can be deleted.
\n@cdeotte 's awesome notebook https://www.kaggle.com/cdeotte/rapids-cuml-knn-find-duplicates can be used.

", "rawMarkdown": "I think there can be more which will affect our cv score; this problem can be solved if we could somehow plot it in a 2 axis plane and used knn to detect which images are similar and can be deleted.\n@cdeotte 's awesome notebook https://www.kaggle.com/cdeotte/rapids-cuml-knn-find-duplicates can be used.", "votes": null }, { "id": "1087422", "postDate": "11/22/2020 17:38:07", "content": "

Noticing how these two have different labels. Does this mean one image/plant can have multiple diseases? Seems reasonable but would be nice to know for modelling.

\n
\n

'1562043567.jpg' and '3551135685.jpg'
\n \"\"

\n
", "rawMarkdown": "Noticing how these two have different labels. Does this mean one image/plant can have multiple diseases? Seems reasonable but would be nice to know for modelling.\n\n> '1562043567.jpg' and '3551135685.jpg'\n> ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F468287%2F8ed540241d6752eace502411d4a76b93%2Fduplicate1.png?generation=1605859984077292&alt=media)", "votes": null }, { "id": "1087439", "postDate": "11/22/2020 17:50:14", "content": "

Yes I think in this case you are correct; can it be now treated as a multi-label problem🤔🤔

", "rawMarkdown": "Yes I think in this case you are correct; can it be now treated as a multi-label problem🤔🤔", "votes": null }, { "id": "1087650", "postDate": "11/23/2020 00:39:55", "content": "

It cannot be multilabel because the label 4 is healthy and label 0 is \"Cassava Bacterial Blight (CBB)\". It is not possible to have both labels exist at the same time.
\nso I think is a mislabel

", "rawMarkdown": "It cannot be multilabel because the label 4 is healthy and label 0 is \"Cassava Bacterial Blight (CBB)\". It is not possible to have both labels exist at the same time.\nso I think is a mislabel", "votes": null }, { "id": "1087950", "postDate": "11/23/2020 07:25:48", "content": "

Multi label is possible when that plant has multiple diseases. But in this case It has both healthy and diseased labels which is not possible. I think this problem is due to croud sourcing the data. Mistakenly they have classified it. The organizers has to take some precautions to avoid such casses.
\nWe can compare pixel values to detect them and we can remove those pictures from training data

", "rawMarkdown": "Multi label is possible when that plant has multiple diseases. But in this case It has both healthy and diseased labels which is not possible. I think this problem is due to croud sourcing the data. Mistakenly they have classified it. The organizers has to take some precautions to avoid such casses.\nWe can compare pixel values to detect them and we can remove those pictures from training data", "votes": null }, { "id": "1088057", "postDate": "11/23/2020 09:25:26", "content": "

Ya I agree
\nSo I think the best option here is to remove the duplicated image and the mislabel image

", "rawMarkdown": "Ya I agree\nSo I think the best option here is to remove the duplicated image and the mislabel image", "votes": null } ], "comments": [ { "id": 1087373, "author_name": "mrinath", "author_url": "", "post_date": "11/22/2020 16:40:03", "content": "

I think there can be more which will affect our cv score; this problem can be solved if we could somehow plot it in a 2 axis plane and used knn to detect which images are similar and can be deleted.
\n@cdeotte 's awesome notebook https://www.kaggle.com/cdeotte/rapids-cuml-knn-find-duplicates can be used.

", "votes": null, "replies": [] }, { "id": 1087422, "author_name": "nicholasjhana", "author_url": "", "post_date": "11/22/2020 17:38:07", "content": "

Noticing how these two have different labels. Does this mean one image/plant can have multiple diseases? Seems reasonable but would be nice to know for modelling.

\n
\n

'1562043567.jpg' and '3551135685.jpg'
\n \"\"

\n
", "votes": null, "replies": [ { "id": 1087439, "author_name": "mrinath", "author_url": "", "post_date": "11/22/2020 17:50:14", "content": "

Yes I think in this case you are correct; can it be now treated as a multi-label problem🤔🤔

", "votes": null, "replies": [] }, { "id": 1087650, "author_name": "slm37102", "author_url": "", "post_date": "11/23/2020 00:39:55", "content": "

It cannot be multilabel because the label 4 is healthy and label 0 is \"Cassava Bacterial Blight (CBB)\". It is not possible to have both labels exist at the same time.
\nso I think is a mislabel

", "votes": null, "replies": [] }, { "id": 1087950, "author_name": "vinaykakara", "author_url": "", "post_date": "11/23/2020 07:25:48", "content": "

Multi label is possible when that plant has multiple diseases. But in this case It has both healthy and diseased labels which is not possible. I think this problem is due to croud sourcing the data. Mistakenly they have classified it. The organizers has to take some precautions to avoid such casses.
\nWe can compare pixel values to detect them and we can remove those pictures from training data

", "votes": null, "replies": [] }, { "id": 1088057, "author_name": "slm37102", "author_url": "", "post_date": "11/23/2020 09:25:26", "content": "

Ya I agree
\nSo I think the best option here is to remove the duplicated image and the mislabel image

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1084630": "There seems to be duplicate images.\n'1562043567.jpg' and '3551135685.jpg'\n'2252529694.jpg' and '911861181.jpg'\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F468287%2F8ed540241d6752eace502411d4a76b93%2Fduplicate1.png?generation=1605859984077292&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F468287%2F2c629f4d147b738be97f17bee406d04b%2Fduplicate2.png?generation=1605860061245765&alt=media)\n\n1562043567.jpg and 3551135685.jpg seem to be given different labels.\n\n\nThe code is available below.\nhttps://www.kaggle.com/nakajima/duplicate-train-images?scriptVersionId=47295222", "1087373": "I think there can be more which will affect our cv score; this problem can be solved if we could somehow plot it in a 2 axis plane and used knn to detect which images are similar and can be deleted.\n@cdeotte 's awesome notebook https://www.kaggle.com/cdeotte/rapids-cuml-knn-find-duplicates can be used.", "1087422": "Noticing how these two have different labels. Does this mean one image/plant can have multiple diseases? Seems reasonable but would be nice to know for modelling.\n\n> '1562043567.jpg' and '3551135685.jpg'\n> ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F468287%2F8ed540241d6752eace502411d4a76b93%2Fduplicate1.png?generation=1605859984077292&alt=media)", "1087439": "Yes I think in this case you are correct; can it be now treated as a multi-label problem🤔🤔", "1087650": "It cannot be multilabel because the label 4 is healthy and label 0 is \"Cassava Bacterial Blight (CBB)\". It is not possible to have both labels exist at the same time.\nso I think is a mislabel", "1087950": "Multi label is possible when that plant has multiple diseases. But in this case It has both healthy and diseased labels which is not possible. I think this problem is due to croud sourcing the data. Mistakenly they have classified it. The organizers has to take some precautions to avoid such casses.\nWe can compare pixel values to detect them and we can remove those pictures from training data", "1088057": "Ya I agree\nSo I think the best option here is to remove the duplicated image and the mislabel image" }, "source": "meta" }