Hi There,
\nI have talked about this issue in a couple of comments to @nickuzmenkov and @buinyi notebooks and decided to make it as a discussion so that more people and the host @fruitpathology might help all of us to figure it out.
\nWhat I mean is, there are two class labels cider_apple_rust and rust and, interestingly enough, the class label rust appears only among images with multiple class labels while the class label cider_apple_rust appears only with single class labeled images. In particular, the class label rust appears only with these two combinations of labels:
\nHere: numbers of images quoted are before deleting any duplicates, i.e., as they are in the original metadata CSV file.
\nWhat if rust and cider_apple_rust are the same class? I looked at images with both of these labels present and the spots which look like rust do look alike on images labeled cider_apple_rust and rust frog_eye_leaf_spot or rust complex.
\nIf the same pattern of assigning the labels for the class rust persists in the test data, what it might mean is that whenever a prediction is made such that it is for just single class rust then we might need to use cider_apple_rust in the prediction string. However, in a situation when prediction is made such that besides rust other classes detected, then for the rust class we might need to use rust in the prediction string.
\nI investigated a little bit this possibility in this notebook. Especial coincidence which I noted when I was exploring duplicate images I detected is that most of them have inconsistent target labels and in particular those inconsistencies were among images labeled as cider_apple_rust vs rust complex which speaks more to the possibility that cider_apple_rust and rust labels might belong to the same class.
", "messages": [ { "id": "1250325", "postDate": "03/24/2021 00:37:51", "content": "Hi There,
\nI have talked about this issue in a couple of comments to @nickuzmenkov and @buinyi notebooks and decided to make it as a discussion so that more people and the host @fruitpathology might help all of us to figure it out.
\nWhat I mean is, there are two class labels cider_apple_rust and rust and, interestingly enough, the class label rust appears only among images with multiple class labels while the class label cider_apple_rust appears only with single class labeled images. In particular, the class label rust appears only with these two combinations of labels:
\nHere: numbers of images quoted are before deleting any duplicates, i.e., as they are in the original metadata CSV file.
\nWhat if rust and cider_apple_rust are the same class? I looked at images with both of these labels present and the spots which look like rust do look alike on images labeled cider_apple_rust and rust frog_eye_leaf_spot or rust complex.
\nIf the same pattern of assigning the labels for the class rust persists in the test data, what it might mean is that whenever a prediction is made such that it is for just single class rust then we might need to use cider_apple_rust in the prediction string. However, in a situation when prediction is made such that besides rust other classes detected, then for the rust class we might need to use rust in the prediction string.
\nI investigated a little bit this possibility in this notebook. Especial coincidence which I noted when I was exploring duplicate images I detected is that most of them have inconsistent target labels and in particular those inconsistencies were among images labeled as cider_apple_rust vs rust complex which speaks more to the possibility that cider_apple_rust and rust labels might belong to the same class.
", "rawMarkdown": "Hi There,\n\nI have talked about this issue in a couple of comments to @nickuzmenkov and @buinyi notebooks and decided to make it as a discussion so that more people and the host @fruitpathology might help all of us to figure it out.\n\nWhat I mean is, there are two class labels **cider_apple_rust** and **rust** and, interestingly enough, the class label **rust** appears only among images with multiple class labels while the class label **cider_apple_rust** appears only with single class labeled images. In particular, the class label **rust** appears only with these two combinations of labels:\n\n1. **rust** with **frog_eye_leaf_spot** in 120 images\n2. **rust** with **complex** in 97 images\n\nHere: numbers of images quoted are before deleting any duplicates, i.e., as they are in the original metadata CSV file.\n\nWhat if **rust** and **cider_apple_rust** are the same class? I looked at images with both of these labels present and the spots which look like rust do look alike on images labeled **cider_apple_rust** and **rust frog_eye_leaf_spot** or **rust complex**.\n\nIf the same pattern of assigning the labels for the class rust persists in the test data, what it might mean is that whenever a prediction is made such that it is for just single class rust then we might need to use **cider_apple_rust** in the prediction string. However, in a situation when prediction is made such that besides rust other classes detected, then for the rust class we might need to use **rust** in the prediction string.\n\nI investigated a little bit this possibility in this [notebook](https://www.kaggle.com/datasciencegeek/eda-plantpathology2021-fgvc8). Especial coincidence which I noted when I was exploring duplicate images I detected is that most of them have inconsistent target labels and in particular those inconsistencies were among images labeled as **cider_apple_rust** vs **rust complex** which speaks more to the possibility that **cider_apple_rust** and **rust** labels might belong to the same class.", "votes": null }, { "id": "1251112", "postDate": "03/24/2021 13:52:41", "content": "Yes, cider apple rust and rust are labels for same class. The label rust is given for images with multi class labels and the label cider apple rust is given for single class labels.
", "rawMarkdown": "Yes, cider apple rust and rust are labels for same class. The label rust is given for images with multi class labels and the label cider apple rust is given for single class labels.", "votes": null }, { "id": "1251226", "postDate": "03/24/2021 15:40:52", "content": "Thank you very much for clarifying it for us!
", "rawMarkdown": "Thank you very much for clarifying it for us!", "votes": null }, { "id": "1251327", "postDate": "03/24/2021 17:20:51", "content": "I'm working on an update that will merge the two labels. Thanks again for the catch!
", "rawMarkdown": "I'm working on an update that will merge the two labels. Thanks again for the catch!", "votes": null }, { "id": "1251400", "postDate": "03/24/2021 18:32:48", "content": "Thank you, @datasciencegeek, for a great catch!
\nFirstly I thought that there are multiple apple sorts in the pictures, so in a rather fancy pipeline we have to first detect whether it's a cider apple tree or not (BTW, is it possible to do so having only a single leaf picture even for a human specialist?). But now things appear to be even worse. Too many strange points in this competition.
Yes, cider apple rust and rust are labels for same class. The label rust is given for images with multi class labels and the label cider apple rust is given for single class labels.
", "votes": null, "replies": [ { "id": 1251226, "author_name": "datasciencegeek", "author_url": "", "post_date": "03/24/2021 15:40:52", "content": "Thank you very much for clarifying it for us!
", "votes": null, "replies": [] }, { "id": 1251327, "author_name": "sohier", "author_url": "", "post_date": "03/24/2021 17:20:51", "content": "I'm working on an update that will merge the two labels. Thanks again for the catch!
", "votes": null, "replies": [] } ] }, { "id": 1251400, "author_name": "nickuzmenkov", "author_url": "", "post_date": "03/24/2021 18:32:48", "content": "Thank you, @datasciencegeek, for a great catch!
\nFirstly I thought that there are multiple apple sorts in the pictures, so in a rather fancy pipeline we have to first detect whether it's a cider apple tree or not (BTW, is it possible to do so having only a single leaf picture even for a human specialist?). But now things appear to be even worse. Too many strange points in this competition.