![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F998934%2F35a51084292cb9bba23312b4d134f356%2F.png?generation=1597755825010323&alt=media\")
I read the data description but I can't find the detail of the columns 'diagnosis'. It's not clear how this label is made. It is very strange that most of the diagnosis is unknown, which is a label not seen in previous competitions. Type of diagnosis is described in ISIC 2019 and most of them are by histopathology. I think most of the types of diagnosis of this competition's data are not by histopathology because there are about 30 images per each patient and performing 30 biopsies per patient is not realistic. I suspect that diagnosis unknown simply means that a biopsy was not performed. If it's true, what we predict is not whether a skin lesion is a melanoma or not, but whether a skin lesion will be performed a biopsy by a dermatologist.
", "messages": [ { "id": "975679", "postDate": "08/18/2020 12:22:53", "content": "I read the data description but I can't find the detail of the columns 'diagnosis'. It's not clear how this label is made. It is very strange that most of the diagnosis is unknown, which is a label not seen in previous competitions. Type of diagnosis is described in ISIC 2019 and most of them are by histopathology. I think most of the types of diagnosis of this competition's data are not by histopathology because there are about 30 images per each patient and performing 30 biopsies per patient is not realistic. I suspect that diagnosis unknown simply means that a biopsy was not performed. If it's true, what we predict is not whether a skin lesion is a melanoma or not, but whether a skin lesion will be performed a biopsy by a dermatologist.
", "rawMarkdown": "I read the data description but I can't find the detail of the columns 'diagnosis'. It's not clear how this label is made. It is very strange that most of the diagnosis is unknown, which is a label not seen in previous competitions. Type of diagnosis is described in ISIC 2019 and most of them are by histopathology. I think most of the types of diagnosis of this competition's data are not by histopathology because there are about 30 images per each patient and performing 30 biopsies per patient is not realistic. I suspect that diagnosis unknown simply means that a biopsy was not performed. If it's true, what we predict is not whether a skin lesion is a melanoma or not, but whether a skin lesion will be performed a biopsy by a dermatologist.", "votes": null }, { "id": "975710", "postDate": "08/18/2020 12:35:00", "content": "", "rawMarkdown": "veronicarotemberg", "votes": null }, { "id": "975716", "postDate": "08/18/2020 12:41:50", "content": "Copying what I wrote in another thread.
\nWe found out the following thing:
\nSo there definitely is some difference between those.
", "rawMarkdown": "Copying what I wrote in another thread.\n\nWe found out the following thing:\n\n- Only training and evaluating on 2020 data\n- Removing all diagnosis=nevus from train (keep them in val)\n- Score drops a lot!\n- Remove similar amount of diagnosis=unknown\n- Score does not change (only tiny random range).\n\nSo there definitely is some difference between those.", "votes": null }, { "id": "975757", "postDate": "08/18/2020 13:05:10", "content": "At least, there is a correlation between the number of images per patient and the diagnosis unknown rate per patient.
\n
This is an excellent point. In case \"unknown\" indeed means that the physician did not deem it necessary to have a biopsy, then we might have false negatives in our ground truth (melanomas that were not spotted by the dermatologist and are labeled as unknown).
", "rawMarkdown": "This is an excellent point. In case \"unknown\" indeed means that the physician did not deem it necessary to have a biopsy, then we might have false negatives in our ground truth (melanomas that were not spotted by the dermatologist and are labeled as unknown).", "votes": null }, { "id": "976271", "postDate": "08/18/2020 18:53:00", "content": "Yes - this is my current hypothesis (see comment below). Those labeled as unknown seem way less reliable.
", "rawMarkdown": "Yes - this is my current hypothesis (see comment below). Those labeled as unknown seem way less reliable.", "votes": null } ], "comments": [ { "id": 975710, "author_name": "s903124", "author_url": "", "post_date": "08/18/2020 12:35:00", "content": "", "votes": null, "replies": [] }, { "id": 975716, "author_name": "philippsinger", "author_url": "", "post_date": "08/18/2020 12:41:50", "content": "Copying what I wrote in another thread.
\nWe found out the following thing:
\nSo there definitely is some difference between those.
", "votes": null, "replies": [] }, { "id": 975757, "author_name": "osciiart", "author_url": "", "post_date": "08/18/2020 13:05:10", "content": "At least, there is a correlation between the number of images per patient and the diagnosis unknown rate per patient.
\n
This is an excellent point. In case \"unknown\" indeed means that the physician did not deem it necessary to have a biopsy, then we might have false negatives in our ground truth (melanomas that were not spotted by the dermatologist and are labeled as unknown).
", "votes": null, "replies": [ { "id": 976271, "author_name": "philippsinger", "author_url": "", "post_date": "08/18/2020 18:53:00", "content": "Yes - this is my current hypothesis (see comment below). Those labeled as unknown seem way less reliable.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "975679": "I read the data description but I can't find the detail of the columns 'diagnosis'. It's not clear how this label is made. It is very strange that most of the diagnosis is unknown, which is a label not seen in previous competitions. Type of diagnosis is described in ISIC 2019 and most of them are by histopathology. I think most of the types of diagnosis of this competition's data are not by histopathology because there are about 30 images per each patient and performing 30 biopsies per patient is not realistic. I suspect that diagnosis unknown simply means that a biopsy was not performed. If it's true, what we predict is not whether a skin lesion is a melanoma or not, but whether a skin lesion will be performed a biopsy by a dermatologist.", "975710": "veronicarotemberg", "975716": "Copying what I wrote in another thread.\n\nWe found out the following thing:\n\n- Only training and evaluating on 2020 data\n- Removing all diagnosis=nevus from train (keep them in val)\n- Score drops a lot!\n- Remove similar amount of diagnosis=unknown\n- Score does not change (only tiny random range).\n\nSo there definitely is some difference between those.", "975757": "At least, there is a correlation between the number of images per patient and the diagnosis unknown rate per patient.\n\n", "976249": "This is an excellent point. In case \"unknown\" indeed means that the physician did not deem it necessary to have a biopsy, then we might have false negatives in our ground truth (melanomas that were not spotted by the dermatologist and are labeled as unknown).", "976271": "Yes - this is my current hypothesis (see comment below). Those labeled as unknown seem way less reliable." }, "source": "meta" }