![\"\"](\"https://img.medscapestatic.com/article/733/861/733861-fig1.jpg\")
this is probably the most important objective of the competition.
\n\n\"Currently, dermatologists evaluate every one of a patient's moles to identify outlier lesions or “ugly ducklings” that are most likely to be melanoma.\"
\n\nwe can simply treat this as an image recognition problem (like the past ISIC competitions), but can we do better than that?
\n\nFirst let us understand what is \"Ugly Duckling Concept\"\nhttps://www.medscape.com/viewarticle/733861_3
\n\n\"In 1998, Grob, et al[7] introduced the ugly duckling concept-the observation that nevi in the same individual tend to resemble one another, and that MM often deviates from this nevus pattern. This clinical realization pointed to the importance of not just evaluating the morphology of the lesion in question, but also comparing it to that of surrounding lesions, looking for an outlier in the background of similar-appearing moles. For example, the outlier lesion can be larger and darker than the surrounding moles (Figure 1 A), or conversely, small and red in the background of multiple large dark moles (Figure 1 B). Finally, if the patient has few or no other moles (Figure 1 C). any changing lesion should be considered a suspicious outlier.\"
\n\n
so what does it means? This competition is really about \"outlier detection\". can you measure differences for a given set of mole images? And then using the measured difference to predict melanoma?
\n\nif you find papers and works on using ML/deep learning for \"Ugly Duckling Concept\", please share it here.
\n\nI estimate pure image recognition (without meta data) to be around AUC 0.95 ~ 0.96. If \"Ugly Duckling Concept\" works, maybe we can get to 0.97~0.98 ???
", "messages": [ { "id": "869887", "postDate": "06/01/2020 10:52:31", "content": "this is probably the most important objective of the competition.
\n\n\"Currently, dermatologists evaluate every one of a patient's moles to identify outlier lesions or “ugly ducklings” that are most likely to be melanoma.\"
\n\nwe can simply treat this as an image recognition problem (like the past ISIC competitions), but can we do better than that?
\n\nFirst let us understand what is \"Ugly Duckling Concept\"\nhttps://www.medscape.com/viewarticle/733861_3
\n\n\"In 1998, Grob, et al[7] introduced the ugly duckling concept-the observation that nevi in the same individual tend to resemble one another, and that MM often deviates from this nevus pattern. This clinical realization pointed to the importance of not just evaluating the morphology of the lesion in question, but also comparing it to that of surrounding lesions, looking for an outlier in the background of similar-appearing moles. For example, the outlier lesion can be larger and darker than the surrounding moles (Figure 1 A), or conversely, small and red in the background of multiple large dark moles (Figure 1 B). Finally, if the patient has few or no other moles (Figure 1 C). any changing lesion should be considered a suspicious outlier.\"
\n\n
so what does it means? This competition is really about \"outlier detection\". can you measure differences for a given set of mole images? And then using the measured difference to predict melanoma?
\n\nif you find papers and works on using ML/deep learning for \"Ugly Duckling Concept\", please share it here.
\n\nI estimate pure image recognition (without meta data) to be around AUC 0.95 ~ 0.96. If \"Ugly Duckling Concept\" works, maybe we can get to 0.97~0.98 ???
", "rawMarkdown": "this is probably the most important objective of the competition.\n\n\"Currently, dermatologists evaluate every one of a patient's moles to identify outlier lesions or “ugly ducklings” that are most likely to be melanoma.\"\n\nwe can simply treat this as an image recognition problem (like the past ISIC competitions), but can we do better than that?\n\nFirst let us understand what is \"Ugly Duckling Concept\"\nhttps://www.medscape.com/viewarticle/733861_3\n\n\"In 1998, Grob, et al[7] introduced the ugly duckling concept-the observation that nevi in the same individual tend to resemble one another, and that MM often deviates from this nevus pattern. This clinical realization pointed to the importance of not just evaluating the morphology of the lesion in question, but also comparing it to that of surrounding lesions, looking for an outlier in the background of similar-appearing moles. For example, the outlier lesion can be larger and darker than the surrounding moles (Figure 1 A), or conversely, small and red in the background of multiple large dark moles (Figure 1 B). Finally, if the patient has few or no other moles (Figure 1 C). any changing lesion should be considered a suspicious outlier.\"\n\n\n\nso what does it means? This competition is really about \"outlier detection\". can you measure differences for a given set of mole images? And then using the measured difference to predict melanoma?\n\nif you find papers and works on using ML/deep learning for \"Ugly Duckling Concept\", please share it here.\n\nI estimate pure image recognition (without meta data) to be around AUC 0.95 ~ 0.96. If \"Ugly Duckling Concept\" works, maybe we can get to 0.97~0.98 ???", "votes": null }, { "id": "869893", "postDate": "06/01/2020 10:55:42", "content": "\"Skin cancer detection based on deep learning and entropy to detect outlier samples\"\n- https://arxiv.org/pdf/1909.04525.pdf
\n\n\"The “Ugly Duckling” Sign Agreement Between Observers\"\n- https://www.researchgate.net/publication/5642885_The_Ugly_Duckling_Sign_Agreement_Between_Observers
", "rawMarkdown": "\"Skin cancer detection based on deep learning and entropy to detect outlier samples\"\n- https://arxiv.org/pdf/1909.04525.pdf\n\n\"The “Ugly Duckling” Sign Agreement Between Observers\"\n- https://www.researchgate.net/publication/5642885_The_Ugly_Duckling_Sign_Agreement_Between_Observers", "votes": null }, { "id": "871603", "postDate": "06/02/2020 13:56:03", "content": "https://ai.googleblog.com/2019/09/using-deep-learning-to-inform.html\n“A Deep Learning System for Differential Diagnosis of Skin Diseases” \n![\"\"](\"https://1.bp.blogspot.com/-b3HBRhjUZs0/XXoeACfbsmI/AAAAAAAAEoc/f4aL1TH8J2g6Ix5Tlfj-rYXemzRgPmVXwCEwYBhgL/s640/image1.png\")
![\"\"](\"https://1.bp.blogspot.com/-wMnHrj5PPys/XXoeAKVc2xI/AAAAAAAAEoI/64V1-8jUT6s-BuBRjc-EmjIpWvR7W2yZQCLcBGAsYHQ/s640/image3.png\")
Great idea Heng. This is a way to use the patient_id feature.
Surprising that although this seems so important very few people have been discussing this. So how exactly would you go about comparing images with the same patient_id? Simply based on the predicted result and giving extra weight to outliers. Or is it worth doing feature extraction on the images themselves and clustering? Would it be necessary to autocrop images first?
", "rawMarkdown": "Surprising that although this seems so important very few people have been discussing this. So how exactly would you go about comparing images with the same patient_id? Simply based on the predicted result and giving extra weight to outliers. Or is it worth doing feature extraction on the images themselves and clustering? Would it be necessary to autocrop images first?", "votes": null }, { "id": "900196", "postDate": "06/24/2020 17:20:40", "content": "I was wondering whether tiling say four images from the same patient and training a model with binary cross entropy on the class of each image would allow it to learn to pick up differences in features across the tiled images.
\n\nFor example, the input to the model could by 512x512x3 comprised of four 256x256x3 images from the same patient and the labels would be a 4x1 vector.
", "rawMarkdown": "I was wondering whether tiling say four images from the same patient and training a model with binary cross entropy on the class of each image would allow it to learn to pick up differences in features across the tiled images.\n\nFor example, the input to the model could by 512x512x3 comprised of four 256x256x3 images from the same patient and the labels would be a 4x1 vector.", "votes": null }, { "id": "900245", "postDate": "06/24/2020 17:46:01", "content": "Interesting approach, Caleb. But what if there are 20, are more images for a given patient. Should be make smaller images and incorporate all of them or stick to a fixed number?\nI was thinking of a different approach. Creating image embeddings and then do some clustering or other comparison to see if I can find different groups and outliers. I would then use this info to update the probabilities for the given image for this patient. \nAnyway, pretty new to this, but I'm sure this is where we need to apply our ingenuity.\nReally interested to hearing other ideas.\nI remember DETR Network being mentioned in another post as an option....any others?
", "rawMarkdown": "Interesting approach, Caleb. But what if there are 20, are more images for a given patient. Should be make smaller images and incorporate all of them or stick to a fixed number?\nI was thinking of a different approach. Creating image embeddings and then do some clustering or other comparison to see if I can find different groups and outliers. I would then use this info to update the probabilities for the given image for this patient. \nAnyway, pretty new to this, but I'm sure this is where we need to apply our ingenuity.\nReally interested to hearing other ideas.\nI remember DETR Network being mentioned in another post as an option....any others?", "votes": null }, { "id": "900317", "postDate": "06/24/2020 18:30:28", "content": "Cool, interested to see the embedding / clustering approach. I was thinking that you would select images at random for each patient, applying augmentation to each before assembling the tiles to feed in to the model, so that it would see different combinations of images from each patient for each epoch.
", "rawMarkdown": "Cool, interested to see the embedding / clustering approach. I was thinking that you would select images at random for each patient, applying augmentation to each before assembling the tiles to feed in to the model, so that it would see different combinations of images from each patient for each epoch.", "votes": null }, { "id": "901312", "postDate": "06/25/2020 11:36:19", "content": "Caleb, I think your idea has most promise\nBut if you combine say 4 images wont you have 2**4 output options? So you will need something like categorical cross entropy? Apologies if I dont understand.\nOther option I see is using multi input and then you can either concatenate with similar result as above or multi-output and then you can train each image separately using binary
", "rawMarkdown": "Caleb, I think your idea has most promise\nBut if you combine say 4 images wont you have 2**4 output options? So you will need something like categorical cross entropy? Apologies if I dont understand.\nOther option I see is using multi input and then you can either concatenate with similar result as above or multi-output and then you can train each image separately using binary", "votes": null }, { "id": "901994", "postDate": "06/25/2020 21:03:14", "content": "I was thinking you would just have a 4x1 output vector that represented the target for each of the four images with a sigmoid activation so that you got a predicted probability between 0 and 1 for each element. Loss function would be binary cross entropy I think so that you could have more than one positive image per sequence. Nice discussion here
\n\nAnother way to do this would be to create a sequence of a set number of images from the same patient, run them through a cnn with a TimeDistributed layer wrapped around it, concat embeddings and predict the same 4x1 vector.
", "rawMarkdown": "I was thinking you would just have a 4x1 output vector that represented the target for each of the four images with a sigmoid activation so that you got a predicted probability between 0 and 1 for each element. Loss function would be binary cross entropy I think so that you could have more than one positive image per sequence. Nice discussion [here](https://gombru.github.io/2018/05/23/cross_entropy_loss/)\n\nAnother way to do this would be to create a sequence of a set number of images from the same patient, run them through a cnn with a TimeDistributed layer wrapped around it, concat embeddings and predict the same 4x1 vector.", "votes": null } ], "comments": [ { "id": 869893, "author_name": "hengck23", "author_url": "", "post_date": "06/01/2020 10:55:42", "content": "\"Skin cancer detection based on deep learning and entropy to detect outlier samples\"\n- https://arxiv.org/pdf/1909.04525.pdf
\n\n\"The “Ugly Duckling” Sign Agreement Between Observers\"\n- https://www.researchgate.net/publication/5642885_The_Ugly_Duckling_Sign_Agreement_Between_Observers
", "votes": null, "replies": [] }, { "id": 871603, "author_name": "hengck23", "author_url": "", "post_date": "06/02/2020 13:56:03", "content": "https://ai.googleblog.com/2019/09/using-deep-learning-to-inform.html\n“A Deep Learning System for Differential Diagnosis of Skin Diseases” \n
Great idea Heng. This is a way to use the patient_id feature.
Surprising that although this seems so important very few people have been discussing this. So how exactly would you go about comparing images with the same patient_id? Simply based on the predicted result and giving extra weight to outliers. Or is it worth doing feature extraction on the images themselves and clustering? Would it be necessary to autocrop images first?
", "votes": null, "replies": [] }, { "id": 900196, "author_name": "calebeverett", "author_url": "", "post_date": "06/24/2020 17:20:40", "content": "I was wondering whether tiling say four images from the same patient and training a model with binary cross entropy on the class of each image would allow it to learn to pick up differences in features across the tiled images.
\n\nFor example, the input to the model could by 512x512x3 comprised of four 256x256x3 images from the same patient and the labels would be a 4x1 vector.
", "votes": null, "replies": [ { "id": 901312, "author_name": "daisenthal", "author_url": "", "post_date": "06/25/2020 11:36:19", "content": "Caleb, I think your idea has most promise\nBut if you combine say 4 images wont you have 2**4 output options? So you will need something like categorical cross entropy? Apologies if I dont understand.\nOther option I see is using multi input and then you can either concatenate with similar result as above or multi-output and then you can train each image separately using binary
", "votes": null, "replies": [] }, { "id": 901994, "author_name": "calebeverett", "author_url": "", "post_date": "06/25/2020 21:03:14", "content": "I was thinking you would just have a 4x1 output vector that represented the target for each of the four images with a sigmoid activation so that you got a predicted probability between 0 and 1 for each element. Loss function would be binary cross entropy I think so that you could have more than one positive image per sequence. Nice discussion here
\n\nAnother way to do this would be to create a sequence of a set number of images from the same patient, run them through a cnn with a TimeDistributed layer wrapped around it, concat embeddings and predict the same 4x1 vector.
", "votes": null, "replies": [] } ] }, { "id": 900245, "author_name": "daisenthal", "author_url": "", "post_date": "06/24/2020 17:46:01", "content": "Interesting approach, Caleb. But what if there are 20, are more images for a given patient. Should be make smaller images and incorporate all of them or stick to a fixed number?\nI was thinking of a different approach. Creating image embeddings and then do some clustering or other comparison to see if I can find different groups and outliers. I would then use this info to update the probabilities for the given image for this patient. \nAnyway, pretty new to this, but I'm sure this is where we need to apply our ingenuity.\nReally interested to hearing other ideas.\nI remember DETR Network being mentioned in another post as an option....any others?
", "votes": null, "replies": [ { "id": 900317, "author_name": "calebeverett", "author_url": "", "post_date": "06/24/2020 18:30:28", "content": "Cool, interested to see the embedding / clustering approach. I was thinking that you would select images at random for each patient, applying augmentation to each before assembling the tiles to feed in to the model, so that it would see different combinations of images from each patient for each epoch.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "869887": "this is probably the most important objective of the competition.\n\n\"Currently, dermatologists evaluate every one of a patient's moles to identify outlier lesions or “ugly ducklings” that are most likely to be melanoma.\"\n\nwe can simply treat this as an image recognition problem (like the past ISIC competitions), but can we do better than that?\n\nFirst let us understand what is \"Ugly Duckling Concept\"\nhttps://www.medscape.com/viewarticle/733861_3\n\n\"In 1998, Grob, et al[7] introduced the ugly duckling concept-the observation that nevi in the same individual tend to resemble one another, and that MM often deviates from this nevus pattern. This clinical realization pointed to the importance of not just evaluating the morphology of the lesion in question, but also comparing it to that of surrounding lesions, looking for an outlier in the background of similar-appearing moles. For example, the outlier lesion can be larger and darker than the surrounding moles (Figure 1 A), or conversely, small and red in the background of multiple large dark moles (Figure 1 B). Finally, if the patient has few or no other moles (Figure 1 C). any changing lesion should be considered a suspicious outlier.\"\n\n\n\nso what does it means? This competition is really about \"outlier detection\". can you measure differences for a given set of mole images? And then using the measured difference to predict melanoma?\n\nif you find papers and works on using ML/deep learning for \"Ugly Duckling Concept\", please share it here.\n\nI estimate pure image recognition (without meta data) to be around AUC 0.95 ~ 0.96. If \"Ugly Duckling Concept\" works, maybe we can get to 0.97~0.98 ???", "869893": "\"Skin cancer detection based on deep learning and entropy to detect outlier samples\"\n- https://arxiv.org/pdf/1909.04525.pdf\n\n\"The “Ugly Duckling” Sign Agreement Between Observers\"\n- https://www.researchgate.net/publication/5642885_The_Ugly_Duckling_Sign_Agreement_Between_Observers", "871603": "https://ai.googleblog.com/2019/09/using-deep-learning-to-inform.html\n“A Deep Learning System for Differential Diagnosis of Skin Diseases” \n\n\n", "871738": "Great idea Heng. This is a way to use the `patient_id` feature.", "899717": "Surprising that although this seems so important very few people have been discussing this. So how exactly would you go about comparing images with the same patient_id? Simply based on the predicted result and giving extra weight to outliers. Or is it worth doing feature extraction on the images themselves and clustering? Would it be necessary to autocrop images first?", "900196": "I was wondering whether tiling say four images from the same patient and training a model with binary cross entropy on the class of each image would allow it to learn to pick up differences in features across the tiled images.\n\nFor example, the input to the model could by 512x512x3 comprised of four 256x256x3 images from the same patient and the labels would be a 4x1 vector.", "900245": "Interesting approach, Caleb. But what if there are 20, are more images for a given patient. Should be make smaller images and incorporate all of them or stick to a fixed number?\nI was thinking of a different approach. Creating image embeddings and then do some clustering or other comparison to see if I can find different groups and outliers. I would then use this info to update the probabilities for the given image for this patient. \nAnyway, pretty new to this, but I'm sure this is where we need to apply our ingenuity.\nReally interested to hearing other ideas.\nI remember DETR Network being mentioned in another post as an option....any others?", "900317": "Cool, interested to see the embedding / clustering approach. I was thinking that you would select images at random for each patient, applying augmentation to each before assembling the tiles to feed in to the model, so that it would see different combinations of images from each patient for each epoch.", "901312": "Caleb, I think your idea has most promise\nBut if you combine say 4 images wont you have 2**4 output options? So you will need something like categorical cross entropy? Apologies if I dont understand.\nOther option I see is using multi input and then you can either concatenate with similar result as above or multi-output and then you can train each image separately using binary", "901994": "I was thinking you would just have a 4x1 output vector that represented the target for each of the four images with a sigmoid activation so that you got a predicted probability between 0 and 1 for each element. Loss function would be binary cross entropy I think so that you could have more than one positive image per sequence. Nice discussion [here](https://gombru.github.io/2018/05/23/cross_entropy_loss/)\n\nAnother way to do this would be to create a sequence of a set number of images from the same patient, run them through a cnn with a TimeDistributed layer wrapped around it, concat embeddings and predict the same 4x1 vector." }, "source": "meta" }