As it has already been mentioned, dermatologist use the \"Ugly Duckling Effect\" to detect melanomas : they look at all the moles of the patient, if one looks weird and very different from other moles then they get suspicious. This is based on the fact that it's impossible (or very very unlikely) that some one would get 20 melanomas at the same time, so if you have 20 moles looking alike then there is probably nothing to worry about. But knowing this, have you been able to use it successfully in your models?
\nI personally tried a quite unusual approach without much success but I thought it would still be nice to share it publicly and create this thread so that people can discuss about their approach here.
\nI created a graph for each patient :
\nI tried to use a Message Passing Neural network approach using MetaLayer from the excellent pytorch-geometric library (https://github.com/rusty1s/pytorch_geometric). This consist of 3 models for Edges, Nodes and GlobalFeatures that learn how to exchange information from each other in order to do Node classification.
So I stacked 3 Metalayers and then a simple MLP in order to get the class for each node (each image). The idea was that you are using the entire graph in order to take a decision, and the model had access to enough information to learn the \"Ugly Duckling\" concept, if you had a lot of edges with small distances maybe you're not very different from others so you are not a melanoma.
\nMy model trained and I could get something like CV score of 0.935~0.94 (but this was used with multiple oof predictions so a simple averaging would have done better ~0.95). So I did not manage to push this idea to a useful submission but I think there is some potential into it. As I'm no expert in GNNs I would be happy to receive advice on this!
\nMaybe this approach could work but we would need to find a better way to encode the image than with pretrained embeddings, maybe some smarter distances would help…
\nAs it has already been mentioned, dermatologist use the \"Ugly Duckling Effect\" to detect melanomas : they look at all the moles of the patient, if one looks weird and very different from other moles then they get suspicious. This is based on the fact that it's impossible (or very very unlikely) that some one would get 20 melanomas at the same time, so if you have 20 moles looking alike then there is probably nothing to worry about. But knowing this, have you been able to use it successfully in your models?
\nI personally tried a quite unusual approach without much success but I thought it would still be nice to share it publicly and create this thread so that people can discuss about their approach here.
\nI created a graph for each patient :
\nI tried to use a Message Passing Neural network approach using MetaLayer from the excellent pytorch-geometric library (https://github.com/rusty1s/pytorch_geometric). This consist of 3 models for Edges, Nodes and GlobalFeatures that learn how to exchange information from each other in order to do Node classification.
So I stacked 3 Metalayers and then a simple MLP in order to get the class for each node (each image). The idea was that you are using the entire graph in order to take a decision, and the model had access to enough information to learn the \"Ugly Duckling\" concept, if you had a lot of edges with small distances maybe you're not very different from others so you are not a melanoma.
\nMy model trained and I could get something like CV score of 0.935~0.94 (but this was used with multiple oof predictions so a simple averaging would have done better ~0.95). So I did not manage to push this idea to a useful submission but I think there is some potential into it. As I'm no expert in GNNs I would be happy to receive advice on this!
\nMaybe this approach could work but we would need to find a better way to encode the image than with pretrained embeddings, maybe some smarter distances would help…
\nWe had a similar idea, but didnt execute it in the end. Couldn't you perhaps train it end-to-end such that you learn both representations of your images (CNN) and how the images and metadata of a same patient can be combined/aggregated (GNN)?
", "rawMarkdown": "We had a similar idea, but didnt execute it in the end. Couldn't you perhaps train it end-to-end such that you learn both representations of your images (CNN) and how the images and metadata of a same patient can be combined/aggregated (GNN)?", "votes": null }, { "id": "976014", "postDate": "08/18/2020 15:28:35", "content": "Yeah that seems a good approach but it would be a huge model as you'll have to provide the entire image for each node. If you want to have a batch size of say 8 : you would need to have 8xnum_images in your model seems huge.
\nIt would be very interesting to try though!
", "rawMarkdown": "Yeah that seems a good approach but it would be a huge model as you'll have to provide the entire image for each node. If you want to have a batch size of say 8 : you would need to have 8xnum_images in your model seems huge.\n\nIt would be very interesting to try though!", "votes": null }, { "id": "976969", "postDate": "08/19/2020 07:53:06", "content": "This is a very interesting approach! Did you try checking performance on LB with a late submission?
\nWe did some experiments on “ugly duckling” using a simpler strategy: based on within-patient image distances extracted from CNN embeddings and based on predictions distribution within patients but did not succeed as well. I posted a brief overview of our experiments here.
", "rawMarkdown": "This is a very interesting approach! Did you try checking performance on LB with a late submission?\n\nWe did some experiments on “ugly duckling” using a simpler strategy: based on within-patient image distances extracted from CNN embeddings and based on predictions distribution within patients but did not succeed as well. I posted a brief overview of our experiments [here](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175500).", "votes": null }, { "id": "977138", "postDate": "08/19/2020 10:11:03", "content": "I'll try a late submission and update you with the results!
", "rawMarkdown": "I'll try a late submission and update you with the results!", "votes": null } ], "comments": [ { "id": 976009, "author_name": "group16", "author_url": "", "post_date": "08/18/2020 15:24:59", "content": "We had a similar idea, but didnt execute it in the end. Couldn't you perhaps train it end-to-end such that you learn both representations of your images (CNN) and how the images and metadata of a same patient can be combined/aggregated (GNN)?
", "votes": null, "replies": [ { "id": 976014, "author_name": "optimo", "author_url": "", "post_date": "08/18/2020 15:28:35", "content": "Yeah that seems a good approach but it would be a huge model as you'll have to provide the entire image for each node. If you want to have a batch size of say 8 : you would need to have 8xnum_images in your model seems huge.
\nIt would be very interesting to try though!
", "votes": null, "replies": [] } ] }, { "id": 976969, "author_name": "kozodoi", "author_url": "", "post_date": "08/19/2020 07:53:06", "content": "This is a very interesting approach! Did you try checking performance on LB with a late submission?
\nWe did some experiments on “ugly duckling” using a simpler strategy: based on within-patient image distances extracted from CNN embeddings and based on predictions distribution within patients but did not succeed as well. I posted a brief overview of our experiments here.
", "votes": null, "replies": [ { "id": 977138, "author_name": "optimo", "author_url": "", "post_date": "08/19/2020 10:11:03", "content": "I'll try a late submission and update you with the results!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "975993": "# How did you deal with \"Ugly Duckling Effect\"?\n\nAs it has already been mentioned, dermatologist use the \"Ugly Duckling Effect\" to detect melanomas : they look at all the moles of the patient, if one looks weird and very different from other moles then they get suspicious. This is based on the fact that it's impossible (or very very unlikely) that some one would get 20 melanomas at the same time, so if you have 20 moles looking alike then there is probably nothing to worry about. But knowing this, have you been able to use it successfully in your models?\n\n\n# Graph Neural Networks\n\nI personally tried a quite unusual approach without much success but I thought it would still be nice to share it publicly and create this thread so that people can discuss about their approach here.\n\n## The Graph\n\nI created a graph for each patient :\n- **each node**: consisted in meta features related to one image, like age or location AND the final head of pretrained model (about 2048 features). I also added the OOF prediction score from the pretrained model. With this every node has information about one specific image.\n- **each edge** : every node was connected to any other node, so the graph was fully connected and undirected. Edges contained only two information (but more could have been added) : L2 distance between pertained embeddings and Age difference between the two nodes\n- **global features** : each graph has patient specific features like sex.\n\n## The Model\n\nI tried to use a Message Passing Neural network approach using `MetaLayer` from the excellent pytorch-geometric library (https://github.com/rusty1s/pytorch_geometric). This consist of 3 models for Edges, Nodes and GlobalFeatures that learn how to exchange information from each other in order to do Node classification.\n\nSo I stacked 3 Metalayers and then a simple MLP in order to get the class for each node (each image). The idea was that you are using the entire graph in order to take a decision, and the model had access to enough information to learn the \"Ugly Duckling\" concept, if you had a lot of edges with small distances maybe you're not very different from others so you are not a melanoma.\n\n\n## The results\n\nMy model trained and I could get something like CV score of 0.935~0.94 (but this was used with multiple oof predictions so a simple averaging would have done better ~0.95). So I did not manage to push this idea to a useful submission but I think there is some potential into it. As I'm no expert in GNNs I would be happy to receive advice on this!\n\nMaybe this approach could work but we would need to find a better way to encode the image than with pretrained embeddings, maybe some smarter distances would help...\n\n\n### What was your approach ?", "976009": "We had a similar idea, but didnt execute it in the end. Couldn't you perhaps train it end-to-end such that you learn both representations of your images (CNN) and how the images and metadata of a same patient can be combined/aggregated (GNN)?", "976014": "Yeah that seems a good approach but it would be a huge model as you'll have to provide the entire image for each node. If you want to have a batch size of say 8 : you would need to have 8xnum_images in your model seems huge.\n\nIt would be very interesting to try though!", "976969": "This is a very interesting approach! Did you try checking performance on LB with a late submission?\n\nWe did some experiments on “ugly duckling” using a simpler strategy: based on within-patient image distances extracted from CNN embeddings and based on predictions distribution within patients but did not succeed as well. I posted a brief overview of our experiments [here](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175500).", "977138": "I'll try a late submission and update you with the results!" }, "source": "meta" }