\"https://www.kaggle.com/c/siim-isic-melanoma-classification/overview/prizes\"
\n\nThere are two special prize for with and without context. It is mentioned these are for \"Top-scoring\" model, does it mean \"inference\" model?
\n\nFor example, I may use context information for training and at inference i used only image (without context ).
\n\nSay i trained a multitask network to predict melanoma, age,sex, location at training. melanoma is the target output while the rest are auxiliary output. At inference, I only output melanoma prediction. Is such an approach considered with or without context for the special prize?
\n\ni could also use patent_id for augmentation, e.g. mixup images of the same patient for training only. or using GAN/auto-encoder to interpolate images within the same patient to create more training data.
", "messages": [ { "id": "875253", "postDate": "06/05/2020 15:52:29", "content": "\"https://www.kaggle.com/c/siim-isic-melanoma-classification/overview/prizes\"
\n\nThere are two special prize for with and without context. It is mentioned these are for \"Top-scoring\" model, does it mean \"inference\" model?
\n\nFor example, I may use context information for training and at inference i used only image (without context ).
\n\nSay i trained a multitask network to predict melanoma, age,sex, location at training. melanoma is the target output while the rest are auxiliary output. At inference, I only output melanoma prediction. Is such an approach considered with or without context for the special prize?
\n\ni could also use patent_id for augmentation, e.g. mixup images of the same patient for training only. or using GAN/auto-encoder to interpolate images within the same patient to create more training data.
", "rawMarkdown": "\"https://www.kaggle.com/c/siim-isic-melanoma-classification/overview/prizes\"\n\n\nThere are two special prize for with and without context. It is mentioned these are for \"Top-scoring\" model, does it mean \"inference\" model?\n\nFor example, I may use context information for training and at inference i used only image (without context ). \n\nSay i trained a multitask network to predict melanoma, age,sex, location at training. melanoma is the target output while the rest are auxiliary output. At inference, I only output melanoma prediction. Is such an approach considered with or without context for the special prize?\n\ni could also use patent_id for augmentation, e.g. mixup images of the same patient for training only. or using GAN/auto-encoder to interpolate images within the same patient to create more training data.", "votes": null }, { "id": "875378", "postDate": "06/05/2020 18:12:49", "content": "\n\n\n\"Patient-level contextual information\" is any information gleaned from a patient's overall set of images, versus information from a single image. Age and sex are not outlined as patient-level contextual information for the purposes of special prize eligibility.\n \"Without Context\" refers to approaches which do not in any way consider the variable patient_id to cluster images.
\n
I think \"contextual information\" refers to patient_id. Age, sex and anatomical site can be used in a solution and still be eligible to win the \"without context\" prize.
", "rawMarkdown": "> \"Patient-level contextual information\" is any information gleaned from a patient's overall set of images, versus information from a single image. **Age and sex are not outlined as patient-level contextual information for the purposes of special prize eligibility.**\n\"Without Context\" refers to approaches which do not in any way consider the variable patient_id to cluster images.\n\nI think \"contextual information\" refers to patient_id. Age, sex and anatomical site can be used in a solution and still be eligible to win the \"without context\" prize.", "votes": null }, { "id": "875417", "postDate": "06/05/2020 19:01:43", "content": "Age, sex can be used - nothing else - so I remember.\nGuess it will be difficult to control the usage of something else in all the ensembled scores.
", "rawMarkdown": "Age, sex can be used - nothing else - so I remember.\nGuess it will be difficult to control the usage of something else in all the ensembled scores.", "votes": null }, { "id": "878169", "postDate": "06/08/2020 10:14:49", "content": "@hengck23 I think \"context\" refers to what you mentioned before: the \"ugly duckling\" effect.\nUsing multiple lesions information from a same patient to make the diagnosis (whether it's a count of images or the images themselves).
\n\nI think it's mostly for differentiating the real life usage: should practitioners take a single shot of a lesion and look at the score, or should they start a specific session for a patient and take multiple shots before looking at the results. In that sense, I don't think the way you train your model matters, only the way you make inference. But I guess clarification from the hosts would be nice.
", "rawMarkdown": "hengck23 I think \"context\" refers to what you mentioned before: the \"ugly duckling\" effect.\nUsing multiple lesions information from a same patient to make the diagnosis (whether it's a count of images or the images themselves). \n\nI think it's mostly for differentiating the real life usage: should practitioners take a single shot of a lesion and look at the score, or should they start a specific session for a patient and take multiple shots before looking at the results. In that sense, I don't think the way you train your model matters, only the way you make inference. But I guess clarification from the hosts would be nice.", "votes": null }, { "id": "878798", "postDate": "06/08/2020 21:17:54", "content": "Good question. To clarify, \"Context\" means the contextual images were used for both training and inference. So to count as eligible for either the context or without context awards, your submission must entirely use or not use context through training and inference.
\n\nSo in your example @hengck23, age and sex are not considered patient contextual information (see Prizes page), so training to output that information and then predicting on the target at inference would be considered \"without context.\" Likewise, if you train using patient_id & sets of images across each patient, then that would necessarily have to be consistently used for inference in order to qualify as \"with context.\"
", "rawMarkdown": "Good question. To clarify, \"Context\" means the contextual images were used for both training and inference. So to count as eligible for either the context or without context awards, your submission must entirely use or not use context through training *and* inference. \n\nSo in your example @hengck23, age and sex are **not** considered patient contextual information (see [Prizes](https://www.kaggle.com/c/siim-isic-melanoma-classification/overview/prizes) page), so training to output that information and then predicting on the target at inference would be considered \"without context.\" Likewise, if you train using patient_id & sets of images across each patient, then that would necessarily have to be consistently used for inference in order to qualify as \"with context.\"", "votes": null } ], "comments": [ { "id": 875378, "author_name": "yeeseng", "author_url": "", "post_date": "06/05/2020 18:12:49", "content": "\n\n\n\"Patient-level contextual information\" is any information gleaned from a patient's overall set of images, versus information from a single image. Age and sex are not outlined as patient-level contextual information for the purposes of special prize eligibility.\n \"Without Context\" refers to approaches which do not in any way consider the variable patient_id to cluster images.
\n
I think \"contextual information\" refers to patient_id. Age, sex and anatomical site can be used in a solution and still be eligible to win the \"without context\" prize.
", "votes": null, "replies": [] }, { "id": 875417, "author_name": "romanweilguny", "author_url": "", "post_date": "06/05/2020 19:01:43", "content": "Age, sex can be used - nothing else - so I remember.\nGuess it will be difficult to control the usage of something else in all the ensembled scores.
", "votes": null, "replies": [] }, { "id": 878169, "author_name": "optimo", "author_url": "", "post_date": "06/08/2020 10:14:49", "content": "@hengck23 I think \"context\" refers to what you mentioned before: the \"ugly duckling\" effect.\nUsing multiple lesions information from a same patient to make the diagnosis (whether it's a count of images or the images themselves).
\n\nI think it's mostly for differentiating the real life usage: should practitioners take a single shot of a lesion and look at the score, or should they start a specific session for a patient and take multiple shots before looking at the results. In that sense, I don't think the way you train your model matters, only the way you make inference. But I guess clarification from the hosts would be nice.
", "votes": null, "replies": [] }, { "id": 878798, "author_name": "juliaelliott", "author_url": "", "post_date": "06/08/2020 21:17:54", "content": "Good question. To clarify, \"Context\" means the contextual images were used for both training and inference. So to count as eligible for either the context or without context awards, your submission must entirely use or not use context through training and inference.
\n\nSo in your example @hengck23, age and sex are not considered patient contextual information (see Prizes page), so training to output that information and then predicting on the target at inference would be considered \"without context.\" Likewise, if you train using patient_id & sets of images across each patient, then that would necessarily have to be consistently used for inference in order to qualify as \"with context.\"
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "875253": "\"https://www.kaggle.com/c/siim-isic-melanoma-classification/overview/prizes\"\n\n\nThere are two special prize for with and without context. It is mentioned these are for \"Top-scoring\" model, does it mean \"inference\" model?\n\nFor example, I may use context information for training and at inference i used only image (without context ). \n\nSay i trained a multitask network to predict melanoma, age,sex, location at training. melanoma is the target output while the rest are auxiliary output. At inference, I only output melanoma prediction. Is such an approach considered with or without context for the special prize?\n\ni could also use patent_id for augmentation, e.g. mixup images of the same patient for training only. or using GAN/auto-encoder to interpolate images within the same patient to create more training data.", "875378": "> \"Patient-level contextual information\" is any information gleaned from a patient's overall set of images, versus information from a single image. **Age and sex are not outlined as patient-level contextual information for the purposes of special prize eligibility.**\n\"Without Context\" refers to approaches which do not in any way consider the variable patient_id to cluster images.\n\nI think \"contextual information\" refers to patient_id. Age, sex and anatomical site can be used in a solution and still be eligible to win the \"without context\" prize.", "875417": "Age, sex can be used - nothing else - so I remember.\nGuess it will be difficult to control the usage of something else in all the ensembled scores.", "878169": "hengck23 I think \"context\" refers to what you mentioned before: the \"ugly duckling\" effect.\nUsing multiple lesions information from a same patient to make the diagnosis (whether it's a count of images or the images themselves). \n\nI think it's mostly for differentiating the real life usage: should practitioners take a single shot of a lesion and look at the score, or should they start a specific session for a patient and take multiple shots before looking at the results. In that sense, I don't think the way you train your model matters, only the way you make inference. But I guess clarification from the hosts would be nice.", "878798": "Good question. To clarify, \"Context\" means the contextual images were used for both training and inference. So to count as eligible for either the context or without context awards, your submission must entirely use or not use context through training *and* inference. \n\nSo in your example @hengck23, age and sex are **not** considered patient contextual information (see [Prizes](https://www.kaggle.com/c/siim-isic-melanoma-classification/overview/prizes) page), so training to output that information and then predicting on the target at inference would be considered \"without context.\" Likewise, if you train using patient_id & sets of images across each patient, then that would necessarily have to be consistently used for inference in order to qualify as \"with context.\"" }, "source": "meta" }