Dear kagglers,\nI am glad to welcome you in the competition.
\n\nI am still a bit confused a bout Confidence column we supposed to provide in submission file.\nIt is already pretty clear that we are talking about standard deviation.
\n\nHowever I am still a bit confused how we sould estimate it. Is it std of a single patient or a single week or just std over entire predicted FVC column?
\n\nMoreover, may you provide me a clarification, there are already tones of kernels where this parameter ( Confidence) is optimized with scipy. Could you please explain me why does it make sense?
", "messages": [ { "id": "928267", "postDate": "07/13/2020 20:36:13", "content": "Dear kagglers,\nI am glad to welcome you in the competition.
\n\nI am still a bit confused a bout Confidence column we supposed to provide in submission file.\nIt is already pretty clear that we are talking about standard deviation.
\n\nHowever I am still a bit confused how we sould estimate it. Is it std of a single patient or a single week or just std over entire predicted FVC column?
\n\nMoreover, may you provide me a clarification, there are already tones of kernels where this parameter ( Confidence) is optimized with scipy. Could you please explain me why does it make sense?
", "rawMarkdown": "Dear kagglers,\nI am glad to welcome you in the competition.\n\nI am still a bit confused a bout Confidence column we supposed to provide in submission file.\nIt is already pretty clear that we are talking about standard deviation.\n\nHowever I am still a bit confused how we sould estimate it. Is it std of a single patient or a single week or just std over entire predicted FVC column?\n\nMoreover, may you provide me a clarification, there are already tones of kernels where this parameter ( Confidence) is optimized with scipy. Could you please explain me why does it make sense?", "votes": null }, { "id": "931656", "postDate": "07/16/2020 11:02:40", "content": "In my opinion you can think of this as a measure of \"risk\" given how the metric is designed.\nWhen sigma is large, it is OK if your prediction is off the true value, but the penalty of sigma itself is high.\nWhen sigma is small, the penalty of sigma itself is low, but the prediction needs to be close to the true value, otherwise the loss would be high.\nWith this in mind, you can sort of understand why sigma is called \"confidence\". If you are confident that your predictions are perfect, you can set sigma to the minimal value so that the penalty of it is minimized.
", "rawMarkdown": "In my opinion you can think of this as a measure of \"risk\" given how the metric is designed.\nWhen sigma is large, it is OK if your prediction is off the true value, but the penalty of sigma itself is high.\nWhen sigma is small, the penalty of sigma itself is low, but the prediction needs to be close to the true value, otherwise the loss would be high.\nWith this in mind, you can sort of understand why sigma is called \"confidence\". If you are confident that your predictions are perfect, you can set sigma to the minimal value so that the penalty of it is minimized.", "votes": null }, { "id": "938037", "postDate": "07/21/2020 10:02:08", "content": "Thank you for the clear response. I have already got some idea about confidence . I am preparing kind of summary on this parameter and going to post it here sooner or later.
", "rawMarkdown": "Thank you for the clear response. I have already got some idea about confidence . I am preparing kind of summary on this parameter and going to post it here sooner or later.", "votes": null }, { "id": "943730", "postDate": "07/24/2020 14:39:00", "content": "Thanks for great explanation
", "rawMarkdown": "Thanks for great explanation", "votes": null }, { "id": "952448", "postDate": "07/30/2020 23:38:31", "content": "Why do you think it's called standard deviation in the evaluation page? Thanks in advance!
", "rawMarkdown": "Why do you think it's called standard deviation in the evaluation page? Thanks in advance!", "votes": null }, { "id": "952768", "postDate": "07/31/2020 07:57:39", "content": "It comes from the definition of the evaluation metric. Take a look at the definition of the Laplace distribution, and the variance of the distribution.
", "rawMarkdown": "It comes from the definition of the evaluation metric. Take a look at the definition of the Laplace distribution, and the variance of the distribution.", "votes": null }, { "id": "953630", "postDate": "07/31/2020 23:46:32", "content": "ah, got it. Thank you.
", "rawMarkdown": "ah, got it. Thank you.", "votes": null }, { "id": "1006350", "postDate": "09/11/2020 07:41:50", "content": "@YALICKJ First of all thank you for the clear idea.
\nMy question is then, is it going to be the same confidence for all the predicted FVC for a particular patient??
\nor we have to consider the deviation of each point from its average value.
@YALICKJ First of all thank you for the clear idea.
\nMy question is then, is it going to be the same confidence for all the predicted FVC for a particular patient??
\nor we have to consider the deviation of each point from its average value.
In my opinion you can think of this as a measure of \"risk\" given how the metric is designed.\nWhen sigma is large, it is OK if your prediction is off the true value, but the penalty of sigma itself is high.\nWhen sigma is small, the penalty of sigma itself is low, but the prediction needs to be close to the true value, otherwise the loss would be high.\nWith this in mind, you can sort of understand why sigma is called \"confidence\". If you are confident that your predictions are perfect, you can set sigma to the minimal value so that the penalty of it is minimized.
", "votes": null, "replies": [ { "id": 938037, "author_name": "maksimbahdanchyk", "author_url": "", "post_date": "07/21/2020 10:02:08", "content": "Thank you for the clear response. I have already got some idea about confidence . I am preparing kind of summary on this parameter and going to post it here sooner or later.
", "votes": null, "replies": [] }, { "id": 943730, "author_name": "vivekgopalramaswamy", "author_url": "", "post_date": "07/24/2020 14:39:00", "content": "Thanks for great explanation
", "votes": null, "replies": [] }, { "id": 952448, "author_name": "unkownhihi", "author_url": "", "post_date": "07/30/2020 23:38:31", "content": "Why do you think it's called standard deviation in the evaluation page? Thanks in advance!
", "votes": null, "replies": [] }, { "id": 952768, "author_name": "yalickj", "author_url": "", "post_date": "07/31/2020 07:57:39", "content": "It comes from the definition of the evaluation metric. Take a look at the definition of the Laplace distribution, and the variance of the distribution.
", "votes": null, "replies": [] }, { "id": 953630, "author_name": "unkownhihi", "author_url": "", "post_date": "07/31/2020 23:46:32", "content": "ah, got it. Thank you.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "928267": "Dear kagglers,\nI am glad to welcome you in the competition.\n\nI am still a bit confused a bout Confidence column we supposed to provide in submission file.\nIt is already pretty clear that we are talking about standard deviation.\n\nHowever I am still a bit confused how we sould estimate it. Is it std of a single patient or a single week or just std over entire predicted FVC column?\n\nMoreover, may you provide me a clarification, there are already tones of kernels where this parameter ( Confidence) is optimized with scipy. Could you please explain me why does it make sense?", "931656": "In my opinion you can think of this as a measure of \"risk\" given how the metric is designed.\nWhen sigma is large, it is OK if your prediction is off the true value, but the penalty of sigma itself is high.\nWhen sigma is small, the penalty of sigma itself is low, but the prediction needs to be close to the true value, otherwise the loss would be high.\nWith this in mind, you can sort of understand why sigma is called \"confidence\". If you are confident that your predictions are perfect, you can set sigma to the minimal value so that the penalty of it is minimized.", "938037": "Thank you for the clear response. I have already got some idea about confidence . I am preparing kind of summary on this parameter and going to post it here sooner or later.", "943730": "Thanks for great explanation", "952448": "Why do you think it's called standard deviation in the evaluation page? Thanks in advance!", "952768": "It comes from the definition of the evaluation metric. Take a look at the definition of the Laplace distribution, and the variance of the distribution.", "953630": "ah, got it. Thank you.", "1006350": "YALICKJ First of all thank you for the clear idea.\nMy question is then, is it going to be the same confidence for all the predicted FVC for a particular patient??\nor we have to consider the deviation of each point from its average value." }, "source": "meta" }