Dear fellow Kagglers,
\nlet me start with a quick summary of what we know so far:
\nWe know so far that CV is definitely the way to go instead of relying purely on the public LB-score, which is only based on 15% of the data.
GroupKFold strongly beats the usage of basic KFold, as it reduces leakage.
IMG data only grants very limited additional value (except for tissue/lung-area feature) and it's hard to engineer useful features out of the few pictures we have.
\n\nPercent- a computed field which approximates the patient's FVC as a percent of the typical FVC for a person of similar characteristics
\n
Let me quickly kickstart the debate:
\nAs I explained here NOT using percent at all seems kind of wrong to me. Using it for all training will introduce leakage, as the combination of AGE and PERCENT is enough to numerically calculate the correct FVC. So my idea is to only use it for the first row for each patient, as it's also available in the test data and gives us insigts.
Dear fellow Kagglers,
\nlet me start with a quick summary of what we know so far:
\nWe know so far that CV is definitely the way to go instead of relying purely on the public LB-score, which is only based on 15% of the data.
GroupKFold strongly beats the usage of basic KFold, as it reduces leakage.
IMG data only grants very limited additional value (except for tissue/lung-area feature) and it's hard to engineer useful features out of the few pictures we have.
\n\nPercent- a computed field which approximates the patient's FVC as a percent of the typical FVC for a person of similar characteristics
\n
Let me quickly kickstart the debate:
\nAs I explained here NOT using percent at all seems kind of wrong to me. Using it for all training will introduce leakage, as the combination of AGE and PERCENT is enough to numerically calculate the correct FVC. So my idea is to only use it for the first row for each patient, as it's also available in the test data and gives us insigts.
@ChristianDenich About PERCENT: As per my analysis, two people of the same gender and age don't have standard FVC values as same (if you divide the patient-specific FVC by % and find out 'standard' FVC for that person/gender/age combination). Standard FVC seems to be the function of body weight/height/build/BMI/race/ethnicity etc. also. In that sense % in incomplete information as provided here.
", "rawMarkdown": "@ChristianDenich About PERCENT: As per my analysis, two people of the same gender and age don't have standard FVC values as same (if you divide the patient-specific FVC by % and find out 'standard' FVC for that person/gender/age combination). Standard FVC seems to be the function of body weight/height/build/BMI/race/ethnicity etc. also. In that sense % in incomplete information as provided here.", "votes": 1 }, { "id": 1033862, "postDate": "2020-10-01T10:18:38.033Z", "content": "If the Percent feature is wrong to use, a shakeup is inevitable. There is a big gap in CV/LB when it's not included.
", "rawMarkdown": "If the Percent feature is wrong to use, a shakeup is inevitable. There is a big gap in CV/LB when it's not included.", "votes": 2, "replies": [ { "id": 1034675, "postDate": "2020-10-02T04:31:31.657Z", "content": "Problem is how do you compute the Percent for each week? All I can say is for each Patient the typical FVC = FVC / Percent * 100 is constant. Maybe the typical FVC can provide some useful hidden information since Patients with same Age, Sex, Smoking Status do not necessary have the same typical FVC.
", "rawMarkdown": "Problem is how do you compute the Percent for each week? All I can say is for each Patient the typical FVC = FVC / Percent * 100 is constant. Maybe the typical FVC can provide some useful hidden information since Patients with same Age, Sex, Smoking Status do not necessary have the same typical FVC. ", "votes": 1 }, { "id": 1034853, "postDate": "2020-10-02T08:39:00.937Z", "content": "You are right that it could contain hidden info about a patient. I believe you can use either the typical FVC or the base Percent of a patient to capture that
", "rawMarkdown": "You are right that it could contain hidden info about a patient. I believe you can use either the typical FVC or the base Percent of a patient to capture that" } ] }, { "id": 1032637, "postDate": "2020-09-30T10:28:21.483Z", "content": "@gannadolinska, @ulrich07, @carlossouza any insights you want to share?
", "rawMarkdown": "@gannadolinska, @ulrich07, @carlossouza any insights you want to share?", "replies": [ { "id": 1032967, "postDate": "2020-09-30T15:03:25.890Z", "content": "So far, our best submission uses GroupKFold, uses the images to predict the betas (slopes for the FVC declines), uses all features including Percent (although I personally think it's a form of data leak), and blends models that predict the confidence with (very different) methods…
\nWe tried hundreds of different approaches: some that we thought it would work failed miserably, and some others that we didn't thought much surprised us… go figure :)
Reflecting on your (very good) question, I learned 2 \"recipes\" in this competition:
\nWow, it seems that our team is perfectly aligned with @carlossouza's team: our \"recipes\" are basically the same with some tweaks, we too experimented with several approaches, most of them resulting in no to very minor changes in the model's performance.
\nWe have learnt a lot from our submissions, and we haven't stopped yet. With each submission, we're getting new insights, some contradicting our hypothesis for selecting which one would be the ideal submission.
", "rawMarkdown": "Wow, it seems that our team is perfectly aligned with @carlossouza's team: our **\"recipes\"** are basically the same with some tweaks, we too experimented with several approaches, most of them resulting in no to very minor changes in the model's performance.\n\nWe have learnt a lot from our submissions, and we haven't stopped yet. With each submission, we're getting new insights, some contradicting our hypothesis for selecting which one would be the ideal submission.", "votes": 1 }, { "id": 1033158, "postDate": "2020-09-30T17:54:52.627Z", "content": "@aadhavvignesh We are using the same recipes as well and some additional things as well ,
\n@carlossouza I too have experimented on a lot of things that haven't work well for me though on lb , but I am very much trusting the ideas to work on private . We are yet to ensemble models as we were waiting for the last week 😉😉😉 . The only thing was we didn't have the full support of the team as they had prior commitments but nevertheless I have learned a lot here because all the pipelines I have written myself , I hope it ends well for us
\nThanks Chris for this post
@ChristianDenich About PERCENT: As per my analysis, two people of the same gender and age don't have standard FVC values as same (if you divide the patient-specific FVC by % and find out 'standard' FVC for that person/gender/age combination). Standard FVC seems to be the function of body weight/height/build/BMI/race/ethnicity etc. also. In that sense % in incomplete information as provided here.
", "votes": 1, "replies": [] }, { "id": 1033862, "author_name": "Rafi Hai", "author_url": "", "post_date": "2020-10-01T10:18:38.033000", "content": "If the Percent feature is wrong to use, a shakeup is inevitable. There is a big gap in CV/LB when it's not included.
", "votes": 2, "replies": [ { "id": 1034675, "author_name": "Quan", "author_url": "", "post_date": "2020-10-02T04:31:31.657000", "content": "Problem is how do you compute the Percent for each week? All I can say is for each Patient the typical FVC = FVC / Percent * 100 is constant. Maybe the typical FVC can provide some useful hidden information since Patients with same Age, Sex, Smoking Status do not necessary have the same typical FVC.
", "votes": 1, "replies": [] }, { "id": 1034853, "author_name": "Rafi Hai", "author_url": "", "post_date": "2020-10-02T08:39:00.937000", "content": "You are right that it could contain hidden info about a patient. I believe you can use either the typical FVC or the base Percent of a patient to capture that
", "votes": 0, "replies": [] } ] }, { "id": 1032637, "author_name": "from coffee import *", "author_url": "", "post_date": "2020-09-30T10:28:21.483000", "content": "@gannadolinska, @ulrich07, @carlossouza any insights you want to share?
", "votes": 0, "replies": [ { "id": 1032967, "author_name": "Carlos Souza", "author_url": "", "post_date": "2020-09-30T15:03:25.890000", "content": "So far, our best submission uses GroupKFold, uses the images to predict the betas (slopes for the FVC declines), uses all features including Percent (although I personally think it's a form of data leak), and blends models that predict the confidence with (very different) methods…
\nWe tried hundreds of different approaches: some that we thought it would work failed miserably, and some others that we didn't thought much surprised us… go figure :)
Reflecting on your (very good) question, I learned 2 \"recipes\" in this competition:
\nWow, it seems that our team is perfectly aligned with @carlossouza's team: our \"recipes\" are basically the same with some tweaks, we too experimented with several approaches, most of them resulting in no to very minor changes in the model's performance.
\nWe have learnt a lot from our submissions, and we haven't stopped yet. With each submission, we're getting new insights, some contradicting our hypothesis for selecting which one would be the ideal submission.
", "votes": 1, "replies": [] }, { "id": 1033158, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2020-09-30T17:54:52.627000", "content": "@aadhavvignesh We are using the same recipes as well and some additional things as well ,
\n@carlossouza I too have experimented on a lot of things that haven't work well for me though on lb , but I am very much trusting the ideas to work on private . We are yet to ensemble models as we were waiting for the last week 😉😉😉 . The only thing was we didn't have the full support of the team as they had prior commitments but nevertheless I have learned a lot here because all the pipelines I have written myself , I hope it ends well for us
\nThanks Chris for this post