![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F890965%2Fbcf666d9a4bda8590beff62d544d7b2f%2Fimg1.png?generation=1601394385051620&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F890965%2F70b14d557f8f7b72e92ffc1f4d9c6b4e%2Fimg2.png?generation=1601394420503521&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F890965%2Fda2b02ec2b751887880453ea8e6c8a0a%2Fimg3.png?generation=1601394434530201&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F890965%2F95b995e471f93c5d630012d346e7223c%2Fimg4.png?generation=1601394451574647&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F890965%2F5bbffc87da9494077b193ade393d841c%2Fimg5.png?generation=1601394463485458&alt=media\")
Hello,
\nI have used a 3D VGG like CNN trained from scratch with the competition images (reduced to 48x48x48) as one input, and the features that nearly everyone is using as another input into a small MLP, then the output of the CNN (around 300 features) is concatenated to the output of the the MLP and then 2 Dense layers and then the pinball loss is used to optimize the whole model.
This got me to -6.86 on the public test set.
\nThe images below are what I have when I predict the 5 samples from the submission test set.
\nI would be interested to know if anyone can share the same kind of results. All of the predicted curves look very much the same with slight variations of the slope, seems corrected by the base FVC and have a confidence that gently increases with the distance from the base week. Do you have a different kind of results ?
\nThanks
\n
\n
\n
\n
\n
Hello,
\nI have used a 3D VGG like CNN trained from scratch with the competition images (reduced to 48x48x48) as one input, and the features that nearly everyone is using as another input into a small MLP, then the output of the CNN (around 300 features) is concatenated to the output of the the MLP and then 2 Dense layers and then the pinball loss is used to optimize the whole model.
This got me to -6.86 on the public test set.
\nThe images below are what I have when I predict the 5 samples from the submission test set.
\nI would be interested to know if anyone can share the same kind of results. All of the predicted curves look very much the same with slight variations of the slope, seems corrected by the base FVC and have a confidence that gently increases with the distance from the base week. Do you have a different kind of results ?
\nThanks
\n
\n
\n
\n
\n
Thanks for your informative share. The lb score from this prediction is -7.2, so take it as a bad example;) Yet my observation shares your view:
\nAll of the predicted curves look very much the same with slight variations of the slope, seems corrected by the base FVC and have a confidence that gently increases with the distance from the base week.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F590240%2Ffcf26f75f93c0000bb21fefd0451e05f%2F2020-10-02%209.37.25.png?generation=1601599104778073&alt=media\")
Thanks for sharing your results. Do you somehow impose linearity with respect to the weeks feature or does your NN construct comes up with this by itself?
\nAnd just to ensure that your aware of this: Probably, you used the 5 patients from the test folder as well to train your network (or did you explicitly exclude them from training?). So you should expect the model to work less well than in your plots for the private LB.
", "rawMarkdown": "Thanks for sharing your results. Do you somehow impose linearity with respect to the weeks feature or does your NN construct comes up with this by itself?\n\nAnd just to ensure that your aware of this: Probably, you used the 5 patients from the test folder as well to train your network (or did you explicitly exclude them from training?). So you should expect the model to work less well than in your plots for the private LB.", "votes": null }, { "id": "1034836", "postDate": "10/02/2020 08:15:35", "content": "You are welcome. I don't impose anything, it's the result from the NN (averaged on 5 folds for prediction and confidence). And it's not totally linear.
\nI know that the 5 patients are in the training set (thanks), I did a training session by excluding them and I got the same kind of results with a slighly lesser overall score.
", "rawMarkdown": "You are welcome. I don't impose anything, it's the result from the NN (averaged on 5 folds for prediction and confidence). And it's not totally linear.\n\nI know that the 5 patients are in the training set (thanks), I did a training session by excluding them and I got the same kind of results with a slighly lesser overall score.", "votes": null }, { "id": "1034838", "postDate": "10/02/2020 08:18:04", "content": "Thanks for sharing, you have slightly more complex results than mines. What kind of model did you use ?
", "rawMarkdown": "Thanks for sharing, you have slightly more complex results than mines. What kind of model did you use ?", "votes": null }, { "id": "1034861", "postDate": "10/02/2020 08:46:59", "content": "This is from an ensemble of multiple quantile regression models using only tabular data (without percent feature). I wanted to make a 3DCNN model like you did but could not manage;(
", "rawMarkdown": "This is from an ensemble of multiple quantile regression models using only tabular data (without percent feature). I wanted to make a 3DCNN model like you did but could not manage;(", "votes": null }, { "id": "1034882", "postDate": "10/02/2020 09:06:40", "content": "Interesting. I wasn't able to build a NN that does not completely overfit, but you and @code1110 apparently succeeded. It seems that I still have to learn many things about NNs.
\nFor example my prediction for the first patient from sub (left for FVC and right for confidence prediction) looks like this:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F966537%2F6ffdd459ad28ac35c963e48ac675b5ff%2Ftmp.png?generation=1601629453564955&alt=media\")
This seems really bad to me. Especially that the FVC prediction is increasing that strongly for the last weeks.
", "rawMarkdown": "Interesting. I wasn't able to build a NN that does not completely overfit, but you and @code1110 apparently succeeded. It seems that I still have to learn many things about NNs.\n\nFor example my prediction for the first patient from sub (left for FVC and right for confidence prediction) looks like this:\n\n\nThis seems really bad to me. Especially that the FVC prediction is increasing that strongly for the last weeks.", "votes": null }, { "id": "1034962", "postDate": "10/02/2020 10:38:06", "content": "I may be wrong but it looks like you have a training set that uses only one base week for every patient, since your FVC is spot on for the first week and bad for the others. My training set is built by using all
\nentries from the same patient at starting points to predict the rest of the entries. It looks like this :
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F890965%2F1776713c6ef4dabd101e562a981279a5%2Ftrset.png?generation=1601634961930766&alt=media\")
I am not sure that it's the right way to do it, but I got better results with it.
", "rawMarkdown": "I may be wrong but it looks like you have a training set that uses only one base week for every patient, since your FVC is spot on for the first week and bad for the others. My training set is built by using all \nentries from the same patient at starting points to predict the rest of the entries. It looks like this :\n\n\nI am not sure that it's the right way to do it, but I got better results with it.", "votes": null }, { "id": "1034967", "postDate": "10/02/2020 10:43:39", "content": "Thanks. I had a lot of trouble with the dicom images to build the 3D representation of the scans. THe scan quality is very variable and their metadata is a mess.
", "rawMarkdown": "Thanks. I had a lot of trouble with the dicom images to build the 3D representation of the scans. THe scan quality is very variable and their metadata is a mess.", "votes": null }, { "id": "1034978", "postDate": "10/02/2020 10:58:31", "content": "Nice. I approximately double my training data, by doing this for the second visit of each patient. For some reason however, I didn't do it for the third, fourth and so on visit as you did.
\n'You got better results with it' means in CV or on the LB?
", "rawMarkdown": "Nice. I approximately double my training data, by doing this for the second visit of each patient. For some reason however, I didn't do it for the third, fourth and so on visit as you did. \n\n'You got better results with it' means in CV or on the LB?", "votes": null }, { "id": "1034990", "postDate": "10/02/2020 11:12:55", "content": "Both, at this point I have a good correlation between my CV and the LB. But I don't have a very good score….
", "rawMarkdown": "Both, at this point I have a good correlation between my CV and the LB. But I don't have a very good score....", "votes": null }, { "id": "1034994", "postDate": "10/02/2020 11:15:53", "content": "If your private LB score will be what your public LB score is now, I would place a bet that you get to the top 20.
", "rawMarkdown": "If your private LB score will be what your public LB score is now, I would place a bet that you get to the top 20.", "votes": null } ], "comments": [ { "id": 1034601, "author_name": "code1110", "author_url": "", "post_date": "10/02/2020 00:42:20", "content": "Thanks for your informative share. The lb score from this prediction is -7.2, so take it as a bad example;) Yet my observation shares your view:
\nAll of the predicted curves look very much the same with slight variations of the slope, seems corrected by the base FVC and have a confidence that gently increases with the distance from the base week.\n
Thanks for sharing, you have slightly more complex results than mines. What kind of model did you use ?
", "votes": null, "replies": [] }, { "id": 1034861, "author_name": "code1110", "author_url": "", "post_date": "10/02/2020 08:46:59", "content": "This is from an ensemble of multiple quantile regression models using only tabular data (without percent feature). I wanted to make a 3DCNN model like you did but could not manage;(
", "votes": null, "replies": [] }, { "id": 1034967, "author_name": "filipix", "author_url": "", "post_date": "10/02/2020 10:43:39", "content": "Thanks. I had a lot of trouble with the dicom images to build the 3D representation of the scans. THe scan quality is very variable and their metadata is a mess.
", "votes": null, "replies": [] } ] }, { "id": 1034796, "author_name": "tobiit", "author_url": "", "post_date": "10/02/2020 07:12:59", "content": "Thanks for sharing your results. Do you somehow impose linearity with respect to the weeks feature or does your NN construct comes up with this by itself?
\nAnd just to ensure that your aware of this: Probably, you used the 5 patients from the test folder as well to train your network (or did you explicitly exclude them from training?). So you should expect the model to work less well than in your plots for the private LB.
", "votes": null, "replies": [ { "id": 1034836, "author_name": "filipix", "author_url": "", "post_date": "10/02/2020 08:15:35", "content": "You are welcome. I don't impose anything, it's the result from the NN (averaged on 5 folds for prediction and confidence). And it's not totally linear.
\nI know that the 5 patients are in the training set (thanks), I did a training session by excluding them and I got the same kind of results with a slighly lesser overall score.
", "votes": null, "replies": [] }, { "id": 1034882, "author_name": "tobiit", "author_url": "", "post_date": "10/02/2020 09:06:40", "content": "Interesting. I wasn't able to build a NN that does not completely overfit, but you and @code1110 apparently succeeded. It seems that I still have to learn many things about NNs.
\nFor example my prediction for the first patient from sub (left for FVC and right for confidence prediction) looks like this:
\n
This seems really bad to me. Especially that the FVC prediction is increasing that strongly for the last weeks.
", "votes": null, "replies": [] }, { "id": 1034962, "author_name": "filipix", "author_url": "", "post_date": "10/02/2020 10:38:06", "content": "I may be wrong but it looks like you have a training set that uses only one base week for every patient, since your FVC is spot on for the first week and bad for the others. My training set is built by using all
\nentries from the same patient at starting points to predict the rest of the entries. It looks like this :
\n
I am not sure that it's the right way to do it, but I got better results with it.
", "votes": null, "replies": [] }, { "id": 1034978, "author_name": "tobiit", "author_url": "", "post_date": "10/02/2020 10:58:31", "content": "Nice. I approximately double my training data, by doing this for the second visit of each patient. For some reason however, I didn't do it for the third, fourth and so on visit as you did.
\n'You got better results with it' means in CV or on the LB?
", "votes": null, "replies": [] }, { "id": 1034990, "author_name": "filipix", "author_url": "", "post_date": "10/02/2020 11:12:55", "content": "Both, at this point I have a good correlation between my CV and the LB. But I don't have a very good score….
", "votes": null, "replies": [] }, { "id": 1034994, "author_name": "tobiit", "author_url": "", "post_date": "10/02/2020 11:15:53", "content": "If your private LB score will be what your public LB score is now, I would place a bet that you get to the top 20.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1031682": "Hello,\nI have used a 3D VGG like CNN trained from scratch with the competition images (reduced to 48x48x48) as one input, and the features that nearly everyone is using as another input into a small MLP, then the output of the CNN (around 300 features) is concatenated to the output of the the MLP and then 2 Dense layers and then the pinball loss is used to optimize the whole model.\n\nThis got me to -6.86 on the public test set.\n\nThe images below are what I have when I predict the 5 samples from the submission test set.\n\nI would be interested to know if anyone can share the same kind of results. All of the predicted curves look very much the same with slight variations of the slope, seems corrected by the base FVC and have a confidence that gently increases with the distance from the base week. Do you have a different kind of results ?\n\nThanks\n\n\n\n\n\n", "1034601": "Thanks for your informative share. The lb score from this prediction is -7.2, so take it as a bad example;) Yet my observation shares your view: \n\n```\nAll of the predicted curves look very much the same with slight variations of the slope, seems corrected by the base FVC and have a confidence that gently increases with the distance from the base week.\n```\n\n", "1034796": "Thanks for sharing your results. Do you somehow impose linearity with respect to the weeks feature or does your NN construct comes up with this by itself?\n\nAnd just to ensure that your aware of this: Probably, you used the 5 patients from the test folder as well to train your network (or did you explicitly exclude them from training?). So you should expect the model to work less well than in your plots for the private LB.", "1034836": "You are welcome. I don't impose anything, it's the result from the NN (averaged on 5 folds for prediction and confidence). And it's not totally linear.\n\nI know that the 5 patients are in the training set (thanks), I did a training session by excluding them and I got the same kind of results with a slighly lesser overall score.", "1034838": "Thanks for sharing, you have slightly more complex results than mines. What kind of model did you use ?", "1034861": "This is from an ensemble of multiple quantile regression models using only tabular data (without percent feature). I wanted to make a 3DCNN model like you did but could not manage;(", "1034882": "Interesting. I wasn't able to build a NN that does not completely overfit, but you and @code1110 apparently succeeded. It seems that I still have to learn many things about NNs.\n\nFor example my prediction for the first patient from sub (left for FVC and right for confidence prediction) looks like this:\n\n\nThis seems really bad to me. Especially that the FVC prediction is increasing that strongly for the last weeks.", "1034962": "I may be wrong but it looks like you have a training set that uses only one base week for every patient, since your FVC is spot on for the first week and bad for the others. My training set is built by using all \nentries from the same patient at starting points to predict the rest of the entries. It looks like this :\n\n\nI am not sure that it's the right way to do it, but I got better results with it.", "1034967": "Thanks. I had a lot of trouble with the dicom images to build the 3D representation of the scans. THe scan quality is very variable and their metadata is a mess.", "1034978": "Nice. I approximately double my training data, by doing this for the second visit of each patient. For some reason however, I didn't do it for the third, fourth and so on visit as you did. \n\n'You got better results with it' means in CV or on the LB?", "1034990": "Both, at this point I have a good correlation between my CV and the LB. But I don't have a very good score....", "1034994": "If your private LB score will be what your public LB score is now, I would place a bet that you get to the top 20." }, "source": "meta" }