![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18538441%2Fd8444bbcb835cc99b73291dda9d13cb2%2FWechatIMG13130.jpg?generation=1734944346161013&alt=media\")
I tried to validate on TS_6_4 like public discussion, but failed to get a good correlation between CV and LB. I even got 0.818 locally with bad lb scores, and it's overfitting obviously.
\nOnce I tried following the local score to improve my pipeline, but it went to a dead end.
\nAny suggestions? many thanks.
\n
I tried to validate on TS_6_4 like public discussion, but failed to get a good correlation between CV and LB. I even got 0.818 locally with bad lb scores, and it's overfitting obviously.
\nOnce I tried following the local score to improve my pipeline, but it went to a dead end.
\nAny suggestions? many thanks.
\n
Hi, I think I might have seen your id on xhs before haha.
\nBack to the topic, I suggest you can try running local cross validations and calculate the mean&std. For example, train on six data points and val on the rest one, do this 7 times until you get mean cv score for all the runs. If you observe a significant boost in local cv score, that might be a progress on lb.
\n\nHi, I think I might have seen your id on xhs before haha.
\n
Yeah it's me hahaha
\n\n\nBack to the topic, I suggest you can try running local cross validations and calculate the mean&std. For example, train on six data points and val on the rest one, do this 7 times until you get mean cv score for all the runs. If you observe a significant boost in local cv score, that might be a progress on lb.
\n
Thank you, I just tried only one-fold validation, and I'll do it later.
", "rawMarkdown": ">Hi, I think I might have seen your id on xhs before haha.\n\nYeah it's me hahaha\n\n>Back to the topic, I suggest you can try running local cross validations and calculate the mean&std. For example, train on six data points and val on the rest one, do this 7 times until you get mean cv score for all the runs. If you observe a significant boost in local cv score, that might be a progress on lb.\n\nThank you, I just tried only one-fold validation, and I'll do it later.", "votes": null }, { "id": "3080302", "postDate": "12/25/2024 01:44:32", "content": "I tried an emsemble of 7 folds, and cv is quite good while lb is not. Maybe I should improve one-fold model then blend 7 folds.
", "rawMarkdown": "I tried an emsemble of 7 folds, and cv is quite good while lb is not. Maybe I should improve one-fold model then blend 7 folds.", "votes": null }, { "id": "3082361", "postDate": "12/28/2024 03:47:59", "content": "I think there might be a little bit of different understanding here. 🫣
\nNormally I don't go straight up for 7 fold emsembles on LB submissions. My local 7 fold validations are to verify if any changes would make a good impact on the overall score. If any change does that, I will try further experiments and submit on LB.
\nBut with tta on, I never make 7 fold emsembles becase my notebook will run out of time haha.
Thank you, I'll think about that.
", "rawMarkdown": "Thank you, I'll think about that.", "votes": null } ], "comments": [ { "id": 3079193, "author_name": "yksinyoung", "author_url": "", "post_date": "12/23/2024 10:39:42", "content": "Hi, I think I might have seen your id on xhs before haha.
\nBack to the topic, I suggest you can try running local cross validations and calculate the mean&std. For example, train on six data points and val on the rest one, do this 7 times until you get mean cv score for all the runs. If you observe a significant boost in local cv score, that might be a progress on lb.
\n\nHi, I think I might have seen your id on xhs before haha.
\n
Yeah it's me hahaha
\n\n\nBack to the topic, I suggest you can try running local cross validations and calculate the mean&std. For example, train on six data points and val on the rest one, do this 7 times until you get mean cv score for all the runs. If you observe a significant boost in local cv score, that might be a progress on lb.
\n
Thank you, I just tried only one-fold validation, and I'll do it later.
", "votes": null, "replies": [] }, { "id": 3080302, "author_name": "sweetyheehee", "author_url": "", "post_date": "12/25/2024 01:44:32", "content": "I tried an emsemble of 7 folds, and cv is quite good while lb is not. Maybe I should improve one-fold model then blend 7 folds.
", "votes": null, "replies": [ { "id": 3082361, "author_name": "yksinyoung", "author_url": "", "post_date": "12/28/2024 03:47:59", "content": "I think there might be a little bit of different understanding here. 🫣
\nNormally I don't go straight up for 7 fold emsembles on LB submissions. My local 7 fold validations are to verify if any changes would make a good impact on the overall score. If any change does that, I will try further experiments and submit on LB.
\nBut with tta on, I never make 7 fold emsembles becase my notebook will run out of time haha.
Thank you, I'll think about that.
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "3079115": "I tried to validate on TS_6_4 like public discussion, but failed to get a good correlation between CV and LB. I even got 0.818 locally with bad lb scores, and it's overfitting obviously.\n\nOnce I tried following the local score to improve my pipeline, but it went to a dead end.\n \nAny suggestions? many thanks.\n\n", "3079193": "Hi, I think I might have seen your id on xhs before haha.\nBack to the topic, I suggest you can try running local cross validations and calculate the mean&std. For example, train on six data points and val on the rest one, do this 7 times until you get mean cv score for all the runs. If you observe a significant boost in local cv score, that might be a progress on lb.", "3079208": ">Hi, I think I might have seen your id on xhs before haha.\n\nYeah it's me hahaha\n\n>Back to the topic, I suggest you can try running local cross validations and calculate the mean&std. For example, train on six data points and val on the rest one, do this 7 times until you get mean cv score for all the runs. If you observe a significant boost in local cv score, that might be a progress on lb.\n\nThank you, I just tried only one-fold validation, and I'll do it later.", "3080302": "I tried an emsemble of 7 folds, and cv is quite good while lb is not. Maybe I should improve one-fold model then blend 7 folds.", "3082361": "I think there might be a little bit of different understanding here. 🫣\nNormally I don't go straight up for 7 fold emsembles on LB submissions. My local 7 fold validations are to verify if any changes would make a good impact on the overall score. If any change does that, I will try further experiments and submit on LB. \nBut with tta on, I never make 7 fold emsembles becase my notebook will run out of time haha.", "3082466": "Thank you, I'll think about that." }, "source": "meta" }