\n
valid_inputs = [all_inputs[i] for i in range(0, len(all_inputs), 8)] \n
train_inputs = [f for f in all_inputs if not f in valid_inputs] \nCV/LB correlation is close to linear for me.
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2126325%2Fc43402605550c92b95535e142b27b04b%2FCV_LB.png?generation=1746708569089070&alt=media\")
| Dataset | \nCV | \nWeight | \nCV Weight | \n
|---|---|---|---|
| FlatVel_A | \n13.62 | \n6.38% | \n0.87 | \n
| FlatFault_A | \n16.99 | \n11.49% | \n1.95 | \n
| FlatVel_B | \n38.82 | \n6.38% | \n2.48 | \n
| CurveFault_A | \n25.84 | \n11.49% | \n2.97 | \n
| CurveVel_A | \n56.9 | \n6.38% | \n3.63 | \n
| CurveVel_B | \n132.38 | \n6.38% | \n8.45 | \n
| Style_A | \n77.39 | \n14.26% | \n11.04 | \n
| Style_B | \n85.86 | \n14.26% | \n12.24 | \n
| FlatFault_B | \n108.89 | \n11.49% | \n12.51 | \n
| CurveFault_B | \n178.53 | \n11.49% | \n20.51 | \n
| \n | CV => 76.65 | \nLB => 87.9 | \n\n |
| Dataset | \nCV | \nWeight | \nCV Weight | \n
|---|---|---|---|
| FlatVel_A | \n13.62 | \n6.38% | \n0.87 | \n
| FlatFault_A | \n16.99 | \n11.49% | \n1.95 | \n
| FlatVel_B | \n38.82 | \n6.38% | \n2.48 | \n
| CurveFault_A | \n25.84 | \n11.49% | \n2.97 | \n
| CurveVel_A | \n56.9 | \n6.38% | \n3.63 | \n
| CurveVel_B | \n132.38 | \n6.38% | \n8.45 | \n
| Style_A | \n77.39 | \n14.26% | \n11.04 | \n
| Style_B | \n85.86 | \n14.26% | \n12.24 | \n
| FlatFault_B | \n108.89 | \n11.49% | \n12.51 | \n
| CurveFault_B | \n178.53 | \n11.49% | \n20.51 | \n
| \n | CV => 76.65 | \nLB => 87.9 | \n\n |
I use every 8th file for the validation, i.e.
\nvalid_inputs = [all_inputs[i] for i in range(0, len(all_inputs), 8)]
\ntrain_inputs = [f for f in all_inputs if not f in valid_inputs]
\nCV/LB correlation is close to linear for me.
\n
Thanks for sharing your stable CV strategy @egortrushin. I will also will try.
\n\n", "rawMarkdown": "Thanks for sharing your stable CV strategy @egortrushin. I will also will try.\n\n> @egortrushin could you share the # of input files per dataset in the cv?" } ] }, { "id": 3200494, "postDate": "2025-05-12T16:48:00.270Z", "content": "@egortrushin could you share the # of input files per dataset in the cv?
\n
single model: CV: 42.73 LB: 45.1 stratified split
Folder | Loss\n----------------------\nCurveFault_A | \nCurveVel_A | \nFlatFault_A | \nFlatVel_A | \nStyle_A | \nCurveFault_B | \nCurveVel_B | \nFlatFault_B | \nFlatVel_B | \nStyle_B | \nmay i ask which model you used?
", "rawMarkdown": "may i ask which model you used?", "isDeleted": true } ] }, { "id": 3197559, "postDate": "2025-05-08T09:34:14.830Z", "content": "Thank you for sharing the information! I have a quick question. Since the training dataset is quite large, using a large k(like 6+) is computationally difficult for me. If you're using something like k-fold CV, I’d really appreciate it if you could share any strategies or workarounds you’ve found helpful.
", "rawMarkdown": "Thank you for sharing the information! I have a quick question. Since the training dataset is quite large, using a large k(like 6+) is computationally difficult for me. If you're using something like k-fold CV, I’d really appreciate it if you could share any strategies or workarounds you’ve found helpful.", "votes": 1, "replies": [ { "id": 3197598, "postDate": "2025-05-08T10:40:33.567Z", "content": "Hi @haruiig, i trained till now individual datasets and used cv as author suggested cv split.
", "rawMarkdown": "Hi @haruiig, i trained till now individual datasets and used cv as author suggested cv split. " } ] }, { "id": 3214990, "postDate": "2025-06-01T11:45:01.087Z", "content": "\n \n\nBug in the code, ignore below results
\n
LB: 41.9
\nCV: 6.77384
| \n | dataset | \nmean_l1_loss | \n
|---|---|---|
| 6 | \nFlatVel_A | \n9.80455 | \n
| 4 | \nFlatFault_A | \n11.2967 | \n
| 0 | \nCurveFault_A | \n11.9547 | \n
| 2 | \nCurveVel_A | \n12.8543 | \n
| 7 | \nFlatVel_B | \n12.9903 | \n
| 8 | \nStyle_A | \n15.6347 | \n
| 5 | \nFlatFault_B | \n18.0968 | \n
| 9 | \nStyle_B | \n19.0094 | \n
| 3 | \nCurveVel_B | \n24.8558 | \n
| 1 | \nCurveFault_B | \n34.3181 | \n
LB: 57.4
\nCV: 17.00776
| \n | dataset | \nmean_l1_loss | \n
|---|---|---|
| 6 | \nFlatVel_A | \n1.73058 | \n
| 4 | \nFlatFault_A | \n2.13168 | \n
| 0 | \nCurveFault_A | \n2.51763 | \n
| 2 | \nCurveVel_A | \n3.38419 | \n
| 7 | \nFlatVel_B | \n3.86001 | \n
| 5 | \nFlatFault_B | \n7.36532 | \n
| 9 | \nStyle_B | \n8.74178 | \n
| 8 | \nStyle_A | \n9.96799 | \n
| 3 | \nCurveVel_B | \n11.4148 | \n
| 1 | \nCurveFault_B | \n16.6244 | \n
It might not be just overfitting… I do not fit a model to hit 6.x on train and it comes nowhere near that score for validation.
\nThere might be some mistakes or leaks where your CV is 6.x or 17.x, because from my experiments, there is no world where you can get Style_A below 20 letalone below 10, when your FlatVel_A has not even broken 1.0 (comparitively and realisticly easy to break below 1)
\nThe LB vs CV should be very stable otherwise.
", "rawMarkdown": "It might not be just overfitting... I do not fit a model to hit 6.x on train and it comes nowhere near that score for validation.\n\nThere might be some mistakes or leaks where your CV is 6.x or 17.x, because from my experiments, there is no world where you can get Style_A below 20 letalone below 10, when your FlatVel_A has not even broken 1.0 (comparitively and realisticly easy to break below 1)\n\nThe LB vs CV should be very stable otherwise.", "votes": 1, "replies": [ { "id": 3215017, "postDate": "2025-06-01T12:59:44.327Z", "content": "Thanks, i will check with Leakage in my code.
", "rawMarkdown": "Thanks, i will check with Leakage in my code." } ] } ] } ], "comments": [ { "id": 3197677, "author_name": "Egor Trushin", "author_url": "", "post_date": "2025-05-08T12:49:56.207000", "content": "I use every 8th file for the validation, i.e.
\nvalid_inputs = [all_inputs[i] for i in range(0, len(all_inputs), 8)]
\ntrain_inputs = [f for f in all_inputs if not f in valid_inputs]
\nCV/LB correlation is close to linear for me.
\n
Thanks for sharing your stable CV strategy @egortrushin. I will also will try.
\n\n", "votes": 0, "replies": [] } ] }, { "id": 3200494, "author_name": "DrHB", "author_url": "", "post_date": "2025-05-12T16:48:00.270000", "content": "@egortrushin could you share the # of input files per dataset in the cv?
\n
single model: CV: 42.73 LB: 45.1 stratified split
Folder | Loss\n----------------------\nCurveFault_A | \nCurveVel_A | \nFlatFault_A | \nFlatVel_A | \nStyle_A | \nCurveFault_B | \nCurveVel_B | \nFlatFault_B | \nFlatVel_B | \nStyle_B | \nmay i ask which model you used?
", "votes": 0, "replies": [] } ] }, { "id": 3197559, "author_name": "Harui-ig", "author_url": "", "post_date": "2025-05-08T09:34:14.830000", "content": "Thank you for sharing the information! I have a quick question. Since the training dataset is quite large, using a large k(like 6+) is computationally difficult for me. If you're using something like k-fold CV, I’d really appreciate it if you could share any strategies or workarounds you’ve found helpful.
", "votes": 1, "replies": [ { "id": 3197598, "author_name": "SeshuRaju 🧘♂️", "author_url": "", "post_date": "2025-05-08T10:40:33.567000", "content": "Hi @haruiig, i trained till now individual datasets and used cv as author suggested cv split.
", "votes": 0, "replies": [] } ] }, { "id": 3214990, "author_name": "SeshuRaju 🧘♂️", "author_url": "", "post_date": "2025-06-01T11:45:01.087000", "content": "\n \n\nBug in the code, ignore below results
\n
LB: 41.9
\nCV: 6.77384
| \n | dataset | \nmean_l1_loss | \n
|---|---|---|
| 6 | \nFlatVel_A | \n9.80455 | \n
| 4 | \nFlatFault_A | \n11.2967 | \n
| 0 | \nCurveFault_A | \n11.9547 | \n
| 2 | \nCurveVel_A | \n12.8543 | \n
| 7 | \nFlatVel_B | \n12.9903 | \n
| 8 | \nStyle_A | \n15.6347 | \n
| 5 | \nFlatFault_B | \n18.0968 | \n
| 9 | \nStyle_B | \n19.0094 | \n
| 3 | \nCurveVel_B | \n24.8558 | \n
| 1 | \nCurveFault_B | \n34.3181 | \n
LB: 57.4
\nCV: 17.00776
| \n | dataset | \nmean_l1_loss | \n
|---|---|---|
| 6 | \nFlatVel_A | \n1.73058 | \n
| 4 | \nFlatFault_A | \n2.13168 | \n
| 0 | \nCurveFault_A | \n2.51763 | \n
| 2 | \nCurveVel_A | \n3.38419 | \n
| 7 | \nFlatVel_B | \n3.86001 | \n
| 5 | \nFlatFault_B | \n7.36532 | \n
| 9 | \nStyle_B | \n8.74178 | \n
| 8 | \nStyle_A | \n9.96799 | \n
| 3 | \nCurveVel_B | \n11.4148 | \n
| 1 | \nCurveFault_B | \n16.6244 | \n
It might not be just overfitting… I do not fit a model to hit 6.x on train and it comes nowhere near that score for validation.
\nThere might be some mistakes or leaks where your CV is 6.x or 17.x, because from my experiments, there is no world where you can get Style_A below 20 letalone below 10, when your FlatVel_A has not even broken 1.0 (comparitively and realisticly easy to break below 1)
\nThe LB vs CV should be very stable otherwise.
", "votes": 1, "replies": [ { "id": 3215017, "author_name": "SeshuRaju 🧘♂️", "author_url": "", "post_date": "2025-06-01T12:59:44.327000", "content": "Thanks, i will check with Leakage in my code.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3191561": "| Dataset | CV | Weight | CV Weight |\n|:-------------|-------:|---------:|------------:|\n| FlatVel_A | 13.62 | 6.38% | 0.87 |\n| FlatFault_A | 16.99 | 11.49% | 1.95 |\n| FlatVel_B | 38.82 | 6.38% | 2.48 |\n| CurveFault_A | 25.84 | 11.49% | 2.97 |\n| CurveVel_A | 56.9 | 6.38% | 3.63 |\n| CurveVel_B | 132.38 | 6.38% | 8.45 |\n| Style_A | 77.39 | 14.26% | 11.04 |\n| Style_B | 85.86 | 14.26% | 12.24 |\n| FlatFault_B | 108.89 | 11.49% | 12.51 |\n| CurveFault_B | 178.53 | 11.49% | 20.51 |\n| | **CV => 76.65** | **LB => 87.9** | |\n\n## weights based on dataset distribution", "3197677": "I use every 8th file for the validation, i.e. \n`valid_inputs = [all_inputs[i] for i in range(0, len(all_inputs), 8)]` \n`train_inputs = [f for f in all_inputs if not f in valid_inputs]` \nCV/LB correlation is close to linear for me.\n", "3200494": "single model: CV: `42.73` LB: `45.1` stratified split\n```python\nFolder | Loss\n----------------------\nCurveFault_A | 11.3136\nCurveVel_A | 23.8266\nFlatFault_A | 7.2014\nFlatVel_A | 4.2391\nStyle_A | 42.0223\nCurveFault_B | 116.1918\nCurveVel_B | 70.8718\nFlatFault_B | 50.1331\nFlatVel_B | 15.6909\nStyle_B | 57.4357\n```", "3197559": "Thank you for sharing the information! I have a quick question. Since the training dataset is quite large, using a large k(like 6+) is computationally difficult for me. If you're using something like k-fold CV, I’d really appreciate it if you could share any strategies or workarounds you’ve found helpful.", "3214990": "> ~~My overfitting cv and train -- but no luck with LB~~\nBug in the code, ignore below results\n---\n\n**LB: 41.9**\n**CV: 6.77384**\n| | dataset | mean_l1_loss |\n|---:|:-------------|---------------:|\n| 6 | FlatVel_A | 9.80455 |\n| 4 | FlatFault_A | 11.2967 |\n| 0 | CurveFault_A | 11.9547 |\n| 2 | CurveVel_A | 12.8543 |\n| 7 | FlatVel_B | 12.9903 |\n| 8 | Style_A | 15.6347 |\n| 5 | FlatFault_B | 18.0968 |\n| 9 | Style_B | 19.0094 |\n| 3 | CurveVel_B | 24.8558 |\n| 1 | CurveFault_B | 34.3181 |\n\n---\n\n**LB: 57.4**\n**CV: 17.00776**\n\n| | dataset | mean_l1_loss |\n|---:|:-------------|---------------:|\n| 6 | FlatVel_A | 1.73058 |\n| 4 | FlatFault_A | 2.13168 |\n| 0 | CurveFault_A | 2.51763 |\n| 2 | CurveVel_A | 3.38419 |\n| 7 | FlatVel_B | 3.86001 |\n| 5 | FlatFault_B | 7.36532 |\n| 9 | Style_B | 8.74178 |\n| 8 | Style_A | 9.96799 |\n| 3 | CurveVel_B | 11.4148 |\n| 1 | CurveFault_B | 16.6244 |~~\n" } }