Here's mine - a few weeks ago I was contacted by a member of The Zoo with an offer of teaming up. I was flattered but politely declined since I wanted to challenge myself and see how well I could do on my own. And now I know the answer - a silver medal. But if only...!
\n\nCongratulations to the winning team, I look forward to seeing your solution!
", "messages": [ { "id": 542462, "postDate": "2019-06-04T00:12:37.760Z", "content": "Easy: if only we selected the best submission, we would now be 2nd ;)
", "rawMarkdown": "Easy: if only we selected the best submission, we would now be 2nd ;)", "votes": 7, "replies": [ { "id": 542469, "postDate": "2019-06-04T00:16:23.777Z", "content": "Funny, I have the same story ;)
", "rawMarkdown": "Funny, I have the same story ;)\n", "votes": 3 }, { "id": 542475, "postDate": "2019-06-04T00:19:37.107Z", "content": "If only I stopped working after my first test submission....... :D\nNot second, but top 30
", "rawMarkdown": "If only I stopped working after my first test submission....... :D\nNot second, but top 30", "votes": 7 }, { "id": 542483, "postDate": "2019-06-04T00:23:15.363Z", "content": "my 2nd and 3rd submission are my best.. i should have stayed away..lol
", "rawMarkdown": "my 2nd and 3rd submission are my best.. i should have stayed away..lol", "votes": 2 }, { "id": 542484, "postDate": "2019-06-04T00:24:35.427Z", "content": "Private LB mean was 6.669 wow
", "rawMarkdown": "Private LB mean was 6.669 wow", "votes": 3 }, { "id": 542493, "postDate": "2019-06-04T00:26:55.407Z", "content": "There are some submissions at the bottom where the error is so large, I think they got confused about what they were supposed to do (or were just messing around). I feel sorry for the team that was in 21st and is now 3691st - that's why you don't overfit the LB!
", "rawMarkdown": "There are some submissions at the bottom where the error is so large, I think they got confused about what they were supposed to do (or were just messing around). I feel sorry for the team that was in 21st and is now 3691st - that's why you don't overfit the LB!", "votes": 2 }, { "id": 542494, "postDate": "2019-06-04T00:27:22.490Z", "content": "Shows you how easily mean can be skewed......
", "rawMarkdown": "Shows you how easily mean can be skewed......", "votes": 1 }, { "id": 542549, "postDate": "2019-06-04T01:20:36.343Z", "content": "According the paper the mean of the testset is around 6.3 ~ 6.5.
\n\nTaking into account the mean of Public is < 5 makes sense that the mean of Private > 6.5
", "rawMarkdown": "According the paper the mean of the testset is around 6.3 ~ 6.5.\n\nTaking into account the mean of Public is < 5 makes sense that the mean of Private > 6.5", "votes": 2 } ] }, { "id": 542459, "postDate": "2019-06-04T00:10:50.027Z", "content": "Here's mine - a few weeks ago I was contacted by a member of The Zoo with an offer of teaming up. I was flattered but politely declined since I wanted to challenge myself and see how well I could do on my own. And now I know the answer - a silver medal. But if only...!
\n\nCongratulations to the winning team, I look forward to seeing your solution!
", "rawMarkdown": "Here's mine - a few weeks ago I was contacted by a member of The Zoo with an offer of teaming up. I was flattered but politely declined since I wanted to challenge myself and see how well I could do on my own. And now I know the answer - a silver medal. But if only...!\n\nCongratulations to the winning team, I look forward to seeing your solution!", "votes": 7 }, { "id": 542542, "postDate": "2019-06-04T01:11:30.840Z", "content": "One of my best solution:
\n\nmodel = RandomForestRegressor(max_depth=500, random_state=0, n_estimators=500, n_jobs=6)\nmodel.fit(train_features, train_target)
\ntest_target = model.predict(test_features)
That's all !!
\n\nno parameter optimization, no CV, learned directly from whole train set (32 features used)
\n\nPrivate Score: 2.45256
\n\nCompared to that, my final solution has about 1000 lines of code (combination of 3-fold CV, 5-fold CV, 16-fold CV, a lot of classifiers, stacking, etc.)
\n\n... and I lost ~ 1000 positions on LB with it :)))
\n\nSo, I probably don't understand the meaning of the word \"overfitting\". :)
", "rawMarkdown": "One of my best solution:\n\nmodel = RandomForestRegressor(max_depth=500, random_state=0, n_estimators=500, n_jobs=6)\nmodel.fit(train_features, train_target) \ntest_target = model.predict(test_features)\n\n\nThat's all !!\n\nno parameter optimization, no CV, learned directly from whole train set (32 features used)\n\nPrivate Score: 2.45256\n\n-----------------------\n\nCompared to that, my final solution has about 1000 lines of code (combination of 3-fold CV, 5-fold CV, 16-fold CV, a lot of classifiers, stacking, etc.)\n\n... and I lost ~ 1000 positions on LB with it :)))\n\nSo, I probably don't understand the meaning of the word \"overfitting\". :)", "votes": 6 }, { "id": 542566, "postDate": "2019-06-04T01:37:44.313Z", "content": "Had I not joined VSB, I would not have survived this shake up. I lost my medal there for choosing an overly complicated model. This time round, one submission is reserved for simplicity with decent CV, and it happens to be my third best.
", "rawMarkdown": "Had I not joined VSB, I would not have survived this shake up. I lost my medal there for choosing an overly complicated model. This time round, one submission is reserved for simplicity with decent CV, and it happens to be my third best. ", "votes": 3 }, { "id": 542490, "postDate": "2019-06-04T00:26:32.320Z", "content": "A model I did a month ago would have gotten a gold. I need to write better comments on my submission because I'm not even sure what I did. My public score went up 0.1 so I went in a different direction. oh well. It was fun.
", "rawMarkdown": "A model I did a month ago would have gotten a gold. I need to write better comments on my submission because I'm not even sure what I did. My public score went up 0.1 so I went in a different direction. oh well. It was fun.", "votes": 3 }, { "id": 542991, "postDate": "2019-06-04T08:56:16.340Z", "content": "If only I haven't spend 9000 hours on this competition trying to come up with something good, I would be even higher on LB than I am now :)
", "rawMarkdown": "If only I haven't spend 9000 hours on this competition trying to come up with something good, I would be even higher on LB than I am now :)", "votes": 1 }, { "id": 542527, "postDate": "2019-06-04T00:55:30.563Z", "content": "Honestly I'm really excited we didn't drop further. My \"if only\" is that I wish I had studied the images from the published papers more 😄
", "rawMarkdown": "Honestly I'm really excited we didn't drop further. My \"if only\" is that I wish I had studied the images from the published papers more 😄 ", "votes": 1 }, { "id": 542464, "postDate": "2019-06-04T00:14:37.110Z", "content": "If only I submitted a weighted trained submission file that is adapted to my estimation of private LB (which I didn't), I would end up at the last gold position. I didn't want to be adventurous, and it turns out that I regret it a little...
", "rawMarkdown": "If only I submitted a weighted trained submission file that is adapted to my estimation of private LB (which I didn't), I would end up at the last gold position. I didn't want to be adventurous, and it turns out that I regret it a little...", "votes": 1 }, { "id": 542675, "postDate": "2019-06-04T03:23:09.517Z", "content": "BTW congrats buddy!!
", "rawMarkdown": "BTW congrats buddy!! ", "votes": 2 }, { "id": 542473, "postDate": "2019-06-04T00:17:54.993Z", "content": "The model that made most sense to me scored 1.6 on public LB, so I gave up on it, but I would have gotten a silver on top 100 :(
\n\nI wish there was something that they could do to not mislead so many people with huge differences on private/public LB. Lesson learned: Trust your gut.
", "rawMarkdown": "The model that made most sense to me scored 1.6 on public LB, so I gave up on it, but I would have gotten a silver on top 100 :(\n\nI wish there was something that they could do to not mislead so many people with huge differences on private/public LB. Lesson learned: Trust your gut.", "votes": 2, "replies": [ { "id": 542480, "postDate": "2019-06-04T00:21:22.043Z", "content": "The huge difference between private and public LB was obvious to all of us from the beginning. It is on us that couldn't resist temptations of going up on public rankings :D
\n\nit's ok though. next time ;)
", "rawMarkdown": "The huge difference between private and public LB was obvious to all of us from the beginning. It is on us that couldn't resist temptations of going up on public rankings :D \n\nit's ok though. next time ;)" } ] }, { "id": 542488, "postDate": "2019-06-04T00:25:35.063Z", "content": "Turns out private test was high TTF. I made a submission that overfit on only high TTF EQs that turned out to be my best private test 2.36 and public 2.30 . No way to know. Trained on [1,2,4,7,10,11,14], EQ 1 starts the full EQ cycle, EQ 0 was throw into validation set.
", "rawMarkdown": "Turns out private test was high TTF. I made a submission that overfit on only high TTF EQs that turned out to be my best private test 2.36 and public 2.30 . No way to know. Trained on [1,2,4,7,10,11,14], EQ 1 starts the full EQ cycle, EQ 0 was throw into validation set.", "votes": 1 }, { "id": 542546, "postDate": "2019-06-04T01:17:18.810Z", "content": "My conservative selection scored ~top150 and the \"risk\" selection top11 ;)\nIf I had changed only 1 single parameter of my risk sub it was placed in the money :_(
", "rawMarkdown": "My conservative selection scored ~top150 and the \"risk\" selection top11 ;)\nIf I had changed only 1 single parameter of my risk sub it was placed in the money :_(\n", "votes": -1, "replies": [ { "id": 542561, "postDate": "2019-06-04T01:29:31.493Z", "content": "And if you had changed 2 single parameters you would fall into LB hell :)
", "rawMarkdown": "And if you had changed 2 single parameters you would fall into LB hell :)", "votes": 4 }, { "id": 542562, "postDate": "2019-06-04T01:30:12.517Z", "content": "and congratulations to 11th place !!
", "rawMarkdown": "and congratulations to 11th place !!", "votes": 1 }, { "id": 542598, "postDate": "2019-06-04T01:56:11.770Z", "content": "Changing more parameters we would have won ;)
", "rawMarkdown": "Changing more parameters we would have won ;)", "votes": -1 } ] }, { "id": 542523, "postDate": "2019-06-04T00:50:53.847Z", "content": "https://www.screencast.com/t/Xz6bbOlz7gr just other cv scheme could work better
", "rawMarkdown": "https://www.screencast.com/t/Xz6bbOlz7gr just other cv scheme could work better" }, { "id": 542501, "postDate": "2019-06-04T00:30:39.590Z", "content": "My final submissions were my best lb model trained on shuffled cv and a neural net that i stitched together in half a day for fun. If i submitted a lgb model with a cv that i trusted i'd be placed ~20-100, gg wp xD\nGuess next time i'll choose models more carefully for final submissions.
\n\nAnyway, it was a great learning experience with lots of interesting ideas and feedback, thanks for that, Congratz to the winners and we'll meet again in future competitions :)
", "rawMarkdown": "My final submissions were my best lb model trained on shuffled cv and a neural net that i stitched together in half a day for fun. If i submitted a lgb model with a cv that i trusted i'd be placed ~20-100, gg wp xD\nGuess next time i'll choose models more carefully for final submissions. \n\nAnyway, it was a great learning experience with lots of interesting ideas and feedback, thanks for that, Congratz to the winners and we'll meet again in future competitions :)" }, { "id": 542491, "postDate": "2019-06-04T00:26:46.327Z", "content": "If only I have not doubted the new sklearn's VotingRegressor instead of mlxtend's StackingRegressor it would have been very different! Selecting the final submission was my nightmare and a last minute change led to 800 drop for me!
", "rawMarkdown": "If only I have not doubted the new sklearn's VotingRegressor instead of mlxtend's StackingRegressor it would have been very different! Selecting the final submission was my nightmare and a last minute change led to 800 drop for me!" }, { "id": 542976, "postDate": "2019-06-04T08:48:42.900Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 542462, "author_name": "bluetrain", "author_url": "", "post_date": "2019-06-04T00:12:37.760000", "content": "Easy: if only we selected the best submission, we would now be 2nd ;)
", "votes": 7, "replies": [ { "id": 542469, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-04T00:16:23.777000", "content": "Funny, I have the same story ;)
", "votes": 3, "replies": [] }, { "id": 542475, "author_name": "Jelmer", "author_url": "", "post_date": "2019-06-04T00:19:37.107000", "content": "If only I stopped working after my first test submission....... :D\nNot second, but top 30
", "votes": 7, "replies": [] }, { "id": 542483, "author_name": "Holar", "author_url": "", "post_date": "2019-06-04T00:23:15.363000", "content": "my 2nd and 3rd submission are my best.. i should have stayed away..lol
", "votes": 2, "replies": [] }, { "id": 542484, "author_name": "Psi", "author_url": "", "post_date": "2019-06-04T00:24:35.427000", "content": "Private LB mean was 6.669 wow
", "votes": 3, "replies": [] }, { "id": 542493, "author_name": "RNA", "author_url": "", "post_date": "2019-06-04T00:26:55.407000", "content": "There are some submissions at the bottom where the error is so large, I think they got confused about what they were supposed to do (or were just messing around). I feel sorry for the team that was in 21st and is now 3691st - that's why you don't overfit the LB!
", "votes": 2, "replies": [] }, { "id": 542494, "author_name": "Jelmer", "author_url": "", "post_date": "2019-06-04T00:27:22.490000", "content": "Shows you how easily mean can be skewed......
", "votes": 1, "replies": [] }, { "id": 542549, "author_name": "Giba", "author_url": "", "post_date": "2019-06-04T01:20:36.343000", "content": "According the paper the mean of the testset is around 6.3 ~ 6.5.
\n\nTaking into account the mean of Public is < 5 makes sense that the mean of Private > 6.5
", "votes": 2, "replies": [] } ] }, { "id": 542542, "author_name": "Dre@mer", "author_url": "", "post_date": "2019-06-04T01:11:30.840000", "content": "One of my best solution:
\n\nmodel = RandomForestRegressor(max_depth=500, random_state=0, n_estimators=500, n_jobs=6)\nmodel.fit(train_features, train_target)
\ntest_target = model.predict(test_features)
That's all !!
\n\nno parameter optimization, no CV, learned directly from whole train set (32 features used)
\n\nPrivate Score: 2.45256
\n\nCompared to that, my final solution has about 1000 lines of code (combination of 3-fold CV, 5-fold CV, 16-fold CV, a lot of classifiers, stacking, etc.)
\n\n... and I lost ~ 1000 positions on LB with it :)))
\n\nSo, I probably don't understand the meaning of the word \"overfitting\". :)
", "votes": 6, "replies": [] }, { "id": 542566, "author_name": "yukiya", "author_url": "", "post_date": "2019-06-04T01:37:44.313000", "content": "Had I not joined VSB, I would not have survived this shake up. I lost my medal there for choosing an overly complicated model. This time round, one submission is reserved for simplicity with decent CV, and it happens to be my third best.
", "votes": 3, "replies": [] }, { "id": 542490, "author_name": "Eric Freeman", "author_url": "", "post_date": "2019-06-04T00:26:32.320000", "content": "A model I did a month ago would have gotten a gold. I need to write better comments on my submission because I'm not even sure what I did. My public score went up 0.1 so I went in a different direction. oh well. It was fun.
", "votes": 3, "replies": [] }, { "id": 542991, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-06-04T08:56:16.340000", "content": "If only I haven't spend 9000 hours on this competition trying to come up with something good, I would be even higher on LB than I am now :)
", "votes": 1, "replies": [] }, { "id": 542527, "author_name": "Rob Mulla", "author_url": "", "post_date": "2019-06-04T00:55:30.563000", "content": "Honestly I'm really excited we didn't drop further. My \"if only\" is that I wish I had studied the images from the published papers more 😄
", "votes": 1, "replies": [] }, { "id": 542464, "author_name": "Kha Vo", "author_url": "", "post_date": "2019-06-04T00:14:37.110000", "content": "If only I submitted a weighted trained submission file that is adapted to my estimation of private LB (which I didn't), I would end up at the last gold position. I didn't want to be adventurous, and it turns out that I regret it a little...
", "votes": 1, "replies": [] }, { "id": 542675, "author_name": "Himanshu Soni", "author_url": "", "post_date": "2019-06-04T03:23:09.517000", "content": "BTW congrats buddy!!
", "votes": 2, "replies": [] }, { "id": 542473, "author_name": "Felipe Mello", "author_url": "", "post_date": "2019-06-04T00:17:54.993000", "content": "The model that made most sense to me scored 1.6 on public LB, so I gave up on it, but I would have gotten a silver on top 100 :(
\n\nI wish there was something that they could do to not mislead so many people with huge differences on private/public LB. Lesson learned: Trust your gut.
", "votes": 2, "replies": [ { "id": 542480, "author_name": "Massoud Hosseinali", "author_url": "", "post_date": "2019-06-04T00:21:22.043000", "content": "The huge difference between private and public LB was obvious to all of us from the beginning. It is on us that couldn't resist temptations of going up on public rankings :D
\n\nit's ok though. next time ;)
", "votes": 0, "replies": [] } ] }, { "id": 542488, "author_name": "Tim Yee", "author_url": "", "post_date": "2019-06-04T00:25:35.063000", "content": "Turns out private test was high TTF. I made a submission that overfit on only high TTF EQs that turned out to be my best private test 2.36 and public 2.30 . No way to know. Trained on [1,2,4,7,10,11,14], EQ 1 starts the full EQ cycle, EQ 0 was throw into validation set.
", "votes": 1, "replies": [] }, { "id": 542546, "author_name": "Giba", "author_url": "", "post_date": "2019-06-04T01:17:18.810000", "content": "My conservative selection scored ~top150 and the \"risk\" selection top11 ;)\nIf I had changed only 1 single parameter of my risk sub it was placed in the money :_(
", "votes": -1, "replies": [ { "id": 542561, "author_name": "Dre@mer", "author_url": "", "post_date": "2019-06-04T01:29:31.493000", "content": "And if you had changed 2 single parameters you would fall into LB hell :)
", "votes": 4, "replies": [] }, { "id": 542562, "author_name": "Dre@mer", "author_url": "", "post_date": "2019-06-04T01:30:12.517000", "content": "and congratulations to 11th place !!
", "votes": 1, "replies": [] }, { "id": 542598, "author_name": "Giba", "author_url": "", "post_date": "2019-06-04T01:56:11.770000", "content": "Changing more parameters we would have won ;)
", "votes": -1, "replies": [] } ] }, { "id": 542523, "author_name": "Artem Kuznetsov", "author_url": "", "post_date": "2019-06-04T00:50:53.847000", "content": "https://www.screencast.com/t/Xz6bbOlz7gr just other cv scheme could work better
", "votes": 0, "replies": [] }, { "id": 542501, "author_name": "Sven Hinderer", "author_url": "", "post_date": "2019-06-04T00:30:39.590000", "content": "My final submissions were my best lb model trained on shuffled cv and a neural net that i stitched together in half a day for fun. If i submitted a lgb model with a cv that i trusted i'd be placed ~20-100, gg wp xD\nGuess next time i'll choose models more carefully for final submissions.
\n\nAnyway, it was a great learning experience with lots of interesting ideas and feedback, thanks for that, Congratz to the winners and we'll meet again in future competitions :)
", "votes": 0, "replies": [] }, { "id": 542491, "author_name": "Sia", "author_url": "", "post_date": "2019-06-04T00:26:46.327000", "content": "If only I have not doubted the new sklearn's VotingRegressor instead of mlxtend's StackingRegressor it would have been very different! Selecting the final submission was my nightmare and a last minute change led to 800 drop for me!
", "votes": 0, "replies": [] }, { "id": 542976, "author_name": "", "author_url": "", "post_date": "2019-06-04T08:48:42.900000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "542462": "Easy: if only we selected the best submission, we would now be 2nd ;)", "542459": "Here's mine - a few weeks ago I was contacted by a member of The Zoo with an offer of teaming up. I was flattered but politely declined since I wanted to challenge myself and see how well I could do on my own. And now I know the answer - a silver medal. But if only...!\n\nCongratulations to the winning team, I look forward to seeing your solution!", "542542": "One of my best solution:\n\nmodel = RandomForestRegressor(max_depth=500, random_state=0, n_estimators=500, n_jobs=6)\nmodel.fit(train_features, train_target) \ntest_target = model.predict(test_features)\n\n\nThat's all !!\n\nno parameter optimization, no CV, learned directly from whole train set (32 features used)\n\nPrivate Score: 2.45256\n\n-----------------------\n\nCompared to that, my final solution has about 1000 lines of code (combination of 3-fold CV, 5-fold CV, 16-fold CV, a lot of classifiers, stacking, etc.)\n\n... and I lost ~ 1000 positions on LB with it :)))\n\nSo, I probably don't understand the meaning of the word \"overfitting\". :)", "542566": "Had I not joined VSB, I would not have survived this shake up. I lost my medal there for choosing an overly complicated model. This time round, one submission is reserved for simplicity with decent CV, and it happens to be my third best. ", "542490": "A model I did a month ago would have gotten a gold. I need to write better comments on my submission because I'm not even sure what I did. My public score went up 0.1 so I went in a different direction. oh well. It was fun.", "542991": "If only I haven't spend 9000 hours on this competition trying to come up with something good, I would be even higher on LB than I am now :)", "542527": "Honestly I'm really excited we didn't drop further. My \"if only\" is that I wish I had studied the images from the published papers more 😄 ", "542464": "If only I submitted a weighted trained submission file that is adapted to my estimation of private LB (which I didn't), I would end up at the last gold position. I didn't want to be adventurous, and it turns out that I regret it a little...", "542675": "BTW congrats buddy!! ", "542473": "The model that made most sense to me scored 1.6 on public LB, so I gave up on it, but I would have gotten a silver on top 100 :(\n\nI wish there was something that they could do to not mislead so many people with huge differences on private/public LB. Lesson learned: Trust your gut.", "542488": "Turns out private test was high TTF. I made a submission that overfit on only high TTF EQs that turned out to be my best private test 2.36 and public 2.30 . No way to know. Trained on [1,2,4,7,10,11,14], EQ 1 starts the full EQ cycle, EQ 0 was throw into validation set.", "542546": "My conservative selection scored ~top150 and the \"risk\" selection top11 ;)\nIf I had changed only 1 single parameter of my risk sub it was placed in the money :_(\n", "542523": "https://www.screencast.com/t/Xz6bbOlz7gr just other cv scheme could work better", "542501": "My final submissions were my best lb model trained on shuffled cv and a neural net that i stitched together in half a day for fun. If i submitted a lgb model with a cv that i trusted i'd be placed ~20-100, gg wp xD\nGuess next time i'll choose models more carefully for final submissions. \n\nAnyway, it was a great learning experience with lots of interesting ideas and feedback, thanks for that, Congratz to the winners and we'll meet again in future competitions :)", "542491": "If only I have not doubted the new sklearn's VotingRegressor instead of mlxtend's StackingRegressor it would have been very different! Selecting the final submission was my nightmare and a last minute change led to 800 drop for me!", "542976": "" } }