Given the possibility of three-submissions It will be interesting to see the shakeup in this competition.\nWhat do you think?
", "messages": [ { "id": "269067", "postDate": "01/16/2018 06:06:09", "content": "Given the possibility of three-submissions It will be interesting to see the shakeup in this competition.\nWhat do you think?
", "rawMarkdown": "Given the possibility of three-submissions It will be interesting to see the shakeup in this competition.\nWhat do you think?", "votes": null }, { "id": "269208", "postDate": "01/16/2018 13:08:28", "content": "I think so too :-)
", "rawMarkdown": "I think so too :-)", "votes": null }, { "id": "269334", "postDate": "01/16/2018 16:43:13", "content": "Data are quite noisy and teams are very close to each other. Relatively moderate shakeup is expected to be.
", "rawMarkdown": "Data are quite noisy and teams are very close to each other. Relatively moderate shakeup is expected to be.", "votes": null }, { "id": "269343", "postDate": "01/16/2018 16:47:06", "content": "I'm actually counting on a shakeup. More like, I've staked a lot on a shakeup, which might seem like a stupid thing to do I admit, but it was just intuition. I might not have learned much from prev DL competitions, but I did learn one thing: the leaderboard can be a very good liar at times. A lot of it can be attributed to the fact that the private lb/public lb split is 70/30. But beyond that, it's the presence of \"unknowns\". That, combined with a 70% private lb can very well make this competition turn into something like one of those 2-stage competitions. Who knows what demons hide in there.
\n\nBut it's apparent that I am hit by the LB paranoia bug so either way, I think I'll end up learning much more than all the people who grab them silvers and golds :D :D.
", "rawMarkdown": "I'm actually counting on a shakeup. More like, I've staked a lot on a shakeup, which might seem like a stupid thing to do I admit, but it was just intuition. I might not have learned much from prev DL competitions, but I did learn one thing: the leaderboard can be a very good liar at times. A lot of it can be attributed to the fact that the private lb/public lb split is 70/30. But beyond that, it's the presence of \"unknowns\". That, combined with a 70% private lb can very well make this competition turn into something like one of those 2-stage competitions. Who knows what demons hide in there.\n\nBut it's apparent that I am hit by the LB paranoia bug so either way, I think I'll end up learning much more than all the people who grab them silvers and golds :D :D.", "votes": null }, { "id": "269365", "postDate": "01/16/2018 17:37:31", "content": "I'm afraid that my score in LB is overfiting, so I expect a moderate shakeup tonight.
", "rawMarkdown": "I'm afraid that my score in LB is overfiting, so I expect a moderate shakeup tonight.", "votes": null }, { "id": "269369", "postDate": "01/16/2018 17:43:13", "content": "How do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...
", "rawMarkdown": "How do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...", "votes": null }, { "id": "269371", "postDate": "01/16/2018 17:46:43", "content": "Relatively moderate shakeup will be there i am sure.
", "rawMarkdown": "Relatively moderate shakeup will be there i am sure.", "votes": null }, { "id": "269372", "postDate": "01/16/2018 17:47:21", "content": "All the best!
", "rawMarkdown": "All the best!", "votes": null }, { "id": "269382", "postDate": "01/16/2018 18:15:09", "content": "What validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!
", "rawMarkdown": "What validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!", "votes": null }, { "id": "269388", "postDate": "01/16/2018 18:20:56", "content": "normal validation/test scores somehow translate to the public LB if your model is deep and big enough:\n0.96-0.97 to 87% LB, >0.97 to 88% LB (DPN92 was exception and scored only 87%)
", "rawMarkdown": "normal validation/test scores somehow translate to the public LB if your model is deep and big enough:\n0.96-0.97 to 87% LB, >0.97 to 88% LB (DPN92 was exception and scored only 87%)", "votes": null }, { "id": "269395", "postDate": "01/16/2018 18:40:26", "content": "Yeah, I'm seeing something similar, but the exceptions make me think that something else is going on... We'll see soon enough. Good luck!
", "rawMarkdown": "Yeah, I'm seeing something similar, but the exceptions make me think that something else is going on... We'll see soon enough. Good luck!", "votes": null }, { "id": "269398", "postDate": "01/16/2018 18:51:59", "content": "\n\n\nfergusoci wrote
\n \n\n\nWhat validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!
\n
I just splitted the data using user_id. I had correlation with the LB but a very big gap between them. My models have a cv score of 0.99 vs the lb score of 0.89.
", "rawMarkdown": "> **fergusoci wrote**\n> \n> > What validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!\n\nI just splitted the data using user_id. I had correlation with the LB but a very big gap between them. My models have a cv score of 0.99 vs the lb score of 0.89.", "votes": null }, { "id": "269399", "postDate": "01/16/2018 18:52:28", "content": "\n\n\ntugstugi wrote
\n \n\n\nHow do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...
\n
It's just intuition. All the things that I tried in the last week didn't improve the score. So I believe I had a lucky submission and that it won't hold on the private LB.
", "rawMarkdown": "> **tugstugi wrote**\n> \n> > How do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...\n\nIt's just intuition. All the things that I tried in the last week didn't improve the score. So I believe I had a lucky submission and that it won't hold on the private LB.", "votes": null } ], "comments": [ { "id": 269208, "author_name": "youngguy131", "author_url": "", "post_date": "01/16/2018 13:08:28", "content": "I think so too :-)
", "votes": null, "replies": [] }, { "id": 269334, "author_name": "shvetsiya", "author_url": "", "post_date": "01/16/2018 16:43:13", "content": "Data are quite noisy and teams are very close to each other. Relatively moderate shakeup is expected to be.
", "votes": null, "replies": [ { "id": 269371, "author_name": "outliar", "author_url": "", "post_date": "01/16/2018 17:46:43", "content": "Relatively moderate shakeup will be there i am sure.
", "votes": null, "replies": [] } ] }, { "id": 269343, "author_name": "yadavsarthak", "author_url": "", "post_date": "01/16/2018 16:47:06", "content": "I'm actually counting on a shakeup. More like, I've staked a lot on a shakeup, which might seem like a stupid thing to do I admit, but it was just intuition. I might not have learned much from prev DL competitions, but I did learn one thing: the leaderboard can be a very good liar at times. A lot of it can be attributed to the fact that the private lb/public lb split is 70/30. But beyond that, it's the presence of \"unknowns\". That, combined with a 70% private lb can very well make this competition turn into something like one of those 2-stage competitions. Who knows what demons hide in there.
\n\nBut it's apparent that I am hit by the LB paranoia bug so either way, I think I'll end up learning much more than all the people who grab them silvers and golds :D :D.
", "votes": null, "replies": [ { "id": 269372, "author_name": "outliar", "author_url": "", "post_date": "01/16/2018 17:47:21", "content": "All the best!
", "votes": null, "replies": [] } ] }, { "id": 269365, "author_name": "ironbar", "author_url": "", "post_date": "01/16/2018 17:37:31", "content": "I'm afraid that my score in LB is overfiting, so I expect a moderate shakeup tonight.
", "votes": null, "replies": [ { "id": 269369, "author_name": "tugstugi", "author_url": "", "post_date": "01/16/2018 17:43:13", "content": "How do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...
", "votes": null, "replies": [] }, { "id": 269382, "author_name": "fergusoci", "author_url": "", "post_date": "01/16/2018 18:15:09", "content": "What validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!
", "votes": null, "replies": [] }, { "id": 269388, "author_name": "tugstugi", "author_url": "", "post_date": "01/16/2018 18:20:56", "content": "normal validation/test scores somehow translate to the public LB if your model is deep and big enough:\n0.96-0.97 to 87% LB, >0.97 to 88% LB (DPN92 was exception and scored only 87%)
", "votes": null, "replies": [] }, { "id": 269395, "author_name": "fergusoci", "author_url": "", "post_date": "01/16/2018 18:40:26", "content": "Yeah, I'm seeing something similar, but the exceptions make me think that something else is going on... We'll see soon enough. Good luck!
", "votes": null, "replies": [] }, { "id": 269398, "author_name": "ironbar", "author_url": "", "post_date": "01/16/2018 18:51:59", "content": "\n\n\nfergusoci wrote
\n \n\n\nWhat validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!
\n
I just splitted the data using user_id. I had correlation with the LB but a very big gap between them. My models have a cv score of 0.99 vs the lb score of 0.89.
", "votes": null, "replies": [] }, { "id": 269399, "author_name": "ironbar", "author_url": "", "post_date": "01/16/2018 18:52:28", "content": "\n\n\ntugstugi wrote
\n \n\n\nHow do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...
\n
It's just intuition. All the things that I tried in the last week didn't improve the score. So I believe I had a lucky submission and that it won't hold on the private LB.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "269067": "Given the possibility of three-submissions It will be interesting to see the shakeup in this competition.\nWhat do you think?", "269208": "I think so too :-)", "269334": "Data are quite noisy and teams are very close to each other. Relatively moderate shakeup is expected to be.", "269343": "I'm actually counting on a shakeup. More like, I've staked a lot on a shakeup, which might seem like a stupid thing to do I admit, but it was just intuition. I might not have learned much from prev DL competitions, but I did learn one thing: the leaderboard can be a very good liar at times. A lot of it can be attributed to the fact that the private lb/public lb split is 70/30. But beyond that, it's the presence of \"unknowns\". That, combined with a 70% private lb can very well make this competition turn into something like one of those 2-stage competitions. Who knows what demons hide in there.\n\nBut it's apparent that I am hit by the LB paranoia bug so either way, I think I'll end up learning much more than all the people who grab them silvers and golds :D :D.", "269365": "I'm afraid that my score in LB is overfiting, so I expect a moderate shakeup tonight.", "269369": "How do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...", "269371": "Relatively moderate shakeup will be there i am sure.", "269372": "All the best!", "269382": "What validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!", "269388": "normal validation/test scores somehow translate to the public LB if your model is deep and big enough:\n0.96-0.97 to 87% LB, >0.97 to 88% LB (DPN92 was exception and scored only 87%)", "269395": "Yeah, I'm seeing something similar, but the exceptions make me think that something else is going on... We'll see soon enough. Good luck!", "269398": "> **fergusoci wrote**\n> \n> > What validation scheme did you come up with to match the leaderboard? I tried several but couldn't get anything that matched it... Which makes me think I'm going to be part of the shakeup, and that I won't likely benefit from it!\n\nI just splitted the data using user_id. I had correlation with the LB but a very big gap between them. My models have a cv score of 0.99 vs the lb score of 0.89.", "269399": "> **tugstugi wrote**\n> \n> > How do you know that you are overfiting the public LB? Until now, all our validation accuracy matched somehow the public LB scores. So I am hoping...\n\nIt's just intuition. All the things that I tried in the last week didn't improve the score. So I believe I had a lucky submission and that it won't hold on the private LB." }, "source": "meta" }