The kernel reruns for Stage 2 have been completed, the Leaderboard has been finalized.
\n\nFor those who are wondering, the drop in teams was mostly due to teams not selecting submissions, and thus not moving into Stage 2. For the Kernels that were selected, over 95% of them ran successfully. The most common reasons for kernels failing reruns were exceeding the time constraint or not correctly writing a submission file. Because selected kernels were re-run on the entire dataset (Public and Private), model variability may have caused different public scores for those selections during the re-run.
\n\nMedals and points are awarded based on the total teams that participated in Stage 1.
\n\nWe are looking into the any irregularities that are reported in the forums.
", "messages": [ { "id": 471177, "postDate": "2019-02-14T05:50:46.747Z", "content": "The kernel reruns for Stage 2 have been completed, the Leaderboard has been finalized.
\n\nFor those who are wondering, the drop in teams was mostly due to teams not selecting submissions, and thus not moving into Stage 2. For the Kernels that were selected, over 95% of them ran successfully. The most common reasons for kernels failing reruns were exceeding the time constraint or not correctly writing a submission file. Because selected kernels were re-run on the entire dataset (Public and Private), model variability may have caused different public scores for those selections during the re-run.
\n\nMedals and points are awarded based on the total teams that participated in Stage 1.
\n\nWe are looking into the any irregularities that are reported in the forums.
", "rawMarkdown": "The kernel reruns for Stage 2 have been completed, the Leaderboard has been finalized.\n\nFor those who are wondering, the drop in teams was mostly due to teams not selecting submissions, and thus not moving into Stage 2. For the Kernels that were selected, over 95% of them ran successfully. The most common reasons for kernels failing reruns were exceeding the time constraint or not correctly writing a submission file. Because selected kernels were re-run on the entire dataset (Public and Private), model variability may have caused different public scores for those selections during the re-run.\n\nMedals and points are awarded based on the total teams that participated in Stage 1.\n\nWe are looking into the any irregularities that are reported in the forums.", "votes": 30 }, { "id": 472022, "postDate": "2019-02-15T08:20:10.990Z", "content": "This is my first NLP and kernel based competition on Kaggle. I have learned a lot from public kernels and discussions. And finally got a Bronze Medal. Many thanks again. :)
", "rawMarkdown": "This is my first NLP and kernel based competition on Kaggle. I have learned a lot from public kernels and discussions. And finally got a Bronze Medal. Many thanks again. :)", "votes": 4 }, { "id": 471278, "postDate": "2019-02-14T08:35:08.587Z", "content": "Thank you for the hard work organizing this.
", "rawMarkdown": "Thank you for the hard work organizing this. ", "votes": 1 }, { "id": 471216, "postDate": "2019-02-14T06:40:36.027Z", "content": "This is my first NLP competition on Kaggle. I learned a lot from public kernels and discussions. Thanks, @kaggle...
", "rawMarkdown": "This is my first NLP competition on Kaggle. I learned a lot from public kernels and discussions. Thanks, @kaggle...", "votes": 1 }, { "id": 471346, "postDate": "2019-02-14T10:34:29.720Z", "content": "Maybe send out an email, too, since I just caught this by chance and some others might have missed it :)
\n\nThanks for organizing this great competition! I came in 44st, which is better than I had hoped. Also, the Toulouse Goose didn't make it in the top 1000, so they did overfit the public data with some trick it seems :)
", "rawMarkdown": "Maybe send out an email, too, since I just caught this by chance and some others might have missed it :)\n\nThanks for organizing this great competition! I came in 44st, which is better than I had hoped. Also, the Toulouse Goose didn't make it in the top 1000, so they did overfit the public data with some trick it seems :)", "votes": 2, "replies": [ { "id": 471364, "postDate": "2019-02-14T11:00:33.513Z", "content": "@mschumacher, congrats and you are right about overfitting. When I read about some people making a case that choosing the right seed is a legitimate parameter to tune, I thought oh boy! Regarding public LB, I tend to err on the side of caution which occasionally leaves me behind but works most of the time.
", "rawMarkdown": "@mschumacher, congrats and you are right about overfitting. When I read about some people making a case that choosing the right seed is a legitimate parameter to tune, I thought oh boy! Regarding public LB, I tend to err on the side of caution which occasionally leaves me behind but works most of the time.", "votes": 1 }, { "id": 471535, "postDate": "2019-02-14T15:27:52.067Z", "content": "Congrat both of you again!
\n\nJust want to mention that even though Toulouse Goose did overfit the public LB, I still think that their overfitting method is a rare gem, and would very love to see how they did it.
", "rawMarkdown": "Congrat both of you again!\n\nJust want to mention that even though Toulouse Goose did overfit the public LB, I still think that their overfitting method is a rare gem, and would very love to see how they did it.", "votes": 2 }, { "id": 471576, "postDate": "2019-02-14T16:02:22.490Z", "content": "I agree! I can only imagine they ensembled several BERTs or similar and then hard-coded the predictions in the kernel, though they said there was no Bert involved...
\n\n@sheriytm Congrats to you too!
", "rawMarkdown": "I agree! I can only imagine they ensembled several BERTs or similar and then hard-coded the predictions in the kernel, though they said there was no Bert involved...\n\n@sheriytm Congrats to you too!", "votes": 2 }, { "id": 471577, "postDate": "2019-02-14T16:03:16.293Z", "content": "@ratthachat, I am guessing perhaps something like this https://www.kaggle.com/c/restaurant-revenue-prediction/discussion/13950 Or this one https://www.kaggle.com/hengck23/the-perfect-score-script
\n\nThanks @mschumacher.
", "rawMarkdown": "@ratthachat, I am guessing perhaps something like this https://www.kaggle.com/c/restaurant-revenue-prediction/discussion/13950 Or this one https://www.kaggle.com/hengck23/the-perfect-score-script\n\nThanks @mschumacher.", "votes": 1 } ] }, { "id": 471214, "postDate": "2019-02-14T06:37:55.293Z", "content": "Surprised to see the competition results were already in but pleasantly surprised trusting our CV ended up paying off.
", "rawMarkdown": "Surprised to see the competition results were already in but pleasantly surprised trusting our CV ended up paying off. ", "votes": 2 }, { "id": 471195, "postDate": "2019-02-14T06:15:55.270Z", "content": "Thanks for all your work getting all of these kernels run and the final results posted. I wasn't in the running, but this was my first competition and it was fun. I'm more motivated than ever to get involved in some other competitions!
", "rawMarkdown": "Thanks for all your work getting all of these kernels run and the final results posted. I wasn't in the running, but this was my first competition and it was fun. I'm more motivated than ever to get involved in some other competitions!", "votes": 2, "replies": [ { "id": 471197, "postDate": "2019-02-14T06:18:31.303Z", "content": "Thanks for participating, and I hope you learned a lot from the experience! :-)
", "rawMarkdown": "Thanks for participating, and I hope you learned a lot from the experience! :-)", "votes": 3 } ] }, { "id": 542918, "postDate": "2019-06-04T07:43:00.947Z", "content": "If my submission page says - invalidated for stage 2 rerun, does that mean that my submission was not selected for stage 2 or does it mean that it was selected and failed to run successfully?
", "rawMarkdown": "If my submission page says - invalidated for stage 2 rerun, does that mean that my submission was not selected for stage 2 or does it mean that it was selected and failed to run successfully?" }, { "id": 471813, "postDate": "2019-02-14T23:10:30.327Z", "content": "I worked quite hard in the last 2 weeks and did not select the submissions... i assumed it would pick the best LB. is there a way to run and check my private score? sad story...
", "rawMarkdown": "I worked quite hard in the last 2 weeks and did not select the submissions... i assumed it would pick the best LB. is there a way to run and check my private score? sad story...", "replies": [ { "id": 471817, "postDate": "2019-02-14T23:29:50.850Z", "content": "I'm sorry to hear that. You're welcome to make a late submission.
", "rawMarkdown": "I'm sorry to hear that. You're welcome to make a late submission." } ] }, { "id": 471375, "postDate": "2019-02-14T11:13:24.603Z", "content": "My both selected kernels show \"Invalidated for stage 2 reruns\". Does it mean that both failed the fixed time limit?
", "rawMarkdown": "My both selected kernels show \"Invalidated for stage 2 reruns\". Does it mean that both failed the fixed time limit? ", "replies": [ { "id": 471429, "postDate": "2019-02-14T12:52:24.727Z", "content": "@ashwinids, it might be that you have something hard coded that would give an error with the change in test data size. I think time limit failure will put you somewhere at the bottom of the LB from my experience in the Mercari Price competition. I may be wrong since this is only my 2nd kernel competition.
", "rawMarkdown": "@ashwinids, it might be that you have something hard coded that would give an error with the change in test data size. I think time limit failure will put you somewhere at the bottom of the LB from my experience in the Mercari Price competition. I may be wrong since this is only my 2nd kernel competition." }, { "id": 471527, "postDate": "2019-02-14T15:21:15.643Z", "rawMarkdown": "", "isDeleted": true }, { "id": 471531, "postDate": "2019-02-14T15:23:52.103Z", "content": "@YaGana Sheriff-Hussaini, I checked my selected kernels. I hardcoded the number of rows in the test dataset. I don't know when and why. Thanks, for suggesting the possible cause of the error.
", "rawMarkdown": "@YaGana Sheriff-Hussaini, I checked my selected kernels. I hardcoded the number of rows in the test dataset. I don't know when and why. Thanks, for suggesting the possible cause of the error.", "votes": 1 }, { "id": 471557, "postDate": "2019-02-14T15:51:23.117Z", "content": "@ashwinids, you are welcome and sorry about your experience. It happens to the best of us, put it down to lesson learned and see you in the next one. Happy kaggling.
", "rawMarkdown": "@ashwinids, you are welcome and sorry about your experience. It happens to the best of us, put it down to lesson learned and see you in the next one. Happy kaggling." } ] }, { "id": 471213, "postDate": "2019-02-14T06:35:50.263Z", "content": "@inversion, thanks for getting this done faster than planned.
\n\nIf the medals and points are awarded based on stage-1 number of teams, I do not understand the 336 points I got. Can you please check my account and let me know if everything is correct? My last competition before this one is the travelling santa - prime path.
", "rawMarkdown": "@inversion, thanks for getting this done faster than planned.\n\nIf the medals and points are awarded based on stage-1 number of teams, I do not understand the 336 points I got. Can you please check my account and let me know if everything is correct? My last competition before this one is the travelling santa - prime path.", "replies": [ { "id": 471656, "postDate": "2019-02-14T18:16:32.443Z", "content": "@sheriytm - We've double-checked points calculations, and they are confirmed correct -- in line with the total number of teams being 4037 (not 1401). I'd recommend re-checking your points and ensuring that the calculation you're doing on expected number of points is also accurate.
", "rawMarkdown": "@sheriytm - We've double-checked points calculations, and they are confirmed correct -- in line with the total number of teams being 4037 (not 1401). I'd recommend re-checking your points and ensuring that the calculation you're doing on expected number of points is also accurate.", "votes": 1 } ] }, { "id": 471212, "postDate": "2019-02-14T06:34:33.680Z", "content": "Thanks your all effort to re-run the kernel. I learned a lot in this competition!
", "rawMarkdown": "Thanks your all effort to re-run the kernel. I learned a lot in this competition!" }, { "id": 471181, "postDate": "2019-02-14T05:57:26.873Z", "content": "This question may be silly. Is public test set is the subset of private test set ?
\n\nNote : I do know that in kaggle competitions that private and public test sets don't have any intersection. Just want to confirm
", "rawMarkdown": "This question may be silly. Is public test set is the subset of private test set ?\n\nNote : I do know that in kaggle competitions that private and public test sets don't have any intersection. Just want to confirm ", "replies": [ { "id": 471186, "postDate": "2019-02-14T06:06:05.427Z", "content": "You are correct. The private test set was completely different data than the public test set. The test set that is currently on the data page is a combination of both the public and private rows.
", "rawMarkdown": "You are correct. The private test set was completely different data than the public test set. The test set that is currently on the data page is a combination of both the public and private rows." }, { "id": 471191, "postDate": "2019-02-14T06:12:04.233Z", "content": "If it is a combination of both private and public then why does the size of test is showing as 376k. It must show 376k(private size) + 56k(public part ) = 432k.
\n\nOr Am I missing something ?
", "rawMarkdown": "If it is a combination of both private and public then why does the size of test is showing as 376k. It must show 376k(private size) + 56k(public part ) = 432k. \n\nOr Am I missing something ?" }, { "id": 471196, "postDate": "2019-02-14T06:17:30.393Z", "content": "The new test set is 376k (total) = 56k (public) + 320k (private)
", "rawMarkdown": "The new test set is 376k (total) = 56k (public) + 320k (private)", "votes": 1 }, { "id": 471277, "postDate": "2019-02-14T08:33:08.230Z", "content": "Thanks for the reply \nSo scoring is done on 320k ?
", "rawMarkdown": "Thanks for the reply \nSo scoring is done on 320k ?", "votes": 1 }, { "id": 471510, "postDate": "2019-02-14T15:06:39.723Z", "content": "Correct. The final rank is based on the score from the private 320k rows.
", "rawMarkdown": "Correct. The final rank is based on the score from the private 320k rows." } ] }, { "id": 472085, "postDate": "2019-02-15T10:08:16.940Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 472022, "author_name": "Qing Liu", "author_url": "", "post_date": "2019-02-15T08:20:10.990000", "content": "This is my first NLP and kernel based competition on Kaggle. I have learned a lot from public kernels and discussions. And finally got a Bronze Medal. Many thanks again. :)
", "votes": 4, "replies": [] }, { "id": 471278, "author_name": "yukiya", "author_url": "", "post_date": "2019-02-14T08:35:08.587000", "content": "Thank you for the hard work organizing this.
", "votes": 1, "replies": [] }, { "id": 471216, "author_name": "Karthik Chowdary Tsaliki", "author_url": "", "post_date": "2019-02-14T06:40:36.027000", "content": "This is my first NLP competition on Kaggle. I learned a lot from public kernels and discussions. Thanks, @kaggle...
", "votes": 1, "replies": [] }, { "id": 471346, "author_name": "Max Schumacher", "author_url": "", "post_date": "2019-02-14T10:34:29.720000", "content": "Maybe send out an email, too, since I just caught this by chance and some others might have missed it :)
\n\nThanks for organizing this great competition! I came in 44st, which is better than I had hoped. Also, the Toulouse Goose didn't make it in the top 1000, so they did overfit the public data with some trick it seems :)
", "votes": 2, "replies": [ { "id": 471364, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-14T11:00:33.513000", "content": "@mschumacher, congrats and you are right about overfitting. When I read about some people making a case that choosing the right seed is a legitimate parameter to tune, I thought oh boy! Regarding public LB, I tend to err on the side of caution which occasionally leaves me behind but works most of the time.
", "votes": 1, "replies": [] }, { "id": 471535, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2019-02-14T15:27:52.067000", "content": "Congrat both of you again!
\n\nJust want to mention that even though Toulouse Goose did overfit the public LB, I still think that their overfitting method is a rare gem, and would very love to see how they did it.
", "votes": 2, "replies": [] }, { "id": 471576, "author_name": "Max Schumacher", "author_url": "", "post_date": "2019-02-14T16:02:22.490000", "content": "I agree! I can only imagine they ensembled several BERTs or similar and then hard-coded the predictions in the kernel, though they said there was no Bert involved...
\n\n@sheriytm Congrats to you too!
", "votes": 2, "replies": [] }, { "id": 471577, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-14T16:03:16.293000", "content": "@ratthachat, I am guessing perhaps something like this https://www.kaggle.com/c/restaurant-revenue-prediction/discussion/13950 Or this one https://www.kaggle.com/hengck23/the-perfect-score-script
\n\nThanks @mschumacher.
", "votes": 1, "replies": [] } ] }, { "id": 471214, "author_name": "ryches", "author_url": "", "post_date": "2019-02-14T06:37:55.293000", "content": "Surprised to see the competition results were already in but pleasantly surprised trusting our CV ended up paying off.
", "votes": 2, "replies": [] }, { "id": 471195, "author_name": "Ryan Roper", "author_url": "", "post_date": "2019-02-14T06:15:55.270000", "content": "Thanks for all your work getting all of these kernels run and the final results posted. I wasn't in the running, but this was my first competition and it was fun. I'm more motivated than ever to get involved in some other competitions!
", "votes": 2, "replies": [ { "id": 471197, "author_name": "inversion", "author_url": "", "post_date": "2019-02-14T06:18:31.303000", "content": "Thanks for participating, and I hope you learned a lot from the experience! :-)
", "votes": 3, "replies": [] } ] }, { "id": 542918, "author_name": "Ramya C", "author_url": "", "post_date": "2019-06-04T07:43:00.947000", "content": "If my submission page says - invalidated for stage 2 rerun, does that mean that my submission was not selected for stage 2 or does it mean that it was selected and failed to run successfully?
", "votes": 0, "replies": [] }, { "id": 471813, "author_name": "Shubin", "author_url": "", "post_date": "2019-02-14T23:10:30.327000", "content": "I worked quite hard in the last 2 weeks and did not select the submissions... i assumed it would pick the best LB. is there a way to run and check my private score? sad story...
", "votes": 0, "replies": [ { "id": 471817, "author_name": "Julia Elliott", "author_url": "", "post_date": "2019-02-14T23:29:50.850000", "content": "I'm sorry to hear that. You're welcome to make a late submission.
", "votes": 0, "replies": [] } ] }, { "id": 471375, "author_name": "vishleshak", "author_url": "", "post_date": "2019-02-14T11:13:24.603000", "content": "My both selected kernels show \"Invalidated for stage 2 reruns\". Does it mean that both failed the fixed time limit?
", "votes": 0, "replies": [ { "id": 471429, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-14T12:52:24.727000", "content": "@ashwinids, it might be that you have something hard coded that would give an error with the change in test data size. I think time limit failure will put you somewhere at the bottom of the LB from my experience in the Mercari Price competition. I may be wrong since this is only my 2nd kernel competition.
", "votes": 0, "replies": [] }, { "id": 471527, "author_name": "", "author_url": "", "post_date": "2019-02-14T15:21:15.643000", "content": "", "votes": 0, "replies": [] }, { "id": 471531, "author_name": "vishleshak", "author_url": "", "post_date": "2019-02-14T15:23:52.103000", "content": "@YaGana Sheriff-Hussaini, I checked my selected kernels. I hardcoded the number of rows in the test dataset. I don't know when and why. Thanks, for suggesting the possible cause of the error.
", "votes": 1, "replies": [] }, { "id": 471557, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-14T15:51:23.117000", "content": "@ashwinids, you are welcome and sorry about your experience. It happens to the best of us, put it down to lesson learned and see you in the next one. Happy kaggling.
", "votes": 0, "replies": [] } ] }, { "id": 471213, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-02-14T06:35:50.263000", "content": "@inversion, thanks for getting this done faster than planned.
\n\nIf the medals and points are awarded based on stage-1 number of teams, I do not understand the 336 points I got. Can you please check my account and let me know if everything is correct? My last competition before this one is the travelling santa - prime path.
", "votes": 0, "replies": [ { "id": 471656, "author_name": "Julia Elliott", "author_url": "", "post_date": "2019-02-14T18:16:32.443000", "content": "@sheriytm - We've double-checked points calculations, and they are confirmed correct -- in line with the total number of teams being 4037 (not 1401). I'd recommend re-checking your points and ensuring that the calculation you're doing on expected number of points is also accurate.
", "votes": 1, "replies": [] } ] }, { "id": 471212, "author_name": "0xFunky", "author_url": "", "post_date": "2019-02-14T06:34:33.680000", "content": "Thanks your all effort to re-run the kernel. I learned a lot in this competition!
", "votes": 0, "replies": [] }, { "id": 471181, "author_name": "redshellspy", "author_url": "", "post_date": "2019-02-14T05:57:26.873000", "content": "This question may be silly. Is public test set is the subset of private test set ?
\n\nNote : I do know that in kaggle competitions that private and public test sets don't have any intersection. Just want to confirm
", "votes": 0, "replies": [ { "id": 471186, "author_name": "inversion", "author_url": "", "post_date": "2019-02-14T06:06:05.427000", "content": "You are correct. The private test set was completely different data than the public test set. The test set that is currently on the data page is a combination of both the public and private rows.
", "votes": 0, "replies": [] }, { "id": 471191, "author_name": "redshellspy", "author_url": "", "post_date": "2019-02-14T06:12:04.233000", "content": "If it is a combination of both private and public then why does the size of test is showing as 376k. It must show 376k(private size) + 56k(public part ) = 432k.
\n\nOr Am I missing something ?
", "votes": 0, "replies": [] }, { "id": 471196, "author_name": "inversion", "author_url": "", "post_date": "2019-02-14T06:17:30.393000", "content": "The new test set is 376k (total) = 56k (public) + 320k (private)
", "votes": 1, "replies": [] }, { "id": 471277, "author_name": "redshellspy", "author_url": "", "post_date": "2019-02-14T08:33:08.230000", "content": "Thanks for the reply \nSo scoring is done on 320k ?
", "votes": 1, "replies": [] }, { "id": 471510, "author_name": "inversion", "author_url": "", "post_date": "2019-02-14T15:06:39.723000", "content": "Correct. The final rank is based on the score from the private 320k rows.
", "votes": 0, "replies": [] } ] }, { "id": 472085, "author_name": "", "author_url": "", "post_date": "2019-02-15T10:08:16.940000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "471177": "The kernel reruns for Stage 2 have been completed, the Leaderboard has been finalized.\n\nFor those who are wondering, the drop in teams was mostly due to teams not selecting submissions, and thus not moving into Stage 2. For the Kernels that were selected, over 95% of them ran successfully. The most common reasons for kernels failing reruns were exceeding the time constraint or not correctly writing a submission file. Because selected kernels were re-run on the entire dataset (Public and Private), model variability may have caused different public scores for those selections during the re-run.\n\nMedals and points are awarded based on the total teams that participated in Stage 1.\n\nWe are looking into the any irregularities that are reported in the forums.", "472022": "This is my first NLP and kernel based competition on Kaggle. I have learned a lot from public kernels and discussions. And finally got a Bronze Medal. Many thanks again. :)", "471278": "Thank you for the hard work organizing this. ", "471216": "This is my first NLP competition on Kaggle. I learned a lot from public kernels and discussions. Thanks, @kaggle...", "471346": "Maybe send out an email, too, since I just caught this by chance and some others might have missed it :)\n\nThanks for organizing this great competition! I came in 44st, which is better than I had hoped. Also, the Toulouse Goose didn't make it in the top 1000, so they did overfit the public data with some trick it seems :)", "471214": "Surprised to see the competition results were already in but pleasantly surprised trusting our CV ended up paying off. ", "471195": "Thanks for all your work getting all of these kernels run and the final results posted. I wasn't in the running, but this was my first competition and it was fun. I'm more motivated than ever to get involved in some other competitions!", "542918": "If my submission page says - invalidated for stage 2 rerun, does that mean that my submission was not selected for stage 2 or does it mean that it was selected and failed to run successfully?", "471813": "I worked quite hard in the last 2 weeks and did not select the submissions... i assumed it would pick the best LB. is there a way to run and check my private score? sad story...", "471375": "My both selected kernels show \"Invalidated for stage 2 reruns\". Does it mean that both failed the fixed time limit? ", "471213": "@inversion, thanks for getting this done faster than planned.\n\nIf the medals and points are awarded based on stage-1 number of teams, I do not understand the 336 points I got. Can you please check my account and let me know if everything is correct? My last competition before this one is the travelling santa - prime path.", "471212": "Thanks your all effort to re-run the kernel. I learned a lot in this competition!", "471181": "This question may be silly. Is public test set is the subset of private test set ?\n\nNote : I do know that in kaggle competitions that private and public test sets don't have any intersection. Just want to confirm ", "472085": "" } }