As some users have noted, there was a four hour window yesterday where newly-created notebooks were defaulted to Public, creating unintentional sharing. (See this forum post for additional details.)
\nThis bug impacted a small number of available public notebooks during the last few hours of the G2Net competition. Unfortunately, there is no clean way to rewind the impact without also creating secondary disruptions. We have carefully considered alternatives and believe they are equally or more disruptive to a fair competition outcome. Therefore, we will proceed with the G2Net leaderboard as it currently stands.
\nKaggle's goal has always been to provide competitions that are as fair as possible, recognizing that there are sometimes data, platform, and human issues that complicate this mission. We regret the issues this bug has caused in the final hours of what was otherwise an interesting and challenging competition and hope that those affected will continue to see Kaggle as a place to be challenged and to hone skills alongside a community of people who are so willing to share and help others succeed.
\nThe Kaggle Team
", "messages": [ { "id": "1530068", "postDate": "09/30/2021 22:40:50", "content": "As some users have noted, there was a four hour window yesterday where newly-created notebooks were defaulted to Public, creating unintentional sharing. (See this forum post for additional details.)
\nThis bug impacted a small number of available public notebooks during the last few hours of the G2Net competition. Unfortunately, there is no clean way to rewind the impact without also creating secondary disruptions. We have carefully considered alternatives and believe they are equally or more disruptive to a fair competition outcome. Therefore, we will proceed with the G2Net leaderboard as it currently stands.
\nKaggle's goal has always been to provide competitions that are as fair as possible, recognizing that there are sometimes data, platform, and human issues that complicate this mission. We regret the issues this bug has caused in the final hours of what was otherwise an interesting and challenging competition and hope that those affected will continue to see Kaggle as a place to be challenged and to hone skills alongside a community of people who are so willing to share and help others succeed.
\nThe Kaggle Team
", "rawMarkdown": "As some users have noted, there was a four hour window yesterday where newly-created notebooks were defaulted to Public, creating unintentional sharing. (See this [forum post](https://www.kaggle.com/general/275572) for additional details.)\n\nThis bug impacted a small number of available public notebooks during the last few hours of the G2Net competition. Unfortunately, there is no clean way to rewind the impact without also creating secondary disruptions. We have carefully considered alternatives and believe they are equally or more disruptive to a fair competition outcome. Therefore, we will proceed with the G2Net leaderboard as it currently stands.\n\nKaggle's goal has always been to provide competitions that are as fair as possible, recognizing that there are sometimes data, platform, and human issues that complicate this mission. We regret the issues this bug has caused in the final hours of what was otherwise an interesting and challenging competition and hope that those affected will continue to see Kaggle as a place to be challenged and to hone skills alongside a community of people who are so willing to share and help others succeed.\n\nThe Kaggle Team", "votes": null }, { "id": "1530077", "postDate": "09/30/2021 22:51:02", "content": "sorry, i'm not good at English.
\nDoes it means that private leaderboad fixed?
@yamanity - Yes, it means that the preliminary standings are confirmed. Congratulations on your silver medal.
", "rawMarkdown": "yamanity - Yes, it means that the preliminary standings are confirmed. Congratulations on your silver medal.", "votes": null }, { "id": "1530084", "postDate": "09/30/2021 23:06:27", "content": "thank you.I got it.arigato!
", "rawMarkdown": "thank you.I got it.arigato!", "votes": null }, { "id": "1530365", "postDate": "10/01/2021 05:58:33", "content": "Dear Kaggle Team,
\nI'm sorry, but doing nothing at all is the wrong signal. I can understand that doing a hard reset of the leaderboard to the time point when the bug occurred would cause damage to several teams that may also have not used the high scoring public notebook. But allowing this bug to influence the leaderboard is completely unfair. Only a small group of people were able to use this notebook but people like me that already had to go to sleep had no chance to even use this notebook at all. There was no chance to react!
\nBesides this bug another damaging error was that computational resources were frozen so only participants with external cloud computing resources were able to even do further computations for predictions that could have an influence on the final submissions.
\nWith doing nothing you just showed that blending public notebooks seems to be more important than doing computations on your own. Really sad. In my opinion the discussion forum and the notebooks should be generally closed or frozen a week before a competition ends.
\n:-(
", "rawMarkdown": "Dear Kaggle Team,\n\nI'm sorry, but doing nothing at all is the wrong signal. I can understand that doing a hard reset of the leaderboard to the time point when the bug occurred would cause damage to several teams that may also have not used the high scoring public notebook. But allowing this bug to influence the leaderboard is completely unfair. Only a small group of people were able to use this notebook but people like me that already had to go to sleep had no chance to even use this notebook at all. There was no chance to react!\n\nBesides this bug another damaging error was that computational resources were frozen so only participants with external cloud computing resources were able to even do further computations for predictions that could have an influence on the final submissions.\n\nWith doing nothing you just showed that blending public notebooks seems to be more important than doing computations on your own. Really sad. In my opinion the discussion forum and the notebooks should be generally closed or frozen a week before a competition ends. \n\n:-(", "votes": null }, { "id": "1530414", "postDate": "10/01/2021 06:42:38", "content": "\n\ndoing nothing at all is the wrong signal
\n
I second that
", "rawMarkdown": "> doing nothing at all is the wrong signal\n\nI second that", "votes": null }, { "id": "1530723", "postDate": "10/01/2021 11:13:10", "content": "I'm really starting to ask myself why I tried to climb up the leaderboard honestly during the last 4 weeks… with 2 little kids at home, interrupted nights and nightmare flood that destroyed so much. Why should anyone try to do that?! Why should I still continue with Kaggle when you don't care about the rule to \"not share high scoring public notebooks\" in the end of a competition (regardless now whether this was a bug or not)? Really frustrating.
\nI can easily live without this bronze medal but not so easily with breaking own principles and rules.
", "rawMarkdown": "I'm really starting to ask myself why I tried to climb up the leaderboard honestly during the last 4 weeks... with 2 little kids at home, interrupted nights and nightmare flood that destroyed so much. Why should anyone try to do that?! Why should I still continue with Kaggle when you don't care about the rule to \"not share high scoring public notebooks\" in the end of a competition (regardless now whether this was a bug or not)? Really frustrating. \n\nI can easily live without this bronze medal but not so easily with breaking own principles and rules.", "votes": null }, { "id": "1530739", "postDate": "10/01/2021 11:29:12", "content": "This is what I think a \"minimum detriment\" solution would look like:
\nThoughts?
", "rawMarkdown": "This is what I think a \"minimum detriment\" solution would look like:\n\n- Draw up the final private leaderboard as now, and also the private leaderboard immediately before the bug (the latter has to be based on best submission, since selections weren't universally finalised)\n- Identify the 121 teams who would have got medals under the pre-bug version\n- Find the one of these 121 teams that fell to the lowest position in the final LB\n- Draw a new lower-bronze borderline under that team in the final LB\n- Repeat procedure with the 60 teams that would have won at least silver, and again with the 12 teams that would have won gold under the pre-bug LB, drawing new lower boundaries for silver and gold on top of the same rankings we have now\n- I think this should mean that every team gets the better of their pre-bug and post-bug medal, together with their post-bug ranking position\n- The only 'cost' I see is slight grade inflation on medals. A few teams may get 'bonus' medals.\n\nThoughts?", "votes": null }, { "id": "1531005", "postDate": "10/01/2021 14:38:14", "content": "Dear Kaggle Staff members,
\nI fully agree with everything what Laura wrote hear. But 2 points still remain unadressed.
\nThe system-wide bug can't have suddenly appeared by itself! In my opinion there are only 2 possible causes (correct me if I am wrong). Either somebody hacked Kaggle system or some active Kaggle Staff member was \"fixing\" something just 5 hours prior to the end of the competition. Both possibilities sound little bit scary to me.
\nUnlike Laura I was up during the \"hours of magics\" watching me sliding down the leaderboard minute by minute. I could have theoretically done something because I still had my last submission reserved for the case of miracle happening - my last notebook successfully finishing after more than 30 hours of attempts to make it run on TPU (it didn't happen).
\nBut what should I have done? Should I have searched actively for a leaked top notebook and copied the output of some else's work to my own ensemble only because others had done it? Keeping my standards of ethics has always been more important for me than earning any benefits. I fully understand the depth of disappointment of those who lost a well deserved medal though it in the end didn't happen to me.
Blanka
", "rawMarkdown": "Dear Kaggle Staff members,\n\nI fully agree with everything what Laura wrote hear. But 2 points still remain unadressed.\n\nThe system-wide bug can't have suddenly appeared by itself! In my opinion there are only 2 possible causes (correct me if I am wrong). Either somebody hacked Kaggle system or some active Kaggle Staff member was \"fixing\" something just 5 hours prior to the end of the competition. Both possibilities sound little bit scary to me.\n\nUnlike Laura I was up during the \"hours of magics\" watching me sliding down the leaderboard minute by minute. I could have theoretically done something because I still had my last submission reserved for the case of miracle happening - my last notebook successfully finishing after more than 30 hours of attempts to make it run on TPU (it didn't happen).\nBut what should I have done? Should I have searched actively for a leaked top notebook and copied the output of some else's work to my own ensemble only because others had done it? Keeping my standards of ethics has always been more important for me than earning any benefits. I fully understand the depth of disappointment of those who lost a well deserved medal though it in the end didn't happen to me.\n\nBlanka", "votes": null }, { "id": "1531168", "postDate": "10/01/2021 17:21:28", "content": "\n\nthough it in the end didn't happen to me
\n
It might happen to you, not losing a medal but losing a deserved silver, since you are not very far away from silver. Because based on my observation, my rank dropped by 12 from 45 to 57 within 30minutes of the leakage happening. (current Public LB 53 is due to the removal of some suspicious accounts). Among these 12 teams, maybe one or two of them really had some breakthrough in the last hour of the competition, but very likely most of them used the leakage.
\nIt ruined the months of efforts of the hard working guys, especially for those near the medal boundries, at least for current rank 61 to 70, they probably should have earned a silver medal if these 12 teams I observed lose the 'help' of leakage.
\nThanks for keeping a high standard of ethics. And for those who used the leakage, I hope you can feel sorry for yourself.
\nI do hope this bug will not happen again. It is so scary. You cannot imagine the feeling of watching yourself sliding down the leaderboard to almost out of silver range and you can do nothing to stop it.
With my sentence : \" I fully understand the depth of disappointment of those who lost a well deserved medal…\" I of course meant all those who lost a medal of any quality, including those whose medal turned from silver to bronze.
\nI estimate my lost to be 15 to 20 positions.
\nAnd of course both of us would like to hear from Kaggle Staff some assurance that they will do their best to prevent repeating of such a calamity.
I have to say I am very disappointed with Kaggle in their inability to protect participants time and efforts. This conversation has been happening for way too long
", "rawMarkdown": "I have to say I am very disappointed with Kaggle in their inability to protect participants time and efforts. This conversation has been happening for way too long", "votes": null }, { "id": "1531391", "postDate": "10/02/2021 00:05:23", "content": "This just happened in the GLR competition as well - https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275173
", "rawMarkdown": "This just happened in the GLR competition as well - https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275173", "votes": null } ], "comments": [ { "id": 1530077, "author_name": "yamanity", "author_url": "", "post_date": "09/30/2021 22:51:02", "content": "sorry, i'm not good at English.
\nDoes it means that private leaderboad fixed?
@yamanity - Yes, it means that the preliminary standings are confirmed. Congratulations on your silver medal.
", "votes": null, "replies": [] }, { "id": 1530084, "author_name": "yamanity", "author_url": "", "post_date": "09/30/2021 23:06:27", "content": "thank you.I got it.arigato!
", "votes": null, "replies": [] } ] }, { "id": 1530365, "author_name": "allunia", "author_url": "", "post_date": "10/01/2021 05:58:33", "content": "Dear Kaggle Team,
\nI'm sorry, but doing nothing at all is the wrong signal. I can understand that doing a hard reset of the leaderboard to the time point when the bug occurred would cause damage to several teams that may also have not used the high scoring public notebook. But allowing this bug to influence the leaderboard is completely unfair. Only a small group of people were able to use this notebook but people like me that already had to go to sleep had no chance to even use this notebook at all. There was no chance to react!
\nBesides this bug another damaging error was that computational resources were frozen so only participants with external cloud computing resources were able to even do further computations for predictions that could have an influence on the final submissions.
\nWith doing nothing you just showed that blending public notebooks seems to be more important than doing computations on your own. Really sad. In my opinion the discussion forum and the notebooks should be generally closed or frozen a week before a competition ends.
\n:-(
", "votes": null, "replies": [ { "id": 1530414, "author_name": "zarif98sjs", "author_url": "", "post_date": "10/01/2021 06:42:38", "content": "\n\ndoing nothing at all is the wrong signal
\n
I second that
", "votes": null, "replies": [] }, { "id": 1530723, "author_name": "allunia", "author_url": "", "post_date": "10/01/2021 11:13:10", "content": "I'm really starting to ask myself why I tried to climb up the leaderboard honestly during the last 4 weeks… with 2 little kids at home, interrupted nights and nightmare flood that destroyed so much. Why should anyone try to do that?! Why should I still continue with Kaggle when you don't care about the rule to \"not share high scoring public notebooks\" in the end of a competition (regardless now whether this was a bug or not)? Really frustrating.
\nI can easily live without this bronze medal but not so easily with breaking own principles and rules.
", "votes": null, "replies": [] }, { "id": 1530739, "author_name": "jbomitchell", "author_url": "", "post_date": "10/01/2021 11:29:12", "content": "This is what I think a \"minimum detriment\" solution would look like:
\nThoughts?
", "votes": null, "replies": [] } ] }, { "id": 1531005, "author_name": "blankaf", "author_url": "", "post_date": "10/01/2021 14:38:14", "content": "Dear Kaggle Staff members,
\nI fully agree with everything what Laura wrote hear. But 2 points still remain unadressed.
\nThe system-wide bug can't have suddenly appeared by itself! In my opinion there are only 2 possible causes (correct me if I am wrong). Either somebody hacked Kaggle system or some active Kaggle Staff member was \"fixing\" something just 5 hours prior to the end of the competition. Both possibilities sound little bit scary to me.
\nUnlike Laura I was up during the \"hours of magics\" watching me sliding down the leaderboard minute by minute. I could have theoretically done something because I still had my last submission reserved for the case of miracle happening - my last notebook successfully finishing after more than 30 hours of attempts to make it run on TPU (it didn't happen).
\nBut what should I have done? Should I have searched actively for a leaked top notebook and copied the output of some else's work to my own ensemble only because others had done it? Keeping my standards of ethics has always been more important for me than earning any benefits. I fully understand the depth of disappointment of those who lost a well deserved medal though it in the end didn't happen to me.
Blanka
", "votes": null, "replies": [ { "id": 1531168, "author_name": "buumoo", "author_url": "", "post_date": "10/01/2021 17:21:28", "content": "\n\nthough it in the end didn't happen to me
\n
It might happen to you, not losing a medal but losing a deserved silver, since you are not very far away from silver. Because based on my observation, my rank dropped by 12 from 45 to 57 within 30minutes of the leakage happening. (current Public LB 53 is due to the removal of some suspicious accounts). Among these 12 teams, maybe one or two of them really had some breakthrough in the last hour of the competition, but very likely most of them used the leakage.
\nIt ruined the months of efforts of the hard working guys, especially for those near the medal boundries, at least for current rank 61 to 70, they probably should have earned a silver medal if these 12 teams I observed lose the 'help' of leakage.
\nThanks for keeping a high standard of ethics. And for those who used the leakage, I hope you can feel sorry for yourself.
\nI do hope this bug will not happen again. It is so scary. You cannot imagine the feeling of watching yourself sliding down the leaderboard to almost out of silver range and you can do nothing to stop it.
With my sentence : \" I fully understand the depth of disappointment of those who lost a well deserved medal…\" I of course meant all those who lost a medal of any quality, including those whose medal turned from silver to bronze.
\nI estimate my lost to be 15 to 20 positions.
\nAnd of course both of us would like to hear from Kaggle Staff some assurance that they will do their best to prevent repeating of such a calamity.
I have to say I am very disappointed with Kaggle in their inability to protect participants time and efforts. This conversation has been happening for way too long
", "votes": null, "replies": [ { "id": 1531391, "author_name": "rdizzl3", "author_url": "", "post_date": "10/02/2021 00:05:23", "content": "This just happened in the GLR competition as well - https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275173
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1530068": "As some users have noted, there was a four hour window yesterday where newly-created notebooks were defaulted to Public, creating unintentional sharing. (See this [forum post](https://www.kaggle.com/general/275572) for additional details.)\n\nThis bug impacted a small number of available public notebooks during the last few hours of the G2Net competition. Unfortunately, there is no clean way to rewind the impact without also creating secondary disruptions. We have carefully considered alternatives and believe they are equally or more disruptive to a fair competition outcome. Therefore, we will proceed with the G2Net leaderboard as it currently stands.\n\nKaggle's goal has always been to provide competitions that are as fair as possible, recognizing that there are sometimes data, platform, and human issues that complicate this mission. We regret the issues this bug has caused in the final hours of what was otherwise an interesting and challenging competition and hope that those affected will continue to see Kaggle as a place to be challenged and to hone skills alongside a community of people who are so willing to share and help others succeed.\n\nThe Kaggle Team", "1530077": "sorry, i'm not good at English.\nDoes it means that private leaderboad fixed?", "1530080": "yamanity - Yes, it means that the preliminary standings are confirmed. Congratulations on your silver medal.", "1530084": "thank you.I got it.arigato!", "1530365": "Dear Kaggle Team,\n\nI'm sorry, but doing nothing at all is the wrong signal. I can understand that doing a hard reset of the leaderboard to the time point when the bug occurred would cause damage to several teams that may also have not used the high scoring public notebook. But allowing this bug to influence the leaderboard is completely unfair. Only a small group of people were able to use this notebook but people like me that already had to go to sleep had no chance to even use this notebook at all. There was no chance to react!\n\nBesides this bug another damaging error was that computational resources were frozen so only participants with external cloud computing resources were able to even do further computations for predictions that could have an influence on the final submissions.\n\nWith doing nothing you just showed that blending public notebooks seems to be more important than doing computations on your own. Really sad. In my opinion the discussion forum and the notebooks should be generally closed or frozen a week before a competition ends. \n\n:-(", "1530414": "> doing nothing at all is the wrong signal\n\nI second that", "1530723": "I'm really starting to ask myself why I tried to climb up the leaderboard honestly during the last 4 weeks... with 2 little kids at home, interrupted nights and nightmare flood that destroyed so much. Why should anyone try to do that?! Why should I still continue with Kaggle when you don't care about the rule to \"not share high scoring public notebooks\" in the end of a competition (regardless now whether this was a bug or not)? Really frustrating. \n\nI can easily live without this bronze medal but not so easily with breaking own principles and rules.", "1530739": "This is what I think a \"minimum detriment\" solution would look like:\n\n- Draw up the final private leaderboard as now, and also the private leaderboard immediately before the bug (the latter has to be based on best submission, since selections weren't universally finalised)\n- Identify the 121 teams who would have got medals under the pre-bug version\n- Find the one of these 121 teams that fell to the lowest position in the final LB\n- Draw a new lower-bronze borderline under that team in the final LB\n- Repeat procedure with the 60 teams that would have won at least silver, and again with the 12 teams that would have won gold under the pre-bug LB, drawing new lower boundaries for silver and gold on top of the same rankings we have now\n- I think this should mean that every team gets the better of their pre-bug and post-bug medal, together with their post-bug ranking position\n- The only 'cost' I see is slight grade inflation on medals. A few teams may get 'bonus' medals.\n\nThoughts?", "1531005": "Dear Kaggle Staff members,\n\nI fully agree with everything what Laura wrote hear. But 2 points still remain unadressed.\n\nThe system-wide bug can't have suddenly appeared by itself! In my opinion there are only 2 possible causes (correct me if I am wrong). Either somebody hacked Kaggle system or some active Kaggle Staff member was \"fixing\" something just 5 hours prior to the end of the competition. Both possibilities sound little bit scary to me.\n\nUnlike Laura I was up during the \"hours of magics\" watching me sliding down the leaderboard minute by minute. I could have theoretically done something because I still had my last submission reserved for the case of miracle happening - my last notebook successfully finishing after more than 30 hours of attempts to make it run on TPU (it didn't happen).\nBut what should I have done? Should I have searched actively for a leaked top notebook and copied the output of some else's work to my own ensemble only because others had done it? Keeping my standards of ethics has always been more important for me than earning any benefits. I fully understand the depth of disappointment of those who lost a well deserved medal though it in the end didn't happen to me.\n\nBlanka", "1531168": "> though it in the end didn't happen to me\n\nIt might happen to you, not losing a medal but losing a deserved silver, since you are not very far away from silver. Because based on my observation, my rank dropped by 12 from 45 to 57 within 30minutes of the leakage happening. (current Public LB 53 is due to the removal of some suspicious accounts). Among these 12 teams, maybe one or two of them really had some breakthrough in the last hour of the competition, but very likely most of them used the leakage.\nIt ruined the months of efforts of the hard working guys, especially for those near the medal boundries, at least for current rank 61 to 70, they probably should have earned a silver medal if these 12 teams I observed lose the 'help' of leakage.\nThanks for keeping a high standard of ethics. And for those who used the leakage, I hope you can feel sorry for yourself.\nI do hope this bug will not happen again. It is so scary. You cannot imagine the feeling of watching yourself sliding down the leaderboard to almost out of silver range and you can do nothing to stop it.", "1531185": "With my sentence : \" I fully understand the depth of disappointment of those who lost a well deserved medal...\" I of course meant all those who lost a medal of any quality, including those whose medal turned from silver to bronze. \nI estimate my lost to be 15 to 20 positions. \nAnd of course both of us would like to hear from Kaggle Staff some assurance that they will do their best to prevent repeating of such a calamity.", "1531307": "I have to say I am very disappointed with Kaggle in their inability to protect participants time and efforts. This conversation has been happening for way too long", "1531391": "This just happened in the GLR competition as well - https://www.kaggle.com/c/landmark-retrieval-2021/discussion/275173" }, "source": "meta" }