Thank you all for your hard work in this competition (and your patience with the new dataset). We are always very impressed with the community. We also would like to give big thanks to Ann and Arden from KKBox, they have worked hard for this competition and we're glad to have presented the Kaggle community with the dataset.
\n\nWe've cleaned the leaderboard and removed some cheaters. If you think you were removed by mistake, please contact compliance. Please fill in all the fields honestly.
\n\nIf you were a top finisher, be on the lookout for additional information from the WSDM/KKBox teams regarding the prize money, as well as the opportunity to speak at WSDM for eligible winners.
\n\nWe hope to see you in your next competitions!
\n\nHappy Modeling!
", "messages": [ { "id": "261122", "postDate": "12/21/2017 20:50:36", "content": "Thank you all for your hard work in this competition (and your patience with the new dataset). We are always very impressed with the community. We also would like to give big thanks to Ann and Arden from KKBox, they have worked hard for this competition and we're glad to have presented the Kaggle community with the dataset.
\n\nWe've cleaned the leaderboard and removed some cheaters. If you think you were removed by mistake, please contact compliance. Please fill in all the fields honestly.
\n\nIf you were a top finisher, be on the lookout for additional information from the WSDM/KKBox teams regarding the prize money, as well as the opportunity to speak at WSDM for eligible winners.
\n\nWe hope to see you in your next competitions!
\n\nHappy Modeling!
", "rawMarkdown": "Thank you all for your hard work in this competition (and your patience with the new dataset). We are always very impressed with the community. We also would like to give big thanks to Ann and Arden from KKBox, they have worked hard for this competition and we're glad to have presented the Kaggle community with the dataset.\n\nWe've cleaned the leaderboard and removed some cheaters. If you think you were removed by mistake, please contact [compliance][1]. Please fill in all the fields honestly.\n\nIf you were a top finisher, be on the lookout for additional information from the WSDM/KKBox teams regarding the prize money, as well as the opportunity to speak at WSDM for eligible winners.\n\nWe hope to see you in your next competitions!\n\nHappy Modeling!\n\n\n [1]: https://www.kaggle.com/compliance", "votes": null }, { "id": "261161", "postDate": "12/21/2017 22:48:42", "content": "Hi Addison,
\n\nI wonder if the organizer gave serious thoughts on the issue I raised here. At least 3 teams out of top 4 teams in the public LB seemed to choose submissions that score worse on both public and private LBs to comply the competition rule.
\n\nWithout proper validation on whether the solutions selected by top teams comply the rules or not, it will be the lose-lose situation for both the organizer and top finishers.
\n\nAny comments on the matter will be appreciated.
\n\nBest,\nJeong
", "rawMarkdown": "Hi Addison,\n\nI wonder if the organizer gave serious thoughts on the issue I raised [here][1]. At least 3 teams out of top 4 teams in the public LB seemed to choose submissions that score worse on both public and private LBs to comply the competition rule.\n\nWithout proper validation on whether the solutions selected by top teams comply the rules or not, it will be the lose-lose situation for both the organizer and top finishers.\n\nAny comments on the matter will be appreciated.\n\nBest,\nJeong\n\n [1]: https://www.kaggle.com/c/kkbox-churn-prediction-challenge/discussion/45921", "votes": null }, { "id": "261228", "postDate": "12/22/2017 03:00:59", "content": "Hi Jeong-Loon,
\n\nKaggle does not elaborate on our anti-cheating detection/validation mechanisms. Thanks for looking out!
\n\nBest,
", "rawMarkdown": "Hi Jeong-Loon,\n\nKaggle does not elaborate on our anti-cheating detection/validation mechanisms. Thanks for looking out!\n\nBest,", "votes": null }, { "id": "261233", "postDate": "12/22/2017 03:24:36", "content": "Thanks for your prompt reply, Addison,
\n\nYou must have mistaken what I meant by \"validation\". It's less about how Kaggle detects cheating in general, which I'm very familiar with, but more about how to deal with the specific leak (expiration dates in members.csv) introduced by the organizer accidentally at the beginning of the competition - then later banned by the organizer.
\n\nThere is no way for Kaggle/organizer to verify whether a team exploited such a leak without cooperation from the team.
\n\nAs an organizer to another competition, I understand that it's not easy to do it right, and don't mind if the organizer decides not to do so given the timeline and resources available.
\n\nHowever, it would be better if the organizer would have officially addressed the issue raised several times both before and after the competition deadline.
\n\nI feel that the response from both Kaggle and organizer could've been better.
\n\nBest,\nJeong
", "rawMarkdown": "Thanks for your prompt reply, Addison,\n\nYou must have mistaken what I meant by \"validation\". It's less about how Kaggle detects cheating in general, which I'm very familiar with, but more about how to deal with the specific leak (expiration dates in members.csv) introduced by the organizer accidentally at the beginning of the competition - then later banned by the organizer. \n\nThere is no way for Kaggle/organizer to verify whether a team exploited such a leak without cooperation from the team. \n\nAs an organizer to another competition, I understand that it's not easy to do it right, and don't mind if the organizer decides not to do so given the timeline and resources available. \n\nHowever, it would be better if the organizer would have officially addressed the issue raised several times both before and after the competition deadline.\n\nI feel that the response from both Kaggle and organizer could've been better.\n\nBest,\nJeong", "votes": null }, { "id": "261240", "postDate": "12/22/2017 03:43:52", "content": "Jeong,
\n\nSorry, I should have clarified. We also cannot elaborate much on the level of validation we do/do not perform for something like this. If we extend that information fully, the nuances could be exploited by those hoping to \"game the system.\"
\n\nWe did note in the forums (see here) and here (which you also cited) that these datasets were not allowed. Per the rules, the competition sponsors still have the opportunity to validate the winning models and examine the code specifically for these issues - therefore if a winning team's model cannot reproduce the leaderboard scores per the rules of the competition in a satisfactory manner, the prizes will be extended to the next on the leaderboard.
\n\nWe apologize this was not clearer at the time and have been discussing internally ways to clarify non-standard competition rules in clear, concise ways going forward.
\n\nThanks for the great feedback!
", "rawMarkdown": "Jeong,\n\nSorry, I should have clarified. We also cannot elaborate much on the level of validation we do/do not perform for something like this. If we extend that information fully, the nuances could be exploited by those hoping to \"game the system.\"\n\nWe did note in the forums (see [here][1]) and [here][2] (which you also cited) that these datasets were not allowed. Per the rules, the competition sponsors still have the opportunity to validate the winning models and examine the code specifically for these issues - therefore if a winning team's model cannot reproduce the leaderboard scores per the rules of the competition in a satisfactory manner, the prizes will be extended to the next on the leaderboard.\n\nWe apologize this was not clearer at the time and have been discussing internally ways to clarify non-standard competition rules in clear, concise ways going forward. \n\nThanks for the great feedback!\n\n\n [1]: https://www.kaggle.com/c/kkbox-churn-prediction-challenge/discussion/43169#255915\n [2]: https://www.kaggle.com/c/kkbox-churn-prediction-challenge/discussion/44363#250087", "votes": null }, { "id": "261562", "postDate": "12/23/2017 01:18:08", "content": "Addison, I totaly agree with Jeong. I was one of the teams that changed the final submission to comply with the rules. Indeed, I was one of the competitors that asked for a clear statement about the data that we were allowed to use. By the time that statement was made, I was the first of the leadeboard, and I immediately stopped using the leaked file.
\n\nLooking at the final leaderboard now, and comparing it to the public leaderboard, I cannot be sure that all the top scorers followed the rules!
", "rawMarkdown": "Addison, I totaly agree with Jeong. I was one of the teams that changed the final submission to comply with the rules. Indeed, I was one of the competitors that asked for a clear statement about the data that we were allowed to use. By the time that statement was made, I was the first of the leadeboard, and I immediately stopped using the leaked file. \n\nLooking at the final leaderboard now, and comparing it to the public leaderboard, I cannot be sure that all the top scorers followed the rules!", "votes": null } ], "comments": [ { "id": 261161, "author_name": "jeongyoonlee", "author_url": "", "post_date": "12/21/2017 22:48:42", "content": "Hi Addison,
\n\nI wonder if the organizer gave serious thoughts on the issue I raised here. At least 3 teams out of top 4 teams in the public LB seemed to choose submissions that score worse on both public and private LBs to comply the competition rule.
\n\nWithout proper validation on whether the solutions selected by top teams comply the rules or not, it will be the lose-lose situation for both the organizer and top finishers.
\n\nAny comments on the matter will be appreciated.
\n\nBest,\nJeong
", "votes": null, "replies": [ { "id": 261228, "author_name": "addisonhoward", "author_url": "", "post_date": "12/22/2017 03:00:59", "content": "Hi Jeong-Loon,
\n\nKaggle does not elaborate on our anti-cheating detection/validation mechanisms. Thanks for looking out!
\n\nBest,
", "votes": null, "replies": [] }, { "id": 261233, "author_name": "jeongyoonlee", "author_url": "", "post_date": "12/22/2017 03:24:36", "content": "Thanks for your prompt reply, Addison,
\n\nYou must have mistaken what I meant by \"validation\". It's less about how Kaggle detects cheating in general, which I'm very familiar with, but more about how to deal with the specific leak (expiration dates in members.csv) introduced by the organizer accidentally at the beginning of the competition - then later banned by the organizer.
\n\nThere is no way for Kaggle/organizer to verify whether a team exploited such a leak without cooperation from the team.
\n\nAs an organizer to another competition, I understand that it's not easy to do it right, and don't mind if the organizer decides not to do so given the timeline and resources available.
\n\nHowever, it would be better if the organizer would have officially addressed the issue raised several times both before and after the competition deadline.
\n\nI feel that the response from both Kaggle and organizer could've been better.
\n\nBest,\nJeong
", "votes": null, "replies": [] }, { "id": 261240, "author_name": "addisonhoward", "author_url": "", "post_date": "12/22/2017 03:43:52", "content": "Jeong,
\n\nSorry, I should have clarified. We also cannot elaborate much on the level of validation we do/do not perform for something like this. If we extend that information fully, the nuances could be exploited by those hoping to \"game the system.\"
\n\nWe did note in the forums (see here) and here (which you also cited) that these datasets were not allowed. Per the rules, the competition sponsors still have the opportunity to validate the winning models and examine the code specifically for these issues - therefore if a winning team's model cannot reproduce the leaderboard scores per the rules of the competition in a satisfactory manner, the prizes will be extended to the next on the leaderboard.
\n\nWe apologize this was not clearer at the time and have been discussing internally ways to clarify non-standard competition rules in clear, concise ways going forward.
\n\nThanks for the great feedback!
", "votes": null, "replies": [] }, { "id": 261562, "author_name": "aloisiodn", "author_url": "", "post_date": "12/23/2017 01:18:08", "content": "Addison, I totaly agree with Jeong. I was one of the teams that changed the final submission to comply with the rules. Indeed, I was one of the competitors that asked for a clear statement about the data that we were allowed to use. By the time that statement was made, I was the first of the leadeboard, and I immediately stopped using the leaked file.
\n\nLooking at the final leaderboard now, and comparing it to the public leaderboard, I cannot be sure that all the top scorers followed the rules!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "261122": "Thank you all for your hard work in this competition (and your patience with the new dataset). We are always very impressed with the community. We also would like to give big thanks to Ann and Arden from KKBox, they have worked hard for this competition and we're glad to have presented the Kaggle community with the dataset.\n\nWe've cleaned the leaderboard and removed some cheaters. If you think you were removed by mistake, please contact [compliance][1]. Please fill in all the fields honestly.\n\nIf you were a top finisher, be on the lookout for additional information from the WSDM/KKBox teams regarding the prize money, as well as the opportunity to speak at WSDM for eligible winners.\n\nWe hope to see you in your next competitions!\n\nHappy Modeling!\n\n\n [1]: https://www.kaggle.com/compliance", "261161": "Hi Addison,\n\nI wonder if the organizer gave serious thoughts on the issue I raised [here][1]. At least 3 teams out of top 4 teams in the public LB seemed to choose submissions that score worse on both public and private LBs to comply the competition rule.\n\nWithout proper validation on whether the solutions selected by top teams comply the rules or not, it will be the lose-lose situation for both the organizer and top finishers.\n\nAny comments on the matter will be appreciated.\n\nBest,\nJeong\n\n [1]: https://www.kaggle.com/c/kkbox-churn-prediction-challenge/discussion/45921", "261228": "Hi Jeong-Loon,\n\nKaggle does not elaborate on our anti-cheating detection/validation mechanisms. Thanks for looking out!\n\nBest,", "261233": "Thanks for your prompt reply, Addison,\n\nYou must have mistaken what I meant by \"validation\". It's less about how Kaggle detects cheating in general, which I'm very familiar with, but more about how to deal with the specific leak (expiration dates in members.csv) introduced by the organizer accidentally at the beginning of the competition - then later banned by the organizer. \n\nThere is no way for Kaggle/organizer to verify whether a team exploited such a leak without cooperation from the team. \n\nAs an organizer to another competition, I understand that it's not easy to do it right, and don't mind if the organizer decides not to do so given the timeline and resources available. \n\nHowever, it would be better if the organizer would have officially addressed the issue raised several times both before and after the competition deadline.\n\nI feel that the response from both Kaggle and organizer could've been better.\n\nBest,\nJeong", "261240": "Jeong,\n\nSorry, I should have clarified. We also cannot elaborate much on the level of validation we do/do not perform for something like this. If we extend that information fully, the nuances could be exploited by those hoping to \"game the system.\"\n\nWe did note in the forums (see [here][1]) and [here][2] (which you also cited) that these datasets were not allowed. Per the rules, the competition sponsors still have the opportunity to validate the winning models and examine the code specifically for these issues - therefore if a winning team's model cannot reproduce the leaderboard scores per the rules of the competition in a satisfactory manner, the prizes will be extended to the next on the leaderboard.\n\nWe apologize this was not clearer at the time and have been discussing internally ways to clarify non-standard competition rules in clear, concise ways going forward. \n\nThanks for the great feedback!\n\n\n [1]: https://www.kaggle.com/c/kkbox-churn-prediction-challenge/discussion/43169#255915\n [2]: https://www.kaggle.com/c/kkbox-churn-prediction-challenge/discussion/44363#250087", "261562": "Addison, I totaly agree with Jeong. I was one of the teams that changed the final submission to comply with the rules. Indeed, I was one of the competitors that asked for a clear statement about the data that we were allowed to use. By the time that statement was made, I was the first of the leadeboard, and I immediately stopped using the leaked file. \n\nLooking at the final leaderboard now, and comparing it to the public leaderboard, I cannot be sure that all the top scorers followed the rules!" }, "source": "meta" }