{
  "id": 384844,
  "title": "Update on Competition Compliance",
  "url": "/competitions/otto-recommender-system/discussion/384844",
  "author_name": "",
  "post_date": "2023-02-09T18:39:47.423174800Z",
  "votes": 68,
  "comment_count": 21,
  "views": 0,
  "content": "<p>First and most importantly, thanks to everyone who participated in this exciting and challenging competition! And of course, thanks to the host OTTO for proving such an interesting problem and fantastic dataset for the Kaggle community.</p>\n<p>Addressing a few points on the competition compliance, recognizing the uncertainty and waiting has been very frustrating to everyone involved:</p>\n<p><strong>Q:</strong> Why were there changes to the Leaderboard after it was verified?</p>\n<p><strong>A:</strong> There are instances where the decision to remove a team for compliance issues is very straightforward. On occasion, a proper investigation requires significantly more time. Even then, there are times when we uncover additional information after a leaderboard has been verified. In these instances, it is possible that additional teams are removed from the leaderboard after verification.</p>\n<p><strong>Q:</strong> Can you provide more details why teams are removed?</p>\n<p><strong>A:</strong> Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.</p>\n<p><strong>Q:</strong> What is Kaggle doing to prevent cheating in the future?</p>\n<p><strong>A:</strong> The vast majority of Kagglers abide by competition rules. Unfortunately, the unethical actions of a select few participants degrades the overall experience of a competition. As has always been the case, Kaggle will continue to monitor for actions that do not abide by competition rules and remove teams found to be out of compliance. Repeat offenders will receive account bans. Discovery of unethical behavior (e.g., account selling/buying) will also result in account bans. In addition to these ongoing activities, Kaggle is working on tools and processes to make cheating of any kind more difficult and to improve our detection.</p>\n<p>Thanks to everyone for being part of this amazing community!</p>",
  "messages": [
    {
      "id": "2137127",
      "postDate": "02/09/2023 18:39:47",
      "content": "<p>First and most importantly, thanks to everyone who participated in this exciting and challenging competition! And of course, thanks to the host OTTO for proving such an interesting problem and fantastic dataset for the Kaggle community.</p>\n<p>Addressing a few points on the competition compliance, recognizing the uncertainty and waiting has been very frustrating to everyone involved:</p>\n<p><strong>Q:</strong> Why were there changes to the Leaderboard after it was verified?</p>\n<p><strong>A:</strong> There are instances where the decision to remove a team for compliance issues is very straightforward. On occasion, a proper investigation requires significantly more time. Even then, there are times when we uncover additional information after a leaderboard has been verified. In these instances, it is possible that additional teams are removed from the leaderboard after verification.</p>\n<p><strong>Q:</strong> Can you provide more details why teams are removed?</p>\n<p><strong>A:</strong> Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.</p>\n<p><strong>Q:</strong> What is Kaggle doing to prevent cheating in the future?</p>\n<p><strong>A:</strong> The vast majority of Kagglers abide by competition rules. Unfortunately, the unethical actions of a select few participants degrades the overall experience of a competition. As has always been the case, Kaggle will continue to monitor for actions that do not abide by competition rules and remove teams found to be out of compliance. Repeat offenders will receive account bans. Discovery of unethical behavior (e.g., account selling/buying) will also result in account bans. In addition to these ongoing activities, Kaggle is working on tools and processes to make cheating of any kind more difficult and to improve our detection.</p>\n<p>Thanks to everyone for being part of this amazing community!</p>",
      "rawMarkdown": "First and most importantly, thanks to everyone who participated in this exciting and challenging competition! And of course, thanks to the host OTTO for proving such an interesting problem and fantastic dataset for the Kaggle community.\n\nAddressing a few points on the competition compliance, recognizing the uncertainty and waiting has been very frustrating to everyone involved:\n\n**Q:** Why were there changes to the Leaderboard after it was verified?\n\n**A:** There are instances where the decision to remove a team for compliance issues is very straightforward. On occasion, a proper investigation requires significantly more time. Even then, there are times when we uncover additional information after a leaderboard has been verified. In these instances, it is possible that additional teams are removed from the leaderboard after verification.\n\n\n**Q:** Can you provide more details why teams are removed?\n\n**A:** Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.\n\n\n**Q:** What is Kaggle doing to prevent cheating in the future?\n\n**A:** The vast majority of Kagglers abide by competition rules. Unfortunately, the unethical actions of a select few participants degrades the overall experience of a competition. As has always been the case, Kaggle will continue to monitor for actions that do not abide by competition rules and remove teams found to be out of compliance. Repeat offenders will receive account bans. Discovery of unethical behavior (e.g., account selling/buying) will also result in account bans. In addition to these ongoing activities, Kaggle is working on tools and processes to make cheating of any kind more difficult and to improve our detection.\n\nThanks to everyone for being part of this amazing community!",
      "votes": null
    },
    {
      "id": "2137246",
      "postDate": "02/09/2023 20:18:54",
      "content": "<p>A move in the right direction, but for somebody too late, too little. <br>\nWithout a huge public \"uprising\" things would stay as before and another team would be heading to the same bitter fate as Makotu's team in this competition. </p>",
      "rawMarkdown": "A move in the right direction, but for somebody too late, too little. \nWithout a huge public \"uprising\" things would stay as before and another team would be heading to the same bitter fate as Makotu's team in this competition.",
      "votes": null
    },
    {
      "id": "2137360",
      "postDate": "02/09/2023 22:39:02",
      "content": "<p>Our gold medal is gone, but it is much more important that kaggle is a healthy platform!<br>\nEveryone, be careful when merging teams. It was a good learning experience.</p>",
      "rawMarkdown": "Our gold medal is gone, but it is much more important that kaggle is a healthy platform!\nEveryone, be careful when merging teams. It was a good learning experience.",
      "votes": null
    },
    {
      "id": "2137407",
      "postDate": "02/10/2023 00:50:11",
      "content": "<p>Sorry to hear about your loss. Nevertheless, thank you and <a href=\"https://www.kaggle.com/bruceqdu\" target=\"_blank\">@bruceqdu</a> for your honesty. It adds much value to the Kaggle community by showing that Kagglers have high ethical standards in addition to exceptional technical knowledge and skills. I wish you all the best.</p>",
      "rawMarkdown": "Sorry to hear about your loss. Nevertheless, thank you and @bruceqdu for your honesty. It adds much value to the Kaggle community by showing that Kagglers have high ethical standards in addition to exceptional technical knowledge and skills. I wish you all the best.",
      "votes": null
    },
    {
      "id": "2137425",
      "postDate": "02/10/2023 01:38:17",
      "content": "<p><a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> Regarding the removal of the team, Our team submit a form to Kaggle compliance a few days ago, Could you get a response and told us the specific reason why we are removed through email? And Which teammate is responsible for it? </p>\n<p>Thanks, we enjoyed this competition very much. And we also work very hard on this comp. In recent days, we feel sad about it and cannot figure out why we are removed. Looking forward to ur reply. 🙏</p>",
      "rawMarkdown": "inversion Regarding the removal of the team, Our team submit a form to Kaggle compliance a few days ago, Could you get a response and told us the specific reason why we are removed through email? And Which teammate is responsible for it? \n\nThanks, we enjoyed this competition very much. And we also work very hard on this comp. In recent days, we feel sad about it and cannot figure out why we are removed. Looking forward to ur reply. 🙏",
      "votes": null
    },
    {
      "id": "2137427",
      "postDate": "02/10/2023 01:40:53",
      "content": "<p>Our gold medal is lost, but I respect your decision.  It was a very expensive study.</p>\n<p>However, I am concerned that this result will make us timid about blowing the whistle in the future. It would be great to have a system to bail out even during the competition when we have an unintended team merge. I also think that banning obvious cheaters even during competition will reduce the number of unfortunate team merges. I hope that the Kaggle team will properly address these issues from a systems perspective.</p>",
      "rawMarkdown": "Our gold medal is lost, but I respect your decision.  It was a very expensive study.\n\nHowever, I am concerned that this result will make us timid about blowing the whistle in the future. It would be great to have a system to bail out even during the competition when we have an unintended team merge. I also think that banning obvious cheaters even during competition will reduce the number of unfortunate team merges. I hope that the Kaggle team will properly address these issues from a systems perspective.",
      "votes": null
    },
    {
      "id": "2137439",
      "postDate": "02/10/2023 02:14:46",
      "content": "<p>Great respect to those who pointed them out!</p>",
      "rawMarkdown": "Great respect to those who pointed them out!",
      "votes": null
    },
    {
      "id": "2137445",
      "postDate": "02/10/2023 02:30:00",
      "content": "<blockquote>\n  <p>A: Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.</p>\n</blockquote>\n<p>By the way, doesn't this case deserve that rare instances? From some of the threads I've seen, I believe that this is a significant incident that has occurred in the past. Grandmaster being banned is a very big incident and there should be a detailed explanation as there has been in <a href=\"https://www.kaggle.com/competitions/petfinder-adoption-prediction/discussion/125436\" target=\"_blank\">the past Petfinder competitions</a>.</p>\n<p>At least our team is a party to this and would like to know more details.</p>",
      "rawMarkdown": "> A: Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.\n\nBy the way, doesn't this case deserve that rare instances? From some of the threads I've seen, I believe that this is a significant incident that has occurred in the past. Grandmaster being banned is a very big incident and there should be a detailed explanation as there has been in [the past Petfinder competitions](https://www.kaggle.com/competitions/petfinder-adoption-prediction/discussion/125436).\n\nAt least our team is a party to this and would like to know more details.",
      "votes": null
    },
    {
      "id": "2137446",
      "postDate": "02/10/2023 02:37:20",
      "content": "<p>Being removed pains me so much because of the countless hours spent on this competition and we all knowing ourselves' innocence. But it will pain me more if cheaters go unpunished. <br>\nThough I prefer another way of dealing with this thing, I respect your decision. <br>\nLooking forward to joining the next recsys competition.</p>",
      "rawMarkdown": "Being removed pains me so much because of the countless hours spent on this competition and we all knowing ourselves' innocence. But it will pain me more if cheaters go unpunished. \nThough I prefer another way of dealing with this thing, I respect your decision. \nLooking forward to joining the next recsys competition.",
      "votes": null
    },
    {
      "id": "2137455",
      "postDate": "02/10/2023 02:57:18",
      "content": "<p>Within 24 hours after forming the team, we four draw a line with Poteman and I emailed <a href=\"https://www.kaggle.com/bruceqdu\" target=\"_blank\">@bruceqdu</a> to notice runningZ;<br>\nWithin 48 hours after forming the team, <a href=\"https://www.kaggle.com/mhyodo\" target=\"_blank\">@mhyodo</a> and <a href=\"https://www.kaggle.com/bruceqdu\" target=\"_blank\">@bruceqdu</a> made the public announcements about their cheating.</p>\n<p>To be honest, I am a little bit disappointed.😣</p>",
      "rawMarkdown": "Within 24 hours after forming the team, we four draw a line with Poteman and I emailed @bruceqdu to notice runningZ;\nWithin 48 hours after forming the team, @mhyodo and @bruceqdu made the public announcements about their cheating.\n\n\nTo be honest, I am a little bit disappointed.😣",
      "votes": null
    },
    {
      "id": "2137584",
      "postDate": "02/10/2023 06:52:46",
      "content": "<p>Great work!</p>",
      "rawMarkdown": "Great work!",
      "votes": null
    },
    {
      "id": "2137628",
      "postDate": "02/10/2023 07:40:51",
      "content": "<p>IMHO, It should not be a system where honest people lose out. They sacrificed themselves for the sake of justice. Could kaggle ban the cheaters if that honest people had kept the secret?<br>\n<a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> Do you think there are another compensation?<br>\nPlease take a look into the behavior of honest people</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/381318\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/381318</a></li>\n<li><a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/381321\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/381321</a></li>\n</ul>",
      "rawMarkdown": "IMHO, It should not be a system where honest people lose out. They sacrificed themselves for the sake of justice. Could kaggle ban the cheaters if that honest people had kept the secret?\n@inversion Do you think there are another compensation?\nPlease take a look into the behavior of honest people\n- https://www.kaggle.com/competitions/otto-recommender-system/discussion/381318\n- https://www.kaggle.com/competitions/otto-recommender-system/discussion/381321",
      "votes": null
    },
    {
      "id": "2140653",
      "postDate": "02/12/2023 02:11:22",
      "content": "<p>I suggest kaggle should verify the codes of top-ranked teams (such as gold medal teams or top 50 teams).<br>\nMany data mining competitions held by ACM conferences (such as KDD CUP) or large platform (such as Tianchi) always did this.<br>\nFrom the code verification, kaggle could make sure each team could reproduce their submissions and detect similar and suspicious codes.</p>",
      "rawMarkdown": "I suggest kaggle should verify the codes of top-ranked teams (such as gold medal teams or top 50 teams).\nMany data mining competitions held by ACM conferences (such as KDD CUP) or large platform (such as Tianchi) always did this.\nFrom the code verification, kaggle could make sure each team could reproduce their submissions and detect similar and suspicious codes.",
      "votes": null
    },
    {
      "id": "2140654",
      "postDate": "02/12/2023 02:21:23",
      "content": "<p>Agree. If top50 teams are too huge to verify, <code>np.random.choice(top_teams, size=int(len(top_teams) * 0.1))</code> would also work</p>",
      "rawMarkdown": "Agree. If top50 teams are too huge to verify, `np.random.choice(top_teams, size=int(len(top_teams) * 0.1))` would also work",
      "votes": null
    },
    {
      "id": "2140728",
      "postDate": "02/12/2023 05:14:19",
      "content": "<p>I agree with your suggestion.  It's a heavy work and Kaggle team need to hire more people😂. I also joined KDDCup 2017 once and lots of Tianchi comps. In fact, they didn't check code very carefully. There are many cheaters passed the code verification including two famous ones we know.</p>",
      "rawMarkdown": "I agree with your suggestion.  It's a heavy work and Kaggle team need to hire more people😂. I also joined KDDCup 2017 once and lots of Tianchi comps. In fact, they didn't check code very carefully. There are many cheaters passed the code verification including two famous ones we know.",
      "votes": null
    },
    {
      "id": "2140801",
      "postDate": "02/12/2023 06:43:47",
      "content": "<p>Perhaps kaggle could collect all the codes from top-ranked teams and publish them to all the competitors.<br>\nThe competitors may find cheaters from the codes after a few days.</p>",
      "rawMarkdown": "Perhaps kaggle could collect all the codes from top-ranked teams and publish them to all the competitors.\nThe competitors may find cheaters from the codes after a few days.",
      "votes": null
    },
    {
      "id": "2140984",
      "postDate": "02/12/2023 10:49:47",
      "content": "<p>This.   We might never have finally developed enough evidence to remove poteman from Kaggle if not for them.  We collectively owe them a debt of gratitude.  And truly, what a tragedy, it must have been heartbreaking.   On top of that, they'll probably develop this massive paranoia about other people.</p>\n<p>Truly, I place the blame squarely on Kaggle for not <a href=\"https://www.kaggle.com/discussions/product-feedback/373889\" target=\"_blank\">normalizing basic activities</a> through some simple guidance that could avoid these situations.   This would be extremely low hanging fruit to implement - just some highly visible text on the team page.</p>\n<p>My security experience has taught me that bad/flawed actors will always exist, it's a fundamental law of any complex system.  What's far less practical is trying to rid the multiverse of bad actors than just fixing the system to mitigate their damage.  </p>",
      "rawMarkdown": "This.   We might never have finally developed enough evidence to remove poteman from Kaggle if not for them.  We collectively owe them a debt of gratitude.  And truly, what a tragedy, it must have been heartbreaking.   On top of that, they'll probably develop this massive paranoia about other people.\n\nTruly, I place the blame squarely on Kaggle for not [normalizing basic activities](https://www.kaggle.com/discussions/product-feedback/373889) through some simple guidance that could avoid these situations.   This would be extremely low hanging fruit to implement - just some highly visible text on the team page.\n\nMy security experience has taught me that bad/flawed actors will always exist, it's a fundamental law of any complex system.  What's far less practical is trying to rid the multiverse of bad actors than just fixing the system to mitigate their damage.",
      "votes": null
    },
    {
      "id": "2141044",
      "postDate": "02/12/2023 12:03:19",
      "content": "<p><a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> Have you considered account verification, be it via some ID picture or some credit card? Make verification mandatory for any competition medal win for instance.</p>",
      "rawMarkdown": "inversion Have you considered account verification, be it via some ID picture or some credit card? Make verification mandatory for any competition medal win for instance.",
      "votes": null
    },
    {
      "id": "2141232",
      "postDate": "02/12/2023 14:52:40",
      "content": "<p>Lol, someone doesn't want to have his/her account verified. I must be on the right track!</p>",
      "rawMarkdown": "Lol, someone doesn't want to have his/her account verified. I must be on the right track!",
      "votes": null
    },
    {
      "id": "2141792",
      "postDate": "02/13/2023 05:09:07",
      "content": "<p>Thanks for the update on the competition compliance. It would be good to also validate the top scorer's code/model. Keep up the good work!</p>",
      "rawMarkdown": "Thanks for the update on the competition compliance. It would be good to also validate the top scorer's code/model. Keep up the good work!",
      "votes": null
    },
    {
      "id": "2141905",
      "postDate": "02/13/2023 07:06:11",
      "content": "<p>Indeed, there could be a bounty for helping catch cheaters.</p>",
      "rawMarkdown": "Indeed, there could be a bounty for helping catch cheaters.",
      "votes": null
    },
    {
      "id": "2143449",
      "postDate": "02/14/2023 09:14:21",
      "content": "<p>what the competition abot</p>",
      "rawMarkdown": "what the competition abot",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2137246,
      "author_name": "blankaf",
      "author_url": "",
      "post_date": "02/09/2023 20:18:54",
      "content": "<p>A move in the right direction, but for somebody too late, too little. <br>\nWithout a huge public \"uprising\" things would stay as before and another team would be heading to the same bitter fate as Makotu's team in this competition. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2137360,
      "author_name": "mhyodo",
      "author_url": "",
      "post_date": "02/09/2023 22:39:02",
      "content": "<p>Our gold medal is gone, but it is much more important that kaggle is a healthy platform!<br>\nEveryone, be careful when merging teams. It was a good learning experience.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2137407,
          "author_name": "habedi",
          "author_url": "",
          "post_date": "02/10/2023 00:50:11",
          "content": "<p>Sorry to hear about your loss. Nevertheless, thank you and <a href=\"https://www.kaggle.com/bruceqdu\" target=\"_blank\">@bruceqdu</a> for your honesty. It adds much value to the Kaggle community by showing that Kagglers have high ethical standards in addition to exceptional technical knowledge and skills. I wish you all the best.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2137425,
      "author_name": "alvinai9603",
      "author_url": "",
      "post_date": "02/10/2023 01:38:17",
      "content": "<p><a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> Regarding the removal of the team, Our team submit a form to Kaggle compliance a few days ago, Could you get a response and told us the specific reason why we are removed through email? And Which teammate is responsible for it? </p>\n<p>Thanks, we enjoyed this competition very much. And we also work very hard on this comp. In recent days, we feel sad about it and cannot figure out why we are removed. Looking forward to ur reply. 🙏</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2137427,
      "author_name": "shimacos",
      "author_url": "",
      "post_date": "02/10/2023 01:40:53",
      "content": "<p>Our gold medal is lost, but I respect your decision.  It was a very expensive study.</p>\n<p>However, I am concerned that this result will make us timid about blowing the whistle in the future. It would be great to have a system to bail out even during the competition when we have an unintended team merge. I also think that banning obvious cheaters even during competition will reduce the number of unfortunate team merges. I hope that the Kaggle team will properly address these issues from a systems perspective.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2137439,
      "author_name": "hydantess",
      "author_url": "",
      "post_date": "02/10/2023 02:14:46",
      "content": "<p>Great respect to those who pointed them out!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2137445,
      "author_name": "shimacos",
      "author_url": "",
      "post_date": "02/10/2023 02:30:00",
      "content": "<blockquote>\n  <p>A: Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.</p>\n</blockquote>\n<p>By the way, doesn't this case deserve that rare instances? From some of the threads I've seen, I believe that this is a significant incident that has occurred in the past. Grandmaster being banned is a very big incident and there should be a detailed explanation as there has been in <a href=\"https://www.kaggle.com/competitions/petfinder-adoption-prediction/discussion/125436\" target=\"_blank\">the past Petfinder competitions</a>.</p>\n<p>At least our team is a party to this and would like to know more details.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2137455,
          "author_name": "sirius81",
          "author_url": "",
          "post_date": "02/10/2023 02:57:18",
          "content": "<p>Within 24 hours after forming the team, we four draw a line with Poteman and I emailed <a href=\"https://www.kaggle.com/bruceqdu\" target=\"_blank\">@bruceqdu</a> to notice runningZ;<br>\nWithin 48 hours after forming the team, <a href=\"https://www.kaggle.com/mhyodo\" target=\"_blank\">@mhyodo</a> and <a href=\"https://www.kaggle.com/bruceqdu\" target=\"_blank\">@bruceqdu</a> made the public announcements about their cheating.</p>\n<p>To be honest, I am a little bit disappointed.😣</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2137446,
      "author_name": "sirius81",
      "author_url": "",
      "post_date": "02/10/2023 02:37:20",
      "content": "<p>Being removed pains me so much because of the countless hours spent on this competition and we all knowing ourselves' innocence. But it will pain me more if cheaters go unpunished. <br>\nThough I prefer another way of dealing with this thing, I respect your decision. <br>\nLooking forward to joining the next recsys competition.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2137584,
      "author_name": "bakhtiarrasheed",
      "author_url": "",
      "post_date": "02/10/2023 06:52:46",
      "content": "<p>Great work!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2137628,
      "author_name": "onodera",
      "author_url": "",
      "post_date": "02/10/2023 07:40:51",
      "content": "<p>IMHO, It should not be a system where honest people lose out. They sacrificed themselves for the sake of justice. Could kaggle ban the cheaters if that honest people had kept the secret?<br>\n<a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> Do you think there are another compensation?<br>\nPlease take a look into the behavior of honest people</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/381318\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/381318</a></li>\n<li><a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/381321\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/381321</a></li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 2140984,
          "author_name": "kaggleqrdl",
          "author_url": "",
          "post_date": "02/12/2023 10:49:47",
          "content": "<p>This.   We might never have finally developed enough evidence to remove poteman from Kaggle if not for them.  We collectively owe them a debt of gratitude.  And truly, what a tragedy, it must have been heartbreaking.   On top of that, they'll probably develop this massive paranoia about other people.</p>\n<p>Truly, I place the blame squarely on Kaggle for not <a href=\"https://www.kaggle.com/discussions/product-feedback/373889\" target=\"_blank\">normalizing basic activities</a> through some simple guidance that could avoid these situations.   This would be extremely low hanging fruit to implement - just some highly visible text on the team page.</p>\n<p>My security experience has taught me that bad/flawed actors will always exist, it's a fundamental law of any complex system.  What's far less practical is trying to rid the multiverse of bad actors than just fixing the system to mitigate their damage.  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2141905,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "02/13/2023 07:06:11",
          "content": "<p>Indeed, there could be a bounty for helping catch cheaters.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2140653,
      "author_name": "bruceqdu",
      "author_url": "",
      "post_date": "02/12/2023 02:11:22",
      "content": "<p>I suggest kaggle should verify the codes of top-ranked teams (such as gold medal teams or top 50 teams).<br>\nMany data mining competitions held by ACM conferences (such as KDD CUP) or large platform (such as Tianchi) always did this.<br>\nFrom the code verification, kaggle could make sure each team could reproduce their submissions and detect similar and suspicious codes.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2140654,
          "author_name": "onodera",
          "author_url": "",
          "post_date": "02/12/2023 02:21:23",
          "content": "<p>Agree. If top50 teams are too huge to verify, <code>np.random.choice(top_teams, size=int(len(top_teams) * 0.1))</code> would also work</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2140728,
          "author_name": "hydantess",
          "author_url": "",
          "post_date": "02/12/2023 05:14:19",
          "content": "<p>I agree with your suggestion.  It's a heavy work and Kaggle team need to hire more people😂. I also joined KDDCup 2017 once and lots of Tianchi comps. In fact, they didn't check code very carefully. There are many cheaters passed the code verification including two famous ones we know.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2140801,
              "author_name": "bruceqdu",
              "author_url": "",
              "post_date": "02/12/2023 06:43:47",
              "content": "<p>Perhaps kaggle could collect all the codes from top-ranked teams and publish them to all the competitors.<br>\nThe competitors may find cheaters from the codes after a few days.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2141044,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "02/12/2023 12:03:19",
      "content": "<p><a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> Have you considered account verification, be it via some ID picture or some credit card? Make verification mandatory for any competition medal win for instance.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2141232,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "02/12/2023 14:52:40",
          "content": "<p>Lol, someone doesn't want to have his/her account verified. I must be on the right track!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2141792,
      "author_name": "giranntu",
      "author_url": "",
      "post_date": "02/13/2023 05:09:07",
      "content": "<p>Thanks for the update on the competition compliance. It would be good to also validate the top scorer's code/model. Keep up the good work!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2143449,
      "author_name": "chan143",
      "author_url": "",
      "post_date": "02/14/2023 09:14:21",
      "content": "<p>what the competition abot</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2137127": "First and most importantly, thanks to everyone who participated in this exciting and challenging competition! And of course, thanks to the host OTTO for proving such an interesting problem and fantastic dataset for the Kaggle community.\n\nAddressing a few points on the competition compliance, recognizing the uncertainty and waiting has been very frustrating to everyone involved:\n\n**Q:** Why were there changes to the Leaderboard after it was verified?\n\n**A:** There are instances where the decision to remove a team for compliance issues is very straightforward. On occasion, a proper investigation requires significantly more time. Even then, there are times when we uncover additional information after a leaderboard has been verified. In these instances, it is possible that additional teams are removed from the leaderboard after verification.\n\n\n**Q:** Can you provide more details why teams are removed?\n\n**A:** Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.\n\n\n**Q:** What is Kaggle doing to prevent cheating in the future?\n\n**A:** The vast majority of Kagglers abide by competition rules. Unfortunately, the unethical actions of a select few participants degrades the overall experience of a competition. As has always been the case, Kaggle will continue to monitor for actions that do not abide by competition rules and remove teams found to be out of compliance. Repeat offenders will receive account bans. Discovery of unethical behavior (e.g., account selling/buying) will also result in account bans. In addition to these ongoing activities, Kaggle is working on tools and processes to make cheating of any kind more difficult and to improve our detection.\n\nThanks to everyone for being part of this amazing community!",
    "2137246": "A move in the right direction, but for somebody too late, too little. \nWithout a huge public \"uprising\" things would stay as before and another team would be heading to the same bitter fate as Makotu's team in this competition.",
    "2137360": "Our gold medal is gone, but it is much more important that kaggle is a healthy platform!\nEveryone, be careful when merging teams. It was a good learning experience.",
    "2137407": "Sorry to hear about your loss. Nevertheless, thank you and @bruceqdu for your honesty. It adds much value to the Kaggle community by showing that Kagglers have high ethical standards in addition to exceptional technical knowledge and skills. I wish you all the best.",
    "2137425": "inversion Regarding the removal of the team, Our team submit a form to Kaggle compliance a few days ago, Could you get a response and told us the specific reason why we are removed through email? And Which teammate is responsible for it? \n\nThanks, we enjoyed this competition very much. And we also work very hard on this comp. In recent days, we feel sad about it and cannot figure out why we are removed. Looking forward to ur reply. 🙏",
    "2137427": "Our gold medal is lost, but I respect your decision.  It was a very expensive study.\n\nHowever, I am concerned that this result will make us timid about blowing the whistle in the future. It would be great to have a system to bail out even during the competition when we have an unintended team merge. I also think that banning obvious cheaters even during competition will reduce the number of unfortunate team merges. I hope that the Kaggle team will properly address these issues from a systems perspective.",
    "2137439": "Great respect to those who pointed them out!",
    "2137445": "> A: Kaggle does not share the methods we use to detect cheating, nor do we (except in rare instances) publicly comment about individuals or teams.\n\nBy the way, doesn't this case deserve that rare instances? From some of the threads I've seen, I believe that this is a significant incident that has occurred in the past. Grandmaster being banned is a very big incident and there should be a detailed explanation as there has been in [the past Petfinder competitions](https://www.kaggle.com/competitions/petfinder-adoption-prediction/discussion/125436).\n\nAt least our team is a party to this and would like to know more details.",
    "2137446": "Being removed pains me so much because of the countless hours spent on this competition and we all knowing ourselves' innocence. But it will pain me more if cheaters go unpunished. \nThough I prefer another way of dealing with this thing, I respect your decision. \nLooking forward to joining the next recsys competition.",
    "2137455": "Within 24 hours after forming the team, we four draw a line with Poteman and I emailed @bruceqdu to notice runningZ;\nWithin 48 hours after forming the team, @mhyodo and @bruceqdu made the public announcements about their cheating.\n\n\nTo be honest, I am a little bit disappointed.😣",
    "2137584": "Great work!",
    "2137628": "IMHO, It should not be a system where honest people lose out. They sacrificed themselves for the sake of justice. Could kaggle ban the cheaters if that honest people had kept the secret?\n@inversion Do you think there are another compensation?\nPlease take a look into the behavior of honest people\n- https://www.kaggle.com/competitions/otto-recommender-system/discussion/381318\n- https://www.kaggle.com/competitions/otto-recommender-system/discussion/381321",
    "2140653": "I suggest kaggle should verify the codes of top-ranked teams (such as gold medal teams or top 50 teams).\nMany data mining competitions held by ACM conferences (such as KDD CUP) or large platform (such as Tianchi) always did this.\nFrom the code verification, kaggle could make sure each team could reproduce their submissions and detect similar and suspicious codes.",
    "2140654": "Agree. If top50 teams are too huge to verify, `np.random.choice(top_teams, size=int(len(top_teams) * 0.1))` would also work",
    "2140728": "I agree with your suggestion.  It's a heavy work and Kaggle team need to hire more people😂. I also joined KDDCup 2017 once and lots of Tianchi comps. In fact, they didn't check code very carefully. There are many cheaters passed the code verification including two famous ones we know.",
    "2140801": "Perhaps kaggle could collect all the codes from top-ranked teams and publish them to all the competitors.\nThe competitors may find cheaters from the codes after a few days.",
    "2140984": "This.   We might never have finally developed enough evidence to remove poteman from Kaggle if not for them.  We collectively owe them a debt of gratitude.  And truly, what a tragedy, it must have been heartbreaking.   On top of that, they'll probably develop this massive paranoia about other people.\n\nTruly, I place the blame squarely on Kaggle for not [normalizing basic activities](https://www.kaggle.com/discussions/product-feedback/373889) through some simple guidance that could avoid these situations.   This would be extremely low hanging fruit to implement - just some highly visible text on the team page.\n\nMy security experience has taught me that bad/flawed actors will always exist, it's a fundamental law of any complex system.  What's far less practical is trying to rid the multiverse of bad actors than just fixing the system to mitigate their damage.",
    "2141044": "inversion Have you considered account verification, be it via some ID picture or some credit card? Make verification mandatory for any competition medal win for instance.",
    "2141232": "Lol, someone doesn't want to have his/her account verified. I must be on the right track!",
    "2141792": "Thanks for the update on the competition compliance. It would be good to also validate the top scorer's code/model. Keep up the good work!",
    "2141905": "Indeed, there could be a bounty for helping catch cheaters.",
    "2143449": "what the competition abot"
  },
  "source": "meta"
}