Hi,
\nI want to suggest an improvement with the Kaggle cheat detection system. My team (hungry for gold, 16th preliminar place) was dropped from the leaderboard because one of the members used two accounts at the beginning of the competition (before anyone joined the team).
\nThis situation was very frustrating for me because I worked really hard, spent a lot of time (all my free time was dedicated to this) and also money (paying for colab pro subscription and extra GPUs on GCP).
\nMy point is, this situation can be totally avoided by dropping cheaters earlier. Right now Kaggle waits until the competition ends to drop people that used two accounts. It is necessary to wait until this point? I think not, it shouldn't be difficult to identify and drop them earlier.
\nThe most worrying thing is that my teammate that made the mistake was not a novice -> joining a team with experienced people is not a guarantee of not being dropped in the end.
\nWith the current cheat detection system I think it is really really risky to join a team.
\nHope @inversion and @juliaelliott can read this and think about it.
", "messages": [ { "id": "1315544", "postDate": "05/19/2021 22:20:37", "content": "Hi,
\nI want to suggest an improvement with the Kaggle cheat detection system. My team (hungry for gold, 16th preliminar place) was dropped from the leaderboard because one of the members used two accounts at the beginning of the competition (before anyone joined the team).
\nThis situation was very frustrating for me because I worked really hard, spent a lot of time (all my free time was dedicated to this) and also money (paying for colab pro subscription and extra GPUs on GCP).
\nMy point is, this situation can be totally avoided by dropping cheaters earlier. Right now Kaggle waits until the competition ends to drop people that used two accounts. It is necessary to wait until this point? I think not, it shouldn't be difficult to identify and drop them earlier.
\nThe most worrying thing is that my teammate that made the mistake was not a novice -> joining a team with experienced people is not a guarantee of not being dropped in the end.
\nWith the current cheat detection system I think it is really really risky to join a team.
\nHope @inversion and @juliaelliott can read this and think about it.
", "rawMarkdown": "Hi,\n\nI want to suggest an improvement with the Kaggle cheat detection system. My team (hungry for gold, 16th preliminar place) was dropped from the leaderboard because one of the members used two accounts at the beginning of the competition (before anyone joined the team). \n\nThis situation was very frustrating for me because I worked really hard, spent a lot of time (all my free time was dedicated to this) and also money (paying for colab pro subscription and extra GPUs on GCP).\n\nMy point is, this situation can be totally avoided by dropping cheaters earlier. Right now Kaggle waits until the competition ends to drop people that used two accounts. It is necessary to wait until this point? I think not, it shouldn't be difficult to identify and drop them earlier.\n\nThe most worrying thing is that my teammate that made the mistake was not a novice -> joining a team with experienced people is not a guarantee of not being dropped in the end. \n\nWith the current cheat detection system I think it is really really risky to join a team. \n\nHope @inversion and @juliaelliott can read this and think about it.", "votes": null }, { "id": "1316097", "postDate": "05/20/2021 09:50:15", "content": "@mavillan I have lots of sympathy with your situation, similar to a runner who spends four years training for the Olympic relay and then loses their medal because a teammate took drugs. I note also that there's a similar post on the RPS forum to your previous one on here, from a competitor who seems to have lost a medal retrospectively without much explanation (at least prior to their posting). While I can't comment on the handling of it since I don't know when the transgression came to light, I am very sorry that this happened to you and the disqualification is no reflection on your fine efforts in this competition.
", "rawMarkdown": "mavillan I have lots of sympathy with your situation, similar to a runner who spends four years training for the Olympic relay and then loses their medal because a teammate took drugs. I note also that there's a [similar post on the RPS forum](https://www.kaggle.com/c/rock-paper-scissors/discussion/235791) to your previous one on here, from a competitor who seems to have lost a medal retrospectively without much explanation (at least prior to their posting). While I can't comment on the handling of it since I don't know when the transgression came to light, I am very sorry that this happened to you and the disqualification is no reflection on your fine efforts in this competition.", "votes": null }, { "id": "1316496", "postDate": "05/20/2021 15:41:32", "content": "I really feel your pain. The risk is just too big for me. You'll be surprised that masters and GMs can just cheat as well. Most of the time I only consider teaming up with someone I know or someone with a good reputation. I'm truly sorry for you. Kaggle should have done a better job. I know it is challenging for them and they might not have enough manpower but something should be done.
", "rawMarkdown": "I really feel your pain. The risk is just too big for me. You'll be surprised that masters and GMs can just cheat as well. Most of the time I only consider teaming up with someone I know or someone with a good reputation. I'm truly sorry for you. Kaggle should have done a better job. I know it is challenging for them and they might not have enough manpower but something should be done.", "votes": null }, { "id": "1316528", "postDate": "05/20/2021 16:19:26", "content": "I am a witness of Mavillan's work , he had done commendable work not only with building models but also with exploring predictions in order to find potential leaks . I really feel for my awesome teammates here , the 16th place solution was really unique and Kaggle and the whole community would have benifitted from it but now it will go unknown.
", "rawMarkdown": "I am a witness of Mavillan's work , he had done commendable work not only with building models but also with exploring predictions in order to find potential leaks . I really feel for my awesome teammates here , the 16th place solution was really unique and Kaggle and the whole community would have benifitted from it but now it will go unknown.", "votes": null }, { "id": "1316915", "postDate": "05/21/2021 02:47:10", "content": "Thanks for the feedback. I'm the the one who have been dropped from the leaderboard too (the final ranking is 20/11xx), despite of very hard work. The reason is also be using two account from the beginning, but I really don't know that is prohibited until the final week of competition because I'm a very beginner. But if kaggle staff drop me earlier, or send me something like warning message, I would'n waste my efforts. Anw, thanks for the very very interesting competition !
", "rawMarkdown": "Thanks for the feedback. I'm the the one who have been dropped from the leaderboard too (the final ranking is 20/11xx), despite of very hard work. The reason is also be using two account from the beginning, but I really don't know that is prohibited until the final week of competition because I'm a very beginner. But if kaggle staff drop me earlier, or send me something like warning message, I would'n waste my efforts. Anw, thanks for the very very interesting competition !", "votes": null }, { "id": "1317246", "postDate": "05/21/2021 08:48:44", "content": "It happens frequently and I believe Kaggle's said (or implied) that they apply the anti-cheating measures at the end of competitions to prevent cheaters from finding work-arounds. That makes sense.
\nWhat I've found work (admittedly with a small sample of competitions) is to be up-front when forming a team with strangers and explictly ask every member - \"Could you please confirm that you're not engaging in anything that might cause the team to be disqualified e.g., duplicate accounts?\").
\nI think barring hardened cheaters, most people would be honest about this.
", "rawMarkdown": "It happens frequently and I believe Kaggle's said (or implied) that they apply the anti-cheating measures at the end of competitions to prevent cheaters from finding work-arounds. That makes sense. \n\nWhat I've found work (admittedly with a small sample of competitions) is to be up-front when forming a team with strangers and explictly ask every member - \"Could you please confirm that you're not engaging in anything that might cause the team to be disqualified e.g., duplicate accounts?\").\n\nI think barring hardened cheaters, most people would be honest about this.", "votes": null }, { "id": "1317346", "postDate": "05/21/2021 10:47:20", "content": "If one plays the game fairly, they don't need to check the rules. Could you answer the following questions?
\nI have experienced this before and lost around 2 silver medals because of some team members having duplicate accounts. I understand your pain, but at the same time I feel Kaggle cant identify those accounts before the competition ends. you are putting a lot on the team that is running and managing multiple competitions at the same time. Just be more cautious about whom you merge with, in future. Good luck!
", "rawMarkdown": "I have experienced this before and lost around 2 silver medals because of some team members having duplicate accounts. I understand your pain, but at the same time I feel Kaggle cant identify those accounts before the competition ends. you are putting a lot on the team that is running and managing multiple competitions at the same time. Just be more cautious about whom you merge with, in future. Good luck!", "votes": null }, { "id": "1325152", "postDate": "05/27/2021 14:32:04", "content": "Yeah plus even when we know that someone is cheating, we can not un-merge all efforts of every single member goes waste. Kaggle should introduce a concept of un-merging.
", "rawMarkdown": "Yeah plus even when we know that someone is cheating, we can not **un-merge** all efforts of every single member goes waste. Kaggle should introduce a concept of un-merging.", "votes": null }, { "id": "1327092", "postDate": "05/28/2021 22:18:30", "content": "I don't think un-merge is feasible. Once you join a team, you have access to all their submissions and private content. It will not be fair for anyone if some team member un-merges having had access to all that information.
\nThis is way I think early dropping of cheaters is the best option to solve this problem.
", "rawMarkdown": "I don't think un-merge is feasible. Once you join a team, you have access to all their submissions and private content. It will not be fair for anyone if some team member un-merges having had access to all that information.\n\nThis is way I think early dropping of cheaters is the best option to solve this problem.", "votes": null }, { "id": "1327094", "postDate": "05/28/2021 22:23:16", "content": "I know it is not easy to implement, but it is not something extremely complicated either. It seems that the process of identifying cheaters is already automated (it only took a few minutes in this competition). If you run this same process once a day, it should be enough.
", "rawMarkdown": "I know it is not easy to implement, but it is not something extremely complicated either. It seems that the process of identifying cheaters is already automated (it only took a few minutes in this competition). If you run this same process once a day, it should be enough.", "votes": null }, { "id": "1327105", "postDate": "05/28/2021 22:55:11", "content": "I understand your part - it is automated, only when the leaderboard is finalized.
", "rawMarkdown": "I understand your part - it is automated, only when the leaderboard is finalized.", "votes": null }, { "id": "1327107", "postDate": "05/28/2021 23:00:53", "content": "One of the reasons, people create multiple accounts - is to get more freedom on the number of the submissions. You have double the number of submissions, - that is double a chance to move up on the leaderboard. This is really unfair to the people, who spend hours in getting those top positions.
", "rawMarkdown": "One of the reasons, people create multiple accounts - is to get more freedom on the number of the submissions. You have double the number of submissions, - that is double a chance to move up on the leaderboard. This is really unfair to the people, who spend hours in getting those top positions.", "votes": null } ], "comments": [ { "id": 1316097, "author_name": "jbomitchell", "author_url": "", "post_date": "05/20/2021 09:50:15", "content": "@mavillan I have lots of sympathy with your situation, similar to a runner who spends four years training for the Olympic relay and then loses their medal because a teammate took drugs. I note also that there's a similar post on the RPS forum to your previous one on here, from a competitor who seems to have lost a medal retrospectively without much explanation (at least prior to their posting). While I can't comment on the handling of it since I don't know when the transgression came to light, I am very sorry that this happened to you and the disqualification is no reflection on your fine efforts in this competition.
", "votes": null, "replies": [] }, { "id": 1316496, "author_name": "jiweiliu", "author_url": "", "post_date": "05/20/2021 15:41:32", "content": "I really feel your pain. The risk is just too big for me. You'll be surprised that masters and GMs can just cheat as well. Most of the time I only consider teaming up with someone I know or someone with a good reputation. I'm truly sorry for you. Kaggle should have done a better job. I know it is challenging for them and they might not have enough manpower but something should be done.
", "votes": null, "replies": [] }, { "id": 1316528, "author_name": "tanulsingh077", "author_url": "", "post_date": "05/20/2021 16:19:26", "content": "I am a witness of Mavillan's work , he had done commendable work not only with building models but also with exploring predictions in order to find potential leaks . I really feel for my awesome teammates here , the 16th place solution was really unique and Kaggle and the whole community would have benifitted from it but now it will go unknown.
", "votes": null, "replies": [] }, { "id": 1316915, "author_name": "nocturnebflat123", "author_url": "", "post_date": "05/21/2021 02:47:10", "content": "Thanks for the feedback. I'm the the one who have been dropped from the leaderboard too (the final ranking is 20/11xx), despite of very hard work. The reason is also be using two account from the beginning, but I really don't know that is prohibited until the final week of competition because I'm a very beginner. But if kaggle staff drop me earlier, or send me something like warning message, I would'n waste my efforts. Anw, thanks for the very very interesting competition !
", "votes": null, "replies": [ { "id": 1317346, "author_name": "jiweiliu", "author_url": "", "post_date": "05/21/2021 10:47:20", "content": "If one plays the game fairly, they don't need to check the rules. Could you answer the following questions?
\nOne of the reasons, people create multiple accounts - is to get more freedom on the number of the submissions. You have double the number of submissions, - that is double a chance to move up on the leaderboard. This is really unfair to the people, who spend hours in getting those top positions.
", "votes": null, "replies": [] } ] }, { "id": 1317246, "author_name": "leecming", "author_url": "", "post_date": "05/21/2021 08:48:44", "content": "It happens frequently and I believe Kaggle's said (or implied) that they apply the anti-cheating measures at the end of competitions to prevent cheaters from finding work-arounds. That makes sense.
\nWhat I've found work (admittedly with a small sample of competitions) is to be up-front when forming a team with strangers and explictly ask every member - \"Could you please confirm that you're not engaging in anything that might cause the team to be disqualified e.g., duplicate accounts?\").
\nI think barring hardened cheaters, most people would be honest about this.
", "votes": null, "replies": [] }, { "id": 1321567, "author_name": "roydatascience", "author_url": "", "post_date": "05/24/2021 19:01:05", "content": "I have experienced this before and lost around 2 silver medals because of some team members having duplicate accounts. I understand your pain, but at the same time I feel Kaggle cant identify those accounts before the competition ends. you are putting a lot on the team that is running and managing multiple competitions at the same time. Just be more cautious about whom you merge with, in future. Good luck!
", "votes": null, "replies": [ { "id": 1327094, "author_name": "mavillan", "author_url": "", "post_date": "05/28/2021 22:23:16", "content": "I know it is not easy to implement, but it is not something extremely complicated either. It seems that the process of identifying cheaters is already automated (it only took a few minutes in this competition). If you run this same process once a day, it should be enough.
", "votes": null, "replies": [] }, { "id": 1327105, "author_name": "roydatascience", "author_url": "", "post_date": "05/28/2021 22:55:11", "content": "I understand your part - it is automated, only when the leaderboard is finalized.
", "votes": null, "replies": [] } ] }, { "id": 1325152, "author_name": "rsinda", "author_url": "", "post_date": "05/27/2021 14:32:04", "content": "Yeah plus even when we know that someone is cheating, we can not un-merge all efforts of every single member goes waste. Kaggle should introduce a concept of un-merging.
", "votes": null, "replies": [ { "id": 1327092, "author_name": "mavillan", "author_url": "", "post_date": "05/28/2021 22:18:30", "content": "I don't think un-merge is feasible. Once you join a team, you have access to all their submissions and private content. It will not be fair for anyone if some team member un-merges having had access to all that information.
\nThis is way I think early dropping of cheaters is the best option to solve this problem.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1315544": "Hi,\n\nI want to suggest an improvement with the Kaggle cheat detection system. My team (hungry for gold, 16th preliminar place) was dropped from the leaderboard because one of the members used two accounts at the beginning of the competition (before anyone joined the team). \n\nThis situation was very frustrating for me because I worked really hard, spent a lot of time (all my free time was dedicated to this) and also money (paying for colab pro subscription and extra GPUs on GCP).\n\nMy point is, this situation can be totally avoided by dropping cheaters earlier. Right now Kaggle waits until the competition ends to drop people that used two accounts. It is necessary to wait until this point? I think not, it shouldn't be difficult to identify and drop them earlier.\n\nThe most worrying thing is that my teammate that made the mistake was not a novice -> joining a team with experienced people is not a guarantee of not being dropped in the end. \n\nWith the current cheat detection system I think it is really really risky to join a team. \n\nHope @inversion and @juliaelliott can read this and think about it.", "1316097": "mavillan I have lots of sympathy with your situation, similar to a runner who spends four years training for the Olympic relay and then loses their medal because a teammate took drugs. I note also that there's a [similar post on the RPS forum](https://www.kaggle.com/c/rock-paper-scissors/discussion/235791) to your previous one on here, from a competitor who seems to have lost a medal retrospectively without much explanation (at least prior to their posting). While I can't comment on the handling of it since I don't know when the transgression came to light, I am very sorry that this happened to you and the disqualification is no reflection on your fine efforts in this competition.", "1316496": "I really feel your pain. The risk is just too big for me. You'll be surprised that masters and GMs can just cheat as well. Most of the time I only consider teaming up with someone I know or someone with a good reputation. I'm truly sorry for you. Kaggle should have done a better job. I know it is challenging for them and they might not have enough manpower but something should be done.", "1316528": "I am a witness of Mavillan's work , he had done commendable work not only with building models but also with exploring predictions in order to find potential leaks . I really feel for my awesome teammates here , the 16th place solution was really unique and Kaggle and the whole community would have benifitted from it but now it will go unknown.", "1316915": "Thanks for the feedback. I'm the the one who have been dropped from the leaderboard too (the final ranking is 20/11xx), despite of very hard work. The reason is also be using two account from the beginning, but I really don't know that is prohibited until the final week of competition because I'm a very beginner. But if kaggle staff drop me earlier, or send me something like warning message, I would'n waste my efforts. Anw, thanks for the very very interesting competition !", "1317246": "It happens frequently and I believe Kaggle's said (or implied) that they apply the anti-cheating measures at the end of competitions to prevent cheaters from finding work-arounds. That makes sense. \n\nWhat I've found work (admittedly with a small sample of competitions) is to be up-front when forming a team with strangers and explictly ask every member - \"Could you please confirm that you're not engaging in anything that might cause the team to be disqualified e.g., duplicate accounts?\").\n\nI think barring hardened cheaters, most people would be honest about this.", "1317346": "If one plays the game fairly, they don't need to check the rules. Could you answer the following questions?\n- why did you create two accounts and submit with two accounts in the first place?\n- have you wondered if that was unfair for people with one account?\n- why didn't you ask in the forum if that was allowed?", "1321567": "I have experienced this before and lost around 2 silver medals because of some team members having duplicate accounts. I understand your pain, but at the same time I feel Kaggle cant identify those accounts before the competition ends. you are putting a lot on the team that is running and managing multiple competitions at the same time. Just be more cautious about whom you merge with, in future. Good luck!", "1325152": "Yeah plus even when we know that someone is cheating, we can not **un-merge** all efforts of every single member goes waste. Kaggle should introduce a concept of un-merging.", "1327092": "I don't think un-merge is feasible. Once you join a team, you have access to all their submissions and private content. It will not be fair for anyone if some team member un-merges having had access to all that information.\n\nThis is way I think early dropping of cheaters is the best option to solve this problem.", "1327094": "I know it is not easy to implement, but it is not something extremely complicated either. It seems that the process of identifying cheaters is already automated (it only took a few minutes in this competition). If you run this same process once a day, it should be enough.", "1327105": "I understand your part - it is automated, only when the leaderboard is finalized.", "1327107": "One of the reasons, people create multiple accounts - is to get more freedom on the number of the submissions. You have double the number of submissions, - that is double a chance to move up on the leaderboard. This is really unfair to the people, who spend hours in getting those top positions." }, "source": "meta" }