{ "id": 381495, "title": "【Maybe massive cheating】 Weird LB changes around silver", "url": "/competitions/otto-recommender-system/discussion/381495", "author_name": "wakaka", "post_date": "2023-01-27T00:22:59.073000", "votes": 24, "comment_count": 28, "views": 0, "content": "

Hi all,

\n

This may really just be my imagination, but I'm just a little curious, so I'm posting this.

\n

I feel like there are suddenly more Contributer teams(most of them are composed of 4) around 80~100 in the silver rank, I don't remember what teams were in that rank 3 or 4 days ago so if they were in that rank originally, I will delete this post. However, in a previous competition (Open Problems - Multimodal Single-Cell Integration), we saw a large number of Contributer teams suddenly appear in the medal standings just before the end of the competition, and we are concerned that something similar may happen. I fear that a similar thing will happen (https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313).

\n

Of course, I understand that it is possible to legitimately reach the silver range of LB suddenly. But I have made this discussion because I find it odd that these Contributer teams are clumped together in similar rankings…

\n

\"\"

", "messages": [ { "id": 2117072, "postDate": "2023-01-27T00:22:59.073Z", "content": "

Hi all,

\n

This may really just be my imagination, but I'm just a little curious, so I'm posting this.

\n

I feel like there are suddenly more Contributer teams(most of them are composed of 4) around 80~100 in the silver rank, I don't remember what teams were in that rank 3 or 4 days ago so if they were in that rank originally, I will delete this post. However, in a previous competition (Open Problems - Multimodal Single-Cell Integration), we saw a large number of Contributer teams suddenly appear in the medal standings just before the end of the competition, and we are concerned that something similar may happen. I fear that a similar thing will happen (https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313).

\n

Of course, I understand that it is possible to legitimately reach the silver range of LB suddenly. But I have made this discussion because I find it odd that these Contributer teams are clumped together in similar rankings…

\n

\"\"

", "rawMarkdown": "Hi all, \n\nThis may really just be my imagination, but I'm just a little curious, so I'm posting this.\n\nI feel like there are suddenly more Contributer teams(most of them are composed of 4) around 80~100 in the silver rank, I don't remember what teams were in that rank 3 or 4 days ago so if they were in that rank originally, I will delete this post. However, in a previous competition (Open Problems - Multimodal Single-Cell Integration), we saw a large number of Contributer teams suddenly appear in the medal standings just before the end of the competition, and we are concerned that something similar may happen. I fear that a similar thing will happen (https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313). \n\nOf course, I understand that it is possible to legitimately reach the silver range of LB suddenly. But I have made this discussion because I find it odd that these Contributer teams are clumped together in similar rankings...\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F11991501%2Fd7967b50d12f3c510bf0578c05ac3bbb%2F(91).png?generation=1674790349676747&alt=media)", "votes": 24 }, { "id": 2122023, "postDate": "2023-01-30T17:03:23.847Z", "content": "

In the previous competition (multimodal cell) about 1/3 of teams in medals were disqualified. I guess something like that is going to happen this competition. Just have some patience.

\n

I've moved from 62nd position to current 100th in a few days (hope to upload better results in a few hours when notebook finishes).

", "rawMarkdown": "In the previous competition (multimodal cell) about 1/3 of teams in medals were disqualified. I guess something like that is going to happen this competition. Just have some patience.\n\nI've moved from 62nd position to current 100th in a few days (hope to upload better results in a few hours when notebook finishes).", "votes": 8, "replies": [ { "id": 2122199, "postDate": "2023-01-30T18:59:47.347Z", "content": "

damn, one third… so this is a very serious and organized attack on kaggle's reputation and medals value

", "rawMarkdown": "damn, one third... so this is a very serious and organized attack on kaggle's reputation and medals value", "replies": [ { "id": 2122437, "postDate": "2023-01-30T23:56:59.120Z", "content": "

I moved from 123rd place to 88th after results were finalized and cheaters removed.
\nThe fact there are cheaters proves that medals are viewed as important - otherwise noone would bother. I guess organizers are doing pretty good job finding the cheaters.
\nAnd there are of course cases when people break the rules because they misunderstand them, like sharing code before accepting someone into the team or some teams get removed because of a single member.
\nIn cases like a real organized cheating, like buying/selling results maybe real names of cheaters should be published - this case, being caught would bring much bigger problems than possible profit from getting that medal.

", "rawMarkdown": "I moved from 123rd place to 88th after results were finalized and cheaters removed.\nThe fact there are cheaters proves that medals are viewed as important - otherwise noone would bother. I guess organizers are doing pretty good job finding the cheaters.\nAnd there are of course cases when people break the rules because they misunderstand them, like sharing code before accepting someone into the team or some teams get removed because of a single member. \nIn cases like a real organized cheating, like buying/selling results maybe real names of cheaters should be published - this case, being caught would bring much bigger problems than possible profit from getting that medal.", "votes": 2 } ] } ] }, { "id": 2117759, "postDate": "2023-01-27T14:17:00.473Z", "content": "

Even though I haven't been coming to kaggle for long, I have so far observed the buying and selling of silver medals in almost all competitions. In the G2Net competition, there were also several suspicious teams in the silver medal zone in the last few days, and they exactly fit the description here).

\n

This is not something that can be easily solved, unless the kaggle team really takes this issue seriously and decides to put a lot of effort into it on a consistent basis. I and others have made some suggestions in previous posts, but haven't seen any substantial action from the kaggle team.

\n

In fact, even after this competition, the kaggle team conducted a pretty long investigation, and when you go back to LB now, you can still identify several teams that are clearly suspect. They still won the silver medal as expected. As you can see, even clearly distinguishing cheating teams is a very difficult task.

\n

To be honest, I am disappointed with this phenomenon and the results. But the world is just not perfect and sometimes we just have to accept it, put our heads down and keep on doing the right thing, we can still keep learning on kaggle and moving towards the master and grandmaster.🙂

", "rawMarkdown": "Even though I haven't been coming to kaggle for long, I have so far observed the buying and selling of silver medals in almost all competitions. In the [G2Net competition](https://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves/leaderboard), there were also several suspicious teams in the silver medal zone in the last few days, and they exactly fit the description [here](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313)). \n\nThis is not something that can be easily solved, unless the kaggle team really takes this issue seriously and decides to put a lot of effort into it on a consistent basis. I and others have made some [suggestions](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313#2033961) in previous posts, but haven't seen any substantial action from the kaggle team.\n\nIn fact, even after this competition, the kaggle team conducted a pretty long investigation, and when you go back to [LB](https://www.kaggle.com/competitions/open-problems-multimodal/leaderboard) now, you can still identify several teams that are clearly suspect. They still won the silver medal as expected. As you can see, even clearly distinguishing cheating teams is a very difficult task.\n\nTo be honest, I am disappointed with this phenomenon and the results. But the world is just not perfect and sometimes we just have to accept it, put our heads down and keep on doing the right thing, we can still keep learning on kaggle and moving towards the master and grandmaster.🙂", "votes": 8, "replies": [ { "id": 2117914, "postDate": "2023-01-27T16:29:19.647Z", "content": "
\n

haven't seen any substantial action from the kaggle team.

\n
\n

We take this seriously and have been working behind the scenes on processes/projects to combat this. These projects take time, necessarily have to happen in private, and have to be balanced with other priorities. While I know it may be frustrating to not see immediate drastic changes, it's not a sign that we are ignoring this.

\n
\n

you can still identify several teams that are clearly suspect

\n
\n

We are careful to use a standard of evidence that goes beyond merely looking suspect (the harm of removing a legitimate team is a much worse outcome than leaving a cheating team). Thankfully, cheating by sharing solutions or using multiple accounts is a shortsighted move that does not lead to knowledge & growth, so while these people may gain a short-term boost in stature, it's not a sustainable path to doing better.

\n
\n

I am disappointed with this phenomenon and the results. But the world is just not perfect and sometimes we just have to accept it, put our heads down and keep on doing the right thing

\n
\n

Well said! This attitude is what makes this community great, and the reason we can keep doing these in spite of a few bad apples.

", "rawMarkdown": "> haven't seen any substantial action from the kaggle team.\n\nWe take this seriously and have been working behind the scenes on processes/projects to combat this. These projects take time, necessarily have to happen in private, and have to be balanced with other priorities. While I know it may be frustrating to not see immediate drastic changes, it's not a sign that we are ignoring this.\n\n> you can still identify several teams that are clearly suspect\n\nWe are careful to use a standard of evidence that goes beyond merely looking suspect (the harm of removing a legitimate team is a much worse outcome than leaving a cheating team). Thankfully, cheating by sharing solutions or using multiple accounts is a shortsighted move that does not lead to knowledge & growth, so while these people may gain a short-term boost in stature, it's not a sustainable path to doing better.\n\n> I am disappointed with this phenomenon and the results. But the world is just not perfect and sometimes we just have to accept it, put our heads down and keep on doing the right thing\n\nWell said! This attitude is what makes this community great, and the reason we can keep doing these in spite of a few bad apples.", "votes": 27, "replies": [ { "id": 2118119, "postDate": "2023-01-27T19:27:41.910Z", "content": "

There are some very low hanging fruit things Kaggle could do.

\n

eg: On the team page, just explain/normalize what is expected before teaming with a stranger.

\n

One baseline might be - Get on a zoom call to talk about technical experience and discuss expectations around code sharing and how the team will operate.

\n

This would cut down significantly on the biggest and most thorniest issue - which is prize winning teams getting disqualified because they were too trusting.

\n

https://kaggle.com/discussions/product-feedback/373889#2118013

", "rawMarkdown": "There are some very low hanging fruit things Kaggle could do. \n\neg: On the team page, just explain/normalize what is expected before teaming with a stranger.\n\nOne baseline might be - Get on a zoom call to talk about technical experience and discuss expectations around code sharing and how the team will operate.\n\nThis would cut down significantly on the biggest and most thorniest issue - which is prize winning teams getting disqualified because they were too trusting.\n\nhttps://kaggle.com/discussions/product-feedback/373889#2118013\n", "votes": 3 }, { "id": 2118702, "postDate": "2023-01-28T08:04:31.363Z", "content": "

@wcukierski
\nCant you make a statement like this in advance??
\n\"We will remove the questionable team from the LB once. If you later submit the code and prove your innocence, we will put you back on the finalized version of the LB. We are sorry, but we need your cooperation\"

", "rawMarkdown": "@wcukierski \nCant you make a statement like this in advance??\n\"We will remove the questionable team from the LB once. If you later submit the code and prove your innocence, we will put you back on the finalized version of the LB. We are sorry, but we need your cooperation\"", "votes": 1 }, { "id": 2121889, "postDate": "2023-01-30T16:07:54.447Z", "content": "

@wcukierski

\n
\n

have to be balanced with other priorities

\n
\n

Could you tell us, please, what can be of a higher priority than keeping competitions fair and with equal chances for all participants?

", "rawMarkdown": "@wcukierski\n>have to be balanced with other priorities\n\nCould you tell us, please, what can be of a **higher priority** than keeping competitions fair and with equal chances for all participants?", "votes": 3 } ] }, { "id": 2118353, "postDate": "2023-01-28T00:39:32.220Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 2118756, "postDate": "2023-01-28T08:50:36.257Z", "content": "

Indeed, it seems that the lower area of the silver medals has been quickly invaded… which has been more than less common and very, very noticeable in recent months…

", "rawMarkdown": "Indeed, it seems that the lower area of the silver medals has been quickly invaded... which has been more than less common and very, very noticeable in recent months...", "votes": 6, "replies": [ { "id": 2123779, "postDate": "2023-01-31T16:59:22.327Z", "content": "

I think this also happened in https://www.kaggle.com/competitions/santa-2022. Because in the final weeks of the competition, many new competitors appeared out of nowhere and got into medal tiers.

", "rawMarkdown": "I think this also happened in https://www.kaggle.com/competitions/santa-2022. Because in the final weeks of the competition, many new competitors appeared out of nowhere and got into medal tiers. " } ] }, { "id": 2122661, "postDate": "2023-01-31T04:41:30.577Z", "content": "

Echoing the same thought
\nI've spent 2 months in the competition and was in the 84th for a month.
\nWithin a week I've moved from 84th to 148th.

\n

Given the fact that until 10 Jan there were still few teams that surpassed the \"big wall\" LB 0.579,
\nIt's very hard to believe some new joiners with only FEW DAYS can achieve LB > 0.579

\n

attaching analysis from @tatamikenn
\n\"Big wall in LB: only 4.12% of teams scored >= 0.579\": https://www.kaggle.com/competitions/otto-recommender-system/discussion/377149

", "rawMarkdown": "Echoing the same thought\nI've spent 2 months in the competition and was in the 84th for a month. \nWithin a week I've moved from 84th to 148th.\n\nGiven the fact that until 10 Jan there were still few teams that surpassed the \"big wall\" LB 0.579,\nIt's very hard to believe some new joiners with only FEW DAYS can achieve LB > 0.579\n\nattaching analysis from @tatamikenn \n\"Big wall in LB: only 4.12% of teams scored >= 0.579\": https://www.kaggle.com/competitions/otto-recommender-system/discussion/377149", "votes": 3 }, { "id": 2122641, "postDate": "2023-01-31T04:28:00.247Z", "content": "

For each user, we could only submit up to 5 submissions per day, right? When I checked some users' profile, I accidentally found one user(lkkcharlie0126) submit 42 submissions per day! And has submitted more than 5 times multiple days for this competition since this is the only competition that user participated.

\n

Is there any exceptions that you could submit >5 times, if not where should I report this wired case to?

", "rawMarkdown": "For each user, we could only submit up to 5 submissions per day, right? When I checked some users' profile, I accidentally found one user(lkkcharlie0126) submit 42 submissions per day! And has submitted more than 5 times multiple days for this competition since this is the only competition that user participated. \n\nIs there any exceptions that you could submit >5 times, if not where should I report this wired case to? ", "votes": 4, "replies": [ { "id": 2123537, "postDate": "2023-01-31T15:13:04.733Z", "content": "
\n

I accidentally found one user(lkkcharlie0126) submit 42 submissions per day! And has submitted more than 5 times multiple days for this competition since this is the only competition that user participated.

\n
\n

On what are you basing this claim? I reviewed this team and found no instances of exceeding the daily limits. In fact, our on-call will get paged if any team on any competition were to violate a daily limit.

", "rawMarkdown": "> I accidentally found one user(lkkcharlie0126) submit 42 submissions per day! And has submitted more than 5 times multiple days for this competition since this is the only competition that user participated.\n\nOn what are you basing this claim? I reviewed this team and found no instances of exceeding the daily limits. In fact, our on-call will get paged if any team on any competition were to violate a daily limit.", "votes": 1, "replies": [ { "id": 2123716, "postDate": "2023-01-31T16:35:28.220Z", "content": "

\"From

", "rawMarkdown": "![From the user's profile? Not sure if it's the website problem](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12423127%2F4721244b8df85b297d3afd8c4b9fe7a8%2FScreen%20Shot%202023-01-30%20at%2021.14.46.png?generation=1675182884222416&alt=media)" }, { "id": 2123719, "postDate": "2023-01-31T16:36:00.853Z", "content": "

From the user profile? Nor sure if it's the website problem though

", "rawMarkdown": "From the user profile? Nor sure if it's the website problem though" }, { "id": 2123770, "postDate": "2023-01-31T16:54:21.410Z", "content": "

@wcukierski the last finished competition was Santa 2022 which allowed 100 submissions per day.

\n

With a nearly traditional Kaggle staff unaibility to keep competitions fair and transparent I wouldn't be wondering if somewhere deep in the system the number of allowed submissions per day wouldn't have been reset to five.
\nOf course, you will never admit any technical mistake.

", "rawMarkdown": "@wcukierski the last finished competition was Santa 2022 which allowed 100 submissions per day.\n\nWith a nearly traditional Kaggle staff unaibility to keep competitions fair and transparent I wouldn't be wondering if somewhere deep in the system the number of allowed submissions per day wouldn't have been reset to five.\nOf course, you will never admit any technical mistake. ", "votes": 1 }, { "id": 2123875, "postDate": "2023-01-31T17:43:51.707Z", "content": "

@sherrypomi that activity graph includes submissions to closed competitions (which do not show up as having been joined on a user's profile).

", "rawMarkdown": "@sherrypomi that activity graph includes submissions to closed competitions (which do not show up as having been joined on a user's profile)." }, { "id": 2123915, "postDate": "2023-01-31T18:11:29.020Z", "content": "

@wcukierski

\n
\n

… submissions to closed competitions (which do not show up as having been joined on a user's profile)

\n
\n

As usual, nicely put words but unfortunately saying nothing.
\nCan you list the competitions? Or are they secret?
\nOr did the user submit to several competitions but then he was eliminated from the leaderboard for real or alleged rule violation? Or how else could this happen?
\nWhere is the transparency of information?

", "rawMarkdown": "@wcukierski \n>... submissions to closed competitions (which do not show up as having been joined on a user's profile)\n\nAs usual, nicely put words but unfortunately saying nothing. \nCan you list the competitions? Or are they secret? \nOr did the user submit to several competitions but then he was eliminated from the leaderboard for real or alleged rule violation? Or how else could this happen?\nWhere is the transparency of information?", "votes": 1 }, { "id": 2123928, "postDate": "2023-01-31T18:22:41.203Z", "content": "

@blankaf if you submit to a closed competition (one where the deadline has passed), that submission shows as submission activity on the Public Activity part of your profile. Submissions to closed competitions are unlimited, so if you submit 1000 times in a day, your activity will say 1000 submissions for that day.

", "rawMarkdown": "@blankaf if you submit to a closed competition (one where the deadline has passed), that submission shows as submission activity on the `Public Activity` part of your profile. Submissions to closed competitions are unlimited, so if you submit 1000 times in a day, your activity will say 1000 submissions for that day." }, { "id": 2123930, "postDate": "2023-01-31T18:25:22.840Z", "content": "

oh I see, sorry about the confusion. Nice to know this.

", "rawMarkdown": "oh I see, sorry about the confusion. Nice to know this.", "votes": 1 }, { "id": 2123995, "postDate": "2023-01-31T19:32:31.697Z", "content": "

@wcukierski
\nOK. The confusion could have been excluded if you had just said: \"The total count includes also Late submissions\".
\nOr is \"Late submission\" a prohibited term in Kaggle discussions?
\nIt is an interesting phenomenon that in every disputed Code Competition (Foursquare, DFL) late submissions are not opened. The explanation is apparent.

", "rawMarkdown": "@wcukierski \nOK. The confusion could have been excluded if you had just said: \"The total count includes also Late submissions\".\nOr is \"Late submission\" a prohibited term in Kaggle discussions?\nIt is an interesting phenomenon that in every disputed Code Competition (Foursquare, DFL) late submissions are not opened. The explanation is apparent.", "votes": 1 } ] } ] }, { "id": 2122033, "postDate": "2023-01-30T17:08:10.583Z", "content": "

Let's see how they will move when competitions ends. If teams close to each other move in the same direction, it could be fishy.

", "rawMarkdown": "Let's see how they will move when competitions ends. If teams close to each other move in the same direction, it could be fishy.", "votes": 4 }, { "id": 2117086, "postDate": "2023-01-27T00:55:30.387Z", "content": "

I also felt a weird activity around silver zone (100-150) recently

\n

I spent something like 7 days on the same position (115th), and on last ~24h dropped 30 positions. Can be a coindence due to team merger effect (two days ago)

\n

Another curious thing is that the number of teams on this comp went up from ~2500 to 2530 recently

", "rawMarkdown": "I also felt a weird activity around silver zone (100-150) recently\n\nI spent something like 7 days on the same position (115th), and on last ~24h dropped 30 positions. Can be a coindence due to team merger effect (two days ago)\n\nAnother curious thing is that the number of teams on this comp went up from ~2500 to 2530 recently", "votes": 4, "replies": [ { "id": 2117088, "postDate": "2023-01-27T01:24:12.797Z", "content": "

Yes, the rank changes became so intense after merger deadline. But, This is probably usual case in kaggle competition, so we cannot state this is because of cheaters. But, its suspicious…

", "rawMarkdown": "Yes, the rank changes became so intense after merger deadline. But, This is probably usual case in kaggle competition, so we cannot state this is because of cheaters. But, its suspicious...", "votes": 1, "replies": [ { "id": 2117445, "postDate": "2023-01-27T09:24:13.520Z", "content": "

I also noticed that big drop two days ago !

", "rawMarkdown": "I also noticed that big drop two days ago !", "votes": 2 } ] } ] }, { "id": 2123343, "postDate": "2023-01-31T13:08:32.767Z", "content": "

In the last week the lower limit of the silver medal range has gone up from 0.579 to 0.584, is this normal for any competition?

", "rawMarkdown": "In the last week the lower limit of the silver medal range has gone up from 0.579 to 0.584, is this normal for any competition?", "votes": 1 }, { "id": 2118798, "postDate": "2023-01-28T09:39:57.240Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 2122023, "author_name": "Artem Fedorov", "author_url": "", "post_date": "2023-01-30T17:03:23.847000", "content": "

In the previous competition (multimodal cell) about 1/3 of teams in medals were disqualified. I guess something like that is going to happen this competition. Just have some patience.

\n

I've moved from 62nd position to current 100th in a few days (hope to upload better results in a few hours when notebook finishes).

", "votes": 8, "replies": [ { "id": 2122199, "author_name": "HinePo", "author_url": "", "post_date": "2023-01-30T18:59:47.347000", "content": "

damn, one third… so this is a very serious and organized attack on kaggle's reputation and medals value

", "votes": 0, "replies": [ { "id": 2122437, "author_name": "Artem Fedorov", "author_url": "", "post_date": "2023-01-30T23:56:59.120000", "content": "

I moved from 123rd place to 88th after results were finalized and cheaters removed.
\nThe fact there are cheaters proves that medals are viewed as important - otherwise noone would bother. I guess organizers are doing pretty good job finding the cheaters.
\nAnd there are of course cases when people break the rules because they misunderstand them, like sharing code before accepting someone into the team or some teams get removed because of a single member.
\nIn cases like a real organized cheating, like buying/selling results maybe real names of cheaters should be published - this case, being caught would bring much bigger problems than possible profit from getting that medal.

", "votes": 2, "replies": [] } ] } ] }, { "id": 2117759, "author_name": "Chen Lin", "author_url": "", "post_date": "2023-01-27T14:17:00.473000", "content": "

Even though I haven't been coming to kaggle for long, I have so far observed the buying and selling of silver medals in almost all competitions. In the G2Net competition, there were also several suspicious teams in the silver medal zone in the last few days, and they exactly fit the description here).

\n

This is not something that can be easily solved, unless the kaggle team really takes this issue seriously and decides to put a lot of effort into it on a consistent basis. I and others have made some suggestions in previous posts, but haven't seen any substantial action from the kaggle team.

\n

In fact, even after this competition, the kaggle team conducted a pretty long investigation, and when you go back to LB now, you can still identify several teams that are clearly suspect. They still won the silver medal as expected. As you can see, even clearly distinguishing cheating teams is a very difficult task.

\n

To be honest, I am disappointed with this phenomenon and the results. But the world is just not perfect and sometimes we just have to accept it, put our heads down and keep on doing the right thing, we can still keep learning on kaggle and moving towards the master and grandmaster.🙂

", "votes": 8, "replies": [ { "id": 2117914, "author_name": "Will Cukierski", "author_url": "", "post_date": "2023-01-27T16:29:19.647000", "content": "
\n

haven't seen any substantial action from the kaggle team.

\n
\n

We take this seriously and have been working behind the scenes on processes/projects to combat this. These projects take time, necessarily have to happen in private, and have to be balanced with other priorities. While I know it may be frustrating to not see immediate drastic changes, it's not a sign that we are ignoring this.

\n
\n

you can still identify several teams that are clearly suspect

\n
\n

We are careful to use a standard of evidence that goes beyond merely looking suspect (the harm of removing a legitimate team is a much worse outcome than leaving a cheating team). Thankfully, cheating by sharing solutions or using multiple accounts is a shortsighted move that does not lead to knowledge & growth, so while these people may gain a short-term boost in stature, it's not a sustainable path to doing better.

\n
\n

I am disappointed with this phenomenon and the results. But the world is just not perfect and sometimes we just have to accept it, put our heads down and keep on doing the right thing

\n
\n

Well said! This attitude is what makes this community great, and the reason we can keep doing these in spite of a few bad apples.

", "votes": 27, "replies": [ { "id": 2118119, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2023-01-27T19:27:41.910000", "content": "

There are some very low hanging fruit things Kaggle could do.

\n

eg: On the team page, just explain/normalize what is expected before teaming with a stranger.

\n

One baseline might be - Get on a zoom call to talk about technical experience and discuss expectations around code sharing and how the team will operate.

\n

This would cut down significantly on the biggest and most thorniest issue - which is prize winning teams getting disqualified because they were too trusting.

\n

https://kaggle.com/discussions/product-feedback/373889#2118013

", "votes": 3, "replies": [] }, { "id": 2118702, "author_name": "wakaka", "author_url": "", "post_date": "2023-01-28T08:04:31.363000", "content": "

@wcukierski
\nCant you make a statement like this in advance??
\n\"We will remove the questionable team from the LB once. If you later submit the code and prove your innocence, we will put you back on the finalized version of the LB. We are sorry, but we need your cooperation\"

", "votes": 1, "replies": [] }, { "id": 2121889, "author_name": "Allie K.", "author_url": "", "post_date": "2023-01-30T16:07:54.447000", "content": "

@wcukierski

\n
\n

have to be balanced with other priorities

\n
\n

Could you tell us, please, what can be of a higher priority than keeping competitions fair and with equal chances for all participants?

", "votes": 3, "replies": [] } ] }, { "id": 2118353, "author_name": "", "author_url": "", "post_date": "2023-01-28T00:39:32.220000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 2118756, "author_name": "Manuel Campos", "author_url": "", "post_date": "2023-01-28T08:50:36.257000", "content": "

Indeed, it seems that the lower area of the silver medals has been quickly invaded… which has been more than less common and very, very noticeable in recent months…

", "votes": 6, "replies": [ { "id": 2123779, "author_name": "Hassan Abedi", "author_url": "", "post_date": "2023-01-31T16:59:22.327000", "content": "

I think this also happened in https://www.kaggle.com/competitions/santa-2022. Because in the final weeks of the competition, many new competitors appeared out of nowhere and got into medal tiers.

", "votes": 0, "replies": [] } ] }, { "id": 2122661, "author_name": "Aji Samudra", "author_url": "", "post_date": "2023-01-31T04:41:30.577000", "content": "

Echoing the same thought
\nI've spent 2 months in the competition and was in the 84th for a month.
\nWithin a week I've moved from 84th to 148th.

\n

Given the fact that until 10 Jan there were still few teams that surpassed the \"big wall\" LB 0.579,
\nIt's very hard to believe some new joiners with only FEW DAYS can achieve LB > 0.579

\n

attaching analysis from @tatamikenn
\n\"Big wall in LB: only 4.12% of teams scored >= 0.579\": https://www.kaggle.com/competitions/otto-recommender-system/discussion/377149

", "votes": 3, "replies": [] }, { "id": 2122641, "author_name": "Sherry_pomi", "author_url": "", "post_date": "2023-01-31T04:28:00.247000", "content": "

For each user, we could only submit up to 5 submissions per day, right? When I checked some users' profile, I accidentally found one user(lkkcharlie0126) submit 42 submissions per day! And has submitted more than 5 times multiple days for this competition since this is the only competition that user participated.

\n

Is there any exceptions that you could submit >5 times, if not where should I report this wired case to?

", "votes": 4, "replies": [ { "id": 2123537, "author_name": "Will Cukierski", "author_url": "", "post_date": "2023-01-31T15:13:04.733000", "content": "
\n

I accidentally found one user(lkkcharlie0126) submit 42 submissions per day! And has submitted more than 5 times multiple days for this competition since this is the only competition that user participated.

\n
\n

On what are you basing this claim? I reviewed this team and found no instances of exceeding the daily limits. In fact, our on-call will get paged if any team on any competition were to violate a daily limit.

", "votes": 1, "replies": [ { "id": 2123716, "author_name": "Sherry_pomi", "author_url": "", "post_date": "2023-01-31T16:35:28.220000", "content": "

\"From

", "votes": 0, "replies": [] }, { "id": 2123719, "author_name": "Sherry_pomi", "author_url": "", "post_date": "2023-01-31T16:36:00.853000", "content": "

From the user profile? Nor sure if it's the website problem though

", "votes": 0, "replies": [] }, { "id": 2123770, "author_name": "Allie K.", "author_url": "", "post_date": "2023-01-31T16:54:21.410000", "content": "

@wcukierski the last finished competition was Santa 2022 which allowed 100 submissions per day.

\n

With a nearly traditional Kaggle staff unaibility to keep competitions fair and transparent I wouldn't be wondering if somewhere deep in the system the number of allowed submissions per day wouldn't have been reset to five.
\nOf course, you will never admit any technical mistake.

", "votes": 1, "replies": [] }, { "id": 2123875, "author_name": "Will Cukierski", "author_url": "", "post_date": "2023-01-31T17:43:51.707000", "content": "

@sherrypomi that activity graph includes submissions to closed competitions (which do not show up as having been joined on a user's profile).

", "votes": 0, "replies": [] }, { "id": 2123915, "author_name": "Allie K.", "author_url": "", "post_date": "2023-01-31T18:11:29.020000", "content": "

@wcukierski

\n
\n

… submissions to closed competitions (which do not show up as having been joined on a user's profile)

\n
\n

As usual, nicely put words but unfortunately saying nothing.
\nCan you list the competitions? Or are they secret?
\nOr did the user submit to several competitions but then he was eliminated from the leaderboard for real or alleged rule violation? Or how else could this happen?
\nWhere is the transparency of information?

", "votes": 1, "replies": [] }, { "id": 2123928, "author_name": "Will Cukierski", "author_url": "", "post_date": "2023-01-31T18:22:41.203000", "content": "

@blankaf if you submit to a closed competition (one where the deadline has passed), that submission shows as submission activity on the Public Activity part of your profile. Submissions to closed competitions are unlimited, so if you submit 1000 times in a day, your activity will say 1000 submissions for that day.

", "votes": 0, "replies": [] }, { "id": 2123930, "author_name": "Sherry_pomi", "author_url": "", "post_date": "2023-01-31T18:25:22.840000", "content": "

oh I see, sorry about the confusion. Nice to know this.

", "votes": 1, "replies": [] }, { "id": 2123995, "author_name": "Allie K.", "author_url": "", "post_date": "2023-01-31T19:32:31.697000", "content": "

@wcukierski
\nOK. The confusion could have been excluded if you had just said: \"The total count includes also Late submissions\".
\nOr is \"Late submission\" a prohibited term in Kaggle discussions?
\nIt is an interesting phenomenon that in every disputed Code Competition (Foursquare, DFL) late submissions are not opened. The explanation is apparent.

", "votes": 1, "replies": [] } ] } ] }, { "id": 2122033, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-01-30T17:08:10.583000", "content": "

Let's see how they will move when competitions ends. If teams close to each other move in the same direction, it could be fishy.

", "votes": 4, "replies": [] }, { "id": 2117086, "author_name": "HinePo", "author_url": "", "post_date": "2023-01-27T00:55:30.387000", "content": "

I also felt a weird activity around silver zone (100-150) recently

\n

I spent something like 7 days on the same position (115th), and on last ~24h dropped 30 positions. Can be a coindence due to team merger effect (two days ago)

\n

Another curious thing is that the number of teams on this comp went up from ~2500 to 2530 recently

", "votes": 4, "replies": [ { "id": 2117088, "author_name": "wakaka", "author_url": "", "post_date": "2023-01-27T01:24:12.797000", "content": "

Yes, the rank changes became so intense after merger deadline. But, This is probably usual case in kaggle competition, so we cannot state this is because of cheaters. But, its suspicious…

", "votes": 1, "replies": [ { "id": 2117445, "author_name": "Rayan-aay", "author_url": "", "post_date": "2023-01-27T09:24:13.520000", "content": "

I also noticed that big drop two days ago !

", "votes": 2, "replies": [] } ] } ] }, { "id": 2123343, "author_name": "theta", "author_url": "", "post_date": "2023-01-31T13:08:32.767000", "content": "

In the last week the lower limit of the silver medal range has gone up from 0.579 to 0.584, is this normal for any competition?

", "votes": 1, "replies": [] }, { "id": 2118798, "author_name": "", "author_url": "", "post_date": "2023-01-28T09:39:57.240000", "content": "", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2117072": "Hi all, \n\nThis may really just be my imagination, but I'm just a little curious, so I'm posting this.\n\nI feel like there are suddenly more Contributer teams(most of them are composed of 4) around 80~100 in the silver rank, I don't remember what teams were in that rank 3 or 4 days ago so if they were in that rank originally, I will delete this post. However, in a previous competition (Open Problems - Multimodal Single-Cell Integration), we saw a large number of Contributer teams suddenly appear in the medal standings just before the end of the competition, and we are concerned that something similar may happen. I fear that a similar thing will happen (https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313). \n\nOf course, I understand that it is possible to legitimately reach the silver range of LB suddenly. But I have made this discussion because I find it odd that these Contributer teams are clumped together in similar rankings...\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F11991501%2Fd7967b50d12f3c510bf0578c05ac3bbb%2F(91).png?generation=1674790349676747&alt=media)", "2122023": "In the previous competition (multimodal cell) about 1/3 of teams in medals were disqualified. I guess something like that is going to happen this competition. Just have some patience.\n\nI've moved from 62nd position to current 100th in a few days (hope to upload better results in a few hours when notebook finishes).", "2117759": "Even though I haven't been coming to kaggle for long, I have so far observed the buying and selling of silver medals in almost all competitions. In the [G2Net competition](https://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves/leaderboard), there were also several suspicious teams in the silver medal zone in the last few days, and they exactly fit the description [here](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313)). \n\nThis is not something that can be easily solved, unless the kaggle team really takes this issue seriously and decides to put a lot of effort into it on a consistent basis. I and others have made some [suggestions](https://www.kaggle.com/competitions/open-problems-multimodal/discussion/366313#2033961) in previous posts, but haven't seen any substantial action from the kaggle team.\n\nIn fact, even after this competition, the kaggle team conducted a pretty long investigation, and when you go back to [LB](https://www.kaggle.com/competitions/open-problems-multimodal/leaderboard) now, you can still identify several teams that are clearly suspect. They still won the silver medal as expected. As you can see, even clearly distinguishing cheating teams is a very difficult task.\n\nTo be honest, I am disappointed with this phenomenon and the results. But the world is just not perfect and sometimes we just have to accept it, put our heads down and keep on doing the right thing, we can still keep learning on kaggle and moving towards the master and grandmaster.🙂", "2118756": "Indeed, it seems that the lower area of the silver medals has been quickly invaded... which has been more than less common and very, very noticeable in recent months...", "2122661": "Echoing the same thought\nI've spent 2 months in the competition and was in the 84th for a month. \nWithin a week I've moved from 84th to 148th.\n\nGiven the fact that until 10 Jan there were still few teams that surpassed the \"big wall\" LB 0.579,\nIt's very hard to believe some new joiners with only FEW DAYS can achieve LB > 0.579\n\nattaching analysis from @tatamikenn \n\"Big wall in LB: only 4.12% of teams scored >= 0.579\": https://www.kaggle.com/competitions/otto-recommender-system/discussion/377149", "2122641": "For each user, we could only submit up to 5 submissions per day, right? When I checked some users' profile, I accidentally found one user(lkkcharlie0126) submit 42 submissions per day! And has submitted more than 5 times multiple days for this competition since this is the only competition that user participated. \n\nIs there any exceptions that you could submit >5 times, if not where should I report this wired case to? ", "2122033": "Let's see how they will move when competitions ends. If teams close to each other move in the same direction, it could be fishy.", "2117086": "I also felt a weird activity around silver zone (100-150) recently\n\nI spent something like 7 days on the same position (115th), and on last ~24h dropped 30 positions. Can be a coindence due to team merger effect (two days ago)\n\nAnother curious thing is that the number of teams on this comp went up from ~2500 to 2530 recently", "2123343": "In the last week the lower limit of the silver medal range has gone up from 0.579 to 0.584, is this normal for any competition?", "2118798": "" } }