![\"\"](\"https://i.pinimg.com/originals/db/9a/7b/db9a7b446fb26d4c7d687dcd0eb7984b.jpg\")
Hello everybody, I wanna report about why current top-5 jumped in 1 days and achieve such great results.\nAnswer is simple — they scrapped https://www.metmuseum.org/art/collection\nIt already was mentioned in another post here (that it is possible to reconstruct original labels for this competition)\nI got this info during conversation with one of the current top-5 at the leaderboard.\nSo, they overfitted neural networks on ALL scrapped data, it means that they will not fall on the stage 2 (because I think that stage 2 data available at site)
\n\nSo, as it against rules, @codingcliff, @maggiemd what’s next?
", "messages": [ { "id": 543332, "postDate": "2019-06-04T13:31:58.870Z", "content": "Hello everybody, I wanna report about why current top-5 jumped in 1 days and achieve such great results.\nAnswer is simple — they scrapped https://www.metmuseum.org/art/collection\nIt already was mentioned in another post here (that it is possible to reconstruct original labels for this competition)\nI got this info during conversation with one of the current top-5 at the leaderboard.\nSo, they overfitted neural networks on ALL scrapped data, it means that they will not fall on the stage 2 (because I think that stage 2 data available at site)
\n\nSo, as it against rules, @codingcliff, @maggiemd what’s next?
", "rawMarkdown": "Hello everybody, I wanna report about why current top-5 jumped in 1 days and achieve such great results.\nAnswer is simple — they scrapped https://www.metmuseum.org/art/collection\nIt already was mentioned in another post here (that it is possible to reconstruct original labels for this competition)\nI got this info during conversation with one of the current top-5 at the leaderboard.\nSo, they overfitted neural networks on ALL scrapped data, it means that they will not fall on the stage 2 (because I think that stage 2 data available at site)\n\nSo, as it against rules, @codingcliff, @maggiemd what’s next?", "votes": 110 }, { "id": 543405, "postDate": "2019-06-04T14:14:17.543Z", "content": "Kaggle should be serious handling this matter. It is not like exploiting a leak already in the data in a regular competition. It is a RESEARCH competition spelling that NO EXTERNAL DATA allowed, with no prize money so that people do it for the common good. If such misbehavior and direct violation of rules is not punished, I'll be deeply disappointed and the brand value of kaggle is damaged.
", "rawMarkdown": "Kaggle should be serious handling this matter. It is not like exploiting a leak already in the data in a regular competition. It is a RESEARCH competition spelling that NO EXTERNAL DATA allowed, with no prize money so that people do it for the common good. If such misbehavior and direct violation of rules is not punished, I'll be deeply disappointed and the brand value of kaggle is damaged.\n", "votes": 48 }, { "id": 543457, "postDate": "2019-06-04T14:33:49.963Z", "content": "All participants should be checked at least person who gets gold medal in this kind of competition.\nEveryone can get gold easily. That means it would ruin the value of here.
", "rawMarkdown": "All participants should be checked at least person who gets gold medal in this kind of competition.\nEveryone can get gold easily. That means it would ruin the value of here.", "votes": 37 }, { "id": 543394, "postDate": "2019-06-04T14:07:03.157Z", "content": "This is absolutely insane! Do they expect to present this nonsense at CVPR? What for? Some ranking points? Anything but honor!
", "rawMarkdown": "This is absolutely insane! Do they expect to present this nonsense at CVPR? What for? Some ranking points? Anything but honor!", "votes": 24, "replies": [ { "id": 547565, "postDate": "2019-06-07T21:55:03.263Z", "content": "@jiweiliu Agree 👍
", "rawMarkdown": "@jiweiliu Agree 👍 " } ] }, { "id": 543432, "postDate": "2019-06-04T14:25:39.350Z", "content": "> ValeriyBabushkin wrote\n> \n> > and they all are masters/grandmasters
\n\n\n\n(sad face)
\n\n(no offense)
\n\nhttps://www.kaggle.com/c/imet-2019-fgvc6/discussion/94256#542190
", "rawMarkdown": "> **ValeriyBabushkin wrote**\n> \n> > and they all are masters/grandmasters\n\n@venheads \n\n(sad face)\n\n(no offense)\n\nhttps://www.kaggle.com/c/imet-2019-fgvc6/discussion/94256#542190", "votes": 19 }, { "id": 545935, "postDate": "2019-06-06T04:04:41.877Z", "content": "https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/88064#525061
\n\n\n\n\nban the account entirely
\n
I think this is the justice
", "rawMarkdown": "https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/88064#525061\n> ban the account entirely\n\nI think this is the justice", "votes": 20, "replies": [ { "id": 546354, "postDate": "2019-06-06T14:03:08.257Z", "content": "They can create a new one easily and keep competing.
", "rawMarkdown": "They can create a new one easily and keep competing.", "votes": 1 }, { "id": 546361, "postDate": "2019-06-06T14:08:51.410Z", "content": "But it won't be easy to restore Master / Grandmaster title, for example.
", "rawMarkdown": "But it won't be easy to restore Master / Grandmaster title, for example.", "votes": 4 }, { "id": 546419, "postDate": "2019-06-06T15:15:51.943Z", "content": "\n\n\nThey can create a new one easily and keep competing.
\n
You're right. Because they don't have disadvantage. What if kaggle sue them?
", "rawMarkdown": "> They can create a new one easily and keep competing.\n\nYou're right. Because they don't have disadvantage. What if kaggle sue them?", "votes": 1 }, { "id": 546428, "postDate": "2019-06-06T15:26:34.963Z", "content": "\n\n\nWhat if kaggle sue them?
\n
斯斯。。 斯阔衣！
", "rawMarkdown": "> What if kaggle sue them?\n\n斯斯。。 斯阔衣！", "votes": 2 }, { "id": 546435, "postDate": "2019-06-06T15:32:48.633Z", "content": "Anway kaggle team need to take serious action for these bahaviors
", "rawMarkdown": "Anway kaggle team need to take serious action for these bahaviors" }, { "id": 546514, "postDate": "2019-06-06T16:24:34.593Z", "content": "let's review everyone's code. Our code is ready to be reviewed. Actually since it is research, we can even do peer review.
", "rawMarkdown": "let's review everyone's code. Our code is ready to be reviewed. Actually since it is research, we can even do peer review.", "votes": 6 } ] }, { "id": 543462, "postDate": "2019-06-04T14:40:07.267Z", "content": "Sounds like private sharing is common for some people. How sad.
", "rawMarkdown": "Sounds like private sharing is common for some people. How sad.", "votes": 14, "replies": [ { "id": 543485, "postDate": "2019-06-04T14:55:38.910Z", "content": "Some says \"sharing idea is allowed\", because the Rule tab says \"Privately sharing code or data outside of teams is not permitted\".\nThe idea is not code and data.
", "rawMarkdown": "Some says \"sharing idea is allowed\", because the Rule tab says \"Privately sharing code or data outside of teams is not permitted\".\nThe idea is not code and data.", "votes": 4 }, { "id": 543491, "postDate": "2019-06-04T15:01:36.313Z", "content": "I mentioned in a previous competition, Kaggle's rule regarding 'private sharing' is ambiguous. Actually it relies heavily on people's moral standard.
", "rawMarkdown": "I mentioned in a previous competition, Kaggle's rule regarding 'private sharing' is ambiguous. Actually it relies heavily on people's moral standard. ", "votes": 2 }, { "id": 543502, "postDate": "2019-06-04T15:09:39.387Z", "content": "What is ambiguous? It is forbidden to share code or data privately outside teams. If you give some code or some data to another competitor without teaming then you violate the rules.
", "rawMarkdown": "What is ambiguous? It is forbidden to share code or data privately outside teams. If you give some code or some data to another competitor without teaming then you violate the rules.", "votes": 9 }, { "id": 543506, "postDate": "2019-06-04T15:10:38.950Z", "content": "I mean ideas. Clearly lots of people think sharing ideas are also violating the rule.
", "rawMarkdown": "I mean ideas. Clearly lots of people think sharing ideas are also violating the rule. ", "votes": 2 }, { "id": 543508, "postDate": "2019-06-04T15:12:16.843Z", "content": "Idea is gray zone indeed. But here we speak about data scrapped from a website. Rules are crystal clear for data.
", "rawMarkdown": "Idea is gray zone indeed. But here we speak about data scrapped from a website. Rules are crystal clear for data.", "votes": 3 }, { "id": 543513, "postDate": "2019-06-04T15:14:02.890Z", "content": "Yes. Two things happened here. \n- They violated rules by scrapping external data. \n- They privately shared the idea of scrapping.
", "rawMarkdown": "Yes. Two things happened here. \n- They violated rules by scrapping external data. \n- They privately shared the idea of scrapping.", "votes": 5 }, { "id": 543518, "postDate": "2019-06-04T15:16:16.160Z", "content": "Between some talented people, code is not necessary. Just sharing idea is enough, but this stuff is same as code sharing.
", "rawMarkdown": "Between some talented people, code is not necessary. Just sharing idea is enough, but this stuff is same as code sharing.", "votes": 15 }, { "id": 543537, "postDate": "2019-06-04T15:25:49.343Z", "content": "Given some work at the same company I'd rather think they shared the data, i.e. web scrapped it only once.
", "rawMarkdown": "Given some work at the same company I'd rather think they shared the data, i.e. web scrapped it only once.", "votes": 3 }, { "id": 543548, "postDate": "2019-06-04T15:31:46.140Z", "content": "Regarding \"They privately shared the idea of scrapping.\"\nNo. I highly doubt each of them scraped the site. Pretty sure they just shared scrapped data.
", "rawMarkdown": "Regarding \"They privately shared the idea of scrapping.\"\nNo. I highly doubt each of them scraped the site. Pretty sure they just shared scrapped data.", "votes": 4 }, { "id": 544288, "postDate": "2019-06-05T11:05:02.130Z", "content": "@artyomp Right. In their situation the only ethical decision would be to create a team and play as a team since they were sharing their possible solutions inside one company.
\n\nI am not speaking now about scraping of external data which is not of course allowed any way.
", "rawMarkdown": "@artyomp Right. In their situation the only ethical decision would be to create a team and play as a team since they were sharing their possible solutions inside one company.\n\nI am not speaking now about scraping of external data which is not of course allowed any way." } ] }, { "id": 543419, "postDate": "2019-06-04T14:19:49.587Z", "content": "Thank you for your information! I expect your great solution :)\nI got this info during conversation with one of the current top-5 at the leaderboard.
\nBut I think this is private sharing.
Not private given it is disclosed here.
", "rawMarkdown": "Not private given it is disclosed here.", "votes": 6 }, { "id": 543438, "postDate": "2019-06-04T14:27:46.773Z", "content": "The ultimate private sharing is to make it public in the last day ==!
", "rawMarkdown": "The ultimate private sharing is to make it public in the last day ==!", "votes": 5 }, { "id": 543443, "postDate": "2019-06-04T14:28:31.263Z", "content": "Because of the training time ( 5 fold usually take me 2 days), disclosed on the last day almost mean nothing to other competitors
", "rawMarkdown": "Because of the training time ( 5 fold usually take me 2 days), disclosed on the last day almost mean nothing to other competitors", "votes": 2 }, { "id": 543633, "postDate": "2019-06-04T16:43:19.137Z", "content": "@phalanx I think he did a great job to let us know to truth... you made me remember this meme lol\n
It is ultimate private sharing! Thanks for your reply :)
", "rawMarkdown": "It is ultimate private sharing! Thanks for your reply :)", "votes": 1 }, { "id": 544059, "postDate": "2019-06-05T04:41:28.347Z", "content": "lol, this is both funny and a valid point 😄 .
", "rawMarkdown": "lol, this is both funny and a valid point 😄 . ", "votes": 1 } ] }, { "id": 544647, "postDate": "2019-06-05T18:21:33.473Z", "content": "I think Kaggle should ask for solution from all the gold medal winners, not only prize winners. It is also sad that some people use their titles to cover private sharing.
", "rawMarkdown": "I think Kaggle should ask for solution from all the gold medal winners, not only prize winners. It is also sad that some people use their titles to cover private sharing.", "votes": 8, "replies": [ { "id": 546224, "postDate": "2019-06-06T11:18:40.350Z", "content": "True, it is a sad reality that this is what can be expected from them.
", "rawMarkdown": "True, it is a sad reality that this is what can be expected from them." } ] }, { "id": 543406, "postDate": "2019-06-04T14:14:24.253Z", "content": "Why doing that? This competition even hasn't got money prize... I won't buy food at my X5 local stores for at least a month :)
\n\nSpeaking seriously, hope Kaggle will investigate it and this nice competition will have real winner.
", "rawMarkdown": "Why doing that? This competition even hasn't got money prize... I won't buy food at my X5 local stores for at least a month :)\n\nSpeaking seriously, hope Kaggle will investigate it and this nice competition will have real winner.", "votes": 8 }, { "id": 548562, "postDate": "2019-06-09T15:17:15.503Z", "content": "> ValeriyBabushkin wrote\n> \n> > based on the huge difference in imet score leaderboard I think the Leak has been found there
\n\nGuess someone is afraid of facing everyone here, so he chose to response in another competition.
\n\nSo I decide to do him a favor by letting everyone here know the latest response from one of our masters/grandmasters in the Top-5.
", "rawMarkdown": "> **ValeriyBabushkin wrote**\n> \n> > based on the huge difference in imet score leaderboard I think the Leak has been found there\n\n\nGuess someone is afraid of facing everyone here, so he chose to response in another [competition](https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/95025#548538). \n\nSo I decide to do him a favor by letting everyone here know the latest response from one of our masters/grandmasters in the Top-5. ", "votes": 8, "replies": [ { "id": 549045, "postDate": "2019-06-10T08:26:26.933Z", "rawMarkdown": "", "votes": -20 }, { "id": 549681, "postDate": "2019-06-10T23:50:37.263Z", "content": "I guess you can explain where is the leak now. People here would be very interesting to learn how to overfit public leaderboard to that extent. @venheads
", "rawMarkdown": "I guess you can explain where is the leak now. People here would be very interesting to learn how to overfit public leaderboard to that extent. @venheads ", "votes": 5 }, { "id": 549863, "postDate": "2019-06-11T05:16:30.663Z", "rawMarkdown": "", "votes": -22 }, { "id": 549886, "postDate": "2019-06-11T05:41:40.680Z", "content": "Talk is cheap, show us the code, show us the detail of the leak you talked about..\n(no offense)
", "rawMarkdown": "Talk is cheap, show us the code, show us the detail of the leak you talked about..\n(no offense)", "votes": 1 }, { "id": 549895, "postDate": "2019-06-11T05:55:14.320Z", "content": "This competition has one of the most stable CV/LB results over past competitions. You can simply provide Kaggle the code you used to \"overfit\" to the public LB. Then everything will be revealed.
", "rawMarkdown": "This competition has one of the most stable CV/LB results over past competitions. You can simply provide Kaggle the code you used to \"overfit\" to the public LB. Then everything will be revealed.", "votes": 2 }, { "id": 549901, "postDate": "2019-06-11T06:03:03.410Z", "rawMarkdown": "", "votes": -26 }, { "id": 549909, "postDate": "2019-06-11T06:13:51.647Z", "content": "I think Kaggle has the right to ask anyone in this competition to provide the code.
", "rawMarkdown": "I think Kaggle has the right to ask anyone in this competition to provide the code." }, { "id": 549939, "postDate": "2019-06-11T06:44:21.813Z", "content": "Shame on you.\n我从未见过如此厚颜无耻之人
", "rawMarkdown": "Shame on you.\n我从未见过如此厚颜无耻之人", "votes": 8 }, { "id": 549942, "postDate": "2019-06-11T06:48:24.273Z", "content": "\n\n\nI think Kaggle has the right to ask anyone in this competition to provide the code.
\n
Where do you see that written? Of course they don't. IN some competition, like Zillow, there were competition specific rules that indeed enforce this, but this is not the default rules.
", "rawMarkdown": "> I think Kaggle has the right to ask anyone in this competition to provide the code.\n\nWhere do you see that written? Of course they don't. IN some competition, like Zillow, there were competition specific rules that indeed enforce this, but this is not the default rules.", "votes": 2 }, { "id": 549963, "postDate": "2019-06-11T07:16:43.657Z", "content": "https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/88064#525061
\n\n\n", "rawMarkdown": "[https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/88064#525061]()\n> Just a reminder that Kaggle and the Freesound team reserve the right to review all kernels, not just those that win the competition, for adherence to the competition rules. Should it be determined that a participant intentioned tried to \"fly under the radar\" by finishing high enough for strong Kaggle points/medals, but hoping to not have their code reviewed, we reserve the right to not only disqualify that team from the competition, but also to ban the account entirely.\n\n>These rules have been the same for all competitions which prohibit external data, and the integrity of the Kaggle community has resulted in only a few violations to this effect.", "votes": 9 }, { "id": 549972, "postDate": "2019-06-11T07:24:29.290Z", "content": "Just a reminder that Kaggle and the Freesound team reserve the right to review all kernels, not just those that win the competition, for adherence to the competition rules. Should it be determined that a participant intentioned tried to \"fly under the radar\" by finishing high enough for strong Kaggle points/medals, but hoping to not have their code reviewed, we reserve the right to not only disqualify that team from the competition, but also to ban the account entirely.
\n \nThese rules have been the same for all competitions which prohibit external data, and the integrity of the Kaggle community has resulted in only a few violations to this effect.
\n
That's pretty clear
\n\n\n", "rawMarkdown": "That's pretty clear\n> These rules have been the same for all competitions which prohibit external data, and the integrity of the Kaggle community has resulted in only a few violations to this effect.", "votes": 9 }, { "id": 550001, "postDate": "2019-06-11T07:59:15.453Z", "rawMarkdown": "", "votes": -23 }, { "id": 550024, "postDate": "2019-06-11T08:22:15.910Z", "content": "These rules have been the same for all competitions which prohibit external data, and the integrity of the Kaggle community has resulted in only a few violations to this effect.
\n
我从未见过如此厚颜无耻之人
", "rawMarkdown": "我从未见过如此厚颜无耻之人", "votes": 6 }, { "id": 550053, "postDate": "2019-06-11T09:00:20.177Z", "content": "我从未见过如此厚颜无耻之人
", "rawMarkdown": "我从未见过如此厚颜无耻之人", "votes": 4 }, { "id": 550060, "postDate": "2019-06-11T09:06:38.390Z", "content": "我从未见过如此厚颜无耻之人
", "rawMarkdown": "我从未见过如此厚颜无耻之人", "votes": 5 }, { "id": 550365, "postDate": "2019-06-11T14:43:05.017Z", "content": "我从未见过如此厚颜无耻之人
", "rawMarkdown": "我从未见过如此厚颜无耻之人", "votes": 4 }, { "id": 550379, "postDate": "2019-06-11T15:10:38.460Z", "content": "本菜鸡也从未见过如此厚颜无耻之人
", "rawMarkdown": "本菜鸡也从未见过如此厚颜无耻之人", "votes": 2 }, { "id": 550389, "postDate": "2019-06-11T15:24:00.560Z", "content": "\n\n\ndon't think place outside medals could be reviewed as strong
\n
Unless for kernels competition...I remember on Mercari competition, Kaggle re-runned all our kernels on 2nd stage...
\n\nAll those who cheated with external data were removed.
", "rawMarkdown": "> don't think place outside medals could be reviewed as strong\n\nUnless for kernels competition...I remember on Mercari competition, Kaggle re-runned all our kernels on 2nd stage...\n\nAll those who cheated with external data were removed. " }, { "id": 550411, "postDate": "2019-06-11T15:47:20.787Z", "rawMarkdown": "", "votes": -11 }, { "id": 550445, "postDate": "2019-06-11T16:23:32.340Z", "content": "本菜鸡也从未见过如此厚颜无耻之人
", "rawMarkdown": "本菜鸡也从未见过如此厚颜无耻之人" }, { "id": 551135, "postDate": "2019-06-12T11:33:08.437Z", "content": "本菜鸡也从未见过如此厚颜无耻之人
", "rawMarkdown": "本菜鸡也从未见过如此厚颜无耻之人", "votes": -1 } ] }, { "id": 543354, "postDate": "2019-06-04T13:43:20.140Z", "content": "", "rawMarkdown": "@inversion ", "votes": 5, "replies": [ { "id": 543359, "postDate": "2019-06-04T13:46:38.607Z", "content": "in stage 2, I suggest that top participants should reproduce the training processes for their uploaded weights to prove that they didn't violate the rules.
", "rawMarkdown": "in stage 2, I suggest that top participants should reproduce the training processes for their uploaded weights to prove that they didn't violate the rules.", "votes": 8 }, { "id": 543413, "postDate": "2019-06-04T14:15:42.897Z", "content": "That's ture.
", "rawMarkdown": "That's ture.", "votes": 1 }, { "id": 543428, "postDate": "2019-06-04T14:25:10.197Z", "content": "Thanks @YourVenn. @wcukierski is the data scientist for this competition and aware of the issues.
", "rawMarkdown": "Thanks @YourVenn. @wcukierski is the data scientist for this competition and aware of the issues.", "votes": 13 }, { "id": 544065, "postDate": "2019-06-05T04:48:45.527Z", "content": "But is this more politics than data science? What do you actually sign up for at Kaggle 😁
", "rawMarkdown": "But is this more politics than data science? What do you actually sign up for at Kaggle 😁 ", "votes": 1 }, { "id": 551093, "postDate": "2019-06-12T10:21:25.607Z", "content": "We are waiting for your comment about this problem, @wcukierski @inversion \nThank you for recognizing this problem, but what is the next action?
", "rawMarkdown": "We are waiting for your comment about this problem, @wcukierski @inversion \nThank you for recognizing this problem, but what is the next action?" } ] }, { "id": 543353, "postDate": "2019-06-04T13:43:09.730Z", "content": "The truth.
", "rawMarkdown": "The truth.", "votes": 6 }, { "id": 547206, "postDate": "2019-06-07T12:08:00.580Z", "content": "one day i'll become a grandmaster remember that 💯
", "rawMarkdown": "one day i'll become a grandmaster remember that 💯 ", "votes": 3 }, { "id": 543349, "postDate": "2019-06-04T13:41:34Z", "content": "That's what I guess after seen an over 0.07 jump from 619 to 690 last night.
", "rawMarkdown": "That's what I guess after seen an over 0.07 jump from 619 to 690 last night. ", "votes": 3 }, { "id": 543440, "postDate": "2019-06-04T14:28:00.470Z", "content": "If they stay at the top they will have to reproduce how they trained their models. Then it will be clear if they used that external data or not.
", "rawMarkdown": "If they stay at the top they will have to reproduce how they trained their models. Then it will be clear if they used that external data or not. ", "votes": 4, "replies": [ { "id": 543452, "postDate": "2019-06-04T14:32:39.903Z", "content": "Not necessarily. NN can approximate any functions. so one could encode this external label into some crazy hyper parameters with a exotic NN structure. Something like: f1(image, external label) === f2(image, original label) and claim they come up with f2.
I'm sure organizers will want to deep dive on:
\n\n\n", "rawMarkdown": "I'm sure organizers will want to deep dive on:\n\n> so one could encode this external label\n", "votes": 1 }, { "id": 543570, "postDate": "2019-06-04T15:45:51.313Z", "content": "so one could encode this external label
\n
If someone wants to abuse external data without training directly on it, they can simply use this leaked data as a validation set that mimics test and overfit their models.\nSo abuse of external data can be difficult to prove if done properly.
", "rawMarkdown": "If someone wants to abuse external data without training directly on it, they can simply use this leaked data as a validation set that mimics test and overfit their models.\nSo abuse of external data can be difficult to prove if done properly.", "votes": 4 }, { "id": 543970, "postDate": "2019-06-05T01:52:07.353Z", "content": "Still they will need to explain how they trained their model.
", "rawMarkdown": "Still they will need to explain how they trained their model." }, { "id": 544052, "postDate": "2019-06-05T04:23:05.617Z", "content": "Train using all data, with external data as validation, and then identify the best number of epochs to stop at, or make LR high enough so that it doesnt overfit quickly. Then feign you just \"felt\" it was a good number of epochs and didnt bother to spend much time tuning. even if it doesn't give enough lift for in the money, you can snag a gold like that.
", "rawMarkdown": "Train using all data, with external data as validation, and then identify the best number of epochs to stop at, or make LR high enough so that it doesnt overfit quickly. Then feign you just \"felt\" it was a good number of epochs and didnt bother to spend much time tuning. even if it doesn't give enough lift for in the money, you can snag a gold like that.", "votes": 3 } ] }, { "id": 543351, "postDate": "2019-06-04T13:41:51.557Z", "content": "Thank you for explaining this ‘coincidence’ for us.
", "rawMarkdown": "Thank you for explaining this ‘coincidence’ for us.", "votes": 4 }, { "id": 544062, "postDate": "2019-06-05T04:46:38.197Z", "content": "I never quite understood the point of kernels only competitions where you can still upload your own data. What is the real difference? As shown here, unfortunately, as expected, some people will find ways to cheat whenever possible.
", "rawMarkdown": "I never quite understood the point of kernels only competitions where you can still upload your own data. What is the real difference? As shown here, unfortunately, as expected, some people will find ways to cheat whenever possible.", "votes": 1, "replies": [ { "id": 544213, "postDate": "2019-06-05T09:13:25.903Z", "content": "The point of kernel competitions is to get a reproducible inference.
", "rawMarkdown": "The point of kernel competitions is to get a reproducible inference.", "votes": 2 }, { "id": 544702, "postDate": "2019-06-05T19:35:05.027Z", "content": "I understand that point. But if you can train on whatever you like, and with whatever you like, and finally just upload the weights, the overall process is not very reproducible. Unless you require them to redo all the training in a kernel. But even that is not doable in a kernel if you had big machines/resources of your own, and especially if you trained using external data.
\n\nOf course, as you say, you can reproduce inference to the point of running a model with some weights. But allowing uploading random datasets you will never know how they come up with the weights.
", "rawMarkdown": "I understand that point. But if you can train on whatever you like, and with whatever you like, and finally just upload the weights, the overall process is not very reproducible. Unless you require them to redo all the training in a kernel. But even that is not doable in a kernel if you had big machines/resources of your own, and especially if you trained using external data.\n\nOf course, as you say, you can reproduce inference to the point of running a model with some weights. But allowing uploading random datasets you will never know how they come up with the weights.", "votes": 1 } ] }, { "id": 543704, "postDate": "2019-06-04T17:57:42.307Z", "content": "They wanted to be data scientist within a single day ! They are trying to fool the entire community with this kind of behavior.
", "rawMarkdown": "They wanted to be data scientist within a single day ! They are trying to fool the entire community with this kind of behavior.", "votes": 1 }, { "id": 549943, "postDate": "2019-06-11T06:49:10.603Z", "content": "@dempton Well done, your disclosure had the effect you wanted, namely that they no longer are at the top of LB.
", "rawMarkdown": "@dempton Well done, your disclosure had the effect you wanted, namely that they no longer are at the top of LB.", "votes": 1, "replies": [ { "id": 550073, "postDate": "2019-06-11T09:14:31.473Z", "content": "They will still get solo gold, huh?
", "rawMarkdown": "They will still get solo gold, huh?", "votes": 1 } ] }, { "id": 546081, "postDate": "2019-06-06T08:35:01.837Z", "content": "What happens if they do not select the submission/kernel that they are violated?. \nIf they drop their position in stage 2, it shows that they are scared the justice and select the non-cheating versions.
\nI cant wait to see the final result :)
For some reasons I am quite sure that it will happen as you predicted.
", "rawMarkdown": "For some reasons I am quite sure that it will happen as you predicted.", "votes": 3 } ] }, { "id": 543429, "postDate": "2019-06-04T14:25:25.023Z", "content": "These guys violated the rules. Their behaviors should be strictly reviewed.
", "rawMarkdown": "These guys violated the rules. Their behaviors should be strictly reviewed. ", "votes": 2 }, { "id": 543374, "postDate": "2019-06-04T13:57:19.287Z", "content": "You prove my assumption, pure image classification cannot achieve that high, most top teams stuck at 660+
", "rawMarkdown": "You prove my assumption, pure image classification cannot achieve that high, most top teams stuck at 660+", "votes": 2, "replies": [ { "id": 543378, "postDate": "2019-06-04T14:01:24.103Z", "content": "But 677 is real ;)\nI will share my solution later in another post)
", "rawMarkdown": "But 677 is real ;)\nI will share my solution later in another post)", "votes": 32 }, { "id": 543445, "postDate": "2019-06-04T14:29:05.697Z", "content": "677 is not that far, I am waiting to see your solution!
", "rawMarkdown": "677 is not that far, I am waiting to see your solution!" }, { "id": 543535, "postDate": "2019-06-04T15:24:03.160Z", "content": "High score solution doesn't seem to be allowed to share in public near the end of the game, you can share it after the competition.
", "rawMarkdown": "High score solution doesn't seem to be allowed to share in public near the end of the game, you can share it after the competition.", "votes": 2 }, { "id": 544456, "postDate": "2019-06-05T14:26:16.877Z", "content": "Hey, are you willing to share your solution at the workshop?
", "rawMarkdown": "Hey, are you willing to share your solution at the workshop?" } ] }, { "id": 543453, "postDate": "2019-06-04T14:32:56.933Z", "content": "scrapped data is not allowed. we can use only white listed data.
", "rawMarkdown": "scrapped data is not allowed. we can use only white listed data." }, { "id": 544202, "postDate": "2019-06-05T09:04:24.903Z", "content": "This is why it is hard to get a gold medal, because many people are always grouped together， once there is one gold medal, everyone in the group has one gold modal, really sad to see this.
\n\nEDIT: I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.
", "rawMarkdown": "This is why it is hard to get a gold medal, because many people are always grouped together， once there is one gold medal, everyone in the group has one gold modal, really sad to see this.\n\nEDIT: I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.", "votes": 1, "replies": [ { "id": 544306, "postDate": "2019-06-05T11:28:47.100Z", "content": "Please don't label a group because of a case.
", "rawMarkdown": "Please don't label a group because of a case.", "votes": 5 }, { "id": 544344, "postDate": "2019-06-05T12:21:30.507Z", "content": "When they team, people use only one spot in the leaderboard, which means more medals for others. That's good actually.
", "rawMarkdown": "When they team, people use only one spot in the leaderboard, which means more medals for others. That's good actually.", "votes": 7 }, { "id": 544356, "postDate": "2019-06-05T12:33:38.257Z", "content": "I agree with CPMP, teaming is actually good. If there is no limit on team size, I wish all masters/grandmasters except me are in one group in all the competitions I join. ^_^
", "rawMarkdown": "I agree with CPMP, teaming is actually good. If there is no limit on team size, I wish all masters/grandmasters except me are in one group in all the competitions I join. ^_^", "votes": 6 }, { "id": 544370, "postDate": "2019-06-05T12:48:41.540Z", "content": "Xuan Cao lol
", "rawMarkdown": "Xuan Cao lol" }, { "id": 544490, "postDate": "2019-06-05T15:04:41.020Z", "content": "I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.
", "rawMarkdown": "I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.", "votes": 2 }, { "id": 544492, "postDate": "2019-06-05T15:05:31.797Z", "content": "yeah, you are right, I should not be so radical
", "rawMarkdown": "yeah, you are right, I should not be so radical" }, { "id": 544499, "postDate": "2019-06-05T15:19:55.253Z", "content": "小学弟加油！现在kaggle kernel 有免费的P100 可以用，很好的学习机会！
", "rawMarkdown": "小学弟加油！现在kaggle kernel 有免费的P100 可以用，很好的学习机会！", "votes": 4 }, { "id": 544505, "postDate": "2019-06-05T15:25:06.313Z", "content": "多谢大佬的鼓励 ：）
", "rawMarkdown": "多谢大佬的鼓励 ：）" }, { "id": 544654, "postDate": "2019-06-05T18:28:49.490Z", "content": "\n\n\ncompete-for-learning wrote
\n \n\n\n \nThis is why it is hard to get a gold medal, because many people are always grouped together， once there is one gold medal, everyone in the group has one gold modal, really sad to see this.
\nEDIT: I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.
\n
Suspicious private sharing between top teams never happened in my limited Kaggle experience... Focus on what you can learn from each competition you entered, you will get your gold medal soon.
", "rawMarkdown": "> **compete-for-learning wrote**\n> \n> > This is why it is hard to get a gold medal, because many people are always grouped together， once there is one gold medal, everyone in the group has one gold modal, really sad to see this.\n> \n> EDIT: I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.\n\n\n\n\nSuspicious private sharing between top teams never happened in my limited Kaggle experience... Focus on what you can learn from each competition you entered, you will get your gold medal soon. ", "votes": 1 }, { "id": 544703, "postDate": "2019-06-05T19:35:30.897Z", "content": "\n\n\nSuspicious private sharing between top teams never happened in my limited Kaggle experience…
\n
Now it begins...
", "rawMarkdown": "> Suspicious private sharing between top teams never happened in my limited Kaggle experience…\n\nNow it begins..." }, { "id": 545973, "postDate": "2019-06-06T05:13:39.220Z", "content": "It happened before for sure.
", "rawMarkdown": "It happened before for sure.", "votes": 3 } ] }, { "id": 543610, "postDate": "2019-06-04T16:21:02.787Z", "content": "Is it possible to get 100% accuracy with a model and some feature engineering? As i see people are getting 100% accuracy. I am new in machine learning just trying to understand what is going on.
", "rawMarkdown": "Is it possible to get 100% accuracy with a model and some feature engineering? As i see people are getting 100% accuracy. I am new in machine learning just trying to understand what is going on.", "replies": [ { "id": 546971, "postDate": "2019-06-07T04:34:35.413Z", "content": "What I fail to understand is why this comment was downvoted 3 times.\nNice way to demoralize beginners slow claps
", "rawMarkdown": "What I fail to understand is why this comment was downvoted 3 times.\nNice way to demoralize beginners *slow claps*", "votes": 1 }, { "id": 547539, "postDate": "2019-06-07T20:46:28.487Z", "content": "Maybe because feature engineering has nothing to do with deep learning. Automatic feature inference in ConvNets is a basic concept that every beginner should understand. Also, accuracy is not a metric here.
", "rawMarkdown": "Maybe because feature engineering has nothing to do with deep learning. Automatic feature inference in ConvNets is a basic concept that every beginner should understand. Also, accuracy is not a metric here." }, { "id": 547710, "postDate": "2019-06-08T06:43:19.857Z", "content": "Thank you for taking the time to actually explain that rather than downvoting my comment as well.\nAs I see, giving a cold shoulder to people breeds more toxicity.
", "rawMarkdown": "Thank you for taking the time to actually explain that rather than downvoting my comment as well.\nAs I see, giving a cold shoulder to people breeds more toxicity." } ] }, { "id": 566330, "postDate": "2019-07-02T04:07:40.777Z", "content": "sound good it's amazing
", "rawMarkdown": "sound good it's amazing ", "votes": -2 }, { "id": 545881, "postDate": "2019-06-06T02:03:57.563Z", "content": "no wonder so many 'pros' these days, kind of sad.
", "rawMarkdown": "no wonder so many 'pros' these days, kind of sad." }, { "id": 544529, "postDate": "2019-06-05T16:04:20.800Z", "content": "le petro de das
", "rawMarkdown": "le petro de das" }, { "id": 543968, "postDate": "2019-06-05T01:48:27.913Z", "content": "I think competition holders should be aware of how \"Kernel Only\" impact the justice of competition.
\n\nThe biggest problem in the competition is that: \nOn one hand, no external data.\nBut on the other hand, people are allowed to submit their own model files. \nIllegal manipulations do happen in their own model files, but it is hard to figure it out.
\n\nEither \"Kernel Only and No Own Model\" Or \"Normal Competition\" will be better than that.
", "rawMarkdown": "I think competition holders should be aware of how \"Kernel Only\" impact the justice of competition.\n\nThe biggest problem in the competition is that: \nOn one hand, no external data.\nBut on the other hand, people are allowed to submit their own model files. \nIllegal manipulations do happen in their own model files, but it is hard to figure it out.\n\nEither \"Kernel Only and No Own Model\" Or \"Normal Competition\" will be better than that.", "replies": [ { "id": 543986, "postDate": "2019-06-05T02:30:37.453Z", "content": "No, in the normal competition there may be more such things= =maybe we just don't know
", "rawMarkdown": "No, in the normal competition there may be more such things= =maybe we just don't know" } ] }, { "id": 546333, "postDate": "2019-06-06T13:33:21.517Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 543405, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-04T14:14:17.543000", "content": "Kaggle should be serious handling this matter. It is not like exploiting a leak already in the data in a regular competition. It is a RESEARCH competition spelling that NO EXTERNAL DATA allowed, with no prize money so that people do it for the common good. If such misbehavior and direct violation of rules is not punished, I'll be deeply disappointed and the brand value of kaggle is damaged.
", "votes": 48, "replies": [] }, { "id": 543457, "author_name": "ONODERA", "author_url": "", "post_date": "2019-06-04T14:33:49.963000", "content": "All participants should be checked at least person who gets gold medal in this kind of competition.\nEveryone can get gold easily. That means it would ruin the value of here.
", "votes": 37, "replies": [] }, { "id": 543394, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-04T14:07:03.157000", "content": "This is absolutely insane! Do they expect to present this nonsense at CVPR? What for? Some ranking points? Anything but honor!
", "votes": 24, "replies": [ { "id": 547565, "author_name": "Amit Kumar Jaiswal", "author_url": "", "post_date": "2019-06-07T21:55:03.263000", "content": "@jiweiliu Agree 👍
", "votes": 0, "replies": [] } ] }, { "id": 543432, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-04T14:25:39.350000", "content": "> ValeriyBabushkin wrote\n> \n> > and they all are masters/grandmasters
\n\n\n\n(sad face)
\n\n(no offense)
\n\nhttps://www.kaggle.com/c/imet-2019-fgvc6/discussion/94256#542190
", "votes": 19, "replies": [] }, { "id": 545935, "author_name": "ONODERA", "author_url": "", "post_date": "2019-06-06T04:04:41.877000", "content": "https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/88064#525061
\n\n\n\n\nban the account entirely
\n
I think this is the justice
", "votes": 20, "replies": [ { "id": 546354, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-06T14:03:08.257000", "content": "They can create a new one easily and keep competing.
", "votes": 1, "replies": [] }, { "id": 546361, "author_name": "Dmytro Panchenko", "author_url": "", "post_date": "2019-06-06T14:08:51.410000", "content": "But it won't be easy to restore Master / Grandmaster title, for example.
", "votes": 4, "replies": [] }, { "id": 546419, "author_name": "ONODERA", "author_url": "", "post_date": "2019-06-06T15:15:51.943000", "content": "\n\n\nThey can create a new one easily and keep competing.
\n
You're right. Because they don't have disadvantage. What if kaggle sue them?
", "votes": 1, "replies": [] }, { "id": 546428, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-06T15:26:34.963000", "content": "\n\n\nWhat if kaggle sue them?
\n
斯斯。。 斯阔衣！
", "votes": 2, "replies": [] }, { "id": 546435, "author_name": "Strideradu", "author_url": "", "post_date": "2019-06-06T15:32:48.633000", "content": "Anway kaggle team need to take serious action for these bahaviors
", "votes": 0, "replies": [] }, { "id": 546514, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-06T16:24:34.593000", "content": "let's review everyone's code. Our code is ready to be reviewed. Actually since it is research, we can even do peer review.
", "votes": 6, "replies": [] } ] }, { "id": 543462, "author_name": "Appian", "author_url": "", "post_date": "2019-06-04T14:40:07.267000", "content": "Sounds like private sharing is common for some people. How sad.
", "votes": 14, "replies": [ { "id": 543485, "author_name": "ONODERA", "author_url": "", "post_date": "2019-06-04T14:55:38.910000", "content": "Some says \"sharing idea is allowed\", because the Rule tab says \"Privately sharing code or data outside of teams is not permitted\".\nThe idea is not code and data.
", "votes": 4, "replies": [] }, { "id": 543491, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-04T15:01:36.313000", "content": "I mentioned in a previous competition, Kaggle's rule regarding 'private sharing' is ambiguous. Actually it relies heavily on people's moral standard.
", "votes": 2, "replies": [] }, { "id": 543502, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-04T15:09:39.387000", "content": "What is ambiguous? It is forbidden to share code or data privately outside teams. If you give some code or some data to another competitor without teaming then you violate the rules.
", "votes": 9, "replies": [] }, { "id": 543506, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-04T15:10:38.950000", "content": "I mean ideas. Clearly lots of people think sharing ideas are also violating the rule.
", "votes": 2, "replies": [] }, { "id": 543508, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-04T15:12:16.843000", "content": "Idea is gray zone indeed. But here we speak about data scrapped from a website. Rules are crystal clear for data.
", "votes": 3, "replies": [] }, { "id": 543513, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-04T15:14:02.890000", "content": "Yes. Two things happened here. \n- They violated rules by scrapping external data. \n- They privately shared the idea of scrapping.
", "votes": 5, "replies": [] }, { "id": 543518, "author_name": "ONODERA", "author_url": "", "post_date": "2019-06-04T15:16:16.160000", "content": "Between some talented people, code is not necessary. Just sharing idea is enough, but this stuff is same as code sharing.
", "votes": 15, "replies": [] }, { "id": 543537, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-04T15:25:49.343000", "content": "Given some work at the same company I'd rather think they shared the data, i.e. web scrapped it only once.
", "votes": 3, "replies": [] }, { "id": 543548, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2019-06-04T15:31:46.140000", "content": "Regarding \"They privately shared the idea of scrapping.\"\nNo. I highly doubt each of them scraped the site. Pretty sure they just shared scrapped data.
", "votes": 4, "replies": [] }, { "id": 544288, "author_name": "Anna Novikova", "author_url": "", "post_date": "2019-06-05T11:05:02.130000", "content": "@artyomp Right. In their situation the only ethical decision would be to create a team and play as a team since they were sharing their possible solutions inside one company.
\n\nI am not speaking now about scraping of external data which is not of course allowed any way.
", "votes": 0, "replies": [] } ] }, { "id": 543419, "author_name": "phalanx", "author_url": "", "post_date": "2019-06-04T14:19:49.587000", "content": "Thank you for your information! I expect your great solution :)\nI got this info during conversation with one of the current top-5 at the leaderboard.
\nBut I think this is private sharing.
Not private given it is disclosed here.
", "votes": 6, "replies": [] }, { "id": 543438, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-04T14:27:46.773000", "content": "The ultimate private sharing is to make it public in the last day ==!
", "votes": 5, "replies": [] }, { "id": 543443, "author_name": "Strideradu", "author_url": "", "post_date": "2019-06-04T14:28:31.263000", "content": "Because of the training time ( 5 fold usually take me 2 days), disclosed on the last day almost mean nothing to other competitors
", "votes": 2, "replies": [] }, { "id": 543633, "author_name": "Firas Baba", "author_url": "", "post_date": "2019-06-04T16:43:19.137000", "content": "@phalanx I think he did a great job to let us know to truth... you made me remember this meme lol\n
It is ultimate private sharing! Thanks for your reply :)
", "votes": 1, "replies": [] }, { "id": 544059, "author_name": "averagemn", "author_url": "", "post_date": "2019-06-05T04:41:28.347000", "content": "lol, this is both funny and a valid point 😄 .
", "votes": 1, "replies": [] } ] }, { "id": 544647, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-06-05T18:21:33.473000", "content": "I think Kaggle should ask for solution from all the gold medal winners, not only prize winners. It is also sad that some people use their titles to cover private sharing.
", "votes": 8, "replies": [ { "id": 546224, "author_name": "Ashwin Balachandar S", "author_url": "", "post_date": "2019-06-06T11:18:40.350000", "content": "True, it is a sad reality that this is what can be expected from them.
", "votes": 0, "replies": [] } ] }, { "id": 543406, "author_name": "Anna Novikova", "author_url": "", "post_date": "2019-06-04T14:14:24.253000", "content": "Why doing that? This competition even hasn't got money prize... I won't buy food at my X5 local stores for at least a month :)
\n\nSpeaking seriously, hope Kaggle will investigate it and this nice competition will have real winner.
", "votes": 8, "replies": [] }, { "id": 548562, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-09T15:17:15.503000", "content": "> ValeriyBabushkin wrote\n> \n> > based on the huge difference in imet score leaderboard I think the Leak has been found there
\n\nGuess someone is afraid of facing everyone here, so he chose to response in another competition.
\n\nSo I decide to do him a favor by letting everyone here know the latest response from one of our masters/grandmasters in the Top-5.
", "votes": 8, "replies": [ { "id": 549045, "author_name": "ValeriyBabushkin", "author_url": "", "post_date": "2019-06-10T08:26:26.933000", "content": "", "votes": -20, "replies": [] }, { "id": 549681, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-10T23:50:37.263000", "content": "I guess you can explain where is the leak now. People here would be very interesting to learn how to overfit public leaderboard to that extent. @venheads
", "votes": 5, "replies": [] }, { "id": 549863, "author_name": "ValeriyBabushkin", "author_url": "", "post_date": "2019-06-11T05:16:30.663000", "content": "", "votes": -22, "replies": [] }, { "id": 549886, "author_name": "Gary", "author_url": "", "post_date": "2019-06-11T05:41:40.680000", "content": "Talk is cheap, show us the code, show us the detail of the leak you talked about..\n(no offense)
", "votes": 1, "replies": [] }, { "id": 549895, "author_name": "Kulbear", "author_url": "", "post_date": "2019-06-11T05:55:14.320000", "content": "This competition has one of the most stable CV/LB results over past competitions. You can simply provide Kaggle the code you used to \"overfit\" to the public LB. Then everything will be revealed.
", "votes": 2, "replies": [] }, { "id": 549901, "author_name": "ValeriyBabushkin", "author_url": "", "post_date": "2019-06-11T06:03:03.410000", "content": "", "votes": -26, "replies": [] }, { "id": 549909, "author_name": "SeuTao", "author_url": "", "post_date": "2019-06-11T06:13:51.647000", "content": "I think Kaggle has the right to ask anyone in this competition to provide the code.
", "votes": 0, "replies": [] }, { "id": 549939, "author_name": "wh1te", "author_url": "", "post_date": "2019-06-11T06:44:21.813000", "content": "Shame on you.\n我从未见过如此厚颜无耻之人
", "votes": 8, "replies": [] }, { "id": 549942, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-11T06:48:24.273000", "content": "\n\n\nI think Kaggle has the right to ask anyone in this competition to provide the code.
\n
Where do you see that written? Of course they don't. IN some competition, like Zillow, there were competition specific rules that indeed enforce this, but this is not the default rules.
", "votes": 2, "replies": [] }, { "id": 549963, "author_name": "SeuTao", "author_url": "", "post_date": "2019-06-11T07:16:43.657000", "content": "https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/88064#525061
\n\n\n", "votes": 9, "replies": [] }, { "id": 549972, "author_name": "SeuTao", "author_url": "", "post_date": "2019-06-11T07:24:29.290000", "content": "Just a reminder that Kaggle and the Freesound team reserve the right to review all kernels, not just those that win the competition, for adherence to the competition rules. Should it be determined that a participant intentioned tried to \"fly under the radar\" by finishing high enough for strong Kaggle points/medals, but hoping to not have their code reviewed, we reserve the right to not only disqualify that team from the competition, but also to ban the account entirely.
\n \nThese rules have been the same for all competitions which prohibit external data, and the integrity of the Kaggle community has resulted in only a few violations to this effect.
\n
That's pretty clear
\n\n\n", "votes": 9, "replies": [] }, { "id": 550001, "author_name": "ValeriyBabushkin", "author_url": "", "post_date": "2019-06-11T07:59:15.453000", "content": "", "votes": -23, "replies": [] }, { "id": 550024, "author_name": "SeuTao", "author_url": "", "post_date": "2019-06-11T08:22:15.910000", "content": "These rules have been the same for all competitions which prohibit external data, and the integrity of the Kaggle community has resulted in only a few violations to this effect.
\n
我从未见过如此厚颜无耻之人
", "votes": 6, "replies": [] }, { "id": 550053, "author_name": "Kulbear", "author_url": "", "post_date": "2019-06-11T09:00:20.177000", "content": "我从未见过如此厚颜无耻之人
", "votes": 4, "replies": [] }, { "id": 550060, "author_name": "Gary", "author_url": "", "post_date": "2019-06-11T09:06:38.390000", "content": "我从未见过如此厚颜无耻之人
", "votes": 5, "replies": [] }, { "id": 550365, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-11T14:43:05.017000", "content": "我从未见过如此厚颜无耻之人
", "votes": 4, "replies": [] }, { "id": 550379, "author_name": "jionie", "author_url": "", "post_date": "2019-06-11T15:10:38.460000", "content": "本菜鸡也从未见过如此厚颜无耻之人
", "votes": 2, "replies": [] }, { "id": 550389, "author_name": "Serigne ", "author_url": "", "post_date": "2019-06-11T15:24:00.560000", "content": "\n\n\ndon't think place outside medals could be reviewed as strong
\n
Unless for kernels competition...I remember on Mercari competition, Kaggle re-runned all our kernels on 2nd stage...
\n\nAll those who cheated with external data were removed.
", "votes": 0, "replies": [] }, { "id": 550411, "author_name": "ValeriyBabushkin", "author_url": "", "post_date": "2019-06-11T15:47:20.787000", "content": "", "votes": -11, "replies": [] }, { "id": 550445, "author_name": "seefun", "author_url": "", "post_date": "2019-06-11T16:23:32.340000", "content": "本菜鸡也从未见过如此厚颜无耻之人
", "votes": 0, "replies": [] }, { "id": 551135, "author_name": "sheep", "author_url": "", "post_date": "2019-06-12T11:33:08.437000", "content": "本菜鸡也从未见过如此厚颜无耻之人
", "votes": -1, "replies": [] } ] }, { "id": 543354, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-06-04T13:43:20.140000", "content": "", "votes": 5, "replies": [ { "id": 543359, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-06-04T13:46:38.607000", "content": "in stage 2, I suggest that top participants should reproduce the training processes for their uploaded weights to prove that they didn't violate the rules.
", "votes": 8, "replies": [] }, { "id": 543413, "author_name": "Zheng Li", "author_url": "", "post_date": "2019-06-04T14:15:42.897000", "content": "That's ture.
", "votes": 1, "replies": [] }, { "id": 543428, "author_name": "inversion", "author_url": "", "post_date": "2019-06-04T14:25:10.197000", "content": "Thanks @YourVenn. @wcukierski is the data scientist for this competition and aware of the issues.
", "votes": 13, "replies": [] }, { "id": 544065, "author_name": "averagemn", "author_url": "", "post_date": "2019-06-05T04:48:45.527000", "content": "But is this more politics than data science? What do you actually sign up for at Kaggle 😁
", "votes": 1, "replies": [] }, { "id": 551093, "author_name": "rskmoi", "author_url": "", "post_date": "2019-06-12T10:21:25.607000", "content": "We are waiting for your comment about this problem, @wcukierski @inversion \nThank you for recognizing this problem, but what is the next action?
", "votes": 0, "replies": [] } ] }, { "id": 543353, "author_name": "Gary", "author_url": "", "post_date": "2019-06-04T13:43:09.730000", "content": "The truth.
", "votes": 6, "replies": [] }, { "id": 547206, "author_name": "mohaned abid", "author_url": "", "post_date": "2019-06-07T12:08:00.580000", "content": "one day i'll become a grandmaster remember that 💯
", "votes": 3, "replies": [] }, { "id": 543349, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-04T13:41:34", "content": "That's what I guess after seen an over 0.07 jump from 619 to 690 last night.
", "votes": 3, "replies": [] }, { "id": 543440, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-04T14:28:00.470000", "content": "If they stay at the top they will have to reproduce how they trained their models. Then it will be clear if they used that external data or not.
", "votes": 4, "replies": [ { "id": 543452, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-04T14:32:39.903000", "content": "Not necessarily. NN can approximate any functions. so one could encode this external label into some crazy hyper parameters with a exotic NN structure. Something like: f1(image, external label) === f2(image, original label) and claim they come up with f2.
I'm sure organizers will want to deep dive on:
\n\n\n", "votes": 1, "replies": [] }, { "id": 543570, "author_name": "Dmytro Panchenko", "author_url": "", "post_date": "2019-06-04T15:45:51.313000", "content": "so one could encode this external label
\n
If someone wants to abuse external data without training directly on it, they can simply use this leaked data as a validation set that mimics test and overfit their models.\nSo abuse of external data can be difficult to prove if done properly.
", "votes": 4, "replies": [] }, { "id": 543970, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-05T01:52:07.353000", "content": "Still they will need to explain how they trained their model.
", "votes": 0, "replies": [] }, { "id": 544052, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-06-05T04:23:05.617000", "content": "Train using all data, with external data as validation, and then identify the best number of epochs to stop at, or make LR high enough so that it doesnt overfit quickly. Then feign you just \"felt\" it was a good number of epochs and didnt bother to spend much time tuning. even if it doesn't give enough lift for in the money, you can snag a gold like that.
", "votes": 3, "replies": [] } ] }, { "id": 543351, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-04T13:41:51.557000", "content": "Thank you for explaining this ‘coincidence’ for us.
", "votes": 4, "replies": [] }, { "id": 544062, "author_name": "averagemn", "author_url": "", "post_date": "2019-06-05T04:46:38.197000", "content": "I never quite understood the point of kernels only competitions where you can still upload your own data. What is the real difference? As shown here, unfortunately, as expected, some people will find ways to cheat whenever possible.
", "votes": 1, "replies": [ { "id": 544213, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2019-06-05T09:13:25.903000", "content": "The point of kernel competitions is to get a reproducible inference.
", "votes": 2, "replies": [] }, { "id": 544702, "author_name": "averagemn", "author_url": "", "post_date": "2019-06-05T19:35:05.027000", "content": "I understand that point. But if you can train on whatever you like, and with whatever you like, and finally just upload the weights, the overall process is not very reproducible. Unless you require them to redo all the training in a kernel. But even that is not doable in a kernel if you had big machines/resources of your own, and especially if you trained using external data.
\n\nOf course, as you say, you can reproduce inference to the point of running a model with some weights. But allowing uploading random datasets you will never know how they come up with the weights.
", "votes": 1, "replies": [] } ] }, { "id": 543704, "author_name": "Shahules", "author_url": "", "post_date": "2019-06-04T17:57:42.307000", "content": "They wanted to be data scientist within a single day ! They are trying to fool the entire community with this kind of behavior.
", "votes": 1, "replies": [] }, { "id": 549943, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-11T06:49:10.603000", "content": "@dempton Well done, your disclosure had the effect you wanted, namely that they no longer are at the top of LB.
", "votes": 1, "replies": [ { "id": 550073, "author_name": "See--", "author_url": "", "post_date": "2019-06-11T09:14:31.473000", "content": "They will still get solo gold, huh?
", "votes": 1, "replies": [] } ] }, { "id": 546081, "author_name": "cab", "author_url": "", "post_date": "2019-06-06T08:35:01.837000", "content": "What happens if they do not select the submission/kernel that they are violated?. \nIf they drop their position in stage 2, it shows that they are scared the justice and select the non-cheating versions.
\nI cant wait to see the final result :)
For some reasons I am quite sure that it will happen as you predicted.
", "votes": 3, "replies": [] } ] }, { "id": 543429, "author_name": "Zheng Li", "author_url": "", "post_date": "2019-06-04T14:25:25.023000", "content": "These guys violated the rules. Their behaviors should be strictly reviewed.
", "votes": 2, "replies": [] }, { "id": 543374, "author_name": "Strideradu", "author_url": "", "post_date": "2019-06-04T13:57:19.287000", "content": "You prove my assumption, pure image classification cannot achieve that high, most top teams stuck at 660+
", "votes": 2, "replies": [ { "id": 543378, "author_name": "Konstantin Gavrilchik", "author_url": "", "post_date": "2019-06-04T14:01:24.103000", "content": "But 677 is real ;)\nI will share my solution later in another post)
", "votes": 32, "replies": [] }, { "id": 543445, "author_name": "Strideradu", "author_url": "", "post_date": "2019-06-04T14:29:05.697000", "content": "677 is not that far, I am waiting to see your solution!
", "votes": 0, "replies": [] }, { "id": 543535, "author_name": "", "author_url": "", "post_date": "2019-06-04T15:24:03.160000", "content": "High score solution doesn't seem to be allowed to share in public near the end of the game, you can share it after the competition.
", "votes": 2, "replies": [] }, { "id": 544456, "author_name": "Chenyang Zhang", "author_url": "", "post_date": "2019-06-05T14:26:16.877000", "content": "Hey, are you willing to share your solution at the workshop?
", "votes": 0, "replies": [] } ] }, { "id": 543453, "author_name": "takuoko", "author_url": "", "post_date": "2019-06-04T14:32:56.933000", "content": "scrapped data is not allowed. we can use only white listed data.
", "votes": 0, "replies": [] }, { "id": 544202, "author_name": "good good study", "author_url": "", "post_date": "2019-06-05T09:04:24.903000", "content": "This is why it is hard to get a gold medal, because many people are always grouped together， once there is one gold medal, everyone in the group has one gold modal, really sad to see this.
\n\nEDIT: I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.
", "votes": 1, "replies": [ { "id": 544306, "author_name": "qrfaction", "author_url": "", "post_date": "2019-06-05T11:28:47.100000", "content": "Please don't label a group because of a case.
", "votes": 5, "replies": [] }, { "id": 544344, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-05T12:21:30.507000", "content": "When they team, people use only one spot in the leaderboard, which means more medals for others. That's good actually.
", "votes": 7, "replies": [] }, { "id": 544356, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-05T12:33:38.257000", "content": "I agree with CPMP, teaming is actually good. If there is no limit on team size, I wish all masters/grandmasters except me are in one group in all the competitions I join. ^_^
", "votes": 6, "replies": [] }, { "id": 544370, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2019-06-05T12:48:41.540000", "content": "Xuan Cao lol
", "votes": 0, "replies": [] }, { "id": 544490, "author_name": "good good study", "author_url": "", "post_date": "2019-06-05T15:04:41.020000", "content": "I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.
", "votes": 2, "replies": [] }, { "id": 544492, "author_name": "good good study", "author_url": "", "post_date": "2019-06-05T15:05:31.797000", "content": "yeah, you are right, I should not be so radical
", "votes": 0, "replies": [] }, { "id": 544499, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-06-05T15:19:55.253000", "content": "小学弟加油！现在kaggle kernel 有免费的P100 可以用，很好的学习机会！
", "votes": 4, "replies": [] }, { "id": 544505, "author_name": "good good study", "author_url": "", "post_date": "2019-06-05T15:25:06.313000", "content": "多谢大佬的鼓励 ：）
", "votes": 0, "replies": [] }, { "id": 544654, "author_name": "Xuan Cao", "author_url": "", "post_date": "2019-06-05T18:28:49.490000", "content": "\n\n\ncompete-for-learning wrote
\n \n\n\n \nThis is why it is hard to get a gold medal, because many people are always grouped together， once there is one gold medal, everyone in the group has one gold modal, really sad to see this.
\nEDIT: I do not mean team is bad, I mean some people share idea(or even code, data) privately but do not team up, this is what I am trying to argue.
\n
Suspicious private sharing between top teams never happened in my limited Kaggle experience... Focus on what you can learn from each competition you entered, you will get your gold medal soon.
", "votes": 1, "replies": [] }, { "id": 544703, "author_name": "Dmytro Panchenko", "author_url": "", "post_date": "2019-06-05T19:35:30.897000", "content": "\n\n\nSuspicious private sharing between top teams never happened in my limited Kaggle experience…
\n
Now it begins...
", "votes": 0, "replies": [] }, { "id": 545973, "author_name": "CPMP", "author_url": "", "post_date": "2019-06-06T05:13:39.220000", "content": "It happened before for sure.
", "votes": 3, "replies": [] } ] }, { "id": 543610, "author_name": "Labib Abdullah", "author_url": "", "post_date": "2019-06-04T16:21:02.787000", "content": "Is it possible to get 100% accuracy with a model and some feature engineering? As i see people are getting 100% accuracy. I am new in machine learning just trying to understand what is going on.
", "votes": 0, "replies": [ { "id": 546971, "author_name": "Rafid Abyaad", "author_url": "", "post_date": "2019-06-07T04:34:35.413000", "content": "What I fail to understand is why this comment was downvoted 3 times.\nNice way to demoralize beginners slow claps
", "votes": 1, "replies": [] }, { "id": 547539, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2019-06-07T20:46:28.487000", "content": "Maybe because feature engineering has nothing to do with deep learning. Automatic feature inference in ConvNets is a basic concept that every beginner should understand. Also, accuracy is not a metric here.
", "votes": 0, "replies": [] }, { "id": 547710, "author_name": "Rafid Abyaad", "author_url": "", "post_date": "2019-06-08T06:43:19.857000", "content": "Thank you for taking the time to actually explain that rather than downvoting my comment as well.\nAs I see, giving a cold shoulder to people breeds more toxicity.
", "votes": 0, "replies": [] } ] }, { "id": 566330, "author_name": "Abhishek Mishra", "author_url": "", "post_date": "2019-07-02T04:07:40.777000", "content": "sound good it's amazing
", "votes": -2, "replies": [] }, { "id": 545881, "author_name": "Yizhi", "author_url": "", "post_date": "2019-06-06T02:03:57.563000", "content": "no wonder so many 'pros' these days, kind of sad.
", "votes": 0, "replies": [] }, { "id": 544529, "author_name": "Utkarsh", "author_url": "", "post_date": "2019-06-05T16:04:20.800000", "content": "le petro de das
", "votes": 0, "replies": [] }, { "id": 543968, "author_name": "Dilapsky Lee", "author_url": "", "post_date": "2019-06-05T01:48:27.913000", "content": "I think competition holders should be aware of how \"Kernel Only\" impact the justice of competition.
\n\nThe biggest problem in the competition is that: \nOn one hand, no external data.\nBut on the other hand, people are allowed to submit their own model files. \nIllegal manipulations do happen in their own model files, but it is hard to figure it out.
\n\nEither \"Kernel Only and No Own Model\" Or \"Normal Competition\" will be better than that.
", "votes": 0, "replies": [ { "id": 543986, "author_name": "Strideradu", "author_url": "", "post_date": "2019-06-05T02:30:37.453000", "content": "No, in the normal competition there may be more such things= =maybe we just don't know
", "votes": 0, "replies": [] } ] }, { "id": 546333, "author_name": "", "author_url": "", "post_date": "2019-06-06T13:33:21.517000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "543332": "Hello everybody, I wanna report about why current top-5 jumped in 1 days and achieve such great results.\nAnswer is simple — they scrapped https://www.metmuseum.org/art/collection\nIt already was mentioned in another post here (that it is possible to reconstruct original labels for this competition)\nI got this info during conversation with one of the current top-5 at the leaderboard.\nSo, they overfitted neural networks on ALL scrapped data, it means that they will not fall on the stage 2 (because I think that stage 2 data available at site)\n\nSo, as it against rules, @codingcliff, @maggiemd what’s next?", "543405": "Kaggle should be serious handling this matter. It is not like exploiting a leak already in the data in a regular competition. It is a RESEARCH competition spelling that NO EXTERNAL DATA allowed, with no prize money so that people do it for the common good. If such misbehavior and direct violation of rules is not punished, I'll be deeply disappointed and the brand value of kaggle is damaged.\n", "543457": "All participants should be checked at least person who gets gold medal in this kind of competition.\nEveryone can get gold easily. That means it would ruin the value of here.", "543394": "This is absolutely insane! Do they expect to present this nonsense at CVPR? What for? Some ranking points? Anything but honor!", "543432": "> **ValeriyBabushkin wrote**\n> \n> > and they all are masters/grandmasters\n\n@venheads \n\n(sad face)\n\n(no offense)\n\nhttps://www.kaggle.com/c/imet-2019-fgvc6/discussion/94256#542190", "545935": "https://www.kaggle.com/c/freesound-audio-tagging-2019/discussion/88064#525061\n> ban the account entirely\n\nI think this is the justice", "543462": "Sounds like private sharing is common for some people. How sad.", "543419": "Thank you for your information! I expect your great solution :)\n`I got this info during conversation with one of the current top-5 at the leaderboard.`