I was surprised to see (in this competition also!) a public notebook with a high score (0.665) comparative with the public available baseline(0.650), notebook published today with 8 days before the deadline.
\nLegally it may be according to Kaggle rules but morally I do not think is good for the community.
\nPractically, this notebook does not introduce any new methodology, analytics or sustain and test a theory, it's just an inference with the model being public available for everybody for just copy-edit-submit and be almost in the silver zone. Is hard enough for people without GPU horsepower to come with a creative solution in order to advance in the leaderboard.
\nI am a believer in Kaggle sharing mindset and I also learned a lot on Kaggle but this kind of notebooks in the latest phases of a competition are doing more harm than good.
\nWhat I do not understand is what is making people do that ? Is it for the notebooks medals, or …?
I was surprised to see (in this competition also!) a public notebook with a high score (0.665) comparative with the public available baseline(0.650), notebook published today with 8 days before the deadline.
\nLegally it may be according to Kaggle rules but morally I do not think is good for the community.
\nPractically, this notebook does not introduce any new methodology, analytics or sustain and test a theory, it's just an inference with the model being public available for everybody for just copy-edit-submit and be almost in the silver zone. Is hard enough for people without GPU horsepower to come with a creative solution in order to advance in the leaderboard.
\nI am a believer in Kaggle sharing mindset and I also learned a lot on Kaggle but this kind of notebooks in the latest phases of a competition are doing more harm than good.
\nWhat I do not understand is what is making people do that ? Is it for the notebooks medals, or …?
In this particular case it doesn't really make a difference.The top score is over 0.79. It is unlikely that someone with a 0.66 score will win the competition. At this stage everyone has to run their own race if not near the top 50
", "rawMarkdown": "In this particular case it doesn't really make a difference.The top score is over 0.79. It is unlikely that someone with a 0.66 score will win the competition. At this stage everyone has to run their own race if not near the top 50", "votes": null }, { "id": "1678636", "postDate": "02/06/2022 17:18:24", "content": "It's not all about winning the competition, it's also about being in the silver and bronze medal areas. This notebook public model is almost in the silver medal area, extremely high and hard to get zone (100 places from 2000 teams).
", "rawMarkdown": "It's not all about winning the competition, it's also about being in the silver and bronze medal areas. This notebook public model is almost in the silver medal area, extremely high and hard to get zone (100 places from 2000 teams).", "votes": null }, { "id": "1678856", "postDate": "02/06/2022 20:33:42", "content": "Jep… Leaderboard inflation incoming :/
", "rawMarkdown": "Jep... Leaderboard inflation incoming :/", "votes": null }, { "id": "1678986", "postDate": "02/06/2022 22:19:12", "content": "Agree, that guy can simply just share the ideas in whatever detail level he wants and let the others implement and train the model themselves. Posting the entire model and the code at this stage took my and many others' motivation to participate in this competition any longer. I do NOT have any GPU either (Colab pro), but although I really like him to share the ideas, I do NOT need his fully trained model to get improvements. Some people just do not participate in the entire competition and just copy-submit the model without any improvement. Unfair!
", "rawMarkdown": "Agree, that guy can simply just share the ideas in whatever detail level he wants and let the others implement and train the model themselves. Posting the entire model and the code at this stage took my and many others' motivation to participate in this competition any longer. I do NOT have any GPU either (Colab pro), but although I really like him to share the ideas, I do NOT need his fully trained model to get improvements. Some people just do not participate in the entire competition and just copy-submit the model without any improvement. Unfair!", "votes": null }, { "id": "1678993", "postDate": "02/06/2022 22:26:42", "content": "Yes. We should encourage people to keep Kaggle a place for sharing creative ideas, state of the art methodologies and learning from each other, it is indeed an amazing community with the best data science worldwide, but not to share already trained models just for people to copy-submit them.
\nThere are a lot of people who are struggling to find solution, to be creative and then they discover one morning they dropped 300 places in leaderboard due to \"copy-submit kernel\". Practically anybody can come and with 2 clicks can be in the silver/bronze area above many good data scientist
tricks is weaker than hardware （and some other factors）in the competition.If anyone need more detail，they can find it in my notebook comment. but these tricks should（and now，can） carried out on a good baseline.
", "rawMarkdown": "tricks is weaker than hardware （and some other factors）in the competition.If anyone need more detail，they can find it in my notebook comment. but these tricks should（and now，can） carried out on a good baseline.", "votes": null }, { "id": "1679132", "postDate": "02/07/2022 02:27:45", "content": "any notebook could be shared if not in the money zone on author's wish!!!!!!
", "rawMarkdown": "any notebook could be shared if not in the money zone on author's wish!!!!!!", "votes": null }, { "id": "1679158", "postDate": "02/07/2022 03:04:10", "content": "The issue in this particular competition is - at least some part was to get TensorFlow solutions -
\n\"Achieve a private leaderboard score better than or equal to the private leaderboard score of the Top 10% of teams on the private leaderboard. For example, if there are 1000 teams, any selected submission with a score greater than the 100th place team's private score will be eligible.\"
\nUnless there is a separate LB for TF? Perhaps some at the top have managed to achieve something in this area for the competition, but with all the copies and forks of high scoring notebooks, it has pretty much killed TF work here.
\nFor some code competitions, think like internet off, only private models should be allowed in public notebooks and submissions. That might help with some of the fork/submits and get better diversity of solutions and less discouragement overall. Too many competitions at the moment end up with a one trick pony show based on one type of model from one original notebook which probably deserves all the medals. (Prize winners are a different story usually.)
\nHowever, it is possible that all of these overfit the public LB and will see a big shake up at the end. So maybe try not to be discouraged and work to your own CV and solutions.
", "rawMarkdown": "The issue in this particular competition is - at least some part was to get TensorFlow solutions -\n\n\"Achieve a private leaderboard score better than or equal to the private leaderboard score of the Top 10% of teams on the private leaderboard. For example, if there are 1000 teams, any selected submission with a score greater than the 100th place team's private score will be eligible.\"\n\nUnless there is a separate LB for TF? Perhaps some at the top have managed to achieve something in this area for the competition, but with all the copies and forks of high scoring notebooks, it has pretty much killed TF work here.\n\nFor some code competitions, think like internet off, only private models should be allowed in public notebooks and submissions. That might help with some of the fork/submits and get better diversity of solutions and less discouragement overall. Too many competitions at the moment end up with a one trick pony show based on one type of model from one original notebook which probably deserves all the medals. (Prize winners are a different story usually.)\n\nHowever, it is possible that all of these overfit the public LB and will see a big shake up at the end. So maybe try not to be discouraged and work to your own CV and solutions.", "votes": null }, { "id": "1679170", "postDate": "02/07/2022 03:37:15", "content": "I think high score notebooks are likely to be overfit.
\nIt's easy to look at PublicLB scores, but what matters is PrivateLB, which is rated at 75% of the test data.
people have different motivation. May for him, it is about becoming a notebooks grandmaster, in which case you can't really fault him for affecting your chances at a medal.
", "rawMarkdown": "people have different motivation. May for him, it is about becoming a notebooks grandmaster, in which case you can't really fault him for affecting your chances at a medal.", "votes": null }, { "id": "1680044", "postDate": "02/07/2022 14:46:46", "content": "Sharing this kind of notebook is useless for the people and community, people learn nothing from it. And many people originally in the silver or bronze zone now will be defeated by a bonch of copy-paste master, not so fair.
\ni think a real notebook grandmaster should share something benefit people and the community, not spam.
", "rawMarkdown": "Sharing this kind of notebook is useless for the people and community, people learn nothing from it. And many people originally in the silver or bronze zone now will be defeated by a bonch of copy-paste master, not so fair.\n\ni think a real notebook grandmaster should share something benefit people and the community, not spam.", "votes": null }, { "id": "1680207", "postDate": "02/07/2022 16:37:55", "content": "I agree and I already discussed this before.
\nI think kaggle can solve this by adding one more deadline like a public notebook deadline or something, Last 2-to 3 weeks can be the deadline for to anyone can public their notebook.
\nImprovement can be discussed in the discussion section.
Well, the rules are that any notebook can be shared, except in the last 7 days.
", "rawMarkdown": "Well, the rules are that any notebook can be shared, except in the last 7 days.", "votes": null }, { "id": "1684737", "postDate": "02/10/2022 17:10:01", "content": "Can you provide a link to this notebook?
", "rawMarkdown": "Can you provide a link to this notebook?", "votes": null }, { "id": "1684761", "postDate": "02/10/2022 17:34:30", "content": "Well…if you cannot find it, then you don't deserve to copy-submit it 😃
", "rawMarkdown": "Well...if you cannot find it, then you don't deserve to copy-submit it 😃", "votes": null }, { "id": "1684767", "postDate": "02/10/2022 17:46:53", "content": "You didn't have to reply and provide a link, but you definitely had to refrain from posting this disgraceful comment.
", "rawMarkdown": "You didn't have to reply and provide a link, but you definitely had to refrain from posting this disgraceful comment.", "votes": null }, { "id": "1684772", "postDate": "02/10/2022 17:53:01", "content": "@yurikreinin what did you expect when asking for direction to find a notebook which does not introduce any new methodology, analytics or sustain and test a theory ? it's just an inference with the model being public available for everybody for just copy-edit-submit and be almost in the silver zone.
\nIf you would had ask for help regarding a specific methodology or a notebook with a good EDA or one explaining a proposed architecture everybody here would had help you.
@vladvdv,
\nYou recent reply is absolutely alright and you could post it without sounding disrespecting and absolutely wrong assumptions.
\nYour are right, that there is not too much useful information that can be retrieved from strictly inference notebook. However, I was interested to see some nuances, like what kind of pre-processing is done? Normalization? Is input in RGB format or something else? May be something else that I didn't take in account? I just wanted to save my and others time by skipping questions. More of that, I am too busy with my professional activities right now, in order to discuss this game nuances or search for notebook, which I, perhaps, may open in a couple of weeks for curiosity, if I have a time.
In this particular case it doesn't really make a difference.The top score is over 0.79. It is unlikely that someone with a 0.66 score will win the competition. At this stage everyone has to run their own race if not near the top 50
", "votes": null, "replies": [ { "id": 1678636, "author_name": "vladvdv", "author_url": "", "post_date": "02/06/2022 17:18:24", "content": "It's not all about winning the competition, it's also about being in the silver and bronze medal areas. This notebook public model is almost in the silver medal area, extremely high and hard to get zone (100 places from 2000 teams).
", "votes": null, "replies": [] }, { "id": 1679852, "author_name": "nymfree", "author_url": "", "post_date": "02/07/2022 12:45:40", "content": "people have different motivation. May for him, it is about becoming a notebooks grandmaster, in which case you can't really fault him for affecting your chances at a medal.
", "votes": null, "replies": [] } ] }, { "id": 1678856, "author_name": "meesz9", "author_url": "", "post_date": "02/06/2022 20:33:42", "content": "Jep… Leaderboard inflation incoming :/
", "votes": null, "replies": [] }, { "id": 1678986, "author_name": "anhnhunhat", "author_url": "", "post_date": "02/06/2022 22:19:12", "content": "Agree, that guy can simply just share the ideas in whatever detail level he wants and let the others implement and train the model themselves. Posting the entire model and the code at this stage took my and many others' motivation to participate in this competition any longer. I do NOT have any GPU either (Colab pro), but although I really like him to share the ideas, I do NOT need his fully trained model to get improvements. Some people just do not participate in the entire competition and just copy-submit the model without any improvement. Unfair!
", "votes": null, "replies": [ { "id": 1678993, "author_name": "vladvdv", "author_url": "", "post_date": "02/06/2022 22:26:42", "content": "Yes. We should encourage people to keep Kaggle a place for sharing creative ideas, state of the art methodologies and learning from each other, it is indeed an amazing community with the best data science worldwide, but not to share already trained models just for people to copy-submit them.
\nThere are a lot of people who are struggling to find solution, to be creative and then they discover one morning they dropped 300 places in leaderboard due to \"copy-submit kernel\". Practically anybody can come and with 2 clicks can be in the silver/bronze area above many good data scientist
tricks is weaker than hardware （and some other factors）in the competition.If anyone need more detail，they can find it in my notebook comment. but these tricks should（and now，can） carried out on a good baseline.
", "votes": null, "replies": [] }, { "id": 1679132, "author_name": "dragonzhang", "author_url": "", "post_date": "02/07/2022 02:27:45", "content": "any notebook could be shared if not in the money zone on author's wish!!!!!!
", "votes": null, "replies": [ { "id": 1682338, "author_name": "kaggleqrdl", "author_url": "", "post_date": "02/09/2022 05:17:01", "content": "Well, the rules are that any notebook can be shared, except in the last 7 days.
", "votes": null, "replies": [] } ] }, { "id": 1679158, "author_name": "something4kag", "author_url": "", "post_date": "02/07/2022 03:04:10", "content": "The issue in this particular competition is - at least some part was to get TensorFlow solutions -
\n\"Achieve a private leaderboard score better than or equal to the private leaderboard score of the Top 10% of teams on the private leaderboard. For example, if there are 1000 teams, any selected submission with a score greater than the 100th place team's private score will be eligible.\"
\nUnless there is a separate LB for TF? Perhaps some at the top have managed to achieve something in this area for the competition, but with all the copies and forks of high scoring notebooks, it has pretty much killed TF work here.
\nFor some code competitions, think like internet off, only private models should be allowed in public notebooks and submissions. That might help with some of the fork/submits and get better diversity of solutions and less discouragement overall. Too many competitions at the moment end up with a one trick pony show based on one type of model from one original notebook which probably deserves all the medals. (Prize winners are a different story usually.)
\nHowever, it is possible that all of these overfit the public LB and will see a big shake up at the end. So maybe try not to be discouraged and work to your own CV and solutions.
", "votes": null, "replies": [] }, { "id": 1679170, "author_name": "syurenuko", "author_url": "", "post_date": "02/07/2022 03:37:15", "content": "I think high score notebooks are likely to be overfit.
\nIt's easy to look at PublicLB scores, but what matters is PrivateLB, which is rated at 75% of the test data.
Sharing this kind of notebook is useless for the people and community, people learn nothing from it. And many people originally in the silver or bronze zone now will be defeated by a bonch of copy-paste master, not so fair.
\ni think a real notebook grandmaster should share something benefit people and the community, not spam.
", "votes": null, "replies": [] }, { "id": 1680207, "author_name": "mahipalsingh", "author_url": "", "post_date": "02/07/2022 16:37:55", "content": "I agree and I already discussed this before.
\nI think kaggle can solve this by adding one more deadline like a public notebook deadline or something, Last 2-to 3 weeks can be the deadline for to anyone can public their notebook.
\nImprovement can be discussed in the discussion section.
Can you provide a link to this notebook?
", "votes": null, "replies": [ { "id": 1684761, "author_name": "vladvdv", "author_url": "", "post_date": "02/10/2022 17:34:30", "content": "Well…if you cannot find it, then you don't deserve to copy-submit it 😃
", "votes": null, "replies": [] }, { "id": 1684767, "author_name": "yurikreinin", "author_url": "", "post_date": "02/10/2022 17:46:53", "content": "You didn't have to reply and provide a link, but you definitely had to refrain from posting this disgraceful comment.
", "votes": null, "replies": [] }, { "id": 1684772, "author_name": "vladvdv", "author_url": "", "post_date": "02/10/2022 17:53:01", "content": "@yurikreinin what did you expect when asking for direction to find a notebook which does not introduce any new methodology, analytics or sustain and test a theory ? it's just an inference with the model being public available for everybody for just copy-edit-submit and be almost in the silver zone.
\nIf you would had ask for help regarding a specific methodology or a notebook with a good EDA or one explaining a proposed architecture everybody here would had help you.
@vladvdv,
\nYou recent reply is absolutely alright and you could post it without sounding disrespecting and absolutely wrong assumptions.
\nYour are right, that there is not too much useful information that can be retrieved from strictly inference notebook. However, I was interested to see some nuances, like what kind of pre-processing is done? Normalization? Is input in RGB format or something else? May be something else that I didn't take in account? I just wanted to save my and others time by skipping questions. More of that, I am too busy with my professional activities right now, in order to discuss this game nuances or search for notebook, which I, perhaps, may open in a couple of weeks for curiosity, if I have a time.