This competition's early sharing prize is an experiment. We would love hear any feedback or questions here. I'll post a follow up thread in two weeks once the early sharing prize window has closed.
", "messages": [ { "id": 2107393, "postDate": "2023-01-19T18:48:55.120Z", "content": "This competition's early sharing prize is an experiment. We would love hear any feedback or questions here. I'll post a follow up thread in two weeks once the early sharing prize window has closed.
", "rawMarkdown": "This competition's [early sharing prize](https://www.kaggle.com/competitions/icecube-neutrinos-in-deep-ice/overview/early-sharing-prize) is an experiment. We would love hear any feedback or questions here. I'll post a follow up thread in two weeks once the early sharing prize window has closed.", "votes": 18 }, { "id": 2107411, "postDate": "2023-01-19T19:10:13.530Z", "content": "My initial thought is what's stopping people from just blending together the publicly available notebooks that people spent a lot of time on. Then it basically becomes a lottery to see who got lucky choosing the weights that did the best on the Private LB. Is there a submission limit for notebooks eligible for the prize or could someone potentially make 70 submissions (14 days * 5 subs/day)?
\nI guess people should wait until the very last second of the two week window to make their notebook public.
", "rawMarkdown": "My initial thought is what's stopping people from just blending together the publicly available notebooks that people spent a lot of time on. Then it basically becomes a lottery to see who got lucky choosing the weights that did the best on the Private LB. Is there a submission limit for notebooks eligible for the prize or could someone potentially make 70 submissions (14 days * 5 subs/day)?\n\nI guess people should wait until the very last second of the two week window to make their notebook public. ", "votes": 8, "replies": [ { "id": 2107461, "postDate": "2023-01-19T19:42:32.930Z", "content": "Exactly, public ensembles should not be eligible for it. I believe it should also take into account how early it was shared (weighted on both score as well as duration).
", "rawMarkdown": "Exactly, public ensembles should not be eligible for it. I believe it should also take into account how early it was shared (weighted on both score as well as duration).", "votes": 2 }, { "id": 2107490, "postDate": "2023-01-19T20:17:10.037Z", "content": "You're right, it doesn't make sense to share anything until the last second (if you're competing for the prize), but is that really a problem? Those two weeks will go by very quickly.
\nThink of it more as a two phase competition, where the first phase, the early sharing prize is to create the best baseline notebook folks will build upon for the rest of the comp.
\nThis should hopefully raise the bar significantly for the eventual outcome.
\nI think this is a hugely important step in the right direction for Kaggle. So glad they did this.
", "rawMarkdown": "You're right, it doesn't make sense to share anything until the last second (if you're competing for the prize), but is that really a problem? Those two weeks will go by very quickly.\n\nThink of it more as a two phase competition, where the first phase, the early sharing prize is to create the best baseline notebook folks will build upon for the rest of the comp.\n\nThis should hopefully raise the bar significantly for the eventual outcome.\n\nI think this is a hugely important step in the right direction for Kaggle. So glad they did this.\n", "votes": 9, "replies": [ { "id": 2107500, "postDate": "2023-01-19T20:25:48.717Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2107565, "postDate": "2023-01-19T21:33:29.823Z", "content": "\n\nYou're right, it doesn't make sense to share anything until the last second (if you're competing for the prize), but is that really a problem? Those two weeks will go by very quickly.
\n
For anyone who is available to make their notebook public at 23:59 UTC, it shouldn't be a problem. But not everyone is available to do that, so they will have to make it public some time before that which leaves them open to having their work used.
", "rawMarkdown": "> You're right, it doesn't make sense to share anything until the last second (if you're competing for the prize), but is that really a problem? Those two weeks will go by very quickly.\n\nFor anyone who is available to make their notebook public at 23:59 UTC, it shouldn't be a problem. But not everyone is available to do that, so they will have to make it public some time before that which leaves them open to having their work used.\n", "votes": 2 } ] }, { "id": 2107586, "postDate": "2023-01-19T21:57:38.540Z", "content": "Anyway, I like the idea. I think a possible solution for the future would be to add an option to the notebooks that will automatically make the notebook public at the end of the Early Sharing window.
", "rawMarkdown": "Anyway, I like the idea. I think a possible solution for the future would be to add an option to the notebooks that will automatically make the notebook public at the end of the Early Sharing window.", "votes": 3, "replies": [ { "id": 2107630, "postDate": "2023-01-19T23:11:12.593Z", "content": "Thanks for the feedback, @brandenkmurray. I've updated the early sharing prize rules in an attempt to address the points that you raised. The relevant language now reads:
\n\n", "rawMarkdown": "Thanks for the feedback, @brandenkmurray. I've updated the early sharing prize rules in an attempt to address the points that you raised. The relevant language now reads:\n>Submit one or more notebooks before February 2, 2023 11:59PM UTC and publish them before February 4, 2023 11:59PM UTC. The delayed publication date is intended to allow you to participate in the early sharing prize without being concerned that someone else might re-submit your work.", "votes": 14 } ] } ] }, { "id": 2122282, "postDate": "2023-01-30T20:27:53.343Z", "content": "Submit one or more notebooks before February 2, 2023 11:59PM UTC and publish them before February 4, 2023 11:59PM UTC. The delayed publication date is intended to allow you to participate in the early sharing prize without being concerned that someone else might re-submit your work.
\n
Great initiative.
\nOne suggestion:
\nI would like to see this for the most significant contributions, not just submissions but notebooks/discussions (could be for a smaller prize in addition to the submission prize).
\nI have often seen a person open source a notebook, idea, or solution early in the competition that some of, if not all of, the top final solutions end up using or relying on. By rewarding this behaviour of sharing ideas and approaches, I think you will continue to grow the community and those within it.
\nThanks for your efforts and I hope this goes well!
", "rawMarkdown": "Great initiative. \n\nOne suggestion:\n\nI would like to see this for the most significant contributions, not just submissions but notebooks/discussions (could be for a smaller prize in addition to the submission prize). \n\nI have often seen a person open source a notebook, idea, or solution early in the competition that some of, if not all of, the top final solutions end up using or relying on. By rewarding this behaviour of sharing ideas and approaches, I think you will continue to grow the community and those within it.\n\nThanks for your efforts and I hope this goes well!", "votes": 3 }, { "id": 2108849, "postDate": "2023-01-20T21:42:18.273Z", "content": "@sohier is the early sharing prize meant to be awarded to people who share models, or is it open to teams who share any kind of information? To provide context: thoughtful and informative EDAs contain some fantastic insights and suggestions for features that are often picked up by other teams and used extensively throughout competitions with little acknowledgement to who actually discovered them. It would be wonderful to reward sharing those types of notebooks, since they are so vital to the construction and deployment of useful models.
", "rawMarkdown": "@sohier is the early sharing prize meant to be awarded to people who share models, or is it open to teams who share any kind of information? To provide context: thoughtful and informative EDAs contain some fantastic insights and suggestions for features that are often picked up by other teams and used extensively throughout competitions with little acknowledgement to who actually discovered them. It would be wonderful to reward sharing those types of notebooks, since they are so vital to the construction and deployment of useful models. ", "votes": 3, "replies": [ { "id": 2125663, "postDate": "2023-02-01T19:29:27.893Z", "content": "This time we're only accepting models so we can administer the prize efficiently.
", "rawMarkdown": "This time we're only accepting models so we can administer the prize efficiently. ", "votes": 1 } ] }, { "id": 2120857, "postDate": "2023-01-29T21:20:37.913Z", "content": "Shouldn't the early sharing prize entrants be required to also make public all the notebooks they used to generated any data used by their (winning) notebook? For example, a notebook that uses \"bigfilewithcompresseddata.bin\" as an input could be made public and not expose much useful information. Consider a CNN - just knowing the weights used doesn't help if they don't explain the model and training method behind it. It would expose the features (image, sizes, channels) being fed in, but not much else.
", "rawMarkdown": "Shouldn't the early sharing prize entrants be required to also make public all the notebooks they used to generated any data used by their (winning) notebook? For example, a notebook that uses \"bigfilewithcompresseddata.bin\" as an input could be made public and not expose much useful information. Consider a CNN - just knowing the weights used doesn't help if they don't explain the model and training method behind it. It would expose the features (image, sizes, channels) being fed in, but not much else.", "votes": 4, "replies": [ { "id": 2122071, "postDate": "2023-01-30T17:29:42.173Z", "content": "This is a great point and a constant issue with incomplete notebooks around kaggle. Training & hyperparameter selection should all be automated and the code for that should be made available and public.
\nThe rules are specific on this point - Submissions may not use or incorporate information from hand labeling
", "rawMarkdown": "This is a great point and a constant issue with incomplete notebooks around kaggle. Training & hyperparameter selection should all be automated and the code for that should be made available and public.\n\nThe rules are specific on this point - Submissions may not use or incorporate information from hand labeling ", "votes": 1 } ] }, { "id": 2108935, "postDate": "2023-01-21T00:47:30.333Z", "content": "All data must be public, right?
", "rawMarkdown": "All data must be public, right?", "votes": 1, "replies": [ { "id": 2108941, "postDate": "2023-01-21T00:54:33.330Z", "content": "Correct. I'll update the prize description to make that more clear.
\nEdit: never mind, that is already covered: Keep the notebooks and any datasets it uses publicly available until the prize is awarded at the end of the competition.
In the prizes tab, it says
\n\n\n$5,000 will be awarded to the team with the highest private leaderboard …
\n
Should that be \"public leaderboard\"? If we know which notebook is at the top of private leaderboard, won't we get some leaked information, since the notebooks public leaderboard position will tell us if the public leaderboard and private leaderboard correlate?
", "rawMarkdown": "In the prizes tab, it says \n> $5,000 will be awarded to the team with the highest private leaderboard ...\n\nShould that be \"public leaderboard\"? If we know which notebook is at the top of private leaderboard, won't we get some leaked information, since the notebooks public leaderboard position will tell us if the public leaderboard and private leaderboard correlate?", "votes": 1, "replies": [ { "id": 2107558, "postDate": "2023-01-19T21:24:56.830Z", "content": "They aren't revealing the winners of the Early Sharing Prize until after the whole competition is over
", "rawMarkdown": "They aren't revealing the winners of the Early Sharing Prize until after the whole competition is over ", "votes": 5, "replies": [ { "id": 2107589, "postDate": "2023-01-19T22:12:46.553Z", "content": "@brandenkmurray is correct.
", "rawMarkdown": "@brandenkmurray is correct.", "votes": 3 } ] } ] }, { "id": 2111145, "postDate": "2023-01-22T16:24:17.150Z", "content": "I think it will be good work. But I have a question. The sooner you post, the more points you get.
", "rawMarkdown": "I think it will be good work. But I have a question. The sooner you post, the more points you get." } ], "comments": [ { "id": 2107411, "author_name": "Branden Murray", "author_url": "", "post_date": "2023-01-19T19:10:13.530000", "content": "My initial thought is what's stopping people from just blending together the publicly available notebooks that people spent a lot of time on. Then it basically becomes a lottery to see who got lucky choosing the weights that did the best on the Private LB. Is there a submission limit for notebooks eligible for the prize or could someone potentially make 70 submissions (14 days * 5 subs/day)?
\nI guess people should wait until the very last second of the two week window to make their notebook public.
", "votes": 8, "replies": [ { "id": 2107461, "author_name": "tarick.morty", "author_url": "", "post_date": "2023-01-19T19:42:32.930000", "content": "Exactly, public ensembles should not be eligible for it. I believe it should also take into account how early it was shared (weighted on both score as well as duration).
", "votes": 2, "replies": [] }, { "id": 2107490, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2023-01-19T20:17:10.037000", "content": "You're right, it doesn't make sense to share anything until the last second (if you're competing for the prize), but is that really a problem? Those two weeks will go by very quickly.
\nThink of it more as a two phase competition, where the first phase, the early sharing prize is to create the best baseline notebook folks will build upon for the rest of the comp.
\nThis should hopefully raise the bar significantly for the eventual outcome.
\nI think this is a hugely important step in the right direction for Kaggle. So glad they did this.
", "votes": 9, "replies": [ { "id": 2107500, "author_name": "", "author_url": "", "post_date": "2023-01-19T20:25:48.717000", "content": "", "votes": 0, "replies": [] }, { "id": 2107565, "author_name": "Branden Murray", "author_url": "", "post_date": "2023-01-19T21:33:29.823000", "content": "\n\nYou're right, it doesn't make sense to share anything until the last second (if you're competing for the prize), but is that really a problem? Those two weeks will go by very quickly.
\n
For anyone who is available to make their notebook public at 23:59 UTC, it shouldn't be a problem. But not everyone is available to do that, so they will have to make it public some time before that which leaves them open to having their work used.
", "votes": 2, "replies": [] } ] }, { "id": 2107586, "author_name": "Branden Murray", "author_url": "", "post_date": "2023-01-19T21:57:38.540000", "content": "Anyway, I like the idea. I think a possible solution for the future would be to add an option to the notebooks that will automatically make the notebook public at the end of the Early Sharing window.
", "votes": 3, "replies": [ { "id": 2107630, "author_name": "Sohier Dane", "author_url": "", "post_date": "2023-01-19T23:11:12.593000", "content": "Thanks for the feedback, @brandenkmurray. I've updated the early sharing prize rules in an attempt to address the points that you raised. The relevant language now reads:
\n\n", "votes": 14, "replies": [] } ] } ] }, { "id": 2122282, "author_name": "Darien Schettler", "author_url": "", "post_date": "2023-01-30T20:27:53.343000", "content": "Submit one or more notebooks before February 2, 2023 11:59PM UTC and publish them before February 4, 2023 11:59PM UTC. The delayed publication date is intended to allow you to participate in the early sharing prize without being concerned that someone else might re-submit your work.
\n
Great initiative.
\nOne suggestion:
\nI would like to see this for the most significant contributions, not just submissions but notebooks/discussions (could be for a smaller prize in addition to the submission prize).
\nI have often seen a person open source a notebook, idea, or solution early in the competition that some of, if not all of, the top final solutions end up using or relying on. By rewarding this behaviour of sharing ideas and approaches, I think you will continue to grow the community and those within it.
\nThanks for your efforts and I hope this goes well!
", "votes": 3, "replies": [] }, { "id": 2108849, "author_name": "Craig Thomas", "author_url": "", "post_date": "2023-01-20T21:42:18.273000", "content": "@sohier is the early sharing prize meant to be awarded to people who share models, or is it open to teams who share any kind of information? To provide context: thoughtful and informative EDAs contain some fantastic insights and suggestions for features that are often picked up by other teams and used extensively throughout competitions with little acknowledgement to who actually discovered them. It would be wonderful to reward sharing those types of notebooks, since they are so vital to the construction and deployment of useful models.
", "votes": 3, "replies": [ { "id": 2125663, "author_name": "Sohier Dane", "author_url": "", "post_date": "2023-02-01T19:29:27.893000", "content": "This time we're only accepting models so we can administer the prize efficiently.
", "votes": 1, "replies": [] } ] }, { "id": 2120857, "author_name": "SolverWorld", "author_url": "", "post_date": "2023-01-29T21:20:37.913000", "content": "Shouldn't the early sharing prize entrants be required to also make public all the notebooks they used to generated any data used by their (winning) notebook? For example, a notebook that uses \"bigfilewithcompresseddata.bin\" as an input could be made public and not expose much useful information. Consider a CNN - just knowing the weights used doesn't help if they don't explain the model and training method behind it. It would expose the features (image, sizes, channels) being fed in, but not much else.
", "votes": 4, "replies": [ { "id": 2122071, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2023-01-30T17:29:42.173000", "content": "This is a great point and a constant issue with incomplete notebooks around kaggle. Training & hyperparameter selection should all be automated and the code for that should be made available and public.
\nThe rules are specific on this point - Submissions may not use or incorporate information from hand labeling
", "votes": 1, "replies": [] } ] }, { "id": 2108935, "author_name": "YujiAriyasu", "author_url": "", "post_date": "2023-01-21T00:47:30.333000", "content": "All data must be public, right?
", "votes": 1, "replies": [ { "id": 2108941, "author_name": "Sohier Dane", "author_url": "", "post_date": "2023-01-21T00:54:33.330000", "content": "Correct. I'll update the prize description to make that more clear.
\nEdit: never mind, that is already covered: Keep the notebooks and any datasets it uses publicly available until the prize is awarded at the end of the competition.
In the prizes tab, it says
\n\n\n$5,000 will be awarded to the team with the highest private leaderboard …
\n
Should that be \"public leaderboard\"? If we know which notebook is at the top of private leaderboard, won't we get some leaked information, since the notebooks public leaderboard position will tell us if the public leaderboard and private leaderboard correlate?
", "votes": 1, "replies": [ { "id": 2107558, "author_name": "Branden Murray", "author_url": "", "post_date": "2023-01-19T21:24:56.830000", "content": "They aren't revealing the winners of the Early Sharing Prize until after the whole competition is over
", "votes": 5, "replies": [ { "id": 2107589, "author_name": "Sohier Dane", "author_url": "", "post_date": "2023-01-19T22:12:46.553000", "content": "@brandenkmurray is correct.
", "votes": 3, "replies": [] } ] } ] }, { "id": 2111145, "author_name": "ibrahim karatas", "author_url": "", "post_date": "2023-01-22T16:24:17.150000", "content": "I think it will be good work. But I have a question. The sooner you post, the more points you get.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2107393": "This competition's [early sharing prize](https://www.kaggle.com/competitions/icecube-neutrinos-in-deep-ice/overview/early-sharing-prize) is an experiment. We would love hear any feedback or questions here. I'll post a follow up thread in two weeks once the early sharing prize window has closed.", "2107411": "My initial thought is what's stopping people from just blending together the publicly available notebooks that people spent a lot of time on. Then it basically becomes a lottery to see who got lucky choosing the weights that did the best on the Private LB. Is there a submission limit for notebooks eligible for the prize or could someone potentially make 70 submissions (14 days * 5 subs/day)?\n\nI guess people should wait until the very last second of the two week window to make their notebook public. ", "2122282": "Great initiative. \n\nOne suggestion:\n\nI would like to see this for the most significant contributions, not just submissions but notebooks/discussions (could be for a smaller prize in addition to the submission prize). \n\nI have often seen a person open source a notebook, idea, or solution early in the competition that some of, if not all of, the top final solutions end up using or relying on. By rewarding this behaviour of sharing ideas and approaches, I think you will continue to grow the community and those within it.\n\nThanks for your efforts and I hope this goes well!", "2108849": "@sohier is the early sharing prize meant to be awarded to people who share models, or is it open to teams who share any kind of information? To provide context: thoughtful and informative EDAs contain some fantastic insights and suggestions for features that are often picked up by other teams and used extensively throughout competitions with little acknowledgement to who actually discovered them. It would be wonderful to reward sharing those types of notebooks, since they are so vital to the construction and deployment of useful models. ", "2120857": "Shouldn't the early sharing prize entrants be required to also make public all the notebooks they used to generated any data used by their (winning) notebook? For example, a notebook that uses \"bigfilewithcompresseddata.bin\" as an input could be made public and not expose much useful information. Consider a CNN - just knowing the weights used doesn't help if they don't explain the model and training method behind it. It would expose the features (image, sizes, channels) being fed in, but not much else.", "2108935": "All data must be public, right?", "2107515": "In the prizes tab, it says \n> $5,000 will be awarded to the team with the highest private leaderboard ...\n\nShould that be \"public leaderboard\"? If we know which notebook is at the top of private leaderboard, won't we get some leaked information, since the notebooks public leaderboard position will tell us if the public leaderboard and private leaderboard correlate?", "2111145": "I think it will be good work. But I have a question. The sooner you post, the more points you get." } }