Thanks all for your work on this competition! Just a reminder to select your final submissions for the private leaderboard prior to close.
\nNote: many teams have submissions that have a private score of 0. Make sure, when selecting your final submissions, that you do not select those!
", "messages": [ { "id": "1300652", "postDate": "05/10/2021 16:00:10", "content": "Thanks all for your work on this competition! Just a reminder to select your final submissions for the private leaderboard prior to close.
\nNote: many teams have submissions that have a private score of 0. Make sure, when selecting your final submissions, that you do not select those!
", "rawMarkdown": "Thanks all for your work on this competition! Just a reminder to select your final submissions for the private leaderboard prior to close.\n\nNote: many teams have submissions that have a private score of 0. **Make sure, when selecting your final submissions, that you do not select those!**", "votes": null }, { "id": "1300668", "postDate": "05/10/2021 16:09:33", "content": "Currently, submissions are queued. Will there be an extension of one day, if today's submission finish after the deadline.
", "rawMarkdown": "Currently, submissions are queued. Will there be an extension of one day, if today's submission finish after the deadline.", "votes": null }, { "id": "1300790", "postDate": "05/10/2021 17:58:13", "content": "agree! my submission is queued now!
", "rawMarkdown": "agree! my submission is queued now!", "votes": null }, { "id": "1301008", "postDate": "05/10/2021 22:18:41", "content": "Don't all 'successful' submission have a private score?
\nWhat is really meant here?
@abdulkadirguner If you make submissions that deliberately skip the private LB through some mechanism, then you probably have a private score of 0 on those submissions. If you're not deliberately skipping the private LB (i.e. if you are making predictions for all test samples) you should not have a problem.
", "rawMarkdown": "abdulkadirguner If you make submissions that deliberately skip the private LB through some mechanism, then you probably have a private score of 0 on those submissions. If you're not deliberately skipping the private LB (i.e. if you are making predictions for all test samples) you should not have a problem.", "votes": null }, { "id": "1301157", "postDate": "05/11/2021 00:57:04", "content": "I should have noticed this… There are almost 700 teams that got 0 on PB including my team lol. Only the first 530 teams are effectively scored.
", "rawMarkdown": "I should have noticed this... There are almost **700 teams that got 0 on PB** including my team lol. Only the first 530 teams are effectively scored.", "votes": null }, { "id": "1301160", "postDate": "05/11/2021 00:59:43", "content": "https://www.kaggle.com/lililycai/infer-hubmap-deepflash2-bce?scriptVersionId=62398987
\nI didn't deliberately skip the private LB. No submission error, and everything looks normal. Months of efforts seem to be a joke. And nearly 700 teams got zero score. Please fix this ASAP.
agreed. !00% sure there is a bug.
", "rawMarkdown": "agreed. !00% sure there is a bug.", "votes": null }, { "id": "1301318", "postDate": "05/11/2021 02:58:17", "content": "It is quite unfair to those who are using the previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors. Can Kaggle do something about this before finalizing?
", "rawMarkdown": "It is quite unfair to those who are using the previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors. Can Kaggle do something about this before finalizing?", "votes": null }, { "id": "1301451", "postDate": "05/11/2021 04:54:11", "content": "I kindly request the organizers to look into the bug that resulted in so many zero values.
\nI used previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors.
I don't want my hard work to get wasted. Same applies to so many kagglers in this competition.
\nKindly make sure that this doesn't occur in the future
Is there something that can be done to update the existing results ?
", "rawMarkdown": "I kindly request the organizers to look into the bug that resulted in so many zero values. \nI used previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors. \n\nI don't want my hard work to get wasted. Same applies to so many kagglers in this competition.\nKindly make sure that this doesn't occur in the future\n\nIs there something that can be done to update the existing results ?", "votes": null }, { "id": "1301479", "postDate": "05/11/2021 05:26:59", "content": "Dear Organizers,
\nPlease help in resolving the bug which resulted in zero private score before finalizing the results. I didn't deliberately skip the private scoring and the submissions were all successful with no errors or indications and the public scores were getting generated. This made all of us think that the private scoring was also successful. The successful submission and the public score results also prompted us to select those submissions for final scoring.
\nIt is extremely unfair and unfortunate to see months and months of effort with no score or medals. Thank you.
", "rawMarkdown": "Dear Organizers,\n\nPlease help in resolving the bug which resulted in zero private score before finalizing the results. I didn't deliberately skip the private scoring and the submissions were all successful with no errors or indications and the public scores were getting generated. This made all of us think that the private scoring was also successful. The successful submission and the public score results also prompted us to select those submissions for final scoring.\n\nIt is extremely unfair and unfortunate to see months and months of effort with no score or medals. Thank you.", "votes": null }, { "id": "1301500", "postDate": "05/11/2021 05:40:01", "content": "@philculliton - I would respectfully ask WHICH sample submission file was used in the private LB? This competition is a bit odd c.f. others because normally there is only ONE. But here there appears to have been one inserted in the test folder as well. Most people who have competed in code competitions would expect the primary competition folder to have the one to use. AND that it has the CORRECT ids for the private submission. I am not sure if using the one under the test folder would have helped the case here with 0 scores. If so, then it seems like a bit of a trap for young players to have structured it this way. Fine if that is a lesson to be learned, but on top of all the queue issues and depending on your time zone how that affected you and whether you had the opportunity to actually select your submissions. Personally I gave up.
\nIt appears only if the sample submission files were ignored and the image ids extracted from the test folder names worked. If that is how it was going to be done then a heads up would have been nice.
", "rawMarkdown": "philculliton - I would respectfully ask WHICH sample submission file was used in the private LB? This competition is a bit odd c.f. others because normally there is only ONE. But here there appears to have been one inserted in the test folder as well. Most people who have competed in code competitions would expect the primary competition folder to have the one to use. AND that it has the CORRECT ids for the private submission. I am not sure if using the one under the test folder would have helped the case here with 0 scores. If so, then it seems like a bit of a trap for young players to have structured it this way. Fine if that is a lesson to be learned, but on top of all the queue issues and depending on your time zone how that affected you and whether you had the opportunity to actually select your submissions. Personally I gave up. \n\nIt appears only if the sample submission files were ignored and the image ids extracted from the test folder names worked. If that is how it was going to be done then a heads up would have been nice.", "votes": null }, { "id": "1301624", "postDate": "05/11/2021 07:05:01", "content": "@something4kag I used the sample submission file and received a private LB score, so it has nothing to do with that.
\nMaybe the deepflash inference pipeline had an issue where it would keep running even after encountering an error in the private test set and submitting empty strings for the private set images, but run perfectly on the public test set. This would give you a public LB score, but 0.0 for private LB score.
\nBut I agree this can be frustrating, so maybe Kaggle can notify participants in the future if there is an error in private scoring or getting an abnormal value such as 0.0 when the public LB score has no issue.
", "rawMarkdown": "something4kag I used the sample submission file and received a private LB score, so it has nothing to do with that. \n\nMaybe the deepflash inference pipeline had an issue where it would keep running even after encountering an error in the private test set and submitting empty strings for the private set images, but run perfectly on the public test set. This would give you a public LB score, but 0.0 for private LB score.\n\nBut I agree this can be frustrating, so maybe Kaggle can notify participants in the future if there is an error in private scoring or getting an abnormal value such as 0.0 when the public LB score has no issue.", "votes": null }, { "id": "1301653", "postDate": "05/11/2021 07:18:54", "content": "In investigating further, it appears the sub notebook for deepflash was changed quietly 4 days ago V15 to use rasterio instead of tifffile library to read images due to a potential large image in private test that the latter would not cope with. I get it, who cares that 600+ took an earlier version that may or may not fail. If it were my work I probably would have posted a topic in the forum to let people know. But caveat emptor I guess.
\nHad other notebooks using rasterio but in conjunction extracted the ids from the test folder so it clouded the problem as subs with them worked. And with queue issues did not get the chance to use final versions but it makes no diff. What matters is to know why and how to avoid in future personally and for Kaggle.
", "rawMarkdown": "In investigating further, it appears the sub notebook for deepflash was changed quietly 4 days ago V15 to use rasterio instead of tifffile library to read images due to a potential large image in private test that the latter would not cope with. I get it, who cares that 600+ took an earlier version that may or may not fail. If it were my work I probably would have posted a topic in the forum to let people know. But caveat emptor I guess. \n\nHad other notebooks using rasterio but in conjunction extracted the ids from the test folder so it clouded the problem as subs with them worked. And with queue issues did not get the chance to use final versions but it makes no diff. What matters is to know why and how to avoid in future personally and for Kaggle.", "votes": null }, { "id": "1301675", "postDate": "05/11/2021 07:32:44", "content": "Hi,
\nIn accordance with Organizer and supporting teams, as of people using deepflash2 inference pipeline, the private LB to fall to zero of many teams. If the pipeline is error, then how could the public LB gives such good results. So, don't blame the participants while the error/mistake from your end.
And so, I'm requesting to reconsider those submission for finalizing the results. That might helpful for many users and their hard work is meaningful.
", "rawMarkdown": "Hi,\nIn accordance with Organizer and supporting teams, as of people using deepflash2 inference pipeline, the private LB to fall to zero of many teams. If the pipeline is error, then how could the public LB gives such good results. So, don't blame the participants while the error/mistake from your end.\n\nAnd so, I'm requesting to reconsider those submission for finalizing the results. That might helpful for many users and their hard work is meaningful.", "votes": null }, { "id": "1302183", "postDate": "05/11/2021 12:32:32", "content": "Disclaimer: this is my understanding.
\nThe public LB is computed instantly, based on the csv generated when you save the notebook. Hence the good public result. Private dataset and/or sample csv can cause the process to trip on the private dataset.
", "rawMarkdown": "Disclaimer: this is my understanding.\n\nThe public LB is computed instantly, based on the csv generated when you save the notebook. Hence the good public result. Private dataset and/or sample csv can cause the process to trip on the private dataset.", "votes": null }, { "id": "1304868", "postDate": "05/13/2021 00:32:19", "content": "just curious as to why my message was downvoted by so many. Is my statement inaccurate? Would love to know.
", "rawMarkdown": "just curious as to why my message was downvoted by so many. Is my statement inaccurate? Would love to know.", "votes": null }, { "id": "1304899", "postDate": "05/13/2021 01:11:37", "content": "@andrasferenczi I don't know why you got downvoted. But i can point out an error in your post.
\nPublic is not computed instantly. The submission.csv file that you can download from public notebooks is not the same submission.csv that gets submitted and scored during submission. For example, view this public notebook, the public submission.csv file only has 1 row because the commit notebook only infers one public image.
When you submit, your notebook gets exectcuted a second time and infers both the public test images and private test images and creates a new submission file with 15 rows (5 public test labels and 10 private test labels), this new second submission file is what gets scored by Kaggle.
", "rawMarkdown": "andrasferenczi I don't know why you got downvoted. But i can point out an error in your post.\n\nPublic is not computed instantly. The `submission.csv` file that you can download from public notebooks is not the same `submission.csv` that gets submitted and scored during submission. For example, view [this public notebook][1], the public `submission.csv` file only has 1 row because the commit notebook only infers one public image.\n\nWhen you submit, your notebook gets exectcuted a second time and infers both the public test images and private test images and creates a new submission file with 15 rows (5 public test labels and 10 private test labels), this new second submission file is what gets scored by Kaggle.\n\n[1]: https://www.kaggle.com/mistag/inference-hubmap-fpn-single-model-ii/output", "votes": null }, { "id": "1305131", "postDate": "05/13/2021 05:36:11", "content": "@andrasferenczi - don't worry too much about downvotes or take them personally. It is a Kaggle aberration and have thought for some time bots are responsible. So even this one with the word downvote in it may get targeted. And this particular thread has downvotes of its own (see 9 upvoters but score is 5) so could be targeted.
\nIf people in Kaggle disagree or want to correct you they will post a reply. That is what matters.
Dear competitions organizers and support team,
\nI ask you to allow me to submit one more time. My story is simple. I am a Kaggle novice.
\nDuring the competition I was training models on my laptop, then created submission.csv file and then uploaded it to the competition. So I checked my score on public LB just by uploading the answer file directly to the kaggle platform.
To participate on Private LB I loaded my trained model https://www.kaggle.com/boyarintsev/last-best-model to the competition. I made a notebook https://www.kaggle.com/boyarintsev/make-answer-new/ to run the model on test dataset on kaggle platform on 2-3 of May. And then I stuck. Because the notebook I created was consuming too much memory. It was really strange, because that notebook was just a copy from my local research and works fine on my laptop. I spent 5-6 days just to avoid the issue of memory consumption, but I didn't succeed :(
\nI gave up, when the competition was over, but then I noticed the button \"late submission\". So I decided that I still have a chance to upload my solution. And I started again to struggle against the memory consumption. I succeeded ten minutes ago, but then I found out that I can't upload new solutions :(
Please allow me to compete with that solution.
\nYou can check that the model I am using was uploaded on 2nd of May https://www.kaggle.com/boyarintsev/last-best-model.
I just really upset about the situation because now I am only a top of 86% in this competition and it is really disappointing me, as I treat kaggle account as a part of my portfolio. I spent a lot of time to solve this competition, because I knew just basic things about the segmentation task before.
\nAnyway, thanks for the competition! I liked it!
\nBest regards,
\nArtem Boyarintsev.
@boyarintsev I'm sorry you feel this way, but unfortunately there are no do overs on Kaggle unless it affects all participants. The competition rules are clearly mentioned and you can find additional documentation here https://www.kaggle.com/docs/competitions#kernels-only-FAQ for future reference.
\nThe reason you faced memory issues is because the private test set contained an extremely large image that causes you to run out of RAM when using the Tiffle library. I think this was discussed once or twice on the discussion forum which is why several public kernels switched from tiffle to rasterio.
\nThe best thing you can do is to learn from this mistake and move on to the next competition. I can guarantee you that there will be more disappointments in future competitions, but that is a part of competing here on Kaggle. Wish you the best of luck in your next competition.
", "rawMarkdown": "boyarintsev I'm sorry you feel this way, but unfortunately there are no do overs on Kaggle unless it affects all participants. The competition rules are clearly mentioned and you can find additional documentation here https://www.kaggle.com/docs/competitions#kernels-only-FAQ for future reference.\n\nThe reason you faced memory issues is because the private test set contained an extremely large image that causes you to run out of RAM when using the Tiffle library. I think this was discussed once or twice on the discussion forum which is why several public kernels switched from tiffle to rasterio.\n\nThe best thing you can do is to learn from this mistake and move on to the next competition. I can guarantee you that there will be more disappointments in future competitions, but that is a part of competing here on Kaggle. Wish you the best of luck in your next competition.", "votes": null }, { "id": "1306380", "postDate": "05/13/2021 18:17:02", "content": "I'm sorry for you feelings, but as Yovin points out, this is a code competition and you need to submit codes to get scored in private set.
", "rawMarkdown": "I'm sorry for you feelings, but as Yovin points out, this is a code competition and you need to submit codes to get scored in private set.", "votes": null } ], "comments": [ { "id": 1300668, "author_name": "mustafaa", "author_url": "", "post_date": "05/10/2021 16:09:33", "content": "Currently, submissions are queued. Will there be an extension of one day, if today's submission finish after the deadline.
", "votes": null, "replies": [ { "id": 1300790, "author_name": "puyuzhou", "author_url": "", "post_date": "05/10/2021 17:58:13", "content": "agree! my submission is queued now!
", "votes": null, "replies": [] } ] }, { "id": 1301008, "author_name": "abdulkadirguner", "author_url": "", "post_date": "05/10/2021 22:18:41", "content": "Don't all 'successful' submission have a private score?
\nWhat is really meant here?
@abdulkadirguner If you make submissions that deliberately skip the private LB through some mechanism, then you probably have a private score of 0 on those submissions. If you're not deliberately skipping the private LB (i.e. if you are making predictions for all test samples) you should not have a problem.
", "votes": null, "replies": [] }, { "id": 1301160, "author_name": "lililycai", "author_url": "", "post_date": "05/11/2021 00:59:43", "content": "https://www.kaggle.com/lililycai/infer-hubmap-deepflash2-bce?scriptVersionId=62398987
\nI didn't deliberately skip the private LB. No submission error, and everything looks normal. Months of efforts seem to be a joke. And nearly 700 teams got zero score. Please fix this ASAP.
I should have noticed this… There are almost 700 teams that got 0 on PB including my team lol. Only the first 530 teams are effectively scored.
", "votes": null, "replies": [ { "id": 1301194, "author_name": "rpsantosakaggle", "author_url": "", "post_date": "05/11/2021 01:27:36", "content": "agreed. !00% sure there is a bug.
", "votes": null, "replies": [] } ] }, { "id": 1301318, "author_name": "css919", "author_url": "", "post_date": "05/11/2021 02:58:17", "content": "It is quite unfair to those who are using the previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors. Can Kaggle do something about this before finalizing?
", "votes": null, "replies": [] }, { "id": 1301451, "author_name": "dhakshiin1601", "author_url": "", "post_date": "05/11/2021 04:54:11", "content": "I kindly request the organizers to look into the bug that resulted in so many zero values.
\nI used previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors.
I don't want my hard work to get wasted. Same applies to so many kagglers in this competition.
\nKindly make sure that this doesn't occur in the future
Is there something that can be done to update the existing results ?
", "votes": null, "replies": [] }, { "id": 1301479, "author_name": "sreevishnudamodaran", "author_url": "", "post_date": "05/11/2021 05:26:59", "content": "Dear Organizers,
\nPlease help in resolving the bug which resulted in zero private score before finalizing the results. I didn't deliberately skip the private scoring and the submissions were all successful with no errors or indications and the public scores were getting generated. This made all of us think that the private scoring was also successful. The successful submission and the public score results also prompted us to select those submissions for final scoring.
\nIt is extremely unfair and unfortunate to see months and months of effort with no score or medals. Thank you.
", "votes": null, "replies": [] }, { "id": 1301500, "author_name": "something4kag", "author_url": "", "post_date": "05/11/2021 05:40:01", "content": "@philculliton - I would respectfully ask WHICH sample submission file was used in the private LB? This competition is a bit odd c.f. others because normally there is only ONE. But here there appears to have been one inserted in the test folder as well. Most people who have competed in code competitions would expect the primary competition folder to have the one to use. AND that it has the CORRECT ids for the private submission. I am not sure if using the one under the test folder would have helped the case here with 0 scores. If so, then it seems like a bit of a trap for young players to have structured it this way. Fine if that is a lesson to be learned, but on top of all the queue issues and depending on your time zone how that affected you and whether you had the opportunity to actually select your submissions. Personally I gave up.
\nIt appears only if the sample submission files were ignored and the image ids extracted from the test folder names worked. If that is how it was going to be done then a heads up would have been nice.
", "votes": null, "replies": [ { "id": 1301624, "author_name": "yovinyahathugoda", "author_url": "", "post_date": "05/11/2021 07:05:01", "content": "@something4kag I used the sample submission file and received a private LB score, so it has nothing to do with that.
\nMaybe the deepflash inference pipeline had an issue where it would keep running even after encountering an error in the private test set and submitting empty strings for the private set images, but run perfectly on the public test set. This would give you a public LB score, but 0.0 for private LB score.
\nBut I agree this can be frustrating, so maybe Kaggle can notify participants in the future if there is an error in private scoring or getting an abnormal value such as 0.0 when the public LB score has no issue.
", "votes": null, "replies": [] }, { "id": 1301653, "author_name": "something4kag", "author_url": "", "post_date": "05/11/2021 07:18:54", "content": "In investigating further, it appears the sub notebook for deepflash was changed quietly 4 days ago V15 to use rasterio instead of tifffile library to read images due to a potential large image in private test that the latter would not cope with. I get it, who cares that 600+ took an earlier version that may or may not fail. If it were my work I probably would have posted a topic in the forum to let people know. But caveat emptor I guess.
\nHad other notebooks using rasterio but in conjunction extracted the ids from the test folder so it clouded the problem as subs with them worked. And with queue issues did not get the chance to use final versions but it makes no diff. What matters is to know why and how to avoid in future personally and for Kaggle.
", "votes": null, "replies": [] } ] }, { "id": 1301675, "author_name": "poornesh5", "author_url": "", "post_date": "05/11/2021 07:32:44", "content": "Hi,
\nIn accordance with Organizer and supporting teams, as of people using deepflash2 inference pipeline, the private LB to fall to zero of many teams. If the pipeline is error, then how could the public LB gives such good results. So, don't blame the participants while the error/mistake from your end.
And so, I'm requesting to reconsider those submission for finalizing the results. That might helpful for many users and their hard work is meaningful.
", "votes": null, "replies": [ { "id": 1302183, "author_name": "andrasferenczi", "author_url": "", "post_date": "05/11/2021 12:32:32", "content": "Disclaimer: this is my understanding.
\nThe public LB is computed instantly, based on the csv generated when you save the notebook. Hence the good public result. Private dataset and/or sample csv can cause the process to trip on the private dataset.
", "votes": null, "replies": [] }, { "id": 1304868, "author_name": "andrasferenczi", "author_url": "", "post_date": "05/13/2021 00:32:19", "content": "just curious as to why my message was downvoted by so many. Is my statement inaccurate? Would love to know.
", "votes": null, "replies": [] }, { "id": 1304899, "author_name": "cdeotte", "author_url": "", "post_date": "05/13/2021 01:11:37", "content": "@andrasferenczi I don't know why you got downvoted. But i can point out an error in your post.
\nPublic is not computed instantly. The submission.csv file that you can download from public notebooks is not the same submission.csv that gets submitted and scored during submission. For example, view this public notebook, the public submission.csv file only has 1 row because the commit notebook only infers one public image.
When you submit, your notebook gets exectcuted a second time and infers both the public test images and private test images and creates a new submission file with 15 rows (5 public test labels and 10 private test labels), this new second submission file is what gets scored by Kaggle.
", "votes": null, "replies": [] }, { "id": 1305131, "author_name": "something4kag", "author_url": "", "post_date": "05/13/2021 05:36:11", "content": "@andrasferenczi - don't worry too much about downvotes or take them personally. It is a Kaggle aberration and have thought for some time bots are responsible. So even this one with the word downvote in it may get targeted. And this particular thread has downvotes of its own (see 9 upvoters but score is 5) so could be targeted.
\nIf people in Kaggle disagree or want to correct you they will post a reply. That is what matters.
Dear competitions organizers and support team,
\nI ask you to allow me to submit one more time. My story is simple. I am a Kaggle novice.
\nDuring the competition I was training models on my laptop, then created submission.csv file and then uploaded it to the competition. So I checked my score on public LB just by uploading the answer file directly to the kaggle platform.
To participate on Private LB I loaded my trained model https://www.kaggle.com/boyarintsev/last-best-model to the competition. I made a notebook https://www.kaggle.com/boyarintsev/make-answer-new/ to run the model on test dataset on kaggle platform on 2-3 of May. And then I stuck. Because the notebook I created was consuming too much memory. It was really strange, because that notebook was just a copy from my local research and works fine on my laptop. I spent 5-6 days just to avoid the issue of memory consumption, but I didn't succeed :(
\nI gave up, when the competition was over, but then I noticed the button \"late submission\". So I decided that I still have a chance to upload my solution. And I started again to struggle against the memory consumption. I succeeded ten minutes ago, but then I found out that I can't upload new solutions :(
Please allow me to compete with that solution.
\nYou can check that the model I am using was uploaded on 2nd of May https://www.kaggle.com/boyarintsev/last-best-model.
I just really upset about the situation because now I am only a top of 86% in this competition and it is really disappointing me, as I treat kaggle account as a part of my portfolio. I spent a lot of time to solve this competition, because I knew just basic things about the segmentation task before.
\nAnyway, thanks for the competition! I liked it!
\nBest regards,
\nArtem Boyarintsev.
@boyarintsev I'm sorry you feel this way, but unfortunately there are no do overs on Kaggle unless it affects all participants. The competition rules are clearly mentioned and you can find additional documentation here https://www.kaggle.com/docs/competitions#kernels-only-FAQ for future reference.
\nThe reason you faced memory issues is because the private test set contained an extremely large image that causes you to run out of RAM when using the Tiffle library. I think this was discussed once or twice on the discussion forum which is why several public kernels switched from tiffle to rasterio.
\nThe best thing you can do is to learn from this mistake and move on to the next competition. I can guarantee you that there will be more disappointments in future competitions, but that is a part of competing here on Kaggle. Wish you the best of luck in your next competition.
", "votes": null, "replies": [] }, { "id": 1306380, "author_name": "lililycai", "author_url": "", "post_date": "05/13/2021 18:17:02", "content": "I'm sorry for you feelings, but as Yovin points out, this is a code competition and you need to submit codes to get scored in private set.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1300652": "Thanks all for your work on this competition! Just a reminder to select your final submissions for the private leaderboard prior to close.\n\nNote: many teams have submissions that have a private score of 0. **Make sure, when selecting your final submissions, that you do not select those!**", "1300668": "Currently, submissions are queued. Will there be an extension of one day, if today's submission finish after the deadline.", "1300790": "agree! my submission is queued now!", "1301008": "Don't all 'successful' submission have a private score?\nWhat is really meant here?", "1301010": "abdulkadirguner If you make submissions that deliberately skip the private LB through some mechanism, then you probably have a private score of 0 on those submissions. If you're not deliberately skipping the private LB (i.e. if you are making predictions for all test samples) you should not have a problem.", "1301157": "I should have noticed this... There are almost **700 teams that got 0 on PB** including my team lol. Only the first 530 teams are effectively scored.", "1301160": "https://www.kaggle.com/lililycai/infer-hubmap-deepflash2-bce?scriptVersionId=62398987\nI didn't deliberately skip the private LB. No submission error, and everything looks normal. Months of efforts seem to be a joke. And nearly 700 teams got zero score. Please fix this ASAP.", "1301194": "agreed. !00% sure there is a bug.", "1301318": "It is quite unfair to those who are using the previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors. Can Kaggle do something about this before finalizing?", "1301451": "I kindly request the organizers to look into the bug that resulted in so many zero values. \nI used previous deepflash2 inference pipeline, the private score just turns out to be 0 without any errors. \n\nI don't want my hard work to get wasted. Same applies to so many kagglers in this competition.\nKindly make sure that this doesn't occur in the future\n\nIs there something that can be done to update the existing results ?", "1301479": "Dear Organizers,\n\nPlease help in resolving the bug which resulted in zero private score before finalizing the results. I didn't deliberately skip the private scoring and the submissions were all successful with no errors or indications and the public scores were getting generated. This made all of us think that the private scoring was also successful. The successful submission and the public score results also prompted us to select those submissions for final scoring.\n\nIt is extremely unfair and unfortunate to see months and months of effort with no score or medals. Thank you.", "1301500": "philculliton - I would respectfully ask WHICH sample submission file was used in the private LB? This competition is a bit odd c.f. others because normally there is only ONE. But here there appears to have been one inserted in the test folder as well. Most people who have competed in code competitions would expect the primary competition folder to have the one to use. AND that it has the CORRECT ids for the private submission. I am not sure if using the one under the test folder would have helped the case here with 0 scores. If so, then it seems like a bit of a trap for young players to have structured it this way. Fine if that is a lesson to be learned, but on top of all the queue issues and depending on your time zone how that affected you and whether you had the opportunity to actually select your submissions. Personally I gave up. \n\nIt appears only if the sample submission files were ignored and the image ids extracted from the test folder names worked. If that is how it was going to be done then a heads up would have been nice.", "1301624": "something4kag I used the sample submission file and received a private LB score, so it has nothing to do with that. \n\nMaybe the deepflash inference pipeline had an issue where it would keep running even after encountering an error in the private test set and submitting empty strings for the private set images, but run perfectly on the public test set. This would give you a public LB score, but 0.0 for private LB score.\n\nBut I agree this can be frustrating, so maybe Kaggle can notify participants in the future if there is an error in private scoring or getting an abnormal value such as 0.0 when the public LB score has no issue.", "1301653": "In investigating further, it appears the sub notebook for deepflash was changed quietly 4 days ago V15 to use rasterio instead of tifffile library to read images due to a potential large image in private test that the latter would not cope with. I get it, who cares that 600+ took an earlier version that may or may not fail. If it were my work I probably would have posted a topic in the forum to let people know. But caveat emptor I guess. \n\nHad other notebooks using rasterio but in conjunction extracted the ids from the test folder so it clouded the problem as subs with them worked. And with queue issues did not get the chance to use final versions but it makes no diff. What matters is to know why and how to avoid in future personally and for Kaggle.", "1301675": "Hi,\nIn accordance with Organizer and supporting teams, as of people using deepflash2 inference pipeline, the private LB to fall to zero of many teams. If the pipeline is error, then how could the public LB gives such good results. So, don't blame the participants while the error/mistake from your end.\n\nAnd so, I'm requesting to reconsider those submission for finalizing the results. That might helpful for many users and their hard work is meaningful.", "1302183": "Disclaimer: this is my understanding.\n\nThe public LB is computed instantly, based on the csv generated when you save the notebook. Hence the good public result. Private dataset and/or sample csv can cause the process to trip on the private dataset.", "1304868": "just curious as to why my message was downvoted by so many. Is my statement inaccurate? Would love to know.", "1304899": "andrasferenczi I don't know why you got downvoted. But i can point out an error in your post.\n\nPublic is not computed instantly. The `submission.csv` file that you can download from public notebooks is not the same `submission.csv` that gets submitted and scored during submission. For example, view [this public notebook][1], the public `submission.csv` file only has 1 row because the commit notebook only infers one public image.\n\nWhen you submit, your notebook gets exectcuted a second time and infers both the public test images and private test images and creates a new submission file with 15 rows (5 public test labels and 10 private test labels), this new second submission file is what gets scored by Kaggle.\n\n[1]: https://www.kaggle.com/mistag/inference-hubmap-fpn-single-model-ii/output", "1305131": "andrasferenczi - don't worry too much about downvotes or take them personally. It is a Kaggle aberration and have thought for some time bots are responsible. So even this one with the word downvote in it may get targeted. And this particular thread has downvotes of its own (see 9 upvoters but score is 5) so could be targeted. \nIf people in Kaggle disagree or want to correct you they will post a reply. That is what matters.", "1305904": "Dear competitions organizers and support team,\n\nI ask you to allow me to submit one more time. My story is simple. I am a Kaggle novice. \nDuring the competition I was training models on my laptop, then created submission.csv file and then uploaded it to the competition. So I checked my score on public LB just by uploading the answer file directly to the kaggle platform.\n\nTo participate on Private LB I loaded my trained model [https://www.kaggle.com/boyarintsev/last-best-model](url) to the competition. I made a notebook [https://www.kaggle.com/boyarintsev/make-answer-new/](url) to run the model on test dataset on kaggle platform on 2-3 of May. And then I stuck. Because the notebook I created was consuming too much memory. It was really strange, because that notebook was just a copy from my local research and works fine on my laptop. I spent 5-6 days just to avoid the issue of memory consumption, but I didn't succeed :( \nI gave up, when the competition was over, but then I noticed the button \"late submission\". So I decided that I still have a chance to upload my solution. And I started again to struggle against the memory consumption. I succeeded ten minutes ago, but then I found out that I can't upload new solutions :( \n\nPlease allow me to compete with that solution. \nYou can check that the model I am using was uploaded on 2nd of May [https://www.kaggle.com/boyarintsev/last-best-model](url). \n\nI just really upset about the situation because now I am only a top of 86% in this competition and it is really disappointing me, as I treat kaggle account as a part of my portfolio. I spent a lot of time to solve this competition, because I knew just basic things about the segmentation task before. \n\nAnyway, thanks for the competition! I liked it!\n\nBest regards,\nArtem Boyarintsev.", "1306065": "boyarintsev I'm sorry you feel this way, but unfortunately there are no do overs on Kaggle unless it affects all participants. The competition rules are clearly mentioned and you can find additional documentation here https://www.kaggle.com/docs/competitions#kernels-only-FAQ for future reference.\n\nThe reason you faced memory issues is because the private test set contained an extremely large image that causes you to run out of RAM when using the Tiffle library. I think this was discussed once or twice on the discussion forum which is why several public kernels switched from tiffle to rasterio.\n\nThe best thing you can do is to learn from this mistake and move on to the next competition. I can guarantee you that there will be more disappointments in future competitions, but that is a part of competing here on Kaggle. Wish you the best of luck in your next competition.", "1306380": "I'm sorry for you feelings, but as Yovin points out, this is a code competition and you need to submit codes to get scored in private set." }, "source": "meta" }