{ "id": 238011, "title": "Only 529 (less than half of the competitors) got scores different from zero on private datasets. ", "url": "/competitions/hubmap-kidney-segmentation/discussion/238011", "author_name": "rpsantosa_kaggle", "post_date": "2021-05-11T01:09:57.122000", "votes": 7, "comment_count": 15, "views": 0, "content": "

Hope this isnt the final result.

", "messages": [ { "id": 1301171, "postDate": "2021-05-11T01:09:57.123Z", "content": "

Hope this isnt the final result.

", "rawMarkdown": "Hope this isnt the final result. ", "votes": 7 }, { "id": 1301178, "postDate": "2021-05-11T01:14:37.967Z", "content": "

is it possible it's all due to the way deepflash2 kernel handles errors?

", "rawMarkdown": "is it possible it's all due to the way deepflash2 kernel handles errors?", "votes": 1, "replies": [ { "id": 1301221, "postDate": "2021-05-11T01:43:03.347Z", "content": "

not really. My team, for example, didnt. Over 700 teams using the same strats on submission seems very hard to happen.

", "rawMarkdown": "not really. My team, for example, didnt. Over 700 teams using the same strats on submission seems very hard to happen.", "votes": 1 }, { "id": 1301298, "postDate": "2021-05-11T02:36:39.523Z", "content": "

you are right, however, 429 teams have not made a single submission for at least 1 to 6 months. I am assuming these folks gave up due to all the labeling issues. So the question is about 344 teams. Wouldn't be surprised if some are new to kaggle and just submitted the public notebooks, and then you have the unfortunate folks (a couple hundred of them), who used the deepflash2 notebook, not realizing it was failing on the private data.
\nNot sure if this is the case, just trying to get answers….

", "rawMarkdown": "you are right, however, 429 teams have not made a single submission for at least 1 to 6 months. I am assuming these folks gave up due to all the labeling issues. So the question is about 344 teams. Wouldn't be surprised if some are new to kaggle and just submitted the public notebooks, and then you have the unfortunate folks (a couple hundred of them), who used the deepflash2 notebook, not realizing it was failing on the private data.\nNot sure if this is the case, just trying to get answers...." } ] }, { "id": 1301516, "postDate": "2021-05-11T05:49:28.107Z", "content": "

I am waiting to hear any confirmation but it is possible the issue with 0 scores has to do with the sample submission csv and there being another copy in the test folder (normally not the case). So only if you derived the ids from the image names did your submissions work. And of course if you got past the queue issues.

\n

UPDATE - see https://www.kaggle.com/matjes/hubmap-efficient-sampling-deepflash2-sub/comments v15 changes 4 days ago to use rasterio instead of tifffile due to potential large image in private test.

", "rawMarkdown": "I am waiting to hear any confirmation but it is possible the issue with 0 scores has to do with the sample submission csv and there being another copy in the test folder (normally not the case). So only if you derived the ids from the image names did your submissions work. And of course if you got past the queue issues. \n\nUPDATE - see https://www.kaggle.com/matjes/hubmap-efficient-sampling-deepflash2-sub/comments v15 changes 4 days ago to use rasterio instead of tifffile due to potential large image in private test. ", "votes": 2, "replies": [ { "id": 1301809, "postDate": "2021-05-11T08:53:08.857Z", "content": "

Yes, I did a complete rewrite in v15 because we were not sure whether the submission on the private data was working - we also contacted @addisonhoward on this matter but we didn't get a satisfying answer. We did not know if v14 and lower were working until the end of the challenge!

", "rawMarkdown": "Yes, I did a complete rewrite in v15 because we were not sure whether the submission on the private data was working - we also contacted @addisonhoward on this matter but we didn't get a satisfying answer. We did not know if v14 and lower were working until the end of the challenge!", "votes": 2 }, { "id": 1302499, "postDate": "2021-05-11T15:19:22.323Z", "content": "

I tried both with/without using sample submission csv. And it is not restrict to deepflash notebooks.

", "rawMarkdown": "I tried both with/without using sample submission csv. And it is not restrict to deepflash notebooks. \n\n" }, { "id": 1302605, "postDate": "2021-05-11T16:15:04.657Z", "content": "

The sample submissions - both copies - in the hidden test set contained all test IDs, both public and private.

", "rawMarkdown": "The sample submissions - both copies - in the hidden test set contained all test IDs, both public and private." }, { "id": 1303009, "postDate": "2021-05-11T21:05:34.203Z", "content": "

Please find out what happens.

", "rawMarkdown": "Please find out what happens." } ] }, { "id": 1301737, "postDate": "2021-05-11T08:09:40.407Z", "content": "

the number of 529 doesn't change over the time… bug competition?

", "rawMarkdown": "the number of 529 doesn't change over the time... bug competition? " }, { "id": 1301641, "postDate": "2021-05-11T07:13:53.177Z", "content": "

The game with \"thresholds\" is too risky. And seems like most code get 0 cuz can't load big image.
\nTo be honest: most of submitters just copied public code with such 0 score

", "rawMarkdown": "The game with \"thresholds\" is too risky. And seems like most code get 0 cuz can't load big image.\nTo be honest: most of submitters just copied public code with such 0 score" }, { "id": 1301415, "postDate": "2021-05-11T04:21:29.190Z", "content": "

Same here. Our best-performing score on the public LB scored 0 on the private. This is beyond frustrating.

", "rawMarkdown": "Same here. Our best-performing score on the public LB scored 0 on the private. This is beyond frustrating.", "replies": [ { "id": 1301817, "postDate": "2021-05-11T08:57:01.983Z", "content": "

Do you understand what the reason of it?

", "rawMarkdown": "Do you understand what the reason of it? " }, { "id": 1302484, "postDate": "2021-05-11T15:14:52.957Z", "content": "

I'm resubmitting my notebooks to see how it works.

", "rawMarkdown": "I'm resubmitting my notebooks to see how it works." } ] }, { "id": 1301256, "postDate": "2021-05-11T02:02:55.187Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 1301262, "postDate": "2021-05-11T02:05:39.090Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 1301178, "author_name": "andras", "author_url": "", "post_date": "2021-05-11T01:14:37.967000", "content": "

is it possible it's all due to the way deepflash2 kernel handles errors?

", "votes": 1, "replies": [ { "id": 1301221, "author_name": "rpsantosa_kaggle", "author_url": "", "post_date": "2021-05-11T01:43:03.347000", "content": "

not really. My team, for example, didnt. Over 700 teams using the same strats on submission seems very hard to happen.

", "votes": 1, "replies": [] }, { "id": 1301298, "author_name": "andras", "author_url": "", "post_date": "2021-05-11T02:36:39.523000", "content": "

you are right, however, 429 teams have not made a single submission for at least 1 to 6 months. I am assuming these folks gave up due to all the labeling issues. So the question is about 344 teams. Wouldn't be surprised if some are new to kaggle and just submitted the public notebooks, and then you have the unfortunate folks (a couple hundred of them), who used the deepflash2 notebook, not realizing it was failing on the private data.
\nNot sure if this is the case, just trying to get answers….

", "votes": 0, "replies": [] } ] }, { "id": 1301516, "author_name": "something4kag", "author_url": "", "post_date": "2021-05-11T05:49:28.107000", "content": "

I am waiting to hear any confirmation but it is possible the issue with 0 scores has to do with the sample submission csv and there being another copy in the test folder (normally not the case). So only if you derived the ids from the image names did your submissions work. And of course if you got past the queue issues.

\n

UPDATE - see https://www.kaggle.com/matjes/hubmap-efficient-sampling-deepflash2-sub/comments v15 changes 4 days ago to use rasterio instead of tifffile due to potential large image in private test.

", "votes": 2, "replies": [ { "id": 1301809, "author_name": "Matthias Griebel", "author_url": "", "post_date": "2021-05-11T08:53:08.857000", "content": "

Yes, I did a complete rewrite in v15 because we were not sure whether the submission on the private data was working - we also contacted @addisonhoward on this matter but we didn't get a satisfying answer. We did not know if v14 and lower were working until the end of the challenge!

", "votes": 2, "replies": [] }, { "id": 1302499, "author_name": "rpsantosa_kaggle", "author_url": "", "post_date": "2021-05-11T15:19:22.323000", "content": "

I tried both with/without using sample submission csv. And it is not restrict to deepflash notebooks.

", "votes": 0, "replies": [] }, { "id": 1302605, "author_name": "Phil Culliton", "author_url": "", "post_date": "2021-05-11T16:15:04.657000", "content": "

The sample submissions - both copies - in the hidden test set contained all test IDs, both public and private.

", "votes": 0, "replies": [] }, { "id": 1303009, "author_name": "anthony", "author_url": "", "post_date": "2021-05-11T21:05:34.203000", "content": "

Please find out what happens.

", "votes": 0, "replies": [] } ] }, { "id": 1301737, "author_name": "anthony", "author_url": "", "post_date": "2021-05-11T08:09:40.407000", "content": "

the number of 529 doesn't change over the time… bug competition?

", "votes": 0, "replies": [] }, { "id": 1301641, "author_name": "Leigh", "author_url": "", "post_date": "2021-05-11T07:13:53.177000", "content": "

The game with \"thresholds\" is too risky. And seems like most code get 0 cuz can't load big image.
\nTo be honest: most of submitters just copied public code with such 0 score

", "votes": 0, "replies": [] }, { "id": 1301415, "author_name": "DennisSakva", "author_url": "", "post_date": "2021-05-11T04:21:29.190000", "content": "

Same here. Our best-performing score on the public LB scored 0 on the private. This is beyond frustrating.

", "votes": 0, "replies": [ { "id": 1301817, "author_name": "anthony", "author_url": "", "post_date": "2021-05-11T08:57:01.983000", "content": "

Do you understand what the reason of it?

", "votes": 0, "replies": [] }, { "id": 1302484, "author_name": "DennisSakva", "author_url": "", "post_date": "2021-05-11T15:14:52.957000", "content": "

I'm resubmitting my notebooks to see how it works.

", "votes": 0, "replies": [] } ] }, { "id": 1301256, "author_name": "", "author_url": "", "post_date": "2021-05-11T02:02:55.187000", "content": "", "votes": 0, "replies": [ { "id": 1301262, "author_name": "", "author_url": "", "post_date": "2021-05-11T02:05:39.090000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1301171": "Hope this isnt the final result. ", "1301178": "is it possible it's all due to the way deepflash2 kernel handles errors?", "1301516": "I am waiting to hear any confirmation but it is possible the issue with 0 scores has to do with the sample submission csv and there being another copy in the test folder (normally not the case). So only if you derived the ids from the image names did your submissions work. And of course if you got past the queue issues. \n\nUPDATE - see https://www.kaggle.com/matjes/hubmap-efficient-sampling-deepflash2-sub/comments v15 changes 4 days ago to use rasterio instead of tifffile due to potential large image in private test. ", "1301737": "the number of 529 doesn't change over the time... bug competition? ", "1301641": "The game with \"thresholds\" is too risky. And seems like most code get 0 cuz can't load big image.\nTo be honest: most of submitters just copied public code with such 0 score", "1301415": "Same here. Our best-performing score on the public LB scored 0 on the private. This is beyond frustrating.", "1301256": "" } }