I read the suggestion made by @abdrah in https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/454732
\n\n\nI suspect that the issue with your submissions might be related to the use of \"1 0\" to indicate the absence of a mask in your predictions. I observed that a very basic model of mine, which predicted masks for all inputs, had its submission accepted. However, when I used a more sophisticated model that included a couple of instances with \"1 0\" (indicating no mask), the submission was not successful.
\n
I submitted two notebooks, the one with \"1 0\" as all predictions failed and the one with all predictions \"1 1\" subbed succesfully
\nSample Code was from @jirkaborovec 's notebook
\nLink to my submission notebook:
\nhttps://www.kaggle.com/code/harshitsheoran/notebooke824b39459/notebook
There seems to be an error when we predict \"1 0\" which is supposed to be for no-mask predicted situations, if all values are in the submission \"1 0\", the submission fails
\n", "messages": [ { "id": "2522701", "postDate": "11/13/2023 00:28:14", "content": "I read the suggestion made by @abdrah in https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/454732
\n\n\nI suspect that the issue with your submissions might be related to the use of \"1 0\" to indicate the absence of a mask in your predictions. I observed that a very basic model of mine, which predicted masks for all inputs, had its submission accepted. However, when I used a more sophisticated model that included a couple of instances with \"1 0\" (indicating no mask), the submission was not successful.
\n
I submitted two notebooks, the one with \"1 0\" as all predictions failed and the one with all predictions \"1 1\" subbed succesfully
\nSample Code was from @jirkaborovec 's notebook
\nLink to my submission notebook:
\nhttps://www.kaggle.com/code/harshitsheoran/notebooke824b39459/notebook
There seems to be an error when we predict \"1 0\" which is supposed to be for no-mask predicted situations, if all values are in the submission \"1 0\", the submission fails
\n", "rawMarkdown": "I read the suggestion made by @abdrah in https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/454732\n\n>I suspect that the issue with your submissions might be related to the use of \"1 0\" to indicate the absence of a mask in your predictions. I observed that a very basic model of mine, which predicted masks for all inputs, had its submission accepted. However, when I used a more sophisticated model that included a couple of instances with \"1 0\" (indicating no mask), the submission was not successful.\n\nI submitted two notebooks, the one with \"1 0\" as all predictions failed and the one with all predictions \"1 1\" subbed succesfully\n\nSample Code was from @jirkaborovec 's [notebook](https://www.kaggle.com/code/jirkaborovec/sennet-hoa-rle-decode-encode-demo-submission) \n\nLink to my submission notebook:\nhttps://www.kaggle.com/code/harshitsheoran/notebooke824b39459/notebook\n\nThere seems to be an error when we predict \"1 0\" which is supposed to be for no-mask predicted situations, if all values are in the submission \"1 0\", the submission fails\n\n@sohier \n@ryanholbrook", "votes": null }, { "id": "2522781", "postDate": "11/13/2023 03:44:29", "content": "As you said, using \"1 1\" instead of \"1 0\", the submission was successful without any issues! Thank you!
", "rawMarkdown": "As you said, using \"1 1\" instead of \"1 0\", the submission was successful without any issues! Thank you!", "votes": null }, { "id": "2522801", "postDate": "11/13/2023 04:38:55", "content": "I did change the empty mask prediction from \"1 0\" to \"1 1\" and now it is graded correctly.
\nNo doubt, \"1 0\" must be causing some kind of problem.
\nI am curious as to why this behavior is happening, but I have published my baseline.
\nI would be happy if it would be of help to anyone else having trouble with submitting.
[Train] https://www.kaggle.com/code/kashiwaba/sennet-hoa-train-unet-simple-baseline
\n[Inference] https://www.kaggle.com/code/kashiwaba/sennet-hoa-inference-unet-simple-baseline
Thanks for the find but 1 1 is suboptimal as it produces 1 pixel in the top left corner and is thus penalized by the metrics.
", "rawMarkdown": "Thanks for the find but 1 1 is suboptimal as it produces 1 pixel in the top left corner and is thus penalized by the metrics.", "votes": null }, { "id": "2523152", "postDate": "11/13/2023 10:12:38", "content": "Yes, you are right, although, this might be the best we can do to avoid this before Kaggle team finds out what is going on
\nOne thing I noticed is that with \"1 0\" the submission did run \"Scoring submission\" status for 2 minutes, so maybe it is selective ids that cause errors when they are \"1 0\" and not all
", "rawMarkdown": "Yes, you are right, although, this might be the best we can do to avoid this before Kaggle team finds out what is going on\n\nOne thing I noticed is that with \"1 0\" the submission did run \"Scoring submission\" status for 2 minutes, so maybe it is selective ids that cause errors when they are \"1 0\" and not all", "votes": null }, { "id": "2523259", "postDate": "11/13/2023 11:27:33", "content": "I have tried \"0 0\" for the empty mask, which passes the scoring.
\nBut it just gave me a 0 grade.
Hmmm, i just tried 0 0 and submission failed
", "rawMarkdown": "Hmmm, i just tried 0 0 and submission failed", "votes": null }, { "id": "2523590", "postDate": "11/13/2023 15:34:08", "content": "I tried \"2 0\" before and got a Scoring Error.
\nPerhaps a length of 0 would cause some kind of error.
\nhttps://www.kaggle.com/code/kashiwaba/sennet-hoa-inference-unet-simple-baseline/notebook?scriptVersionId=150464285
Hi everyone,
\nI'm looking into it. I'll try to have a fix up sometime today. Thank you.
", "rawMarkdown": "Hi everyone,\n\nI'm looking into it. I'll try to have a fix up sometime today. Thank you.", "votes": null }, { "id": "2523689", "postDate": "11/13/2023 16:38:41", "content": "There is an error, but still we can use this.
", "rawMarkdown": "There is an error, but still we can use this.", "votes": null }, { "id": "2524045", "postDate": "11/14/2023 00:28:58", "content": "Glad it worked out for you siwooyong! 🥳🥳🥳
\nI used \"1 0\" and was able to submit two notebooks successfully.
\nIn one notebook, I retrained the model locally, uploaded the model and submit it using the same RLE code, notebook ran but submission failed. Then, i updatd the RLE code via replacing the emptry string from \"1 0\" to \"1 1\", and submission failed again.
Will be taking a break 😴 and wait for Ryan's update :)
", "rawMarkdown": "Glad it worked out for you siwooyong! 🥳🥳🥳\n\nI used \"1 0\" and was able to submit two notebooks successfully.\nIn one notebook, I retrained the model locally, uploaded the model and submit it using the same RLE code, notebook ran but submission failed. Then, i updatd the RLE code via replacing the emptry string from \"1 0\" to \"1 1\", and submission failed again.\n\nWill be taking a break 😴 and wait for Ryan's update :)", "votes": null }, { "id": "2527921", "postDate": "11/17/2023 00:37:22", "content": "Hi @kashiwaba , I just tried tour inference notebook and I got an error lol… It is very strange
", "rawMarkdown": "Hi @kashiwaba , I just tried tour inference notebook and I got an error lol... It is very strange", "votes": null }, { "id": "2527947", "postDate": "11/17/2023 01:51:38", "content": "I am in the same situation.
\nI have tried submitting both the version that previously achieved 0.147 and the version that changed the empty mask to \"1 0\" and both are getting a Scoring Error.
\nAt least there is no random element in my baseline notebook and it has passed the scoring process in the past, so there should be no formatting problem. Nine times out of ten, this is due to staff modifying the scoring system, so I don't think there is anything we can do at this point. All we can do is hope that the staff will address the issue soon.
@kashiwaba Thanks for the update, let's wait then :)
", "rawMarkdown": "kashiwaba Thanks for the update, let's wait then :)", "votes": null } ], "comments": [ { "id": 2522781, "author_name": "siwooyong", "author_url": "", "post_date": "11/13/2023 03:44:29", "content": "As you said, using \"1 1\" instead of \"1 0\", the submission was successful without any issues! Thank you!
", "votes": null, "replies": [ { "id": 2524045, "author_name": "pangyuteng", "author_url": "", "post_date": "11/14/2023 00:28:58", "content": "Glad it worked out for you siwooyong! 🥳🥳🥳
\nI used \"1 0\" and was able to submit two notebooks successfully.
\nIn one notebook, I retrained the model locally, uploaded the model and submit it using the same RLE code, notebook ran but submission failed. Then, i updatd the RLE code via replacing the emptry string from \"1 0\" to \"1 1\", and submission failed again.
Will be taking a break 😴 and wait for Ryan's update :)
", "votes": null, "replies": [] } ] }, { "id": 2522801, "author_name": "kashiwaba", "author_url": "", "post_date": "11/13/2023 04:38:55", "content": "I did change the empty mask prediction from \"1 0\" to \"1 1\" and now it is graded correctly.
\nNo doubt, \"1 0\" must be causing some kind of problem.
\nI am curious as to why this behavior is happening, but I have published my baseline.
\nI would be happy if it would be of help to anyone else having trouble with submitting.
[Train] https://www.kaggle.com/code/kashiwaba/sennet-hoa-train-unet-simple-baseline
\n[Inference] https://www.kaggle.com/code/kashiwaba/sennet-hoa-inference-unet-simple-baseline
Hi @kashiwaba , I just tried tour inference notebook and I got an error lol… It is very strange
", "votes": null, "replies": [ { "id": 2527947, "author_name": "kashiwaba", "author_url": "", "post_date": "11/17/2023 01:51:38", "content": "I am in the same situation.
\nI have tried submitting both the version that previously achieved 0.147 and the version that changed the empty mask to \"1 0\" and both are getting a Scoring Error.
\nAt least there is no random element in my baseline notebook and it has passed the scoring process in the past, so there should be no formatting problem. Nine times out of ten, this is due to staff modifying the scoring system, so I don't think there is anything we can do at this point. All we can do is hope that the staff will address the issue soon.
@kashiwaba Thanks for the update, let's wait then :)
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2523035, "author_name": "sakvaua", "author_url": "", "post_date": "11/13/2023 07:41:51", "content": "Thanks for the find but 1 1 is suboptimal as it produces 1 pixel in the top left corner and is thus penalized by the metrics.
", "votes": null, "replies": [ { "id": 2523152, "author_name": "harshitsheoran", "author_url": "", "post_date": "11/13/2023 10:12:38", "content": "Yes, you are right, although, this might be the best we can do to avoid this before Kaggle team finds out what is going on
\nOne thing I noticed is that with \"1 0\" the submission did run \"Scoring submission\" status for 2 minutes, so maybe it is selective ids that cause errors when they are \"1 0\" and not all
", "votes": null, "replies": [] } ] }, { "id": 2523259, "author_name": "james8lo", "author_url": "", "post_date": "11/13/2023 11:27:33", "content": "I have tried \"0 0\" for the empty mask, which passes the scoring.
\nBut it just gave me a 0 grade.
Hmmm, i just tried 0 0 and submission failed
", "votes": null, "replies": [ { "id": 2523590, "author_name": "kashiwaba", "author_url": "", "post_date": "11/13/2023 15:34:08", "content": "I tried \"2 0\" before and got a Scoring Error.
\nPerhaps a length of 0 would cause some kind of error.
\nhttps://www.kaggle.com/code/kashiwaba/sennet-hoa-inference-unet-simple-baseline/notebook?scriptVersionId=150464285
Hi everyone,
\nI'm looking into it. I'll try to have a fix up sometime today. Thank you.
", "votes": null, "replies": [] }, { "id": 2523689, "author_name": "vikrampython", "author_url": "", "post_date": "11/13/2023 16:38:41", "content": "There is an error, but still we can use this.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2522701": "I read the suggestion made by @abdrah in https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/454732\n\n>I suspect that the issue with your submissions might be related to the use of \"1 0\" to indicate the absence of a mask in your predictions. I observed that a very basic model of mine, which predicted masks for all inputs, had its submission accepted. However, when I used a more sophisticated model that included a couple of instances with \"1 0\" (indicating no mask), the submission was not successful.\n\nI submitted two notebooks, the one with \"1 0\" as all predictions failed and the one with all predictions \"1 1\" subbed succesfully\n\nSample Code was from @jirkaborovec 's [notebook](https://www.kaggle.com/code/jirkaborovec/sennet-hoa-rle-decode-encode-demo-submission) \n\nLink to my submission notebook:\nhttps://www.kaggle.com/code/harshitsheoran/notebooke824b39459/notebook\n\nThere seems to be an error when we predict \"1 0\" which is supposed to be for no-mask predicted situations, if all values are in the submission \"1 0\", the submission fails\n\n@sohier \n@ryanholbrook", "2522781": "As you said, using \"1 1\" instead of \"1 0\", the submission was successful without any issues! Thank you!", "2522801": "I did change the empty mask prediction from \"1 0\" to \"1 1\" and now it is graded correctly.\nNo doubt, \"1 0\" must be causing some kind of problem.\nI am curious as to why this behavior is happening, but I have published my baseline.\nI would be happy if it would be of help to anyone else having trouble with submitting.\n\n[Train] https://www.kaggle.com/code/kashiwaba/sennet-hoa-train-unet-simple-baseline\n[Inference] https://www.kaggle.com/code/kashiwaba/sennet-hoa-inference-unet-simple-baseline", "2523035": "Thanks for the find but 1 1 is suboptimal as it produces 1 pixel in the top left corner and is thus penalized by the metrics.", "2523152": "Yes, you are right, although, this might be the best we can do to avoid this before Kaggle team finds out what is going on\n\nOne thing I noticed is that with \"1 0\" the submission did run \"Scoring submission\" status for 2 minutes, so maybe it is selective ids that cause errors when they are \"1 0\" and not all", "2523259": "I have tried \"0 0\" for the empty mask, which passes the scoring.\nBut it just gave me a 0 grade.", "2523515": "Hmmm, i just tried 0 0 and submission failed", "2523590": "I tried \"2 0\" before and got a Scoring Error.\nPerhaps a length of 0 would cause some kind of error.\nhttps://www.kaggle.com/code/kashiwaba/sennet-hoa-inference-unet-simple-baseline/notebook?scriptVersionId=150464285", "2523619": "Hi everyone,\n\nI'm looking into it. I'll try to have a fix up sometime today. Thank you.", "2523689": "There is an error, but still we can use this.", "2524045": "Glad it worked out for you siwooyong! 🥳🥳🥳\n\nI used \"1 0\" and was able to submit two notebooks successfully.\nIn one notebook, I retrained the model locally, uploaded the model and submit it using the same RLE code, notebook ran but submission failed. Then, i updatd the RLE code via replacing the emptry string from \"1 0\" to \"1 1\", and submission failed again.\n\nWill be taking a break 😴 and wait for Ryan's update :)", "2527921": "Hi @kashiwaba , I just tried tour inference notebook and I got an error lol... It is very strange", "2527947": "I am in the same situation.\nI have tried submitting both the version that previously achieved 0.147 and the version that changed the empty mask to \"1 0\" and both are getting a Scoring Error.\nAt least there is no random element in my baseline notebook and it has passed the scoring process in the past, so there should be no formatting problem. Nine times out of ten, this is due to staff modifying the scoring system, so I don't think there is anything we can do at this point. All we can do is hope that the staff will address the issue soon.", "2527956": "kashiwaba Thanks for the update, let's wait then :)" }, "source": "meta" }