My personal kernel keep got the \"Submission Scoring Error\" when I submit to the leaderboard (although there is no error from the public part). I am trying to figure out what is the root cause and think the forum may have some people who may have insights to help me.
\nPotential cause:
\nsub_df.loc[sub_df['ImageWidth'] == 2048]. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\" PS Wondering Kaggle and organizer can provide more support on the error for the private part. This is a kind of black box and hard to debug the issue.
", "messages": [ { "id": 1266733, "postDate": "2021-04-08T03:20:14.587Z", "content": "My personal kernel keep got the \"Submission Scoring Error\" when I submit to the leaderboard (although there is no error from the public part). I am trying to figure out what is the root cause and think the forum may have some people who may have insights to help me.
\nPotential cause:
\nsub_df.loc[sub_df['ImageWidth'] == 2048]. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\" PS Wondering Kaggle and organizer can provide more support on the error for the private part. This is a kind of black box and hard to debug the issue.
", "rawMarkdown": "My personal kernel keep got the \"Submission Scoring Error\" when I submit to the leaderboard (although there is no error from the public part). I am trying to figure out what is the root cause and think the forum may have some people who may have insights to help me.\n\nPotential cause:\n1. Out of time. Unlikely, the kernel is only using 4900s for the public eval dataset, so it is unlikely the public + private will take more than 9 hours (also wondering if there will be other error showing if out of time?)\n2. Memory issue? Personally, I have no idea whether the memory will become an issue. Not sure if someone has figured out a way to debug this?\n3. Format issue. Possible but I do have a successful submission (0.2x) when I filter the data using `sub_df.loc[sub_df['ImageWidth'] == 2048]`. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\" \n\nPS Wondering Kaggle and organizer can provide more support on the error for the private part. This is a kind of black box and hard to debug the issue. ", "votes": 3 }, { "id": 1274228, "postDate": "2021-04-15T05:55:33.960Z", "content": "I got \"Submission Scoring Error\" when I submitted huge submission file. Approximately 8 GiB or so including public and private.
", "rawMarkdown": "I got \"Submission Scoring Error\" when I submitted huge submission file. Approximately 8 GiB or so including public and private.", "votes": 1 }, { "id": 1267949, "postDate": "2021-04-09T01:50:21.423Z", "content": "Found 2 issues in my code
\nIn my previous submission, I got the same error message if there are more than 1 mask for the same 1 cell.
\nI guess “1 mask for 1 cell” is required to be scored properly.
\nAnother possibility is the format of mask encoding string.
Sorry what do you mean by 1 mask for 1 cell? Can you give me an example of your error submission?
", "rawMarkdown": "Sorry what do you mean by 1 mask for 1 cell? Can you give me an example of your error submission?" }, { "id": 1266945, "postDate": "2021-04-08T07:34:44.040Z", "content": "For example, suppose that \"eNoLCAgIsAQABJ4Beg==\" is the mask of the most left upper corner cell of the image id X, we have to write \"0 0.001 eNoLCAgIsAQABJ4Beg== 1 0.1 eNoLCAgIsAQABJ4Beg== 2 0.005 eNoLCAgIsAQABJ4Beg==…and so on\"
\nI guess we shouldn't have another mask string for that cell, like \"0 0.001 eNoLCAgIsAQABJ4Beg== 1 0.1 HIAaagsCAgIssagJIANDIiOKFA== 2 0.005 eNoLCAgIsAQABJ4Beg==\".
I see your point. Let me check the code. But my code do work if i only select 2048 images, and after that what i did is just combine [2048] + [3072] + … and output to the df
", "rawMarkdown": "I see your point. Let me check the code. But my code do work if i only select 2048 images, and after that what i did is just combine [2048] + [3072] + ... and output to the df", "votes": 1 }, { "id": 1266965, "postDate": "2021-04-08T07:58:42.697Z", "content": "I see. How about selecting only 1728 or 3072 images?
", "rawMarkdown": "I see. How about selecting only 1728 or 3072 images?" }, { "id": 1267883, "postDate": "2021-04-08T22:50:44.453Z", "content": "Seems the issue is from those images not equal to 2048.
", "rawMarkdown": "Seems the issue is from those images not equal to 2048. " } ] }, { "id": 1274494, "postDate": "2021-04-15T11:05:20.030Z", "content": "Hiii, I'm having the same issues.
\nI am processing batches of images of the same size, but my notebook can't handle all the data, it takes too long.
\nThe only sumbission I achieved is running only the most frequent size images. Now I'm trying processing the most 2 frequent sizes, but let's see if I don't get a Notebook Timeout.
I think for segmentation you shouldn't get time out. I guess you may want to optimize your code (but I think you can also try fast submit which only process the public test dataset for now)
", "rawMarkdown": "I think for segmentation you shouldn't get time out. I guess you may want to optimize your code (but I think you can also try fast submit which only process the public test dataset for now)" } ] }, { "id": 1266971, "postDate": "2021-04-08T08:03:25.540Z", "content": "Did you find the reason for the error?
", "rawMarkdown": "Did you find the reason for the error?" }, { "id": 1266913, "postDate": "2021-04-08T07:04:21.337Z", "content": "@lnhtrang Wondering if we can somehow get access to the error info for the private part. Right now we can only guess what happend to debug
", "rawMarkdown": "@lnhtrang Wondering if we can somehow get access to the error info for the private part. Right now we can only guess what happend to debug", "replies": [ { "id": 1267129, "postDate": "2021-04-08T10:37:08.787Z", "content": "Hi! I can only see your submission error here:
\n<Error>\n<Code>NoSuchKey</Code>\n<Message>The specified key does not exist.</Message>\n<Details>No such object: kaggle-competitions-submissions/23823/20381194.raw</Details>\n</Error>\nNoSuchKey\nThanks for the information! Seems there is some error in my code caused it cannot work for the image with 3072 size
", "rawMarkdown": "Thanks for the information! Seems there is some error in my code caused it cannot work for the image with 3072 size" }, { "id": 1277894, "postDate": "2021-04-19T10:52:14.103Z", "content": "Hi, after I come access some posts having same issue, I understand that submission notebook has to read file sample_submission.csv and produce encode string for all images and make prediction with each segmentation cell.
\nPlease correct my thought if it is wrong. Thank you very much.
Format issue. Possible but I do have a successful submission (0.2x) when I filter the data using sub_df.loc[sub_df['ImageWidth'] == 2048]. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\"\nCould it be that the mask exceeds the size of the image?
", "rawMarkdown": "```\nFormat issue. Possible but I do have a successful submission (0.2x) when I filter the data using sub_df.loc[sub_df['ImageWidth'] == 2048]. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\"\n```\nCould it be that the mask exceeds the size of the image?", "replies": [ { "id": 1266774, "postDate": "2021-04-08T04:28:55.527Z", "content": "Not sure, have you met such error before?
", "rawMarkdown": "Not sure, have you met such error before?", "votes": 1 }, { "id": 1266789, "postDate": "2021-04-08T04:44:58.823Z", "content": "No. I am just guessing. You can find the difference between success and failure.
", "rawMarkdown": "No. I am just guessing. You can find the difference between success and failure." } ] } ], "comments": [ { "id": 1274228, "author_name": "ONODERA", "author_url": "", "post_date": "2021-04-15T05:55:33.960000", "content": "I got \"Submission Scoring Error\" when I submitted huge submission file. Approximately 8 GiB or so including public and private.
", "votes": 1, "replies": [] }, { "id": 1267949, "author_name": "Strideradu", "author_url": "", "post_date": "2021-04-09T01:50:21.423000", "content": "Found 2 issues in my code
\nIn my previous submission, I got the same error message if there are more than 1 mask for the same 1 cell.
\nI guess “1 mask for 1 cell” is required to be scored properly.
\nAnother possibility is the format of mask encoding string.
Sorry what do you mean by 1 mask for 1 cell? Can you give me an example of your error submission?
", "votes": 0, "replies": [] }, { "id": 1266945, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-04-08T07:34:44.040000", "content": "For example, suppose that \"eNoLCAgIsAQABJ4Beg==\" is the mask of the most left upper corner cell of the image id X, we have to write \"0 0.001 eNoLCAgIsAQABJ4Beg== 1 0.1 eNoLCAgIsAQABJ4Beg== 2 0.005 eNoLCAgIsAQABJ4Beg==…and so on\"
\nI guess we shouldn't have another mask string for that cell, like \"0 0.001 eNoLCAgIsAQABJ4Beg== 1 0.1 HIAaagsCAgIssagJIANDIiOKFA== 2 0.005 eNoLCAgIsAQABJ4Beg==\".
I see your point. Let me check the code. But my code do work if i only select 2048 images, and after that what i did is just combine [2048] + [3072] + … and output to the df
", "votes": 1, "replies": [] }, { "id": 1266965, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-04-08T07:58:42.697000", "content": "I see. How about selecting only 1728 or 3072 images?
", "votes": 0, "replies": [] }, { "id": 1267883, "author_name": "Strideradu", "author_url": "", "post_date": "2021-04-08T22:50:44.453000", "content": "Seems the issue is from those images not equal to 2048.
", "votes": 0, "replies": [] } ] }, { "id": 1274494, "author_name": "glopezzz", "author_url": "", "post_date": "2021-04-15T11:05:20.030000", "content": "Hiii, I'm having the same issues.
\nI am processing batches of images of the same size, but my notebook can't handle all the data, it takes too long.
\nThe only sumbission I achieved is running only the most frequent size images. Now I'm trying processing the most 2 frequent sizes, but let's see if I don't get a Notebook Timeout.
I think for segmentation you shouldn't get time out. I guess you may want to optimize your code (but I think you can also try fast submit which only process the public test dataset for now)
", "votes": 0, "replies": [] } ] }, { "id": 1266971, "author_name": "Zaakcii Ru", "author_url": "", "post_date": "2021-04-08T08:03:25.540000", "content": "Did you find the reason for the error?
", "votes": 0, "replies": [] }, { "id": 1266913, "author_name": "Strideradu", "author_url": "", "post_date": "2021-04-08T07:04:21.337000", "content": "@lnhtrang Wondering if we can somehow get access to the error info for the private part. Right now we can only guess what happend to debug
", "votes": 0, "replies": [ { "id": 1267129, "author_name": "Trang Le", "author_url": "", "post_date": "2021-04-08T10:37:08.787000", "content": "Hi! I can only see your submission error here:
\n<Error>\n<Code>NoSuchKey</Code>\n<Message>The specified key does not exist.</Message>\n<Details>No such object: kaggle-competitions-submissions/23823/20381194.raw</Details>\n</Error>\nThanks for the information! Seems there is some error in my code caused it cannot work for the image with 3072 size
", "votes": 0, "replies": [] }, { "id": 1277894, "author_name": "Anh-Vu Mai-Nguyen", "author_url": "", "post_date": "2021-04-19T10:52:14.103000", "content": "Hi, after I come access some posts having same issue, I understand that submission notebook has to read file sample_submission.csv and produce encode string for all images and make prediction with each segmentation cell.
\nPlease correct my thought if it is wrong. Thank you very much.
Format issue. Possible but I do have a successful submission (0.2x) when I filter the data using sub_df.loc[sub_df['ImageWidth'] == 2048]. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\"\nCould it be that the mask exceeds the size of the image?
", "votes": 0, "replies": [ { "id": 1266774, "author_name": "Strideradu", "author_url": "", "post_date": "2021-04-08T04:28:55.527000", "content": "Not sure, have you met such error before?
", "votes": 1, "replies": [] }, { "id": 1266789, "author_name": "Alien", "author_url": "", "post_date": "2021-04-08T04:44:58.823000", "content": "No. I am just guessing. You can find the difference between success and failure.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1266733": "My personal kernel keep got the \"Submission Scoring Error\" when I submit to the leaderboard (although there is no error from the public part). I am trying to figure out what is the root cause and think the forum may have some people who may have insights to help me.\n\nPotential cause:\n1. Out of time. Unlikely, the kernel is only using 4900s for the public eval dataset, so it is unlikely the public + private will take more than 9 hours (also wondering if there will be other error showing if out of time?)\n2. Memory issue? Personally, I have no idea whether the memory will become an issue. Not sure if someone has figured out a way to debug this?\n3. Format issue. Possible but I do have a successful submission (0.2x) when I filter the data using `sub_df.loc[sub_df['ImageWidth'] == 2048]`. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\" \n\nPS Wondering Kaggle and organizer can provide more support on the error for the private part. This is a kind of black box and hard to debug the issue. ", "1274228": "I got \"Submission Scoring Error\" when I submitted huge submission file. Approximately 8 GiB or so including public and private.", "1267949": "Found 2 issues in my code\n1. I hardcoded mask size to be 2048 in the dataset although I have arg for this in init() ... I guess this caused the scoring error\n2. Another issue I set padding = False and that will cause an issue when segment 1728 images\nNow I am save and update new submission hope this can work", "1266881": "In my previous submission, I got the same error message if there are more than 1 mask for the same 1 cell.\nI guess “1 mask for 1 cell” is required to be scored properly.\nAnother possibility is the format of mask encoding string.", "1274494": "Hiii, I'm having the same issues.\n\nI am processing batches of images of the same size, but my notebook can't handle all the data, it takes too long.\nThe only sumbission I achieved is running only the most frequent size images. Now I'm trying processing the most 2 frequent sizes, but let's see if I don't get a `Notebook Timeout`.", "1266971": "Did you find the reason for the error?", "1266913": "@lnhtrang Wondering if we can somehow get access to the error info for the private part. Right now we can only guess what happend to debug", "1266764": "```\nFormat issue. Possible but I do have a successful submission (0.2x) when I filter the data using sub_df.loc[sub_df['ImageWidth'] == 2048]. But later on when I changed 2048 to other image sizes and concatenate all results together then I got the \"Submission Scoring Error\"\n```\nCould it be that the mask exceeds the size of the image?" } }