I have submitted a notebook running for more than 5 days despite the error message like \"Submission CSV Not Found,\" \"Submission Scoring Error,\" etc.
\nAfter my investigation, finally, I make a successful kernel submission.
\n1. The reason for the \"Submission Scoring Error\" is because the submission CSV file did not follow the scoring order. To solve it, you need to read the sample CSV from ../input/hubmap-kidney-segmentation/test/sample_submission.csv as your template.
For example, my submission with \"Submission Scoring Error\" showed my predicted order is
\naa05346ff
\n2ec3f1bb9
\n3589adb90
\nd488c759a
\n57512b7f1
But the right predicted order should be
\n2ec3f1bb9
\n3589adb90
\nd488c759a
\naa05346ff
\n57512b7f1
2. Submission CSV Not Found. This is because the test names of the images from the public dataset and the private are different. Previously, I used
\n\"p = pathlib.Path(DATA_PATH)
\nfor i, filename in tqdm(enumerate(p.glob('test/*.tiff'))):\"
to get the test tiff image. However, it seems that the images in the private dataset did not follow the rules used in the public dataset.
\nThus, please read sample_submission.csv from the test as the reference.
\nIf you did not follow the rules, then your kernel submission will fall into endless running.
\nCheck the working notebook https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter-sub for example that is written by Iafoss, https://www.kaggle.com/tikutiku
\nThanks very much, Iafoss.
\nYour notebook leads to my successful submission.
I have submitted a notebook running for more than 5 days despite the error message like \"Submission CSV Not Found,\" \"Submission Scoring Error,\" etc.
\nAfter my investigation, finally, I make a successful kernel submission.
\n1. The reason for the \"Submission Scoring Error\" is because the submission CSV file did not follow the scoring order. To solve it, you need to read the sample CSV from ../input/hubmap-kidney-segmentation/test/sample_submission.csv as your template.
For example, my submission with \"Submission Scoring Error\" showed my predicted order is
\naa05346ff
\n2ec3f1bb9
\n3589adb90
\nd488c759a
\n57512b7f1
But the right predicted order should be
\n2ec3f1bb9
\n3589adb90
\nd488c759a
\naa05346ff
\n57512b7f1
2. Submission CSV Not Found. This is because the test names of the images from the public dataset and the private are different. Previously, I used
\n\"p = pathlib.Path(DATA_PATH)
\nfor i, filename in tqdm(enumerate(p.glob('test/*.tiff'))):\"
to get the test tiff image. However, it seems that the images in the private dataset did not follow the rules used in the public dataset.
\nThus, please read sample_submission.csv from the test as the reference.
\nIf you did not follow the rules, then your kernel submission will fall into endless running.
\nCheck the working notebook https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter-sub for example that is written by Iafoss, https://www.kaggle.com/tikutiku
\nThanks very much, Iafoss.
\nYour notebook leads to my successful submission.
@goldgarudagoldgaruda Congrats to solve the problem, but the notebook is written by @iafoss :)
", "rawMarkdown": "goldgarudagoldgaruda Congrats to solve the problem, but the notebook is written by @iafoss :)", "votes": null }, { "id": "1244999", "postDate": "03/19/2021 11:43:43", "content": "Thanks. Updated. :)
", "rawMarkdown": "Thanks. Updated. :)", "votes": null }, { "id": "1247405", "postDate": "03/21/2021 17:24:01", "content": "Thanks all! This was driving me crazy. If you use glob to get the public test tiffs and ignore the private ones it works. So couldn't figure out what was going on.
\nAlso there are 2 sample_submission.csv files both seemed to work for me. Now I can start trying to use the new data instead of trying to get my submissions to work :)
", "rawMarkdown": "Thanks all! This was driving me crazy. If you use glob to get the public test tiffs and ignore the private ones it works. So couldn't figure out what was going on.\n\nAlso there are 2 sample_submission.csv files both seemed to work for me. Now I can start trying to use the new data instead of trying to get my submissions to work :)", "votes": null }, { "id": "1250267", "postDate": "03/23/2021 22:26:31", "content": "For me its still not working
", "rawMarkdown": "For me its still not working", "votes": null }, { "id": "1250831", "postDate": "03/24/2021 10:01:52", "content": "We just solved this issue now. But still we don't know why this code is working suddenly
\nfor step,idx enumerate(test_files):\n\n sample_submission.reset_index().to_csv('/kaggle/working/submission.csv',index=False)\n sample_submission.reset_index().to_csv('submission.csv', index = False)\n\nsample_submission = sample_submission.reset_index()\nsample_submission.to_csv('/kaggle/working/submission.csv',index=False)\nsample_submission.to_csv('submission.csv', index = False)\nPerhaps, your notebook collapsed when dealing with the private tiff. So optimize your memory at first. I suspect that there may be larger tiff images in the private dataset.
", "rawMarkdown": "Perhaps, your notebook collapsed when dealing with the private tiff. So optimize your memory at first. I suspect that there may be larger tiff images in the private dataset.", "votes": null }, { "id": "1251426", "postDate": "03/24/2021 18:56:54", "content": "Its probably not that, because the error is \"Notebook Threw Exception\". I had this memory problem before in the previous dataset, and i solved that mapping the image from disk. So actually im not dealing with the whole image in memory. Using try catch i discovered that the problem is in the function tifffile.imread(), so probably is not reading the file, but im using this sample_submission.csv as template.
", "rawMarkdown": "Its probably not that, because the error is \"Notebook Threw Exception\". I had this memory problem before in the previous dataset, and i solved that mapping the image from disk. So actually im not dealing with the whole image in memory. Using try catch i discovered that the problem is in the function tifffile.imread(), so probably is not reading the file, but im using this sample_submission.csv as template.", "votes": null }, { "id": "1252785", "postDate": "03/26/2021 03:44:05", "content": "I trained a different model, then followed the submission code by @iafoss, but still get the 'Submission Scoring Error'. The scoring order is right, so I don't know what is the problem. May be I try to use both train and test code by @iafoss.
", "rawMarkdown": "I trained a different model, then followed the submission code by @iafoss, but still get the 'Submission Scoring Error'. The scoring order is right, so I don't know what is the problem. May be I try to use both train and test code by @iafoss.", "votes": null }, { "id": "1254574", "postDate": "03/27/2021 20:27:27", "content": "Have you been able to fix the issue?
", "rawMarkdown": "Have you been able to fix the issue?", "votes": null }, { "id": "1257074", "postDate": "03/30/2021 13:56:23", "content": "I am busy at last weekend, so I can't try other ways to solve it. May be I will try again in next days.
", "rawMarkdown": "I am busy at last weekend, so I can't try other ways to solve it. May be I will try again in next days.", "votes": null }, { "id": "1260877", "postDate": "04/02/2021 13:38:00", "content": "Thanks a lot @goldgarudagoldgaruda for pointing to solution and @iafoss for the notebook!
\nEventually I was able to solve the \"endless submission\" problem using following:
\nYou are very welcome, I'm happy that it is solved.
", "rawMarkdown": "You are very welcome, I'm happy that it is solved.", "votes": null }, { "id": "1265821", "postDate": "04/07/2021 08:28:02", "content": "@goldgarudagoldgaruda i follow the order same as there in sample submission.
\nI get sub scoring error after an hour time.
\nCan there be GPU out of memory error ?
That might be one possible reason. Be careful, there are two sample_submission.csv files under 2 different folders. sub scoring error often means your csv file is not satisfying the scoring rule.
", "rawMarkdown": "That might be one possible reason. Be careful, there are two sample_submission.csv files under 2 different folders. sub scoring error often means your csv file is not satisfying the scoring rule.", "votes": null } ], "comments": [ { "id": 1244998, "author_name": "tikutiku", "author_url": "", "post_date": "03/19/2021 11:41:49", "content": "@goldgarudagoldgaruda Congrats to solve the problem, but the notebook is written by @iafoss :)
", "votes": null, "replies": [] }, { "id": 1244999, "author_name": "goldgarudagoldgaruda", "author_url": "", "post_date": "03/19/2021 11:43:43", "content": "Thanks. Updated. :)
", "votes": null, "replies": [] }, { "id": 1247405, "author_name": "governor", "author_url": "", "post_date": "03/21/2021 17:24:01", "content": "Thanks all! This was driving me crazy. If you use glob to get the public test tiffs and ignore the private ones it works. So couldn't figure out what was going on.
\nAlso there are 2 sample_submission.csv files both seemed to work for me. Now I can start trying to use the new data instead of trying to get my submissions to work :)
", "votes": null, "replies": [] }, { "id": 1250267, "author_name": "joaorodriguez", "author_url": "", "post_date": "03/23/2021 22:26:31", "content": "For me its still not working
", "votes": null, "replies": [ { "id": 1250917, "author_name": "goldgarudagoldgaruda", "author_url": "", "post_date": "03/24/2021 11:00:25", "content": "Perhaps, your notebook collapsed when dealing with the private tiff. So optimize your memory at first. I suspect that there may be larger tiff images in the private dataset.
", "votes": null, "replies": [] }, { "id": 1251426, "author_name": "joaorodriguez", "author_url": "", "post_date": "03/24/2021 18:56:54", "content": "Its probably not that, because the error is \"Notebook Threw Exception\". I had this memory problem before in the previous dataset, and i solved that mapping the image from disk. So actually im not dealing with the whole image in memory. Using try catch i discovered that the problem is in the function tifffile.imread(), so probably is not reading the file, but im using this sample_submission.csv as template.
", "votes": null, "replies": [] } ] }, { "id": 1250831, "author_name": "jinssaa", "author_url": "", "post_date": "03/24/2021 10:01:52", "content": "We just solved this issue now. But still we don't know why this code is working suddenly
\nfor step,idx enumerate(test_files):\n\n sample_submission.reset_index().to_csv('/kaggle/working/submission.csv',index=False)\n sample_submission.reset_index().to_csv('submission.csv', index = False)\n\nsample_submission = sample_submission.reset_index()\nsample_submission.to_csv('/kaggle/working/submission.csv',index=False)\nsample_submission.to_csv('submission.csv', index = False)\nI trained a different model, then followed the submission code by @iafoss, but still get the 'Submission Scoring Error'. The scoring order is right, so I don't know what is the problem. May be I try to use both train and test code by @iafoss.
", "votes": null, "replies": [ { "id": 1254574, "author_name": "nauyan", "author_url": "", "post_date": "03/27/2021 20:27:27", "content": "Have you been able to fix the issue?
", "votes": null, "replies": [] }, { "id": 1257074, "author_name": "ntthientn", "author_url": "", "post_date": "03/30/2021 13:56:23", "content": "I am busy at last weekend, so I can't try other ways to solve it. May be I will try again in next days.
", "votes": null, "replies": [] } ] }, { "id": 1260877, "author_name": "eytankats", "author_url": "", "post_date": "04/02/2021 13:38:00", "content": "Thanks a lot @goldgarudagoldgaruda for pointing to solution and @iafoss for the notebook!
\nEventually I was able to solve the \"endless submission\" problem using following:
\nYou are very welcome, I'm happy that it is solved.
", "votes": null, "replies": [] } ] }, { "id": 1265821, "author_name": "jaideepvalani", "author_url": "", "post_date": "04/07/2021 08:28:02", "content": "@goldgarudagoldgaruda i follow the order same as there in sample submission.
\nI get sub scoring error after an hour time.
\nCan there be GPU out of memory error ?
That might be one possible reason. Be careful, there are two sample_submission.csv files under 2 different folders. sub scoring error often means your csv file is not satisfying the scoring rule.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1244981": "I have submitted a notebook running for more than 5 days despite the error message like \"Submission CSV Not Found,\" \"Submission Scoring Error,\" etc.\n\nAfter my investigation, finally, I make a successful kernel submission.\n**1.** The reason for the \"Submission Scoring Error\" is because the submission CSV file did not follow the scoring order. To solve it, you need to read the sample CSV from ../input/hubmap-kidney-segmentation/test/sample_submission.csv as your template.\n\nFor example, my submission with \"Submission Scoring Error\" showed my predicted order is \naa05346ff\n2ec3f1bb9\n3589adb90\nd488c759a\n57512b7f1\n\nBut the right predicted order should be \n2ec3f1bb9\n3589adb90\nd488c759a\naa05346ff\n57512b7f1\n\n**2.** Submission CSV Not Found. This is because the test names of the images from the public dataset and the private are different. Previously, I used \n\n\"p = pathlib.Path(DATA_PATH)\nfor i, filename in tqdm(enumerate(p.glob('test/*.tiff'))):\"\n\nto get the test tiff image. However, it seems that the images in the private dataset did not follow the rules used in the public dataset.\n\nThus, please read sample_submission.csv from the test as the reference.\n\nIf you did not follow the rules, then your kernel submission will fall into endless running.\n\nCheck the working notebook https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter-sub for example that is written by Iafoss, https://www.kaggle.com/tikutiku \n\nThanks very much, Iafoss. \nYour notebook leads to my successful submission.", "1244998": "goldgarudagoldgaruda Congrats to solve the problem, but the notebook is written by @iafoss :)", "1244999": "Thanks. Updated. :)", "1247405": "Thanks all! This was driving me crazy. If you use glob to get the public test tiffs and ignore the private ones it works. So couldn't figure out what was going on.\n\nAlso there are 2 sample_submission.csv files both seemed to work for me. Now I can start trying to use the new data instead of trying to get my submissions to work :)", "1250267": "For me its still not working", "1250831": "We just solved this issue now. But still we don't know why this code is working suddenly\n\n1. memory optimization\n2. save sample_submission per sample idx(seems like weird but just share it for whom need to try)\n\n```\nfor step,idx enumerate(test_files):\n\n sample_submission.reset_index().to_csv('/kaggle/working/submission.csv',index=False)\n sample_submission.reset_index().to_csv('submission.csv', index = False)\n\nsample_submission = sample_submission.reset_index()\nsample_submission.to_csv('/kaggle/working/submission.csv',index=False)\nsample_submission.to_csv('submission.csv', index = False)\n```", "1250917": "Perhaps, your notebook collapsed when dealing with the private tiff. So optimize your memory at first. I suspect that there may be larger tiff images in the private dataset.", "1251426": "Its probably not that, because the error is \"Notebook Threw Exception\". I had this memory problem before in the previous dataset, and i solved that mapping the image from disk. So actually im not dealing with the whole image in memory. Using try catch i discovered that the problem is in the function tifffile.imread(), so probably is not reading the file, but im using this sample_submission.csv as template.", "1252785": "I trained a different model, then followed the submission code by @iafoss, but still get the 'Submission Scoring Error'. The scoring order is right, so I don't know what is the problem. May be I try to use both train and test code by @iafoss.", "1254574": "Have you been able to fix the issue?", "1257074": "I am busy at last weekend, so I can't try other ways to solve it. May be I will try again in next days.", "1260877": "Thanks a lot @goldgarudagoldgaruda for pointing to solution and @iafoss for the notebook!\n\nEventually I was able to solve the \"endless submission\" problem using following:\n- proper reading images with *rasterio* package.\n- prediction order based on *sample_submission.csv*.", "1261339": "You are very welcome, I'm happy that it is solved.", "1265821": "goldgarudagoldgaruda i follow the order same as there in sample submission. \nI get sub scoring error after an hour time. \nCan there be GPU out of memory error ?", "1266024": "That might be one possible reason. Be careful, there are two sample_submission.csv files under 2 different folders. sub scoring error often means your csv file is not satisfying the scoring rule." }, "source": "meta" }