![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3654943%2Fbaac80f7c496430a84752c5c32f875a2%2FScreenshot%202019-12-30%20at%206.24.59%20PM.png?generation=1577710531029449&alt=media\")
I failed submitting 13 times, and finally succeeded in submission.\nHere, I show my faults. I hope this will help.\n(Let me apologize for my poor English. )
\n\nAs this is already pointed out in other discussion threads, we should delete variables that we don't use any more. In addition, the following snippet may be helpful for searching variables using high memory:\n```\nimport sys
\n\nprint(\"{}{: >25}{}{: >10}{}\".format('|','Variable Name','|','Memory','|'))\nprint(\" ------------------------------------ \")\nfor var_name in dir():\n if not var_name.startswith(\"_\") and sys.getsizeof(eval(var_name)) > 10000: # change this\n print(\"{}{: >25}{}{: >10}{}\".format('|',var_name,'|',sys.getsizeof(eval(var_name)),'|'))\n```\ncf: https://qiita.com/AnchorBlues/items/883790e43417640140aa
\n\n[update]\n\"Submission scoring error\" seems also related to memory error.\nSo, try to delete variables not used anymore.
\n\nThis is also discussed in other thread.\nBecause test dataset size is similar to train dataset size, I tried to run by replacing test_image_data_[0~3].parquet with train_image_data_[0~3].parquet.\nAnd I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).
\n\nIn my case, predicted values of vowel_diacritic and of consonant_diacritic were mistakenly replaced. The range of these two label are different; vowel_diacritic 11, consonant_diacritic 7. I think before scoring the submission file, the values are checked for validity. So, I got \"Submission CSV Not Found\".
\n\n[update]
\n\n(This is a quote from the discussion below)
\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.
\n\nDocker version may be related to the error.\nhttps://www.kaggle.com/c/google-quest-challenge/discussion/121445\nYou may need run twice even after changing docker version.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/123355
", "messages": [ { "id": "705629", "postDate": "12/29/2019 07:46:44", "content": "I failed submitting 13 times, and finally succeeded in submission.\nHere, I show my faults. I hope this will help.\n(Let me apologize for my poor English. )
\n\nAs this is already pointed out in other discussion threads, we should delete variables that we don't use any more. In addition, the following snippet may be helpful for searching variables using high memory:\n```\nimport sys
\n\nprint(\"{}{: >25}{}{: >10}{}\".format('|','Variable Name','|','Memory','|'))\nprint(\" ------------------------------------ \")\nfor var_name in dir():\n if not var_name.startswith(\"_\") and sys.getsizeof(eval(var_name)) > 10000: # change this\n print(\"{}{: >25}{}{: >10}{}\".format('|',var_name,'|',sys.getsizeof(eval(var_name)),'|'))\n```\ncf: https://qiita.com/AnchorBlues/items/883790e43417640140aa
\n\n[update]\n\"Submission scoring error\" seems also related to memory error.\nSo, try to delete variables not used anymore.
\n\nThis is also discussed in other thread.\nBecause test dataset size is similar to train dataset size, I tried to run by replacing test_image_data_[0~3].parquet with train_image_data_[0~3].parquet.\nAnd I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).
\n\nIn my case, predicted values of vowel_diacritic and of consonant_diacritic were mistakenly replaced. The range of these two label are different; vowel_diacritic 11, consonant_diacritic 7. I think before scoring the submission file, the values are checked for validity. So, I got \"Submission CSV Not Found\".
\n\n[update]
\n\n(This is a quote from the discussion below)
\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.
\n\nDocker version may be related to the error.\nhttps://www.kaggle.com/c/google-quest-challenge/discussion/121445\nYou may need run twice even after changing docker version.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/123355
", "rawMarkdown": "I failed submitting 13 times, and finally succeeded in submission.\nHere, I show my faults. I hope this will help.\n(Let me apologize for my poor English. )\n\n### - **Notebook Exceeded Allowed Compute**\nAs this is already pointed out in other discussion threads, we should delete variables that we don't use any more. In addition, the following snippet may be helpful for searching variables using high memory:\n```\nimport sys\n\nprint(\"{}{: >25}{}{: >10}{}\".format('|','Variable Name','|','Memory','|'))\nprint(\" ------------------------------------ \")\nfor var_name in dir():\n if not var_name.startswith(\"_\") and sys.getsizeof(eval(var_name)) > 10000: # change this\n print(\"{}{: >25}{}{: >10}{}\".format('|',var_name,'|',sys.getsizeof(eval(var_name)),'|'))\n```\ncf: [https://qiita.com/AnchorBlues/items/883790e43417640140aa](https://qiita.com/AnchorBlues/items/883790e43417640140aa)\n\n[update]\n**\"Submission scoring error\"** seems also related to memory error.\nSo, try to delete variables not used anymore.\n\n### - **Notebook Timeout**\nThis is also discussed in other thread.\nBecause test dataset size is similar to train dataset size, I tried to run by replacing **test_image_data_[0~3].parquet** with **train_image_data_[0~3].parquet**.\nAnd I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).\n\n### - **Submission CSV Not Found**\nIn my case, predicted values of vowel_diacritic and of consonant_diacritic were mistakenly replaced. The range of these two label are different; vowel_diacritic 11, consonant_diacritic 7. I think before scoring the submission file, the values are checked for validity. So, I got \"Submission CSV Not Found\".\n\n[update]\n\n### - **Notebook threw exception**\n(This is a quote from the discussion below)\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.\n\nDocker version may be related to the error.\nhttps://www.kaggle.com/c/google-quest-challenge/discussion/121445\nYou may need run twice even after changing docker version.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/123355", "votes": null }, { "id": "705650", "postDate": "12/29/2019 08:36:33", "content": "Hey what about 'Notebook threw exception' error? What does this error mean? Is it the same as Notebook timeout?
", "rawMarkdown": "Hey what about 'Notebook threw exception' error? What does this error mean? Is it the same as Notebook timeout?", "votes": null }, { "id": "705716", "postDate": "12/29/2019 10:49:34", "content": "I didn't come across \"Notebook threw exception\"... but it seems different from \"Notebook timeout\".
\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.
\n\nI hope this may be useful.
", "rawMarkdown": "I didn't come across \"Notebook threw exception\"... but it seems different from \"Notebook timeout\".\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.\n\n1. Docker version may be related to the error.\nhttps://www.kaggle.com/c/google-quest-challenge/discussion/121445\n2. You may need run twice even after changing docker version.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/123355\n\nI hope this may be useful.", "votes": null }, { "id": "705758", "postDate": "12/29/2019 12:30:41", "content": "Thank you so much!!! It was a problem with Docker version. Changing it to 'Latest available' fixed it.
", "rawMarkdown": "Thank you so much!!! It was a problem with Docker version. Changing it to 'Latest available' fixed it.", "votes": null }, { "id": "706229", "postDate": "12/30/2019 05:12:59", "content": "Good luck!!!
", "rawMarkdown": "Good luck!!!", "votes": null }, { "id": "706452", "postDate": "12/30/2019 12:18:26", "content": "Hi Inoichan, this is a pretty handy post! Thank you very much for investigating all the possible errors and posting the possible solutions. Could you add the comment about the \"\"Notebook threw exception\" to the main body of the post so that new participants may easily find it?
", "rawMarkdown": "Hi Inoichan, this is a pretty handy post! Thank you very much for investigating all the possible errors and posting the possible solutions. Could you add the comment about the \"\"Notebook threw exception\" to the main body of the post so that new participants may easily find it?", "votes": null }, { "id": "706479", "postDate": "12/30/2019 12:55:49", "content": "Hi, what about \"Submission Scoring Error\" ? \n
Hi NJ7, I found a discussion about that error in the comments here,\nhttps://www.kaggle.com/c/bengaliai-cv19/discussion/123212\nIt could be a memory error.
", "rawMarkdown": "Hi NJ7, I found a discussion about that error in the comments here,\nhttps://www.kaggle.com/c/bengaliai-cv19/discussion/123212\nIt could be a memory error.", "votes": null }, { "id": "706595", "postDate": "12/30/2019 15:54:24", "content": "OK!
", "rawMarkdown": "OK!", "votes": null }, { "id": "706612", "postDate": "12/30/2019 16:07:12", "content": "I got Submission CSV Not Found 2 times and it finally turns out to be memory problem.\nActually we have to read only one parquat file into memory at one time when predicting, then delete it and load another parquat file. Repeat this 4 times...
In my case, when I tried to read all parquets at the same time, I got Notebook Exceeded Allowed Compute error. Why is it different...\nAnyway, I think, reading / deleting parquets 4 times is troublesome and wasteful.
\n\n\n\n
\n- Notebook Timeout\n This is also discussed in other thread.\n Because test dataset size is similar to train dataset size, I tried to run by replacing testimagedata[0~3].parquet with trainimagedata[0~3].parquet.\n And I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).
\n
I also had Notebook Timeout and using the train test everything seemed fine (i.e. GPU time 1h, CPU 4h).\nSolved optimizing the procedure for writing \"submission.csv\" file (can take a while when using the real test set).
", "rawMarkdown": "> - Notebook Timeout\nThis is also discussed in other thread.\nBecause test dataset size is similar to train dataset size, I tried to run by replacing testimagedata[0~3].parquet with trainimagedata[0~3].parquet.\nAnd I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).\n\nI also had Notebook Timeout and using the train test everything seemed fine (i.e. GPU time 1h, CPU 4h).\nSolved optimizing the procedure for writing \"submission.csv\" file (can take a while when using the real test set).", "votes": null }, { "id": "710978", "postDate": "01/05/2020 13:55:49", "content": "Happy Ney Year to all Kagglers!!\nIn my case I got multiple times Submission Scorring Error and Submission CSV Not Found.
\n\nI have checked my inference scheme against the train images and it finishes on time! Also the submission I'm sure is in the required format. Anyone else facing the same issues ? Any ideas are welcome.
\n\nI attach the inference code (it is the standard snippet found in many kernels, see: https://www.kaggle.com/kaushal2896/bengali-graphemes-starter-eda-multi-output-cnn)
\n\n```\npreds_dict = {\n 'grapheme_root': [],\n 'vowel_diacritic': [],\n 'consonant_diacritic': []}
\n\ncomponents = ['consonant_diacritic', 'grapheme_root', 'vowel_diacritic']\ntarget = [] # model predictions placeholder\nrow_id = [] # row_id place holder
\n\nfor fname in INFERENCE: # for i in range(4):\n print('+' * 20)\n print('Loading test file:', fname)\n df_test_img = pd.read_parquet(fname)
\n df_test_img.set_index('image_id', inplace=True)
print('Preprocessing test file:', fname)\nX_test = resize(df_test_img, need_progress_bar=False)\nX_test = X_test.values.reshape(-1, IMG_SIZE, IMG_SIZE, NO_CHANNELS)\n\nprint('Inference test file..')\npreds = model.predict(X_test, batch_size=BATCH_SIZE, verbose=1) \n\nfor i, p in tqdm(enumerate(preds_dict)):\n preds_dict[p] = np.argmax(preds[i], axis=1)\n\nfor k,id in enumerate(df_test_img.index.values): \n for i,comp in enumerate(components):\n id_sample=id+'_'+comp\n row_id.append(id_sample)\n target.append(preds_dict[comp][k])\n```
\n\nand for the submission file I have tried both ways as per below
\n\na)\n```\ndf_subm['target'] = np.array(target)
\n\ndf_subm.to_csv('submission.csv', index=False)\n\nb)\n
df_sample = pd.DataFrame({'row_id': row_id, 'target': np.array(target)}, \n columns = ['row_id','target'])
\n\ndf_sample .to_csv('submission.csv', index=False)\n```
\n\nwhere, INFERENCE contains the train/testimages for timing and submission respectively.
\nif DEBUG: \n print('Inference on Train Images for Timing..\\n')\n INFERENCE = TRAIN\nelse: \n print('Inference on Test Images for Submission..\\n')\n INFERENCE = TEST\n
Please, any advice/help will be more than welcome!
", "rawMarkdown": "Happy Ney Year to all Kagglers!!\nIn my case I got multiple times **Submission Scorring Error** and **Submission CSV Not Found**. \n\nI have checked my inference scheme against the train images and it finishes on time! Also the submission I'm sure is in the required format. Anyone else facing the same issues ? Any ideas are welcome. \n\n\nI attach the inference code (it is the standard snippet found in many kernels, see: https://www.kaggle.com/kaushal2896/bengali-graphemes-starter-eda-multi-output-cnn)\n\n```\npreds_dict = {\n 'grapheme_root': [],\n 'vowel_diacritic': [],\n 'consonant_diacritic': []}\n\ncomponents = ['consonant_diacritic', 'grapheme_root', 'vowel_diacritic']\ntarget = [] # model predictions placeholder\nrow_id = [] # row_id place holder\n\n\nfor fname in INFERENCE: # for i in range(4):\n print('+' * 20)\n print('Loading test file:', fname)\n df_test_img = pd.read_parquet(fname) \n df_test_img.set_index('image_id', inplace=True)\n \n print('Preprocessing test file:', fname)\n X_test = resize(df_test_img, need_progress_bar=False)\n X_test = X_test.values.reshape(-1, IMG_SIZE, IMG_SIZE, NO_CHANNELS)\n \n print('Inference test file..')\n preds = model.predict(X_test, batch_size=BATCH_SIZE, verbose=1) \n \n for i, p in tqdm(enumerate(preds_dict)):\n preds_dict[p] = np.argmax(preds[i], axis=1)\n \n for k,id in enumerate(df_test_img.index.values): \n for i,comp in enumerate(components):\n id_sample=id+'_'+comp\n row_id.append(id_sample)\n target.append(preds_dict[comp][k])\n```\n\nand for the submission file I have tried both ways as per below \n\na)\n```\ndf_subm['target'] = np.array(target)\n\n# create submission file\ndf_subm.to_csv('submission.csv', index=False)\n```\nb)\n```\n# # create a dataframe with the solutions \ndf_sample = pd.DataFrame({'row_id': row_id, 'target': np.array(target)}, \n columns = ['row_id','target'])\n\ndf_sample .to_csv('submission.csv', index=False)\n```\n\nwhere, `INFERENCE ` contains the train/testimages for timing and submission respectively. \n\n```\nif DEBUG: \n print('Inference on Train Images for Timing..\\n')\n INFERENCE = TRAIN\nelse: \n print('Inference on Test Images for Submission..\\n')\n INFERENCE = TEST\n```\n\nPlease, any advice/help will be more than welcome!", "votes": null }, { "id": "733415", "postDate": "01/31/2020 05:03:38", "content": "It is another competition, and tips seem to be posted to help solve it.
\n\n\n\n\n\n
\n- Resolving Submission Errors: Below is a quick rundown of the some errors that users may receive on submission. Note that we are not in a position to share more granular details of any precise exceptions/errors, as doing so would make the competition vulnerable to error-probing by savvy participants.
\n- Notebook Timeout: Your submission notebook exceeded the allowed runtime cap for the competition. Review the competition's Code Requirements for details, and note that the privately held rerun test set could be orders of magnitude larger than the publicly shared validation set.
\n- Submission Scoring Error: Your notebook generated a submission.csv file with incorrect format. Some examples causing this are having the wrong number of rows or columns, empty values, an incorrect datatype for a value, or invalid submission values from what is expected for the privately rerun test set.
\n- Notebook Threw Exception: While rerunning your code on the public test set (privately held), your notebook hit an error.
\n- Submission CSV Not Found: Your Notebook did not output a submission.csv file. Some examples causing this are if your Notebook times out, or if it runs into an error. There is a chance this will be the error returned instead of \"Notebook Timeout\" depending on error sequencing.\n No Submit to Competition button on my notebook's output: This indicates that you have enabled something that is prohibited for the submissions on that competition. Review the competition's Code Requirements for details
\n- Notebook Exceeded Allowed Compute: This indicates you have violated a Code Requirement constraint during the rerun.
\n- Kaggle Error: A rare system error. Please try resubmitting to resolve the error and contact Kaggle Support if it persists.
\n
https://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started
", "rawMarkdown": "It is another competition, and tips seem to be posted to help solve it.\n> \n- Resolving Submission Errors: Below is a quick rundown of the some errors that users may receive on submission. Note that we are not in a position to share more granular details of any precise exceptions/errors, as doing so would make the competition vulnerable to error-probing by savvy participants.\n- Notebook Timeout: Your submission notebook exceeded the allowed runtime cap for the competition. Review the competition's Code Requirements for details, and note that the privately held rerun test set could be orders of magnitude larger than the publicly shared validation set.\n- Submission Scoring Error: Your notebook generated a submission.csv file with incorrect format. Some examples causing this are having the wrong number of rows or columns, empty values, an incorrect datatype for a value, or invalid submission values from what is expected for the privately rerun test set.\n- Notebook Threw Exception: While rerunning your code on the public test set (privately held), your notebook hit an error.\n- Submission CSV Not Found: Your Notebook did not output a submission.csv file. Some examples causing this are if your Notebook times out, or if it runs into an error. There is a chance this will be the error returned instead of \"Notebook Timeout\" depending on error sequencing.\nNo Submit to Competition button on my notebook's output: This indicates that you have enabled something that is prohibited for the submissions on that competition. Review the competition's Code Requirements for details\n- Notebook Exceeded Allowed Compute: This indicates you have violated a Code Requirement constraint during the rerun.\n- Kaggle Error: A rare system error. Please try resubmitting to resolve the error and contact Kaggle Support if it persists.\n\nhttps://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started", "votes": null }, { "id": "744286", "postDate": "02/12/2020 17:42:57", "content": "I've been having huge issues with Notebook Exceeded Allowed Compute and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help
", "rawMarkdown": "I've been having huge issues with **Notebook Exceeded Allowed Compute** and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help", "votes": null } ], "comments": [ { "id": 705650, "author_name": "samfc10", "author_url": "", "post_date": "12/29/2019 08:36:33", "content": "Hey what about 'Notebook threw exception' error? What does this error mean? Is it the same as Notebook timeout?
", "votes": null, "replies": [ { "id": 705716, "author_name": "inoueu1", "author_url": "", "post_date": "12/29/2019 10:49:34", "content": "I didn't come across \"Notebook threw exception\"... but it seems different from \"Notebook timeout\".
\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.
\n\nI hope this may be useful.
", "votes": null, "replies": [] }, { "id": 705758, "author_name": "samfc10", "author_url": "", "post_date": "12/29/2019 12:30:41", "content": "Thank you so much!!! It was a problem with Docker version. Changing it to 'Latest available' fixed it.
", "votes": null, "replies": [] }, { "id": 706229, "author_name": "inoueu1", "author_url": "", "post_date": "12/30/2019 05:12:59", "content": "Good luck!!!
", "votes": null, "replies": [] }, { "id": 706452, "author_name": "reasat", "author_url": "", "post_date": "12/30/2019 12:18:26", "content": "Hi Inoichan, this is a pretty handy post! Thank you very much for investigating all the possible errors and posting the possible solutions. Could you add the comment about the \"\"Notebook threw exception\" to the main body of the post so that new participants may easily find it?
", "votes": null, "replies": [] }, { "id": 706595, "author_name": "inoueu1", "author_url": "", "post_date": "12/30/2019 15:54:24", "content": "OK!
", "votes": null, "replies": [] } ] }, { "id": 706479, "author_name": "namanj27", "author_url": "", "post_date": "12/30/2019 12:55:49", "content": "Hi, what about \"Submission Scoring Error\" ? \n
Hi NJ7, I found a discussion about that error in the comments here,\nhttps://www.kaggle.com/c/bengaliai-cv19/discussion/123212\nIt could be a memory error.
", "votes": null, "replies": [] } ] }, { "id": 706612, "author_name": "haqishen", "author_url": "", "post_date": "12/30/2019 16:07:12", "content": "I got Submission CSV Not Found 2 times and it finally turns out to be memory problem.\nActually we have to read only one parquat file into memory at one time when predicting, then delete it and load another parquat file. Repeat this 4 times...
In my case, when I tried to read all parquets at the same time, I got Notebook Exceeded Allowed Compute error. Why is it different...\nAnyway, I think, reading / deleting parquets 4 times is troublesome and wasteful.
\n\n\n\n
\n- Notebook Timeout\n This is also discussed in other thread.\n Because test dataset size is similar to train dataset size, I tried to run by replacing testimagedata[0~3].parquet with trainimagedata[0~3].parquet.\n And I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).
\n
I also had Notebook Timeout and using the train test everything seemed fine (i.e. GPU time 1h, CPU 4h).\nSolved optimizing the procedure for writing \"submission.csv\" file (can take a while when using the real test set).
", "votes": null, "replies": [] }, { "id": 710978, "author_name": "imeintanis", "author_url": "", "post_date": "01/05/2020 13:55:49", "content": "Happy Ney Year to all Kagglers!!\nIn my case I got multiple times Submission Scorring Error and Submission CSV Not Found.
\n\nI have checked my inference scheme against the train images and it finishes on time! Also the submission I'm sure is in the required format. Anyone else facing the same issues ? Any ideas are welcome.
\n\nI attach the inference code (it is the standard snippet found in many kernels, see: https://www.kaggle.com/kaushal2896/bengali-graphemes-starter-eda-multi-output-cnn)
\n\n```\npreds_dict = {\n 'grapheme_root': [],\n 'vowel_diacritic': [],\n 'consonant_diacritic': []}
\n\ncomponents = ['consonant_diacritic', 'grapheme_root', 'vowel_diacritic']\ntarget = [] # model predictions placeholder\nrow_id = [] # row_id place holder
\n\nfor fname in INFERENCE: # for i in range(4):\n print('+' * 20)\n print('Loading test file:', fname)\n df_test_img = pd.read_parquet(fname)
\n df_test_img.set_index('image_id', inplace=True)
print('Preprocessing test file:', fname)\nX_test = resize(df_test_img, need_progress_bar=False)\nX_test = X_test.values.reshape(-1, IMG_SIZE, IMG_SIZE, NO_CHANNELS)\n\nprint('Inference test file..')\npreds = model.predict(X_test, batch_size=BATCH_SIZE, verbose=1) \n\nfor i, p in tqdm(enumerate(preds_dict)):\n preds_dict[p] = np.argmax(preds[i], axis=1)\n\nfor k,id in enumerate(df_test_img.index.values): \n for i,comp in enumerate(components):\n id_sample=id+'_'+comp\n row_id.append(id_sample)\n target.append(preds_dict[comp][k])\n```
\n\nand for the submission file I have tried both ways as per below
\n\na)\n```\ndf_subm['target'] = np.array(target)
\n\ndf_subm.to_csv('submission.csv', index=False)\n\nb)\n
df_sample = pd.DataFrame({'row_id': row_id, 'target': np.array(target)}, \n columns = ['row_id','target'])
\n\ndf_sample .to_csv('submission.csv', index=False)\n```
\n\nwhere, INFERENCE contains the train/testimages for timing and submission respectively.
\nif DEBUG: \n print('Inference on Train Images for Timing..\\n')\n INFERENCE = TRAIN\nelse: \n print('Inference on Test Images for Submission..\\n')\n INFERENCE = TEST\n
Please, any advice/help will be more than welcome!
", "votes": null, "replies": [] }, { "id": 733415, "author_name": "wakamezake", "author_url": "", "post_date": "01/31/2020 05:03:38", "content": "It is another competition, and tips seem to be posted to help solve it.
\n\n\n\n\n\n
\n- Resolving Submission Errors: Below is a quick rundown of the some errors that users may receive on submission. Note that we are not in a position to share more granular details of any precise exceptions/errors, as doing so would make the competition vulnerable to error-probing by savvy participants.
\n- Notebook Timeout: Your submission notebook exceeded the allowed runtime cap for the competition. Review the competition's Code Requirements for details, and note that the privately held rerun test set could be orders of magnitude larger than the publicly shared validation set.
\n- Submission Scoring Error: Your notebook generated a submission.csv file with incorrect format. Some examples causing this are having the wrong number of rows or columns, empty values, an incorrect datatype for a value, or invalid submission values from what is expected for the privately rerun test set.
\n- Notebook Threw Exception: While rerunning your code on the public test set (privately held), your notebook hit an error.
\n- Submission CSV Not Found: Your Notebook did not output a submission.csv file. Some examples causing this are if your Notebook times out, or if it runs into an error. There is a chance this will be the error returned instead of \"Notebook Timeout\" depending on error sequencing.\n No Submit to Competition button on my notebook's output: This indicates that you have enabled something that is prohibited for the submissions on that competition. Review the competition's Code Requirements for details
\n- Notebook Exceeded Allowed Compute: This indicates you have violated a Code Requirement constraint during the rerun.
\n- Kaggle Error: A rare system error. Please try resubmitting to resolve the error and contact Kaggle Support if it persists.
\n
https://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started
", "votes": null, "replies": [] }, { "id": 744286, "author_name": "yeayates21", "author_url": "", "post_date": "02/12/2020 17:42:57", "content": "I've been having huge issues with Notebook Exceeded Allowed Compute and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "705629": "I failed submitting 13 times, and finally succeeded in submission.\nHere, I show my faults. I hope this will help.\n(Let me apologize for my poor English. )\n\n### - **Notebook Exceeded Allowed Compute**\nAs this is already pointed out in other discussion threads, we should delete variables that we don't use any more. In addition, the following snippet may be helpful for searching variables using high memory:\n```\nimport sys\n\nprint(\"{}{: >25}{}{: >10}{}\".format('|','Variable Name','|','Memory','|'))\nprint(\" ------------------------------------ \")\nfor var_name in dir():\n if not var_name.startswith(\"_\") and sys.getsizeof(eval(var_name)) > 10000: # change this\n print(\"{}{: >25}{}{: >10}{}\".format('|',var_name,'|',sys.getsizeof(eval(var_name)),'|'))\n```\ncf: [https://qiita.com/AnchorBlues/items/883790e43417640140aa](https://qiita.com/AnchorBlues/items/883790e43417640140aa)\n\n[update]\n**\"Submission scoring error\"** seems also related to memory error.\nSo, try to delete variables not used anymore.\n\n### - **Notebook Timeout**\nThis is also discussed in other thread.\nBecause test dataset size is similar to train dataset size, I tried to run by replacing **test_image_data_[0~3].parquet** with **train_image_data_[0~3].parquet**.\nAnd I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).\n\n### - **Submission CSV Not Found**\nIn my case, predicted values of vowel_diacritic and of consonant_diacritic were mistakenly replaced. The range of these two label are different; vowel_diacritic 11, consonant_diacritic 7. I think before scoring the submission file, the values are checked for validity. So, I got \"Submission CSV Not Found\".\n\n[update]\n\n### - **Notebook threw exception**\n(This is a quote from the discussion below)\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.\n\nDocker version may be related to the error.\nhttps://www.kaggle.com/c/google-quest-challenge/discussion/121445\nYou may need run twice even after changing docker version.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/123355", "705650": "Hey what about 'Notebook threw exception' error? What does this error mean? Is it the same as Notebook timeout?", "705716": "I didn't come across \"Notebook threw exception\"... but it seems different from \"Notebook timeout\".\n\nAbout \"Notebook threw exception\", I found some discussions in other competitions.\n\n1. Docker version may be related to the error.\nhttps://www.kaggle.com/c/google-quest-challenge/discussion/121445\n2. You may need run twice even after changing docker version.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/123355\n\nI hope this may be useful.", "705758": "Thank you so much!!! It was a problem with Docker version. Changing it to 'Latest available' fixed it.", "706229": "Good luck!!!", "706452": "Hi Inoichan, this is a pretty handy post! Thank you very much for investigating all the possible errors and posting the possible solutions. Could you add the comment about the \"\"Notebook threw exception\" to the main body of the post so that new participants may easily find it?", "706479": "Hi, what about \"Submission Scoring Error\" ? \n", "706510": "Hi NJ7, I found a discussion about that error in the comments here,\nhttps://www.kaggle.com/c/bengaliai-cv19/discussion/123212\nIt could be a memory error.", "706595": "OK!", "706612": "I got `Submission CSV Not Found` 2 times and it finally turns out to be memory problem.\nActually we have to read only one parquat file into memory at one time when predicting, then delete it and load another parquat file. Repeat this 4 times...", "706935": "In my case, when I tried to read all parquets at the same time, I got `Notebook Exceeded Allowed Compute` error. Why is it different...\nAnyway, I think, reading / deleting parquets 4 times is troublesome and wasteful.", "707173": "> - Notebook Timeout\nThis is also discussed in other thread.\nBecause test dataset size is similar to train dataset size, I tried to run by replacing testimagedata[0~3].parquet with trainimagedata[0~3].parquet.\nAnd I checked if all processes are completed within the time limit(gpu: 2h, cpu: 9h).\n\nI also had Notebook Timeout and using the train test everything seemed fine (i.e. GPU time 1h, CPU 4h).\nSolved optimizing the procedure for writing \"submission.csv\" file (can take a while when using the real test set).", "710978": "Happy Ney Year to all Kagglers!!\nIn my case I got multiple times **Submission Scorring Error** and **Submission CSV Not Found**. \n\nI have checked my inference scheme against the train images and it finishes on time! Also the submission I'm sure is in the required format. Anyone else facing the same issues ? Any ideas are welcome. \n\n\nI attach the inference code (it is the standard snippet found in many kernels, see: https://www.kaggle.com/kaushal2896/bengali-graphemes-starter-eda-multi-output-cnn)\n\n```\npreds_dict = {\n 'grapheme_root': [],\n 'vowel_diacritic': [],\n 'consonant_diacritic': []}\n\ncomponents = ['consonant_diacritic', 'grapheme_root', 'vowel_diacritic']\ntarget = [] # model predictions placeholder\nrow_id = [] # row_id place holder\n\n\nfor fname in INFERENCE: # for i in range(4):\n print('+' * 20)\n print('Loading test file:', fname)\n df_test_img = pd.read_parquet(fname) \n df_test_img.set_index('image_id', inplace=True)\n \n print('Preprocessing test file:', fname)\n X_test = resize(df_test_img, need_progress_bar=False)\n X_test = X_test.values.reshape(-1, IMG_SIZE, IMG_SIZE, NO_CHANNELS)\n \n print('Inference test file..')\n preds = model.predict(X_test, batch_size=BATCH_SIZE, verbose=1) \n \n for i, p in tqdm(enumerate(preds_dict)):\n preds_dict[p] = np.argmax(preds[i], axis=1)\n \n for k,id in enumerate(df_test_img.index.values): \n for i,comp in enumerate(components):\n id_sample=id+'_'+comp\n row_id.append(id_sample)\n target.append(preds_dict[comp][k])\n```\n\nand for the submission file I have tried both ways as per below \n\na)\n```\ndf_subm['target'] = np.array(target)\n\n# create submission file\ndf_subm.to_csv('submission.csv', index=False)\n```\nb)\n```\n# # create a dataframe with the solutions \ndf_sample = pd.DataFrame({'row_id': row_id, 'target': np.array(target)}, \n columns = ['row_id','target'])\n\ndf_sample .to_csv('submission.csv', index=False)\n```\n\nwhere, `INFERENCE ` contains the train/testimages for timing and submission respectively. \n\n```\nif DEBUG: \n print('Inference on Train Images for Timing..\\n')\n INFERENCE = TRAIN\nelse: \n print('Inference on Test Images for Submission..\\n')\n INFERENCE = TEST\n```\n\nPlease, any advice/help will be more than welcome!", "733415": "It is another competition, and tips seem to be posted to help solve it.\n> \n- Resolving Submission Errors: Below is a quick rundown of the some errors that users may receive on submission. Note that we are not in a position to share more granular details of any precise exceptions/errors, as doing so would make the competition vulnerable to error-probing by savvy participants.\n- Notebook Timeout: Your submission notebook exceeded the allowed runtime cap for the competition. Review the competition's Code Requirements for details, and note that the privately held rerun test set could be orders of magnitude larger than the publicly shared validation set.\n- Submission Scoring Error: Your notebook generated a submission.csv file with incorrect format. Some examples causing this are having the wrong number of rows or columns, empty values, an incorrect datatype for a value, or invalid submission values from what is expected for the privately rerun test set.\n- Notebook Threw Exception: While rerunning your code on the public test set (privately held), your notebook hit an error.\n- Submission CSV Not Found: Your Notebook did not output a submission.csv file. Some examples causing this are if your Notebook times out, or if it runs into an error. There is a chance this will be the error returned instead of \"Notebook Timeout\" depending on error sequencing.\nNo Submit to Competition button on my notebook's output: This indicates that you have enabled something that is prohibited for the submissions on that competition. Review the competition's Code Requirements for details\n- Notebook Exceeded Allowed Compute: This indicates you have violated a Code Requirement constraint during the rerun.\n- Kaggle Error: A rare system error. Please try resubmitting to resolve the error and contact Kaggle Support if it persists.\n\nhttps://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started", "744286": "I've been having huge issues with **Notebook Exceeded Allowed Compute** and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help" }, "source": "meta" }