I have a notebook that definitely throws an error. If I run it cell by cell, the error is shown.
\nHowever, with \"Save and Run All (Commit)\", the process runs right through the error, and continues to run all the following cells. One of those cells creates a submission file that is of course defective. The commit process is reported as successful. The log shows the error, but execution does not stop.
\n\"Submit Predictions\" then allows me to select that defective version. I assume that the notebook-only process then runs the notebook with the error and scores the defective submission.
\nPlaces I am confused:
\n1) I am a beginner with notebook-only competitions, but I seem to remember that last week a notebook with an error would not commit. Afterwards, you would not be allowed to submit it to the competition. If the notebook passed Commit, but hit an error during submission, you would get \"Submission Scoring Error\". What is going on here? Is my memory at fault, or has something changed?
2) The original error was a \"File not found\" bug. I have since put in \"1/0\" for testing. Executing the cell manually gives an error, but Commit blows right past the error.
\n3) Why this matters: I have a notebook that commits successfully on test data, but fails when submitted. (It's probably a GPU memory error.) If the submission process ignores errors and scores a defective submission, I cannot tell whether my code errors out or my method is wrong.
\nCan anyone shed some light on this issue?
\nCommit log is attached.
", "messages": [ { "id": "1007108", "postDate": "09/11/2020 20:15:53", "content": "I have a notebook that definitely throws an error. If I run it cell by cell, the error is shown.
\nHowever, with \"Save and Run All (Commit)\", the process runs right through the error, and continues to run all the following cells. One of those cells creates a submission file that is of course defective. The commit process is reported as successful. The log shows the error, but execution does not stop.
\n\"Submit Predictions\" then allows me to select that defective version. I assume that the notebook-only process then runs the notebook with the error and scores the defective submission.
\nPlaces I am confused:
\n1) I am a beginner with notebook-only competitions, but I seem to remember that last week a notebook with an error would not commit. Afterwards, you would not be allowed to submit it to the competition. If the notebook passed Commit, but hit an error during submission, you would get \"Submission Scoring Error\". What is going on here? Is my memory at fault, or has something changed?
2) The original error was a \"File not found\" bug. I have since put in \"1/0\" for testing. Executing the cell manually gives an error, but Commit blows right past the error.
\n3) Why this matters: I have a notebook that commits successfully on test data, but fails when submitted. (It's probably a GPU memory error.) If the submission process ignores errors and scores a defective submission, I cannot tell whether my code errors out or my method is wrong.
\nCan anyone shed some light on this issue?
\nCommit log is attached.
", "rawMarkdown": "I have a notebook that definitely throws an error. If I run it cell by cell, the error is shown.\n\nHowever, with \"Save and Run All (Commit)\", the process runs right through the error, and continues to run all the following cells. One of those cells creates a submission file that is of course defective. The commit process is reported as successful. The log shows the error, but execution does not stop.\n\n\"Submit Predictions\" then allows me to select that defective version. I assume that the notebook-only process then runs the notebook with the error and scores the defective submission.\n\nPlaces I am confused:\n1) I am a beginner with notebook-only competitions, but I seem to remember that last week a notebook with an error would not commit. Afterwards, you would not be allowed to submit it to the competition. If the notebook passed Commit, but hit an error during submission, you would get \"Submission Scoring Error\". What is going on here? Is my memory at fault, or has something changed?\n\n2) The original error was a \"File not found\" bug. I have since put in \"1/0\" for testing. Executing the cell manually gives an error, but Commit blows right past the error.\n\n3) Why this matters: I have a notebook that commits successfully on test data, but fails when submitted. (It's probably a GPU memory error.) If the submission process ignores errors and scores a defective submission, I cannot tell whether my code errors out or my method is wrong.\n\nCan anyone shed some light on this issue?\n\nCommit log is attached.", "votes": null }, { "id": "1007191", "postDate": "09/11/2020 22:48:10", "content": "Indeed, execution continues even if cell has an error in Kaggle notebooks.
", "rawMarkdown": "Indeed, execution continues even if cell has an error in Kaggle notebooks.", "votes": null }, { "id": "1007258", "postDate": "09/12/2020 02:01:44", "content": "Thanks so much for replying.
\nOne more question, because you seem to understand the situation well. Is the docker environment used for the submission run the same as the one set for the notebook? I am asking because I switched from GPU to None for the accelerator (in editor), and numpy started throwing errors that did not happen in the GPU environment.
", "rawMarkdown": "Thanks so much for replying.\n\nOne more question, because you seem to understand the situation well. Is the docker environment used for the submission run the same as the one set for the notebook? I am asking because I switched from GPU to None for the accelerator (in editor), and numpy started throwing errors that did not happen in the GPU environment.", "votes": null }, { "id": "1008210", "postDate": "09/12/2020 20:29:30", "content": "Yes, docker is the same if you don't change settings between when you ran your interactive session and the commit.
", "rawMarkdown": "Yes, docker is the same if you don't change settings between when you ran your interactive session and the commit.", "votes": null }, { "id": "1008216", "postDate": "09/12/2020 20:41:01", "content": "This happens with submissions too. Error or not, 1 of our submissions has been running for more than 3 hours now.
", "rawMarkdown": "This happens with submissions too. Error or not, 1 of our submissions has been running for more than 3 hours now.", "votes": null } ], "comments": [ { "id": 1007191, "author_name": "cpmpml", "author_url": "", "post_date": "09/11/2020 22:48:10", "content": "Indeed, execution continues even if cell has an error in Kaggle notebooks.
", "votes": null, "replies": [] }, { "id": 1007258, "author_name": "pomo777", "author_url": "", "post_date": "09/12/2020 02:01:44", "content": "Thanks so much for replying.
\nOne more question, because you seem to understand the situation well. Is the docker environment used for the submission run the same as the one set for the notebook? I am asking because I switched from GPU to None for the accelerator (in editor), and numpy started throwing errors that did not happen in the GPU environment.
", "votes": null, "replies": [ { "id": 1008210, "author_name": "cpmpml", "author_url": "", "post_date": "09/12/2020 20:29:30", "content": "Yes, docker is the same if you don't change settings between when you ran your interactive session and the commit.
", "votes": null, "replies": [] } ] }, { "id": 1008216, "author_name": "mightyrains", "author_url": "", "post_date": "09/12/2020 20:41:01", "content": "This happens with submissions too. Error or not, 1 of our submissions has been running for more than 3 hours now.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1007108": "I have a notebook that definitely throws an error. If I run it cell by cell, the error is shown.\n\nHowever, with \"Save and Run All (Commit)\", the process runs right through the error, and continues to run all the following cells. One of those cells creates a submission file that is of course defective. The commit process is reported as successful. The log shows the error, but execution does not stop.\n\n\"Submit Predictions\" then allows me to select that defective version. I assume that the notebook-only process then runs the notebook with the error and scores the defective submission.\n\nPlaces I am confused:\n1) I am a beginner with notebook-only competitions, but I seem to remember that last week a notebook with an error would not commit. Afterwards, you would not be allowed to submit it to the competition. If the notebook passed Commit, but hit an error during submission, you would get \"Submission Scoring Error\". What is going on here? Is my memory at fault, or has something changed?\n\n2) The original error was a \"File not found\" bug. I have since put in \"1/0\" for testing. Executing the cell manually gives an error, but Commit blows right past the error.\n\n3) Why this matters: I have a notebook that commits successfully on test data, but fails when submitted. (It's probably a GPU memory error.) If the submission process ignores errors and scores a defective submission, I cannot tell whether my code errors out or my method is wrong.\n\nCan anyone shed some light on this issue?\n\nCommit log is attached.", "1007191": "Indeed, execution continues even if cell has an error in Kaggle notebooks.", "1007258": "Thanks so much for replying.\n\nOne more question, because you seem to understand the situation well. Is the docker environment used for the submission run the same as the one set for the notebook? I am asking because I switched from GPU to None for the accelerator (in editor), and numpy started throwing errors that did not happen in the GPU environment.", "1008210": "Yes, docker is the same if you don't change settings between when you ran your interactive session and the commit.", "1008216": "This happens with submissions too. Error or not, 1 of our submissions has been running for more than 3 hours now." }, "source": "meta" }