Hey all. I noticed that my most recent submission is throwing an error - \"Submission Scoring Error.\" Does anyone know the cause of this, or how to correct it?
", "messages": [ { "id": 2178662, "postDate": "2023-03-12T15:43:13.307Z", "content": "Hey all. I noticed that my most recent submission is throwing an error - \"Submission Scoring Error.\" Does anyone know the cause of this, or how to correct it?
", "rawMarkdown": "Hey all. I noticed that my most recent submission is throwing an error - \"Submission Scoring Error.\" Does anyone know the cause of this, or how to correct it?", "votes": 8 }, { "id": 2183642, "postDate": "2023-03-15T19:46:04.843Z", "content": "I've updated to metric to fix this issue. Updates here: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/395073
", "rawMarkdown": "I've updated to metric to fix this issue. Updates here: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/395073", "votes": 1, "replies": [ { "id": 2188219, "postDate": "2023-03-19T12:11:20.857Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 2178675, "postDate": "2023-03-12T16:00:52.177Z", "content": "I also the same error
\n\"Submission Scoring Error\"
\n\n\nYour notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips
\n
~~~~
\nSoved：by np.round(pred,3)
I had this issue before I rounded the predictions to a few significant digits. I assumed that the scoring engine has an issue with interpreting scientific notation of small numbers - and this is how pandas saves them. I didn't verify that assumption, though.
\nTry np.round(y_pred, 4) and see what happens.
", "rawMarkdown": "I had this issue before I rounded the predictions to a few significant digits. I assumed that the scoring engine has an issue with interpreting scientific notation of small numbers - and this is how pandas saves them. I didn't verify that assumption, though.\n\nTry np.round(y_pred, 4) and see what happens.", "votes": 2, "replies": [ { "id": 2179960, "postDate": "2023-03-13T13:53:06.053Z", "content": "I had similar issue. Rounding to 3 decimals worked for me. Submiting raw probs gives me \"Submission Scoring Error\".
", "rawMarkdown": "I had similar issue. Rounding to 3 decimals worked for me. Submiting raw probs gives me \"Submission Scoring Error\"." }, { "id": 2180546, "postDate": "2023-03-13T22:54:08.707Z", "content": "It also seems there's a hard limit on the output values. I am guessing it's beyond (0,1) but there's a limit. have to verify this.
\n@ryanholbrook any guidelines on this?
", "rawMarkdown": "It also seems there's a hard limit on the output values. I am guessing it's beyond (0,1) but there's a limit. have to verify this.\n\n@ryanholbrook any guidelines on this?", "votes": 1, "replies": [ { "id": 2180915, "postDate": "2023-03-14T07:14:16.347Z", "content": "My experiments show that you can do np.round(Y_pred * 10000, 4) - as this keeps the number precision higher. Such a solution was accepted and scored for me.
Anyway - that's a pity we need to do such things, it should work without rounding.
", "rawMarkdown": "My experiments show that you can do `np.round(Y_pred * 10000, 4)` - as this keeps the number precision higher. Such a solution was accepted and scored for me.\n\nAnyway - that's a pity we need to do such things, it should work without rounding.", "votes": 1 }, { "id": 2181014, "postDate": "2023-03-14T08:47:22.460Z", "content": "Rounding to 3 worked, rounding to 4 didn't, rounding to 5 and multiplying by 1000 didn't. Don't round first 1000 rows and set everything else to 0.5 worked.
\nI think it just doesn't like many unique values
", "rawMarkdown": "Rounding to 3 worked, rounding to 4 didn't, rounding to 5 and multiplying by 1000 didn't. Don't round first 1000 rows and set everything else to 0.5 worked. \n\nI think it just doesn't like many unique values" }, { "id": 2181263, "postDate": "2023-03-14T12:32:42Z", "content": "Thanks for the heads up. I'll investigate. My guess is it could be an OOM error when the metric has to process a large number of unique values. AP (like AUC) works by ranking confidence scores, so there's less to store with tied ranks.
", "rawMarkdown": "Thanks for the heads up. I'll investigate. My guess is it could be an OOM error when the metric has to process a large number of unique values. AP (like AUC) works by ranking confidence scores, so there's less to store with tied ranks.", "votes": 1 } ] } ] }, { "id": 2188177, "postDate": "2023-03-19T11:19:01.437Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2183642, "author_name": "Ryan Holbrook", "author_url": "", "post_date": "2023-03-15T19:46:04.843000", "content": "I've updated to metric to fix this issue. Updates here: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/395073
", "votes": 1, "replies": [ { "id": 2188219, "author_name": "", "author_url": "", "post_date": "2023-03-19T12:11:20.857000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2178675, "author_name": "yanqiangmiffy", "author_url": "", "post_date": "2023-03-12T16:00:52.177000", "content": "I also the same error
\n\"Submission Scoring Error\"
\n\n\nYour notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips
\n
~~~~
\nSoved：by np.round(pred,3)
I had this issue before I rounded the predictions to a few significant digits. I assumed that the scoring engine has an issue with interpreting scientific notation of small numbers - and this is how pandas saves them. I didn't verify that assumption, though.
\nTry np.round(y_pred, 4) and see what happens.
", "votes": 2, "replies": [ { "id": 2179960, "author_name": "Giba", "author_url": "", "post_date": "2023-03-13T13:53:06.053000", "content": "I had similar issue. Rounding to 3 decimals worked for me. Submiting raw probs gives me \"Submission Scoring Error\".
", "votes": 0, "replies": [] }, { "id": 2180546, "author_name": "Mayukh Bhattacharyya", "author_url": "", "post_date": "2023-03-13T22:54:08.707000", "content": "It also seems there's a hard limit on the output values. I am guessing it's beyond (0,1) but there's a limit. have to verify this.
\n@ryanholbrook any guidelines on this?
", "votes": 1, "replies": [ { "id": 2180915, "author_name": "Tomasz Bartczak", "author_url": "", "post_date": "2023-03-14T07:14:16.347000", "content": "My experiments show that you can do np.round(Y_pred * 10000, 4) - as this keeps the number precision higher. Such a solution was accepted and scored for me.
Anyway - that's a pity we need to do such things, it should work without rounding.
", "votes": 1, "replies": [] }, { "id": 2181014, "author_name": "RD", "author_url": "", "post_date": "2023-03-14T08:47:22.460000", "content": "Rounding to 3 worked, rounding to 4 didn't, rounding to 5 and multiplying by 1000 didn't. Don't round first 1000 rows and set everything else to 0.5 worked.
\nI think it just doesn't like many unique values
", "votes": 0, "replies": [] }, { "id": 2181263, "author_name": "Ryan Holbrook", "author_url": "", "post_date": "2023-03-14T12:32:42", "content": "Thanks for the heads up. I'll investigate. My guess is it could be an OOM error when the metric has to process a large number of unique values. AP (like AUC) works by ranking confidence scores, so there's less to store with tied ranks.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2188177, "author_name": "", "author_url": "", "post_date": "2023-03-19T11:19:01.437000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2178662": "Hey all. I noticed that my most recent submission is throwing an error - \"Submission Scoring Error.\" Does anyone know the cause of this, or how to correct it?", "2183642": "I've updated to metric to fix this issue. Updates here: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/395073", "2178675": "I also the same error\n\n\"Submission Scoring Error\"\n\n\n>Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips\n\n~~~~\nSoved：by **np.round(pred,3)**", "2179805": "I had this issue before I rounded the predictions to a few significant digits. I assumed that the scoring engine has an issue with interpreting scientific notation of small numbers - and this is how pandas saves them. I didn't verify that assumption, though.\n\nTry np.round(y_pred, 4) and see what happens.", "2188177": "" } }