\n
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F20802231%2Fb506973ba1c37713350fb3c66ccb6b2d%2FSnipaste_2024-07-17_10-57-06.png?generation=1721185127828848&alt=media\")
hye team,I'm facing a problem and have tried many solutions, but nothing is working. I really need help. someone expert help me to fix these :https://www.kaggle.com/code/gtgnhqq/lumbar-competition!
\n
hye team,I'm facing a problem and have tried many solutions, but nothing is working. I really need help. someone expert help me to fix these :https://www.kaggle.com/code/gtgnhqq/lumbar-competition!
\n
@gtgnhqq After checking all your codes are running properly, turn off the internet (Session options >> Internet \"Turn off\").
\nTry to save your note book as a public file (share >> Public)
\nAnd also Install the required Libraries
!pip install pandas==2.2.2
\n!pip install scikit-learn==1.2.2
\n!pip install numpy==1.26.4
For more, check out my notebook, Easy Essay Transformer LAL
", "rawMarkdown": "gtgnhqq After checking all your codes are running properly, turn off the internet (Session options >> Internet \"Turn off\").\nTry to save your note book as a public file (share >> Public) \nAnd also Install the required Libraries \n# Install the required libraries\n!pip install pandas==2.2.2\n!pip install scikit-learn==1.2.2\n!pip install numpy==1.26.4\n\nFor more, check out my notebook, [Easy Essay Transformer LAL](https://www.kaggle.com/code/raja6242/easy-essay-transformer-lal)", "votes": null }, { "id": "2925467", "postDate": "07/17/2024 07:40:01", "content": "I still haven’t resolved it, and I’ve run out of attempts for today.😭
", "rawMarkdown": "I still haven’t resolved it, and I’ve run out of attempts for today.😭", "votes": null }, { "id": "2925536", "postDate": "07/17/2024 08:33:59", "content": "In your code above,
\nT1 = {\n :[, , ,\n , ],\n :[, , ,\n , , ,\n , , ,\n ],\n : [, , ,\n , , ,\n , , ,\n ],\n}\nThis mapping is a correct assumption. But the error occurs, I think, when there is missing Sagittal T2/STIR and Sagittal T1 series (There are training study cases with is issue). This leads to missing row_id values in final submission.
My suggestion:
\n0.3333333333333333 if you want to.combine_first method to fill missing row_id values.ss_df = pd.read_csv()\n submission.row_id.isin(ss_df.row_id).() \nsubmission = submission.set_index().combine_first(ss_df.set_index()).reset_index()\nsubmission.to_csv(, index=)\nI still haven't succeeded, and this is my tenth attempt at submission.😭
", "rawMarkdown": "I still haven't succeeded, and this is my tenth attempt at submission.😭", "votes": null }, { "id": "2926909", "postDate": "07/18/2024 06:01:55", "content": "I want to confirm that Version 8 is still getting \"Submission Scoring Error\" and not \"Notebook Threw Exception\".
\nIf so, assert the following before submission.to_csv('submission.csv', index=False):
submissionI still haven't succeeded, despite all my attempts. It's so difficult. I've been revising for five days, and I don't understand why I can only submit twice a day. Even failed submissions count towards the submission limit. I'm on the verge of breaking down!!!😭
", "rawMarkdown": "I still haven't succeeded, despite all my attempts. It's so difficult. I've been revising for five days, and I don't understand why I can only submit twice a day. Even failed submissions count towards the submission limit. I'm on the verge of breaking down!!!😭", "votes": null }, { "id": "2927123", "postDate": "07/18/2024 09:25:18", "content": "![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F20802231%2F4a55df458f50c7fc27e52a44cd23b1bf%2FQQ20240718172125.png?generation=1721294547337732&alt=media\")
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F20802231%2F22fb6e4fdb5a763aa7a0a4a1549e6fb4%2FQQ20240718172141.png?generation=1721294556443629&alt=media\")
\nIt still shows a submission scoring error.
Are you ensuring each row still sums to 1 after changing float format etc?
", "rawMarkdown": "Are you ensuring each row still sums to 1 after changing float format etc?", "votes": null }, { "id": "2927168", "postDate": "07/18/2024 09:58:34", "content": "submission[['normal_mild', 'moderate', 'severe']] = submission[['normal_mild', 'moderate', 'severe']].div(submission[['normal_mild', 'moderate', 'severe']].sum(axis=1), axis=0)
\nIs it okay if I write it like this?
Just to test if this is the issue, you could set the 3rd column to just be (1-column1-column2) to see if it works. I've had weird problems in that past with rounding, not sure if it's the issue here though
", "rawMarkdown": "Just to test if this is the issue, you could set the 3rd column to just be (1-column1-column2) to see if it works. I've had weird problems in that past with rounding, not sure if it's the issue here though", "votes": null }, { "id": "2927402", "postDate": "07/18/2024 12:56:41", "content": "I still can't get it right; I'm about to collapse.
", "rawMarkdown": "I still can't get it right; I'm about to collapse.", "votes": null }, { "id": "2928625", "postDate": "07/19/2024 12:38:28", "content": "@gtgnhqq I was able to find the core bug in your code:
\nrow_ids = submission[].copy()\nsubmission_transformed = submission.groupby().transform()\nsubmission_transformed[] = row_ids\nsubmission = submission_transformed[[, , , ]]\nIt should be submission = submission.groupby('row_id').mean().reset_index()
If you see transform documentation:
\n\nReturns a DataFrame having the same indexes as the original object filled with the transformed values.
\n
This means that the duplicate row_id values are still present and Version 2 of my modified copy of your original notebook caught that error by adding validation to the merge method: submission = ordered_row_ids.merge(submission, how=\"left\", on=\"row_id\", validate=\"1:1\")
FAKE_TEST as True or False as and len(ss_df) <= 25 will ensure hidden test case will be evaluated during submission.## DEBUG above them, when copying the code back to your notebook with trained weights.Thank you so much, I finally succeeded! After so many days, I'm really grateful.
", "rawMarkdown": "Thank you so much, I finally succeeded! After so many days, I'm really grateful.", "votes": null } ], "comments": [ { "id": 2925317, "author_name": "raja6242", "author_url": "", "post_date": "07/17/2024 05:20:42", "content": "@gtgnhqq After checking all your codes are running properly, turn off the internet (Session options >> Internet \"Turn off\").
\nTry to save your note book as a public file (share >> Public)
\nAnd also Install the required Libraries
!pip install pandas==2.2.2
\n!pip install scikit-learn==1.2.2
\n!pip install numpy==1.26.4
For more, check out my notebook, Easy Essay Transformer LAL
", "votes": null, "replies": [ { "id": 2925467, "author_name": "", "author_url": "", "post_date": "07/17/2024 07:40:01", "content": "I still haven’t resolved it, and I’ve run out of attempts for today.😭
", "votes": null, "replies": [] } ] }, { "id": 2925536, "author_name": "coderrkj", "author_url": "", "post_date": "07/17/2024 08:33:59", "content": "In your code above,
\nT1 = {\n :[, , ,\n , ],\n :[, , ,\n , , ,\n , , ,\n ],\n : [, , ,\n , , ,\n , , ,\n ],\n}\nThis mapping is a correct assumption. But the error occurs, I think, when there is missing Sagittal T2/STIR and Sagittal T1 series (There are training study cases with is issue). This leads to missing row_id values in final submission.
My suggestion:
\n0.3333333333333333 if you want to.combine_first method to fill missing row_id values.ss_df = pd.read_csv()\n submission.row_id.isin(ss_df.row_id).() \nsubmission = submission.set_index().combine_first(ss_df.set_index()).reset_index()\nsubmission.to_csv(, index=)\nI still haven't succeeded, and this is my tenth attempt at submission.😭
", "votes": null, "replies": [ { "id": 2926909, "author_name": "coderrkj", "author_url": "", "post_date": "07/18/2024 06:01:55", "content": "I want to confirm that Version 8 is still getting \"Submission Scoring Error\" and not \"Notebook Threw Exception\".
\nIf so, assert the following before submission.to_csv('submission.csv', index=False):
submissionI still haven't succeeded, despite all my attempts. It's so difficult. I've been revising for five days, and I don't understand why I can only submit twice a day. Even failed submissions count towards the submission limit. I'm on the verge of breaking down!!!😭
", "votes": null, "replies": [ { "id": 2927123, "author_name": "", "author_url": "", "post_date": "07/18/2024 09:25:18", "content": "
\n
\nIt still shows a submission scoring error.
Are you ensuring each row still sums to 1 after changing float format etc?
", "votes": null, "replies": [ { "id": 2927168, "author_name": "", "author_url": "", "post_date": "07/18/2024 09:58:34", "content": "submission[['normal_mild', 'moderate', 'severe']] = submission[['normal_mild', 'moderate', 'severe']].div(submission[['normal_mild', 'moderate', 'severe']].sum(axis=1), axis=0)
\nIs it okay if I write it like this?
Just to test if this is the issue, you could set the 3rd column to just be (1-column1-column2) to see if it works. I've had weird problems in that past with rounding, not sure if it's the issue here though
", "votes": null, "replies": [ { "id": 2927402, "author_name": "", "author_url": "", "post_date": "07/18/2024 12:56:41", "content": "I still can't get it right; I'm about to collapse.
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ] } ] }, { "id": 2928625, "author_name": "coderrkj", "author_url": "", "post_date": "07/19/2024 12:38:28", "content": "@gtgnhqq I was able to find the core bug in your code:
\nrow_ids = submission[].copy()\nsubmission_transformed = submission.groupby().transform()\nsubmission_transformed[] = row_ids\nsubmission = submission_transformed[[, , , ]]\nIt should be submission = submission.groupby('row_id').mean().reset_index()
If you see transform documentation:
\n\nReturns a DataFrame having the same indexes as the original object filled with the transformed values.
\n
This means that the duplicate row_id values are still present and Version 2 of my modified copy of your original notebook caught that error by adding validation to the merge method: submission = ordered_row_ids.merge(submission, how=\"left\", on=\"row_id\", validate=\"1:1\")
FAKE_TEST as True or False as and len(ss_df) <= 25 will ensure hidden test case will be evaluated during submission.## DEBUG above them, when copying the code back to your notebook with trained weights.Thank you so much, I finally succeeded! After so many days, I'm really grateful.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2925229": "hye team,I'm facing a problem and have tried many solutions, but nothing is working. I really need help. someone expert help me to fix these :https://www.kaggle.com/code/gtgnhqq/lumbar-competition!\n", "2925317": "gtgnhqq After checking all your codes are running properly, turn off the internet (Session options >> Internet \"Turn off\").\nTry to save your note book as a public file (share >> Public) \nAnd also Install the required Libraries \n# Install the required libraries\n!pip install pandas==2.2.2\n!pip install scikit-learn==1.2.2\n!pip install numpy==1.26.4\n\nFor more, check out my notebook, [Easy Essay Transformer LAL](https://www.kaggle.com/code/raja6242/easy-essay-transformer-lal)", "2925467": "I still haven’t resolved it, and I’ve run out of attempts for today.😭", "2925536": "In your code above,\n```python\nT1 = {\n 'Sagittal T2/STIR':['spinal_canal_stenosis_l1_l2', 'spinal_canal_stenosis_l2_l3', 'spinal_canal_stenosis_l3_l4',\n 'spinal_canal_stenosis_l4_l5', 'spinal_canal_stenosis_l5_s1'],\n 'Axial T2':['left_subarticular_stenosis_l1_l2', 'left_subarticular_stenosis_l2_l3', 'left_subarticular_stenosis_l3_l4',\n 'left_subarticular_stenosis_l4_l5', 'left_subarticular_stenosis_l5_s1', 'right_subarticular_stenosis_l1_l2',\n 'right_subarticular_stenosis_l2_l3', 'right_subarticular_stenosis_l3_l4', 'right_subarticular_stenosis_l4_l5',\n 'right_subarticular_stenosis_l5_s1'],\n 'Sagittal T1' : ['left_neural_foraminal_narrowing_l1_l2', 'left_neural_foraminal_narrowing_l2_l3', 'left_neural_foraminal_narrowing_l3_l4',\n 'left_neural_foraminal_narrowing_l4_l5', 'left_neural_foraminal_narrowing_l5_s1', 'right_neural_foraminal_narrowing_l1_l2',\n 'right_neural_foraminal_narrowing_l2_l3', 'right_neural_foraminal_narrowing_l3_l4', 'right_neural_foraminal_narrowing_l4_l5',\n 'right_neural_foraminal_narrowing_l5_s1'],\n}\n```\nThis mapping is a correct assumption. But the error occurs, I think, when there is missing `Sagittal T2/STIR` and `Sagittal T1` series (There are training study cases with is issue). This leads to missing `row_id` values in final submission.\n\nMy suggestion:\n- Read sample_submission.csv. Change the default value of `0.3333333333333333` if you want to.\n- Use `combine_first` method to fill missing `row_id` values.\n- As an example, in the last cell: \n```python\nss_df = pd.read_csv(\"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/sample_submission.csv\")\nassert submission.row_id.isin(ss_df.row_id).all() # Check if all `row_id` values are valid (Optional)\nsubmission = submission.set_index(\"row_id\").combine_first(ss_df.set_index(\"row_id\")).reset_index()\nsubmission.to_csv('submission.csv', index=False)\n```", "2926785": "I still haven't succeeded, and this is my tenth attempt at submission.😭", "2926909": "I want to confirm that Version 8 is still getting \"Submission Scoring Error\" and not \"Notebook Threw Exception\".\n\nIf so, assert the following before `submission.to_csv('submission.csv', index=False)`:\n- Assert dtype for last 3 columns are of some float type (or do conversion)\n- Assert no NaN values exist in `submission`\n- Round the float values to 6 decimals (to prevent exponential notation [e.g.: 1.6e-8])", "2927119": "I still haven't succeeded, despite all my attempts. It's so difficult. I've been revising for five days, and I don't understand why I can only submit twice a day. Even failed submissions count towards the submission limit. I'm on the verge of breaking down!!!😭", "2927123": "\n\nIt still shows a submission scoring error.", "2927161": "Are you ensuring each row still sums to 1 after changing float format etc?", "2927168": "submission[['normal_mild', 'moderate', 'severe']] = submission[['normal_mild', 'moderate', 'severe']].div(submission[['normal_mild', 'moderate', 'severe']].sum(axis=1), axis=0)\nIs it okay if I write it like this?", "2927174": "Just to test if this is the issue, you could set the 3rd column to just be (1-column1-column2) to see if it works. I've had weird problems in that past with rounding, not sure if it's the issue here though", "2927402": "I still can't get it right; I'm about to collapse.", "2928625": "gtgnhqq I was able to find the core bug in your code:\n```python\nrow_ids = submission['row_id'].copy()\nsubmission_transformed = submission.groupby('row_id').transform('mean')\nsubmission_transformed['row_id'] = row_ids\nsubmission = submission_transformed[['row_id', 'normal_mild', 'moderate', 'severe']]\n```\nIt should be `submission = submission.groupby('row_id').mean().reset_index()`\n\n### Reason: \nIf you see `transform` [documentation](https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.transform.html):\n> Returns a DataFrame having the same indexes as the original object filled with the transformed values.\n\nThis means that the duplicate `row_id` values are still present and [Version 2](https://www.kaggle.com/code/coderrkj/lumbar-competition-debugging-notebook?scriptVersionId=188924486) of my modified copy of your original notebook caught that error by adding validation to the `merge` method: `submission = ordered_row_ids.merge(submission, how=\"left\", on=\"row_id\", validate=\"1:1\")`\n\n### Additional points:\n- [Version 3](https://www.kaggle.com/code/coderrkj/lumbar-competition-debugging-notebook?scriptVersionId=188947891) shows how to predict over train images and get score from competition metric.\n- [Version 4](https://www.kaggle.com/code/coderrkj/lumbar-competition-debugging-notebook?scriptVersionId=188948027), I have disabled this evaluation and submitted it for testing with hidden data (LB: 1.02 with random weights).\n- You can submit with `FAKE_TEST` as True or False as `and len(ss_df) <= 25` will ensure hidden test case will be evaluated during submission.\n- Remove comments on code with `## DEBUG` above them, when copying the code back to your notebook with trained weights.", "2929637": "Thank you so much, I finally succeeded! After so many days, I'm really grateful." }, "source": "meta" }