![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F856355%2F0f158495a8ffe4f4d2368a52d3406694%2F006.png?generation=1719372819072726&alt=media\")
please take a look: study_id: 3637444890 series_id: 3892989905
\n
please take a look: study_id: 3637444890 series_id: 3892989905
\n
Yes, I can confirm that this is a Sagittal slice of the upper body. The cerebellum can be seen at the top and there are no lumbar vertebrae in the bottom part of the image. Thanks for pointing this case out.
\nFun fact: study_id 3637444890 is the only one where Sagittal T1 is used to label Spinal Canal Stenosis. All other use Sagittal T2/STIR. I guess this is the reason.
", "rawMarkdown": "Yes, I can confirm that this is a Sagittal slice of the upper body. The cerebellum can be seen at the top and there are no lumbar vertebrae in the bottom part of the image. Thanks for pointing this case out.\n\nFun fact: study_id 3637444890 is the only one where Sagittal T1 is used to label Spinal Canal Stenosis. All other use Sagittal T2/STIR. I guess this is the reason.", "votes": null }, { "id": "2890883", "postDate": "06/26/2024 11:09:20", "content": "Yes, it is the cervical spine image, I was about to post it few days ago. Just drop it. I have not found any other incosistencies of this nature anywhere else.
\nThe algorithm your teammate has proposed is actually adds it to the input channels during the dataloading. I have not noticed a big difference in the score though with/or without it.
", "rawMarkdown": "Yes, it is the cervical spine image, I was about to post it few days ago. Just drop it. I have not found any other incosistencies of this nature anywhere else.\n\nThe algorithm your teammate has proposed is actually adds it to the input channels during the dataloading. I have not noticed a big difference in the score though with/or without it.", "votes": null }, { "id": "2891500", "postDate": "06/26/2024 16:57:10", "content": "Thanks for sharing @lihaoweicvch. How did you identify this?
", "rawMarkdown": "Thanks for sharing @lihaoweicvch. How did you identify this?", "votes": null }, { "id": "2891625", "postDate": "06/26/2024 18:23:50", "content": "Idk how the author identified that and you did not ask me, though if you let me it might help answer other people questions. If you go by simple merge test_series_descriptions.csv and train_label_coordinates.csv and sort by missing coordinates - there will be some hits. I also went thru all sagittal examples, I did not find anything else like this.
mrgd = train_desc.merge(train_coord, on=[, ], how=)\n(mrgd[mrgd.x.isna()].to_string())\n| \n | study_id | \nseries_id | \nseries_description | \ninstance_number | \ncondition | \nlevel | \nx | \ny | \n
|---|---|---|---|---|---|---|---|---|
| 33978 | \n3008676218 | \n542282425 | \nSagittal T1 | \nNaN | \nNaN | \nNaN | \nNaN | \nNaN | \n
| 33979 | \n3008676218 | \n3636216534 | \nAxial T2 | \nNaN | \nNaN | \nNaN | \nNaN | \nNaN | \n
| 41256 | \n3637444890 | \n3892989905 | \nSagittal T2/STIR | \nNaN | \nNaN | \nNaN | \nNaN | \nNaN | \n
My fault, @sergiosaharovskiy. I should have asked you both!
", "rawMarkdown": "My fault, @sergiosaharovskiy. I should have asked you both!", "votes": null }, { "id": "2896221", "postDate": "06/29/2024 17:10:35", "content": "Thank you for this valuable insight.
\nI have noticed that total_train_label_cords_desc['instance_number'][41256] produces 5 and total_train_label_cords_desc['condition'][41256] produces 'Right Neural Foraminal Narrowing' and not nan. I am a bit confused by this output has the dataset been modified?
Your responses will be highly valuable.
", "rawMarkdown": "Thank you for this valuable insight. \nI have noticed that total_train_label_cords_desc['instance_number'][41256] produces 5 and total_train_label_cords_desc['condition'][41256] produces 'Right Neural Foraminal Narrowing' and not nan. I am a bit confused by this output has the dataset been modified?\n\nYour responses will be highly valuable.", "votes": null }, { "id": "2896316", "postDate": "06/29/2024 18:41:34", "content": "@devsya, it would be great if you could show a snippet on how you got total_train_label_cords_desc.
\nI reran the code Sergey gave in a Kaggle notebook cell:
pathlib Path\n pandas pd\n\nINPUT_DIR = Path()\ndf_train_label = pd.read_csv(INPUT_DIR / )\ndf_train_desc = pd.read_csv(INPUT_DIR / )\n\nmrgd = df_train_desc.merge(df_train_label, on=[, ], how=)\nmrgd[mrgd.x.isna()]\nand the result is the same as above. To be specific to your question, mrgd.loc[41256] gives:
study_id 3637444890\nseries_id 3892989905\nseries_description Sagittal T2/STIR\ninstance_number NaN\ncondition NaN\nlevel NaN\nx NaN\ny NaN\nName: 41256, dtype: object\n![\"See](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14118568%2Fae6104f614419dae16aa9efd654664e5%2Fscreenshot_train_label_cords.png?generation=1719727922004953&alt=media\")
@coderrkj In the image you would see that 41256 points to a different series ID. So I tried this : train_label_cords[(train_label_cords['study_id']=='3637444890') & (train_label_cords['series_id']=='3892989905')]
\nAnd it returned zero rows. I think that the dataset has been updated. You should also check it by clicking on the three dots of the main folder of the competition dataset and then choosing the check for updates option.
The confusion that you are having is that index number 41256 in df_train_label and df_train_desc.merge(df_train_label, on=['study_id', 'series_id'], how='left') refer to 2 separate things.
\nFirst on is just the 41256th row in train_label_coordinates.csv the other is the 41256th row in left SQL join of train_series_descriptions.csv and train_label_coordinates.csv on the columns study_id and series_id.
\nIf you do print(len(mrgd), len(df_train_label)) from my code cell that I gave above, you get 48695 48692. There are 3 series missing in train_label_coordinates.csv. Those 3 that Sergio Saharovskiy posted are these missing series.
\n\nSo I tried this : train_label_cords[(train_label_cords['study_id']=='3637444890') & (train_label_cords['series_id']=='3892989905')] And it returned zero rows.
\n
Yes, that is what the merge is also depicting, that study_id=3637444890 series_id=3892989905 is missing in train_label_coordinates.csv but present in train_series_descriptions.csv.
Edit: Also adding for this:
\n\n\nYou should also check it by clicking on the three dots of the main folder of the competition dataset and then choosing the check for updates option.
\n
![\"No](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1303569%2F5ad26213148efa9839490b85a2007013%2FScreenshot%202024-06-30%20131955.png?generation=1719733890297553&alt=media\")
Thanks a lot for this clarification, although I have not yet checked it myself.
", "rawMarkdown": "Thanks a lot for this clarification, although I have not yet checked it myself.", "votes": null } ], "comments": [ { "id": 2890326, "author_name": "coderrkj", "author_url": "", "post_date": "06/26/2024 04:29:33", "content": "Yes, I can confirm that this is a Sagittal slice of the upper body. The cerebellum can be seen at the top and there are no lumbar vertebrae in the bottom part of the image. Thanks for pointing this case out.
\nFun fact: study_id 3637444890 is the only one where Sagittal T1 is used to label Spinal Canal Stenosis. All other use Sagittal T2/STIR. I guess this is the reason.
", "votes": null, "replies": [] }, { "id": 2890883, "author_name": "sergiosaharovskiy", "author_url": "", "post_date": "06/26/2024 11:09:20", "content": "Yes, it is the cervical spine image, I was about to post it few days ago. Just drop it. I have not found any other incosistencies of this nature anywhere else.
\nThe algorithm your teammate has proposed is actually adds it to the input channels during the dataloading. I have not noticed a big difference in the score though with/or without it.
", "votes": null, "replies": [] }, { "id": 2891500, "author_name": "brendanartley", "author_url": "", "post_date": "06/26/2024 16:57:10", "content": "Thanks for sharing @lihaoweicvch. How did you identify this?
", "votes": null, "replies": [ { "id": 2891625, "author_name": "sergiosaharovskiy", "author_url": "", "post_date": "06/26/2024 18:23:50", "content": "Idk how the author identified that and you did not ask me, though if you let me it might help answer other people questions. If you go by simple merge test_series_descriptions.csv and train_label_coordinates.csv and sort by missing coordinates - there will be some hits. I also went thru all sagittal examples, I did not find anything else like this.
mrgd = train_desc.merge(train_coord, on=[, ], how=)\n(mrgd[mrgd.x.isna()].to_string())\n| \n | study_id | \nseries_id | \nseries_description | \ninstance_number | \ncondition | \nlevel | \nx | \ny | \n
|---|---|---|---|---|---|---|---|---|
| 33978 | \n3008676218 | \n542282425 | \nSagittal T1 | \nNaN | \nNaN | \nNaN | \nNaN | \nNaN | \n
| 33979 | \n3008676218 | \n3636216534 | \nAxial T2 | \nNaN | \nNaN | \nNaN | \nNaN | \nNaN | \n
| 41256 | \n3637444890 | \n3892989905 | \nSagittal T2/STIR | \nNaN | \nNaN | \nNaN | \nNaN | \nNaN | \n
My fault, @sergiosaharovskiy. I should have asked you both!
", "votes": null, "replies": [] }, { "id": 2896221, "author_name": "devsya", "author_url": "", "post_date": "06/29/2024 17:10:35", "content": "Thank you for this valuable insight.
\nI have noticed that total_train_label_cords_desc['instance_number'][41256] produces 5 and total_train_label_cords_desc['condition'][41256] produces 'Right Neural Foraminal Narrowing' and not nan. I am a bit confused by this output has the dataset been modified?
Your responses will be highly valuable.
", "votes": null, "replies": [ { "id": 2896316, "author_name": "coderrkj", "author_url": "", "post_date": "06/29/2024 18:41:34", "content": "@devsya, it would be great if you could show a snippet on how you got total_train_label_cords_desc.
\nI reran the code Sergey gave in a Kaggle notebook cell:
pathlib Path\n pandas pd\n\nINPUT_DIR = Path()\ndf_train_label = pd.read_csv(INPUT_DIR / )\ndf_train_desc = pd.read_csv(INPUT_DIR / )\n\nmrgd = df_train_desc.merge(df_train_label, on=[, ], how=)\nmrgd[mrgd.x.isna()]\nand the result is the same as above. To be specific to your question, mrgd.loc[41256] gives:
study_id 3637444890\nseries_id 3892989905\nseries_description Sagittal T2/STIR\ninstance_number NaN\ncondition NaN\nlevel NaN\nx NaN\ny NaN\nName: 41256, dtype: object\n
@coderrkj In the image you would see that 41256 points to a different series ID. So I tried this : train_label_cords[(train_label_cords['study_id']=='3637444890') & (train_label_cords['series_id']=='3892989905')]
\nAnd it returned zero rows. I think that the dataset has been updated. You should also check it by clicking on the three dots of the main folder of the competition dataset and then choosing the check for updates option.
The confusion that you are having is that index number 41256 in df_train_label and df_train_desc.merge(df_train_label, on=['study_id', 'series_id'], how='left') refer to 2 separate things.
\nFirst on is just the 41256th row in train_label_coordinates.csv the other is the 41256th row in left SQL join of train_series_descriptions.csv and train_label_coordinates.csv on the columns study_id and series_id.
\nIf you do print(len(mrgd), len(df_train_label)) from my code cell that I gave above, you get 48695 48692. There are 3 series missing in train_label_coordinates.csv. Those 3 that Sergio Saharovskiy posted are these missing series.
\n\nSo I tried this : train_label_cords[(train_label_cords['study_id']=='3637444890') & (train_label_cords['series_id']=='3892989905')] And it returned zero rows.
\n
Yes, that is what the merge is also depicting, that study_id=3637444890 series_id=3892989905 is missing in train_label_coordinates.csv but present in train_series_descriptions.csv.
Edit: Also adding for this:
\n\n\nYou should also check it by clicking on the three dots of the main folder of the competition dataset and then choosing the check for updates option.
\n
Thanks a lot for this clarification, although I have not yet checked it myself.
", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2890264": "please take a look: study_id: 3637444890 series_id: 3892989905\n\n", "2890326": "Yes, I can confirm that this is a Sagittal slice of the upper body. The cerebellum can be seen at the top and there are no lumbar vertebrae in the bottom part of the image. Thanks for pointing this case out.\n\nFun fact: study_id 3637444890 is the only one where Sagittal T1 is used to label Spinal Canal Stenosis. All other use Sagittal T2/STIR. I guess this is the reason.", "2890883": "Yes, it is the cervical spine image, I was about to post it few days ago. Just drop it. I have not found any other incosistencies of this nature anywhere else.\n\nThe algorithm your teammate has proposed is actually adds it to the input channels during the dataloading. I have not noticed a big difference in the score though with/or without it.", "2891500": "Thanks for sharing @lihaoweicvch. How did you identify this?", "2891625": "Idk how the author identified that and you did not ask me, though if you let me it might help answer other people questions. If you go by simple merge `test_series_descriptions.csv` and `train_label_coordinates.csv` and sort by missing coordinates - there will be some hits. I also went thru all sagittal examples, I did not find anything else like this.\n```python\nmrgd = train_desc.merge(train_coord, on=['study_id', 'series_id'], how='left')\nprint(mrgd[mrgd.x.isna()].to_string())\n```\n\n| | study_id | series_id | series_description | instance_number | condition | level | x | y |\n|----|------------|-------------|--------------------|-----------------|-----------|-------|-----|-----|\n| 33978 | 3008676218 | 542282425 | Sagittal T1 | NaN | NaN | NaN | NaN | NaN |\n| 33979 | 3008676218 | 3636216534 | Axial T2 | NaN | NaN | NaN | NaN | NaN |\n| 41256 | 3637444890 | 3892989905 | Sagittal T2/STIR | NaN | NaN | NaN | NaN | NaN |", "2891730": "My fault, @sergiosaharovskiy. I should have asked you both!", "2896221": "Thank you for this valuable insight. \nI have noticed that total_train_label_cords_desc['instance_number'][41256] produces 5 and total_train_label_cords_desc['condition'][41256] produces 'Right Neural Foraminal Narrowing' and not nan. I am a bit confused by this output has the dataset been modified?\n\nYour responses will be highly valuable.", "2896316": "devsya, it would be great if you could show a snippet on how you got `total_train_label_cords_desc`. \nI reran the code Sergey gave in a Kaggle notebook cell:\n```python\nfrom pathlib import Path\nimport pandas as pd\n\nINPUT_DIR = Path(\"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification\")\ndf_train_label = pd.read_csv(INPUT_DIR / 'train_label_coordinates.csv')\ndf_train_desc = pd.read_csv(INPUT_DIR / 'train_series_descriptions.csv')\n\nmrgd = df_train_desc.merge(df_train_label, on=['study_id', 'series_id'], how='left')\nmrgd[mrgd.x.isna()]\n```\nand the result is the same as above. To be specific to your question, `mrgd.loc[41256]` gives:\n```text\nstudy_id 3637444890\nseries_id 3892989905\nseries_description Sagittal T2/STIR\ninstance_number NaN\ncondition NaN\nlevel NaN\nx NaN\ny NaN\nName: 41256, dtype: object\n```", "2896799": "", "2896806": "coderrkj In the image you would see that 41256 points to a different series ID. So I tried this : train_label_cords[(train_label_cords['study_id']=='3637444890') & (train_label_cords['series_id']=='3892989905')]\nAnd it returned zero rows. I think that the dataset has been updated. You should also check it by clicking on the three dots of the main folder of the competition dataset and then choosing the check for updates option.", "2896964": "The confusion that you are having is that index number `41256` in `df_train_label` and `df_train_desc.merge(df_train_label, on=['study_id', 'series_id'], how='left')` refer to 2 separate things.\nFirst on is just the 41256th row in `train_label_coordinates.csv` the other is the 41256th row in **left SQL join** of `train_series_descriptions.csv` and `train_label_coordinates.csv` on the columns `study_id` and `series_id`.\nIf you do `print(len(mrgd), len(df_train_label))` from my code cell that I gave above, you get `48695 48692`. There are 3 series missing in `train_label_coordinates.csv`. Those 3 that Sergio Saharovskiy posted are these missing series.\n\n> So I tried this : train_label_cords[(train_label_cords['study_id']=='3637444890') & (train_label_cords['series_id']=='3892989905')] And it returned zero rows.\n\nYes, that is what the merge is also depicting, that study_id=3637444890 series_id=3892989905 is missing in `train_label_coordinates.csv` but present in `train_series_descriptions.csv`.\n\nEdit: Also adding for this:\n>You should also check it by clicking on the three dots of the main folder of the competition dataset and then choosing the check for updates option.\n\n", "2897087": "Thanks a lot for this clarification, although I have not yet checked it myself." }, "source": "meta" }