{
  "id": 521365,
  "title": "Let's See Null Distribution in Train DataFrame",
  "url": "/competitions/rsna-2024-lumbar-spine-degenerative-classification/discussion/521365",
  "author_name": "",
  "post_date": "2024-07-20T11:07:16.611132200Z",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>There are missing data, This is due to the fact that some of the images do not have those regions visualized</p>\n<p>📌 <code>Unknown</code> : null<br>\n📌 <code>Known</code>: normal_mild, moderate, severe</p>\n<pre><code>def (row):\n    if pd.(row): \n        return \n    else: \n        return \nfor col in df.columns:\n    df[col] = df[col].(lambda x: (x))\n\nspinal_conditions = [, , ]\n\nplt.style.()\nplt.(figsize=(,)) \nfor i, condition in (spinal_conditions):\n    plt.(,,i+)\n    plt.(f, size=)\n\n   column = [col for col in df.columns if condition in col]\n   tmp = df[column].()\n   tmp2 = tmp.()\n\n   sns.(x=tmp2.index, y=tmp2.values)\nplt.()\nplt.()\n</code></pre>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe56d31dc9bd1d5008c4b7d2e7f472575%2Fam.JPG?generation=1721519205778212&amp;alt=media\" alt=\"\"></p>\n<pre><code>def (row):\n      if pd.(row): \n         return \n     else: \n         return \n for col in df.columns:\n     df[col] = df[col].(lambda x: (x))\n\n spinal_level = [,,,,]\n\n plt.style.()\n plt.(figsize=(,)) \n for i, level in (spinal_level):\n     plt.(,,i+)\n     plt.(f, size=)\n\n    column = [col for col in df.columns if condition in col]\n    tmp = df[column].()\n    tmp2 = tmp.()\n\n    sns.(x=tmp2.index, y=tmp2.values)\n plt.()\n plt.()\n</code></pre>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe9eab7e145057245e9793c1a82b93164%2Fl2.JPG?generation=1721519367904413&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": "2929778",
      "postDate": "07/20/2024 11:07:16",
      "content": "<p>There are missing data, This is due to the fact that some of the images do not have those regions visualized</p>\n<p>📌 <code>Unknown</code> : null<br>\n📌 <code>Known</code>: normal_mild, moderate, severe</p>\n<pre><code>def (row):\n    if pd.(row): \n        return \n    else: \n        return \nfor col in df.columns:\n    df[col] = df[col].(lambda x: (x))\n\nspinal_conditions = [, , ]\n\nplt.style.()\nplt.(figsize=(,)) \nfor i, condition in (spinal_conditions):\n    plt.(,,i+)\n    plt.(f, size=)\n\n   column = [col for col in df.columns if condition in col]\n   tmp = df[column].()\n   tmp2 = tmp.()\n\n   sns.(x=tmp2.index, y=tmp2.values)\nplt.()\nplt.()\n</code></pre>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe56d31dc9bd1d5008c4b7d2e7f472575%2Fam.JPG?generation=1721519205778212&amp;alt=media\" alt=\"\"></p>\n<pre><code>def (row):\n      if pd.(row): \n         return \n     else: \n         return \n for col in df.columns:\n     df[col] = df[col].(lambda x: (x))\n\n spinal_level = [,,,,]\n\n plt.style.()\n plt.(figsize=(,)) \n for i, level in (spinal_level):\n     plt.(,,i+)\n     plt.(f, size=)\n\n    column = [col for col in df.columns if condition in col]\n    tmp = df[column].()\n    tmp2 = tmp.()\n\n    sns.(x=tmp2.index, y=tmp2.values)\n plt.()\n plt.()\n</code></pre>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe9eab7e145057245e9793c1a82b93164%2Fl2.JPG?generation=1721519367904413&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "There are missing data, This is due to the fact that some of the images do not have those regions visualized\n\n📌 `Unknown` : null\n📌 `Known`: normal_mild, moderate, severe\n\n    def nan2unknown(row):\n        if pd.isna(row): \n            return 'unknown'\n        else: \n            return 'known'\n    for col in df.columns:\n        df[col] = df[col].apply(lambda x: nan2unknown(x))\n    \n    spinal_conditions = ['canal', 'foraminal', 'subarticular']\n\n    plt.style.use('default')\n    plt.figure(figsize=(12,6)) \n    for i, condition in enumerate(spinal_conditions):\n        plt.subplot(1,3,i+1)\n        plt.title(f'Distribution of Null in {condition}', size=10)\n    \n       column = [col for col in df.columns if condition in col]\n       tmp = df[column].stack()\n       tmp2 = tmp.value_counts()\n    \n       sns.barplot(x=tmp2.index, y=tmp2.values)\n    plt.tight_layout()\n    plt.show()\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe56d31dc9bd1d5008c4b7d2e7f472575%2Fam.JPG?generation=1721519205778212&alt=media)\n\n    def nan2unknown(row):\n          if pd.isna(row): \n             return 'unknown'\n         else: \n             return 'known'\n     for col in df.columns:\n         df[col] = df[col].apply(lambda x: nan2unknown(x))\n    \n     spinal_level = ['l1_l2','l2_l3','l3_l4','l4_l5','l5_s1']\n\n     plt.style.use('default')\n     plt.figure(figsize=(12,6)) \n     for i, level in enumerate(spinal_level):\n         plt.subplot(1,3,i+1)\n         plt.title(f'Distribution of Null in {level}', size=10)\n    \n        column = [col for col in df.columns if condition in col]\n        tmp = df[column].stack()\n        tmp2 = tmp.value_counts()\n    \n        sns.barplot(x=tmp2.index, y=tmp2.values)\n     plt.tight_layout()\n     plt.show()\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe9eab7e145057245e9793c1a82b93164%2Fl2.JPG?generation=1721519367904413&alt=media)",
      "votes": null
    },
    {
      "id": "2971061",
      "postDate": "08/26/2024 18:25:23",
      "content": "<p>import pandas as pd<br>\nimport numpy as np<br>\nimport matplotlib.pyplot as plt<br>\nimport seaborn as sns</p>\n<p>df_train_copy = df_train.copy()<br>\ndef nan2unknown(row):<br>\n    if pd.isna(row):<br>\n        return 'unknown'<br>\n    else:<br>\n        return 'known'<br>\nfor col in df_train.columns:<br>\n    df_train_copy[col] = df_train_copy[col].apply(lambda x: nan2unknown(x))</p>\n<h1>Define a function to convert non-'unknown' entries to 'known'</h1>\n<h1>Define the spinal levels you want to analyze</h1>\n<p>spinal_level = ['l1_l2', 'l2_l3', 'l3_l4', 'l4_l5', 'l5_s1']</p>\n<h1>Plotting setup</h1>\n<p>plt.style.use('default')<br>\nplt.figure(figsize=(15, 6))</p>\n<h1>Create a subplot for each spinal level</h1>\n<p>for i, level in enumerate(spinal_level):<br>\n    plt.subplot(1, 5, i + 1)  # 1 row, 5 columns for each level<br>\n    plt.title(f'Distribution of Null in {level}', size=10)</p>\n<pre><code>\ncolumns_to_plot = [col  col  df_train_copy.columns  level  col]\n\nstacked_data = df_train_copy[columns_to_plot].stack()\nvalue_counts = stacked_data.value_counts()\n\n\nsns.barplot(=value_counts.index, =value_counts.values, palette=[, ])\n</code></pre>\n<p>plt.tight_layout()<br>\nplt.show()</p>\n<p>For your spinal levels</p>",
      "rawMarkdown": "import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n\ndf_train_copy = df_train.copy()\ndef nan2unknown(row):\n    if pd.isna(row):\n        return 'unknown'\n    else:\n        return 'known'\nfor col in df_train.columns:\n    df_train_copy[col] = df_train_copy[col].apply(lambda x: nan2unknown(x))\n\n# Define a function to convert non-'unknown' entries to 'known'\n\n# Define the spinal levels you want to analyze\nspinal_level = ['l1_l2', 'l2_l3', 'l3_l4', 'l4_l5', 'l5_s1']\n\n# Plotting setup\nplt.style.use('default')\nplt.figure(figsize=(15, 6))\n\n# Create a subplot for each spinal level\nfor i, level in enumerate(spinal_level):\n    plt.subplot(1, 5, i + 1)  # 1 row, 5 columns for each level\n    plt.title(f'Distribution of Null in {level}', size=10)\n\n    # Filter columns that correspond to the current spinal level\n    columns_to_plot = [col for col in df_train_copy.columns if level in col]\n    # Stack the data for plotting\n    stacked_data = df_train_copy[columns_to_plot].stack()\n    value_counts = stacked_data.value_counts()\n\n    # Create the bar plot\n    sns.barplot(x=value_counts.index, y=value_counts.values, palette=['#1f77b4', '#ff7f0e'])\n\nplt.tight_layout()\nplt.show()\n\nFor your spinal levels",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2971061,
      "author_name": "amanbawa",
      "author_url": "",
      "post_date": "08/26/2024 18:25:23",
      "content": "<p>import pandas as pd<br>\nimport numpy as np<br>\nimport matplotlib.pyplot as plt<br>\nimport seaborn as sns</p>\n<p>df_train_copy = df_train.copy()<br>\ndef nan2unknown(row):<br>\n    if pd.isna(row):<br>\n        return 'unknown'<br>\n    else:<br>\n        return 'known'<br>\nfor col in df_train.columns:<br>\n    df_train_copy[col] = df_train_copy[col].apply(lambda x: nan2unknown(x))</p>\n<h1>Define a function to convert non-'unknown' entries to 'known'</h1>\n<h1>Define the spinal levels you want to analyze</h1>\n<p>spinal_level = ['l1_l2', 'l2_l3', 'l3_l4', 'l4_l5', 'l5_s1']</p>\n<h1>Plotting setup</h1>\n<p>plt.style.use('default')<br>\nplt.figure(figsize=(15, 6))</p>\n<h1>Create a subplot for each spinal level</h1>\n<p>for i, level in enumerate(spinal_level):<br>\n    plt.subplot(1, 5, i + 1)  # 1 row, 5 columns for each level<br>\n    plt.title(f'Distribution of Null in {level}', size=10)</p>\n<pre><code>\ncolumns_to_plot = [col  col  df_train_copy.columns  level  col]\n\nstacked_data = df_train_copy[columns_to_plot].stack()\nvalue_counts = stacked_data.value_counts()\n\n\nsns.barplot(=value_counts.index, =value_counts.values, palette=[, ])\n</code></pre>\n<p>plt.tight_layout()<br>\nplt.show()</p>\n<p>For your spinal levels</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2929778": "There are missing data, This is due to the fact that some of the images do not have those regions visualized\n\n📌 `Unknown` : null\n📌 `Known`: normal_mild, moderate, severe\n\n    def nan2unknown(row):\n        if pd.isna(row): \n            return 'unknown'\n        else: \n            return 'known'\n    for col in df.columns:\n        df[col] = df[col].apply(lambda x: nan2unknown(x))\n    \n    spinal_conditions = ['canal', 'foraminal', 'subarticular']\n\n    plt.style.use('default')\n    plt.figure(figsize=(12,6)) \n    for i, condition in enumerate(spinal_conditions):\n        plt.subplot(1,3,i+1)\n        plt.title(f'Distribution of Null in {condition}', size=10)\n    \n       column = [col for col in df.columns if condition in col]\n       tmp = df[column].stack()\n       tmp2 = tmp.value_counts()\n    \n       sns.barplot(x=tmp2.index, y=tmp2.values)\n    plt.tight_layout()\n    plt.show()\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe56d31dc9bd1d5008c4b7d2e7f472575%2Fam.JPG?generation=1721519205778212&alt=media)\n\n    def nan2unknown(row):\n          if pd.isna(row): \n             return 'unknown'\n         else: \n             return 'known'\n     for col in df.columns:\n         df[col] = df[col].apply(lambda x: nan2unknown(x))\n    \n     spinal_level = ['l1_l2','l2_l3','l3_l4','l4_l5','l5_s1']\n\n     plt.style.use('default')\n     plt.figure(figsize=(12,6)) \n     for i, level in enumerate(spinal_level):\n         plt.subplot(1,3,i+1)\n         plt.title(f'Distribution of Null in {level}', size=10)\n    \n        column = [col for col in df.columns if condition in col]\n        tmp = df[column].stack()\n        tmp2 = tmp.value_counts()\n    \n        sns.barplot(x=tmp2.index, y=tmp2.values)\n     plt.tight_layout()\n     plt.show()\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16438831%2Fe9eab7e145057245e9793c1a82b93164%2Fl2.JPG?generation=1721519367904413&alt=media)",
    "2971061": "import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n\ndf_train_copy = df_train.copy()\ndef nan2unknown(row):\n    if pd.isna(row):\n        return 'unknown'\n    else:\n        return 'known'\nfor col in df_train.columns:\n    df_train_copy[col] = df_train_copy[col].apply(lambda x: nan2unknown(x))\n\n# Define a function to convert non-'unknown' entries to 'known'\n\n# Define the spinal levels you want to analyze\nspinal_level = ['l1_l2', 'l2_l3', 'l3_l4', 'l4_l5', 'l5_s1']\n\n# Plotting setup\nplt.style.use('default')\nplt.figure(figsize=(15, 6))\n\n# Create a subplot for each spinal level\nfor i, level in enumerate(spinal_level):\n    plt.subplot(1, 5, i + 1)  # 1 row, 5 columns for each level\n    plt.title(f'Distribution of Null in {level}', size=10)\n\n    # Filter columns that correspond to the current spinal level\n    columns_to_plot = [col for col in df_train_copy.columns if level in col]\n    # Stack the data for plotting\n    stacked_data = df_train_copy[columns_to_plot].stack()\n    value_counts = stacked_data.value_counts()\n\n    # Create the bar plot\n    sns.barplot(x=value_counts.index, y=value_counts.values, palette=['#1f77b4', '#ff7f0e'])\n\nplt.tight_layout()\nplt.show()\n\nFor your spinal levels"
  },
  "source": "meta"
}