{
  "id": 334220,
  "title": "Lung Most Difficult Organ Category to Segment?",
  "url": "/competitions/hubmap-organ-segmentation/discussion/334220",
  "author_name": "",
  "post_date": "2022-06-30T13:29:13.880918300Z",
  "votes": 8,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hello Kagglers,</p>\n<p>I just got my first UNET model working with a validation F1 score of 0.838 using a single fold with 20% left out for validation. When further investigating the performance it came to my attention the model performed reasonably well on all organs, except the lung with an F1 score as low as 0.293.</p>\n<table>\n<thead>\n<tr>\n<th>organ</th>\n<th>Best F1</th>\n<th>Best Threshold</th>\n<th>Validation Sample Count</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>kidney</td>\n<td>0.910</td>\n<td>0.333</td>\n<td>20</td>\n</tr>\n<tr>\n<td>largeintestine</td>\n<td>0.872</td>\n<td>0.361</td>\n<td>12</td>\n</tr>\n<tr>\n<td>lung</td>\n<td>0.293</td>\n<td>0.226</td>\n<td>9</td>\n</tr>\n<tr>\n<td>prostate</td>\n<td>0.848</td>\n<td>0.361</td>\n<td>19</td>\n</tr>\n<tr>\n<td>spleen</td>\n<td>0.761</td>\n<td>0.291</td>\n<td>11</td>\n</tr>\n</tbody>\n</table>\n<p>Is your experience the same, are the samples taken from the lung an outlier in terms of segmentation difficulty, or am I missing something?</p>\n<p>I will publish my notebook in the coming days, so you can take a look yourself.</p>\n<p><strong>UPDATE 1</strong> applied a stratification with respect to the organs when splitting the dataset into train/validation for a correct distribution of samples, results are updated accordingly. Validation performance on lung samples increased slightly, but is still by far the most difficult organ.</p>",
  "messages": [
    {
      "id": "1838389",
      "postDate": "06/30/2022 13:29:13",
      "content": "<p>Hello Kagglers,</p>\n<p>I just got my first UNET model working with a validation F1 score of 0.838 using a single fold with 20% left out for validation. When further investigating the performance it came to my attention the model performed reasonably well on all organs, except the lung with an F1 score as low as 0.293.</p>\n<table>\n<thead>\n<tr>\n<th>organ</th>\n<th>Best F1</th>\n<th>Best Threshold</th>\n<th>Validation Sample Count</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>kidney</td>\n<td>0.910</td>\n<td>0.333</td>\n<td>20</td>\n</tr>\n<tr>\n<td>largeintestine</td>\n<td>0.872</td>\n<td>0.361</td>\n<td>12</td>\n</tr>\n<tr>\n<td>lung</td>\n<td>0.293</td>\n<td>0.226</td>\n<td>9</td>\n</tr>\n<tr>\n<td>prostate</td>\n<td>0.848</td>\n<td>0.361</td>\n<td>19</td>\n</tr>\n<tr>\n<td>spleen</td>\n<td>0.761</td>\n<td>0.291</td>\n<td>11</td>\n</tr>\n</tbody>\n</table>\n<p>Is your experience the same, are the samples taken from the lung an outlier in terms of segmentation difficulty, or am I missing something?</p>\n<p>I will publish my notebook in the coming days, so you can take a look yourself.</p>\n<p><strong>UPDATE 1</strong> applied a stratification with respect to the organs when splitting the dataset into train/validation for a correct distribution of samples, results are updated accordingly. Validation performance on lung samples increased slightly, but is still by far the most difficult organ.</p>",
      "rawMarkdown": "Hello Kagglers,\n\nI just got my first UNET model working with a validation F1 score of 0.838 using a single fold with 20% left out for validation. When further investigating the performance it came to my attention the model performed reasonably well on all organs, except the lung with an F1 score as low as 0.293.\n\n| organ | Best F1 | Best Threshold | Validation Sample Count |\n| --- | --- |\n|kidney | 0.910 | 0.333 | 20 |\n| largeintestine | 0.872 |  0.361| 12| \n| lung | 0.293 | 0.226| 9| \n| prostate | 0.848 | 0.361|  19| \n| spleen | 0.761 |  0.291| 11| \n\nIs your experience the same, are the samples taken from the lung an outlier in terms of segmentation difficulty, or am I missing something?\n\nI will publish my notebook in the coming days, so you can take a look yourself.\n\n**UPDATE 1** applied a stratification with respect to the organs when splitting the dataset into train/validation for a correct distribution of samples, results are updated accordingly. Validation performance on lung samples increased slightly, but is still by far the most difficult organ.",
      "votes": null
    },
    {
      "id": "1838498",
      "postDate": "06/30/2022 15:28:06",
      "content": "<p>Yup its so hard to differentiate even through human eyes I wonder how a model going to learn that.</p>",
      "rawMarkdown": "Yup its so hard to differentiate even through human eyes I wonder how a model going to learn that.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1838498,
      "author_name": "muhammad4hmed",
      "author_url": "",
      "post_date": "06/30/2022 15:28:06",
      "content": "<p>Yup its so hard to differentiate even through human eyes I wonder how a model going to learn that.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1838389": "Hello Kagglers,\n\nI just got my first UNET model working with a validation F1 score of 0.838 using a single fold with 20% left out for validation. When further investigating the performance it came to my attention the model performed reasonably well on all organs, except the lung with an F1 score as low as 0.293.\n\n| organ | Best F1 | Best Threshold | Validation Sample Count |\n| --- | --- |\n|kidney | 0.910 | 0.333 | 20 |\n| largeintestine | 0.872 |  0.361| 12| \n| lung | 0.293 | 0.226| 9| \n| prostate | 0.848 | 0.361|  19| \n| spleen | 0.761 |  0.291| 11| \n\nIs your experience the same, are the samples taken from the lung an outlier in terms of segmentation difficulty, or am I missing something?\n\nI will publish my notebook in the coming days, so you can take a look yourself.\n\n**UPDATE 1** applied a stratification with respect to the organs when splitting the dataset into train/validation for a correct distribution of samples, results are updated accordingly. Validation performance on lung samples increased slightly, but is still by far the most difficult organ.",
    "1838498": "Yup its so hard to differentiate even through human eyes I wonder how a model going to learn that."
  },
  "source": "meta"
}