{
  "id": 354598,
  "title": "Sharing individual organ score on Public/Private LB",
  "url": "/competitions/hubmap-organ-segmentation/discussion/354598",
  "author_name": "",
  "post_date": "2022-09-23T02:58:07.359085500Z",
  "votes": 1,
  "comment_count": 12,
  "views": 0,
  "content": "<p>Could you share your individual organ scores on Public and Private ? I would like to compare your score to mine, checking what scores I am having low. And then I want to make my model better. <br>\nI'll go first</p>\n<pre><code>                    public          private\nall:                 0.81             0.80\nlung:                0.09             0.18\nkideny:              0.12             0.16\nlargeintestine:      0.05             0.08\nprostate:            0.14             0.17\nspleen:              0.17             0.18\n</code></pre>\n<p>It seems lung has made a big difference within top 100</p>",
  "messages": [
    {
      "id": "1951367",
      "postDate": "09/23/2022 02:58:07",
      "content": "<p>Could you share your individual organ scores on Public and Private ? I would like to compare your score to mine, checking what scores I am having low. And then I want to make my model better. <br>\nI'll go first</p>\n<pre><code>                    public          private\nall:                 0.81             0.80\nlung:                0.09             0.18\nkideny:              0.12             0.16\nlargeintestine:      0.05             0.08\nprostate:            0.14             0.17\nspleen:              0.17             0.18\n</code></pre>\n<p>It seems lung has made a big difference within top 100</p>",
      "rawMarkdown": "Could you share your individual organ scores on Public and Private ? I would like to compare your score to mine, checking what scores I am having low. And then I want to make my model better. \nI'll go first\n```\n                    public          private\nall:                 0.81             0.80\nlung:                0.09             0.18\nkideny:              0.12             0.16\nlargeintestine:      0.05             0.08\nprostate:            0.14             0.17\nspleen:              0.17             0.18\n```\nIt seems lung has made a big difference within top 100",
      "votes": null
    },
    {
      "id": "1951386",
      "postDate": "09/23/2022 03:15:06",
      "content": "<p>since the number of images, num of FTUs is unknown in each of the public/public split, care has to be taken in interpreting results.</p>\n<p>but i think lung is the greatest contributor in terms of number of images</p>\n<p>all:</p>\n<ul>\n<li>private 0.81</li>\n<li>public 0.83</li>\n</ul>\n<p>lung:</p>\n<ul>\n<li>private 0.20</li>\n<li>public 0.10</li>\n</ul>\n<hr>\n<p>largeintestine:</p>\n<ul>\n<li>private 0.08</li>\n<li>public 0.05</li>\n</ul>\n<p>spleen:</p>\n<ul>\n<li>private 0.18</li>\n<li>public 0.17</li>\n</ul>\n<p>prostate:</p>\n<ul>\n<li>private 0.17</li>\n<li>public 0.14</li>\n</ul>\n<p>kidney:</p>\n<ul>\n<li>private 0.16</li>\n<li>public 0.12</li>\n</ul>",
      "rawMarkdown": "since the number of images, num of FTUs is unknown in each of the public/public split, care has to be taken in interpreting results.\n\nbut i think lung is the greatest contributor in terms of number of images\n\nall:\n- private 0.81\n- public 0.83\n\nlung:\n- private 0.20\n- public 0.10\n\n---\n\nlargeintestine:\n- private 0.08\n- public 0.05\n\nspleen:\n- private 0.18\n- public 0.17\n\nprostate:\n- private 0.17\n- public 0.14\n\n\nkidney:\n- private 0.16\n- public 0.12",
      "votes": null
    },
    {
      "id": "1951452",
      "postDate": "09/23/2022 04:29:41",
      "content": "<p>Here is fine,<br>\nFocus on backbone and decoder:<br>\nBackbone: Swin-base&lt; nextvit-large &lt;convnext-large&lt;pvtv2-b4&lt;coat-lite-medium<br>\nwithout externel data and relabel, public lb=0.81, prostate=0.19, private lb=0.79, prostate=0.17<br>\nFor prostate, try SegNeXt which replace self-attention with large conv kernel, however it is easy to overfit.</p>\n<p>Decoder: dafomer is best, replace stage-3/4 with unext, public lb=0.81.</p>\n<p>Have no time to try relabel and externel data, may be it is useful</p>",
      "rawMarkdown": "Here is fine,\nFocus on backbone and decoder:\nBackbone: Swin-base< nextvit-large <convnext-large<pvtv2-b4<coat-lite-medium\nwithout externel data and relabel, public lb=0.81, prostate=0.19, private lb=0.79, prostate=0.17\nFor prostate, try SegNeXt which replace self-attention with large conv kernel, however it is easy to overfit.\n\nDecoder: dafomer is best, replace stage-3/4 with unext, public lb=0.81.\n\nHave no time to try relabel and externel data, may be it is useful",
      "votes": null
    },
    {
      "id": "1951464",
      "postDate": "09/23/2022 04:41:52",
      "content": "<p>Thank you very much for sharing!</p>",
      "rawMarkdown": "Thank you very much for sharing!",
      "votes": null
    },
    {
      "id": "1951466",
      "postDate": "09/23/2022 04:42:07",
      "content": "<p>Did anyone solve the mystery of the large intestine? We had the most beautiful predicted masks for this class on HPA, GTEX, Kaggle LI dataset, and overall LI had the second largest by-organ dice on local validation (after kidneys), but LB score was just 0.05, which implies that the LB dice for this organ is just around 0.7. What's up with that? It's not too different from our implied lung score and we knew it would be low.</p>",
      "rawMarkdown": "Did anyone solve the mystery of the large intestine? We had the most beautiful predicted masks for this class on HPA, GTEX, Kaggle LI dataset, and overall LI had the second largest by-organ dice on local validation (after kidneys), but LB score was just 0.05, which implies that the LB dice for this organ is just around 0.7. What's up with that? It's not too different from our implied lung score and we knew it would be low.",
      "votes": null
    },
    {
      "id": "1951467",
      "postDate": "09/23/2022 04:42:11",
      "content": "<p>Thank you very much!</p>",
      "rawMarkdown": "Thank you very much!",
      "votes": null
    },
    {
      "id": "1951473",
      "postDate": "09/23/2022 04:51:50",
      "content": "<p>i think  large intestine has the least number of hubmap images (about 41 to 44)</p>",
      "rawMarkdown": "i think  large intestine has the least number of hubmap images (about 41 to 44)",
      "votes": null
    },
    {
      "id": "1951474",
      "postDate": "09/23/2022 04:52:40",
      "content": "<p>Public/Private<br>\nall: 0.81/0.79<br>\nlung: 0.09/0.17<br>\nlargeintestine: 0.05/0.08<br>\nspleen: 0.17/0.18<br>\nprostate: 0.14/0.17<br>\nkidney: 0.12/0.16</p>\n<p>My model was weak in lungs.<br>\nMaybe it's because I only used tile images and Unet models. </p>",
      "rawMarkdown": "Public/Private\nall: 0.81/0.79\nlung: 0.09/0.17\nlargeintestine: 0.05/0.08\nspleen: 0.17/0.18\nprostate: 0.14/0.17\nkidney: 0.12/0.16\n\nMy model was weak in lungs.\nMaybe it's because I only used tile images and Unet models.",
      "votes": null
    },
    {
      "id": "1951481",
      "postDate": "09/23/2022 05:00:03",
      "content": "<p>Thank you for sharing!</p>",
      "rawMarkdown": "Thank you for sharing!",
      "votes": null
    },
    {
      "id": "1951518",
      "postDate": "09/23/2022 05:42:35",
      "content": "<p>Right, but the 0.7 dice I quoted is the average dice for all li images. It is adjusted for the number of images.</p>",
      "rawMarkdown": "Right, but the 0.7 dice I quoted is the average dice for all li images. It is adjusted for the number of images.",
      "votes": null
    },
    {
      "id": "1951892",
      "postDate": "09/23/2022 10:05:50",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> total is 43 exactly. <a href=\"https://www.kaggle.com/sakvaua\" target=\"_blank\">@sakvaua</a> I dont know my partial score for LI but to calculate dice for it will be good to know how many images there are in public/test split . i.e if split 10/33 adjusted dice .7  is probable. (upd adj. .7 is for public lb .05 right?) My std of local cv on LI around .02-.06. And with 'fixed' (manually) annotations LI is close to last place in terms of dice. In terms of data origin i think it is probably true, as on the \"outer edges\" LI annotations are ambiguous</p>",
      "rawMarkdown": "hengck23 total is 43 exactly. @sakvaua I dont know my partial score for LI but to calculate dice for it will be good to know how many images there are in public/test split . i.e if split 10/33 adjusted dice .7  is probable. (upd adj. .7 is for public lb .05 right?) My std of local cv on LI around .02-.06. And with 'fixed' (manually) annotations LI is close to last place in terms of dice. In terms of data origin i think it is probably true, as on the \"outer edges\" LI annotations are ambiguous",
      "votes": null
    },
    {
      "id": "1951916",
      "postDate": "09/23/2022 10:19:37",
      "content": "<p>Hubmap only: .11 public / .22 private at lung,  cnn with a bit tricky setup.<br>\n  .08/.17 with opencv morphology (no parameters), 30 lines of code. </p>",
      "rawMarkdown": "Hubmap only: .11 public / .22 private at lung,  cnn with a bit tricky setup.\n  .08/.17 with opencv morphology (no parameters), 30 lines of code.",
      "votes": null
    },
    {
      "id": "1952797",
      "postDate": "09/23/2022 23:49:51",
      "content": "<p>Thank you for sharing your score! I'm looking forward to seeing your setup when you post your solution</p>",
      "rawMarkdown": "Thank you for sharing your score! I'm looking forward to seeing your setup when you post your solution",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1951386,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "09/23/2022 03:15:06",
      "content": "<p>since the number of images, num of FTUs is unknown in each of the public/public split, care has to be taken in interpreting results.</p>\n<p>but i think lung is the greatest contributor in terms of number of images</p>\n<p>all:</p>\n<ul>\n<li>private 0.81</li>\n<li>public 0.83</li>\n</ul>\n<p>lung:</p>\n<ul>\n<li>private 0.20</li>\n<li>public 0.10</li>\n</ul>\n<hr>\n<p>largeintestine:</p>\n<ul>\n<li>private 0.08</li>\n<li>public 0.05</li>\n</ul>\n<p>spleen:</p>\n<ul>\n<li>private 0.18</li>\n<li>public 0.17</li>\n</ul>\n<p>prostate:</p>\n<ul>\n<li>private 0.17</li>\n<li>public 0.14</li>\n</ul>\n<p>kidney:</p>\n<ul>\n<li>private 0.16</li>\n<li>public 0.12</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 1951464,
          "author_name": "cheulkay",
          "author_url": "",
          "post_date": "09/23/2022 04:41:52",
          "content": "<p>Thank you very much for sharing!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1951452,
      "author_name": "jiageng",
      "author_url": "",
      "post_date": "09/23/2022 04:29:41",
      "content": "<p>Here is fine,<br>\nFocus on backbone and decoder:<br>\nBackbone: Swin-base&lt; nextvit-large &lt;convnext-large&lt;pvtv2-b4&lt;coat-lite-medium<br>\nwithout externel data and relabel, public lb=0.81, prostate=0.19, private lb=0.79, prostate=0.17<br>\nFor prostate, try SegNeXt which replace self-attention with large conv kernel, however it is easy to overfit.</p>\n<p>Decoder: dafomer is best, replace stage-3/4 with unext, public lb=0.81.</p>\n<p>Have no time to try relabel and externel data, may be it is useful</p>",
      "votes": null,
      "replies": [
        {
          "id": 1951467,
          "author_name": "cheulkay",
          "author_url": "",
          "post_date": "09/23/2022 04:42:11",
          "content": "<p>Thank you very much!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1951466,
      "author_name": "sakvaua",
      "author_url": "",
      "post_date": "09/23/2022 04:42:07",
      "content": "<p>Did anyone solve the mystery of the large intestine? We had the most beautiful predicted masks for this class on HPA, GTEX, Kaggle LI dataset, and overall LI had the second largest by-organ dice on local validation (after kidneys), but LB score was just 0.05, which implies that the LB dice for this organ is just around 0.7. What's up with that? It's not too different from our implied lung score and we knew it would be low.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1951473,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "09/23/2022 04:51:50",
          "content": "<p>i think  large intestine has the least number of hubmap images (about 41 to 44)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1951518,
          "author_name": "sakvaua",
          "author_url": "",
          "post_date": "09/23/2022 05:42:35",
          "content": "<p>Right, but the 0.7 dice I quoted is the average dice for all li images. It is adjusted for the number of images.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1951892,
          "author_name": "bakeryproducts",
          "author_url": "",
          "post_date": "09/23/2022 10:05:50",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> total is 43 exactly. <a href=\"https://www.kaggle.com/sakvaua\" target=\"_blank\">@sakvaua</a> I dont know my partial score for LI but to calculate dice for it will be good to know how many images there are in public/test split . i.e if split 10/33 adjusted dice .7  is probable. (upd adj. .7 is for public lb .05 right?) My std of local cv on LI around .02-.06. And with 'fixed' (manually) annotations LI is close to last place in terms of dice. In terms of data origin i think it is probably true, as on the \"outer edges\" LI annotations are ambiguous</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1951474,
      "author_name": "tamotamo",
      "author_url": "",
      "post_date": "09/23/2022 04:52:40",
      "content": "<p>Public/Private<br>\nall: 0.81/0.79<br>\nlung: 0.09/0.17<br>\nlargeintestine: 0.05/0.08<br>\nspleen: 0.17/0.18<br>\nprostate: 0.14/0.17<br>\nkidney: 0.12/0.16</p>\n<p>My model was weak in lungs.<br>\nMaybe it's because I only used tile images and Unet models. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1951481,
          "author_name": "cheulkay",
          "author_url": "",
          "post_date": "09/23/2022 05:00:03",
          "content": "<p>Thank you for sharing!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1951916,
      "author_name": "bakeryproducts",
      "author_url": "",
      "post_date": "09/23/2022 10:19:37",
      "content": "<p>Hubmap only: .11 public / .22 private at lung,  cnn with a bit tricky setup.<br>\n  .08/.17 with opencv morphology (no parameters), 30 lines of code. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1952797,
          "author_name": "cheulkay",
          "author_url": "",
          "post_date": "09/23/2022 23:49:51",
          "content": "<p>Thank you for sharing your score! I'm looking forward to seeing your setup when you post your solution</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1951367": "Could you share your individual organ scores on Public and Private ? I would like to compare your score to mine, checking what scores I am having low. And then I want to make my model better. \nI'll go first\n```\n                    public          private\nall:                 0.81             0.80\nlung:                0.09             0.18\nkideny:              0.12             0.16\nlargeintestine:      0.05             0.08\nprostate:            0.14             0.17\nspleen:              0.17             0.18\n```\nIt seems lung has made a big difference within top 100",
    "1951386": "since the number of images, num of FTUs is unknown in each of the public/public split, care has to be taken in interpreting results.\n\nbut i think lung is the greatest contributor in terms of number of images\n\nall:\n- private 0.81\n- public 0.83\n\nlung:\n- private 0.20\n- public 0.10\n\n---\n\nlargeintestine:\n- private 0.08\n- public 0.05\n\nspleen:\n- private 0.18\n- public 0.17\n\nprostate:\n- private 0.17\n- public 0.14\n\n\nkidney:\n- private 0.16\n- public 0.12",
    "1951452": "Here is fine,\nFocus on backbone and decoder:\nBackbone: Swin-base< nextvit-large <convnext-large<pvtv2-b4<coat-lite-medium\nwithout externel data and relabel, public lb=0.81, prostate=0.19, private lb=0.79, prostate=0.17\nFor prostate, try SegNeXt which replace self-attention with large conv kernel, however it is easy to overfit.\n\nDecoder: dafomer is best, replace stage-3/4 with unext, public lb=0.81.\n\nHave no time to try relabel and externel data, may be it is useful",
    "1951464": "Thank you very much for sharing!",
    "1951466": "Did anyone solve the mystery of the large intestine? We had the most beautiful predicted masks for this class on HPA, GTEX, Kaggle LI dataset, and overall LI had the second largest by-organ dice on local validation (after kidneys), but LB score was just 0.05, which implies that the LB dice for this organ is just around 0.7. What's up with that? It's not too different from our implied lung score and we knew it would be low.",
    "1951467": "Thank you very much!",
    "1951473": "i think  large intestine has the least number of hubmap images (about 41 to 44)",
    "1951474": "Public/Private\nall: 0.81/0.79\nlung: 0.09/0.17\nlargeintestine: 0.05/0.08\nspleen: 0.17/0.18\nprostate: 0.14/0.17\nkidney: 0.12/0.16\n\nMy model was weak in lungs.\nMaybe it's because I only used tile images and Unet models.",
    "1951481": "Thank you for sharing!",
    "1951518": "Right, but the 0.7 dice I quoted is the average dice for all li images. It is adjusted for the number of images.",
    "1951892": "hengck23 total is 43 exactly. @sakvaua I dont know my partial score for LI but to calculate dice for it will be good to know how many images there are in public/test split . i.e if split 10/33 adjusted dice .7  is probable. (upd adj. .7 is for public lb .05 right?) My std of local cv on LI around .02-.06. And with 'fixed' (manually) annotations LI is close to last place in terms of dice. In terms of data origin i think it is probably true, as on the \"outer edges\" LI annotations are ambiguous",
    "1951916": "Hubmap only: .11 public / .22 private at lung,  cnn with a bit tricky setup.\n  .08/.17 with opencv morphology (no parameters), 30 lines of code.",
    "1952797": "Thank you for sharing your score! I'm looking forward to seeing your setup when you post your solution"
  },
  "source": "meta"
}