{
  "id": 207509,
  "title": "[HELP] predicted masks look okay but 0.000 public LB score on submission?",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/207509",
  "author_name": "",
  "post_date": "2020-12-30T01:49:47.588995Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi all,</p>\n<p>Not sure if anyone has encountered the same/similar issue here…</p>\n<p>I have trained an initial model whose prediction (when decoded from rle) show good-ish overlap with test-set glomeruli that can be identified by eye…<a href=\"https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50593131\" target=\"_blank\">https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50593131</a> (example plots, only shown for part of a test .tiff, but the quality is comparable for several other test .tiffs)</p>\n<p>I'm quite confused by the fact that the public LB score turns out to be 0.000 (was expecting a non-0 score), but I have run out of debugging ideas….</p>\n<p>The <code>submission.csv</code> associated with the submission notebook is here <a href=\"https://www.kaggle.com/meowyc/hubmap-test-submission-0-score-why-20201230\" target=\"_blank\">https://www.kaggle.com/meowyc/hubmap-test-submission-0-score-why-20201230</a></p>\n<p>Update: the dice coeffs (one per public test .TIFF) between the problematic submission.csv and some of the best model predictions (<a href=\"https://www.kaggle.com/tivfrvqhs5/global-mask-shift?scriptVersionId=50597377\" target=\"_blank\">https://www.kaggle.com/tivfrvqhs5/global-mask-shift?scriptVersionId=50597377</a>) are between 0.5 and 0.9. Which makes the 0.000 public LB score even more confusing…</p>",
  "messages": [
    {
      "id": "1131802",
      "postDate": "12/30/2020 01:49:47",
      "content": "<p>Hi all,</p>\n<p>Not sure if anyone has encountered the same/similar issue here…</p>\n<p>I have trained an initial model whose prediction (when decoded from rle) show good-ish overlap with test-set glomeruli that can be identified by eye…<a href=\"https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50593131\" target=\"_blank\">https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50593131</a> (example plots, only shown for part of a test .tiff, but the quality is comparable for several other test .tiffs)</p>\n<p>I'm quite confused by the fact that the public LB score turns out to be 0.000 (was expecting a non-0 score), but I have run out of debugging ideas….</p>\n<p>The <code>submission.csv</code> associated with the submission notebook is here <a href=\"https://www.kaggle.com/meowyc/hubmap-test-submission-0-score-why-20201230\" target=\"_blank\">https://www.kaggle.com/meowyc/hubmap-test-submission-0-score-why-20201230</a></p>\n<p>Update: the dice coeffs (one per public test .TIFF) between the problematic submission.csv and some of the best model predictions (<a href=\"https://www.kaggle.com/tivfrvqhs5/global-mask-shift?scriptVersionId=50597377\" target=\"_blank\">https://www.kaggle.com/tivfrvqhs5/global-mask-shift?scriptVersionId=50597377</a>) are between 0.5 and 0.9. Which makes the 0.000 public LB score even more confusing…</p>",
      "rawMarkdown": "Hi all,\n\nNot sure if anyone has encountered the same/similar issue here...\n\nI have trained an initial model whose prediction (when decoded from rle) show good-ish overlap with test-set glomeruli that can be identified by eye...https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50593131 (example plots, only shown for part of a test .tiff, but the quality is comparable for several other test .tiffs)\n\nI'm quite confused by the fact that the public LB score turns out to be 0.000 (was expecting a non-0 score), but I have run out of debugging ideas....\n\nThe `submission.csv` associated with the submission notebook is here https://www.kaggle.com/meowyc/hubmap-test-submission-0-score-why-20201230\n\n\nUpdate: the dice coeffs (one per public test .TIFF) between the problematic submission.csv and some of the best model predictions (https://www.kaggle.com/tivfrvqhs5/global-mask-shift?scriptVersionId=50597377) are between 0.5 and 0.9. Which makes the 0.000 public LB score even more confusing...",
      "votes": null
    },
    {
      "id": "1133142",
      "postDate": "12/31/2020 01:28:54",
      "content": "<p>Update: example plots and dice scores are now included in a new version of the notebook: <a href=\"https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50635537\" target=\"_blank\">https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50635537</a></p>",
      "rawMarkdown": "Update: example plots and dice scores are now included in a new version of the notebook: https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50635537",
      "votes": null
    },
    {
      "id": "1133201",
      "postDate": "12/31/2020 02:47:24",
      "content": "<p>Now that you use the keras framework, you can check out this person’s public notebook to find out the cause of the error<a href=\"url\" target=\"_blank\">https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-subm</a></p>",
      "rawMarkdown": "Now that you use the keras framework, you can check out this person’s public notebook to find out the cause of the error[https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-subm](url)",
      "votes": null
    },
    {
      "id": "1133205",
      "postDate": "12/31/2020 02:49:04",
      "content": "<p>See how others predict, if you have problems during training, you can take a look<a href=\"url\" target=\"_blank\">https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train</a></p>",
      "rawMarkdown": "See how others predict, if you have problems during training, you can take a look[https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train](url)",
      "votes": null
    },
    {
      "id": "1133258",
      "postDate": "12/31/2020 04:08:03",
      "content": "<p>Hi, I've encountered the same problem during very first submissions. My problem is due to the setting of index column (i.e. you need to set \"id\" column as index).</p>\n<p>May it helps</p>",
      "rawMarkdown": "Hi, I've encountered the same problem during very first submissions. My problem is due to the setting of index column (i.e. you need to set \"id\" column as index).\n\nMay it helps",
      "votes": null
    },
    {
      "id": "1133699",
      "postDate": "12/31/2020 12:50:56",
      "content": "<p>Did you transpose your mask before submitting it?</p>",
      "rawMarkdown": "Did you transpose your mask before submitting it?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1133142,
      "author_name": "meowyc",
      "author_url": "",
      "post_date": "12/31/2020 01:28:54",
      "content": "<p>Update: example plots and dice scores are now included in a new version of the notebook: <a href=\"https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50635537\" target=\"_blank\">https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50635537</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1133205,
          "author_name": "zhangeng",
          "author_url": "",
          "post_date": "12/31/2020 02:49:04",
          "content": "<p>See how others predict, if you have problems during training, you can take a look<a href=\"url\" target=\"_blank\">https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1133201,
      "author_name": "zhangeng",
      "author_url": "",
      "post_date": "12/31/2020 02:47:24",
      "content": "<p>Now that you use the keras framework, you can check out this person’s public notebook to find out the cause of the error<a href=\"url\" target=\"_blank\">https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-subm</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1133258,
      "author_name": "lijiaqi96",
      "author_url": "",
      "post_date": "12/31/2020 04:08:03",
      "content": "<p>Hi, I've encountered the same problem during very first submissions. My problem is due to the setting of index column (i.e. you need to set \"id\" column as index).</p>\n<p>May it helps</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1133699,
      "author_name": "sakvaua",
      "author_url": "",
      "post_date": "12/31/2020 12:50:56",
      "content": "<p>Did you transpose your mask before submitting it?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1131802": "Hi all,\n\nNot sure if anyone has encountered the same/similar issue here...\n\nI have trained an initial model whose prediction (when decoded from rle) show good-ish overlap with test-set glomeruli that can be identified by eye...https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50593131 (example plots, only shown for part of a test .tiff, but the quality is comparable for several other test .tiffs)\n\nI'm quite confused by the fact that the public LB score turns out to be 0.000 (was expecting a non-0 score), but I have run out of debugging ideas....\n\nThe `submission.csv` associated with the submission notebook is here https://www.kaggle.com/meowyc/hubmap-test-submission-0-score-why-20201230\n\n\nUpdate: the dice coeffs (one per public test .TIFF) between the problematic submission.csv and some of the best model predictions (https://www.kaggle.com/tivfrvqhs5/global-mask-shift?scriptVersionId=50597377) are between 0.5 and 0.9. Which makes the 0.000 public LB score even more confusing...",
    "1133142": "Update: example plots and dice scores are now included in a new version of the notebook: https://www.kaggle.com/meowyc/hubmap-check-test-pred?scriptVersionId=50635537",
    "1133201": "Now that you use the keras framework, you can check out this person’s public notebook to find out the cause of the error[https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-subm](url)",
    "1133205": "See how others predict, if you have problems during training, you can take a look[https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train](url)",
    "1133258": "Hi, I've encountered the same problem during very first submissions. My problem is due to the setting of index column (i.e. you need to set \"id\" column as index).\n\nMay it helps",
    "1133699": "Did you transpose your mask before submitting it?"
  },
  "source": "meta"
}