{
  "id": 628688,
  "title": "Unlabeled regions",
  "url": "/competitions/vesuvius-challenge-surface-detection/discussion/628688",
  "author_name": "",
  "post_date": "2025-11-17T11:39:31.129661700Z",
  "votes": 6,
  "comment_count": 10,
  "views": 0,
  "content": "<p>Hi everyone, we are confused about what exactly the unlabeled region denotes. Is it the case that there are actually surfaces present in the unlabeled region, but are not annotated? If so , is that the case in the test set as well? For example, this prediction scores about 0.59 using the official metric. <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3459992%2Fbd791caad7f49627d9fefd9390527086%2FScreenshot%202025-11-17%20213429.png?generation=1763378100754305&amp;alt=media\" alt=\"\"></p>\n<p>If we compare the label and prediction, it's almost exactly the same if we consider only label 1 (with a few inconsistencies). Do those small inconsistencies account for all the drop in score, or is it something else? And also , I noticed that there will be more data added later on. Will there also be any refinements done to the current training data in that case?</p>\n<p>Thank you!</p>",
  "messages": [
    {
      "id": "3334032",
      "postDate": "11/17/2025 11:39:31",
      "content": "<p>Hi everyone, we are confused about what exactly the unlabeled region denotes. Is it the case that there are actually surfaces present in the unlabeled region, but are not annotated? If so , is that the case in the test set as well? For example, this prediction scores about 0.59 using the official metric. <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3459992%2Fbd791caad7f49627d9fefd9390527086%2FScreenshot%202025-11-17%20213429.png?generation=1763378100754305&amp;alt=media\" alt=\"\"></p>\n<p>If we compare the label and prediction, it's almost exactly the same if we consider only label 1 (with a few inconsistencies). Do those small inconsistencies account for all the drop in score, or is it something else? And also , I noticed that there will be more data added later on. Will there also be any refinements done to the current training data in that case?</p>\n<p>Thank you!</p>",
      "rawMarkdown": "Hi everyone, we are confused about what exactly the unlabeled region denotes. Is it the case that there are actually surfaces present in the unlabeled region, but are not annotated? If so , is that the case in the test set as well? For example, this prediction scores about 0.59 using the official metric. ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3459992%2Fbd791caad7f49627d9fefd9390527086%2FScreenshot%202025-11-17%20213429.png?generation=1763378100754305&alt=media)\n\nIf we compare the label and prediction, it's almost exactly the same if we consider only label 1 (with a few inconsistencies). Do those small inconsistencies account for all the drop in score, or is it something else? And also , I noticed that there will be more data added later on. Will there also be any refinements done to the current training data in that case?\n\nThank you!",
      "votes": null
    },
    {
      "id": "3334393",
      "postDate": "11/17/2025 14:21:22",
      "content": "<ol>\n<li>There are unlabeled regions also in the test set.</li>\n<li>The score you are observing make sense. The VOI and the Toposcore components in our metrics are greatly affected by topology changes. So if you have a different number of connected components or holes or handles in the predictions, these are going to penalize the score. Our main purpose is having predictions as topological accurate as possible, it's not really important to have them voxel accurate.</li>\n</ol>",
      "rawMarkdown": "1. There are unlabeled regions also in the test set.\n2. The score you are observing make sense. The VOI and the Toposcore components in our metrics are greatly affected by topology changes. So if you have a different number of connected components or holes or handles in the predictions, these are going to penalize the score. Our main purpose is having predictions as topological accurate as possible, it's not really important to have them voxel accurate.",
      "votes": null
    },
    {
      "id": "3335310",
      "postDate": "11/17/2025 23:42:27",
      "content": "<p>Thank you!</p>",
      "rawMarkdown": "Thank you!",
      "votes": null
    },
    {
      "id": "3335532",
      "postDate": "11/18/2025 04:39:10",
      "content": "<p>Hello, I have a question. Do we need to predict the unlabeled regions?</p>",
      "rawMarkdown": "Hello, I have a question. Do we need to predict the unlabeled regions?",
      "votes": null
    },
    {
      "id": "3335562",
      "postDate": "11/18/2025 05:05:03",
      "content": "<p>Nope, no need to predict those.</p>",
      "rawMarkdown": "Nope, no need to predict those.",
      "votes": null
    },
    {
      "id": "3335583",
      "postDate": "11/18/2025 05:21:17",
      "content": "<p>I actually think we should predict across the entire image for the test set.\nWe don’t really know which regions are unlabeled in the test data, do we?</p>",
      "rawMarkdown": "I actually think we should predict across the entire image for the test set.\nWe don’t really know which regions are unlabeled in the test data, do we?",
      "votes": null
    },
    {
      "id": "3335590",
      "postDate": "11/18/2025 05:27:30",
      "content": "<p>We just predict label 1, i.e. the surface, across the entire image. Label 2 (unlabeled) is not required to be predicted. I assumed the question was that \"do we have to predict label 2?\" to which I replied nope, but yes, we will have to predict label 1 across the entire image as you said.</p>",
      "rawMarkdown": "We just predict label 1, i.e. the surface, across the entire image. Label 2 (unlabeled) is not required to be predicted. I assumed the question was that \"do we have to predict label 2?\" to which I replied nope, but yes, we will have to predict label 1 across the entire image as you said.",
      "votes": null
    },
    {
      "id": "3335687",
      "postDate": "11/18/2025 06:24:39",
      "content": "<p>Thanks for the explanation</p>",
      "rawMarkdown": "Thanks for the explanation",
      "votes": null
    },
    {
      "id": "3343654",
      "postDate": "11/21/2025 20:25:20",
      "content": "<p>You do predict across the image, but the unlabeled areas in the test set are not counted against in the score. So you still get the context of the region there, it just doesnt influence your final leaderboard score</p>",
      "rawMarkdown": "You do predict across the image, but the unlabeled areas in the test set are not counted against in the score. So you still get the context of the region there, it just doesnt influence your final leaderboard score",
      "votes": null
    },
    {
      "id": "3351440",
      "postDate": "11/28/2025 12:08:29",
      "content": "<blockquote>\n  <p>There are unlabeled regions also in the test set.</p>\n</blockquote>\n<p>Are these areas then excluded from the scoring? I was doing so when I was <a href=\"https://www.kaggle.com/code/jirkaborovec/surface-train-inference-slice-2d-segmentation\" target=\"_blank\">computing the model evaluation</a>…</p>",
      "rawMarkdown": "> There are unlabeled regions also in the test set.\n\nAre these areas then excluded from the scoring? I was doing so when I was [computing the model evaluation](https://www.kaggle.com/code/jirkaborovec/surface-train-inference-slice-2d-segmentation)...",
      "votes": null
    },
    {
      "id": "3351590",
      "postDate": "11/28/2025 14:19:26",
      "content": "<p>They are indeed! </p>",
      "rawMarkdown": "They are indeed!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3334393,
      "author_name": "giorgioangelotti",
      "author_url": "",
      "post_date": "11/17/2025 14:21:22",
      "content": "<ol>\n<li>There are unlabeled regions also in the test set.</li>\n<li>The score you are observing make sense. The VOI and the Toposcore components in our metrics are greatly affected by topology changes. So if you have a different number of connected components or holes or handles in the predictions, these are going to penalize the score. Our main purpose is having predictions as topological accurate as possible, it's not really important to have them voxel accurate.</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 3335310,
          "author_name": "p4rallax",
          "author_url": "",
          "post_date": "11/17/2025 23:42:27",
          "content": "<p>Thank you!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 3335532,
          "author_name": "moseliao",
          "author_url": "",
          "post_date": "11/18/2025 04:39:10",
          "content": "<p>Hello, I have a question. Do we need to predict the unlabeled regions?</p>",
          "votes": null,
          "replies": [
            {
              "id": 3335562,
              "author_name": "p4rallax",
              "author_url": "",
              "post_date": "11/18/2025 05:05:03",
              "content": "<p>Nope, no need to predict those.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 3335583,
                  "author_name": "sapr3s",
                  "author_url": "",
                  "post_date": "11/18/2025 05:21:17",
                  "content": "<p>I actually think we should predict across the entire image for the test set.\nWe don’t really know which regions are unlabeled in the test data, do we?</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 3335590,
                      "author_name": "p4rallax",
                      "author_url": "",
                      "post_date": "11/18/2025 05:27:30",
                      "content": "<p>We just predict label 1, i.e. the surface, across the entire image. Label 2 (unlabeled) is not required to be predicted. I assumed the question was that \"do we have to predict label 2?\" to which I replied nope, but yes, we will have to predict label 1 across the entire image as you said.</p>",
                      "votes": null,
                      "replies": [
                        {
                          "id": 3335687,
                          "author_name": "moseliao",
                          "author_url": "",
                          "post_date": "11/18/2025 06:24:39",
                          "content": "<p>Thanks for the explanation</p>",
                          "votes": null,
                          "replies": []
                        }
                      ]
                    },
                    {
                      "id": 3343654,
                      "author_name": "seanjohnsonsp",
                      "author_url": "",
                      "post_date": "11/21/2025 20:25:20",
                      "content": "<p>You do predict across the image, but the unlabeled areas in the test set are not counted against in the score. So you still get the context of the region there, it just doesnt influence your final leaderboard score</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        },
        {
          "id": 3351440,
          "author_name": "jirkaborovec",
          "author_url": "",
          "post_date": "11/28/2025 12:08:29",
          "content": "<blockquote>\n  <p>There are unlabeled regions also in the test set.</p>\n</blockquote>\n<p>Are these areas then excluded from the scoring? I was doing so when I was <a href=\"https://www.kaggle.com/code/jirkaborovec/surface-train-inference-slice-2d-segmentation\" target=\"_blank\">computing the model evaluation</a>…</p>",
          "votes": null,
          "replies": [
            {
              "id": 3351590,
              "author_name": "seanjohnsonsp",
              "author_url": "",
              "post_date": "11/28/2025 14:19:26",
              "content": "<p>They are indeed! </p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3334032": "Hi everyone, we are confused about what exactly the unlabeled region denotes. Is it the case that there are actually surfaces present in the unlabeled region, but are not annotated? If so , is that the case in the test set as well? For example, this prediction scores about 0.59 using the official metric. ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3459992%2Fbd791caad7f49627d9fefd9390527086%2FScreenshot%202025-11-17%20213429.png?generation=1763378100754305&alt=media)\n\nIf we compare the label and prediction, it's almost exactly the same if we consider only label 1 (with a few inconsistencies). Do those small inconsistencies account for all the drop in score, or is it something else? And also , I noticed that there will be more data added later on. Will there also be any refinements done to the current training data in that case?\n\nThank you!",
    "3334393": "1. There are unlabeled regions also in the test set.\n2. The score you are observing make sense. The VOI and the Toposcore components in our metrics are greatly affected by topology changes. So if you have a different number of connected components or holes or handles in the predictions, these are going to penalize the score. Our main purpose is having predictions as topological accurate as possible, it's not really important to have them voxel accurate.",
    "3335310": "Thank you!",
    "3335532": "Hello, I have a question. Do we need to predict the unlabeled regions?",
    "3335562": "Nope, no need to predict those.",
    "3335583": "I actually think we should predict across the entire image for the test set.\nWe don’t really know which regions are unlabeled in the test data, do we?",
    "3335590": "We just predict label 1, i.e. the surface, across the entire image. Label 2 (unlabeled) is not required to be predicted. I assumed the question was that \"do we have to predict label 2?\" to which I replied nope, but yes, we will have to predict label 1 across the entire image as you said.",
    "3335687": "Thanks for the explanation",
    "3343654": "You do predict across the image, but the unlabeled areas in the test set are not counted against in the score. So you still get the context of the region there, it just doesnt influence your final leaderboard score",
    "3351440": "> There are unlabeled regions also in the test set.\n\nAre these areas then excluded from the scoring? I was doing so when I was [computing the model evaluation](https://www.kaggle.com/code/jirkaborovec/surface-train-inference-slice-2d-segmentation)...",
    "3351590": "They are indeed!"
  },
  "source": "meta"
}