{
  "id": 400370,
  "title": "Was the surface annotator blinded from the infrared image?",
  "url": "/competitions/vesuvius-challenge-ink-detection/discussion/400370",
  "author_name": "",
  "post_date": "2023-04-08T01:57:11.746068500Z",
  "votes": 8,
  "comment_count": 2,
  "views": 0,
  "content": "<p>After voxelization, slices were made and an annotator used splines to follow the surface.</p>\n<p>Who was this annotator?</p>\n<p>Was it farmed out to workers who got instructions? Did the example instructions come from the same fragment they had to annotate, or from a different fragment? (hopefully not from same fragment)</p>\n<p>If it was not farmed out, there is the risk that the annotator had seen the fragment IR image. This risks inadvertent change in annotation behavior on the known inky region versus known non-inky regions, leaking prior knowledge into the local shape and level of detail of the surface mesh, which then automatically propagates into the normal selection, which then automatically propagates into the pixel selection.</p>",
  "messages": [
    {
      "id": "2213957",
      "postDate": "04/08/2023 01:57:11",
      "content": "<p>After voxelization, slices were made and an annotator used splines to follow the surface.</p>\n<p>Who was this annotator?</p>\n<p>Was it farmed out to workers who got instructions? Did the example instructions come from the same fragment they had to annotate, or from a different fragment? (hopefully not from same fragment)</p>\n<p>If it was not farmed out, there is the risk that the annotator had seen the fragment IR image. This risks inadvertent change in annotation behavior on the known inky region versus known non-inky regions, leaking prior knowledge into the local shape and level of detail of the surface mesh, which then automatically propagates into the normal selection, which then automatically propagates into the pixel selection.</p>",
      "rawMarkdown": "After voxelization, slices were made and an annotator used splines to follow the surface.\n\nWho was this annotator?\n\nWas it farmed out to workers who got instructions? Did the example instructions come from the same fragment they had to annotate, or from a different fragment? (hopefully not from same fragment)\n\nIf it was not farmed out, there is the risk that the annotator had seen the fragment IR image. This risks inadvertent change in annotation behavior on the known inky region versus known non-inky regions, leaking prior knowledge into the local shape and level of detail of the surface mesh, which then automatically propagates into the normal selection, which then automatically propagates into the pixel selection.",
      "votes": null
    },
    {
      "id": "2214597",
      "postDate": "04/08/2023 15:57:07",
      "content": "<p>I was the annotator for all fragments.</p>\n<p>That is a fair concern, but I do not believe is a factor with this data. While I am indeed familiar with the IR images, there are not visual cues when working with the X-ray CT slice data that indicate to me where on the surface I am viewing. There's no visual information leakage that would make it possible for me to influence the segmentation based on the presence of ink. And the segmentation is based on a programmatic method that is applied evenly to the entire surface - there's no manual local guidance.</p>",
      "rawMarkdown": "I was the annotator for all fragments.\n\nThat is a fair concern, but I do not believe is a factor with this data. While I am indeed familiar with the IR images, there are not visual cues when working with the X-ray CT slice data that indicate to me where on the surface I am viewing. There's no visual information leakage that would make it possible for me to influence the segmentation based on the presence of ink. And the segmentation is based on a programmatic method that is applied evenly to the entire surface - there's no manual local guidance.",
      "votes": null
    },
    {
      "id": "2217046",
      "postDate": "04/10/2023 14:52:22",
      "content": "<p>Thanks, that's comforting.</p>\n<p>I was afraid of the spline curvatures or other level of detail variation leaking through, but as the annotator you're best positioned to judge how aware you were of where exactly you were fitting splines.</p>\n<p>What you say makes sense, it would probably take conscious effort to find out where on the fragment one is working on compared to the IR image. Fortunately you assured us this wasn't the case.</p>",
      "rawMarkdown": "Thanks, that's comforting.\n\nI was afraid of the spline curvatures or other level of detail variation leaking through, but as the annotator you're best positioned to judge how aware you were of where exactly you were fitting splines.\n\nWhat you say makes sense, it would probably take conscious effort to find out where on the fragment one is working on compared to the IR image. Fortunately you assured us this wasn't the case.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2214597,
      "author_name": "stephenrparsons",
      "author_url": "",
      "post_date": "04/08/2023 15:57:07",
      "content": "<p>I was the annotator for all fragments.</p>\n<p>That is a fair concern, but I do not believe is a factor with this data. While I am indeed familiar with the IR images, there are not visual cues when working with the X-ray CT slice data that indicate to me where on the surface I am viewing. There's no visual information leakage that would make it possible for me to influence the segmentation based on the presence of ink. And the segmentation is based on a programmatic method that is applied evenly to the entire surface - there's no manual local guidance.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2217046,
          "author_name": "ludwigmaes",
          "author_url": "",
          "post_date": "04/10/2023 14:52:22",
          "content": "<p>Thanks, that's comforting.</p>\n<p>I was afraid of the spline curvatures or other level of detail variation leaking through, but as the annotator you're best positioned to judge how aware you were of where exactly you were fitting splines.</p>\n<p>What you say makes sense, it would probably take conscious effort to find out where on the fragment one is working on compared to the IR image. Fortunately you assured us this wasn't the case.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2213957": "After voxelization, slices were made and an annotator used splines to follow the surface.\n\nWho was this annotator?\n\nWas it farmed out to workers who got instructions? Did the example instructions come from the same fragment they had to annotate, or from a different fragment? (hopefully not from same fragment)\n\nIf it was not farmed out, there is the risk that the annotator had seen the fragment IR image. This risks inadvertent change in annotation behavior on the known inky region versus known non-inky regions, leaking prior knowledge into the local shape and level of detail of the surface mesh, which then automatically propagates into the normal selection, which then automatically propagates into the pixel selection.",
    "2214597": "I was the annotator for all fragments.\n\nThat is a fair concern, but I do not believe is a factor with this data. While I am indeed familiar with the IR images, there are not visual cues when working with the X-ray CT slice data that indicate to me where on the surface I am viewing. There's no visual information leakage that would make it possible for me to influence the segmentation based on the presence of ink. And the segmentation is based on a programmatic method that is applied evenly to the entire surface - there's no manual local guidance.",
    "2217046": "Thanks, that's comforting.\n\nI was afraid of the spline curvatures or other level of detail variation leaking through, but as the annotator you're best positioned to judge how aware you were of where exactly you were fitting splines.\n\nWhat you say makes sense, it would probably take conscious effort to find out where on the fragment one is working on compared to the IR image. Fortunately you assured us this wasn't the case."
  },
  "source": "meta"
}