{
  "id": 296408,
  "title": "Annotation bugs? Which annotations should I trust?",
  "url": "/competitions/sartorius-cell-instance-segmentation/discussion/296408",
  "author_name": "",
  "post_date": "2021-12-21T12:39:57.386113100Z",
  "votes": 3,
  "comment_count": 9,
  "views": 0,
  "content": "<p>I found many annotation errors in the annotation files provided by Livecell. </p>\n<p>There are many images with different annotations. <strong>118 images</strong> at least. </p>\n<p>Examples of annotation errors:</p>\n<p><img src=\"https://i.postimg.cc/Y2sqfwGb/1212.png\" alt=\"\"><br>\n<img src=\"https://i.postimg.cc/RVG8nwPt/1213.png\" alt=\"\"><br>\n<img src=\"https://i.postimg.cc/28Bb1xjc/1214.png\" alt=\"\"></p>",
  "messages": [
    {
      "id": "1624997",
      "postDate": "12/21/2021 12:39:57",
      "content": "<p>I found many annotation errors in the annotation files provided by Livecell. </p>\n<p>There are many images with different annotations. <strong>118 images</strong> at least. </p>\n<p>Examples of annotation errors:</p>\n<p><img src=\"https://i.postimg.cc/Y2sqfwGb/1212.png\" alt=\"\"><br>\n<img src=\"https://i.postimg.cc/RVG8nwPt/1213.png\" alt=\"\"><br>\n<img src=\"https://i.postimg.cc/28Bb1xjc/1214.png\" alt=\"\"></p>",
      "rawMarkdown": "I found many annotation errors in the annotation files provided by Livecell. \n\nThere are many images with different annotations. **118 images** at least. \n\nExamples of annotation errors:\n\n![](https://i.postimg.cc/Y2sqfwGb/1212.png)\n![](https://i.postimg.cc/RVG8nwPt/1213.png)\n![](https://i.postimg.cc/28Bb1xjc/1214.png)",
      "votes": null
    },
    {
      "id": "1625038",
      "postDate": "12/21/2021 13:13:31",
      "content": "<p>Yeah, annotations are inconsistent, I found similar issue for primary dataset -- <a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/290441\" target=\"_blank\">https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/290441</a>.</p>",
      "rawMarkdown": "Yeah, annotations are inconsistent, I found similar issue for primary dataset -- https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/290441.",
      "votes": null
    },
    {
      "id": "1625062",
      "postDate": "12/21/2021 13:45:06",
      "content": "<p>I also found data leakage in the providied annotations. <a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361\" target=\"_blank\">https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361</a></p>",
      "rawMarkdown": "I also found data leakage in the providied annotations. [https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361)",
      "votes": null
    },
    {
      "id": "1625063",
      "postDate": "12/21/2021 13:46:03",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/blueboy97\" target=\"_blank\">@blueboy97</a>, irrelevant but did you try AP evaluation in livecell github? I attempt to find the best evaluation for this competition but failed :( I dont no how the competitors can achieve 0.3x CV</p>",
      "rawMarkdown": "Hi @blueboy97, irrelevant but did you try AP evaluation in livecell github? I attempt to find the best evaluation for this competition but failed :( I dont no how the competitors can achieve 0.3x CV",
      "votes": null
    },
    {
      "id": "1625075",
      "postDate": "12/21/2021 13:53:52",
      "content": "<p>I haven't tried yet. I use metric from this <a href=\"https://www.kaggle.com/theoviel/competition-metric-map-iou\" target=\"_blank\">notebook</a>. Maybe different people calculate CV in different ways. Compare your CV with LB, and you will know the relationship between them.</p>",
      "rawMarkdown": "I haven't tried yet. I use metric from this [notebook](https://www.kaggle.com/theoviel/competition-metric-map-iou). Maybe different people calculate CV in different ways. Compare your CV with LB, and you will know the relationship between them.",
      "votes": null
    },
    {
      "id": "1625179",
      "postDate": "12/21/2021 15:04:25",
      "content": "<p>Thanks for your reply 😄</p>",
      "rawMarkdown": "Thanks for your reply 😄",
      "votes": null
    },
    {
      "id": "1625255",
      "postDate": "12/21/2021 16:14:12",
      "content": "<p>Leakage in LiveCell doesn't not matter in this competition really. I just put all annotations together for train/test/validaiton for Live Cell data for transfer learning purpose.</p>",
      "rawMarkdown": "Leakage in LiveCell doesn't not matter in this competition really. I just put all annotations together for train/test/validaiton for Live Cell data for transfer learning purpose.",
      "votes": null
    },
    {
      "id": "1625305",
      "postDate": "12/21/2021 16:55:23",
      "content": "<p>Yep, that's what I am doing now.</p>",
      "rawMarkdown": "Yep, that's what I am doing now.",
      "votes": null
    },
    {
      "id": "1625592",
      "postDate": "12/22/2021 00:52:49",
      "content": "<p>Are you put together with LiveCell's data (cell_type=3+7)? Have you improved by using this method of transfer learning?</p>",
      "rawMarkdown": "Are you put together with LiveCell's data (cell_type=3+7)? Have you improved by using this method of transfer learning?",
      "votes": null
    },
    {
      "id": "1625794",
      "postDate": "12/22/2021 06:32:48",
      "content": "<p>I do, but I don't oversample them and don't apply the same augmentations to them.</p>",
      "rawMarkdown": "I do, but I don't oversample them and don't apply the same augmentations to them.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1625038,
      "author_name": "rednikotin",
      "author_url": "",
      "post_date": "12/21/2021 13:13:31",
      "content": "<p>Yeah, annotations are inconsistent, I found similar issue for primary dataset -- <a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/290441\" target=\"_blank\">https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/290441</a>.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1625062,
          "author_name": "blueboy97",
          "author_url": "",
          "post_date": "12/21/2021 13:45:06",
          "content": "<p>I also found data leakage in the providied annotations. <a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361\" target=\"_blank\">https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1625255,
          "author_name": "rednikotin",
          "author_url": "",
          "post_date": "12/21/2021 16:14:12",
          "content": "<p>Leakage in LiveCell doesn't not matter in this competition really. I just put all annotations together for train/test/validaiton for Live Cell data for transfer learning purpose.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1625305,
          "author_name": "blueboy97",
          "author_url": "",
          "post_date": "12/21/2021 16:55:23",
          "content": "<p>Yep, that's what I am doing now.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1625592,
          "author_name": "zaopolearning",
          "author_url": "",
          "post_date": "12/22/2021 00:52:49",
          "content": "<p>Are you put together with LiveCell's data (cell_type=3+7)? Have you improved by using this method of transfer learning?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1625794,
          "author_name": "rednikotin",
          "author_url": "",
          "post_date": "12/22/2021 06:32:48",
          "content": "<p>I do, but I don't oversample them and don't apply the same augmentations to them.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1625063,
      "author_name": "nochanged",
      "author_url": "",
      "post_date": "12/21/2021 13:46:03",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/blueboy97\" target=\"_blank\">@blueboy97</a>, irrelevant but did you try AP evaluation in livecell github? I attempt to find the best evaluation for this competition but failed :( I dont no how the competitors can achieve 0.3x CV</p>",
      "votes": null,
      "replies": [
        {
          "id": 1625075,
          "author_name": "blueboy97",
          "author_url": "",
          "post_date": "12/21/2021 13:53:52",
          "content": "<p>I haven't tried yet. I use metric from this <a href=\"https://www.kaggle.com/theoviel/competition-metric-map-iou\" target=\"_blank\">notebook</a>. Maybe different people calculate CV in different ways. Compare your CV with LB, and you will know the relationship between them.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1625179,
          "author_name": "nochanged",
          "author_url": "",
          "post_date": "12/21/2021 15:04:25",
          "content": "<p>Thanks for your reply 😄</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1624997": "I found many annotation errors in the annotation files provided by Livecell. \n\nThere are many images with different annotations. **118 images** at least. \n\nExamples of annotation errors:\n\n![](https://i.postimg.cc/Y2sqfwGb/1212.png)\n![](https://i.postimg.cc/RVG8nwPt/1213.png)\n![](https://i.postimg.cc/28Bb1xjc/1214.png)",
    "1625038": "Yeah, annotations are inconsistent, I found similar issue for primary dataset -- https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/290441.",
    "1625062": "I also found data leakage in the providied annotations. [https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/296361)",
    "1625063": "Hi @blueboy97, irrelevant but did you try AP evaluation in livecell github? I attempt to find the best evaluation for this competition but failed :( I dont no how the competitors can achieve 0.3x CV",
    "1625075": "I haven't tried yet. I use metric from this [notebook](https://www.kaggle.com/theoviel/competition-metric-map-iou). Maybe different people calculate CV in different ways. Compare your CV with LB, and you will know the relationship between them.",
    "1625179": "Thanks for your reply 😄",
    "1625255": "Leakage in LiveCell doesn't not matter in this competition really. I just put all annotations together for train/test/validaiton for Live Cell data for transfer learning purpose.",
    "1625305": "Yep, that's what I am doing now.",
    "1625592": "Are you put together with LiveCell's data (cell_type=3+7)? Have you improved by using this method of transfer learning?",
    "1625794": "I do, but I don't oversample them and don't apply the same augmentations to them."
  },
  "source": "meta"
}