{
  "id": 347960,
  "title": "There is a big gap between the local cv results and the lb results.",
  "url": "/competitions/hubmap-organ-segmentation/discussion/347960",
  "author_name": "",
  "post_date": "2022-08-26T06:06:06.632839400Z",
  "votes": 2,
  "comment_count": 6,
  "views": 0,
  "content": "<p>There is a big gap between the local cv results and the lb results. The local cv results are 0.74, but the lb results are only 0.40. The same model and image processing method. Can anyone come across a similar situation and offer some ideas?</p>",
  "messages": [
    {
      "id": "1914486",
      "postDate": "08/26/2022 06:06:06",
      "content": "<p>There is a big gap between the local cv results and the lb results. The local cv results are 0.74, but the lb results are only 0.40. The same model and image processing method. Can anyone come across a similar situation and offer some ideas?</p>",
      "rawMarkdown": "There is a big gap between the local cv results and the lb results. The local cv results are 0.74, but the lb results are only 0.40. The same model and image processing method. Can anyone come across a similar situation and offer some ideas?",
      "votes": null
    },
    {
      "id": "1914688",
      "postDate": "08/26/2022 10:07:45",
      "content": "<p>Hello Xiong,</p>\n<p>My own comments below. But take into acount that I am a novice …</p>\n<p>The test set is very different from the training set. It's intentional:</p>\n<p>\"This competition uses data from two different consortia, the Human Protein Atlas (HPA) and Human BioMolecular Atlas Program (HuBMAP). The <strong>training dataset consists of data from public HPA data,</strong> the public test set <strong>is a combination of private HPA data and HuBMAP data</strong>, and the private test set contains only HuBMAP data.\"</p>\n<p>Another important point is that you have also to take into account the field \"pixel_size\". With the same models, the score go 0.3 to 0.49 after I manage this size !</p>\n<p>Regards</p>\n<p>Thierry</p>",
      "rawMarkdown": "Hello Xiong,\n\nMy own comments below. But take into acount that I am a novice ...\n\nThe test set is very different from the training set. It's intentional:\n\n\"This competition uses data from two different consortia, the Human Protein Atlas (HPA) and Human BioMolecular Atlas Program (HuBMAP). The **training dataset consists of data from public HPA data,** the public test set **is a combination of private HPA data and HuBMAP data**, and the private test set contains only HuBMAP data.\"\n\nAnother important point is that you have also to take into account the field \"pixel_size\". With the same models, the score go 0.3 to 0.49 after I manage this size !\n\nRegards\n\nThierry",
      "votes": null
    },
    {
      "id": "1914690",
      "postDate": "08/26/2022 10:11:08",
      "content": "<p>We had the same issue. We used hard augmentations and our gap is almost closed.</p>",
      "rawMarkdown": "We had the same issue. We used hard augmentations and our gap is almost closed.",
      "votes": null
    },
    {
      "id": "1914953",
      "postDate": "08/26/2022 15:01:38",
      "content": "<p>Hi Neusius. Thank you so much for your crucial reminder that the training and test sets have different stain styles. i think we should reconsider the right way to normalize them even though we do use the staintools to deal with this problem. <br>\nYes! field \"pixel_size I think will be a very useful point for us and we will try it <br>\nthanks again and good luck to you too :)</p>",
      "rawMarkdown": "Hi Neusius. Thank you so much for your crucial reminder that the training and test sets have different stain styles. i think we should reconsider the right way to normalize them even though we do use the staintools to deal with this problem. \nYes! field \"pixel_size I think will be a very useful point for us and we will try it \nthanks again and good luck to you too :)",
      "votes": null
    },
    {
      "id": "1914968",
      "postDate": "08/26/2022 15:11:23",
      "content": "<p>Thanks much for your providing information to us. i agree that augmentation is the key to solve the problem, cause we only simply use the staintools but i think it may be far from enough…  </p>",
      "rawMarkdown": "Thanks much for your providing information to us. i agree that augmentation is the key to solve the problem, cause we only simply use the staintools but i think it may be far from enough...",
      "votes": null
    },
    {
      "id": "1920748",
      "postDate": "08/31/2022 11:31:52",
      "content": "<p>Don't rule out bug in your submission. See you are doing sigmoid before thresholding.</p>",
      "rawMarkdown": "Don't rule out bug in your submission. See you are doing sigmoid before thresholding.",
      "votes": null
    },
    {
      "id": "1944064",
      "postDate": "09/18/2022 03:28:09",
      "content": "<p>Hi, I am a beginner, can I ask you what kind of strong data augmentation you use to reduce the gap during training and testing, thank you very much!🙏</p>",
      "rawMarkdown": "Hi, I am a beginner, can I ask you what kind of strong data augmentation you use to reduce the gap during training and testing, thank you very much!🙏",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1914688,
      "author_name": "thierryneusius",
      "author_url": "",
      "post_date": "08/26/2022 10:07:45",
      "content": "<p>Hello Xiong,</p>\n<p>My own comments below. But take into acount that I am a novice …</p>\n<p>The test set is very different from the training set. It's intentional:</p>\n<p>\"This competition uses data from two different consortia, the Human Protein Atlas (HPA) and Human BioMolecular Atlas Program (HuBMAP). The <strong>training dataset consists of data from public HPA data,</strong> the public test set <strong>is a combination of private HPA data and HuBMAP data</strong>, and the private test set contains only HuBMAP data.\"</p>\n<p>Another important point is that you have also to take into account the field \"pixel_size\". With the same models, the score go 0.3 to 0.49 after I manage this size !</p>\n<p>Regards</p>\n<p>Thierry</p>",
      "votes": null,
      "replies": [
        {
          "id": 1914953,
          "author_name": "xiongzheli",
          "author_url": "",
          "post_date": "08/26/2022 15:01:38",
          "content": "<p>Hi Neusius. Thank you so much for your crucial reminder that the training and test sets have different stain styles. i think we should reconsider the right way to normalize them even though we do use the staintools to deal with this problem. <br>\nYes! field \"pixel_size I think will be a very useful point for us and we will try it <br>\nthanks again and good luck to you too :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1914690,
      "author_name": "erdican",
      "author_url": "",
      "post_date": "08/26/2022 10:11:08",
      "content": "<p>We had the same issue. We used hard augmentations and our gap is almost closed.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1914968,
          "author_name": "xiongzheli",
          "author_url": "",
          "post_date": "08/26/2022 15:11:23",
          "content": "<p>Thanks much for your providing information to us. i agree that augmentation is the key to solve the problem, cause we only simply use the staintools but i think it may be far from enough…  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1944064,
          "author_name": "xihuguan",
          "author_url": "",
          "post_date": "09/18/2022 03:28:09",
          "content": "<p>Hi, I am a beginner, can I ask you what kind of strong data augmentation you use to reduce the gap during training and testing, thank you very much!🙏</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1920748,
      "author_name": "adityakumarsinha",
      "author_url": "",
      "post_date": "08/31/2022 11:31:52",
      "content": "<p>Don't rule out bug in your submission. See you are doing sigmoid before thresholding.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1914486": "There is a big gap between the local cv results and the lb results. The local cv results are 0.74, but the lb results are only 0.40. The same model and image processing method. Can anyone come across a similar situation and offer some ideas?",
    "1914688": "Hello Xiong,\n\nMy own comments below. But take into acount that I am a novice ...\n\nThe test set is very different from the training set. It's intentional:\n\n\"This competition uses data from two different consortia, the Human Protein Atlas (HPA) and Human BioMolecular Atlas Program (HuBMAP). The **training dataset consists of data from public HPA data,** the public test set **is a combination of private HPA data and HuBMAP data**, and the private test set contains only HuBMAP data.\"\n\nAnother important point is that you have also to take into account the field \"pixel_size\". With the same models, the score go 0.3 to 0.49 after I manage this size !\n\nRegards\n\nThierry",
    "1914690": "We had the same issue. We used hard augmentations and our gap is almost closed.",
    "1914953": "Hi Neusius. Thank you so much for your crucial reminder that the training and test sets have different stain styles. i think we should reconsider the right way to normalize them even though we do use the staintools to deal with this problem. \nYes! field \"pixel_size I think will be a very useful point for us and we will try it \nthanks again and good luck to you too :)",
    "1914968": "Thanks much for your providing information to us. i agree that augmentation is the key to solve the problem, cause we only simply use the staintools but i think it may be far from enough...",
    "1920748": "Don't rule out bug in your submission. See you are doing sigmoid before thresholding.",
    "1944064": "Hi, I am a beginner, can I ask you what kind of strong data augmentation you use to reduce the gap during training and testing, thank you very much!🙏"
  },
  "source": "meta"
}