{
  "id": 229697,
  "title": "LB with or without annonating the test dataset",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/229697",
  "author_name": "",
  "post_date": "2021-03-31T09:24:46.520767600Z",
  "votes": 13,
  "comment_count": 14,
  "views": 0,
  "content": "<p>To make the LB score more reasonable, I want to create a thread to collect whether people annonate the test dataset to get the LB. </p>\n<p>Hope this will help us better understand the LB score.</p>\n<p>===========================</p>\n<p>LB:0.924<br>\nCV:0.923</p>\n<p>WITHOUT ANNONATION </p>",
  "messages": [
    {
      "id": "1258011",
      "postDate": "03/31/2021 09:24:46",
      "content": "<p>To make the LB score more reasonable, I want to create a thread to collect whether people annonate the test dataset to get the LB. </p>\n<p>Hope this will help us better understand the LB score.</p>\n<p>===========================</p>\n<p>LB:0.924<br>\nCV:0.923</p>\n<p>WITHOUT ANNONATION </p>",
      "rawMarkdown": "To make the LB score more reasonable, I want to create a thread to collect whether people annonate the test dataset to get the LB. \n\nHope this will help us better understand the LB score.\n\n===========================\n\nLB:0.924\nCV:0.923\n\nWITHOUT ANNONATION",
      "votes": null
    },
    {
      "id": "1258184",
      "postDate": "03/31/2021 12:23:53",
      "content": "<p>TBH this is the closest LB and CV I've ever seen…</p>",
      "rawMarkdown": "TBH this is the closest LB and CV I've ever seen...",
      "votes": null
    },
    {
      "id": "1258508",
      "postDate": "03/31/2021 17:15:51",
      "content": "<p>Without Annotation: (LB: 0.916, CV: 0.932)</p>\n<p>I am still against ANNOTATING the test dataset. This totally changed the Kaggle competitions scheme.</p>\n<p>As <a href=\"https://www.kaggle.com/lijiaqi96\" target=\"_blank\">@lijiaqi96</a>  said, your LB and CV is so close, which is a plus point for you.</p>",
      "rawMarkdown": "Without Annotation: (LB: 0.916, CV: 0.932)\n\nI am still against ANNOTATING the test dataset. This totally changed the Kaggle competitions scheme.\n\nAs @lijiaqi96  said, your LB and CV is so close, which is a plus point for you.",
      "votes": null
    },
    {
      "id": "1258603",
      "postDate": "03/31/2021 18:41:12",
      "content": "<p>CV: 0.94<br>\nLB: 0.932</p>\n<p>(no hand labeling)</p>\n<p>working on ways to improve score on d488c759a and overall w/o hand labeling test set. it would be very misleading for cross validation otherwise. </p>",
      "rawMarkdown": "CV: 0.94\nLB: 0.932\n\n(no hand labeling)\n\nworking on ways to improve score on d488c759a and overall w/o hand labeling test set. it would be very misleading for cross validation otherwise.",
      "votes": null
    },
    {
      "id": "1261107",
      "postDate": "04/02/2021 17:43:11",
      "content": "<p>What's your CV strategy? </p>",
      "rawMarkdown": "What's your CV strategy?",
      "votes": null
    },
    {
      "id": "1264230",
      "postDate": "04/06/2021 02:14:44",
      "content": "<p>CV:0.938 LB:0.924 TTAs 0.926 without hand labeling</p>",
      "rawMarkdown": "CV:0.938 LB:0.924 TTAs 0.926 without hand labeling",
      "votes": null
    },
    {
      "id": "1265057",
      "postDate": "04/06/2021 15:36:40",
      "content": "<p>Did you use pseudo label on public test dataset or just train dataset ?</p>",
      "rawMarkdown": "Did you use pseudo label on public test dataset or just train dataset ?",
      "votes": null
    },
    {
      "id": "1265084",
      "postDate": "04/06/2021 15:53:35",
      "content": "<p>nope, no pseudo on public test set</p>",
      "rawMarkdown": "nope, no pseudo on public test set",
      "votes": null
    },
    {
      "id": "1265721",
      "postDate": "04/07/2021 06:51:21",
      "content": "<p>excellent, amazing CV&amp;LB👍</p>",
      "rawMarkdown": "excellent, amazing CV&LB👍",
      "votes": null
    },
    {
      "id": "1268715",
      "postDate": "04/09/2021 17:18:12",
      "content": "<p>Hi, Excellent CV and LB. What image size did you use both for train and CV? which model did you use?</p>",
      "rawMarkdown": "Hi, Excellent CV and LB. What image size did you use both for train and CV? which model did you use?",
      "votes": null
    },
    {
      "id": "1270784",
      "postDate": "04/12/2021 02:36:23",
      "content": "<p>My recent experiment (single model, five folds, no hand labeling and pseudo label)<br>\nCV: 0.928 (with best threshold)<br>\nLB: 0.933<br>\nvery heavy augmentations</p>",
      "rawMarkdown": "My recent experiment (single model, five folds, no hand labeling and pseudo label)\nCV: 0.928 (with best threshold)\nLB: 0.933\nvery heavy augmentations",
      "votes": null
    },
    {
      "id": "1270895",
      "postDate": "04/12/2021 06:21:29",
      "content": "<p>for how much epochs are you training ?? </p>",
      "rawMarkdown": "for how much epochs are you training ??",
      "votes": null
    },
    {
      "id": "1274107",
      "postDate": "04/15/2021 02:04:10",
      "content": "<p>did you use single fold or kfolds?</p>",
      "rawMarkdown": "did you use single fold or kfolds?",
      "votes": null
    },
    {
      "id": "1274109",
      "postDate": "04/15/2021 02:08:01",
      "content": "<p>I'm trying kfolds to see its improvement. But I still struggled in the strategies.  </p>",
      "rawMarkdown": "I'm trying kfolds to see its improvement. But I still struggled in the strategies.",
      "votes": null
    },
    {
      "id": "1276235",
      "postDate": "04/17/2021 09:35:31",
      "content": "<p>single fold, no hand labels, no pseudo</p>",
      "rawMarkdown": "single fold, no hand labels, no pseudo",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1258184,
      "author_name": "lijiaqi96",
      "author_url": "",
      "post_date": "03/31/2021 12:23:53",
      "content": "<p>TBH this is the closest LB and CV I've ever seen…</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1258508,
      "author_name": "aramisvesal",
      "author_url": "",
      "post_date": "03/31/2021 17:15:51",
      "content": "<p>Without Annotation: (LB: 0.916, CV: 0.932)</p>\n<p>I am still against ANNOTATING the test dataset. This totally changed the Kaggle competitions scheme.</p>\n<p>As <a href=\"https://www.kaggle.com/lijiaqi96\" target=\"_blank\">@lijiaqi96</a>  said, your LB and CV is so close, which is a plus point for you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1258603,
      "author_name": "rosuluc",
      "author_url": "",
      "post_date": "03/31/2021 18:41:12",
      "content": "<p>CV: 0.94<br>\nLB: 0.932</p>\n<p>(no hand labeling)</p>\n<p>working on ways to improve score on d488c759a and overall w/o hand labeling test set. it would be very misleading for cross validation otherwise. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1265057,
          "author_name": "anthonymarcz",
          "author_url": "",
          "post_date": "04/06/2021 15:36:40",
          "content": "<p>Did you use pseudo label on public test dataset or just train dataset ?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1265084,
          "author_name": "rosuluc",
          "author_url": "",
          "post_date": "04/06/2021 15:53:35",
          "content": "<p>nope, no pseudo on public test set</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1265721,
          "author_name": "anthonymarcz",
          "author_url": "",
          "post_date": "04/07/2021 06:51:21",
          "content": "<p>excellent, amazing CV&amp;LB👍</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1274107,
          "author_name": "southsakura",
          "author_url": "",
          "post_date": "04/15/2021 02:04:10",
          "content": "<p>did you use single fold or kfolds?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1276235,
          "author_name": "rosuluc",
          "author_url": "",
          "post_date": "04/17/2021 09:35:31",
          "content": "<p>single fold, no hand labels, no pseudo</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1261107,
      "author_name": "dragonpg2000",
      "author_url": "",
      "post_date": "04/02/2021 17:43:11",
      "content": "<p>What's your CV strategy? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1264230,
      "author_name": "southsakura",
      "author_url": "",
      "post_date": "04/06/2021 02:14:44",
      "content": "<p>CV:0.938 LB:0.924 TTAs 0.926 without hand labeling</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1268715,
      "author_name": "durbin164",
      "author_url": "",
      "post_date": "04/09/2021 17:18:12",
      "content": "<p>Hi, Excellent CV and LB. What image size did you use both for train and CV? which model did you use?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1270784,
      "author_name": "anthonymarcz",
      "author_url": "",
      "post_date": "04/12/2021 02:36:23",
      "content": "<p>My recent experiment (single model, five folds, no hand labeling and pseudo label)<br>\nCV: 0.928 (with best threshold)<br>\nLB: 0.933<br>\nvery heavy augmentations</p>",
      "votes": null,
      "replies": [
        {
          "id": 1270895,
          "author_name": "trooperog",
          "author_url": "",
          "post_date": "04/12/2021 06:21:29",
          "content": "<p>for how much epochs are you training ?? </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1274109,
          "author_name": "southsakura",
          "author_url": "",
          "post_date": "04/15/2021 02:08:01",
          "content": "<p>I'm trying kfolds to see its improvement. But I still struggled in the strategies.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1258011": "To make the LB score more reasonable, I want to create a thread to collect whether people annonate the test dataset to get the LB. \n\nHope this will help us better understand the LB score.\n\n===========================\n\nLB:0.924\nCV:0.923\n\nWITHOUT ANNONATION",
    "1258184": "TBH this is the closest LB and CV I've ever seen...",
    "1258508": "Without Annotation: (LB: 0.916, CV: 0.932)\n\nI am still against ANNOTATING the test dataset. This totally changed the Kaggle competitions scheme.\n\nAs @lijiaqi96  said, your LB and CV is so close, which is a plus point for you.",
    "1258603": "CV: 0.94\nLB: 0.932\n\n(no hand labeling)\n\nworking on ways to improve score on d488c759a and overall w/o hand labeling test set. it would be very misleading for cross validation otherwise.",
    "1261107": "What's your CV strategy?",
    "1264230": "CV:0.938 LB:0.924 TTAs 0.926 without hand labeling",
    "1265057": "Did you use pseudo label on public test dataset or just train dataset ?",
    "1265084": "nope, no pseudo on public test set",
    "1265721": "excellent, amazing CV&LB👍",
    "1268715": "Hi, Excellent CV and LB. What image size did you use both for train and CV? which model did you use?",
    "1270784": "My recent experiment (single model, five folds, no hand labeling and pseudo label)\nCV: 0.928 (with best threshold)\nLB: 0.933\nvery heavy augmentations",
    "1270895": "for how much epochs are you training ??",
    "1274107": "did you use single fold or kfolds?",
    "1274109": "I'm trying kfolds to see its improvement. But I still struggled in the strategies.",
    "1276235": "single fold, no hand labels, no pseudo"
  },
  "source": "meta"
}