{
  "id": 475328,
  "title": "Correct threshold is all you need🤯",
  "url": "/competitions/blood-vessel-segmentation/discussion/475328",
  "author_name": "",
  "post_date": "2024-02-08T02:17:03.643931100Z",
  "votes": 5,
  "comment_count": 2,
  "views": 0,
  "content": "<p>We submitted some experiments with different thresholds after the competition was over.</p>\n<table>\n<thead>\n<tr>\n<th>threshold</th>\n<th>public score</th>\n<th>private score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.5</td>\n<td><strong>0.895</strong></td>\n<td>0.593</td>\n</tr>\n<tr>\n<td>0.3</td>\n<td>0.887</td>\n<td>0.641</td>\n</tr>\n<tr>\n<td>0.2</td>\n<td>0.869</td>\n<td>0.667</td>\n</tr>\n<tr>\n<td>0.1</td>\n<td>0.824</td>\n<td>0.695</td>\n</tr>\n<tr>\n<td>0.05</td>\n<td>0.763</td>\n<td><strong>0.702</strong></td>\n</tr>\n</tbody>\n</table>\n<p>　<br>\n　<br>\nlocal cv tended to be similar to public score.</p>\n<p>How could we determine the correct threshold?</p>",
  "messages": [
    {
      "id": "2642195",
      "postDate": "02/08/2024 02:17:03",
      "content": "<p>We submitted some experiments with different thresholds after the competition was over.</p>\n<table>\n<thead>\n<tr>\n<th>threshold</th>\n<th>public score</th>\n<th>private score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.5</td>\n<td><strong>0.895</strong></td>\n<td>0.593</td>\n</tr>\n<tr>\n<td>0.3</td>\n<td>0.887</td>\n<td>0.641</td>\n</tr>\n<tr>\n<td>0.2</td>\n<td>0.869</td>\n<td>0.667</td>\n</tr>\n<tr>\n<td>0.1</td>\n<td>0.824</td>\n<td>0.695</td>\n</tr>\n<tr>\n<td>0.05</td>\n<td>0.763</td>\n<td><strong>0.702</strong></td>\n</tr>\n</tbody>\n</table>\n<p>　<br>\n　<br>\nlocal cv tended to be similar to public score.</p>\n<p>How could we determine the correct threshold?</p>",
      "rawMarkdown": "We submitted some experiments with different thresholds after the competition was over.\n\n| threshold | public score | private score|\n| --- | --- | --- |\n| 0.5 | **0.895** |0.593 |\n| 0.3 | 0.887 |0.641 |\n| 0.2 | 0.869 |0.667 |\n| 0.1 | 0.824 |0.695 |\n| 0.05 | 0.763 |**0.702** |\n\n　\n　\nlocal cv tended to be similar to public score.\n\nHow could we determine the correct threshold?",
      "votes": null
    },
    {
      "id": "2642274",
      "postDate": "02/08/2024 04:18:26",
      "content": "<p>yes. you can apply correct threshold to get better results.<br>\nbut the top private scores in the range of privatelb 0.71+ are not just threshold tricks.<br>\ni am still waiting for the solution writeup.</p>\n<hr>\n<p>How could we determine the correct threshold?</p>\n<ul>\n<li>we should have \"imgained\" and \"confirmed\" how the private test data looks like in the model development phrase.</li>\n</ul>\n<p>the huge gap in threshold/score between private and public are not results in over-fitting.<br>\nthey are results of either differenet input or annotation (domain shift)</p>",
      "rawMarkdown": "yes. you can apply correct threshold to get better results.\nbut the top private scores in the range of privatelb 0.71+ are not just threshold tricks.\ni am still waiting for the solution writeup.\n\n---\n\nHow could we determine the correct threshold?\n- we should have \"imgained\" and \"confirmed\" how the private test data looks like in the model development phrase.\n\nthe huge gap in threshold/score between private and public are not results in over-fitting.\nthey are results of either differenet input or annotation (domain shift)",
      "votes": null
    },
    {
      "id": "2642480",
      "postDate": "02/08/2024 07:08:49",
      "content": "<p>Yes. I am particularly interested in the 1st place method, too.<br>\nThey are 1st place in both public and private, so there must be some secret to their success.</p>\n<p>they are results of either differenet input or annotation (domain shift)</p>\n<p>If the inputs were a different looking input, there are ways to deal with it, but I feel that dealing with the shakiness of annotations is now a lottery.<br>\nI don't know the truth though. . .</p>",
      "rawMarkdown": "Yes. I am particularly interested in the 1st place method, too.\nThey are 1st place in both public and private, so there must be some secret to their success.\n\nthey are results of either differenet input or annotation (domain shift)\n\nIf the inputs were a different looking input, there are ways to deal with it, but I feel that dealing with the shakiness of annotations is now a lottery.\nI don't know the truth though. . .",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2642274,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/08/2024 04:18:26",
      "content": "<p>yes. you can apply correct threshold to get better results.<br>\nbut the top private scores in the range of privatelb 0.71+ are not just threshold tricks.<br>\ni am still waiting for the solution writeup.</p>\n<hr>\n<p>How could we determine the correct threshold?</p>\n<ul>\n<li>we should have \"imgained\" and \"confirmed\" how the private test data looks like in the model development phrase.</li>\n</ul>\n<p>the huge gap in threshold/score between private and public are not results in over-fitting.<br>\nthey are results of either differenet input or annotation (domain shift)</p>",
      "votes": null,
      "replies": [
        {
          "id": 2642480,
          "author_name": "qtakka",
          "author_url": "",
          "post_date": "02/08/2024 07:08:49",
          "content": "<p>Yes. I am particularly interested in the 1st place method, too.<br>\nThey are 1st place in both public and private, so there must be some secret to their success.</p>\n<p>they are results of either differenet input or annotation (domain shift)</p>\n<p>If the inputs were a different looking input, there are ways to deal with it, but I feel that dealing with the shakiness of annotations is now a lottery.<br>\nI don't know the truth though. . .</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2642195": "We submitted some experiments with different thresholds after the competition was over.\n\n| threshold | public score | private score|\n| --- | --- | --- |\n| 0.5 | **0.895** |0.593 |\n| 0.3 | 0.887 |0.641 |\n| 0.2 | 0.869 |0.667 |\n| 0.1 | 0.824 |0.695 |\n| 0.05 | 0.763 |**0.702** |\n\n　\n　\nlocal cv tended to be similar to public score.\n\nHow could we determine the correct threshold?",
    "2642274": "yes. you can apply correct threshold to get better results.\nbut the top private scores in the range of privatelb 0.71+ are not just threshold tricks.\ni am still waiting for the solution writeup.\n\n---\n\nHow could we determine the correct threshold?\n- we should have \"imgained\" and \"confirmed\" how the private test data looks like in the model development phrase.\n\nthe huge gap in threshold/score between private and public are not results in over-fitting.\nthey are results of either differenet input or annotation (domain shift)",
    "2642480": "Yes. I am particularly interested in the 1st place method, too.\nThey are 1st place in both public and private, so there must be some secret to their success.\n\nthey are results of either differenet input or annotation (domain shift)\n\nIf the inputs were a different looking input, there are ways to deal with it, but I feel that dealing with the shakiness of annotations is now a lottery.\nI don't know the truth though. . ."
  },
  "source": "meta"
}