{
  "id": 465884,
  "title": "WIll there be Shakeup?",
  "url": "/competitions/blood-vessel-segmentation/discussion/465884",
  "author_name": "",
  "post_date": "2024-01-06T05:33:10.661069100Z",
  "votes": 5,
  "comment_count": 8,
  "views": 0,
  "content": "<p>Do you think this competition will have a shakeup?</p>\n<p>Thanks,</p>\n<p>Bo Peng</p>",
  "messages": [
    {
      "id": "2589128",
      "postDate": "01/06/2024 05:33:10",
      "content": "<p>Do you think this competition will have a shakeup?</p>\n<p>Thanks,</p>\n<p>Bo Peng</p>",
      "rawMarkdown": "Do you think this competition will have a shakeup?\n\nThanks,\n\nBo Peng",
      "votes": null
    },
    {
      "id": "2589176",
      "postDate": "01/06/2024 06:24:36",
      "content": "<p>NO if you knows how to make your model robust to resolution and insensitive to threshold.</p>\n<p>since there is only one private and one public test sample, you can do a lot of probing to determine the sensistivity of your  private prediction, e.g. against threshold, etc</p>",
      "rawMarkdown": "NO if you knows how to make your model robust to resolution and insensitive to threshold.\n\nsince there is only one private and one public test sample, you can do a lot of probing to determine the sensistivity of your  private prediction, e.g. against threshold, etc",
      "votes": null
    },
    {
      "id": "2589523",
      "postDate": "01/06/2024 13:15:21",
      "content": "<p>Since 200 people just copied a public notebook meant to overfit public LB there will necessarily be a large shakeup.<br>\nThe fact that both public and private sets are (probably) very different and that the metric is threshold sensitive are good indicators of a large potential shakeup.</p>",
      "rawMarkdown": "Since 200 people just copied a public notebook meant to overfit public LB there will necessarily be a large shakeup.\nThe fact that both public and private sets are (probably) very different and that the metric is threshold sensitive are good indicators of a large potential shakeup.",
      "votes": null
    },
    {
      "id": "2589636",
      "postDate": "01/06/2024 14:30:14",
      "content": "<p>Looking forward to reading your solution, I'm currently making my baseline and I am greatly benefiting from your topics, thank you for all your sharing and for giving people ideas to improve their approach and overall skills in ML. 😀</p>",
      "rawMarkdown": "Looking forward to reading your solution, I'm currently making my baseline and I am greatly benefiting from your topics, thank you for all your sharing and for giving people ideas to improve their approach and overall skills in ML. 😀",
      "votes": null
    },
    {
      "id": "2592641",
      "postDate": "01/08/2024 17:42:37",
      "content": "<p>how do you make a model insensitive to threshold?<br>\nstatistically, the model need be 'sure' in its predictions (i.e. .1 .009, .91, .89) so that it becomes insensitive to changes in threshold in some intermediate regions… and this might well mean overfitting to train data or public leaderboard badly.</p>",
      "rawMarkdown": "how do you make a model insensitive to threshold?\nstatistically, the model need be 'sure' in its predictions (i.e. .1 .009, .91, .89) so that it becomes insensitive to changes in threshold in some intermediate regions... and this might well mean overfitting to train data or public leaderboard badly.",
      "votes": null
    },
    {
      "id": "2592646",
      "postDate": "01/08/2024 17:46:18",
      "content": "<p>\"how do you make a model insensitive to threshold?\"</p>\n<p>your lb vs threshold (local and public) curve should be flat.</p>\n<p>hint:<br>\nyou can also measure no of lablled pixles vs threshold for private via probing which essentially is a proxy for lb score</p>",
      "rawMarkdown": "\"how do you make a model insensitive to threshold?\"\n\nyour lb vs threshold (local and public) curve should be flat.\n\nhint:\nyou can also measure no of lablled pixles vs threshold for private via probing which essentially is a proxy for lb score",
      "votes": null
    },
    {
      "id": "2592661",
      "postDate": "01/08/2024 17:58:13",
      "content": "<p>I get that and I agree with the notion.<br>\nBut practically this would mean that  'pixelwise' decisions do change 'minimally' with threshold, which is only possible (I think) when the model is 'confident' in its predictions. And that would eventually mean  a 'confident' model based on train data or public leaderboard results, where overfitting might be a higher threat if not carefully crafted. </p>",
      "rawMarkdown": "I get that and I agree with the notion.\nBut practically this would mean that  'pixelwise' decisions do change 'minimally' with threshold, which is only possible (I think) when the model is 'confident' in its predictions. And that would eventually mean  a 'confident' model based on train data or public leaderboard results, where overfitting might be a higher threat if not carefully crafted.",
      "votes": null
    },
    {
      "id": "2592839",
      "postDate": "01/08/2024 20:44:34",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> I appreciate that. Yea you really helped me a couple years ago on this competition: <a href=\"https://www.kaggle.com/competitions/nfl-impact-detection\" target=\"_blank\">https://www.kaggle.com/competitions/nfl-impact-detection</a> when I was on a company team. It's great to see you throughout the years as we progress this life. Thanks</p>",
      "rawMarkdown": "Thanks @hengck23 I appreciate that. Yea you really helped me a couple years ago on this competition: https://www.kaggle.com/competitions/nfl-impact-detection when I was on a company team. It's great to see you throughout the years as we progress this life. Thanks",
      "votes": null
    },
    {
      "id": "2593501",
      "postDate": "01/09/2024 08:58:59",
      "content": "<p>yes public notebook has a lot of subs</p>",
      "rawMarkdown": "yes public notebook has a lot of subs",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2589176,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "01/06/2024 06:24:36",
      "content": "<p>NO if you knows how to make your model robust to resolution and insensitive to threshold.</p>\n<p>since there is only one private and one public test sample, you can do a lot of probing to determine the sensistivity of your  private prediction, e.g. against threshold, etc</p>",
      "votes": null,
      "replies": [
        {
          "id": 2589636,
          "author_name": "janmpia",
          "author_url": "",
          "post_date": "01/06/2024 14:30:14",
          "content": "<p>Looking forward to reading your solution, I'm currently making my baseline and I am greatly benefiting from your topics, thank you for all your sharing and for giving people ideas to improve their approach and overall skills in ML. 😀</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2592641,
          "author_name": "abdulkadirguner",
          "author_url": "",
          "post_date": "01/08/2024 17:42:37",
          "content": "<p>how do you make a model insensitive to threshold?<br>\nstatistically, the model need be 'sure' in its predictions (i.e. .1 .009, .91, .89) so that it becomes insensitive to changes in threshold in some intermediate regions… and this might well mean overfitting to train data or public leaderboard badly.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2592646,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "01/08/2024 17:46:18",
              "content": "<p>\"how do you make a model insensitive to threshold?\"</p>\n<p>your lb vs threshold (local and public) curve should be flat.</p>\n<p>hint:<br>\nyou can also measure no of lablled pixles vs threshold for private via probing which essentially is a proxy for lb score</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2592661,
                  "author_name": "abdulkadirguner",
                  "author_url": "",
                  "post_date": "01/08/2024 17:58:13",
                  "content": "<p>I get that and I agree with the notion.<br>\nBut practically this would mean that  'pixelwise' decisions do change 'minimally' with threshold, which is only possible (I think) when the model is 'confident' in its predictions. And that would eventually mean  a 'confident' model based on train data or public leaderboard results, where overfitting might be a higher threat if not carefully crafted. </p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        },
        {
          "id": 2592839,
          "author_name": "bopengiowa",
          "author_url": "",
          "post_date": "01/08/2024 20:44:34",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> I appreciate that. Yea you really helped me a couple years ago on this competition: <a href=\"https://www.kaggle.com/competitions/nfl-impact-detection\" target=\"_blank\">https://www.kaggle.com/competitions/nfl-impact-detection</a> when I was on a company team. It's great to see you throughout the years as we progress this life. Thanks</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2589523,
      "author_name": "optimo",
      "author_url": "",
      "post_date": "01/06/2024 13:15:21",
      "content": "<p>Since 200 people just copied a public notebook meant to overfit public LB there will necessarily be a large shakeup.<br>\nThe fact that both public and private sets are (probably) very different and that the metric is threshold sensitive are good indicators of a large potential shakeup.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2593501,
      "author_name": "manav2805",
      "author_url": "",
      "post_date": "01/09/2024 08:58:59",
      "content": "<p>yes public notebook has a lot of subs</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2589128": "Do you think this competition will have a shakeup?\n\nThanks,\n\nBo Peng",
    "2589176": "NO if you knows how to make your model robust to resolution and insensitive to threshold.\n\nsince there is only one private and one public test sample, you can do a lot of probing to determine the sensistivity of your  private prediction, e.g. against threshold, etc",
    "2589523": "Since 200 people just copied a public notebook meant to overfit public LB there will necessarily be a large shakeup.\nThe fact that both public and private sets are (probably) very different and that the metric is threshold sensitive are good indicators of a large potential shakeup.",
    "2589636": "Looking forward to reading your solution, I'm currently making my baseline and I am greatly benefiting from your topics, thank you for all your sharing and for giving people ideas to improve their approach and overall skills in ML. 😀",
    "2592641": "how do you make a model insensitive to threshold?\nstatistically, the model need be 'sure' in its predictions (i.e. .1 .009, .91, .89) so that it becomes insensitive to changes in threshold in some intermediate regions... and this might well mean overfitting to train data or public leaderboard badly.",
    "2592646": "\"how do you make a model insensitive to threshold?\"\n\nyour lb vs threshold (local and public) curve should be flat.\n\nhint:\nyou can also measure no of lablled pixles vs threshold for private via probing which essentially is a proxy for lb score",
    "2592661": "I get that and I agree with the notion.\nBut practically this would mean that  'pixelwise' decisions do change 'minimally' with threshold, which is only possible (I think) when the model is 'confident' in its predictions. And that would eventually mean  a 'confident' model based on train data or public leaderboard results, where overfitting might be a higher threat if not carefully crafted.",
    "2592839": "Thanks @hengck23 I appreciate that. Yea you really helped me a couple years ago on this competition: https://www.kaggle.com/competitions/nfl-impact-detection when I was on a company team. It's great to see you throughout the years as we progress this life. Thanks",
    "2593501": "yes public notebook has a lot of subs"
  },
  "source": "meta"
}