{
  "id": 93382,
  "title": "how could we find the best threshold for this problem?",
  "url": "/competitions/imet-2019-fgvc6/discussion/93382",
  "author_name": "",
  "post_date": "2019-05-26T13:14:22.729289400Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>currently I do 5 fold cross validation, and I found out that the best threshold is different for each folds.\nright now I'm just predicting with threshold of 0.1.\nany advice would be apprecieated. thanks!</p>",
  "messages": [
    {
      "id": "537212",
      "postDate": "05/26/2019 13:14:22",
      "content": "<p>currently I do 5 fold cross validation, and I found out that the best threshold is different for each folds.\nright now I'm just predicting with threshold of 0.1.\nany advice would be apprecieated. thanks!</p>",
      "rawMarkdown": "currently I do 5 fold cross validation, and I found out that the best threshold is different for each folds.\nright now I'm just predicting with threshold of 0.1.\nany advice would be apprecieated. thanks!",
      "votes": null
    },
    {
      "id": "537406",
      "postDate": "05/27/2019 00:39:53",
      "content": "<p>What's wrong with using a different threshold or each fold?</p>",
      "rawMarkdown": "What's wrong with using a different threshold or each fold?",
      "votes": null
    },
    {
      "id": "537509",
      "postDate": "05/27/2019 06:34:54",
      "content": "<p>I splited the data into 5 folds then I cross validate using different folds\nand I found that each fold has different best threshold.\nfor example fold1 -&gt; threshold 0.1, fold2 -&gt; threshold 0.15 -&gt; fold3 -&gt; threshold 0.2 etc</p>",
      "rawMarkdown": "I splited the data into 5 folds then I cross validate using different folds\nand I found that each fold has different best threshold.\nfor example fold1 -&gt; threshold 0.1, fold2 -&gt; threshold 0.15 -&gt; fold3 -&gt; threshold 0.2 etc",
      "votes": null
    },
    {
      "id": "537950",
      "postDate": "05/27/2019 22:40:23",
      "content": "<p>so what's stopping you from using the different thresholds when you perform inference with the different models? You anyway have to perform inference with the 5 different models and then average the predictions afterwards.</p>",
      "rawMarkdown": "so what's stopping you from using the different thresholds when you perform inference with the different models? You anyway have to perform inference with the 5 different models and then average the predictions afterwards.",
      "votes": null
    },
    {
      "id": "538308",
      "postDate": "05/28/2019 11:46:27",
      "content": "<p>That's alright. For me average threshold over folds works best but you can also try to find a global threshold over whole train oof predictions.</p>",
      "rawMarkdown": "That's alright. For me average threshold over folds works best but you can also try to find a global threshold over whole train oof predictions.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 537406,
      "author_name": "tanlikesmath",
      "author_url": "",
      "post_date": "05/27/2019 00:39:53",
      "content": "<p>What's wrong with using a different threshold or each fold?</p>",
      "votes": null,
      "replies": [
        {
          "id": 537509,
          "author_name": "chrisyang7",
          "author_url": "",
          "post_date": "05/27/2019 06:34:54",
          "content": "<p>I splited the data into 5 folds then I cross validate using different folds\nand I found that each fold has different best threshold.\nfor example fold1 -&gt; threshold 0.1, fold2 -&gt; threshold 0.15 -&gt; fold3 -&gt; threshold 0.2 etc</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 537950,
          "author_name": "tanlikesmath",
          "author_url": "",
          "post_date": "05/27/2019 22:40:23",
          "content": "<p>so what's stopping you from using the different thresholds when you perform inference with the different models? You anyway have to perform inference with the 5 different models and then average the predictions afterwards.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 538308,
      "author_name": "yaroshevskiy",
      "author_url": "",
      "post_date": "05/28/2019 11:46:27",
      "content": "<p>That's alright. For me average threshold over folds works best but you can also try to find a global threshold over whole train oof predictions.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "537212": "currently I do 5 fold cross validation, and I found out that the best threshold is different for each folds.\nright now I'm just predicting with threshold of 0.1.\nany advice would be apprecieated. thanks!",
    "537406": "What's wrong with using a different threshold or each fold?",
    "537509": "I splited the data into 5 folds then I cross validate using different folds\nand I found that each fold has different best threshold.\nfor example fold1 -&gt; threshold 0.1, fold2 -&gt; threshold 0.15 -&gt; fold3 -&gt; threshold 0.2 etc",
    "537950": "so what's stopping you from using the different thresholds when you perform inference with the different models? You anyway have to perform inference with the 5 different models and then average the predictions afterwards.",
    "538308": "That's alright. For me average threshold over folds works best but you can also try to find a global threshold over whole train oof predictions."
  },
  "source": "meta"
}