{
  "id": 551625,
  "title": "Is my optimization flawed?",
  "url": "/competitions/child-mind-institute-problematic-internet-use/discussion/551625",
  "author_name": "",
  "post_date": "2024-12-14T11:44:59.965555400Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hey,<br>\nthis might be boiled down to the question, if you trust CV or LB.<br>\nMy models are quite stable with different seeds. But if I push model weights or threshold optimization (from ~0.465 CV) to 0.479 CV, my LB score goes down from 0.467 to 0.453.<br>\nDoes anyone else have this problem?<br>\nWhat do you think is the issue?<br>\nIs it just unlucky distribution of LB or overfitting on CV?<br>\nSince the train and LB df have the Same size, the optimal qwk thresholds could easily change. But how to choose qwk thresholds or model weights then?</p>",
  "messages": [
    {
      "id": "3071838",
      "postDate": "12/14/2024 11:44:59",
      "content": "<p>Hey,<br>\nthis might be boiled down to the question, if you trust CV or LB.<br>\nMy models are quite stable with different seeds. But if I push model weights or threshold optimization (from ~0.465 CV) to 0.479 CV, my LB score goes down from 0.467 to 0.453.<br>\nDoes anyone else have this problem?<br>\nWhat do you think is the issue?<br>\nIs it just unlucky distribution of LB or overfitting on CV?<br>\nSince the train and LB df have the Same size, the optimal qwk thresholds could easily change. But how to choose qwk thresholds or model weights then?</p>",
      "rawMarkdown": "Hey,\nthis might be boiled down to the question, if you trust CV or LB.\nMy models are quite stable with different seeds. But if I push model weights or threshold optimization (from ~0.465 CV) to 0.479 CV, my LB score goes down from 0.467 to 0.453.\nDoes anyone else have this problem?\nWhat do you think is the issue?\nIs it just unlucky distribution of LB or overfitting on CV?\nSince the train and LB df have the Same size, the optimal qwk thresholds could easily change. But how to choose qwk thresholds or model weights then?",
      "votes": null
    },
    {
      "id": "3071900",
      "postDate": "12/14/2024 13:29:43",
      "content": "<p>I choose not to use qwk thresholds, cuz it didn't perform well in my local cv.</p>",
      "rawMarkdown": "I choose not to use qwk thresholds, cuz it didn't perform well in my local cv.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3071900,
      "author_name": "shanzhong8",
      "author_url": "",
      "post_date": "12/14/2024 13:29:43",
      "content": "<p>I choose not to use qwk thresholds, cuz it didn't perform well in my local cv.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3071838": "Hey,\nthis might be boiled down to the question, if you trust CV or LB.\nMy models are quite stable with different seeds. But if I push model weights or threshold optimization (from ~0.465 CV) to 0.479 CV, my LB score goes down from 0.467 to 0.453.\nDoes anyone else have this problem?\nWhat do you think is the issue?\nIs it just unlucky distribution of LB or overfitting on CV?\nSince the train and LB df have the Same size, the optimal qwk thresholds could easily change. But how to choose qwk thresholds or model weights then?",
    "3071900": "I choose not to use qwk thresholds, cuz it didn't perform well in my local cv."
  },
  "source": "meta"
}