{
  "id": 180565,
  "title": "Estimating performance from folds",
  "url": "/competitions/osic-pulmonary-fibrosis-progression/discussion/180565",
  "author_name": "Sathvik Bhaskarpandit",
  "post_date": "2020-09-05T14:41:32.109000",
  "votes": 6,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I've used K Fold cross validation and split my data into 5 folds. Then I've calculated the validation score for each fold, with a positive sign instead. What's a good way to measure the performance of my model from each fold's score? Normally I would do <br>\noverall score =  (mean of folds) + (standard deviation of folds) and find the model with the least value but I'm finding that the overall score and public LB score are completely unrelated, that is, a good overall score doesn't imply a good public LB score</p>\n<p>Does anyone have any better idea to get a good overall score from each fold's score, that is closely related to the public LB score?</p>",
  "messages": [
    {
      "id": 999295,
      "postDate": "2020-09-05T14:41:32.110Z",
      "content": "<p>I've used K Fold cross validation and split my data into 5 folds. Then I've calculated the validation score for each fold, with a positive sign instead. What's a good way to measure the performance of my model from each fold's score? Normally I would do <br>\noverall score =  (mean of folds) + (standard deviation of folds) and find the model with the least value but I'm finding that the overall score and public LB score are completely unrelated, that is, a good overall score doesn't imply a good public LB score</p>\n<p>Does anyone have any better idea to get a good overall score from each fold's score, that is closely related to the public LB score?</p>",
      "rawMarkdown": "I've used K Fold cross validation and split my data into 5 folds. Then I've calculated the validation score for each fold, with a positive sign instead. What's a good way to measure the performance of my model from each fold's score? Normally I would do \noverall score =  (mean of folds) + (standard deviation of folds) and find the model with the least value but I'm finding that the overall score and public LB score are completely unrelated, that is, a good overall score doesn't imply a good public LB score\n\nDoes anyone have any better idea to get a good overall score from each fold's score, that is closely related to the public LB score?",
      "votes": 6
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "999295": "I've used K Fold cross validation and split my data into 5 folds. Then I've calculated the validation score for each fold, with a positive sign instead. What's a good way to measure the performance of my model from each fold's score? Normally I would do \noverall score =  (mean of folds) + (standard deviation of folds) and find the model with the least value but I'm finding that the overall score and public LB score are completely unrelated, that is, a good overall score doesn't imply a good public LB score\n\nDoes anyone have any better idea to get a good overall score from each fold's score, that is closely related to the public LB score?"
  }
}