{
  "id": 169136,
  "title": "How to get the CV score on ensemble models?",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/169136",
  "author_name": "",
  "post_date": "2020-07-23T02:45:42.009103700Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi all, I am a beginner of Kaggle and I am confused about how to get the CV score on ensemble models. I have used blending to improve my LB rank but I concerned about the issue of overfitting. Would you mind telling me how to done that?</p>\n\n<p>Thanks in advance for your help!</p>",
  "messages": [
    {
      "id": "940557",
      "postDate": "07/23/2020 02:45:42",
      "content": "<p>Hi all, I am a beginner of Kaggle and I am confused about how to get the CV score on ensemble models. I have used blending to improve my LB rank but I concerned about the issue of overfitting. Would you mind telling me how to done that?</p>\n\n<p>Thanks in advance for your help!</p>",
      "rawMarkdown": "Hi all, I am a beginner of Kaggle and I am confused about how to get the CV score on ensemble models. I have used blending to improve my LB rank but I concerned about the issue of overfitting. Would you mind telling me how to done that?\n\nThanks in advance for your help!",
      "votes": null
    },
    {
      "id": "940583",
      "postDate": "07/23/2020 03:21:58",
      "content": "<p>suppose you have 3 model <strong>M1</strong> and <strong>M2</strong> and <strong>M3</strong>\n1) take  <strong>OOF</strong> and <strong>submission</strong> file for all models\n<code>[calculate roc_auc_score for all models with help of Train_target and OOF_target]</code>\n2) if you want to ensemble model with weights <strong>W1</strong> and <strong>W2</strong> and <strong>W3</strong>\n<code>\nOOF_ensemble  = W1 * OOF_1  + W2 * OOF_2 + W3 * OOF_3\nSubmission_ensemble = W1 * Submission_1 + W2 * Submission_2 + W3 * Submission_3\nroc_auc_score_ensemble = roc_auc_score( OOF_ensemble  , Train_target )\n</code></p>",
      "rawMarkdown": "suppose you have 3 model **M1** and **M2** and **M3**\n1) take  **OOF** and **submission** file for all models\n`[calculate roc_auc_score for all models with help of Train_target and OOF_target]`\n2) if you want to ensemble model with weights **W1** and **W2** and **W3**\n```\nOOF_ensemble  = W1 * OOF_1  + W2 * OOF_2 + W3 * OOF_3\nSubmission_ensemble = W1 * Submission_1 + W2 * Submission_2 + W3 * Submission_3\nroc_auc_score_ensemble = roc_auc_score( OOF_ensemble  , Train_target )\n```",
      "votes": null
    },
    {
      "id": "940632",
      "postDate": "07/23/2020 04:05:26",
      "content": "<p>Thanks for your kind help <a href=\"/vatsalparsaniya\">@vatsalparsaniya</a>!</p>",
      "rawMarkdown": "Thanks for your kind help @vatsalparsaniya!",
      "votes": null
    },
    {
      "id": "941288",
      "postDate": "07/23/2020 06:50:58",
      "content": "<p>One more good point to avoid overfitting is to submit average from the same model, trained on different folds</p>",
      "rawMarkdown": "One more good point to avoid overfitting is to submit average from the same model, trained on different folds",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 940583,
      "author_name": "vatsalparsaniya",
      "author_url": "",
      "post_date": "07/23/2020 03:21:58",
      "content": "<p>suppose you have 3 model <strong>M1</strong> and <strong>M2</strong> and <strong>M3</strong>\n1) take  <strong>OOF</strong> and <strong>submission</strong> file for all models\n<code>[calculate roc_auc_score for all models with help of Train_target and OOF_target]</code>\n2) if you want to ensemble model with weights <strong>W1</strong> and <strong>W2</strong> and <strong>W3</strong>\n<code>\nOOF_ensemble  = W1 * OOF_1  + W2 * OOF_2 + W3 * OOF_3\nSubmission_ensemble = W1 * Submission_1 + W2 * Submission_2 + W3 * Submission_3\nroc_auc_score_ensemble = roc_auc_score( OOF_ensemble  , Train_target )\n</code></p>",
      "votes": null,
      "replies": [
        {
          "id": 940632,
          "author_name": "vincentren1997",
          "author_url": "",
          "post_date": "07/23/2020 04:05:26",
          "content": "<p>Thanks for your kind help <a href=\"/vatsalparsaniya\">@vatsalparsaniya</a>!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 941288,
      "author_name": "vladimirsydor",
      "author_url": "",
      "post_date": "07/23/2020 06:50:58",
      "content": "<p>One more good point to avoid overfitting is to submit average from the same model, trained on different folds</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "940557": "Hi all, I am a beginner of Kaggle and I am confused about how to get the CV score on ensemble models. I have used blending to improve my LB rank but I concerned about the issue of overfitting. Would you mind telling me how to done that?\n\nThanks in advance for your help!",
    "940583": "suppose you have 3 model **M1** and **M2** and **M3**\n1) take  **OOF** and **submission** file for all models\n`[calculate roc_auc_score for all models with help of Train_target and OOF_target]`\n2) if you want to ensemble model with weights **W1** and **W2** and **W3**\n```\nOOF_ensemble  = W1 * OOF_1  + W2 * OOF_2 + W3 * OOF_3\nSubmission_ensemble = W1 * Submission_1 + W2 * Submission_2 + W3 * Submission_3\nroc_auc_score_ensemble = roc_auc_score( OOF_ensemble  , Train_target )\n```",
    "940632": "Thanks for your kind help @vatsalparsaniya!",
    "941288": "One more good point to avoid overfitting is to submit average from the same model, trained on different folds"
  },
  "source": "meta"
}