{
  "id": 498336,
  "title": "some tips to speed up your experiments",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/498336",
  "author_name": "",
  "post_date": "2024-04-27T21:10:51.183896100Z",
  "votes": 17,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I've read in many posts that training LGBM models takes over 30 minutes on a GPU for this competition. While that might be okay for final submissions, it feels too lengthy for experiments.</p>\n<p>There are a few tricks to reduce experiment times:</p>\n<ul>\n<li><p>Reduce the number of folds: With the large dataset, it’s a good practice to decrease the number of folds to 3. The results might be slightly lower, but it’s sufficient if your goal is to test ideas. Comparing new results with the baseline will still be effective, and you’ll achieve a 1.6x speed-up.</p></li>\n<li><p>Increase the learning rate and decrease the number of iterations: This speeds up training and allows you to quickly check improvements against the baseline.</p></li>\n<li><p>Reduce features: Instead of combining all features in each experiment, consider using smaller groups. For instance, training with just 20-50 features might take less than a minute on a CPU.</p></li>\n</ul>\n<p>These approaches can help you conduct many experiments in a day. Once you identify a promising direction, you can reintegrate all features, use a higher number of folds, and fine-tune parameters for more comprehensive training.</p>\n<p>These tips aren't particularly novel, but they've helped me avoid lengthy waits during model training.<br>\nHope it helps somebody else.</p>",
  "messages": [
    {
      "id": "2779851",
      "postDate": "04/27/2024 21:10:51",
      "content": "<p>I've read in many posts that training LGBM models takes over 30 minutes on a GPU for this competition. While that might be okay for final submissions, it feels too lengthy for experiments.</p>\n<p>There are a few tricks to reduce experiment times:</p>\n<ul>\n<li><p>Reduce the number of folds: With the large dataset, it’s a good practice to decrease the number of folds to 3. The results might be slightly lower, but it’s sufficient if your goal is to test ideas. Comparing new results with the baseline will still be effective, and you’ll achieve a 1.6x speed-up.</p></li>\n<li><p>Increase the learning rate and decrease the number of iterations: This speeds up training and allows you to quickly check improvements against the baseline.</p></li>\n<li><p>Reduce features: Instead of combining all features in each experiment, consider using smaller groups. For instance, training with just 20-50 features might take less than a minute on a CPU.</p></li>\n</ul>\n<p>These approaches can help you conduct many experiments in a day. Once you identify a promising direction, you can reintegrate all features, use a higher number of folds, and fine-tune parameters for more comprehensive training.</p>\n<p>These tips aren't particularly novel, but they've helped me avoid lengthy waits during model training.<br>\nHope it helps somebody else.</p>",
      "rawMarkdown": "I've read in many posts that training LGBM models takes over 30 minutes on a GPU for this competition. While that might be okay for final submissions, it feels too lengthy for experiments.\n\nThere are a few tricks to reduce experiment times:\n\n- Reduce the number of folds: With the large dataset, it’s a good practice to decrease the number of folds to 3. The results might be slightly lower, but it’s sufficient if your goal is to test ideas. Comparing new results with the baseline will still be effective, and you’ll achieve a 1.6x speed-up.\n\n- Increase the learning rate and decrease the number of iterations: This speeds up training and allows you to quickly check improvements against the baseline.\n\n- Reduce features: Instead of combining all features in each experiment, consider using smaller groups. For instance, training with just 20-50 features might take less than a minute on a CPU.\n\nThese approaches can help you conduct many experiments in a day. Once you identify a promising direction, you can reintegrate all features, use a higher number of folds, and fine-tune parameters for more comprehensive training.\n\nThese tips aren't particularly novel, but they've helped me avoid lengthy waits during model training.\nHope it helps somebody else.",
      "votes": null
    },
    {
      "id": "2780369",
      "postDate": "04/28/2024 06:48:18",
      "content": "<p>Thanks for the useful tips. I also sometimes use inference training to avoid wasting GPU quotas and to check only some hypotheses when full training is very time and resource consuming.</p>",
      "rawMarkdown": "Thanks for the useful tips. I also sometimes use inference training to avoid wasting GPU quotas and to check only some hypotheses when full training is very time and resource consuming.",
      "votes": null
    },
    {
      "id": "2783749",
      "postDate": "04/29/2024 23:43:57",
      "content": "<p>Thanks for the tips. </p>",
      "rawMarkdown": "Thanks for the tips.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2780369,
      "author_name": "andreynesterov",
      "author_url": "",
      "post_date": "04/28/2024 06:48:18",
      "content": "<p>Thanks for the useful tips. I also sometimes use inference training to avoid wasting GPU quotas and to check only some hypotheses when full training is very time and resource consuming.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2783749,
      "author_name": "majurranvimalan",
      "author_url": "",
      "post_date": "04/29/2024 23:43:57",
      "content": "<p>Thanks for the tips. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2779851": "I've read in many posts that training LGBM models takes over 30 minutes on a GPU for this competition. While that might be okay for final submissions, it feels too lengthy for experiments.\n\nThere are a few tricks to reduce experiment times:\n\n- Reduce the number of folds: With the large dataset, it’s a good practice to decrease the number of folds to 3. The results might be slightly lower, but it’s sufficient if your goal is to test ideas. Comparing new results with the baseline will still be effective, and you’ll achieve a 1.6x speed-up.\n\n- Increase the learning rate and decrease the number of iterations: This speeds up training and allows you to quickly check improvements against the baseline.\n\n- Reduce features: Instead of combining all features in each experiment, consider using smaller groups. For instance, training with just 20-50 features might take less than a minute on a CPU.\n\nThese approaches can help you conduct many experiments in a day. Once you identify a promising direction, you can reintegrate all features, use a higher number of folds, and fine-tune parameters for more comprehensive training.\n\nThese tips aren't particularly novel, but they've helped me avoid lengthy waits during model training.\nHope it helps somebody else.",
    "2780369": "Thanks for the useful tips. I also sometimes use inference training to avoid wasting GPU quotas and to check only some hypotheses when full training is very time and resource consuming.",
    "2783749": "Thanks for the tips."
  },
  "source": "meta"
}