{
  "id": 533586,
  "title": "Is there any method to judge if my tricks improve the model?",
  "url": "/competitions/rsna-2024-lumbar-spine-degenerative-classification/discussion/533586",
  "author_name": "",
  "post_date": "2024-09-12T01:49:37.008353600Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I am just a  beginner for kaggle.I wanna know is there any method to judge if my tricks work?If I restart to train my model with all data, it may cost lots of time.But if I just select some sample data to train,the result may be not so accurate.</p>",
  "messages": [
    {
      "id": "2986727",
      "postDate": "09/12/2024 01:49:37",
      "content": "<p>I am just a  beginner for kaggle.I wanna know is there any method to judge if my tricks work?If I restart to train my model with all data, it may cost lots of time.But if I just select some sample data to train,the result may be not so accurate.</p>",
      "rawMarkdown": "I am just a  beginner for kaggle.I wanna know is there any method to judge if my tricks work?If I restart to train my model with all data, it may cost lots of time.But if I just select some sample data to train,the result may be not so accurate.",
      "votes": null
    },
    {
      "id": "2986976",
      "postDate": "09/12/2024 08:52:09",
      "content": "<p>You can experiment with train/validation splits. And if you want, make a last full train with your optimized pipeline.</p>",
      "rawMarkdown": "You can experiment with train/validation splits. And if you want, make a last full train with your optimized pipeline.",
      "votes": null
    },
    {
      "id": "2987080",
      "postDate": "09/12/2024 11:43:44",
      "content": "<p>Try to compare speed of changing loss with previous solution on first epochs </p>\n<p>Try to use pretrained model from last good submission</p>\n<p>And as previously said you can change frequency of eval model while training (one time per 3 epoch) </p>\n<p>Generally speaking the less time you spend on validation of your idea the less accurate representation you get </p>",
      "rawMarkdown": "Try to compare speed of changing loss with previous solution on first epochs \n\nTry to use pretrained model from last good submission\n\nAnd as previously said you can change frequency of eval model while training (one time per 3 epoch) \n \nGenerally speaking the less time you spend on validation of your idea the less accurate representation you get",
      "votes": null
    },
    {
      "id": "2992905",
      "postDate": "09/19/2024 08:26:54",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/i2nfinit3y\" target=\"_blank\">@i2nfinit3y</a> ,<br>\nyou can cross validate models across folds.<br>\nsay you train 5-models by ignoring one of  5-splits each and then validate and see results on unseen split.<br>\nthis way you will get CV score which is i would say gold standard  for kaggle competitions.</p>\n<p>You can also check score on whole train dataset.</p>",
      "rawMarkdown": "Hey @i2nfinit3y ,\nyou can cross validate models across folds.\nsay you train 5-models by ignoring one of  5-splits each and then validate and see results on unseen split.\nthis way you will get CV score which is i would say gold standard  for kaggle competitions.\n\nYou can also check score on whole train dataset.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2986976,
      "author_name": "sacuscreed",
      "author_url": "",
      "post_date": "09/12/2024 08:52:09",
      "content": "<p>You can experiment with train/validation splits. And if you want, make a last full train with your optimized pipeline.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2987080,
      "author_name": "climbest",
      "author_url": "",
      "post_date": "09/12/2024 11:43:44",
      "content": "<p>Try to compare speed of changing loss with previous solution on first epochs </p>\n<p>Try to use pretrained model from last good submission</p>\n<p>And as previously said you can change frequency of eval model while training (one time per 3 epoch) </p>\n<p>Generally speaking the less time you spend on validation of your idea the less accurate representation you get </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2992905,
      "author_name": "rohitchaudhari25",
      "author_url": "",
      "post_date": "09/19/2024 08:26:54",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/i2nfinit3y\" target=\"_blank\">@i2nfinit3y</a> ,<br>\nyou can cross validate models across folds.<br>\nsay you train 5-models by ignoring one of  5-splits each and then validate and see results on unseen split.<br>\nthis way you will get CV score which is i would say gold standard  for kaggle competitions.</p>\n<p>You can also check score on whole train dataset.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2986727": "I am just a  beginner for kaggle.I wanna know is there any method to judge if my tricks work?If I restart to train my model with all data, it may cost lots of time.But if I just select some sample data to train,the result may be not so accurate.",
    "2986976": "You can experiment with train/validation splits. And if you want, make a last full train with your optimized pipeline.",
    "2987080": "Try to compare speed of changing loss with previous solution on first epochs \n\nTry to use pretrained model from last good submission\n\nAnd as previously said you can change frequency of eval model while training (one time per 3 epoch) \n \nGenerally speaking the less time you spend on validation of your idea the less accurate representation you get",
    "2992905": "Hey @i2nfinit3y ,\nyou can cross validate models across folds.\nsay you train 5-models by ignoring one of  5-splits each and then validate and see results on unseen split.\nthis way you will get CV score which is i would say gold standard  for kaggle competitions.\n\nYou can also check score on whole train dataset."
  },
  "source": "meta"
}