{ "id": 502922, "title": "Cross-validation training for the GPU poor", "url": "/competitions/birdclef-2024/discussion/502922", "author_name": "", "post_date": "2024-05-15T10:42:36.767954400Z", "votes": 1, "comment_count": 5, "views": 0, "content": "

I am starting a thread to understand how participants here manage to perform cross-validation (5 to 10 fold) training in resource-constrained environments. What techniques can be used?

", "messages": [ { "id": "2814485", "postDate": "05/15/2024 10:42:36", "content": "

I am starting a thread to understand how participants here manage to perform cross-validation (5 to 10 fold) training in resource-constrained environments. What techniques can be used?

", "rawMarkdown": "I am starting a thread to understand how participants here manage to perform cross-validation (5 to 10 fold) training in resource-constrained environments. What techniques can be used?", "votes": null }, { "id": "2814515", "postDate": "05/15/2024 11:21:51", "content": "

You can speed up training a lot pre-loading spectrograms into a pickle file, so they aren’t created during training. Also, you could define say 10 folds but stop training after say 3.

", "rawMarkdown": "You can speed up training a lot pre-loading spectrograms into a pickle file, so they aren’t created during training. Also, you could define say 10 folds but stop training after say 3.", "votes": null }, { "id": "2816579", "postDate": "05/16/2024 12:04:53", "content": "

Thanks, are you suggesting finalizing hyperparameters based on a smaller number of epochs for cross-validation and then training for longer epochs on the entire dataset?
\nLet me know if you meant something else.

", "rawMarkdown": "Thanks, are you suggesting finalizing hyperparameters based on a smaller number of epochs for cross-validation and then training for longer epochs on the entire dataset?\nLet me know if you meant something else.", "votes": null }, { "id": "2816678", "postDate": "05/16/2024 13:24:59", "content": "

I meant like in this notebook: https://www.kaggle.com/code/zijiangyang1116/birdclef-24-efficientnetb0-pytorch-train. If you search for \"only training one fold\" you can see that he defines 10 folds but only trains one. It's just a way to create a 10% validation split for a single fold really.

", "rawMarkdown": "I meant like in this notebook: https://www.kaggle.com/code/zijiangyang1116/birdclef-24-efficientnetb0-pytorch-train. If you search for \"only training one fold\" you can see that he defines 10 folds but only trains one. It's just a way to create a 10% validation split for a single fold really.", "votes": null }, { "id": "2816724", "postDate": "05/16/2024 13:59:11", "content": "

I don't think that will help in gauging model's generalization capability. Moreover, it does not take advantage of the full dataset.

", "rawMarkdown": "I don't think that will help in gauging model's generalization capability. Moreover, it does not take advantage of the full dataset.", "votes": null }, { "id": "2816734", "postDate": "05/16/2024 14:04:07", "content": "

No, it's just a way to save time in training.

", "rawMarkdown": "No, it's just a way to save time in training.", "votes": null } ], "comments": [ { "id": 2814515, "author_name": "janhuus", "author_url": "", "post_date": "05/15/2024 11:21:51", "content": "

You can speed up training a lot pre-loading spectrograms into a pickle file, so they aren’t created during training. Also, you could define say 10 folds but stop training after say 3.

", "votes": null, "replies": [ { "id": 2816579, "author_name": "srinath9s", "author_url": "", "post_date": "05/16/2024 12:04:53", "content": "

Thanks, are you suggesting finalizing hyperparameters based on a smaller number of epochs for cross-validation and then training for longer epochs on the entire dataset?
\nLet me know if you meant something else.

", "votes": null, "replies": [ { "id": 2816678, "author_name": "janhuus", "author_url": "", "post_date": "05/16/2024 13:24:59", "content": "

I meant like in this notebook: https://www.kaggle.com/code/zijiangyang1116/birdclef-24-efficientnetb0-pytorch-train. If you search for \"only training one fold\" you can see that he defines 10 folds but only trains one. It's just a way to create a 10% validation split for a single fold really.

", "votes": null, "replies": [ { "id": 2816724, "author_name": "srinath9s", "author_url": "", "post_date": "05/16/2024 13:59:11", "content": "

I don't think that will help in gauging model's generalization capability. Moreover, it does not take advantage of the full dataset.

", "votes": null, "replies": [ { "id": 2816734, "author_name": "janhuus", "author_url": "", "post_date": "05/16/2024 14:04:07", "content": "

No, it's just a way to save time in training.

", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2814485": "I am starting a thread to understand how participants here manage to perform cross-validation (5 to 10 fold) training in resource-constrained environments. What techniques can be used?", "2814515": "You can speed up training a lot pre-loading spectrograms into a pickle file, so they aren’t created during training. Also, you could define say 10 folds but stop training after say 3.", "2816579": "Thanks, are you suggesting finalizing hyperparameters based on a smaller number of epochs for cross-validation and then training for longer epochs on the entire dataset?\nLet me know if you meant something else.", "2816678": "I meant like in this notebook: https://www.kaggle.com/code/zijiangyang1116/birdclef-24-efficientnetb0-pytorch-train. If you search for \"only training one fold\" you can see that he defines 10 folds but only trains one. It's just a way to create a 10% validation split for a single fold really.", "2816724": "I don't think that will help in gauging model's generalization capability. Moreover, it does not take advantage of the full dataset.", "2816734": "No, it's just a way to save time in training." }, "source": "meta" }