To my knowledge all of us have quotas for GPU usage. This quota for me is 30 hours per week. I noticed that each submission takes about 1~3 hours for me. I shouldn't be entirely wrong to guess the teams higher up in the leaderboard should be using much more complex models compared to mine (I speculate some would already be using ensembles of some kind - which I don't, currently.), so surely their solutions take more than merely an hour to submit. That makes less than 20 submissions a week - which means 15 shots are wasted every week.
\nHowever, when I see the leaderboard, quite a few teams somehow seem to have procured the necessary GPU time to make 200-300 submissions over the past eleven weeks. Eleven times twenty is 220, and many teams have submitted more solutions than 220, and many of them are solo teams, too.
\nSo I was wondering: am I missing anything? is 3 hours too long, perhaps? Or is there a way to procure additional GPU time?
", "messages": [ { "id": "1774749", "postDate": "05/02/2022 12:20:02", "content": "To my knowledge all of us have quotas for GPU usage. This quota for me is 30 hours per week. I noticed that each submission takes about 1~3 hours for me. I shouldn't be entirely wrong to guess the teams higher up in the leaderboard should be using much more complex models compared to mine (I speculate some would already be using ensembles of some kind - which I don't, currently.), so surely their solutions take more than merely an hour to submit. That makes less than 20 submissions a week - which means 15 shots are wasted every week.
\nHowever, when I see the leaderboard, quite a few teams somehow seem to have procured the necessary GPU time to make 200-300 submissions over the past eleven weeks. Eleven times twenty is 220, and many teams have submitted more solutions than 220, and many of them are solo teams, too.
\nSo I was wondering: am I missing anything? is 3 hours too long, perhaps? Or is there a way to procure additional GPU time?
", "rawMarkdown": "To my knowledge all of us have quotas for GPU usage. This quota for me is 30 hours per week. I noticed that each submission takes about 1~3 hours for me. I shouldn't be entirely wrong to guess the teams higher up in the leaderboard should be using much more complex models compared to mine (I speculate some would already be using ensembles of some kind - which I don't, currently.), so surely their solutions take more than merely an hour to submit. That makes less than 20 submissions a week - which means 15 shots are wasted every week.\n\nHowever, when I see the leaderboard, quite a few teams somehow seem to have procured the necessary GPU time to make 200-300 submissions over the past eleven weeks. Eleven times twenty is 220, and many teams have submitted more solutions than 220, and many of them are solo teams, too.\n\nSo I was wondering: am I missing anything? is 3 hours too long, perhaps? Or is there a way to procure additional GPU time?", "votes": null }, { "id": "1774800", "postDate": "05/02/2022 12:57:48", "content": "The way public notebooks do submissions is far from optimal,
\nit takes around 10 minutes for my average model to complete scoring
10! That's astonishingly short. Cool to know.
\nIf I may, may I ask for some pointers, for example is there stuff I can do to shorten time, besides trying to pipeline?
For example, this notebook uses batch_size=1 for inference, I think with some tweaking you can speed things up by adding support for larger batches
", "rawMarkdown": "For example, [this](https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer) notebook uses batch_size=1 for inference, I think with some tweaking you can speed things up by adding support for larger batches", "votes": null }, { "id": "1779028", "postDate": "05/06/2022 02:59:44", "content": "In my memory, your GPU quota is not consumed in your submission. This week I have spent about 5 hours on submission and my quota has not been reduced that much.
", "rawMarkdown": "In my memory, your GPU quota is not consumed in your submission. This week I have spent about 5 hours on submission and my quota has not been reduced that much.", "votes": null }, { "id": "1779674", "postDate": "05/06/2022 17:33:51", "content": "That's correct - saving and submissions don't count against your GPU time; only the time that you're running a notebook in the notebook editor counts.
", "rawMarkdown": "That's correct - saving and submissions don't count against your GPU time; only the time that you're running a notebook in the notebook editor counts.", "votes": null }, { "id": "1779855", "postDate": "05/06/2022 22:20:34", "content": "This isn't correct. Submissions don't count but saving always counts.
", "rawMarkdown": "This isn't correct. Submissions don't count but saving always counts.", "votes": null }, { "id": "1780609", "postDate": "05/07/2022 16:21:08", "content": "Ah, my mistake. thanks for the update
", "rawMarkdown": "Ah, my mistake. thanks for the update", "votes": null } ], "comments": [ { "id": 1774800, "author_name": "martynoveduard", "author_url": "", "post_date": "05/02/2022 12:57:48", "content": "The way public notebooks do submissions is far from optimal,
\nit takes around 10 minutes for my average model to complete scoring
10! That's astonishingly short. Cool to know.
\nIf I may, may I ask for some pointers, for example is there stuff I can do to shorten time, besides trying to pipeline?
For example, this notebook uses batch_size=1 for inference, I think with some tweaking you can speed things up by adding support for larger batches
", "votes": null, "replies": [] } ] }, { "id": 1779028, "author_name": "shinmurashinmura", "author_url": "", "post_date": "05/06/2022 02:59:44", "content": "In my memory, your GPU quota is not consumed in your submission. This week I have spent about 5 hours on submission and my quota has not been reduced that much.
", "votes": null, "replies": [ { "id": 1779674, "author_name": "chris62", "author_url": "", "post_date": "05/06/2022 17:33:51", "content": "That's correct - saving and submissions don't count against your GPU time; only the time that you're running a notebook in the notebook editor counts.
", "votes": null, "replies": [] }, { "id": 1779855, "author_name": "blankaf", "author_url": "", "post_date": "05/06/2022 22:20:34", "content": "This isn't correct. Submissions don't count but saving always counts.
", "votes": null, "replies": [] }, { "id": 1780609, "author_name": "chris62", "author_url": "", "post_date": "05/07/2022 16:21:08", "content": "Ah, my mistake. thanks for the update
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1774749": "To my knowledge all of us have quotas for GPU usage. This quota for me is 30 hours per week. I noticed that each submission takes about 1~3 hours for me. I shouldn't be entirely wrong to guess the teams higher up in the leaderboard should be using much more complex models compared to mine (I speculate some would already be using ensembles of some kind - which I don't, currently.), so surely their solutions take more than merely an hour to submit. That makes less than 20 submissions a week - which means 15 shots are wasted every week.\n\nHowever, when I see the leaderboard, quite a few teams somehow seem to have procured the necessary GPU time to make 200-300 submissions over the past eleven weeks. Eleven times twenty is 220, and many teams have submitted more solutions than 220, and many of them are solo teams, too.\n\nSo I was wondering: am I missing anything? is 3 hours too long, perhaps? Or is there a way to procure additional GPU time?", "1774800": "The way public notebooks do submissions is far from optimal, \nit takes around 10 minutes for my average model to complete scoring", "1774810": "10! That's astonishingly short. Cool to know. \nIf I may, may I ask for some pointers, for example is there stuff I can do to shorten time, besides trying to pipeline?", "1775002": "For example, [this](https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer) notebook uses batch_size=1 for inference, I think with some tweaking you can speed things up by adding support for larger batches", "1779028": "In my memory, your GPU quota is not consumed in your submission. This week I have spent about 5 hours on submission and my quota has not been reduced that much.", "1779674": "That's correct - saving and submissions don't count against your GPU time; only the time that you're running a notebook in the notebook editor counts.", "1779855": "This isn't correct. Submissions don't count but saving always counts.", "1780609": "Ah, my mistake. thanks for the update" }, "source": "meta" }