![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2555700%2F492c99bd8664caeeb7d4e7b804884e1e%2FScreenshot%202020-12-02%20220313.png?generation=1606928651668651&alt=media\")
I have used up all my GPU quota of the week and therefore cant commit the submission notebook to run on the GPU. Now, I am afraid my submission will time out on the CPU. Considering that the inference time is an important factor in this competition, is it possible to run submission evaluation on GPU even after quota is used up?
", "messages": [ { "id": "1063636", "postDate": "10/29/2020 06:36:51", "content": "I have used up all my GPU quota of the week and therefore cant commit the submission notebook to run on the GPU. Now, I am afraid my submission will time out on the CPU. Considering that the inference time is an important factor in this competition, is it possible to run submission evaluation on GPU even after quota is used up?
", "rawMarkdown": "I have used up all my GPU quota of the week and therefore cant commit the submission notebook to run on the GPU. Now, I am afraid my submission will time out on the CPU. Considering that the inference time is an important factor in this competition, is it possible to run submission evaluation on GPU even after quota is used up?", "votes": null }, { "id": "1063639", "postDate": "10/29/2020 06:59:43", "content": "The ability of running on GPUs over and above the quotas defeats the purpose of having quotas 🙂
\nThere is plenty of time left in this competition for you to submit in subsequent weeks.
", "rawMarkdown": "The ability of running on GPUs over and above the quotas defeats the purpose of having quotas 🙂\n\nThere is plenty of time left in this competition for you to submit in subsequent weeks.", "votes": null }, { "id": "1063682", "postDate": "10/29/2020 07:54:30", "content": "Haha ! Agreed. However my point was that the GPU quotas should not limit my ability to successfully submit to the competition.
", "rawMarkdown": "Haha ! Agreed. However my point was that the GPU quotas should not limit my ability to successfully submit to the competition.", "votes": null }, { "id": "1063747", "postDate": "10/29/2020 09:09:29", "content": "The quotas come from the GCP platform to get a sense of the maximum utilization for an account. Allowing submissions to run beyond the quotas makes it harder to estimate resource requirements.
\nWe're lucky that we already get so many hours of free GPU/TPU usage. It's also beneficial to plan out, optimize pipelines and use them efficiently.
", "rawMarkdown": "The quotas come from the GCP platform to get a sense of the maximum utilization for an account. Allowing submissions to run beyond the quotas makes it harder to estimate resource requirements.\n\nWe're lucky that we already get so many hours of free GPU/TPU usage. It's also beneficial to plan out, optimize pipelines and use them efficiently.", "votes": null }, { "id": "1063773", "postDate": "10/29/2020 09:58:50", "content": "Got you! 👍
", "rawMarkdown": "Got you! 👍", "votes": null }, { "id": "1099893", "postDate": "12/02/2020 17:07:14", "content": "Hi, just wanted to ask if the GPU usage for the submission is tracked and counted for in the GPU Quota Tab? I made a notebook that was using GPU, and submitted it. But it's timing out (Which lead me to believe that GPU is not available) and GPU usage hasn't changed since.
\nNow I've another submission going on but I don't see anything related to it on the GPU Quota Tab
\n
For submission, we don't need to care about the GPU quota.
\nEven though we run out of the GPU quota, we can submit the GPU notebooks.
\nTo commit, we need the GPU quota.
Thanks for the clarification, have been trying to get my submission pipeline working for over 2 weeks now… :/
", "rawMarkdown": "Thanks for the clarification, have been trying to get my submission pipeline working for over 2 weeks now... :/", "votes": null }, { "id": "1100105", "postDate": "12/02/2020 20:54:34", "content": "To be precise, I observed this in another competition. I think the thing should be the same.
\nI know, building pipeline itself is tough in this comp… I'm working on it too.
Got it working finally!
\ntf.keras has a strange behavior that model.predict() is way slower then model.predict_on_batch() for moderately sized datasets.
\nSince our test data is alots of batches of small size, had to change to model.predict_on_batch() to complete the prediction loop within time.
The ability of running on GPUs over and above the quotas defeats the purpose of having quotas 🙂
\nThere is plenty of time left in this competition for you to submit in subsequent weeks.
", "votes": null, "replies": [ { "id": 1063682, "author_name": "abhimanyud", "author_url": "", "post_date": "10/29/2020 07:54:30", "content": "Haha ! Agreed. However my point was that the GPU quotas should not limit my ability to successfully submit to the competition.
", "votes": null, "replies": [] }, { "id": 1063747, "author_name": "rohanrao", "author_url": "", "post_date": "10/29/2020 09:09:29", "content": "The quotas come from the GCP platform to get a sense of the maximum utilization for an account. Allowing submissions to run beyond the quotas makes it harder to estimate resource requirements.
\nWe're lucky that we already get so many hours of free GPU/TPU usage. It's also beneficial to plan out, optimize pipelines and use them efficiently.
", "votes": null, "replies": [] }, { "id": 1063773, "author_name": "abhimanyud", "author_url": "", "post_date": "10/29/2020 09:58:50", "content": "Got you! 👍
", "votes": null, "replies": [] } ] }, { "id": 1099893, "author_name": "abdurrafae", "author_url": "", "post_date": "12/02/2020 17:07:14", "content": "Hi, just wanted to ask if the GPU usage for the submission is tracked and counted for in the GPU Quota Tab? I made a notebook that was using GPU, and submitted it. But it's timing out (Which lead me to believe that GPU is not available) and GPU usage hasn't changed since.
\nNow I've another submission going on but I don't see anything related to it on the GPU Quota Tab
\n
For submission, we don't need to care about the GPU quota.
\nEven though we run out of the GPU quota, we can submit the GPU notebooks.
\nTo commit, we need the GPU quota.
Thanks for the clarification, have been trying to get my submission pipeline working for over 2 weeks now… :/
", "votes": null, "replies": [] }, { "id": 1100105, "author_name": "kokitanisaka", "author_url": "", "post_date": "12/02/2020 20:54:34", "content": "To be precise, I observed this in another competition. I think the thing should be the same.
\nI know, building pipeline itself is tough in this comp… I'm working on it too.
Got it working finally!
\ntf.keras has a strange behavior that model.predict() is way slower then model.predict_on_batch() for moderately sized datasets.
\nSince our test data is alots of batches of small size, had to change to model.predict_on_batch() to complete the prediction loop within time.