Technically, this is my 1st code competition. I am thinking about debugging my model on my laptop/Cloud Instance and then upload my code and run it upon Kaggle for submission.
\nis it this efficient way of work? better suggestions?
", "messages": [ { "id": "1548781", "postDate": "10/18/2021 14:18:37", "content": "Technically, this is my 1st code competition. I am thinking about debugging my model on my laptop/Cloud Instance and then upload my code and run it upon Kaggle for submission.
\nis it this efficient way of work? better suggestions?
", "rawMarkdown": "Technically, this is my 1st code competition. I am thinking about debugging my model on my laptop/Cloud Instance and then upload my code and run it upon Kaggle for submission. \n\nis it this efficient way of work? better suggestions?", "votes": null }, { "id": "1548893", "postDate": "10/18/2021 15:46:50", "content": "Yes, this is usually my workflow. Develop model locally or on the cloud. Upload the weights on Kaggle as private Dataset attach it to Notebook and run submission.
", "rawMarkdown": "Yes, this is usually my workflow. Develop model locally or on the cloud. Upload the weights on Kaggle as private Dataset attach it to Notebook and run submission.", "votes": null }, { "id": "1551145", "postDate": "10/20/2021 10:57:20", "content": "Hi. Do think it is possible to train a competitive only with help of the computational resources here provided (notebooks with accelerators)? Is it enough, or is it crucial for you to have access to your own resources for training a decent model? Let's say, for instance, using a RTX 3090 graphics card. I'm wondering if it is worth trying, given this is my first competition. I know it depends on the task, but I'm asking in general, from your own experience.
", "rawMarkdown": "Hi. Do think it is possible to train a competitive only with help of the computational resources here provided (notebooks with accelerators)? Is it enough, or is it crucial for you to have access to your own resources for training a decent model? Let's say, for instance, using a RTX 3090 graphics card. I'm wondering if it is worth trying, given this is my first competition. I know it depends on the task, but I'm asking in general, from your own experience.", "votes": null }, { "id": "1551294", "postDate": "10/20/2021 13:39:19", "content": "Hi,
\nYes, it's possible to perform well with the hardware set up that you have. You not only have access to your hardware, but you can utilize google collab and Kaggle GPUs. In my first competition, I won a gold medal (APTOS) entirely using google colab. Just try to be smart about using it. For example, design 3-4 small experiments and run them in parallel on your local machine, kaggle, and google colab. Analyze the results and repeat. Once you have a pretty decent model, and you feel like you can upscale. Register for Google Cloud or Amazon. Google Cloud as a first-time user, you get 300$ credit. For Amazon, I believe it's 100$.
This is more than enough to have a decent number of GPUs.
", "rawMarkdown": "Hi, \nYes, it's possible to perform well with the hardware set up that you have. You not only have access to your hardware, but you can utilize google collab and Kaggle GPUs. In my first competition, I won a gold medal (APTOS) entirely using google colab. Just try to be smart about using it. For example, design 3-4 small experiments and run them in parallel on your local machine, kaggle, and google colab. Analyze the results and repeat. Once you have a pretty decent model, and you feel like you can upscale. Register for Google Cloud or Amazon. Google Cloud as a first-time user, you get 300$ credit. For Amazon, I believe it's 100$. \n\nThis is more than enough to have a decent number of GPUs.", "votes": null }, { "id": "1551491", "postDate": "10/20/2021 17:08:48", "content": "It makes a big difference for me in this competition. The same detectron training code takes 4 minutes on my local PC and 30 minutes on kaggle. And it's not even faster GPU, but the fact that the kaggle GPU notebook has access to only two CPU cores and it's bottlenecked on the input data processing. I can see it on 200% CPU and single percent GPU usage through the whole training while I'm at over 70% GPU utilization locally.
", "rawMarkdown": "It makes a big difference for me in this competition. The same detectron training code takes 4 minutes on my local PC and 30 minutes on kaggle. And it's not even faster GPU, but the fact that the kaggle GPU notebook has access to only two CPU cores and it's bottlenecked on the input data processing. I can see it on 200% CPU and single percent GPU usage through the whole training while I'm at over 70% GPU utilization locally.", "votes": null }, { "id": "1551678", "postDate": "10/20/2021 20:15:46", "content": "That's reassuring. Thank you for the advice, I appreciate it!
", "rawMarkdown": "That's reassuring. Thank you for the advice, I appreciate it!", "votes": null } ], "comments": [ { "id": 1548893, "author_name": "drhabib", "author_url": "", "post_date": "10/18/2021 15:46:50", "content": "Yes, this is usually my workflow. Develop model locally or on the cloud. Upload the weights on Kaggle as private Dataset attach it to Notebook and run submission.
", "votes": null, "replies": [ { "id": 1551145, "author_name": "santiagomarruffo", "author_url": "", "post_date": "10/20/2021 10:57:20", "content": "Hi. Do think it is possible to train a competitive only with help of the computational resources here provided (notebooks with accelerators)? Is it enough, or is it crucial for you to have access to your own resources for training a decent model? Let's say, for instance, using a RTX 3090 graphics card. I'm wondering if it is worth trying, given this is my first competition. I know it depends on the task, but I'm asking in general, from your own experience.
", "votes": null, "replies": [] }, { "id": 1551294, "author_name": "drhabib", "author_url": "", "post_date": "10/20/2021 13:39:19", "content": "Hi,
\nYes, it's possible to perform well with the hardware set up that you have. You not only have access to your hardware, but you can utilize google collab and Kaggle GPUs. In my first competition, I won a gold medal (APTOS) entirely using google colab. Just try to be smart about using it. For example, design 3-4 small experiments and run them in parallel on your local machine, kaggle, and google colab. Analyze the results and repeat. Once you have a pretty decent model, and you feel like you can upscale. Register for Google Cloud or Amazon. Google Cloud as a first-time user, you get 300$ credit. For Amazon, I believe it's 100$.
This is more than enough to have a decent number of GPUs.
", "votes": null, "replies": [] }, { "id": 1551678, "author_name": "santiagomarruffo", "author_url": "", "post_date": "10/20/2021 20:15:46", "content": "That's reassuring. Thank you for the advice, I appreciate it!
", "votes": null, "replies": [] } ] }, { "id": 1551491, "author_name": "slawekbiel", "author_url": "", "post_date": "10/20/2021 17:08:48", "content": "It makes a big difference for me in this competition. The same detectron training code takes 4 minutes on my local PC and 30 minutes on kaggle. And it's not even faster GPU, but the fact that the kaggle GPU notebook has access to only two CPU cores and it's bottlenecked on the input data processing. I can see it on 200% CPU and single percent GPU usage through the whole training while I'm at over 70% GPU utilization locally.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1548781": "Technically, this is my 1st code competition. I am thinking about debugging my model on my laptop/Cloud Instance and then upload my code and run it upon Kaggle for submission. \n\nis it this efficient way of work? better suggestions?", "1548893": "Yes, this is usually my workflow. Develop model locally or on the cloud. Upload the weights on Kaggle as private Dataset attach it to Notebook and run submission.", "1551145": "Hi. Do think it is possible to train a competitive only with help of the computational resources here provided (notebooks with accelerators)? Is it enough, or is it crucial for you to have access to your own resources for training a decent model? Let's say, for instance, using a RTX 3090 graphics card. I'm wondering if it is worth trying, given this is my first competition. I know it depends on the task, but I'm asking in general, from your own experience.", "1551294": "Hi, \nYes, it's possible to perform well with the hardware set up that you have. You not only have access to your hardware, but you can utilize google collab and Kaggle GPUs. In my first competition, I won a gold medal (APTOS) entirely using google colab. Just try to be smart about using it. For example, design 3-4 small experiments and run them in parallel on your local machine, kaggle, and google colab. Analyze the results and repeat. Once you have a pretty decent model, and you feel like you can upscale. Register for Google Cloud or Amazon. Google Cloud as a first-time user, you get 300$ credit. For Amazon, I believe it's 100$. \n\nThis is more than enough to have a decent number of GPUs.", "1551491": "It makes a big difference for me in this competition. The same detectron training code takes 4 minutes on my local PC and 30 minutes on kaggle. And it's not even faster GPU, but the fact that the kaggle GPU notebook has access to only two CPU cores and it's bottlenecked on the input data processing. I can see it on 200% CPU and single percent GPU usage through the whole training while I'm at over 70% GPU utilization locally.", "1551678": "That's reassuring. Thank you for the advice, I appreciate it!" }, "source": "meta" }