This question may be kinda dumb. But, I saw a lot of stuff about higher resolution training and only have kaggle GPU.
", "messages": [ { "id": "1656852", "postDate": "01/19/2022 17:30:12", "content": "This question may be kinda dumb. But, I saw a lot of stuff about higher resolution training and only have kaggle GPU.
", "rawMarkdown": "This question may be kinda dumb. But, I saw a lot of stuff about higher resolution training and only have kaggle GPU.", "votes": null }, { "id": "1656920", "postDate": "01/19/2022 19:08:16", "content": "For more details check these resources
\nIf you like Please ThumbUp
", "rawMarkdown": "####Hi [Ichimaru Gin](@ichimarugin), Just GPU is not enough to get top 100 on the Kaggle competition. many data science libraries cannot take advantage of a GPU. So, GPUs will be valuable for some competition (especially when using deep learning libraries like TensorFlow, Keras, and PyTorch). But you are better off without a GPU for most other competition.\n#### Also to reach the Top 100, you need a good strategy that consists of these four things.\n\n- **Feature Engineering**\n- **Model Selection**\n- **Cross-Validation Techniques**\n- **Hyper Parameter Tuning**.\n\n**For more details check these resources**\n- [CPU vs GPU in Machine Learning Algorithms: Which is Better](https://thinkml.ai/cpu-vs-gpu-in-machine-learning-algorithms-which-is-better/)\n- [Winning Tips on Machine Learning Competitions by Kazanova](https://www.hackerearth.com/practice/machine-learning/advanced-techniques/winning-tips-machine-learning-competitions-kazanova-current-kaggle-3/tutorial/)\n\nIf you like Please ThumbUp", "votes": null }, { "id": "1656947", "postDate": "01/19/2022 19:40:09", "content": "Tomorrow you will be suprised (I will post summary of my research (do I need 8xGPU?) but without details - there won't be information about teams but summary - I respect their hard work and time they spend to jump over other team). I collected information from TOP20 LB submission time. Today I can say that TOP10 record submission time is …. below 60 minutes …. Any other question? 😆😲
", "rawMarkdown": "Tomorrow you will be suprised (I will post summary of my research (do I need 8xGPU?) but without details - there won't be information about teams but summary - I respect their hard work and time they spend to jump over other team). I collected information from TOP20 LB submission time. Today I can say that TOP10 record submission time is .... below 60 minutes .... Any other question? 😆😲", "votes": null }, { "id": "1656948", "postDate": "01/19/2022 19:40:50", "content": "Although I haven't run enough experiments, I conjecture that Kaggle GPU is enough to end in the top 100. This is because:
\n1) There is a 9 hour inference time. There is a clear tradeoff between the size of individual models and the number of models/size of images that one can use.
\n2) A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.
\n3) I believe that the high inference sizes might not be the best for private LB.
Please correct me if I missed something, however.
", "rawMarkdown": "Although I haven't run enough experiments, I conjecture that Kaggle GPU is enough to end in the top 100. This is because:\n\n1) There is a 9 hour inference time. There is a clear tradeoff between the size of individual models and the number of models/size of images that one can use.\n2) A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.\n3) I believe that the high inference sizes might not be the best for private LB.\n\nPlease correct me if I missed something, however.", "votes": null }, { "id": "1656956", "postDate": "01/19/2022 19:44:00", "content": "A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.
Yest, thats right (actually we use Colab but out solution without any problem could be run on Kaggle / even on my laptop).
", "rawMarkdown": "` A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.`\n\nYest, thats right (actually we use Colab but out solution without any problem could be run on Kaggle / even on my laptop).", "votes": null }, { "id": "1657207", "postDate": "01/20/2022 02:44:57", "content": "I am using Colab Pro but our current highest LB model would easily run on Kaggle. It's just that it's all to easy to burn through GPU time messing about trying things.
", "rawMarkdown": "I am using Colab Pro but our current highest LB model would easily run on Kaggle. It's just that it's all to easy to burn through GPU time messing about trying things.", "votes": null }, { "id": "1657312", "postDate": "01/20/2022 05:06:48", "content": "Woah 😍
\nThanks,
\nI'm a fan of your posts in this competition. I'm excited to see your next post. 🔥
Thank you! I am still collecting data …. but it will be published soon. Interesting data :) … 49 minutes is record now (TOP20).
", "rawMarkdown": "Thank you! I am still collecting data .... but it will be published soon. Interesting data :) ... 49 minutes is record now (TOP20).", "votes": null }, { "id": "1657452", "postDate": "01/20/2022 07:36:44", "content": "Btw this is not a regression comp this is CV comp
", "rawMarkdown": "Btw this is not a regression comp this is CV comp", "votes": null }, { "id": "1657893", "postDate": "01/20/2022 14:54:05", "content": "Super enough! but considering time limitations, it's better to move to colab!
", "rawMarkdown": "Super enough! but considering time limitations, it's better to move to colab!", "votes": null }, { "id": "1657980", "postDate": "01/20/2022 16:07:47", "content": "Thanks, now I feel motivated. Yes, I'm moving to colab.
", "rawMarkdown": "Thanks, now I feel motivated. Yes, I'm moving to colab.", "votes": null }, { "id": "1658899", "postDate": "01/21/2022 11:50:23", "content": "Hi Bruce Young, may I know your personal review of Colab Pro ?
\nThanks in Advance.
If we start with Kaggle and if you want to do a lot of experiments and especially with the trend to run longer training sessions in this competition, you quickly realise that you need more time.
\nI can't afford a new high-end graphics card, well nobody can with the massive price inflation going on right now, so paying ~USD$10 a month is cheap. I can't justify paying ~$50 for colab pro+ though. When I said I \"stack potatoes\" for a living in my Bio… I meant it.
\nColab have recently been trying to make it a little harder to run longer sessions as they have implemented an \"I am not a Robot\" pop up, but really that just means you have to watch it for a few mins … click the box and then you are ok. I ran a 12 hr session without any problems. The other issue is that if you are not there to copy the results to somewhere safe when it finishes, you could lose them, as it disconnects after awhile once it finishes. To fix that I simply have a line at the end of my notebook where I copy what I need to my Google drive, which I mounted at the beginning of the run.
\nLike any system, even your own, you'll have to make sure you have similar compatible versions of anything you use between the different platforms.
\nPro gives you the option to have large RAM runtime ~25GB which I've found great in this comp to hold all the training data for quicker epochs. This brings me to the other \"issue\" with Colab… accessing the data. If you choose to train with a TPU it's relatively easy to directly access the data and you don't have to download/upload anything. In this competition however we are using GPUs, so I have put a copy of the data on my Google drive. Yes, I pay another ~$10/month for 2TB of Google drive.
\nUnfortunately it is far to slow to use your Google drive directly in training models. It's ok for a small number of files or things you just want to load to update stuff etc but to try to read thousands of little files… no. In this comp I copy the files I need, which is a subset of the total, to my Colab drive and that takes about 10-15mins each time I run my training. It's persistent if you don't do a \"factory reset\" of the notebook though.
\nColab Pro supposedly only lets you run one instance but I find that you can use 2 if they are not both the same type. TPU and GPU or GPU and CPU etc. It's useful to be able to run one and be editing and testing snippets on another.
\nIn this comp my initial benefit from Colab was simply more GPU time and yes , our current LB is still possible on Kaggle though I have been exploring 12hr runs that obviously you can't do on Kaggle and even if you could, you'd not get much done before you ran out of quota.
\nHope that helps…
\nI have to run now… those potatoes won't stack themselves!
You could get Top 100 if you know deep learning theories well and very experienced. But with a better GPU you can iterate your modelling and fine-tuning process faster and gain more experience.
", "rawMarkdown": "You could get Top 100 if you know deep learning theories well and very experienced. But with a better GPU you can iterate your modelling and fine-tuning process faster and gain more experience.", "votes": null } ], "comments": [ { "id": 1656920, "author_name": "imuhammadismail", "author_url": "", "post_date": "01/19/2022 19:08:16", "content": "For more details check these resources
\nIf you like Please ThumbUp
", "votes": null, "replies": [ { "id": 1657452, "author_name": "mithilsalunkhe", "author_url": "", "post_date": "01/20/2022 07:36:44", "content": "Btw this is not a regression comp this is CV comp
", "votes": null, "replies": [] } ] }, { "id": 1656947, "author_name": "remekkinas", "author_url": "", "post_date": "01/19/2022 19:40:09", "content": "Tomorrow you will be suprised (I will post summary of my research (do I need 8xGPU?) but without details - there won't be information about teams but summary - I respect their hard work and time they spend to jump over other team). I collected information from TOP20 LB submission time. Today I can say that TOP10 record submission time is …. below 60 minutes …. Any other question? 😆😲
", "votes": null, "replies": [ { "id": 1657312, "author_name": "ichimarugin", "author_url": "", "post_date": "01/20/2022 05:06:48", "content": "Woah 😍
\nThanks,
\nI'm a fan of your posts in this competition. I'm excited to see your next post. 🔥
Thank you! I am still collecting data …. but it will be published soon. Interesting data :) … 49 minutes is record now (TOP20).
", "votes": null, "replies": [] } ] }, { "id": 1656948, "author_name": "ther3venant", "author_url": "", "post_date": "01/19/2022 19:40:50", "content": "Although I haven't run enough experiments, I conjecture that Kaggle GPU is enough to end in the top 100. This is because:
\n1) There is a 9 hour inference time. There is a clear tradeoff between the size of individual models and the number of models/size of images that one can use.
\n2) A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.
\n3) I believe that the high inference sizes might not be the best for private LB.
Please correct me if I missed something, however.
", "votes": null, "replies": [ { "id": 1656956, "author_name": "remekkinas", "author_url": "", "post_date": "01/19/2022 19:44:00", "content": "A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.
Yest, thats right (actually we use Colab but out solution without any problem could be run on Kaggle / even on my laptop).
", "votes": null, "replies": [] } ] }, { "id": 1657207, "author_name": "mutantspore", "author_url": "", "post_date": "01/20/2022 02:44:57", "content": "I am using Colab Pro but our current highest LB model would easily run on Kaggle. It's just that it's all to easy to burn through GPU time messing about trying things.
", "votes": null, "replies": [ { "id": 1658899, "author_name": "jayaprakashaluri", "author_url": "", "post_date": "01/21/2022 11:50:23", "content": "Hi Bruce Young, may I know your personal review of Colab Pro ?
\nThanks in Advance.
If we start with Kaggle and if you want to do a lot of experiments and especially with the trend to run longer training sessions in this competition, you quickly realise that you need more time.
\nI can't afford a new high-end graphics card, well nobody can with the massive price inflation going on right now, so paying ~USD$10 a month is cheap. I can't justify paying ~$50 for colab pro+ though. When I said I \"stack potatoes\" for a living in my Bio… I meant it.
\nColab have recently been trying to make it a little harder to run longer sessions as they have implemented an \"I am not a Robot\" pop up, but really that just means you have to watch it for a few mins … click the box and then you are ok. I ran a 12 hr session without any problems. The other issue is that if you are not there to copy the results to somewhere safe when it finishes, you could lose them, as it disconnects after awhile once it finishes. To fix that I simply have a line at the end of my notebook where I copy what I need to my Google drive, which I mounted at the beginning of the run.
\nLike any system, even your own, you'll have to make sure you have similar compatible versions of anything you use between the different platforms.
\nPro gives you the option to have large RAM runtime ~25GB which I've found great in this comp to hold all the training data for quicker epochs. This brings me to the other \"issue\" with Colab… accessing the data. If you choose to train with a TPU it's relatively easy to directly access the data and you don't have to download/upload anything. In this competition however we are using GPUs, so I have put a copy of the data on my Google drive. Yes, I pay another ~$10/month for 2TB of Google drive.
\nUnfortunately it is far to slow to use your Google drive directly in training models. It's ok for a small number of files or things you just want to load to update stuff etc but to try to read thousands of little files… no. In this comp I copy the files I need, which is a subset of the total, to my Colab drive and that takes about 10-15mins each time I run my training. It's persistent if you don't do a \"factory reset\" of the notebook though.
\nColab Pro supposedly only lets you run one instance but I find that you can use 2 if they are not both the same type. TPU and GPU or GPU and CPU etc. It's useful to be able to run one and be editing and testing snippets on another.
\nIn this comp my initial benefit from Colab was simply more GPU time and yes , our current LB is still possible on Kaggle though I have been exploring 12hr runs that obviously you can't do on Kaggle and even if you could, you'd not get much done before you ran out of quota.
\nHope that helps…
\nI have to run now… those potatoes won't stack themselves!
Super enough! but considering time limitations, it's better to move to colab!
", "votes": null, "replies": [ { "id": 1657980, "author_name": "ichimarugin", "author_url": "", "post_date": "01/20/2022 16:07:47", "content": "Thanks, now I feel motivated. Yes, I'm moving to colab.
", "votes": null, "replies": [] } ] }, { "id": 1661452, "author_name": "lonnieqin", "author_url": "", "post_date": "01/23/2022 13:25:26", "content": "You could get Top 100 if you know deep learning theories well and very experienced. But with a better GPU you can iterate your modelling and fine-tuning process faster and gain more experience.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1656852": "This question may be kinda dumb. But, I saw a lot of stuff about higher resolution training and only have kaggle GPU.", "1656920": "####Hi [Ichimaru Gin](@ichimarugin), Just GPU is not enough to get top 100 on the Kaggle competition. many data science libraries cannot take advantage of a GPU. So, GPUs will be valuable for some competition (especially when using deep learning libraries like TensorFlow, Keras, and PyTorch). But you are better off without a GPU for most other competition.\n#### Also to reach the Top 100, you need a good strategy that consists of these four things.\n\n- **Feature Engineering**\n- **Model Selection**\n- **Cross-Validation Techniques**\n- **Hyper Parameter Tuning**.\n\n**For more details check these resources**\n- [CPU vs GPU in Machine Learning Algorithms: Which is Better](https://thinkml.ai/cpu-vs-gpu-in-machine-learning-algorithms-which-is-better/)\n- [Winning Tips on Machine Learning Competitions by Kazanova](https://www.hackerearth.com/practice/machine-learning/advanced-techniques/winning-tips-machine-learning-competitions-kazanova-current-kaggle-3/tutorial/)\n\nIf you like Please ThumbUp", "1656947": "Tomorrow you will be suprised (I will post summary of my research (do I need 8xGPU?) but without details - there won't be information about teams but summary - I respect their hard work and time they spend to jump over other team). I collected information from TOP20 LB submission time. Today I can say that TOP10 record submission time is .... below 60 minutes .... Any other question? 😆😲", "1656948": "Although I haven't run enough experiments, I conjecture that Kaggle GPU is enough to end in the top 100. This is because:\n\n1) There is a 9 hour inference time. There is a clear tradeoff between the size of individual models and the number of models/size of images that one can use.\n2) A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.\n3) I believe that the high inference sizes might not be the best for private LB.\n\nPlease correct me if I missed something, however.", "1656956": "` A lot of teams such as @remekkinas's have reached the top of LB on Kaggle GPU.`\n\nYest, thats right (actually we use Colab but out solution without any problem could be run on Kaggle / even on my laptop).", "1657207": "I am using Colab Pro but our current highest LB model would easily run on Kaggle. It's just that it's all to easy to burn through GPU time messing about trying things.", "1657312": "Woah 😍 \nThanks, \nI'm a fan of your posts in this competition. I'm excited to see your next post. 🔥", "1657388": "Thank you! I am still collecting data .... but it will be published soon. Interesting data :) ... 49 minutes is record now (TOP20).", "1657452": "Btw this is not a regression comp this is CV comp", "1657893": "Super enough! but considering time limitations, it's better to move to colab!", "1657980": "Thanks, now I feel motivated. Yes, I'm moving to colab.", "1658899": "Hi Bruce Young, may I know your personal review of Colab Pro ? \nThanks in Advance.", "1659341": "If we start with Kaggle and if you want to do a lot of experiments and especially with the trend to run longer training sessions in this competition, you quickly realise that you need more time. \n\nI can't afford a new high-end graphics card, well nobody can with the massive price inflation going on right now, so paying ~USD$10 a month is cheap. I can't justify paying ~$50 for colab pro+ though. When I said I \"stack potatoes\" for a living in my Bio... I meant it.\n\nColab have recently been trying to make it a little harder to run longer sessions as they have implemented an \"I am not a Robot\" pop up, but really that just means you have to watch it for a few mins ... click the box and then you are ok. I ran a 12 hr session without any problems. The other issue is that if you are not there to copy the results to somewhere safe when it finishes, you could lose them, as it disconnects after awhile once it finishes. To fix that I simply have a line at the end of my notebook where I copy what I need to my Google drive, which I mounted at the beginning of the run.\n\nLike any system, even your own, you'll have to make sure you have similar compatible versions of anything you use between the different platforms.\n\nPro gives you the option to have large RAM runtime ~25GB which I've found great in this comp to hold all the training data for quicker epochs. This brings me to the other \"issue\" with Colab... accessing the data. If you choose to train with a TPU it's relatively easy to directly access the data and you don't have to download/upload anything. In this competition however we are using GPUs, so I have put a copy of the data on my Google drive. Yes, I pay another ~$10/month for 2TB of Google drive. \n\nUnfortunately it is far to slow to use your Google drive directly in training models. It's ok for a small number of files or things you just want to load to update stuff etc but to try to read thousands of little files... no. In this comp I copy the files I need, which is a subset of the total, to my Colab drive and that takes about 10-15mins each time I run my training. It's persistent if you don't do a \"factory reset\" of the notebook though. \n\nColab Pro supposedly only lets you run one instance but I find that you can use 2 if they are not both the same type. TPU and GPU or GPU and CPU etc. It's useful to be able to run one and be editing and testing snippets on another.\n\nIn this comp my initial benefit from Colab was simply more GPU time and yes , our current LB is still possible on Kaggle though I have been exploring 12hr runs that obviously you can't do on Kaggle and even if you could, you'd not get much done before you ran out of quota.\n\nHope that helps...\nI have to run now... those potatoes won't stack themselves!", "1661452": "You could get Top 100 if you know deep learning theories well and very experienced. But with a better GPU you can iterate your modelling and fine-tuning process faster and gain more experience." }, "source": "meta" }