It seems that this competition will be a heavy compute one (I hope I am wrong though) and Kaggle's 40 hours of generous infra won't be enough I guess.
\nI have a GPU at home but I think a cloud env could be more convenient.
\nI am asking those that have tried it for any new Kaggle competition. Is the price (40 euros) worth it or is it a pass?
\nhttps://colab.research.google.com/signup
", "messages": [ { "id": 1479941, "postDate": "2021-08-18T17:52:49.043Z", "content": "It seems that this competition will be a heavy compute one (I hope I am wrong though) and Kaggle's 40 hours of generous infra won't be enough I guess.
\nI have a GPU at home but I think a cloud env could be more convenient.
\nI am asking those that have tried it for any new Kaggle competition. Is the price (40 euros) worth it or is it a pass?
\nhttps://colab.research.google.com/signup
", "rawMarkdown": "It seems that this competition will be a heavy compute one (I hope I am wrong though) and Kaggle's 40 hours of generous infra won't be enough I guess.\n\nI have a GPU at home but I think a cloud env could be more convenient. \n\nI am asking those that have tried it for any new Kaggle competition. Is the price (40 euros) worth it or is it a pass?\n\nhttps://colab.research.google.com/signup ", "votes": 15 }, { "id": 1512215, "postDate": "2021-09-14T05:23:15.127Z", "content": "I have been using RTX6000 from https://jarvislabs.ai/ and I have found it very useful and affordable . They are cheaper than AWS or GCP instances and provide all the facilities that they do and even more at cheaper rates. It has been getting the job done uptill now for my team
", "rawMarkdown": "I have been using RTX6000 from https://jarvislabs.ai/ and I have found it very useful and affordable . They are cheaper than AWS or GCP instances and provide all the facilities that they do and even more at cheaper rates. It has been getting the job done uptill now for my team", "votes": 6, "replies": [ { "id": 1512262, "postDate": "2021-09-14T06:18:00.587Z", "content": "That's one other option I had in mind (full disclaimer, it is my teammate that has suggested it). Thanks for confirming that this is a viable option. π
", "rawMarkdown": "That's one other option I had in mind (full disclaimer, it is my teammate that has suggested it). Thanks for confirming that this is a viable option. π" } ] }, { "id": 1506977, "postDate": "2021-09-08T17:49:16.917Z", "content": "Well, my story is less positive. After some hesitation I've switched to Pro+ in the middle of August. While I was quite happy initially during first weeks, getting V100 very often, last 5 days are really awful. I'm getting K80/T4 most of the time, rarely P100, but what is even more frustrating is that I can't get more RAM, even requesting High RAM instances, it gives me only 12GB and 2 CPU cores, which is too low for any meaningful task. I was on a plain Pro plan for many months and I never had such bad experience as I have now on Pro+. I see a few users also reporting this issue on github, but looks like no one cares. I also tried to contact their \"support\", but all that I received in response was just a boilerplate message \"In order to be able to offer computational resources at scale, Colab needs to maintain flexibility to adjust usage dynamically\" which doesn't help at all.
\nI wonder if this affects only a few unlucky users like me, or is this a common problem.
", "rawMarkdown": "Well, my story is less positive. After some hesitation I've switched to Pro+ in the middle of August. While I was quite happy initially during first weeks, getting V100 very often, last 5 days are really awful. I'm getting K80/T4 most of the time, rarely P100, but what is even more frustrating is that I can't get more RAM, even requesting High RAM instances, it gives me only 12GB and 2 CPU cores, which is too low for any meaningful task. I was on a plain Pro plan for many months and I never had such bad experience as I have now on Pro+. I see a few users also reporting this issue on github, but looks like no one cares. I also tried to contact their \"support\", but all that I received in response was just a boilerplate message \"In order to be able to offer computational resources at scale, Colab needs to maintain flexibility to adjust usage dynamically\" which doesn't help at all. \n\nI wonder if this affects only a few unlucky users like me, or is this a common problem.", "votes": 6, "replies": [ { "id": 1507008, "postDate": "2021-09-08T18:16:31.800Z", "content": "You are not a unlucky user, this seems to be a common problem. I was a happy Colab pro user for past 7 months , constantly used to P100 and V100 occasionally and could run 3 instances at a time. But recently I was getting Tesla T4 and K80 almost everytime (not even once got a P100) and could run only one instance at a time. I thought there was some issue with my subscription, so I cancelled the subscription and tried re-subscribing again. But Google's throwing some cryptic errors and I can't even subscribe to Colab anymore π. Moved to other cloud GPU providers πͺ
", "rawMarkdown": "You are not a unlucky user, this seems to be a common problem. I was a happy Colab pro user for past 7 months , constantly used to P100 and V100 occasionally and could run 3 instances at a time. But recently I was getting Tesla T4 and K80 almost everytime (not even once got a P100) and could run only one instance at a time. I thought there was some issue with my subscription, so I cancelled the subscription and tried re-subscribing again. But Google's throwing some cryptic errors and I can't even subscribe to Colab anymore π. Moved to other cloud GPU providers πͺ", "votes": 2, "replies": [ { "id": 1507044, "postDate": "2021-09-08T18:51:57.487Z", "content": "Ok, after several attempts at last I received a response that looks more or less written by a human.
\n\n\nHi,
\nThanks for your email. As stated previously, we are attempting to balance demand of resources due to the surge in Colab Pro + subscribers. We have made a change this morning and you should hopefully see improved results today.
\nThanks, Colab Pro Customer Support
\n
Though I don't see any effect yet and I wonder how they did manage quite successfully Pro subscriptions for long time and now they fail with Pro+.
", "rawMarkdown": "Ok, after several attempts at last I received a response that looks more or less written by a human.\n\n> Hi,\n\n> Thanks for your email. As stated previously, we are attempting to balance demand of resources due to the surge in Colab Pro + subscribers. We have made a change this morning and you should hopefully see improved results today.\n\n> Thanks, Colab Pro Customer Support\n\nThough I don't see any effect yet and I wonder how they did manage quite successfully Pro subscriptions for long time and now they fail with Pro+.", "votes": 3 } ] }, { "id": 1508586, "postDate": "2021-09-10T11:14:54.817Z", "content": "I hope it gets better soon. Let us know @stassl how it goes. π
", "rawMarkdown": "I hope it gets better soon. Let us know @stassl how it goes. π", "replies": [ { "id": 1516138, "postDate": "2021-09-18T00:47:45.237Z", "content": "Well, it didnβt get better still. Here is latest update from google:
\n\n\nOver the past weeks, Colab has observed a sharp increase in demand, making it difficult to satisfy all concurrent demand with premium chips. We continue to prioritize our paid subscribers (Pro+, Pro) for high-end GPU assignments, and by and large, looking at aggregate numbers, and deep diving into specific cases that have been highlighted in our support channels, we observe Colab's GPU availability is well worth the subscription.
\nWe are actively addressing concerns by bringing more GPU runtimes online; this adds a mix of chips into the fleet. New capacity has resulted in improvements over the last few days. Note that due to offering on-demand resources at scale, we cannot guarantee a specific chip to any user. We are actively building features that will give you flexibility in acquiring runtimes of your preference, though note that there will always be a tradeoff between cost and chip guarantees.
\nIn cases where users have described issues with concurrent runtimes, note that for Pro+ users, we proactively issue a very long reservation when you start a GPU runtime to give you increased reliability, and this may make it difficult to initiate many concurrent sessions. We are designing ways to give you flexibility to choose between concurrency and length of session, but until then terminating sessions you're done with (using Manage Sessions) may unblock you.
\nI'm closing availability and concurrency related issues as there's not much by way of individual support we can offer on GitHub, and there don't appear to be any known issues with assignments.
\n
For now I moved to preemptible GCP instances - I can rent a 50Gb RAM, 8 CPU, T4 GPU for 5$/day, which is more expensive than Colab Pro+, and of course V100 will cost more, but there are some pros as well: you can easily spin a new instance in a minute, you have persistent disk that is not deleted after instance is terminated, so you can keep all data and installed packages there, much more flexibility - you can temporarily add more CPU/GPU to speed up things, SSH without hacks.
", "rawMarkdown": "Well, it didnβt get better still. Here is latest update from google:\n\n> Over the past weeks, Colab has observed a sharp increase in demand, making it difficult to satisfy all concurrent demand with premium chips. We continue to prioritize our paid subscribers (Pro+, Pro) for high-end GPU assignments, and by and large, looking at aggregate numbers, and deep diving into specific cases that have been highlighted in our support channels, we observe Colab's GPU availability is well worth the subscription.\n\n> We are actively addressing concerns by bringing more GPU runtimes online; this adds a mix of chips into the fleet. New capacity has resulted in improvements over the last few days. Note that due to offering on-demand resources at scale, we cannot guarantee a specific chip to any user. We are actively building features that will give you flexibility in acquiring runtimes of your preference, though note that there will always be a tradeoff between cost and chip guarantees.\n\n> In cases where users have described issues with concurrent runtimes, note that for Pro+ users, we proactively issue a very long reservation when you start a GPU runtime to give you increased reliability, and this may make it difficult to initiate many concurrent sessions. We are designing ways to give you flexibility to choose between concurrency and length of session, but until then terminating sessions you're done with (using Manage Sessions) may unblock you.\n\n> I'm closing availability and concurrency related issues as there's not much by way of individual support we can offer on GitHub, and there don't appear to be any known issues with assignments.\n\nFor now I moved to preemptible GCP instances - I can rent a 50Gb RAM, 8 CPU, T4 GPU for 5$/day, which is more expensive than Colab Pro+, and of course V100 will cost more, but there are some pros as well: you can easily spin a new instance in a minute, you have persistent disk that is not deleted after instance is terminated, so you can keep all data and installed packages there, much more flexibility - you can temporarily add more CPU/GPU to speed up things, SSH without hacks." } ] }, { "id": 1513092, "postDate": "2021-09-14T19:50:49.243Z", "content": "I have question for experienced users of google colab ;) Do colab pro+ has tpu instances? And how long they are available?
", "rawMarkdown": "I have question for experienced users of google colab ;) Do colab pro+ has tpu instances? And how long they are available?" }, { "id": 1513104, "postDate": "2021-09-14T20:13:25.313Z", "content": "I'm using TPUs via Colab Pro+ for few days.
\nIt looks like Colab provides mostly TPU v2 (I had to reduce batch sizes for my models), though I've read somewhere that you can get lucky and obtain TPU v3.
\nI haven't faced any TPU time limitations yet, though I train models nearly 24/7.
I also made several attempts on picking GPU and got K80 only.
", "rawMarkdown": "I'm using TPUs via Colab Pro+ for few days.\nIt looks like Colab provides mostly TPU v2 (I had to reduce batch sizes for my models), though I've read somewhere that you can get lucky and obtain TPU v3.\nI haven't faced any TPU time limitations yet, though I train models nearly 24/7.\n\nI also made several attempts on picking GPU and got K80 only.", "votes": 1 }, { "id": 1513117, "postDate": "2021-09-14T20:35:30.093Z", "content": "Ruslan, thank you for your answer. What's power of TPUv2? It is similar to Kaggle tpu mode?
", "rawMarkdown": "Ruslan, thank you for your answer. What's power of TPUv2? It is similar to Kaggle tpu mode?" }, { "id": 1513121, "postDate": "2021-09-14T20:45:07.063Z", "content": "It provides less memory than Kaggle TPUs - I had to reduce batch sizes compared to the config that ran well on Kaggle.
\nThis, obviously, leads to the growth of execution time.
Yet, I haven't faced any session terminations except for the idle connections (e.g. if you're running a notebook with allocated GPU/TPU and do not perform calculations - Colab will kill your session in ~15min and you'll have to reconnect).
\nExecution goes smoothly for at least 14hrs. Also, it's comfortable with making a mount for Google Drive and saving weights, training history, and some other stuff there.
If it were TPU v3, I'd be completely delighted with Colab TPU.
", "rawMarkdown": "It provides less memory than Kaggle TPUs - I had to reduce batch sizes compared to the config that ran well on Kaggle.\nThis, obviously, leads to the growth of execution time.\n\nYet, I haven't faced any session terminations except for the idle connections (e.g. if you're running a notebook with allocated GPU/TPU and do not perform calculations - Colab will kill your session in ~15min and you'll have to reconnect).\nExecution goes smoothly for at least 14hrs. Also, it's comfortable with making a mount for Google Drive and saving weights, training history, and some other stuff there.\n\nIf it were TPU v3, I'd be completely delighted with Colab TPU.", "votes": 2 }, { "id": 1513130, "postDate": "2021-09-14T21:02:02.307Z", "content": "Thank you. It seems I should try it.
", "rawMarkdown": "Thank you. It seems I should try it." } ] }, { "id": 1507362, "postDate": "2021-09-09T06:12:06.780Z", "content": "Hey, I would say there are alternative that are much cheaper, check out gradient paperspace. I have been using it for past 2 months, now.
", "rawMarkdown": "Hey, I would say there are alternative that are much cheaper, check out gradient paperspace. I have been using it for past 2 months, now.", "votes": 3, "replies": [ { "id": 1508583, "postDate": "2021-09-10T11:12:18.230Z", "content": "Interesting, that's another option I had in mind. How does it compare in terms of compute and pricing? Is it reliable (i.e. instances stay up for long jobs)?
", "rawMarkdown": "Interesting, that's another option I had in mind. How does it compare in terms of compute and pricing? Is it reliable (i.e. instances stay up for long jobs)?" }, { "id": 1509497, "postDate": "2021-09-11T10:57:31.597Z", "content": "You get a P5000 or RTX4000 (your choice) with your normal subscription (ca. 9 euro per month). The bottleneck there is that it just runs 6 hours max (except you pay extra for the runtime). An advantage compared to Colab Pro/Kaggle are the 8 CPUs provided..
", "rawMarkdown": "You get a P5000 or RTX4000 (your choice) with your normal subscription (ca. 9 euro per month). The bottleneck there is that it just runs 6 hours max (except you pay extra for the runtime). An advantage compared to Colab Pro/Kaggle are the 8 CPUs provided..", "votes": 1 }, { "id": 1509503, "postDate": "2021-09-11T11:03:16.803Z", "content": "Hi @hannes82 . Is P5000 or RTX 4000 good ? How is it compared to Kaggle's GPU (Tesla P100). And I saw that they also decreased the free storage from 200GB to 15GB. You have to pay $0.29/GB after you exceed the free storage.
", "rawMarkdown": "Hi @hannes82 . Is P5000 or RTX 4000 good ? How is it compared to Kaggle's GPU (Tesla P100). And I saw that they also decreased the free storage from 200GB to 15GB. You have to pay $0.29/GB after you exceed the free storage." }, { "id": 1509515, "postDate": "2021-09-11T11:13:46.007Z", "content": "Hi @atharvaingle. I think RTX4000 is comparable to P100 in terms of performance, the bottleneck there is that it only has 8 GB. P5000 is definitely slower.. When you buy a subscription you get 15 GB free storage (till about a month ago they actually did not enforce this limit, so you practically had unlimited storage π), and that is permanent storage. Just watch out when you delete things that they are really deleted and not just go into the trash (otherwise you may unintentionially pay extra..).
\nEdit: Just so the 15 GB for the subscription. So this seems to apply for new users..
", "rawMarkdown": "Hi @atharvaingle. I think RTX4000 is comparable to P100 in terms of performance, the bottleneck there is that it only has 8 GB. P5000 is definitely slower.. When you buy a subscription you get ~~200~~15 GB free storage (till about a month ago they actually did not enforce this limit, so you practically had unlimited storage π), and that is *permanent* storage. Just watch out when you delete things that they are really deleted and not just go into the trash (otherwise you may unintentionially pay extra..). ~~I think 15 is for free users. ~~\n\nEdit: Just so the 15 GB for the subscription. So this seems to apply for new users..", "votes": 2 }, { "id": 1513045, "postDate": "2021-09-14T18:51:25.003Z", "content": "@hannes82 Ah, the time limit is a big minus though but said you can pay for more time so should be fixable.
\nThanks for providing all these details. π
\nOut of curiosity and if you want to share, are you using it for this competition or is it based on previous usage?
", "rawMarkdown": "@hannes82 Ah, the time limit is a big minus though but said you can pay for more time so should be fixable.\n\nThanks for providing all these details. π\n\nOut of curiosity and if you want to share, are you using it for this competition or is it based on previous usage?" }, { "id": 1513101, "postDate": "2021-09-14T20:06:37.410Z", "content": "@yassinealouini yes I am using paperspace and colab pro for this competition. And kaggle of course :-)
", "rawMarkdown": "@yassinealouini yes I am using paperspace and colab pro for this competition. And kaggle of course :-)", "votes": 1 }, { "id": 1513453, "postDate": "2021-09-15T07:24:41.447Z", "content": "That's awesome! I heard about paperspace before, might give it a go then. π
", "rawMarkdown": "That's awesome! I heard about paperspace before, might give it a go then. π" } ] }, { "id": 1482308, "postDate": "2021-08-20T02:56:47.700Z", "content": "https://old.reddit.com/r/MachineLearning/comments/p3e8ex/d_google_colab_is_now_offering_a_50month_colab/
\nSeems like the consensus is not worth.
Thanks for the link!
", "rawMarkdown": "Thanks for the link!", "votes": 1 } ] }, { "id": 1480081, "postDate": "2021-08-18T19:30:54.057Z", "content": "You are not wrong on the first line though :) This is indeed a data hungry competition, driving me nuts to optimize the pipeline :(
", "rawMarkdown": "You are not wrong on the first line though :) This is indeed a data hungry competition, driving me nuts to optimize the pipeline :(", "votes": 3, "replies": [ { "id": 1481708, "postDate": "2021-08-19T16:29:54.043Z", "content": "Argh, I was hoping I was wrong for once. π
", "rawMarkdown": "Argh, I was hoping I was wrong for once. π", "votes": 1 } ] }, { "id": 1489607, "postDate": "2021-08-25T06:05:57.503Z", "content": "For those that haven't seen it, @headsortails has made a great blog post about this subject and more: https://heads0rtai1s.github.io/2021/08/24/colab-plus-kaggle-cloud-gpu/.
\nCheck it out, it is really good. π
", "rawMarkdown": "For those that haven't seen it, @headsortails has made a great blog post about this subject and more: https://heads0rtai1s.github.io/2021/08/24/colab-plus-kaggle-cloud-gpu/.\n\nCheck it out, it is really good. π", "votes": 4 }, { "id": 1509455, "postDate": "2021-09-11T10:20:18.163Z", "content": "Also, I think the colab pro + offer is made available so that users thank that colab pro is worth it. This is a common marketing technique. It is called the decoy effect: https://en.wikipedia.org/wiki/Decoy_effect.
", "rawMarkdown": "Also, I think the colab pro + offer is made available so that users thank that colab pro is worth it. This is a common marketing technique. It is called the decoy effect: https://en.wikipedia.org/wiki/Decoy_effect. ", "votes": 1 }, { "id": 1506092, "postDate": "2021-09-07T20:38:14.100Z", "content": "I just signed colab pro+ a week ago, and I am very happy with it (so far). I ran about non-stop 2 instances for almost 5 days now. So far I almost always get V100s and they never denied me a GPU.
", "rawMarkdown": "I just signed colab pro+ a week ago, and I am very happy with it (so far). I ran about non-stop 2 instances for almost 5 days now. So far I almost always get V100s and they never denied me a GPU.", "votes": 2, "replies": [ { "id": 1506956, "postDate": "2021-09-08T17:30:04.733Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1508587, "postDate": "2021-09-10T11:15:46.953Z", "content": "That's great! How much vRAM are you getting from the V100?
", "rawMarkdown": "That's great! How much vRAM are you getting from the V100?" }, { "id": 1508960, "postDate": "2021-09-10T18:02:26.433Z", "content": "Colab provides V100 with vRAM of 16GB
", "rawMarkdown": "Colab provides V100 with vRAM of 16GB", "votes": 2 }, { "id": 1509456, "postDate": "2021-09-11T10:21:07.723Z", "content": "Thanks for the details. Might be worth the try for one month.
", "rawMarkdown": "Thanks for the details. Might be worth the try for one month." } ] }, { "id": 1485279, "postDate": "2021-08-22T00:10:43.713Z", "content": "I actually got banned from colab pro (not pro+) before for opening 3 sessions at the same time, they don't explicitly ban you, they just say that you got suspended for abusing and make your life very hard by randomly timing out your sessions with no explanation whatsoever, the sad part is that there's any kind of quota or notice in advance about how much resource you may use, i guess they count on you for knowing when you are abusing and when you are not.
\nAnyway, after a month i repaid the 10 dollars and now i'm more conservative with the resources there, its enough for my needs, i even trained a few models for this competition with it.
", "rawMarkdown": "I actually got banned from colab pro (not pro+) before for opening 3 sessions at the same time, they don't explicitly ban you, they just say that you got suspended for abusing and make your life very hard by randomly timing out your sessions with no explanation whatsoever, the sad part is that there's any kind of quota or notice in advance about how much resource you may use, i guess they count on you for knowing when you are abusing and when you are not.\n\nAnyway, after a month i repaid the 10 dollars and now i'm more conservative with the resources there, its enough for my needs, i even trained a few models for this competition with it.", "votes": 2, "replies": [ { "id": 1485586, "postDate": "2021-08-22T08:40:47.883Z", "content": "Arhg, that's sad. π¨ Thanks for sharing your experience. π
\nI thought that they would give you some warnings and/or just end your quota for one week let's say (as is done in Kaggle).
\nI am glad you have found a better usage pattern now but this should be explicitly detailed in the docs and not something that you figure for your self. π¬
", "rawMarkdown": "Arhg, that's sad. π¨ Thanks for sharing your experience. π\n\nI thought that they would give you some warnings and/or just end your quota for one week let's say (as is done in Kaggle). \n\nI am glad you have found a better usage pattern now but this should be explicitly detailed in the docs and not something that you figure for your self. π¬\n", "votes": 1 }, { "id": 1486057, "postDate": "2021-08-22T16:22:56.307Z", "content": "I kind of agree, but i guess they don't want to give you explicit details about usage because they don't want to commit to that level of availability, that way they can keep the price as low as possible.
\nI tried renting cloud servers before for training algorithms, all i can say is that you can reach a 10 dollars bill pretty quickly within minutes running any gpu, so i'm glad i can have p100 and even eventually a v100 in colab for a fixed price.
", "rawMarkdown": "I kind of agree, but i guess they don't want to give you explicit details about usage because they don't want to commit to that level of availability, that way they can keep the price as low as possible.\n\nI tried renting cloud servers before for training algorithms, all i can say is that you can reach a 10 dollars bill pretty quickly within minutes running any gpu, so i'm glad i can have p100 and even eventually a v100 in colab for a fixed price.", "votes": 3 }, { "id": 1486283, "postDate": "2021-08-22T20:04:33.060Z", "content": "Yes indeed that's what makes the Colab service affordable to Google to some extent even though I doubt they make a lot of money from it, most likely acts like a marketing tool for their other cloud services?
\nRegarding cloud GPU costs, you are absolutely right: they are expensive and that's why I have bought a GPU for my local setup. Though, I haven't been using it a lot recently since I find cloud settings more convenient. I should work on making my local setup more convenient I guess. π
", "rawMarkdown": "Yes indeed that's what makes the Colab service affordable to Google to some extent even though I doubt they make a lot of money from it, most likely acts like a marketing tool for their other cloud services?\n\nRegarding cloud GPU costs, you are absolutely right: they are expensive and that's why I have bought a GPU for my local setup. Though, I haven't been using it a lot recently since I find cloud settings more convenient. I should work on making my local setup more convenient I guess. π", "votes": 2 }, { "id": 1487503, "postDate": "2021-08-23T16:58:38.880Z", "content": "That's great that you have local resources, may i ask what GPU?
\nI have a laptop that sometimes i take out, so to avoid colab timeouts i got a raspberry pi 4 connected to my router 24/7, its kind of my home hub, when i want to train something that will take several hours on colab, i run the session from there via VNC, that way i can take my laptop anywhere without being timed out for inactivity. If i'm out for long i can connect to it via teamviewer and check the progress.
\nWorks pretty well, i see no reason why this shouldn't work for a local server also
I have a 3090. I had a similar setup to yours (without the raspberry pi) but then I moved and didn't have yet time to reset it.
", "rawMarkdown": "I have a 3090. I had a similar setup to yours (without the raspberry pi) but then I moved and didn't have yet time to reset it. ", "votes": 1 } ] }, { "id": 1484104, "postDate": "2021-08-21T04:29:40.590Z", "content": "I have not try it, in my case I have 2 colab pro, that gives me 6 tpu or gpu at the same time for 24 hours, don't know if you can open 6 sessions with colab pro +. 2 colab pro is 20 dollars, 1 colab pro + is 50β¦
", "rawMarkdown": "I have not try it, in my case I have 2 colab pro, that gives me 6 tpu or gpu at the same time for 24 hours, don't know if you can open 6 sessions with colab pro +. 2 colab pro is 20 dollars, 1 colab pro + is 50...", "votes": 2, "replies": [ { "id": 1484794, "postDate": "2021-08-21T15:18:42.357Z", "content": "That's a very clever use of the Colab Pro account. Out of curiosity: is it allowed to have more than one account per person? π€
", "rawMarkdown": "That's a very clever use of the Colab Pro account. Out of curiosity: is it allowed to have more than one account per person? π€", "votes": 2 }, { "id": 1484802, "postDate": "2021-08-21T15:23:24.537Z", "content": "Yes you can, unless im ignoring a rule haha
", "rawMarkdown": "Yes you can, unless im ignoring a rule haha", "votes": 1 }, { "id": 1484829, "postDate": "2021-08-21T15:38:46Z", "content": "Time to create a cluster of google colab pro subscriptions then. π
", "rawMarkdown": "Time to create a cluster of google colab pro subscriptions then. π", "votes": 1 }, { "id": 1485579, "postDate": "2021-08-22T08:25:34.773Z", "content": "You can be banned for multiple accounts from the same IP address. The better way may be to use tor browser or some IP hiding service. This way it's possible to scale up much more on hardware
", "rawMarkdown": "You can be banned for multiple accounts from the same IP address. The better way may be to use tor browser or some IP hiding service. This way it's possible to scale up much more on hardware ", "votes": 2 }, { "id": 1485588, "postDate": "2021-08-22T08:42:56.367Z", "content": "Yes for sure but given these hurdles, I guess it is maybe better to pay for an actual GPU instance from GCP or AWS. π
", "rawMarkdown": "Yes for sure but given these hurdles, I guess it is maybe better to pay for an actual GPU instance from GCP or AWS. π" }, { "id": 1485797, "postDate": "2021-08-22T12:35:18.397Z", "content": "GCP or AWS might be costly for that matter. You may want to checkout GPU optimized computing platforms like Q Blocks that are decentralized in nature and offer highly cost efficient GPU instances.
\nDisclaimer: I am associated with Q Blocks.
Interesting. Do you have a pricing sheet that compares them to your service?
", "rawMarkdown": "Interesting. Do you have a pricing sheet that compares them to your service?", "votes": 1 }, { "id": 1508513, "postDate": "2021-09-10T10:02:42.300Z", "content": "Yes, once you login you can see the price list of available GPU configs in the \"create new instance\" section. Comparatively you end up saving anywhere from 5-10x.
\nLet me know if I can help at any point. Cheers!
That sounds great. I will give it a try at some point, thanks for the details @genesis96839. π
", "rawMarkdown": "That sounds great. I will give it a try at some point, thanks for the details @genesis96839. π ", "votes": 1 } ] }, { "id": 1480085, "postDate": "2021-08-18T19:32:29.643Z", "content": "I got some idea on using Colab Pro (NOT Pro+) from this discussion comment.
", "rawMarkdown": "I got some idea on using Colab Pro (NOT Pro+) from [this discussion comment](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/254775#1398930).", "votes": 2, "replies": [ { "id": 1481707, "postDate": "2021-08-19T16:29:21.767Z", "content": "Thanks for the link, I will give it a look. π
", "rawMarkdown": "Thanks for the link, I will give it a look. π", "votes": 1 } ] }, { "id": 1559721, "postDate": "2021-10-27T07:09:19.103Z", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
I have been using RTX6000 from https://jarvislabs.ai/ and I have found it very useful and affordable . They are cheaper than AWS or GCP instances and provide all the facilities that they do and even more at cheaper rates. It has been getting the job done uptill now for my team
", "votes": 6, "replies": [ { "id": 1512262, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-14T06:18:00.587000", "content": "That's one other option I had in mind (full disclaimer, it is my teammate that has suggested it). Thanks for confirming that this is a viable option. π
", "votes": 0, "replies": [] } ] }, { "id": 1506977, "author_name": "Stas Sl", "author_url": "", "post_date": "2021-09-08T17:49:16.917000", "content": "Well, my story is less positive. After some hesitation I've switched to Pro+ in the middle of August. While I was quite happy initially during first weeks, getting V100 very often, last 5 days are really awful. I'm getting K80/T4 most of the time, rarely P100, but what is even more frustrating is that I can't get more RAM, even requesting High RAM instances, it gives me only 12GB and 2 CPU cores, which is too low for any meaningful task. I was on a plain Pro plan for many months and I never had such bad experience as I have now on Pro+. I see a few users also reporting this issue on github, but looks like no one cares. I also tried to contact their \"support\", but all that I received in response was just a boilerplate message \"In order to be able to offer computational resources at scale, Colab needs to maintain flexibility to adjust usage dynamically\" which doesn't help at all.
\nI wonder if this affects only a few unlucky users like me, or is this a common problem.
", "votes": 6, "replies": [ { "id": 1507008, "author_name": "Atharva Ingle", "author_url": "", "post_date": "2021-09-08T18:16:31.800000", "content": "You are not a unlucky user, this seems to be a common problem. I was a happy Colab pro user for past 7 months , constantly used to P100 and V100 occasionally and could run 3 instances at a time. But recently I was getting Tesla T4 and K80 almost everytime (not even once got a P100) and could run only one instance at a time. I thought there was some issue with my subscription, so I cancelled the subscription and tried re-subscribing again. But Google's throwing some cryptic errors and I can't even subscribe to Colab anymore π. Moved to other cloud GPU providers πͺ
", "votes": 2, "replies": [ { "id": 1507044, "author_name": "Stas Sl", "author_url": "", "post_date": "2021-09-08T18:51:57.487000", "content": "Ok, after several attempts at last I received a response that looks more or less written by a human.
\n\n\nHi,
\nThanks for your email. As stated previously, we are attempting to balance demand of resources due to the surge in Colab Pro + subscribers. We have made a change this morning and you should hopefully see improved results today.
\nThanks, Colab Pro Customer Support
\n
Though I don't see any effect yet and I wonder how they did manage quite successfully Pro subscriptions for long time and now they fail with Pro+.
", "votes": 3, "replies": [] } ] }, { "id": 1508586, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-10T11:14:54.817000", "content": "I hope it gets better soon. Let us know @stassl how it goes. π
", "votes": 0, "replies": [ { "id": 1516138, "author_name": "Stas Sl", "author_url": "", "post_date": "2021-09-18T00:47:45.237000", "content": "Well, it didnβt get better still. Here is latest update from google:
\n\n\nOver the past weeks, Colab has observed a sharp increase in demand, making it difficult to satisfy all concurrent demand with premium chips. We continue to prioritize our paid subscribers (Pro+, Pro) for high-end GPU assignments, and by and large, looking at aggregate numbers, and deep diving into specific cases that have been highlighted in our support channels, we observe Colab's GPU availability is well worth the subscription.
\nWe are actively addressing concerns by bringing more GPU runtimes online; this adds a mix of chips into the fleet. New capacity has resulted in improvements over the last few days. Note that due to offering on-demand resources at scale, we cannot guarantee a specific chip to any user. We are actively building features that will give you flexibility in acquiring runtimes of your preference, though note that there will always be a tradeoff between cost and chip guarantees.
\nIn cases where users have described issues with concurrent runtimes, note that for Pro+ users, we proactively issue a very long reservation when you start a GPU runtime to give you increased reliability, and this may make it difficult to initiate many concurrent sessions. We are designing ways to give you flexibility to choose between concurrency and length of session, but until then terminating sessions you're done with (using Manage Sessions) may unblock you.
\nI'm closing availability and concurrency related issues as there's not much by way of individual support we can offer on GitHub, and there don't appear to be any known issues with assignments.
\n
For now I moved to preemptible GCP instances - I can rent a 50Gb RAM, 8 CPU, T4 GPU for 5$/day, which is more expensive than Colab Pro+, and of course V100 will cost more, but there are some pros as well: you can easily spin a new instance in a minute, you have persistent disk that is not deleted after instance is terminated, so you can keep all data and installed packages there, much more flexibility - you can temporarily add more CPU/GPU to speed up things, SSH without hacks.
", "votes": 0, "replies": [] } ] }, { "id": 1513092, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2021-09-14T19:50:49.243000", "content": "I have question for experienced users of google colab ;) Do colab pro+ has tpu instances? And how long they are available?
", "votes": 0, "replies": [] }, { "id": 1513104, "author_name": "Ruslan Vdovychenko", "author_url": "", "post_date": "2021-09-14T20:13:25.313000", "content": "I'm using TPUs via Colab Pro+ for few days.
\nIt looks like Colab provides mostly TPU v2 (I had to reduce batch sizes for my models), though I've read somewhere that you can get lucky and obtain TPU v3.
\nI haven't faced any TPU time limitations yet, though I train models nearly 24/7.
I also made several attempts on picking GPU and got K80 only.
", "votes": 1, "replies": [] }, { "id": 1513117, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2021-09-14T20:35:30.093000", "content": "Ruslan, thank you for your answer. What's power of TPUv2? It is similar to Kaggle tpu mode?
", "votes": 0, "replies": [] }, { "id": 1513121, "author_name": "Ruslan Vdovychenko", "author_url": "", "post_date": "2021-09-14T20:45:07.063000", "content": "It provides less memory than Kaggle TPUs - I had to reduce batch sizes compared to the config that ran well on Kaggle.
\nThis, obviously, leads to the growth of execution time.
Yet, I haven't faced any session terminations except for the idle connections (e.g. if you're running a notebook with allocated GPU/TPU and do not perform calculations - Colab will kill your session in ~15min and you'll have to reconnect).
\nExecution goes smoothly for at least 14hrs. Also, it's comfortable with making a mount for Google Drive and saving weights, training history, and some other stuff there.
If it were TPU v3, I'd be completely delighted with Colab TPU.
", "votes": 2, "replies": [] }, { "id": 1513130, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2021-09-14T21:02:02.307000", "content": "Thank you. It seems I should try it.
", "votes": 0, "replies": [] } ] }, { "id": 1507362, "author_name": "RobertLangdonVinci", "author_url": "", "post_date": "2021-09-09T06:12:06.780000", "content": "Hey, I would say there are alternative that are much cheaper, check out gradient paperspace. I have been using it for past 2 months, now.
", "votes": 3, "replies": [ { "id": 1508583, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-10T11:12:18.230000", "content": "Interesting, that's another option I had in mind. How does it compare in terms of compute and pricing? Is it reliable (i.e. instances stay up for long jobs)?
", "votes": 0, "replies": [] }, { "id": 1509497, "author_name": "Hannes Γhler", "author_url": "", "post_date": "2021-09-11T10:57:31.597000", "content": "You get a P5000 or RTX4000 (your choice) with your normal subscription (ca. 9 euro per month). The bottleneck there is that it just runs 6 hours max (except you pay extra for the runtime). An advantage compared to Colab Pro/Kaggle are the 8 CPUs provided..
", "votes": 1, "replies": [] }, { "id": 1509503, "author_name": "Atharva Ingle", "author_url": "", "post_date": "2021-09-11T11:03:16.803000", "content": "Hi @hannes82 . Is P5000 or RTX 4000 good ? How is it compared to Kaggle's GPU (Tesla P100). And I saw that they also decreased the free storage from 200GB to 15GB. You have to pay $0.29/GB after you exceed the free storage.
", "votes": 0, "replies": [] }, { "id": 1509515, "author_name": "Hannes Γhler", "author_url": "", "post_date": "2021-09-11T11:13:46.007000", "content": "Hi @atharvaingle. I think RTX4000 is comparable to P100 in terms of performance, the bottleneck there is that it only has 8 GB. P5000 is definitely slower.. When you buy a subscription you get 15 GB free storage (till about a month ago they actually did not enforce this limit, so you practically had unlimited storage π), and that is permanent storage. Just watch out when you delete things that they are really deleted and not just go into the trash (otherwise you may unintentionially pay extra..).
\nEdit: Just so the 15 GB for the subscription. So this seems to apply for new users..
", "votes": 2, "replies": [] }, { "id": 1513045, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-14T18:51:25.003000", "content": "@hannes82 Ah, the time limit is a big minus though but said you can pay for more time so should be fixable.
\nThanks for providing all these details. π
\nOut of curiosity and if you want to share, are you using it for this competition or is it based on previous usage?
", "votes": 0, "replies": [] }, { "id": 1513101, "author_name": "Hannes Γhler", "author_url": "", "post_date": "2021-09-14T20:06:37.410000", "content": "@yassinealouini yes I am using paperspace and colab pro for this competition. And kaggle of course :-)
", "votes": 1, "replies": [] }, { "id": 1513453, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-15T07:24:41.447000", "content": "That's awesome! I heard about paperspace before, might give it a go then. π
", "votes": 0, "replies": [] } ] }, { "id": 1482308, "author_name": "brachester", "author_url": "", "post_date": "2021-08-20T02:56:47.700000", "content": "https://old.reddit.com/r/MachineLearning/comments/p3e8ex/d_google_colab_is_now_offering_a_50month_colab/
\nSeems like the consensus is not worth.
Thanks for the link!
", "votes": 1, "replies": [] } ] }, { "id": 1480081, "author_name": "Md Zarif Ul Alam", "author_url": "", "post_date": "2021-08-18T19:30:54.057000", "content": "You are not wrong on the first line though :) This is indeed a data hungry competition, driving me nuts to optimize the pipeline :(
", "votes": 3, "replies": [ { "id": 1481708, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-19T16:29:54.043000", "content": "Argh, I was hoping I was wrong for once. π
", "votes": 1, "replies": [] } ] }, { "id": 1489607, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-25T06:05:57.503000", "content": "For those that haven't seen it, @headsortails has made a great blog post about this subject and more: https://heads0rtai1s.github.io/2021/08/24/colab-plus-kaggle-cloud-gpu/.
\nCheck it out, it is really good. π
", "votes": 4, "replies": [] }, { "id": 1509455, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-11T10:20:18.163000", "content": "Also, I think the colab pro + offer is made available so that users thank that colab pro is worth it. This is a common marketing technique. It is called the decoy effect: https://en.wikipedia.org/wiki/Decoy_effect.
", "votes": 1, "replies": [] }, { "id": 1506092, "author_name": "Adriano Passos", "author_url": "", "post_date": "2021-09-07T20:38:14.100000", "content": "I just signed colab pro+ a week ago, and I am very happy with it (so far). I ran about non-stop 2 instances for almost 5 days now. So far I almost always get V100s and they never denied me a GPU.
", "votes": 2, "replies": [ { "id": 1506956, "author_name": "", "author_url": "", "post_date": "2021-09-08T17:30:04.733000", "content": "", "votes": 0, "replies": [] }, { "id": 1508587, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-10T11:15:46.953000", "content": "That's great! How much vRAM are you getting from the V100?
", "votes": 0, "replies": [] }, { "id": 1508960, "author_name": "Atharva Ingle", "author_url": "", "post_date": "2021-09-10T18:02:26.433000", "content": "Colab provides V100 with vRAM of 16GB
", "votes": 2, "replies": [] }, { "id": 1509456, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-09-11T10:21:07.723000", "content": "Thanks for the details. Might be worth the try for one month.
", "votes": 0, "replies": [] } ] }, { "id": 1485279, "author_name": "victorasso", "author_url": "", "post_date": "2021-08-22T00:10:43.713000", "content": "I actually got banned from colab pro (not pro+) before for opening 3 sessions at the same time, they don't explicitly ban you, they just say that you got suspended for abusing and make your life very hard by randomly timing out your sessions with no explanation whatsoever, the sad part is that there's any kind of quota or notice in advance about how much resource you may use, i guess they count on you for knowing when you are abusing and when you are not.
\nAnyway, after a month i repaid the 10 dollars and now i'm more conservative with the resources there, its enough for my needs, i even trained a few models for this competition with it.
", "votes": 2, "replies": [ { "id": 1485586, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-22T08:40:47.883000", "content": "Arhg, that's sad. π¨ Thanks for sharing your experience. π
\nI thought that they would give you some warnings and/or just end your quota for one week let's say (as is done in Kaggle).
\nI am glad you have found a better usage pattern now but this should be explicitly detailed in the docs and not something that you figure for your self. π¬
", "votes": 1, "replies": [] }, { "id": 1486057, "author_name": "victorasso", "author_url": "", "post_date": "2021-08-22T16:22:56.307000", "content": "I kind of agree, but i guess they don't want to give you explicit details about usage because they don't want to commit to that level of availability, that way they can keep the price as low as possible.
\nI tried renting cloud servers before for training algorithms, all i can say is that you can reach a 10 dollars bill pretty quickly within minutes running any gpu, so i'm glad i can have p100 and even eventually a v100 in colab for a fixed price.
", "votes": 3, "replies": [] }, { "id": 1486283, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-22T20:04:33.060000", "content": "Yes indeed that's what makes the Colab service affordable to Google to some extent even though I doubt they make a lot of money from it, most likely acts like a marketing tool for their other cloud services?
\nRegarding cloud GPU costs, you are absolutely right: they are expensive and that's why I have bought a GPU for my local setup. Though, I haven't been using it a lot recently since I find cloud settings more convenient. I should work on making my local setup more convenient I guess. π
", "votes": 2, "replies": [] }, { "id": 1487503, "author_name": "victorasso", "author_url": "", "post_date": "2021-08-23T16:58:38.880000", "content": "That's great that you have local resources, may i ask what GPU?
\nI have a laptop that sometimes i take out, so to avoid colab timeouts i got a raspberry pi 4 connected to my router 24/7, its kind of my home hub, when i want to train something that will take several hours on colab, i run the session from there via VNC, that way i can take my laptop anywhere without being timed out for inactivity. If i'm out for long i can connect to it via teamviewer and check the progress.
\nWorks pretty well, i see no reason why this shouldn't work for a local server also
I have a 3090. I had a similar setup to yours (without the raspberry pi) but then I moved and didn't have yet time to reset it.
", "votes": 1, "replies": [] } ] }, { "id": 1484104, "author_name": "Martin Kovacevic Buvinic", "author_url": "", "post_date": "2021-08-21T04:29:40.590000", "content": "I have not try it, in my case I have 2 colab pro, that gives me 6 tpu or gpu at the same time for 24 hours, don't know if you can open 6 sessions with colab pro +. 2 colab pro is 20 dollars, 1 colab pro + is 50β¦
", "votes": 2, "replies": [ { "id": 1484794, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-21T15:18:42.357000", "content": "That's a very clever use of the Colab Pro account. Out of curiosity: is it allowed to have more than one account per person? π€
", "votes": 2, "replies": [] }, { "id": 1484802, "author_name": "Martin Kovacevic Buvinic", "author_url": "", "post_date": "2021-08-21T15:23:24.537000", "content": "Yes you can, unless im ignoring a rule haha
", "votes": 1, "replies": [] }, { "id": 1484829, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-21T15:38:46", "content": "Time to create a cluster of google colab pro subscriptions then. π
", "votes": 1, "replies": [] }, { "id": 1485579, "author_name": "Sarthak Bhatt", "author_url": "", "post_date": "2021-08-22T08:25:34.773000", "content": "You can be banned for multiple accounts from the same IP address. The better way may be to use tor browser or some IP hiding service. This way it's possible to scale up much more on hardware
", "votes": 2, "replies": [] }, { "id": 1485588, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-22T08:42:56.367000", "content": "Yes for sure but given these hurdles, I guess it is maybe better to pay for an actual GPU instance from GCP or AWS. π
", "votes": 0, "replies": [] }, { "id": 1485797, "author_name": "Gaurav", "author_url": "", "post_date": "2021-08-22T12:35:18.397000", "content": "GCP or AWS might be costly for that matter. You may want to checkout GPU optimized computing platforms like Q Blocks that are decentralized in nature and offer highly cost efficient GPU instances.
\nDisclaimer: I am associated with Q Blocks.
Interesting. Do you have a pricing sheet that compares them to your service?
", "votes": 1, "replies": [] }, { "id": 1508513, "author_name": "Gaurav", "author_url": "", "post_date": "2021-09-10T10:02:42.300000", "content": "Yes, once you login you can see the price list of available GPU configs in the \"create new instance\" section. Comparatively you end up saving anywhere from 5-10x.
\nLet me know if I can help at any point. Cheers!
That sounds great. I will give it a try at some point, thanks for the details @genesis96839. π
", "votes": 1, "replies": [] } ] }, { "id": 1480085, "author_name": "Rohan Paul", "author_url": "", "post_date": "2021-08-18T19:32:29.643000", "content": "I got some idea on using Colab Pro (NOT Pro+) from this discussion comment.
", "votes": 2, "replies": [ { "id": 1481707, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-08-19T16:29:21.767000", "content": "Thanks for the link, I will give it a look. π
", "votes": 1, "replies": [] } ] }, { "id": 1559721, "author_name": "ChristopherZerafa", "author_url": "", "post_date": "2021-10-27T07:09:19.103000", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris