I see a lot of folks getting worried because the data is quite big and you'd definitely need a good amount of hardware resources to be competitive in this comp.
\n\nGCP Offers 300$ worth of credits, for a team of 8 members that equals to 2400$ worth of compute credits + @juliaelliott has confirmed Here that the hosts will provide credits soon.
\n\nThat should be good enough to compete IMO. \nPlease don't let the data size discourage you. Good Luck! π΅
", "messages": [ { "id": "693257", "postDate": "12/12/2019 07:31:31", "content": "I see a lot of folks getting worried because the data is quite big and you'd definitely need a good amount of hardware resources to be competitive in this comp.
\n\nGCP Offers 300$ worth of credits, for a team of 8 members that equals to 2400$ worth of compute credits + @juliaelliott has confirmed Here that the hosts will provide credits soon.
\n\nThat should be good enough to compete IMO. \nPlease don't let the data size discourage you. Good Luck! π΅
", "rawMarkdown": "I see a lot of folks getting worried because the data is quite big and you'd definitely need a good amount of hardware resources to be competitive in this comp.\n\nGCP Offers 300$ worth of credits, for a team of 8 members that equals to 2400$ worth of compute credits + @juliaelliott has confirmed [Here](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121226#692992) that the hosts will provide credits soon. \n\nThat should be good enough to compete IMO. \nPlease don't let the data size discourage you. Good Luck! π΅", "votes": null }, { "id": "693415", "postDate": "12/12/2019 10:49:21", "content": "Don't want to sound overly pessimistic but $300 probably lasted like a week for me in a \"normal\" image competition, I don't even want to think about this one. The extra credits will help for sure it's hard to believe you can rely on it alone.
", "rawMarkdown": "Don't want to sound overly pessimistic but $300 probably lasted like a week for me in a \"normal\" image competition, I don't even want to think about this one. The extra credits will help for sure it's hard to believe you can rely on it alone.", "votes": null }, { "id": "693420", "postDate": "12/12/2019 10:55:56", "content": "Agreed but I created this thread to remove the \"mental\" barrier for many of the people who wouldn't enter otherwise.
", "rawMarkdown": "Agreed but I created this thread to remove the \"mental\" barrier for many of the people who wouldn't enter otherwise.", "votes": null }, { "id": "693514", "postDate": "12/12/2019 13:16:49", "content": "How about buy a large google drive, and use free colab? @init27
", "rawMarkdown": "How about buy a large google drive, and use free colab? @init27", "votes": null }, { "id": "693516", "postDate": "12/12/2019 13:18:26", "content": "@diegojohnson I think GCP might offer better/More powerful compute options than Colab.
", "rawMarkdown": "diegojohnson I think GCP might offer better/More powerful compute options than Colab.", "votes": null }, { "id": "693526", "postDate": "12/12/2019 13:28:46", "content": "π thanks
", "rawMarkdown": "π thanks", "votes": null }, { "id": "693592", "postDate": "12/12/2019 14:45:50", "content": "If they also provide an unlimited TPU quota, as they have done in some other competitions, then $300 will last for months.
", "rawMarkdown": "If they also provide an unlimited TPU quota, as they have done in some other competitions, then $300 will last for months.", "votes": null }, { "id": "693643", "postDate": "12/12/2019 15:58:43", "content": "I'd be curious to see TPU attempts + TPU Attempts using PyTorch. \nThinking ahead of myself-you'd also have to think about training on TPU and inferencing on GPU π€
", "rawMarkdown": "I'd be curious to see TPU attempts + TPU Attempts using PyTorch. \nThinking ahead of myself-you'd also have to think about training on TPU and inferencing on GPU π€", "votes": null }, { "id": "693738", "postDate": "12/12/2019 18:25:00", "content": "TPU training and GPU inferencing is what I did in a previous competition, using free GCP credits.
", "rawMarkdown": "TPU training and GPU inferencing is what I did in a previous competition, using free GCP credits.", "votes": null }, { "id": "693919", "postDate": "12/13/2019 00:36:23", "content": "Isn't the 300$ applicable only once per user per lifetime? I used mine long time ago.
", "rawMarkdown": "Isn't the 300$ applicable only once per user per lifetime? I used mine long time ago.", "votes": null }, { "id": "694213", "postDate": "12/13/2019 09:56:40", "content": "That's the initial sign-up credits. Then they sometimes give $300 top-ups for specific competitions, which is what this thread is about.
", "rawMarkdown": "That's the initial sign-up credits. Then they sometimes give $300 top-ups for specific competitions, which is what [this thread](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121226#693988) is about.", "votes": null }, { "id": "694221", "postDate": "12/13/2019 10:05:07", "content": "Is there a way to use large datasets like this with colab?
", "rawMarkdown": "Is there a way to use large datasets like this with colab?", "votes": null }, { "id": "694251", "postDate": "12/13/2019 10:52:20", "content": "I'd be grateful even if it's 300$ per team.
", "rawMarkdown": "I'd be grateful even if it's 300$ per team.", "votes": null }, { "id": "694256", "postDate": "12/13/2019 10:57:34", "content": "put the file in the google drive? @matthewmasters
", "rawMarkdown": "put the file in the google drive? @matthewmasters", "votes": null }, { "id": "694339", "postDate": "12/13/2019 12:53:23", "content": "would that work for getting credits?
\n\n\n\n>All participants will be able to request a minimum of $1,000 in AWS credits to get started, with additional awards granted in quantities of up to $10,000 as entries demonstrate viability or success in detecting deepfakes. Participants can visit www.aws.amazon.com/aws-ml-research-awards to learn more and request AWS credits.
", "rawMarkdown": "would that work for getting credits?\n\nhttps://aws.amazon.com/blogs/machine-learning/aws-supports-the-deepfake-detection-challenge-with-competition-data-and-aws-credits/\n\n>All participants will be able to request a minimum of $1,000 in AWS credits to get started, with additional awards granted in quantities of up to $10,000 as entries demonstrate viability or success in detecting deepfakes. Participants can visit www.aws.amazon.com/aws-ml-research-awards to learn more and request AWS credits.", "votes": null }, { "id": "694536", "postDate": "12/13/2019 18:43:36", "content": "But the default cap to Google Drive is 15GB. To house the full dataset, you'd need to subscribe at $10/month
", "rawMarkdown": "But the default cap to Google Drive is 15GB. To house the full dataset, you'd need to subscribe at $10/month", "votes": null }, { "id": "694552", "postDate": "12/13/2019 19:14:22", "content": "the 300$ does not include GPU quota though
", "rawMarkdown": "the 300$ does not include GPU quota though", "votes": null }, { "id": "695253", "postDate": "12/14/2019 21:04:45", "content": "Google colab is almost the same(if you use a single gpu and run time is less than 12 hours) as gcp because google colab also offer Tesla P100(randomly). You just need to create new colab file until you got Tesla P100.\nEdit: Colab might be better. You can run lots of notebook at the same time.
", "rawMarkdown": "Google colab is almost the same(if you use a single gpu and run time is less than 12 hours) as gcp because google colab also offer Tesla P100(randomly). You just need to create new colab file until you got Tesla P100.\nEdit: Colab might be better. You can run lots of notebook at the same time.", "votes": null }, { "id": "695999", "postDate": "12/16/2019 01:33:15", "content": "It does. You just have to kindly ask them for GPU number quota increase. I got mine quite quickly.
", "rawMarkdown": "It does. You just have to kindly ask them for GPU number quota increase. I got mine quite quickly.", "votes": null }, { "id": "696041", "postDate": "12/16/2019 03:30:15", "content": "Good job, mateπ @unkownhihi
", "rawMarkdown": "Good job, mateπ @unkownhihi", "votes": null }, { "id": "696660", "postDate": "12/16/2019 22:48:48", "content": "Thanks. Just sharing my own experience.
", "rawMarkdown": "Thanks. Just sharing my own experience.", "votes": null }, { "id": "696881", "postDate": "12/17/2019 07:21:57", "content": "@miwojc, Did you try applying?
", "rawMarkdown": "miwojc, Did you try applying?", "votes": null }, { "id": "697020", "postDate": "12/17/2019 11:28:29", "content": "yes i have. so far still awaiting response to my request.
", "rawMarkdown": "yes i have. so far still awaiting response to my request.", "votes": null }, { "id": "698685", "postDate": "12/19/2019 15:14:58", "content": "Credits are interesting, however I would be primarily interested in guidance on how to start with a cloud environment (GCP or AWS) and use it for this challenge. \nI assisted to the Google course \"TF with GCP\", but it does not allow me to reach the required level to be autonomous for a challenge like this one. Moreover, there won't be many kernels to get inspiration since most work will be done on kaggle.\nI will be looking whether there will be support - like starter kits for this challenge - to people who do not have experience of cloud environments.
", "rawMarkdown": "Credits are interesting, however I would be primarily interested in guidance on how to start with a cloud environment (GCP or AWS) and use it for this challenge. \nI assisted to the Google course \"TF with GCP\", but it does not allow me to reach the required level to be autonomous for a challenge like this one. Moreover, there won't be many kernels to get inspiration since most work will be done on kaggle.\nI will be looking whether there will be support - like starter kits for this challenge - to people who do not have experience of cloud environments.", "votes": null }, { "id": "698728", "postDate": "12/19/2019 16:28:58", "content": "I followed this and I'll figure other details as a need per go basis π
", "rawMarkdown": "I followed [this](https://course.fast.ai/start_gcp.html) and I'll figure other details as a need per go basis π", "votes": null }, { "id": "698730", "postDate": "12/19/2019 16:30:00", "content": "> Like starter kits for this challenge
\n\nI might take this up π΅
", "rawMarkdown": "> Like starter kits for this challenge\n\nI might take this up π΅", "votes": null }, { "id": "699505", "postDate": "12/20/2019 14:19:32", "content": "Thank you Sanyam !
", "rawMarkdown": "Thank you Sanyam !", "votes": null } ], "comments": [ { "id": 693415, "author_name": "suicaokhoailang", "author_url": "", "post_date": "12/12/2019 10:49:21", "content": "Don't want to sound overly pessimistic but $300 probably lasted like a week for me in a \"normal\" image competition, I don't even want to think about this one. The extra credits will help for sure it's hard to believe you can rely on it alone.
", "votes": null, "replies": [ { "id": 693420, "author_name": "init27", "author_url": "", "post_date": "12/12/2019 10:55:56", "content": "Agreed but I created this thread to remove the \"mental\" barrier for many of the people who wouldn't enter otherwise.
", "votes": null, "replies": [] }, { "id": 693592, "author_name": "kenkrige", "author_url": "", "post_date": "12/12/2019 14:45:50", "content": "If they also provide an unlimited TPU quota, as they have done in some other competitions, then $300 will last for months.
", "votes": null, "replies": [] }, { "id": 693643, "author_name": "init27", "author_url": "", "post_date": "12/12/2019 15:58:43", "content": "I'd be curious to see TPU attempts + TPU Attempts using PyTorch. \nThinking ahead of myself-you'd also have to think about training on TPU and inferencing on GPU π€
", "votes": null, "replies": [] }, { "id": 693738, "author_name": "kenkrige", "author_url": "", "post_date": "12/12/2019 18:25:00", "content": "TPU training and GPU inferencing is what I did in a previous competition, using free GCP credits.
", "votes": null, "replies": [] }, { "id": 694251, "author_name": "jesucristo", "author_url": "", "post_date": "12/13/2019 10:52:20", "content": "I'd be grateful even if it's 300$ per team.
", "votes": null, "replies": [] } ] }, { "id": 693514, "author_name": "diegojohnson", "author_url": "", "post_date": "12/12/2019 13:16:49", "content": "How about buy a large google drive, and use free colab? @init27
", "votes": null, "replies": [ { "id": 693516, "author_name": "init27", "author_url": "", "post_date": "12/12/2019 13:18:26", "content": "@diegojohnson I think GCP might offer better/More powerful compute options than Colab.
", "votes": null, "replies": [] }, { "id": 693526, "author_name": "diegojohnson", "author_url": "", "post_date": "12/12/2019 13:28:46", "content": "π thanks
", "votes": null, "replies": [] }, { "id": 694221, "author_name": "matthewmasters", "author_url": "", "post_date": "12/13/2019 10:05:07", "content": "Is there a way to use large datasets like this with colab?
", "votes": null, "replies": [] }, { "id": 694256, "author_name": "diegojohnson", "author_url": "", "post_date": "12/13/2019 10:57:34", "content": "put the file in the google drive? @matthewmasters
", "votes": null, "replies": [] }, { "id": 694536, "author_name": "matthewmasters", "author_url": "", "post_date": "12/13/2019 18:43:36", "content": "But the default cap to Google Drive is 15GB. To house the full dataset, you'd need to subscribe at $10/month
", "votes": null, "replies": [] }, { "id": 695253, "author_name": "unkownhihi", "author_url": "", "post_date": "12/14/2019 21:04:45", "content": "Google colab is almost the same(if you use a single gpu and run time is less than 12 hours) as gcp because google colab also offer Tesla P100(randomly). You just need to create new colab file until you got Tesla P100.\nEdit: Colab might be better. You can run lots of notebook at the same time.
", "votes": null, "replies": [] }, { "id": 696041, "author_name": "diegojohnson", "author_url": "", "post_date": "12/16/2019 03:30:15", "content": "Good job, mateπ @unkownhihi
", "votes": null, "replies": [] }, { "id": 696660, "author_name": "unkownhihi", "author_url": "", "post_date": "12/16/2019 22:48:48", "content": "Thanks. Just sharing my own experience.
", "votes": null, "replies": [] } ] }, { "id": 693919, "author_name": "rohitagarwal", "author_url": "", "post_date": "12/13/2019 00:36:23", "content": "Isn't the 300$ applicable only once per user per lifetime? I used mine long time ago.
", "votes": null, "replies": [ { "id": 694213, "author_name": "kenkrige", "author_url": "", "post_date": "12/13/2019 09:56:40", "content": "That's the initial sign-up credits. Then they sometimes give $300 top-ups for specific competitions, which is what this thread is about.
", "votes": null, "replies": [] } ] }, { "id": 694339, "author_name": "miwojc", "author_url": "", "post_date": "12/13/2019 12:53:23", "content": "would that work for getting credits?
\n\n\n\n>All participants will be able to request a minimum of $1,000 in AWS credits to get started, with additional awards granted in quantities of up to $10,000 as entries demonstrate viability or success in detecting deepfakes. Participants can visit www.aws.amazon.com/aws-ml-research-awards to learn more and request AWS credits.
", "votes": null, "replies": [ { "id": 696881, "author_name": "init27", "author_url": "", "post_date": "12/17/2019 07:21:57", "content": "@miwojc, Did you try applying?
", "votes": null, "replies": [] }, { "id": 697020, "author_name": "miwojc", "author_url": "", "post_date": "12/17/2019 11:28:29", "content": "yes i have. so far still awaiting response to my request.
", "votes": null, "replies": [] } ] }, { "id": 694552, "author_name": "d4v1d3", "author_url": "", "post_date": "12/13/2019 19:14:22", "content": "the 300$ does not include GPU quota though
", "votes": null, "replies": [ { "id": 695999, "author_name": "snufkin77", "author_url": "", "post_date": "12/16/2019 01:33:15", "content": "It does. You just have to kindly ask them for GPU number quota increase. I got mine quite quickly.
", "votes": null, "replies": [] } ] }, { "id": 698685, "author_name": "jeanlucdelassus", "author_url": "", "post_date": "12/19/2019 15:14:58", "content": "Credits are interesting, however I would be primarily interested in guidance on how to start with a cloud environment (GCP or AWS) and use it for this challenge. \nI assisted to the Google course \"TF with GCP\", but it does not allow me to reach the required level to be autonomous for a challenge like this one. Moreover, there won't be many kernels to get inspiration since most work will be done on kaggle.\nI will be looking whether there will be support - like starter kits for this challenge - to people who do not have experience of cloud environments.
", "votes": null, "replies": [ { "id": 698728, "author_name": "init27", "author_url": "", "post_date": "12/19/2019 16:28:58", "content": "I followed this and I'll figure other details as a need per go basis π
", "votes": null, "replies": [] }, { "id": 698730, "author_name": "init27", "author_url": "", "post_date": "12/19/2019 16:30:00", "content": "> Like starter kits for this challenge
\n\nI might take this up π΅
", "votes": null, "replies": [] }, { "id": 699505, "author_name": "jeanlucdelassus", "author_url": "", "post_date": "12/20/2019 14:19:32", "content": "Thank you Sanyam !
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "693257": "I see a lot of folks getting worried because the data is quite big and you'd definitely need a good amount of hardware resources to be competitive in this comp.\n\nGCP Offers 300$ worth of credits, for a team of 8 members that equals to 2400$ worth of compute credits + @juliaelliott has confirmed [Here](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121226#692992) that the hosts will provide credits soon. \n\nThat should be good enough to compete IMO. \nPlease don't let the data size discourage you. Good Luck! π΅", "693415": "Don't want to sound overly pessimistic but $300 probably lasted like a week for me in a \"normal\" image competition, I don't even want to think about this one. The extra credits will help for sure it's hard to believe you can rely on it alone.", "693420": "Agreed but I created this thread to remove the \"mental\" barrier for many of the people who wouldn't enter otherwise.", "693514": "How about buy a large google drive, and use free colab? @init27", "693516": "diegojohnson I think GCP might offer better/More powerful compute options than Colab.", "693526": "π thanks", "693592": "If they also provide an unlimited TPU quota, as they have done in some other competitions, then $300 will last for months.", "693643": "I'd be curious to see TPU attempts + TPU Attempts using PyTorch. \nThinking ahead of myself-you'd also have to think about training on TPU and inferencing on GPU π€", "693738": "TPU training and GPU inferencing is what I did in a previous competition, using free GCP credits.", "693919": "Isn't the 300$ applicable only once per user per lifetime? I used mine long time ago.", "694213": "That's the initial sign-up credits. Then they sometimes give $300 top-ups for specific competitions, which is what [this thread](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121226#693988) is about.", "694221": "Is there a way to use large datasets like this with colab?", "694251": "I'd be grateful even if it's 300$ per team.", "694256": "put the file in the google drive? @matthewmasters", "694339": "would that work for getting credits?\n\nhttps://aws.amazon.com/blogs/machine-learning/aws-supports-the-deepfake-detection-challenge-with-competition-data-and-aws-credits/\n\n>All participants will be able to request a minimum of $1,000 in AWS credits to get started, with additional awards granted in quantities of up to $10,000 as entries demonstrate viability or success in detecting deepfakes. Participants can visit www.aws.amazon.com/aws-ml-research-awards to learn more and request AWS credits.", "694536": "But the default cap to Google Drive is 15GB. To house the full dataset, you'd need to subscribe at $10/month", "694552": "the 300$ does not include GPU quota though", "695253": "Google colab is almost the same(if you use a single gpu and run time is less than 12 hours) as gcp because google colab also offer Tesla P100(randomly). You just need to create new colab file until you got Tesla P100.\nEdit: Colab might be better. You can run lots of notebook at the same time.", "695999": "It does. You just have to kindly ask them for GPU number quota increase. I got mine quite quickly.", "696041": "Good job, mateπ @unkownhihi", "696660": "Thanks. Just sharing my own experience.", "696881": "miwojc, Did you try applying?", "697020": "yes i have. so far still awaiting response to my request.", "698685": "Credits are interesting, however I would be primarily interested in guidance on how to start with a cloud environment (GCP or AWS) and use it for this challenge. \nI assisted to the Google course \"TF with GCP\", but it does not allow me to reach the required level to be autonomous for a challenge like this one. Moreover, there won't be many kernels to get inspiration since most work will be done on kaggle.\nI will be looking whether there will be support - like starter kits for this challenge - to people who do not have experience of cloud environments.", "698728": "I followed [this](https://course.fast.ai/start_gcp.html) and I'll figure other details as a need per go basis π", "698730": "> Like starter kits for this challenge\n\nI might take this up π΅", "699505": "Thank you Sanyam !" }, "source": "meta" }