there are some custom scripts for gcs access. How to access that from colab\nFirst of all, is it allowed in this comp to train outside?
", "messages": [ { "id": "746931", "postDate": "02/15/2020 18:47:23", "content": "there are some custom scripts for gcs access. How to access that from colab\nFirst of all, is it allowed in this comp to train outside?
", "rawMarkdown": "there are some custom scripts for gcs access. How to access that from colab\nFirst of all, is it allowed in this comp to train outside?", "votes": null }, { "id": "747008", "postDate": "02/15/2020 21:01:32", "content": "I don't think it's possible to access Kaggle GCS from Google Colab but you can always move the competition's data to your GCS and work with it.
\n\nIn regards of training outside, I think it's allowed as long as you make your training and inference codes public following the end of the competition.
\n\n\n", "rawMarkdown": "I don't think it's possible to access Kaggle GCS from Google Colab but you can always move the competition's data to your GCS and work with it.\n\nIn regards of training outside, I think it's allowed as long as you make your training and inference codes public following the end of the competition.\n\n> To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition. Prizes are subject to fulfillment of Winners Obligations as specified in the competition's rules.", "votes": null }, { "id": "749569", "postDate": "02/18/2020 19:35:14", "content": "To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition. Prizes are subject to fulfillment of Winners Obligations as specified in the competition's rules.
\n
This is a friendly \"playground\" competition so you can do whatever you want. The only restriction is for prizes. To qualify for the prize spots, the training must happen in the Kaggle Notebook. Training externally and submitting only the results will not make you eligible for prizes.
\n\nAlso, Colab has TPU v2's right now. Kaggle offers TPU v3's which are roughly 2x as powerful.
\n\nIf you want to do some Colab training on the side, you can use the GCS bucket path returned by KaggleDatasets().get_gcs_path(). It's a cache so it will be wiped eventually but it should be good for a couple of days. Training might be slower though, if the TPU and the GCS bucket are not in the same region. On Kaggle, KaggleDatasets().get_gcs_path() always gets you a bucket right next to the TPU you have been granted.
Another option is to use TPUs on GCP. On GCP you have to pay for the resources. You can provision a Cloud AI Platform Notebook instance with a TPU using this script: create-tpu-deep-learning-vm.sh\n\ngcloud init # to set up your project, region, ...\n\nthen\n\n./create-tpu-deep-learning-vm.sh choose-a-name-for-your-vm --tpu-type v3-8\n
Why I want to use colab is test some of the things because kaggle has 30 hours of restrictions.\n@mgornergoogle My second question, when you said training externally is not eligble for prizes but if I use kaggle kernel to train different models and then use those saved models in different kernels ( basically an inference Kernel), is it valid since the kaggle platform is used.? Or 3 hours notebook restrictions is for training as well as my inference.
", "rawMarkdown": "Why I want to use colab is test some of the things because kaggle has 30 hours of restrictions.\n@mgornergoogle My second question, when you said training externally is not eligble for prizes but if I use kaggle kernel to train different models and then use those saved models in different kernels ( basically an inference Kernel), is it valid since the kaggle platform is used.? Or 3 hours notebook restrictions is for training as well as my inference.", "votes": null }, { "id": "750990", "postDate": "02/19/2020 22:23:36", "content": "To be eligible for prizes, the training has to happen in the Kaggle kernel. An inference only notebook is not eligible.
", "rawMarkdown": "To be eligible for prizes, the training has to happen in the Kaggle kernel. An inference only notebook is not eligible.", "votes": null }, { "id": "752328", "postDate": "02/20/2020 22:06:45", "content": "A correction to Martin's statement. Training does not have to be done in Kaggle notebooks. But per the Prizes page, it must have been trained on TPUs, although that can happen outside of Kaggle. We will be validating that is the case when reviewing the candidates for these awards.
\n\n\n", "rawMarkdown": "A correction to Martin's statement. Training **does not** have to be done *in* Kaggle notebooks. But per the [Prizes page](https://www.kaggle.com/c/flower-classification-with-tpus/overview/prizes), it must have been trained on TPUs, although that can happen outside of Kaggle. We will be validating that is the case when reviewing the candidates for these awards.\n\n> To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition.", "votes": null }, { "id": "922509", "postDate": "07/10/2020 06:45:54", "content": "To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition.
\n
Hi Martin,
\n\nI have got the exact path in GCS bucket path returned by KaggleDatasets().get_gcs_path().\nHow do I use this path in Colab as suggested by you ?
\n\nThanks
", "rawMarkdown": "Hi Martin,\n\nI have got the exact path in GCS bucket path returned by KaggleDatasets().get_gcs_path().\nHow do I use this path in Colab as suggested by you ?\n\nThanks", "votes": null }, { "id": "926710", "postDate": "07/12/2020 23:19:38", "content": "I am having the same issue.
", "rawMarkdown": "I am having the same issue.", "votes": null } ], "comments": [ { "id": 747008, "author_name": "msheriey", "author_url": "", "post_date": "02/15/2020 21:01:32", "content": "I don't think it's possible to access Kaggle GCS from Google Colab but you can always move the competition's data to your GCS and work with it.
\n\nIn regards of training outside, I think it's allowed as long as you make your training and inference codes public following the end of the competition.
\n\n\n", "votes": null, "replies": [] }, { "id": 749569, "author_name": "mgorner", "author_url": "", "post_date": "02/18/2020 19:35:14", "content": "To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition. Prizes are subject to fulfillment of Winners Obligations as specified in the competition's rules.
\n
This is a friendly \"playground\" competition so you can do whatever you want. The only restriction is for prizes. To qualify for the prize spots, the training must happen in the Kaggle Notebook. Training externally and submitting only the results will not make you eligible for prizes.
\n\nAlso, Colab has TPU v2's right now. Kaggle offers TPU v3's which are roughly 2x as powerful.
\n\nIf you want to do some Colab training on the side, you can use the GCS bucket path returned by KaggleDatasets().get_gcs_path(). It's a cache so it will be wiped eventually but it should be good for a couple of days. Training might be slower though, if the TPU and the GCS bucket are not in the same region. On Kaggle, KaggleDatasets().get_gcs_path() always gets you a bucket right next to the TPU you have been granted.
Another option is to use TPUs on GCP. On GCP you have to pay for the resources. You can provision a Cloud AI Platform Notebook instance with a TPU using this script: create-tpu-deep-learning-vm.sh\n\ngcloud init # to set up your project, region, ...\n\nthen\n\n./create-tpu-deep-learning-vm.sh choose-a-name-for-your-vm --tpu-type v3-8\n
Why I want to use colab is test some of the things because kaggle has 30 hours of restrictions.\n@mgornergoogle My second question, when you said training externally is not eligble for prizes but if I use kaggle kernel to train different models and then use those saved models in different kernels ( basically an inference Kernel), is it valid since the kaggle platform is used.? Or 3 hours notebook restrictions is for training as well as my inference.
", "votes": null, "replies": [] }, { "id": 750990, "author_name": "mgorner", "author_url": "", "post_date": "02/19/2020 22:23:36", "content": "To be eligible for prizes, the training has to happen in the Kaggle kernel. An inference only notebook is not eligible.
", "votes": null, "replies": [] }, { "id": 752328, "author_name": "juliaelliott", "author_url": "", "post_date": "02/20/2020 22:06:45", "content": "A correction to Martin's statement. Training does not have to be done in Kaggle notebooks. But per the Prizes page, it must have been trained on TPUs, although that can happen outside of Kaggle. We will be validating that is the case when reviewing the candidates for these awards.
\n\n\n", "votes": null, "replies": [] }, { "id": 922509, "author_name": "watzisname", "author_url": "", "post_date": "07/10/2020 06:45:54", "content": "To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition.
\n
Hi Martin,
\n\nI have got the exact path in GCS bucket path returned by KaggleDatasets().get_gcs_path().\nHow do I use this path in Colab as suggested by you ?
\n\nThanks
", "votes": null, "replies": [] }, { "id": 926710, "author_name": "mwazirali", "author_url": "", "post_date": "07/12/2020 23:19:38", "content": "I am having the same issue.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "746931": "there are some custom scripts for gcs access. How to access that from colab\nFirst of all, is it allowed in this comp to train outside?", "747008": "I don't think it's possible to access Kaggle GCS from Google Colab but you can always move the competition's data to your GCS and work with it.\n\nIn regards of training outside, I think it's allowed as long as you make your training and inference codes public following the end of the competition.\n\n> To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition. Prizes are subject to fulfillment of Winners Obligations as specified in the competition's rules.", "749569": "This is a friendly \"playground\" competition so you can do whatever you want. The only restriction is for prizes. To qualify for the prize spots, the training must happen in the Kaggle Notebook. Training externally and submitting only the results will not make you eligible for prizes.\n\nAlso, Colab has TPU v2's right now. Kaggle offers TPU v3's which are roughly 2x as powerful.\n\nIf you want to do some Colab training on the side, you can use the GCS bucket path returned by `KaggleDatasets().get_gcs_path()`. It's a cache so it will be wiped eventually but it should be good for a couple of days. Training might be slower though, if the TPU and the GCS bucket are not in the same region. On Kaggle, `KaggleDatasets().get_gcs_path()` always gets you a bucket right next to the TPU you have been granted. \n\nAnother option is to use TPUs on GCP. On GCP you have to pay for the resources. You can provision a Cloud AI Platform Notebook instance with a TPU using this script: [create-tpu-deep-learning-vm.sh](https://raw.githubusercontent.com/GoogleCloudPlatform/training-data-analyst/master/courses/fast-and-lean-data-science/create-tpu-deep-learning-vm.sh)\n```\ngcloud init # to set up your project, region, ...\n```\nthen\n```\n./create-tpu-deep-learning-vm.sh choose-a-name-for-your-vm --tpu-type v3-8\n```", "750079": "Why I want to use colab is test some of the things because kaggle has 30 hours of restrictions.\n@mgornergoogle My second question, when you said training externally is not eligble for prizes but if I use kaggle kernel to train different models and then use those saved models in different kernels ( basically an inference Kernel), is it valid since the kaggle platform is used.? Or 3 hours notebook restrictions is for training as well as my inference.", "750990": "To be eligible for prizes, the training has to happen in the Kaggle kernel. An inference only notebook is not eligible.", "752328": "A correction to Martin's statement. Training **does not** have to be done *in* Kaggle notebooks. But per the [Prizes page](https://www.kaggle.com/c/flower-classification-with-tpus/overview/prizes), it must have been trained on TPUs, although that can happen outside of Kaggle. We will be validating that is the case when reviewing the candidates for these awards.\n\n> To be eligible for these prizes, your submission must generate predictions from machine learning code fully run on TPUs for both training and inference. That code must be made public following the end of the competition.", "922509": "Hi Martin,\n\nI have got the exact path in GCS bucket path returned by KaggleDatasets().get_gcs_path().\nHow do I use this path in Colab as suggested by you ?\n\nThanks", "926710": "I am having the same issue." }, "source": "meta" }