{ "id": 307935, "title": "How can I use google colab for code competition?", "url": "/competitions/birdclef-2022/discussion/307935", "author_name": "", "post_date": "2022-02-16T09:39:41.180738600Z", "votes": 2, "comment_count": 2, "views": 0, "content": "

I usually use the google colab for the simple competitions. There only need to upload one submission csv file. But here I need to upload the full notebook. Is there any easy way to do this with the kaggle API? Or any reasonable solution?

\n

My temporary solution:
\nI check to see if I'm in the kaggle kernel. If not, I think that I am in google colab.

\n

env=\"kaggle\" if 'kaggle' in os.environ['PYTHONPATH'] else \"colab\"

\n

I then set the environment according to where I am. So I can develop my code on colab and it works well when I submit my notebook on kaggle kernel. I use my google drive on colab and the attached files in kaggle kernel.

\n

if(env=='colab'):

\n
\n

os.environ['KAGGLE_USERNAME'] = 'YOUR_KAGGLE_USERNAME'\nos.environ['KAGGLE_KEY'] = 'YOUR_KAGGLE_APIKEY'

\n
\n
\n

work_dir=\"/kaggle/working\"\nif(env==\"colab\"):

\n
\n

if os.path.isdir(\"/kaggle\") == False:

\n
\n

os.mkdir(\"/kaggle\")

\n
\n

if os.path.isdir(work_dir) == False:

\n
\n

os.mkdir(work_dir)\nos.chdir(work_dir)

\n
\n
\n
\n

input_dir=\"/kaggle/input\"\nif(env==\"colab\"):

\n
\n

from google.colab import drive\ndrive.mount(input_dir, force_remount=True)\ninput_dir+=\"/MyDrive/YOUR_PATH_IN_GDRIVE\"

\n
\n

else:

\n
\n

input_dir+=\"/birdclef-2022\"

\n
\n

I hope this helps you too!

", "messages": [ { "id": "1692859", "postDate": "02/16/2022 09:39:41", "content": "

I usually use the google colab for the simple competitions. There only need to upload one submission csv file. But here I need to upload the full notebook. Is there any easy way to do this with the kaggle API? Or any reasonable solution?

\n

My temporary solution:
\nI check to see if I'm in the kaggle kernel. If not, I think that I am in google colab.

\n

env=\"kaggle\" if 'kaggle' in os.environ['PYTHONPATH'] else \"colab\"

\n

I then set the environment according to where I am. So I can develop my code on colab and it works well when I submit my notebook on kaggle kernel. I use my google drive on colab and the attached files in kaggle kernel.

\n

if(env=='colab'):

\n
\n

os.environ['KAGGLE_USERNAME'] = 'YOUR_KAGGLE_USERNAME'\nos.environ['KAGGLE_KEY'] = 'YOUR_KAGGLE_APIKEY'

\n
\n
\n

work_dir=\"/kaggle/working\"\nif(env==\"colab\"):

\n
\n

if os.path.isdir(\"/kaggle\") == False:

\n
\n

os.mkdir(\"/kaggle\")

\n
\n

if os.path.isdir(work_dir) == False:

\n
\n

os.mkdir(work_dir)\nos.chdir(work_dir)

\n
\n
\n
\n

input_dir=\"/kaggle/input\"\nif(env==\"colab\"):

\n
\n

from google.colab import drive\ndrive.mount(input_dir, force_remount=True)\ninput_dir+=\"/MyDrive/YOUR_PATH_IN_GDRIVE\"

\n
\n

else:

\n
\n

input_dir+=\"/birdclef-2022\"

\n
\n

I hope this helps you too!

", "rawMarkdown": "I usually use the google colab for the simple competitions. There only need to upload one submission csv file. But here I need to upload the full notebook. Is there any easy way to do this with the kaggle API? Or any reasonable solution?\n\n**My temporary solution:**\nI check to see if I'm in the kaggle kernel. If not, I think that I am in google colab.\n\n`env=\"kaggle\" if 'kaggle' in os.environ['PYTHONPATH'] else \"colab\"`\n\nI then set the environment according to where I am. So I can develop my code on colab and it works well when I submit my notebook on kaggle kernel. I use my google drive on colab and the attached files in kaggle kernel.\n\n`if(env=='colab'):`\n>`os.environ['KAGGLE_USERNAME'] = 'YOUR_KAGGLE_USERNAME'\nos.environ['KAGGLE_KEY'] = 'YOUR_KAGGLE_APIKEY'`\n\n\n****\n\n`work_dir=\"/kaggle/working\"\nif(env==\"colab\"):`\n>`if os.path.isdir(\"/kaggle\") == False:`\n>>`os.mkdir(\"/kaggle\")`\n\n>`if os.path.isdir(work_dir) == False:`\n>>`os.mkdir(work_dir)\nos.chdir(work_dir)`\n\n****\n\n`input_dir=\"/kaggle/input\"\nif(env==\"colab\"):`\n>`from google.colab import drive\ndrive.mount(input_dir, force_remount=True)\ninput_dir+=\"/MyDrive/YOUR_PATH_IN_GDRIVE\"`\n\n`else:`\n>`input_dir+=\"/birdclef-2022\"`\n\n**I hope this helps you too!**", "votes": null }, { "id": "1692962", "postDate": "02/16/2022 11:18:52", "content": "

@ambrusattila its usually a good idea to find some resources shared by the hosts or other participants.

\n

There have been many similar competitions in the past, AFAIK Kaggle notebooks and Google Colab use a similar infrastructure so if you run anything on notebooks, it won't be a different experience.

\n

You can also train your models on colab and upload the model weights here for inference :)

\n

Good Luck!

", "rawMarkdown": "ambrusattila its usually a good idea to find some resources shared by the hosts or other participants. \n\nThere have been many similar competitions in the past, AFAIK Kaggle notebooks and Google Colab use a similar infrastructure so if you run anything on notebooks, it won't be a different experience. \n\nYou can also train your models on colab and upload the model weights here for inference :) \n\nGood Luck!", "votes": null }, { "id": "1692987", "postDate": "02/16/2022 11:34:49", "content": "

Files can be attached to the kaggle kernel, but for the colab, they must be copied to the colab. The files are also stored elsewhere. If I upload a notebook made there without any changes, it won't run well because of the different handling of the files. Or will?

\n

And the library versions are also different on the two systems. Others are the installed dependencies.

", "rawMarkdown": "Files can be attached to the kaggle kernel, but for the colab, they must be copied to the colab. The files are also stored elsewhere. If I upload a notebook made there without any changes, it won't run well because of the different handling of the files. Or will?\n\nAnd the library versions are also different on the two systems. Others are the installed dependencies.", "votes": null } ], "comments": [ { "id": 1692962, "author_name": "init27", "author_url": "", "post_date": "02/16/2022 11:18:52", "content": "

@ambrusattila its usually a good idea to find some resources shared by the hosts or other participants.

\n

There have been many similar competitions in the past, AFAIK Kaggle notebooks and Google Colab use a similar infrastructure so if you run anything on notebooks, it won't be a different experience.

\n

You can also train your models on colab and upload the model weights here for inference :)

\n

Good Luck!

", "votes": null, "replies": [ { "id": 1692987, "author_name": "ambrusattila", "author_url": "", "post_date": "02/16/2022 11:34:49", "content": "

Files can be attached to the kaggle kernel, but for the colab, they must be copied to the colab. The files are also stored elsewhere. If I upload a notebook made there without any changes, it won't run well because of the different handling of the files. Or will?

\n

And the library versions are also different on the two systems. Others are the installed dependencies.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1692859": "I usually use the google colab for the simple competitions. There only need to upload one submission csv file. But here I need to upload the full notebook. Is there any easy way to do this with the kaggle API? Or any reasonable solution?\n\n**My temporary solution:**\nI check to see if I'm in the kaggle kernel. If not, I think that I am in google colab.\n\n`env=\"kaggle\" if 'kaggle' in os.environ['PYTHONPATH'] else \"colab\"`\n\nI then set the environment according to where I am. So I can develop my code on colab and it works well when I submit my notebook on kaggle kernel. I use my google drive on colab and the attached files in kaggle kernel.\n\n`if(env=='colab'):`\n>`os.environ['KAGGLE_USERNAME'] = 'YOUR_KAGGLE_USERNAME'\nos.environ['KAGGLE_KEY'] = 'YOUR_KAGGLE_APIKEY'`\n\n\n****\n\n`work_dir=\"/kaggle/working\"\nif(env==\"colab\"):`\n>`if os.path.isdir(\"/kaggle\") == False:`\n>>`os.mkdir(\"/kaggle\")`\n\n>`if os.path.isdir(work_dir) == False:`\n>>`os.mkdir(work_dir)\nos.chdir(work_dir)`\n\n****\n\n`input_dir=\"/kaggle/input\"\nif(env==\"colab\"):`\n>`from google.colab import drive\ndrive.mount(input_dir, force_remount=True)\ninput_dir+=\"/MyDrive/YOUR_PATH_IN_GDRIVE\"`\n\n`else:`\n>`input_dir+=\"/birdclef-2022\"`\n\n**I hope this helps you too!**", "1692962": "ambrusattila its usually a good idea to find some resources shared by the hosts or other participants. \n\nThere have been many similar competitions in the past, AFAIK Kaggle notebooks and Google Colab use a similar infrastructure so if you run anything on notebooks, it won't be a different experience. \n\nYou can also train your models on colab and upload the model weights here for inference :) \n\nGood Luck!", "1692987": "Files can be attached to the kaggle kernel, but for the colab, they must be copied to the colab. The files are also stored elsewhere. If I upload a notebook made there without any changes, it won't run well because of the different handling of the files. Or will?\n\nAnd the library versions are also different on the two systems. Others are the installed dependencies." }, "source": "meta" }