Has anyone managed to run notebooks on colab or tpu in reasonable run time? For me running notebook on colab takes more time than the same notebook on Kaggle. And I couldn’t get it running on tpu. Please share your approach so I can learn from it, thanks!
", "messages": [ { "id": 1800476, "postDate": "2022-05-25T02:02:10.323Z", "content": "I used Colab Pro+ (V100). Training with competition data on V100 GPU took about 5 minutes per epoch for tf_efficientnet_b0_ns. Competition data was loaded with librosa and transformed to mel spectrogram with torchaudio.
", "rawMarkdown": "I used Colab Pro+ (V100). Training with competition data on V100 GPU took about 5 minutes per epoch for tf_efficientnet_b0_ns. Competition data was loaded with librosa and transformed to mel spectrogram with torchaudio.", "votes": 1, "replies": [ { "id": 1800488, "postDate": "2022-05-25T02:13:15.410Z", "content": "Thanks for your response. I used colab with p100, I tried different versions of torch, transformers, timm but could get it faster than 1 hour per epoch. Have you used specific versions/configurations of these libraries or any preparation setup? Are you planning to share you training code, I am interested only in colab setup.
", "rawMarkdown": "Thanks for your response. I used colab with p100, I tried different versions of torch, transformers, timm but could get it faster than 1 hour per epoch. Have you used specific versions/configurations of these libraries or any preparation setup? Are you planning to share you training code, I am interested only in colab setup. " }, { "id": 1800698, "postDate": "2022-05-25T06:35:01.933Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1801137, "postDate": "2022-05-25T13:46:38.287Z", "content": "Did you transfer your google drive files to the /content folder first?
\nI did this and was able to train ~15min/epoch using P100 GPU
", "rawMarkdown": "Did you transfer your google drive files to the /content folder first?\n\nI did this and was able to train ~15min/epoch using P100 GPU", "votes": 1 }, { "id": 1801590, "postDate": "2022-05-25T23:26:51.210Z", "content": "Hi, thanks for your answer. I did transfer the dataset in numpy format to the google drive. And accessed them by connecting to the drive and providing the path to the dataset. One hour notebook running time I said previously is actually related to the one epoch running/ training time.
", "rawMarkdown": "Hi, thanks for your answer. I did transfer the dataset in numpy format to the google drive. And accessed them by connecting to the drive and providing the path to the dataset. One hour notebook running time I said previously is actually related to the one epoch running/ training time. " }, { "id": 1801595, "postDate": "2022-05-25T23:44:54.437Z", "content": "I recommend mel spectrogram calculations in mini-batch processing with torchaudio, which is fast.
", "rawMarkdown": "I recommend mel spectrogram calculations in mini-batch processing with torchaudio, which is fast." }, { "id": 1802525, "postDate": "2022-05-26T20:16:10.333Z", "content": "Hey @azamat25 I don't know if I understood it well.
\nWhat I'm saying is that google drive has a very bad input/output speed. You must not get your files directly from drive.
\nYou should transfer them to /content folder first, so you have them in google virtual machine so you can access these files much faster from the path '/content/…..'
", "rawMarkdown": "Hey @azamat25 I don't know if I understood it well.\n\nWhat I'm saying is that google drive has a very bad input/output speed. You must not get your files directly from drive.\n\nYou should transfer them to /content folder first, so you have them in google virtual machine so you can access these files much faster from the path '/content/.....'", "votes": 1 }, { "id": 1802578, "postDate": "2022-05-26T23:09:38.503Z", "content": "Yes, it is important to place the input data in a local directory. I am using code like below, which downloads and unpacks the data directly from kaggle to a local directory via the kaggle API. You can copy numpy archive files from google drive instead.
\nimport os\nimport sys\n\nif 'google.colab' in sys.modules:\n !echo CPU model : `awk -F: '/^model name/{print $2; exit}' < /proc/cpuinfo`\n !grep cpu.cores /proc/cpuinfo | sort -u\n !echo \"logical processors : \" `awk '/^processor/{n++} END{print n}' < /proc/cpuinfo`\n !head -3 /proc/meminfo\n !nvidia-smi\n from google.colab import drive\n drive.mount('/content/drive')\n if not os.path.exists('../output'):\n !pip install timm\n !pip install --upgrade --force-reinstall --no-deps kaggle > /dev/null\n !mkdir ~/.kaggle\n !cp \"/content/drive/MyDrive/kaggle/kaggle.json\" ~/.kaggle/\n !mkdir -p \"../input/birdclef-2022/\"\n !kaggle competitions download -c birdclef-2022\n !unzip -o \"./birdclef-2022.zip\" -d \"../input/birdclef-2022\" > /dev/null\n !ln -s \"/content/drive/MyDrive/kaggle/BirdCLEF2022/output/\" \"../output\"\nHas anyone managed to run notebooks on colab or tpu in reasonable run time? For me running notebook on colab takes more time than the same notebook on Kaggle. And I couldn’t get it running on tpu. Please share your approach so I can learn from it, thanks!
", "rawMarkdown": "Has anyone managed to run notebooks on colab or tpu in reasonable run time? For me running notebook on colab takes more time than the same notebook on Kaggle. And I couldn’t get it running on tpu. Please share your approach so I can learn from it, thanks!" } ], "comments": [ { "id": 1800476, "author_name": "S. Tomizawa", "author_url": "", "post_date": "2022-05-25T02:02:10.323000", "content": "I used Colab Pro+ (V100). Training with competition data on V100 GPU took about 5 minutes per epoch for tf_efficientnet_b0_ns. Competition data was loaded with librosa and transformed to mel spectrogram with torchaudio.
", "votes": 1, "replies": [ { "id": 1800488, "author_name": "Azamat Tolegen", "author_url": "", "post_date": "2022-05-25T02:13:15.410000", "content": "Thanks for your response. I used colab with p100, I tried different versions of torch, transformers, timm but could get it faster than 1 hour per epoch. Have you used specific versions/configurations of these libraries or any preparation setup? Are you planning to share you training code, I am interested only in colab setup.
", "votes": 0, "replies": [] }, { "id": 1800698, "author_name": "", "author_url": "", "post_date": "2022-05-25T06:35:01.933000", "content": "", "votes": 0, "replies": [] }, { "id": 1801137, "author_name": "HinePo", "author_url": "", "post_date": "2022-05-25T13:46:38.287000", "content": "Did you transfer your google drive files to the /content folder first?
\nI did this and was able to train ~15min/epoch using P100 GPU
", "votes": 1, "replies": [] }, { "id": 1801590, "author_name": "Azamat Tolegen", "author_url": "", "post_date": "2022-05-25T23:26:51.210000", "content": "Hi, thanks for your answer. I did transfer the dataset in numpy format to the google drive. And accessed them by connecting to the drive and providing the path to the dataset. One hour notebook running time I said previously is actually related to the one epoch running/ training time.
", "votes": 0, "replies": [] }, { "id": 1801595, "author_name": "S. Tomizawa", "author_url": "", "post_date": "2022-05-25T23:44:54.437000", "content": "I recommend mel spectrogram calculations in mini-batch processing with torchaudio, which is fast.
", "votes": 0, "replies": [] }, { "id": 1802525, "author_name": "HinePo", "author_url": "", "post_date": "2022-05-26T20:16:10.333000", "content": "Hey @azamat25 I don't know if I understood it well.
\nWhat I'm saying is that google drive has a very bad input/output speed. You must not get your files directly from drive.
\nYou should transfer them to /content folder first, so you have them in google virtual machine so you can access these files much faster from the path '/content/…..'
", "votes": 1, "replies": [] }, { "id": 1802578, "author_name": "S. Tomizawa", "author_url": "", "post_date": "2022-05-26T23:09:38.503000", "content": "Yes, it is important to place the input data in a local directory. I am using code like below, which downloads and unpacks the data directly from kaggle to a local directory via the kaggle API. You can copy numpy archive files from google drive instead.
\nimport os\nimport sys\n\nif 'google.colab' in sys.modules:\n !echo CPU model : `awk -F: '/^model name/{print $2; exit}' < /proc/cpuinfo`\n !grep cpu.cores /proc/cpuinfo | sort -u\n !echo \"logical processors : \" `awk '/^processor/{n++} END{print n}' < /proc/cpuinfo`\n !head -3 /proc/meminfo\n !nvidia-smi\n from google.colab import drive\n drive.mount('/content/drive')\n if not os.path.exists('../output'):\n !pip install timm\n !pip install --upgrade --force-reinstall --no-deps kaggle > /dev/null\n !mkdir ~/.kaggle\n !cp \"/content/drive/MyDrive/kaggle/kaggle.json\" ~/.kaggle/\n !mkdir -p \"../input/birdclef-2022/\"\n !kaggle competitions download -c birdclef-2022\n !unzip -o \"./birdclef-2022.zip\" -d \"../input/birdclef-2022\" > /dev/null\n !ln -s \"/content/drive/MyDrive/kaggle/BirdCLEF2022/output/\" \"../output\"\n