{
  "id": 289261,
  "title": "Training CNN Runtime comparison Kaggle vs. Google Colab\t",
  "url": "/competitions/sartorius-cell-instance-segmentation/discussion/289261",
  "author_name": "Andreas Horlbeck",
  "post_date": "2021-11-19T12:53:48.314000",
  "votes": 10,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hey,    </p>\n<p>because of the limited gpu runtime of kaggle and the paid memberships I wondered    <br>\nhow much benefit would there be for the training of DNNs  if I kaggle on Google Colab.    <br>\nSo I decided to really measure the performance on a single CNN classification     <br>\nfrom the rsna competition some weeks ago.    <br>\n<a href=\"https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification\" target=\"_blank\">https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification</a>    </p>\n<p>The setting was as follows;    </p>\n<ul>\n<li>100.000 images (png grayscale,  (train 80%, validation 20%)    </li>\n<li>images resized to  shape of (64,64)    </li>\n<li>CNN with roughly 120.000 parameters    </li>\n<li>15 epochs    </li>\n</ul>\n<p><strong>Results:</strong></p>\n<table>\n<thead>\n<tr>\n<th>System</th>\n<th>GPU</th>\n<th>Runtime in s</th>\n<th>% to Kaggle</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Kaggle</td>\n<td>Tesla P 100</td>\n<td>937</td>\n<td></td>\n</tr>\n<tr>\n<td>Colab free</td>\n<td>Tesla K 80</td>\n<td>739</td>\n<td>79%</td>\n</tr>\n<tr>\n<td>Colab Pro</td>\n<td>Tesla P 100</td>\n<td>510</td>\n<td>54%</td>\n</tr>\n<tr>\n<td>Colab Pro +</td>\n<td>Tesla V 100</td>\n<td>425</td>\n<td>45%</td>\n</tr>\n</tbody>\n</table>\n<p>For visualization visit here:<br>\n<a href=\"https://www.kaggle.com/andreashorlbeck/results-kaggle-vs-colab?scriptVersionId=79503243\" target=\"_blank\">https://www.kaggle.com/andreashorlbeck/results-kaggle-vs-colab?scriptVersionId=79503243</a></p>\n<p>According to the runtime measurements Colab was really faster to train than Kaggle.    <br>\nSome users said that there is a chance to get also a V100 when using Colab as a pro member,    <br>\nbut in my case it was always a P100 and in the case of Colab pro + it was always a V100, in the case of the free version always a K80.    <br>\nSo in the end for me its a clear evidence that I go on with kaggling on google colab - in my case the pro + variant.    </p>\n<p>Some more benefits if you use Google Colab:    </p>\n<ul>\n<li>Data generators are really faster in reading the images from their destination .    <br>\nKaggle read sthe 100.000 images in about 5  minutes, in Colab (its own storage not drive) it was instantly done.    </li>\n<li>obviously no time limitation on GPU usage - for me especially useful because I start trainng in the last week of the competition     <br>\nand so I dont get as much stress looking on the remaining gpu time.    </li>\n<li>you have access to more ram so I think that other operations might also be more stable and faster (gut feeling from my experience)    </li>\n</ul>\n<p>There are also disadvantages:    </p>\n<ul>\n<li>just to list it up: the membership isn't for free, but I even the free membership is clearly a winner    </li>\n<li>settng up the colab environment is a pain in the ass, but if is properly done ist really great     </li>\n</ul>\n<p>If you need a guide to set up your google colab/drive system in order to get it right,    <br>\nplease go to my post on this topic:    <br>\n<a href=\"https://www.kaggle.com/general/287192\" target=\"_blank\">https://www.kaggle.com/general/287192</a></p>\n<p>I hope the information is helpful, feel free to ask.    </p>",
  "messages": [
    {
      "id": 1588531,
      "postDate": "2021-11-19T12:53:48.313Z",
      "content": "<p>Hey,    </p>\n<p>because of the limited gpu runtime of kaggle and the paid memberships I wondered    <br>\nhow much benefit would there be for the training of DNNs  if I kaggle on Google Colab.    <br>\nSo I decided to really measure the performance on a single CNN classification     <br>\nfrom the rsna competition some weeks ago.    <br>\n<a href=\"https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification\" target=\"_blank\">https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification</a>    </p>\n<p>The setting was as follows;    </p>\n<ul>\n<li>100.000 images (png grayscale,  (train 80%, validation 20%)    </li>\n<li>images resized to  shape of (64,64)    </li>\n<li>CNN with roughly 120.000 parameters    </li>\n<li>15 epochs    </li>\n</ul>\n<p><strong>Results:</strong></p>\n<table>\n<thead>\n<tr>\n<th>System</th>\n<th>GPU</th>\n<th>Runtime in s</th>\n<th>% to Kaggle</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Kaggle</td>\n<td>Tesla P 100</td>\n<td>937</td>\n<td></td>\n</tr>\n<tr>\n<td>Colab free</td>\n<td>Tesla K 80</td>\n<td>739</td>\n<td>79%</td>\n</tr>\n<tr>\n<td>Colab Pro</td>\n<td>Tesla P 100</td>\n<td>510</td>\n<td>54%</td>\n</tr>\n<tr>\n<td>Colab Pro +</td>\n<td>Tesla V 100</td>\n<td>425</td>\n<td>45%</td>\n</tr>\n</tbody>\n</table>\n<p>For visualization visit here:<br>\n<a href=\"https://www.kaggle.com/andreashorlbeck/results-kaggle-vs-colab?scriptVersionId=79503243\" target=\"_blank\">https://www.kaggle.com/andreashorlbeck/results-kaggle-vs-colab?scriptVersionId=79503243</a></p>\n<p>According to the runtime measurements Colab was really faster to train than Kaggle.    <br>\nSome users said that there is a chance to get also a V100 when using Colab as a pro member,    <br>\nbut in my case it was always a P100 and in the case of Colab pro + it was always a V100, in the case of the free version always a K80.    <br>\nSo in the end for me its a clear evidence that I go on with kaggling on google colab - in my case the pro + variant.    </p>\n<p>Some more benefits if you use Google Colab:    </p>\n<ul>\n<li>Data generators are really faster in reading the images from their destination .    <br>\nKaggle read sthe 100.000 images in about 5  minutes, in Colab (its own storage not drive) it was instantly done.    </li>\n<li>obviously no time limitation on GPU usage - for me especially useful because I start trainng in the last week of the competition     <br>\nand so I dont get as much stress looking on the remaining gpu time.    </li>\n<li>you have access to more ram so I think that other operations might also be more stable and faster (gut feeling from my experience)    </li>\n</ul>\n<p>There are also disadvantages:    </p>\n<ul>\n<li>just to list it up: the membership isn't for free, but I even the free membership is clearly a winner    </li>\n<li>settng up the colab environment is a pain in the ass, but if is properly done ist really great     </li>\n</ul>\n<p>If you need a guide to set up your google colab/drive system in order to get it right,    <br>\nplease go to my post on this topic:    <br>\n<a href=\"https://www.kaggle.com/general/287192\" target=\"_blank\">https://www.kaggle.com/general/287192</a></p>\n<p>I hope the information is helpful, feel free to ask.    </p>",
      "rawMarkdown": "Hey,\t\n\t\nbecause of the limited gpu runtime of kaggle and the paid memberships I wondered\t\nhow much benefit would there be for the training of DNNs  if I kaggle on Google Colab.\t\nSo I decided to really measure the performance on a single CNN classification \t\nfrom the rsna competition some weeks ago.\t\nhttps://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification\t\n\t\nThe setting was as follows;\t\n - 100.000 images (png grayscale,  (train 80%, validation 20%)\t\n - images resized to  shape of (64,64)\t\n - CNN with roughly 120.000 parameters\t\n - 15 epochs\t\n\t\n**Results:**\n\n| System| GPU  | Runtime in s  | % to Kaggle|\n| --- | --- |\n|  Kaggle| Tesla P 100 | 937 | \n|  Colab free |Tesla K 80 | 739 | 79%\n|  Colab Pro| Tesla P 100 | 510 | 54%\n|  Colab Pro + | Tesla V 100 | 425 | 45%\n\nFor visualization visit here:\nhttps://www.kaggle.com/andreashorlbeck/results-kaggle-vs-colab?scriptVersionId=79503243\n\n\n\t\nAccording to the runtime measurements Colab was really faster to train than Kaggle.\t\nSome users said that there is a chance to get also a V100 when using Colab as a pro member,\t\nbut in my case it was always a P100 and in the case of Colab pro + it was always a V100, in the case of the free version always a K80.\t\nSo in the end for me its a clear evidence that I go on with kaggling on google colab - in my case the pro + variant.\t\n\t\nSome more benefits if you use Google Colab:\t\n- Data generators are really faster in reading the images from their destination .\t\n   Kaggle read sthe 100.000 images in about 5  minutes, in Colab (its own storage not drive) it was instantly done.\t\n- obviously no time limitation on GPU usage - for me especially useful because I start trainng in the last week of the competition \t\n   and so I dont get as much stress looking on the remaining gpu time.\t\n- you have access to more ram so I think that other operations might also be more stable and faster (gut feeling from my experience)\t\n\t\nThere are also disadvantages:\t\n- just to list it up: the membership isn't for free, but I even the free membership is clearly a winner\t\n-  settng up the colab environment is a pain in the ass, but if is properly done ist really great \t\n\t\nIf you need a guide to set up your google colab/drive system in order to get it right,\t\nplease go to my post on this topic:\t\nhttps://www.kaggle.com/general/287192\n\t\n\t\nI hope the information is helpful, feel free to ask.\t\n\t",
      "votes": 9
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1588531": "Hey,\t\n\t\nbecause of the limited gpu runtime of kaggle and the paid memberships I wondered\t\nhow much benefit would there be for the training of DNNs  if I kaggle on Google Colab.\t\nSo I decided to really measure the performance on a single CNN classification \t\nfrom the rsna competition some weeks ago.\t\nhttps://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification\t\n\t\nThe setting was as follows;\t\n - 100.000 images (png grayscale,  (train 80%, validation 20%)\t\n - images resized to  shape of (64,64)\t\n - CNN with roughly 120.000 parameters\t\n - 15 epochs\t\n\t\n**Results:**\n\n| System| GPU  | Runtime in s  | % to Kaggle|\n| --- | --- |\n|  Kaggle| Tesla P 100 | 937 | \n|  Colab free |Tesla K 80 | 739 | 79%\n|  Colab Pro| Tesla P 100 | 510 | 54%\n|  Colab Pro + | Tesla V 100 | 425 | 45%\n\nFor visualization visit here:\nhttps://www.kaggle.com/andreashorlbeck/results-kaggle-vs-colab?scriptVersionId=79503243\n\n\n\t\nAccording to the runtime measurements Colab was really faster to train than Kaggle.\t\nSome users said that there is a chance to get also a V100 when using Colab as a pro member,\t\nbut in my case it was always a P100 and in the case of Colab pro + it was always a V100, in the case of the free version always a K80.\t\nSo in the end for me its a clear evidence that I go on with kaggling on google colab - in my case the pro + variant.\t\n\t\nSome more benefits if you use Google Colab:\t\n- Data generators are really faster in reading the images from their destination .\t\n   Kaggle read sthe 100.000 images in about 5  minutes, in Colab (its own storage not drive) it was instantly done.\t\n- obviously no time limitation on GPU usage - for me especially useful because I start trainng in the last week of the competition \t\n   and so I dont get as much stress looking on the remaining gpu time.\t\n- you have access to more ram so I think that other operations might also be more stable and faster (gut feeling from my experience)\t\n\t\nThere are also disadvantages:\t\n- just to list it up: the membership isn't for free, but I even the free membership is clearly a winner\t\n-  settng up the colab environment is a pain in the ass, but if is properly done ist really great \t\n\t\nIf you need a guide to set up your google colab/drive system in order to get it right,\t\nplease go to my post on this topic:\t\nhttps://www.kaggle.com/general/287192\n\t\n\t\nI hope the information is helpful, feel free to ask.\t\n\t"
  }
}