{
  "id": 174489,
  "title": "Using TPU on Colab",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/174489",
  "author_name": "",
  "post_date": "2020-08-13T18:26:12.576806300Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I made a <a href=\"https://www.kaggle.com/teeyee314/melanoma-gcs-paths?scriptVersionId=40697061\" target=\"_blank\">script</a> to copy and paste GCS_PATHS over to colab to easily play around with since there is no kaggle_datasets/KaggleDatasets on Colab - for anyone interested. Just copy and paste the following code snippets.</p>\n<pre><code>path_dict = {'isic2019-1024x1024': 'gs://kds-7287b805ab38a9013eb287de365a7ed2327b1e954a2e5c7bde5f2287',\n 'isic2019-128x128': 'gs://kds-c2f9270da6f9434ebe0bac9f8c1bf846462b561a99fbb7b1e392731e',\n 'isic2019-256x256': 'gs://kds-dfae178ddbe4da1a77993af1ac7ede33a6b835ee7c24404c2e618e00',\n 'isic2019-384x384': 'gs://kds-ba3a4a10d62d1e054c5eb06cc1cc58dca6e03bb18f5f06af448dec55',\n 'isic2019-512x512': 'gs://kds-f0e31d98a8a127bdee8ff028fef64d527f41e044629193a390f7610c',\n 'isic2019-768x768': 'gs://kds-d76b4251d97923c2736426394305c4928d933ed129182dc7adc4d77c',\n 'malignant-v2-1024x1024': 'gs://kds-8d7428bfe593499c7ff4a53e1e664d64aad614488f1b447258e92af7',\n 'malignant-v2-128x128': 'gs://kds-0d52848b5502832955aced495fb321f7ae2fb70258ba43602f553b77',\n 'malignant-v2-256x256': 'gs://kds-7a08ac82d57200266dc619d51106e4af5e89cedf0c6556f632deebf9',\n 'malignant-v2-384x384': 'gs://kds-4dacd59e0327aff4eef56d9398ecced45e56a0afa6a43bf6bed7251b',\n 'malignant-v2-512x512': 'gs://kds-e47f613ded612087ba7fcca706cccfd4e9fcd036dcb4127def0d79c4',\n 'malignant-v2-768x768': 'gs://kds-4826fbc5832fe6eb691053599c8fa1b06eb57685db32bc294a545f72',\n 'melanoma-1024x1024': 'gs://kds-9c2d3214eaa8c4b962d888c54805ee19cd8a9ebccdcfae96790932f7',\n 'melanoma-128x128': 'gs://kds-7b5a61673b55b50c8616d7df218b1a1b8e2c2e2a3384ea294b438bd8',\n 'melanoma-256x256': 'gs://kds-fbc00c0b868eb34b554705994009a9d2ea1c168e4e3806326e516ba5',\n 'melanoma-384x384': 'gs://kds-4794b97a7db9238be7b3a7e36e6d03f2de112b355765280d28e2b579',\n 'melanoma-512x512': 'gs://kds-3f2b9fbffe7fc1f1219236db9534c99ce7c277d4d69662c3bbf6a6b0',\n 'melanoma-768x768': 'gs://kds-863a8421e19bd1e8af6e49fd881c4adceae0015e529b674f0b61d4c9'}\n</code></pre>\n<pre><code>GCS_PATH[i] = path_dict[f'melanoma-{k}x{k}']\nGCS_PATH2[i] = path_dict[f'isic2019-{k}x{k}']\n</code></pre>\n<p>For Those who want to export the submission/oof .csv and model.h5 weight files into another notebook for blending/combining, use the snippet below.</p>\n<pre><code>from google.colab import files\nfiles.download('submission.csv')\nfiles.download('oof.csv')\nfor i in range(FOLDS):\n  files.download(f'fold-{i}.h5')\n</code></pre>",
  "messages": [
    {
      "id": "969510",
      "postDate": "08/13/2020 18:26:12",
      "content": "<p>I made a <a href=\"https://www.kaggle.com/teeyee314/melanoma-gcs-paths?scriptVersionId=40697061\" target=\"_blank\">script</a> to copy and paste GCS_PATHS over to colab to easily play around with since there is no kaggle_datasets/KaggleDatasets on Colab - for anyone interested. Just copy and paste the following code snippets.</p>\n<pre><code>path_dict = {'isic2019-1024x1024': 'gs://kds-7287b805ab38a9013eb287de365a7ed2327b1e954a2e5c7bde5f2287',\n 'isic2019-128x128': 'gs://kds-c2f9270da6f9434ebe0bac9f8c1bf846462b561a99fbb7b1e392731e',\n 'isic2019-256x256': 'gs://kds-dfae178ddbe4da1a77993af1ac7ede33a6b835ee7c24404c2e618e00',\n 'isic2019-384x384': 'gs://kds-ba3a4a10d62d1e054c5eb06cc1cc58dca6e03bb18f5f06af448dec55',\n 'isic2019-512x512': 'gs://kds-f0e31d98a8a127bdee8ff028fef64d527f41e044629193a390f7610c',\n 'isic2019-768x768': 'gs://kds-d76b4251d97923c2736426394305c4928d933ed129182dc7adc4d77c',\n 'malignant-v2-1024x1024': 'gs://kds-8d7428bfe593499c7ff4a53e1e664d64aad614488f1b447258e92af7',\n 'malignant-v2-128x128': 'gs://kds-0d52848b5502832955aced495fb321f7ae2fb70258ba43602f553b77',\n 'malignant-v2-256x256': 'gs://kds-7a08ac82d57200266dc619d51106e4af5e89cedf0c6556f632deebf9',\n 'malignant-v2-384x384': 'gs://kds-4dacd59e0327aff4eef56d9398ecced45e56a0afa6a43bf6bed7251b',\n 'malignant-v2-512x512': 'gs://kds-e47f613ded612087ba7fcca706cccfd4e9fcd036dcb4127def0d79c4',\n 'malignant-v2-768x768': 'gs://kds-4826fbc5832fe6eb691053599c8fa1b06eb57685db32bc294a545f72',\n 'melanoma-1024x1024': 'gs://kds-9c2d3214eaa8c4b962d888c54805ee19cd8a9ebccdcfae96790932f7',\n 'melanoma-128x128': 'gs://kds-7b5a61673b55b50c8616d7df218b1a1b8e2c2e2a3384ea294b438bd8',\n 'melanoma-256x256': 'gs://kds-fbc00c0b868eb34b554705994009a9d2ea1c168e4e3806326e516ba5',\n 'melanoma-384x384': 'gs://kds-4794b97a7db9238be7b3a7e36e6d03f2de112b355765280d28e2b579',\n 'melanoma-512x512': 'gs://kds-3f2b9fbffe7fc1f1219236db9534c99ce7c277d4d69662c3bbf6a6b0',\n 'melanoma-768x768': 'gs://kds-863a8421e19bd1e8af6e49fd881c4adceae0015e529b674f0b61d4c9'}\n</code></pre>\n<pre><code>GCS_PATH[i] = path_dict[f'melanoma-{k}x{k}']\nGCS_PATH2[i] = path_dict[f'isic2019-{k}x{k}']\n</code></pre>\n<p>For Those who want to export the submission/oof .csv and model.h5 weight files into another notebook for blending/combining, use the snippet below.</p>\n<pre><code>from google.colab import files\nfiles.download('submission.csv')\nfiles.download('oof.csv')\nfor i in range(FOLDS):\n  files.download(f'fold-{i}.h5')\n</code></pre>",
      "rawMarkdown": "I made a [script](https://www.kaggle.com/teeyee314/melanoma-gcs-paths?scriptVersionId=40697061) to copy and paste GCS_PATHS over to colab to easily play around with since there is no kaggle_datasets/KaggleDatasets on Colab - for anyone interested. Just copy and paste the following code snippets.\n\n```\npath_dict = {'isic2019-1024x1024': 'gs://kds-7287b805ab38a9013eb287de365a7ed2327b1e954a2e5c7bde5f2287',\n 'isic2019-128x128': 'gs://kds-c2f9270da6f9434ebe0bac9f8c1bf846462b561a99fbb7b1e392731e',\n 'isic2019-256x256': 'gs://kds-dfae178ddbe4da1a77993af1ac7ede33a6b835ee7c24404c2e618e00',\n 'isic2019-384x384': 'gs://kds-ba3a4a10d62d1e054c5eb06cc1cc58dca6e03bb18f5f06af448dec55',\n 'isic2019-512x512': 'gs://kds-f0e31d98a8a127bdee8ff028fef64d527f41e044629193a390f7610c',\n 'isic2019-768x768': 'gs://kds-d76b4251d97923c2736426394305c4928d933ed129182dc7adc4d77c',\n 'malignant-v2-1024x1024': 'gs://kds-8d7428bfe593499c7ff4a53e1e664d64aad614488f1b447258e92af7',\n 'malignant-v2-128x128': 'gs://kds-0d52848b5502832955aced495fb321f7ae2fb70258ba43602f553b77',\n 'malignant-v2-256x256': 'gs://kds-7a08ac82d57200266dc619d51106e4af5e89cedf0c6556f632deebf9',\n 'malignant-v2-384x384': 'gs://kds-4dacd59e0327aff4eef56d9398ecced45e56a0afa6a43bf6bed7251b',\n 'malignant-v2-512x512': 'gs://kds-e47f613ded612087ba7fcca706cccfd4e9fcd036dcb4127def0d79c4',\n 'malignant-v2-768x768': 'gs://kds-4826fbc5832fe6eb691053599c8fa1b06eb57685db32bc294a545f72',\n 'melanoma-1024x1024': 'gs://kds-9c2d3214eaa8c4b962d888c54805ee19cd8a9ebccdcfae96790932f7',\n 'melanoma-128x128': 'gs://kds-7b5a61673b55b50c8616d7df218b1a1b8e2c2e2a3384ea294b438bd8',\n 'melanoma-256x256': 'gs://kds-fbc00c0b868eb34b554705994009a9d2ea1c168e4e3806326e516ba5',\n 'melanoma-384x384': 'gs://kds-4794b97a7db9238be7b3a7e36e6d03f2de112b355765280d28e2b579',\n 'melanoma-512x512': 'gs://kds-3f2b9fbffe7fc1f1219236db9534c99ce7c277d4d69662c3bbf6a6b0',\n 'melanoma-768x768': 'gs://kds-863a8421e19bd1e8af6e49fd881c4adceae0015e529b674f0b61d4c9'}\n```\n```\nGCS_PATH[i] = path_dict[f'melanoma-{k}x{k}']\nGCS_PATH2[i] = path_dict[f'isic2019-{k}x{k}']\n```\n\n\nFor Those who want to export the submission/oof .csv and model.h5 weight files into another notebook for blending/combining, use the snippet below.\n\n```\nfrom google.colab import files\nfiles.download('submission.csv')\nfiles.download('oof.csv')\nfor i in range(FOLDS):\n  files.download(f'fold-{i}.h5')\n```",
      "votes": null
    },
    {
      "id": "969585",
      "postDate": "08/13/2020 19:31:40",
      "content": "<p>awesome! thanks for sharing <a href=\"https://www.kaggle.com/tim681\" target=\"_blank\">@tim681</a> ! it was painful to get the files to colab</p>",
      "rawMarkdown": "awesome! thanks for sharing @tim681 ! it was painful to get the files to colab",
      "votes": null
    },
    {
      "id": "969597",
      "postDate": "08/13/2020 19:41:52",
      "content": "<p><a href=\"https://www.kaggle.com/mpsampat\" target=\"_blank\">@mpsampat</a> tagged another Tim haha. you're welcome!</p>",
      "rawMarkdown": "mpsampat tagged another Tim haha. you're welcome!",
      "votes": null
    },
    {
      "id": "969676",
      "postDate": "08/13/2020 21:01:44",
      "content": "<p>sorry about that <a href=\"https://www.kaggle.com/teeyee314\" target=\"_blank\">@teeyee314</a> ! :) thanks again! </p>",
      "rawMarkdown": "sorry about that @teeyee314 ! :) thanks again!",
      "votes": null
    },
    {
      "id": "972507",
      "postDate": "08/16/2020 15:46:48",
      "content": "<p>thanks. another way for downloading is using the kaggle api on colab.</p>",
      "rawMarkdown": "thanks. another way for downloading is using the kaggle api on colab.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 969585,
      "author_name": "mpsampat",
      "author_url": "",
      "post_date": "08/13/2020 19:31:40",
      "content": "<p>awesome! thanks for sharing <a href=\"https://www.kaggle.com/tim681\" target=\"_blank\">@tim681</a> ! it was painful to get the files to colab</p>",
      "votes": null,
      "replies": [
        {
          "id": 969597,
          "author_name": "teeyee314",
          "author_url": "",
          "post_date": "08/13/2020 19:41:52",
          "content": "<p><a href=\"https://www.kaggle.com/mpsampat\" target=\"_blank\">@mpsampat</a> tagged another Tim haha. you're welcome!</p>",
          "votes": null,
          "replies": [
            {
              "id": 969676,
              "author_name": "mpsampat",
              "author_url": "",
              "post_date": "08/13/2020 21:01:44",
              "content": "<p>sorry about that <a href=\"https://www.kaggle.com/teeyee314\" target=\"_blank\">@teeyee314</a> ! :) thanks again! </p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 972507,
      "author_name": "yimacs",
      "author_url": "",
      "post_date": "08/16/2020 15:46:48",
      "content": "<p>thanks. another way for downloading is using the kaggle api on colab.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "969510": "I made a [script](https://www.kaggle.com/teeyee314/melanoma-gcs-paths?scriptVersionId=40697061) to copy and paste GCS_PATHS over to colab to easily play around with since there is no kaggle_datasets/KaggleDatasets on Colab - for anyone interested. Just copy and paste the following code snippets.\n\n```\npath_dict = {'isic2019-1024x1024': 'gs://kds-7287b805ab38a9013eb287de365a7ed2327b1e954a2e5c7bde5f2287',\n 'isic2019-128x128': 'gs://kds-c2f9270da6f9434ebe0bac9f8c1bf846462b561a99fbb7b1e392731e',\n 'isic2019-256x256': 'gs://kds-dfae178ddbe4da1a77993af1ac7ede33a6b835ee7c24404c2e618e00',\n 'isic2019-384x384': 'gs://kds-ba3a4a10d62d1e054c5eb06cc1cc58dca6e03bb18f5f06af448dec55',\n 'isic2019-512x512': 'gs://kds-f0e31d98a8a127bdee8ff028fef64d527f41e044629193a390f7610c',\n 'isic2019-768x768': 'gs://kds-d76b4251d97923c2736426394305c4928d933ed129182dc7adc4d77c',\n 'malignant-v2-1024x1024': 'gs://kds-8d7428bfe593499c7ff4a53e1e664d64aad614488f1b447258e92af7',\n 'malignant-v2-128x128': 'gs://kds-0d52848b5502832955aced495fb321f7ae2fb70258ba43602f553b77',\n 'malignant-v2-256x256': 'gs://kds-7a08ac82d57200266dc619d51106e4af5e89cedf0c6556f632deebf9',\n 'malignant-v2-384x384': 'gs://kds-4dacd59e0327aff4eef56d9398ecced45e56a0afa6a43bf6bed7251b',\n 'malignant-v2-512x512': 'gs://kds-e47f613ded612087ba7fcca706cccfd4e9fcd036dcb4127def0d79c4',\n 'malignant-v2-768x768': 'gs://kds-4826fbc5832fe6eb691053599c8fa1b06eb57685db32bc294a545f72',\n 'melanoma-1024x1024': 'gs://kds-9c2d3214eaa8c4b962d888c54805ee19cd8a9ebccdcfae96790932f7',\n 'melanoma-128x128': 'gs://kds-7b5a61673b55b50c8616d7df218b1a1b8e2c2e2a3384ea294b438bd8',\n 'melanoma-256x256': 'gs://kds-fbc00c0b868eb34b554705994009a9d2ea1c168e4e3806326e516ba5',\n 'melanoma-384x384': 'gs://kds-4794b97a7db9238be7b3a7e36e6d03f2de112b355765280d28e2b579',\n 'melanoma-512x512': 'gs://kds-3f2b9fbffe7fc1f1219236db9534c99ce7c277d4d69662c3bbf6a6b0',\n 'melanoma-768x768': 'gs://kds-863a8421e19bd1e8af6e49fd881c4adceae0015e529b674f0b61d4c9'}\n```\n```\nGCS_PATH[i] = path_dict[f'melanoma-{k}x{k}']\nGCS_PATH2[i] = path_dict[f'isic2019-{k}x{k}']\n```\n\n\nFor Those who want to export the submission/oof .csv and model.h5 weight files into another notebook for blending/combining, use the snippet below.\n\n```\nfrom google.colab import files\nfiles.download('submission.csv')\nfiles.download('oof.csv')\nfor i in range(FOLDS):\n  files.download(f'fold-{i}.h5')\n```",
    "969585": "awesome! thanks for sharing @tim681 ! it was painful to get the files to colab",
    "969597": "mpsampat tagged another Tim haha. you're welcome!",
    "969676": "sorry about that @teeyee314 ! :) thanks again!",
    "972507": "thanks. another way for downloading is using the kaggle api on colab."
  },
  "source": "meta"
}