{
  "id": 309367,
  "title": "TPU/GPU  ram allocation problem",
  "url": "/competitions/happy-whale-and-dolphin/discussion/309367",
  "author_name": "",
  "post_date": "2022-02-23T05:20:47.963710300Z",
  "votes": 5,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Since the TPU only 20hrs/w,  so quick out of quota.</p>\n<p>when I switch the TPU inference notebook which works fine , to GPU accelerator,</p>\n<p>It  shows error message of memory allocation related.</p>\n<p>why this happens?</p>\n<p>Thanks for your explanation!</p>",
  "messages": [
    {
      "id": "1701841",
      "postDate": "02/23/2022 05:20:47",
      "content": "<p>Since the TPU only 20hrs/w,  so quick out of quota.</p>\n<p>when I switch the TPU inference notebook which works fine , to GPU accelerator,</p>\n<p>It  shows error message of memory allocation related.</p>\n<p>why this happens?</p>\n<p>Thanks for your explanation!</p>",
      "rawMarkdown": "Since the TPU only 20hrs/w,  so quick out of quota.\n\nwhen I switch the TPU inference notebook which works fine , to GPU accelerator,\n\nIt  shows error message of memory allocation related.\n\nwhy this happens?\n\nThanks for your explanation!",
      "votes": null
    },
    {
      "id": "1701855",
      "postDate": "02/23/2022 05:42:11",
      "content": "<p>From what I know, TPUs have more memory compared to the GPUs available in Kaggle Kernels. </p>\n<p>The TPUs also have 8 cores, so your <code>batch_size</code> becomes <code>8*batch_size_per_core</code>. Can you try running inference with a smaller <code>batch_size</code> on the GPUs? </p>\n<p>Alternatively, you can try using Colab with TPUs but getting data there is a bit more work. Hope this helps! :) </p>",
      "rawMarkdown": "From what I know, TPUs have more memory compared to the GPUs available in Kaggle Kernels. \n\nThe TPUs also have 8 cores, so your `batch_size` becomes `8*batch_size_per_core`. Can you try running inference with a smaller `batch_size` on the GPUs? \n\nAlternatively, you can try using Colab with TPUs but getting data there is a bit more work. Hope this helps! :)",
      "votes": null
    },
    {
      "id": "1701931",
      "postDate": "02/23/2022 07:14:40",
      "content": "<p>thank you for your reply. </p>\n<p>I will try run on Colab, though Its TPU with less ram compared with Kaggle.</p>",
      "rawMarkdown": "thank you for your reply. \n\nI will try run on Colab, though Its TPU with less ram compared with Kaggle.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1701855,
      "author_name": "init27",
      "author_url": "",
      "post_date": "02/23/2022 05:42:11",
      "content": "<p>From what I know, TPUs have more memory compared to the GPUs available in Kaggle Kernels. </p>\n<p>The TPUs also have 8 cores, so your <code>batch_size</code> becomes <code>8*batch_size_per_core</code>. Can you try running inference with a smaller <code>batch_size</code> on the GPUs? </p>\n<p>Alternatively, you can try using Colab with TPUs but getting data there is a bit more work. Hope this helps! :) </p>",
      "votes": null,
      "replies": [
        {
          "id": 1701931,
          "author_name": "dragonzhang",
          "author_url": "",
          "post_date": "02/23/2022 07:14:40",
          "content": "<p>thank you for your reply. </p>\n<p>I will try run on Colab, though Its TPU with less ram compared with Kaggle.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1701841": "Since the TPU only 20hrs/w,  so quick out of quota.\n\nwhen I switch the TPU inference notebook which works fine , to GPU accelerator,\n\nIt  shows error message of memory allocation related.\n\nwhy this happens?\n\nThanks for your explanation!",
    "1701855": "From what I know, TPUs have more memory compared to the GPUs available in Kaggle Kernels. \n\nThe TPUs also have 8 cores, so your `batch_size` becomes `8*batch_size_per_core`. Can you try running inference with a smaller `batch_size` on the GPUs? \n\nAlternatively, you can try using Colab with TPUs but getting data there is a bit more work. Hope this helps! :)",
    "1701931": "thank you for your reply. \n\nI will try run on Colab, though Its TPU with less ram compared with Kaggle."
  },
  "source": "meta"
}