{
  "id": 138203,
  "title": "Pytorch with TPU (XLA)",
  "url": "/competitions/jigsaw-multilingual-toxic-comment-classification/discussion/138203",
  "author_name": "",
  "post_date": "2020-03-24T06:40:45.571700700Z",
  "votes": 8,
  "comment_count": 9,
  "views": 0,
  "content": "<p>Hi, I made a notebook including how to learn pytorch model in TPU.</p>\n\n<p><a href=\"https://www.kaggle.com/youhanlee/bert-pytorch-huggingface-tpu-version-xla\">https://www.kaggle.com/youhanlee/bert-pytorch-huggingface-tpu-version-xla</a></p>\n\n<p>Thanks for this discussion's author and kaggle team.</p>\n\n<p><a href=\"https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/138137\">https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/138137</a></p>\n\n<p>And this notebook was forked from <a href=\"https://www.kaggle.com/theoviel/bert-pytorch-huggingface-starter\">https://www.kaggle.com/theoviel/bert-pytorch-huggingface-starter</a></p>\n\n<p>Thanks for him, <a href=\"https://www.kaggle.com/theoviel\">https://www.kaggle.com/theoviel</a>.</p>",
  "messages": [
    {
      "id": "784356",
      "postDate": "03/24/2020 06:40:45",
      "content": "<p>Hi, I made a notebook including how to learn pytorch model in TPU.</p>\n\n<p><a href=\"https://www.kaggle.com/youhanlee/bert-pytorch-huggingface-tpu-version-xla\">https://www.kaggle.com/youhanlee/bert-pytorch-huggingface-tpu-version-xla</a></p>\n\n<p>Thanks for this discussion's author and kaggle team.</p>\n\n<p><a href=\"https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/138137\">https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/138137</a></p>\n\n<p>And this notebook was forked from <a href=\"https://www.kaggle.com/theoviel/bert-pytorch-huggingface-starter\">https://www.kaggle.com/theoviel/bert-pytorch-huggingface-starter</a></p>\n\n<p>Thanks for him, <a href=\"https://www.kaggle.com/theoviel\">https://www.kaggle.com/theoviel</a>.</p>",
      "rawMarkdown": "Hi, I made a notebook including how to learn pytorch model in TPU.\n\nhttps://www.kaggle.com/youhanlee/bert-pytorch-huggingface-tpu-version-xla\n\nThanks for this discussion's author and kaggle team.\n\nhttps://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/138137\n\nAnd this notebook was forked from https://www.kaggle.com/theoviel/bert-pytorch-huggingface-starter\n\nThanks for him, https://www.kaggle.com/theoviel.",
      "votes": null
    },
    {
      "id": "784364",
      "postDate": "03/24/2020 06:49:53",
      "content": "<p>You should train on whole then :)\n(It's single core imho)</p>",
      "rawMarkdown": "You should train on whole then :)\n(It's single core imho)",
      "votes": null
    },
    {
      "id": "784374",
      "postDate": "03/24/2020 06:57:19",
      "content": "<p>Good advice. :)</p>\n\n<p>You can check the version 3 of my notebook. </p>\n\n<p>The training time of one epoch on the whole dataset was about 1050s (both training and validation).</p>",
      "rawMarkdown": "Good advice. :)\n\nYou can check the version 3 of my notebook. \n\nThe training time of one epoch on the whole dataset was about 1050s (both training and validation).",
      "votes": null
    },
    {
      "id": "784382",
      "postDate": "03/24/2020 07:05:59",
      "content": "<p>Just leaving a reference here, (self-plug, sorry), <a href=\"https://colab.research.google.com/drive/1asR_PzzzrSpuWLNY8V__Q8XT2rvLbGS2#scrollTo=SWHybzVFRFBz\">colab</a>; Had made Bert running for Quest Comp on TPU's last time following this <a href=\"https://www.youtube.com/watch?v=B_P0ZIXspOU\">yt by Abhishek Thakur</a> for all cores!</p>",
      "rawMarkdown": "Just leaving a reference here, (self-plug, sorry), [colab](https://colab.research.google.com/drive/1asR_PzzzrSpuWLNY8V__Q8XT2rvLbGS2#scrollTo=SWHybzVFRFBz); Had made Bert running for Quest Comp on TPU's last time following this [yt by Abhishek Thakur](https://www.youtube.com/watch?v=B_P0ZIXspOU) for all cores!",
      "votes": null
    },
    {
      "id": "784384",
      "postDate": "03/24/2020 07:08:50",
      "content": "<p>Thanks for sharing :) </p>",
      "rawMarkdown": "Thanks for sharing :)",
      "votes": null
    },
    {
      "id": "784671",
      "postDate": "03/24/2020 12:29:13",
      "content": "<p>I didn't see this when I posted mine. Difference is I avoid downloading pytorch each time.  Forth rest it should be similar.</p>",
      "rawMarkdown": "I didn't see this when I posted mine. Difference is I avoid downloading pytorch each time.  Forth rest it should be similar.",
      "votes": null
    },
    {
      "id": "784679",
      "postDate": "03/24/2020 12:36:24",
      "content": "<p>I didn't want to make some redundant works. If there were many similar works, I want to admire and refer to it.</p>\n\n<p>Actually, I'm beginner at NLP. Please forgive me :)</p>",
      "rawMarkdown": "I didn't want to make some redundant works. If there were many similar works, I want to admire and refer to it.\n\nActually, I'm beginner at NLP. Please forgive me :)",
      "votes": null
    },
    {
      "id": "794814",
      "postDate": "04/02/2020 05:36:45",
      "content": "<p>Thanks for sharing.</p>",
      "rawMarkdown": "Thanks for sharing.",
      "votes": null
    },
    {
      "id": "860003",
      "postDate": "05/25/2020 00:07:12",
      "content": "<p>Using the above installs has resulted in this error when torch_xla is imported:</p>\n\n<p>ImportError: /opt/conda/lib/python3.7/site-packages/XLAC.cpython-37m-x8664-linux-gnu.so: undefined symbol: ZN5torch11CppFunctionC1EN3c1014KernelFunctionESt10uniqueptrINS114FunctionSchemaESt14defaultdeleteIS4_EE</p>\n\n<p>Any idea what the issue is? Anyones help would be appreciated</p>",
      "rawMarkdown": "Using the above installs has resulted in this error when torch_xla is imported:\n\nImportError: /opt/conda/lib/python3.7/site-packages/XLAC.cpython-37m-x8664-linux-gnu.so: undefined symbol: ZN5torch11CppFunctionC1EN3c1014KernelFunctionESt10uniqueptrINS114FunctionSchemaESt14defaultdeleteIS4_EE\n\nAny idea what the issue is? Anyones help would be appreciated",
      "votes": null
    },
    {
      "id": "874303",
      "postDate": "06/04/2020 20:05:11",
      "content": "<p>Fixed issue by restarting kernel. </p>",
      "rawMarkdown": "Fixed issue by restarting kernel.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 784364,
      "author_name": "adityaecdrid",
      "author_url": "",
      "post_date": "03/24/2020 06:49:53",
      "content": "<p>You should train on whole then :)\n(It's single core imho)</p>",
      "votes": null,
      "replies": [
        {
          "id": 784374,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "03/24/2020 06:57:19",
          "content": "<p>Good advice. :)</p>\n\n<p>You can check the version 3 of my notebook. </p>\n\n<p>The training time of one epoch on the whole dataset was about 1050s (both training and validation).</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 784382,
          "author_name": "adityaecdrid",
          "author_url": "",
          "post_date": "03/24/2020 07:05:59",
          "content": "<p>Just leaving a reference here, (self-plug, sorry), <a href=\"https://colab.research.google.com/drive/1asR_PzzzrSpuWLNY8V__Q8XT2rvLbGS2#scrollTo=SWHybzVFRFBz\">colab</a>; Had made Bert running for Quest Comp on TPU's last time following this <a href=\"https://www.youtube.com/watch?v=B_P0ZIXspOU\">yt by Abhishek Thakur</a> for all cores!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 784384,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "03/24/2020 07:08:50",
          "content": "<p>Thanks for sharing :) </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 784671,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "03/24/2020 12:29:13",
      "content": "<p>I didn't see this when I posted mine. Difference is I avoid downloading pytorch each time.  Forth rest it should be similar.</p>",
      "votes": null,
      "replies": [
        {
          "id": 784679,
          "author_name": "youhanlee",
          "author_url": "",
          "post_date": "03/24/2020 12:36:24",
          "content": "<p>I didn't want to make some redundant works. If there were many similar works, I want to admire and refer to it.</p>\n\n<p>Actually, I'm beginner at NLP. Please forgive me :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 794814,
      "author_name": "hemanth007",
      "author_url": "",
      "post_date": "04/02/2020 05:36:45",
      "content": "<p>Thanks for sharing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 860003,
      "author_name": "jewelltaylor",
      "author_url": "",
      "post_date": "05/25/2020 00:07:12",
      "content": "<p>Using the above installs has resulted in this error when torch_xla is imported:</p>\n\n<p>ImportError: /opt/conda/lib/python3.7/site-packages/XLAC.cpython-37m-x8664-linux-gnu.so: undefined symbol: ZN5torch11CppFunctionC1EN3c1014KernelFunctionESt10uniqueptrINS114FunctionSchemaESt14defaultdeleteIS4_EE</p>\n\n<p>Any idea what the issue is? Anyones help would be appreciated</p>",
      "votes": null,
      "replies": [
        {
          "id": 874303,
          "author_name": "jewelltaylor",
          "author_url": "",
          "post_date": "06/04/2020 20:05:11",
          "content": "<p>Fixed issue by restarting kernel. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "784356": "Hi, I made a notebook including how to learn pytorch model in TPU.\n\nhttps://www.kaggle.com/youhanlee/bert-pytorch-huggingface-tpu-version-xla\n\nThanks for this discussion's author and kaggle team.\n\nhttps://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/138137\n\nAnd this notebook was forked from https://www.kaggle.com/theoviel/bert-pytorch-huggingface-starter\n\nThanks for him, https://www.kaggle.com/theoviel.",
    "784364": "You should train on whole then :)\n(It's single core imho)",
    "784374": "Good advice. :)\n\nYou can check the version 3 of my notebook. \n\nThe training time of one epoch on the whole dataset was about 1050s (both training and validation).",
    "784382": "Just leaving a reference here, (self-plug, sorry), [colab](https://colab.research.google.com/drive/1asR_PzzzrSpuWLNY8V__Q8XT2rvLbGS2#scrollTo=SWHybzVFRFBz); Had made Bert running for Quest Comp on TPU's last time following this [yt by Abhishek Thakur](https://www.youtube.com/watch?v=B_P0ZIXspOU) for all cores!",
    "784384": "Thanks for sharing :)",
    "784671": "I didn't see this when I posted mine. Difference is I avoid downloading pytorch each time.  Forth rest it should be similar.",
    "784679": "I didn't want to make some redundant works. If there were many similar works, I want to admire and refer to it.\n\nActually, I'm beginner at NLP. Please forgive me :)",
    "794814": "Thanks for sharing.",
    "860003": "Using the above installs has resulted in this error when torch_xla is imported:\n\nImportError: /opt/conda/lib/python3.7/site-packages/XLAC.cpython-37m-x8664-linux-gnu.so: undefined symbol: ZN5torch11CppFunctionC1EN3c1014KernelFunctionESt10uniqueptrINS114FunctionSchemaESt14defaultdeleteIS4_EE\n\nAny idea what the issue is? Anyones help would be appreciated",
    "874303": "Fixed issue by restarting kernel."
  },
  "source": "meta"
}