You might stick to TF 1.0 just because of TPU support.
\n\nAnyway, you can install almost anything in a Kaggle Notebook without Internet. That's obvious for most of kagglers, but still I show how to install packages in this notebook - in that case it's tensorflow-gpu 1.13.1 and bert-tensorflow installed from this Dataset.
Likewise, you can install transformers by HuggingFace or anything. I personally would stick to PyTorch but for TPU.
You might stick to TF 1.0 just because of TPU support.
\n\nAnyway, you can install almost anything in a Kaggle Notebook without Internet. That's obvious for most of kagglers, but still I show how to install packages in this notebook - in that case it's tensorflow-gpu 1.13.1 and bert-tensorflow installed from this Dataset.
Likewise, you can install transformers by HuggingFace or anything. I personally would stick to PyTorch but for TPU.
Wait, isn't that against the code requirements for this competition? Wouldn't transformers be considered a \"custom package\"?\nHere the code requirements for reference:
\n\nNo, it's fine. That phrase means that you can not install a custom package (through Kernel settings) in a Kernel with GPU turned on.
", "rawMarkdown": "No, it's fine. That phrase means that you can not install a custom package (through Kernel settings) in a Kernel with GPU turned on.", "votes": null }, { "id": "683162", "postDate": "11/28/2019 06:41:33", "content": "Does PyTorch have a good TPU support currently?
", "rawMarkdown": "Does PyTorch have a good TPU support currently?", "votes": null }, { "id": "683224", "postDate": "11/28/2019 08:16:28", "content": "Great, thank you!
", "rawMarkdown": "Great, thank you!", "votes": null }, { "id": "686562", "postDate": "12/03/2019 09:46:09", "content": "@aruchomu They do have it, but I never tried. They say, it's slower than TF 1.0 (as for TPU) and requires more configuration.
", "rawMarkdown": "aruchomu They do have it, but I never tried. They say, it's slower than TF 1.0 (as for TPU) and requires more configuration.", "votes": null }, { "id": "686783", "postDate": "12/03/2019 14:26:14", "content": "TF 2.1 is available now in prerelease and it supports TPU. \nTPUs available in this competition do not work with TF2.1 though. \nI guess the only way to use TF2 is with GPU.
", "rawMarkdown": "TF 2.1 is available now in prerelease and it supports TPU. \nTPUs available in this competition do not work with TF2.1 though. \nI guess the only way to use TF2 is with GPU.", "votes": null }, { "id": "686788", "postDate": "12/03/2019 14:37:02", "content": "Cool, good to know
", "rawMarkdown": "Cool, good to know", "votes": null }, { "id": "695471", "postDate": "12/15/2019 07:44:11", "content": "Update: as of Dec 14, 2015 TF2.1 works with TPUs as per Page Bailey.
", "rawMarkdown": "Update: as of Dec 14, 2015 TF2.1 works with TPUs as per Page Bailey.", "votes": null } ], "comments": [ { "id": 682681, "author_name": "szmoro", "author_url": "", "post_date": "11/27/2019 18:38:50", "content": "Wait, isn't that against the code requirements for this competition? Wouldn't transformers be considered a \"custom package\"?\nHere the code requirements for reference:
\n\nNo, it's fine. That phrase means that you can not install a custom package (through Kernel settings) in a Kernel with GPU turned on.
", "votes": null, "replies": [] }, { "id": 683224, "author_name": "szmoro", "author_url": "", "post_date": "11/28/2019 08:16:28", "content": "Great, thank you!
", "votes": null, "replies": [] } ] }, { "id": 683162, "author_name": "aruchomu", "author_url": "", "post_date": "11/28/2019 06:41:33", "content": "Does PyTorch have a good TPU support currently?
", "votes": null, "replies": [ { "id": 686562, "author_name": "kashnitsky", "author_url": "", "post_date": "12/03/2019 09:46:09", "content": "@aruchomu They do have it, but I never tried. They say, it's slower than TF 1.0 (as for TPU) and requires more configuration.
", "votes": null, "replies": [] } ] }, { "id": 686783, "author_name": "mosicr", "author_url": "", "post_date": "12/03/2019 14:26:14", "content": "TF 2.1 is available now in prerelease and it supports TPU. \nTPUs available in this competition do not work with TF2.1 though. \nI guess the only way to use TF2 is with GPU.
", "votes": null, "replies": [ { "id": 686788, "author_name": "kashnitsky", "author_url": "", "post_date": "12/03/2019 14:37:02", "content": "Cool, good to know
", "votes": null, "replies": [] }, { "id": 695471, "author_name": "mosicr", "author_url": "", "post_date": "12/15/2019 07:44:11", "content": "Update: as of Dec 14, 2015 TF2.1 works with TPUs as per Page Bailey.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "682528": "You might stick to TF 1.0 just because of TPU support. \n\nAnyway, you can install almost anything in a Kaggle Notebook without Internet. That's obvious for most of kagglers, but still I show how to install packages in [this notebook](https://www.kaggle.com/kashnitsky/offline-tensorflow-1-13-1-and-bert-tensorflow) - in that case it's tensorflow-gpu 1.13.1 and `bert-tensorflow` installed from [this Dataset](https://www.kaggle.com/kashnitsky/tensorflow1131-offline-bert). \n\nLikewise, you can install `transformers` by HuggingFace or anything. I personally would stick to PyTorch but for TPU.", "682681": "Wait, isn't that against the code requirements for this competition? Wouldn't *transformers* be considered a \"custom package\"?\nHere the code requirements for reference:\n\n- GPU Notebooks <= 3 hours run-time\n- CPU Notebooks <= 9 hours run-time\n- Internet access off\n- External data is allowed (including training offline and uploading the model)\n- **No custom packages enabled in kernels**\n- Submission files must be named \"submission.csv\"", "682787": "No, it's fine. That phrase means that you can not install a custom package (through Kernel settings) in a Kernel with GPU turned on.", "683162": "Does PyTorch have a good TPU support currently?", "683224": "Great, thank you!", "686562": "aruchomu They do have it, but I never tried. They say, it's slower than TF 1.0 (as for TPU) and requires more configuration.", "686783": "TF 2.1 is available now in prerelease and it supports TPU. \nTPUs available in this competition do not work with TF2.1 though. \nI guess the only way to use TF2 is with GPU.", "686788": "Cool, good to know", "695471": "Update: as of Dec 14, 2015 TF2.1 works with TPUs as per Page Bailey." }, "source": "meta" }