Since we are not allowed internet, you can find weights for BERT and GPT-2 models (pytorch), ready to be used with HuggingFace's Transformers or your own models ;)
\n\nhttps://www.kaggle.com/abhishek/bert-pytorch\nhttps://www.kaggle.com/abhishek/gpt2-pytorch
", "messages": [ { "id": "671010", "postDate": "11/12/2019 06:27:13", "content": "Since we are not allowed internet, you can find weights for BERT and GPT-2 models (pytorch), ready to be used with HuggingFace's Transformers or your own models ;)
\n\nhttps://www.kaggle.com/abhishek/bert-pytorch\nhttps://www.kaggle.com/abhishek/gpt2-pytorch
", "rawMarkdown": "Since we are not allowed internet, you can find weights for BERT and GPT-2 models (pytorch), ready to be used with HuggingFace's Transformers or your own models ;)\n\nhttps://www.kaggle.com/abhishek/bert-pytorch\nhttps://www.kaggle.com/abhishek/gpt2-pytorch", "votes": null }, { "id": "671205", "postDate": "11/12/2019 11:36:19", "content": "Thanks for sharing. Is there a way to use pytorch model's weights in tensorflow 2? because according to competition rules, we are only allowed to use tf2.\nI am not sure if we can use huggingface or not.
", "rawMarkdown": "Thanks for sharing. Is there a way to use pytorch model's weights in tensorflow 2? because according to competition rules, we are only allowed to use tf2.\nI am not sure if we can use [huggingface](https://github.com/huggingface/transformers) or not.", "votes": null }, { "id": "671209", "postDate": "11/12/2019 11:40:13", "content": "where does it say that we are only allowed to use tf2?
\n\nbtw, here are BERT tf weights: https://www.kaggle.com/abhishek/bert-tensorflow
", "rawMarkdown": "where does it say that we are only allowed to use tf2? \n\nbtw, here are BERT tf weights: https://www.kaggle.com/abhishek/bert-tensorflow", "votes": null }, { "id": "671329", "postDate": "11/12/2019 14:31:22", "content": "\"To be eligible for the TensorFlow 2.0 prizes, your code must use TensorFlow 2.0. More specifically:
\n\nit should run with a public release of TensorFlow 2.0 installed, and
should not use any tf.compat.v1 module symbols and should not raise deprecation warnings\"
thanks for your fast response.
", "rawMarkdown": "\"To be eligible for the TensorFlow 2.0 prizes, your code must use TensorFlow 2.0. More specifically:\n\n - it should run with a public release of TensorFlow 2.0 installed, and\n\n - should not use any tf.compat.v1 module symbols and should not raise deprecation warnings\"\n\nthanks for your fast response.", "votes": null }, { "id": "671333", "postDate": "11/12/2019 14:37:02", "content": "Yes. But there is also normal prize :D For that you can use anything you want. I hope I’m not wrong here.
", "rawMarkdown": "Yes. But there is also normal prize :D For that you can use anything you want. I hope I’m not wrong here.", "votes": null }, { "id": "671365", "postDate": "11/12/2019 15:18:03", "content": "You're right. There are two sets of prizes: one from Kaggle and one from TensorFlow.
", "rawMarkdown": "You're right. There are two sets of prizes: one from Kaggle and one from TensorFlow.", "votes": null }, { "id": "671366", "postDate": "11/12/2019 15:19:35", "content": "Is there a description somewhere how to read and use the bin files? What's the structure and format? Is there an equivalent for TF2.0? Thanks.
", "rawMarkdown": "Is there a description somewhere how to read and use the bin files? What's the structure and format? Is there an equivalent for TF2.0? Thanks.", "votes": null }, { "id": "671832", "postDate": "11/13/2019 07:55:06", "content": "Thank you for the weights!! Surely would be helpful for lot of us. \nHow about creating a baseline kernel(from the king of NLP) showing how to use these weights ?
", "rawMarkdown": "Thank you for the weights!! Surely would be helpful for lot of us. \nHow about creating a baseline kernel(from the king of NLP) showing how to use these weights ?", "votes": null }, { "id": "679154", "postDate": "11/22/2019 10:31:38", "content": "Thanks. As pytorch user, I'll definitely go to pytorch. But as I'm not nlp expert, I wonder is it possible to reproduce bert performance by pytorch? Is there an advantage of using TF over pytorch in nlp competition?
", "rawMarkdown": "Thanks. As pytorch user, I'll definitely go to pytorch. But as I'm not nlp expert, I wonder is it possible to reproduce bert performance by pytorch? Is there an advantage of using TF over pytorch in nlp competition?", "votes": null }, { "id": "681177", "postDate": "11/25/2019 19:50:27", "content": "This link might help you with what you want. I have not used pytorch so I am looking to load a pre-trained BERT and then train a layer on top of that.\nhttps://colab.research.google.com/drive/1ywsvwO6thOVOrfagjjfuxEf6xVRxbUNO#scrollTo=3UfxtwQy3axu
", "rawMarkdown": "This link might help you with what you want. I have not used pytorch so I am looking to load a pre-trained BERT and then train a layer on top of that.\nhttps://colab.research.google.com/drive/1ywsvwO6thOVOrfagjjfuxEf6xVRxbUNO#scrollTo=3UfxtwQy3axu", "votes": null }, { "id": "718615", "postDate": "01/14/2020 15:32:06", "content": "Pretrained on what?
", "rawMarkdown": "Pretrained on what?", "votes": null } ], "comments": [ { "id": 671205, "author_name": "shervinshervin", "author_url": "", "post_date": "11/12/2019 11:36:19", "content": "Thanks for sharing. Is there a way to use pytorch model's weights in tensorflow 2? because according to competition rules, we are only allowed to use tf2.\nI am not sure if we can use huggingface or not.
", "votes": null, "replies": [ { "id": 671209, "author_name": "abhishek", "author_url": "", "post_date": "11/12/2019 11:40:13", "content": "where does it say that we are only allowed to use tf2?
\n\nbtw, here are BERT tf weights: https://www.kaggle.com/abhishek/bert-tensorflow
", "votes": null, "replies": [] }, { "id": 671329, "author_name": "shervinshervin", "author_url": "", "post_date": "11/12/2019 14:31:22", "content": "\"To be eligible for the TensorFlow 2.0 prizes, your code must use TensorFlow 2.0. More specifically:
\n\nit should run with a public release of TensorFlow 2.0 installed, and
should not use any tf.compat.v1 module symbols and should not raise deprecation warnings\"
thanks for your fast response.
", "votes": null, "replies": [] }, { "id": 671333, "author_name": "abhishek", "author_url": "", "post_date": "11/12/2019 14:37:02", "content": "Yes. But there is also normal prize :D For that you can use anything you want. I hope I’m not wrong here.
", "votes": null, "replies": [] }, { "id": 671365, "author_name": "arvindpdmn", "author_url": "", "post_date": "11/12/2019 15:18:03", "content": "You're right. There are two sets of prizes: one from Kaggle and one from TensorFlow.
", "votes": null, "replies": [] } ] }, { "id": 671366, "author_name": "arvindpdmn", "author_url": "", "post_date": "11/12/2019 15:19:35", "content": "Is there a description somewhere how to read and use the bin files? What's the structure and format? Is there an equivalent for TF2.0? Thanks.
", "votes": null, "replies": [] }, { "id": 671832, "author_name": "bibek777", "author_url": "", "post_date": "11/13/2019 07:55:06", "content": "Thank you for the weights!! Surely would be helpful for lot of us. \nHow about creating a baseline kernel(from the king of NLP) showing how to use these weights ?
", "votes": null, "replies": [] }, { "id": 679154, "author_name": "niuddd", "author_url": "", "post_date": "11/22/2019 10:31:38", "content": "Thanks. As pytorch user, I'll definitely go to pytorch. But as I'm not nlp expert, I wonder is it possible to reproduce bert performance by pytorch? Is there an advantage of using TF over pytorch in nlp competition?
", "votes": null, "replies": [ { "id": 681177, "author_name": "kevinehsani", "author_url": "", "post_date": "11/25/2019 19:50:27", "content": "This link might help you with what you want. I have not used pytorch so I am looking to load a pre-trained BERT and then train a layer on top of that.\nhttps://colab.research.google.com/drive/1ywsvwO6thOVOrfagjjfuxEf6xVRxbUNO#scrollTo=3UfxtwQy3axu
", "votes": null, "replies": [] } ] }, { "id": 718615, "author_name": "cpmpml", "author_url": "", "post_date": "01/14/2020 15:32:06", "content": "Pretrained on what?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "671010": "Since we are not allowed internet, you can find weights for BERT and GPT-2 models (pytorch), ready to be used with HuggingFace's Transformers or your own models ;)\n\nhttps://www.kaggle.com/abhishek/bert-pytorch\nhttps://www.kaggle.com/abhishek/gpt2-pytorch", "671205": "Thanks for sharing. Is there a way to use pytorch model's weights in tensorflow 2? because according to competition rules, we are only allowed to use tf2.\nI am not sure if we can use [huggingface](https://github.com/huggingface/transformers) or not.", "671209": "where does it say that we are only allowed to use tf2? \n\nbtw, here are BERT tf weights: https://www.kaggle.com/abhishek/bert-tensorflow", "671329": "\"To be eligible for the TensorFlow 2.0 prizes, your code must use TensorFlow 2.0. More specifically:\n\n - it should run with a public release of TensorFlow 2.0 installed, and\n\n - should not use any tf.compat.v1 module symbols and should not raise deprecation warnings\"\n\nthanks for your fast response.", "671333": "Yes. But there is also normal prize :D For that you can use anything you want. I hope I’m not wrong here.", "671365": "You're right. There are two sets of prizes: one from Kaggle and one from TensorFlow.", "671366": "Is there a description somewhere how to read and use the bin files? What's the structure and format? Is there an equivalent for TF2.0? Thanks.", "671832": "Thank you for the weights!! Surely would be helpful for lot of us. \nHow about creating a baseline kernel(from the king of NLP) showing how to use these weights ?", "679154": "Thanks. As pytorch user, I'll definitely go to pytorch. But as I'm not nlp expert, I wonder is it possible to reproduce bert performance by pytorch? Is there an advantage of using TF over pytorch in nlp competition?", "681177": "This link might help you with what you want. I have not used pytorch so I am looking to load a pre-trained BERT and then train a layer on top of that.\nhttps://colab.research.google.com/drive/1ywsvwO6thOVOrfagjjfuxEf6xVRxbUNO#scrollTo=3UfxtwQy3axu", "718615": "Pretrained on what?" }, "source": "meta" }