{
  "id": 127309,
  "title": "Training BERT for different languages",
  "url": "/competitions/tensorflow2-question-answering/discussion/127309",
  "author_name": "",
  "post_date": "2020-01-23T09:01:07.719694700Z",
  "votes": 9,
  "comment_count": 7,
  "views": 0,
  "content": "<p><img src=\"https://s3.amazonaws.com/images.seroundtable.com/google-bert-global-1575952051.jpg\" alt=\"\"></p>\n\n<p>Here I provide a list of blog posts, articles, repositories about applications of BERT to different languages.</p>\n\n<ol>\n<li>Open Sourcing <strong><em>German</em></strong> BERT, <a href=\"https://deepset.ai/german-bert\">https://deepset.ai/german-bert</a>  </li>\n<li>BETO, <strong><em>Spanish</em></strong> BERT, <a href=\"https://medium.com/dair-ai/beto-spanish-bert-420e4860d2c6\">https://medium.com/dair-ai/beto-spanish-bert-420e4860d2c6</a>  </li>\n<li>CamemBERT, A Tasty <strong><em>French</em></strong> Language Model, <a href=\"https://camembert-model.fr/\">https://camembert-model.fr/</a>    </li>\n<li>BERT in DeepPavlov, <strong><em>Russian</em></strong> &amp; Slavic general models, <a href=\"http://docs.deeppavlov.ai/en/master/features/models/bert.html\">http://docs.deeppavlov.ai/en/master/features/models/bert.html</a>   </li>\n<li>BERT with SentencePiece for <strong><em>Japanese</em></strong> text, <a href=\"https://github.com/yoheikikuta/bert-japanese\">https://github.com/yoheikikuta/bert-japanese</a>   </li>\n<li><em><strong>Portuguese</strong></em> Named Entity Recognition using BERT-CRF, <a href=\"https://arxiv.org/abs/1909.10649\">https://arxiv.org/abs/1909.10649</a>   </li>\n<li>ALBERTO: <strong><em>Italian</em></strong> BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets, <a href=\"http://ceur-ws.org/Vol-2481/paper57.pdf\">http://ceur-ws.org/Vol-2481/paper57.pdf</a>  </li>\n<li>BERT for <strong><em>Malay</em></strong> language, <a href=\"https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/bert\">https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/bert</a>   </li>\n<li>BERTje: A <strong><em>Dutch</em></strong> BERT Model, <a href=\"https://arxiv.org/abs/1912.09582\">https://arxiv.org/abs/1912.09582</a>    </li>\n<li>Polish BERT: <a href=\"https://huggingface.co/dkleczek/bert-base-polish-uncased-v1\">https://huggingface.co/dkleczek/bert-base-polish-uncased-v1</a>  </li>\n<li>BERT-BiLSMT-CRF-NER, Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning, <a href=\"https://pypi.org/project/bert-base/\">https://pypi.org/project/bert-base/</a>    </li>\n<li>Multilanguage BERT, <a href=\"https://github.com/google-research/bert/blob/master/multilingual.md\">https://github.com/google-research/bert/blob/master/multilingual.md</a>   </li>\n<li>CLUE results with different BERT solutions, <a href=\"https://github.com/CLUEbenchmark/CLUE\">https://github.com/CLUEbenchmark/CLUE</a>  </li>\n<li>BERT-based Cross-Lingual Question Answering with DeepPavlov, <a href=\"https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f\">https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f</a>    </li>\n<li>How multilingual is Multilingual BERT?, <a href=\"http://www.dhgarrette.com/papers/pires_multilingual_bert_acl2019.pdf\">http://www.dhgarrette.com/papers/pires_multilingual_bert_acl2019.pdf</a>   </li>\n<li>Marianne Stecklina, Hallo multilingual BERT, cómo funcionas?, <a href=\"https://medium.com/omnius/hallo-multilingual-bert-c%C3%B3mo-funcionas-2b3406cc4dc2\">https://medium.com/omnius/hallo-multilingual-bert-c%C3%B3mo-funcionas-2b3406cc4dc2</a>  </li>\n</ol>",
  "messages": [
    {
      "id": "726828",
      "postDate": "01/23/2020 09:01:07",
      "content": "<p><img src=\"https://s3.amazonaws.com/images.seroundtable.com/google-bert-global-1575952051.jpg\" alt=\"\"></p>\n\n<p>Here I provide a list of blog posts, articles, repositories about applications of BERT to different languages.</p>\n\n<ol>\n<li>Open Sourcing <strong><em>German</em></strong> BERT, <a href=\"https://deepset.ai/german-bert\">https://deepset.ai/german-bert</a>  </li>\n<li>BETO, <strong><em>Spanish</em></strong> BERT, <a href=\"https://medium.com/dair-ai/beto-spanish-bert-420e4860d2c6\">https://medium.com/dair-ai/beto-spanish-bert-420e4860d2c6</a>  </li>\n<li>CamemBERT, A Tasty <strong><em>French</em></strong> Language Model, <a href=\"https://camembert-model.fr/\">https://camembert-model.fr/</a>    </li>\n<li>BERT in DeepPavlov, <strong><em>Russian</em></strong> &amp; Slavic general models, <a href=\"http://docs.deeppavlov.ai/en/master/features/models/bert.html\">http://docs.deeppavlov.ai/en/master/features/models/bert.html</a>   </li>\n<li>BERT with SentencePiece for <strong><em>Japanese</em></strong> text, <a href=\"https://github.com/yoheikikuta/bert-japanese\">https://github.com/yoheikikuta/bert-japanese</a>   </li>\n<li><em><strong>Portuguese</strong></em> Named Entity Recognition using BERT-CRF, <a href=\"https://arxiv.org/abs/1909.10649\">https://arxiv.org/abs/1909.10649</a>   </li>\n<li>ALBERTO: <strong><em>Italian</em></strong> BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets, <a href=\"http://ceur-ws.org/Vol-2481/paper57.pdf\">http://ceur-ws.org/Vol-2481/paper57.pdf</a>  </li>\n<li>BERT for <strong><em>Malay</em></strong> language, <a href=\"https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/bert\">https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/bert</a>   </li>\n<li>BERTje: A <strong><em>Dutch</em></strong> BERT Model, <a href=\"https://arxiv.org/abs/1912.09582\">https://arxiv.org/abs/1912.09582</a>    </li>\n<li>Polish BERT: <a href=\"https://huggingface.co/dkleczek/bert-base-polish-uncased-v1\">https://huggingface.co/dkleczek/bert-base-polish-uncased-v1</a>  </li>\n<li>BERT-BiLSMT-CRF-NER, Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning, <a href=\"https://pypi.org/project/bert-base/\">https://pypi.org/project/bert-base/</a>    </li>\n<li>Multilanguage BERT, <a href=\"https://github.com/google-research/bert/blob/master/multilingual.md\">https://github.com/google-research/bert/blob/master/multilingual.md</a>   </li>\n<li>CLUE results with different BERT solutions, <a href=\"https://github.com/CLUEbenchmark/CLUE\">https://github.com/CLUEbenchmark/CLUE</a>  </li>\n<li>BERT-based Cross-Lingual Question Answering with DeepPavlov, <a href=\"https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f\">https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f</a>    </li>\n<li>How multilingual is Multilingual BERT?, <a href=\"http://www.dhgarrette.com/papers/pires_multilingual_bert_acl2019.pdf\">http://www.dhgarrette.com/papers/pires_multilingual_bert_acl2019.pdf</a>   </li>\n<li>Marianne Stecklina, Hallo multilingual BERT, cómo funcionas?, <a href=\"https://medium.com/omnius/hallo-multilingual-bert-c%C3%B3mo-funcionas-2b3406cc4dc2\">https://medium.com/omnius/hallo-multilingual-bert-c%C3%B3mo-funcionas-2b3406cc4dc2</a>  </li>\n</ol>",
      "rawMarkdown": "![](https://s3.amazonaws.com/images.seroundtable.com/google-bert-global-1575952051.jpg)\n\nHere I provide a list of blog posts, articles, repositories about applications of BERT to different languages.\n\n1. Open Sourcing ***German*** BERT, https://deepset.ai/german-bert  \n2.  BETO, ***Spanish*** BERT, https://medium.com/dair-ai/beto-spanish-bert-420e4860d2c6  \n3.  CamemBERT, A Tasty ***French*** Language Model, https://camembert-model.fr/    \n4. BERT in DeepPavlov, ***Russian*** &amp; Slavic general models, http://docs.deeppavlov.ai/en/master/features/models/bert.html   \n5. BERT with SentencePiece for ***Japanese*** text, https://github.com/yoheikikuta/bert-japanese   \n6.  ***Portuguese*** Named Entity Recognition using BERT-CRF, https://arxiv.org/abs/1909.10649   \n7. ALBERTO: ***Italian*** BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets, http://ceur-ws.org/Vol-2481/paper57.pdf  \n8. BERT for ***Malay*** language, https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/bert   \n9. BERTje: A ***Dutch*** BERT Model, https://arxiv.org/abs/1912.09582    \n10. Polish BERT: https://huggingface.co/dkleczek/bert-base-polish-uncased-v1  \n11. BERT-BiLSMT-CRF-NER, Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning, https://pypi.org/project/bert-base/    \n12. Multilanguage BERT, https://github.com/google-research/bert/blob/master/multilingual.md   \n13. CLUE results with different BERT solutions, https://github.com/CLUEbenchmark/CLUE  \n14. BERT-based Cross-Lingual Question Answering with DeepPavlov, https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f    \n15. How multilingual is Multilingual BERT?, http://www.dhgarrette.com/papers/pires_multilingual_bert_acl2019.pdf   \n16. Marianne Stecklina, Hallo multilingual BERT, cómo funcionas?, https://medium.com/omnius/hallo-multilingual-bert-c%C3%B3mo-funcionas-2b3406cc4dc2",
      "votes": null
    },
    {
      "id": "726868",
      "postDate": "01/23/2020 09:39:01",
      "content": "<p>Helpful &amp; Informative share\nThanks!! <a href=\"/gpreda\">@gpreda</a> </p>",
      "rawMarkdown": "Helpful &amp; Informative share\nThanks!! @gpreda",
      "votes": null
    },
    {
      "id": "726916",
      "postDate": "01/23/2020 10:14:05",
      "content": "<p>Thank you. Stay tuned, I will continue to update this post with more info. Also, if you have more examples, please post here, so that I will add to the topic.</p>",
      "rawMarkdown": "Thank you. Stay tuned, I will continue to update this post with more info. Also, if you have more examples, please post here, so that I will add to the topic.",
      "votes": null
    },
    {
      "id": "775531",
      "postDate": "03/16/2020 19:18:04",
      "content": "<p>Polish BERT: <a href=\"https://huggingface.co/dkleczek/bert-base-polish-uncased-v1\">https://huggingface.co/dkleczek/bert-base-polish-uncased-v1</a></p>",
      "rawMarkdown": "Polish BERT: https://huggingface.co/dkleczek/bert-base-polish-uncased-v1",
      "votes": null
    },
    {
      "id": "775535",
      "postDate": "03/16/2020 19:21:14",
      "content": "<p>Thank you, added.</p>",
      "rawMarkdown": "Thank you, added.",
      "votes": null
    },
    {
      "id": "803121",
      "postDate": "04/10/2020 07:18:49",
      "content": "<p>Very useful. Thanks a lot for sharing.</p>",
      "rawMarkdown": "Very useful. Thanks a lot for sharing.",
      "votes": null
    },
    {
      "id": "1998503",
      "postDate": "10/21/2022 16:49:16",
      "content": "<p>Downloading the model on my computer, and hoping to have enough space! </p>",
      "rawMarkdown": "Downloading the model on my computer, and hoping to have enough space!",
      "votes": null
    },
    {
      "id": "1998505",
      "postDate": "10/21/2022 16:50:28",
      "content": "<p>And what can possibly mean \"multilanguage BERT\" ?!</p>",
      "rawMarkdown": "And what can possibly mean \"multilanguage BERT\" ?!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1998503,
      "author_name": "catadanna",
      "author_url": "",
      "post_date": "10/21/2022 16:49:16",
      "content": "<p>Downloading the model on my computer, and hoping to have enough space! </p>",
      "votes": null,
      "replies": [
        {
          "id": 1998505,
          "author_name": "catadanna",
          "author_url": "",
          "post_date": "10/21/2022 16:50:28",
          "content": "<p>And what can possibly mean \"multilanguage BERT\" ?!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 726868,
      "author_name": "veeralakrishna",
      "author_url": "",
      "post_date": "01/23/2020 09:39:01",
      "content": "<p>Helpful &amp; Informative share\nThanks!! <a href=\"/gpreda\">@gpreda</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 726916,
          "author_name": "gpreda",
          "author_url": "",
          "post_date": "01/23/2020 10:14:05",
          "content": "<p>Thank you. Stay tuned, I will continue to update this post with more info. Also, if you have more examples, please post here, so that I will add to the topic.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 775531,
      "author_name": "konradb",
      "author_url": "",
      "post_date": "03/16/2020 19:18:04",
      "content": "<p>Polish BERT: <a href=\"https://huggingface.co/dkleczek/bert-base-polish-uncased-v1\">https://huggingface.co/dkleczek/bert-base-polish-uncased-v1</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 775535,
          "author_name": "gpreda",
          "author_url": "",
          "post_date": "03/16/2020 19:21:14",
          "content": "<p>Thank you, added.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 803121,
      "author_name": "hemanth007",
      "author_url": "",
      "post_date": "04/10/2020 07:18:49",
      "content": "<p>Very useful. Thanks a lot for sharing.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "726828": "![](https://s3.amazonaws.com/images.seroundtable.com/google-bert-global-1575952051.jpg)\n\nHere I provide a list of blog posts, articles, repositories about applications of BERT to different languages.\n\n1. Open Sourcing ***German*** BERT, https://deepset.ai/german-bert  \n2.  BETO, ***Spanish*** BERT, https://medium.com/dair-ai/beto-spanish-bert-420e4860d2c6  \n3.  CamemBERT, A Tasty ***French*** Language Model, https://camembert-model.fr/    \n4. BERT in DeepPavlov, ***Russian*** &amp; Slavic general models, http://docs.deeppavlov.ai/en/master/features/models/bert.html   \n5. BERT with SentencePiece for ***Japanese*** text, https://github.com/yoheikikuta/bert-japanese   \n6.  ***Portuguese*** Named Entity Recognition using BERT-CRF, https://arxiv.org/abs/1909.10649   \n7. ALBERTO: ***Italian*** BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets, http://ceur-ws.org/Vol-2481/paper57.pdf  \n8. BERT for ***Malay*** language, https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/bert   \n9. BERTje: A ***Dutch*** BERT Model, https://arxiv.org/abs/1912.09582    \n10. Polish BERT: https://huggingface.co/dkleczek/bert-base-polish-uncased-v1  \n11. BERT-BiLSMT-CRF-NER, Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning, https://pypi.org/project/bert-base/    \n12. Multilanguage BERT, https://github.com/google-research/bert/blob/master/multilingual.md   \n13. CLUE results with different BERT solutions, https://github.com/CLUEbenchmark/CLUE  \n14. BERT-based Cross-Lingual Question Answering with DeepPavlov, https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f    \n15. How multilingual is Multilingual BERT?, http://www.dhgarrette.com/papers/pires_multilingual_bert_acl2019.pdf   \n16. Marianne Stecklina, Hallo multilingual BERT, cómo funcionas?, https://medium.com/omnius/hallo-multilingual-bert-c%C3%B3mo-funcionas-2b3406cc4dc2",
    "726868": "Helpful &amp; Informative share\nThanks!! @gpreda",
    "726916": "Thank you. Stay tuned, I will continue to update this post with more info. Also, if you have more examples, please post here, so that I will add to the topic.",
    "775531": "Polish BERT: https://huggingface.co/dkleczek/bert-base-polish-uncased-v1",
    "775535": "Thank you, added.",
    "803121": "Very useful. Thanks a lot for sharing.",
    "1998503": "Downloading the model on my computer, and hoping to have enough space!",
    "1998505": "And what can possibly mean \"multilanguage BERT\" ?!"
  },
  "source": "meta"
}