{
  "id": 152447,
  "title": "Alternatives for XLM-Roberta?",
  "url": "/competitions/jigsaw-multilingual-toxic-comment-classification/discussion/152447",
  "author_name": "",
  "post_date": "2020-05-19T22:09:14.848912100Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I see that all of the public kernels with best scores use XLM-Roberta. Are there any good alternatives to XLM-Roberta? </p>\n\n<p>I have found only bert-base-multilingual, xlm-mlm-100-1280, and distilbert-base-multilingual-cased. Did any one tried the mentioned alternatives? If yes, how they compare with XLM-Roberta?</p>",
  "messages": [
    {
      "id": "854239",
      "postDate": "05/19/2020 22:09:14",
      "content": "<p>I see that all of the public kernels with best scores use XLM-Roberta. Are there any good alternatives to XLM-Roberta? </p>\n\n<p>I have found only bert-base-multilingual, xlm-mlm-100-1280, and distilbert-base-multilingual-cased. Did any one tried the mentioned alternatives? If yes, how they compare with XLM-Roberta?</p>",
      "rawMarkdown": "I see that all of the public kernels with best scores use XLM-Roberta. Are there any good alternatives to XLM-Roberta? \n\nI have found only bert-base-multilingual, xlm-mlm-100-1280, and distilbert-base-multilingual-cased. Did any one tried the mentioned alternatives? If yes, how they compare with XLM-Roberta?",
      "votes": null
    },
    {
      "id": "855763",
      "postDate": "05/21/2020 07:23:45",
      "content": "<p>I tried xlm-mlm-100-1280 once. I used the same training routine as for XLM-Roberta and got a score of only 0.902 instead of 0.93xx.</p>",
      "rawMarkdown": "I tried xlm-mlm-100-1280 once. I used the same training routine as for XLM-Roberta and got a score of only 0.902 instead of 0.93xx.",
      "votes": null
    },
    {
      "id": "866752",
      "postDate": "05/29/2020 16:43:21",
      "content": "<p>I will be trying other non-BERT family models as I'm sure it is not the only one that can deliver good results, even in the non-SOTA space.</p>",
      "rawMarkdown": "I will be trying other non-BERT family models as I'm sure it is not the only one that can deliver good results, even in the non-SOTA space.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 855763,
      "author_name": "bergerda",
      "author_url": "",
      "post_date": "05/21/2020 07:23:45",
      "content": "<p>I tried xlm-mlm-100-1280 once. I used the same training routine as for XLM-Roberta and got a score of only 0.902 instead of 0.93xx.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 866752,
      "author_name": "dronych",
      "author_url": "",
      "post_date": "05/29/2020 16:43:21",
      "content": "<p>I will be trying other non-BERT family models as I'm sure it is not the only one that can deliver good results, even in the non-SOTA space.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "854239": "I see that all of the public kernels with best scores use XLM-Roberta. Are there any good alternatives to XLM-Roberta? \n\nI have found only bert-base-multilingual, xlm-mlm-100-1280, and distilbert-base-multilingual-cased. Did any one tried the mentioned alternatives? If yes, how they compare with XLM-Roberta?",
    "855763": "I tried xlm-mlm-100-1280 once. I used the same training routine as for XLM-Roberta and got a score of only 0.902 instead of 0.93xx.",
    "866752": "I will be trying other non-BERT family models as I'm sure it is not the only one that can deliver good results, even in the non-SOTA space."
  },
  "source": "meta"
}