{
  "id": 126801,
  "title": "Any models other than Bert ?",
  "url": "/competitions/tensorflow2-question-answering/discussion/126801",
  "author_name": "",
  "post_date": "2020-01-20T11:37:03.711697900Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Just want to know, any models other than BERT people have used here?</p>",
  "messages": [
    {
      "id": "723728",
      "postDate": "01/20/2020 11:37:03",
      "content": "<p>Just want to know, any models other than BERT people have used here?</p>",
      "rawMarkdown": "Just want to know, any models other than BERT people have used here?",
      "votes": null
    },
    {
      "id": "723736",
      "postDate": "01/20/2020 11:47:29",
      "content": "<p>Do you mean other than transformer encoders? I believe there are no chance to score &gt; 0.6 with old good classic approaches. </p>",
      "rawMarkdown": "Do you mean other than transformer encoders? I believe there are no chance to score &gt; 0.6 with old good classic approaches.",
      "votes": null
    },
    {
      "id": "723740",
      "postDate": "01/20/2020 11:51:54",
      "content": "<p>Totally agree with that. I was wondering about transformer encoders only. Like T5 ( text to text ), ALBERT, etc etc .</p>",
      "rawMarkdown": "Totally agree with that. I was wondering about transformer encoders only. Like T5 ( text to text ), ALBERT, etc etc .",
      "votes": null
    },
    {
      "id": "723907",
      "postDate": "01/20/2020 15:41:04",
      "content": "<p>One of the another idea is pre-train not only encoder but dense layer for position logits.\nI've tried squad pre-trained model by huggingface transformers, however no improvements on LB.</p>",
      "rawMarkdown": "One of the another idea is pre-train not only encoder but dense layer for position logits.\nI've tried squad pre-trained model by huggingface transformers, however no improvements on LB.",
      "votes": null
    },
    {
      "id": "723934",
      "postDate": "01/20/2020 16:01:05",
      "content": "<p>I tried Albert. I think the Crux is post result transformation, rather than choosing models like Bert, xlnet etc.</p>",
      "rawMarkdown": "I tried Albert. I think the Crux is post result transformation, rather than choosing models like Bert, xlnet etc.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 723736,
      "author_name": "yaroshevskiy",
      "author_url": "",
      "post_date": "01/20/2020 11:47:29",
      "content": "<p>Do you mean other than transformer encoders? I believe there are no chance to score &gt; 0.6 with old good classic approaches. </p>",
      "votes": null,
      "replies": [
        {
          "id": 723740,
          "author_name": "s4sarath",
          "author_url": "",
          "post_date": "01/20/2020 11:51:54",
          "content": "<p>Totally agree with that. I was wondering about transformer encoders only. Like T5 ( text to text ), ALBERT, etc etc .</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 723907,
      "author_name": "kentaronakanishi",
      "author_url": "",
      "post_date": "01/20/2020 15:41:04",
      "content": "<p>One of the another idea is pre-train not only encoder but dense layer for position logits.\nI've tried squad pre-trained model by huggingface transformers, however no improvements on LB.</p>",
      "votes": null,
      "replies": [
        {
          "id": 723934,
          "author_name": "s4sarath",
          "author_url": "",
          "post_date": "01/20/2020 16:01:05",
          "content": "<p>I tried Albert. I think the Crux is post result transformation, rather than choosing models like Bert, xlnet etc.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "723728": "Just want to know, any models other than BERT people have used here?",
    "723736": "Do you mean other than transformer encoders? I believe there are no chance to score &gt; 0.6 with old good classic approaches.",
    "723740": "Totally agree with that. I was wondering about transformer encoders only. Like T5 ( text to text ), ALBERT, etc etc .",
    "723907": "One of the another idea is pre-train not only encoder but dense layer for position logits.\nI've tried squad pre-trained model by huggingface transformers, however no improvements on LB.",
    "723934": "I tried Albert. I think the Crux is post result transformation, rather than choosing models like Bert, xlnet etc."
  },
  "source": "meta"
}