![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1855081%2F42d5e4d87b5f9206b1163c6a7254917a%2F2019-10-29_18h16_04.png?generation=1572341681476714&alt=media\")
*S*tanford *Qu*estion *A*nswering *D*ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
\n\nfrom:\nhttps://rajpurkar.github.io/SQuAD-explorer/
\n\nALBERT is \"A *L*ite\" version of BERT, a popular unsupervised language representation learning algorithm. ALBERT uses parameter-reduction techniques that allow for large-scale configurations, overcome previous memory limitations, and achieve better behavior with respect to model degradation.
\n\n
from:\nhttps://github.com/google-research/google-research/tree/master/albert\nhttps://arxiv.org/abs/1909.11942
", "messages": [ { "id": "660531", "postDate": "10/29/2019 09:22:39", "content": "*S*tanford *Qu*estion *A*nswering *D*ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
\n\nfrom:\nhttps://rajpurkar.github.io/SQuAD-explorer/
\n\nALBERT is \"A *L*ite\" version of BERT, a popular unsupervised language representation learning algorithm. ALBERT uses parameter-reduction techniques that allow for large-scale configurations, overcome previous memory limitations, and achieve better behavior with respect to model degradation.
\n\n
from:\nhttps://github.com/google-research/google-research/tree/master/albert\nhttps://arxiv.org/abs/1909.11942
", "rawMarkdown": "## What is SQuAD?\n**S**tanford **Qu**estion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.\n\nfrom:\nhttps://rajpurkar.github.io/SQuAD-explorer/\n\n## Leaderboard in SQuAD\n1. ALBERT (ensemble model)\n2. XLNet + DAAF + Verifier (ensemble)\n3. ALBERT (single model)\n\n\n## What is ALBERT?\nALBERT is \"**A** **L**ite\" version of **BERT**, a popular unsupervised language representation learning algorithm. ALBERT uses parameter-reduction techniques that allow for large-scale configurations, overcome previous memory limitations, and achieve better behavior with respect to model degradation.\n\n\n\n\nfrom:\nhttps://github.com/google-research/google-research/tree/master/albert\nhttps://arxiv.org/abs/1909.11942", "votes": null }, { "id": "662908", "postDate": "11/01/2019 07:17:45", "content": "SQuAD and RACE is just short passage(within in 500 token), not like this task.
", "rawMarkdown": "SQuAD and RACE is just short passage(within in 500 token), not like this task.", "votes": null }, { "id": "662916", "postDate": "11/01/2019 07:32:38", "content": "That's right. That's a difficult and interesting part of this task.\nI thought that SQuAD would be helpful because it is the closest to NQ among the existing NLP tasks. Also, NQ's baseline model (BERT) is pre-trained with SQuAD before fine-tuned with NQ.
\n\nReference:\nhttps://github.com/google-research/language/tree/master/language/question_answering/bert_joint
", "rawMarkdown": "That's right. That's a difficult and interesting part of this task.\nI thought that SQuAD would be helpful because it is the closest to NQ among the existing NLP tasks. Also, NQ's baseline model (BERT) is pre-trained with SQuAD before fine-tuned with NQ.\n\nReference:\nhttps://github.com/google-research/language/tree/master/language/question_answering/bert_joint", "votes": null }, { "id": "701903", "postDate": "12/24/2019 03:12:50", "content": "Hi, I have been looking everywhere but could not find an answer to this...
\n\nCan someone explain me what 'DAAF' and 'Verifier' mean? I mean what exactly are they doing on top of the model to make it better?
", "rawMarkdown": "Hi, I have been looking everywhere but could not find an answer to this...\n\nCan someone explain me what 'DAAF' and 'Verifier' mean? I mean what exactly are they doing on top of the model to make it better?", "votes": null }, { "id": "702337", "postDate": "12/24/2019 15:11:57", "content": "to manage ALBERT, you need at least:\n1. manage embedding layer\n2. managing tokenizing strategy
\n\nAfter I tried ALBERT, my CV has dropped by 6%
", "rawMarkdown": "to manage ALBERT, you need at least:\n1. manage embedding layer\n2. managing tokenizing strategy\n\nAfter I tried ALBERT, my CV has dropped by 6%", "votes": null } ], "comments": [ { "id": 662908, "author_name": "hzk123", "author_url": "", "post_date": "11/01/2019 07:17:45", "content": "SQuAD and RACE is just short passage(within in 500 token), not like this task.
", "votes": null, "replies": [ { "id": 662916, "author_name": "bluexleoxgreen", "author_url": "", "post_date": "11/01/2019 07:32:38", "content": "That's right. That's a difficult and interesting part of this task.\nI thought that SQuAD would be helpful because it is the closest to NQ among the existing NLP tasks. Also, NQ's baseline model (BERT) is pre-trained with SQuAD before fine-tuned with NQ.
\n\nReference:\nhttps://github.com/google-research/language/tree/master/language/question_answering/bert_joint
", "votes": null, "replies": [] } ] }, { "id": 701903, "author_name": "prakharg24", "author_url": "", "post_date": "12/24/2019 03:12:50", "content": "Hi, I have been looking everywhere but could not find an answer to this...
\n\nCan someone explain me what 'DAAF' and 'Verifier' mean? I mean what exactly are they doing on top of the model to make it better?
", "votes": null, "replies": [] }, { "id": 702337, "author_name": "httpwwwfszyc", "author_url": "", "post_date": "12/24/2019 15:11:57", "content": "to manage ALBERT, you need at least:\n1. manage embedding layer\n2. managing tokenizing strategy
\n\nAfter I tried ALBERT, my CV has dropped by 6%
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "660531": "## What is SQuAD?\n**S**tanford **Qu**estion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.\n\nfrom:\nhttps://rajpurkar.github.io/SQuAD-explorer/\n\n## Leaderboard in SQuAD\n1. ALBERT (ensemble model)\n2. XLNet + DAAF + Verifier (ensemble)\n3. ALBERT (single model)\n\n\n## What is ALBERT?\nALBERT is \"**A** **L**ite\" version of **BERT**, a popular unsupervised language representation learning algorithm. ALBERT uses parameter-reduction techniques that allow for large-scale configurations, overcome previous memory limitations, and achieve better behavior with respect to model degradation.\n\n\n\n\nfrom:\nhttps://github.com/google-research/google-research/tree/master/albert\nhttps://arxiv.org/abs/1909.11942", "662908": "SQuAD and RACE is just short passage(within in 500 token), not like this task.", "662916": "That's right. That's a difficult and interesting part of this task.\nI thought that SQuAD would be helpful because it is the closest to NQ among the existing NLP tasks. Also, NQ's baseline model (BERT) is pre-trained with SQuAD before fine-tuned with NQ.\n\nReference:\nhttps://github.com/google-research/language/tree/master/language/question_answering/bert_joint", "701903": "Hi, I have been looking everywhere but could not find an answer to this...\n\nCan someone explain me what 'DAAF' and 'Verifier' mean? I mean what exactly are they doing on top of the model to make it better?", "702337": "to manage ALBERT, you need at least:\n1. manage embedding layer\n2. managing tokenizing strategy\n\nAfter I tried ALBERT, my CV has dropped by 6%" }, "source": "meta" }