Hi, \nI had a question about the training data - there's a flag in the column long_answer_candidates called 'top_level'. Can anyone clarify what this flag is indicating?
\n\nThanks!
", "messages": [ { "id": 665351, "postDate": "2019-11-04T22:54:36.587Z", "content": "from https://github.com/google-research-datasets/natural-questions/blob/master/README.md @afederation
\n\nLong Answer Candidates\nThe first task in Natural Questions is to identify the smallest HTML bounding box that contains all of the information required to infer the answer to a question. These long answers can be paragraphs, lists, list items, tables, or table rows. While the candidates can be inferred directly from the HTML or token sequence, we also include a list of long answer candidates for convenience. Each candidate is defined in terms of offsets into both the HTML and the document tokens. As with all other annotations, start offsets are inclusive and end offsets are exclusive.
\n\n\n\"long_answer_candidates\": [\n { \"start_byte\": 32, \"end_byte\": 106, \"start_token\": 5, \"end_token\": 22, \"top_level\": true },\n { \"start_byte\": 65, \"end_byte\": 102, \"start_token\": 13, \"end_token\": 21, \"top_level\": false },\n\nIn this example, you can see that the second long answer candidate is contained within the first. We do not disallow nested long answer candidates, we just ask annotators to find the smallest candidate containing all of the information required to infer the answer to the question. However, we do observe that 95% of all long answers (including all paragraph answers) are not nested below any other candidates. Since we believe that some users may want to start by only considering non-overlapping candidates, we include a boolean flag top_level that identifies whether a candidate is nested below another (top_level = False) or not (top_level = True). Please be aware that this flag is only included for convenience and it is not related to the task definition in any way. For more information about the distribution of long answer types, please see the data statistics section below.
Hi, \nI had a question about the training data - there's a flag in the column long_answer_candidates called 'top_level'. Can anyone clarify what this flag is indicating?
\n\nThanks!
", "rawMarkdown": "Hi, \nI had a question about the training data - there's a flag in the column long_answer_candidates called 'top_level'. Can anyone clarify what this flag is indicating?\n\nThanks!", "votes": 5 }, { "id": 685638, "postDate": "2019-12-02T03:18:06.923Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 665351, "author_name": "SeshuRaju 🧘♂️", "author_url": "", "post_date": "2019-11-04T22:54:36.587000", "content": "from https://github.com/google-research-datasets/natural-questions/blob/master/README.md @afederation
\n\nLong Answer Candidates\nThe first task in Natural Questions is to identify the smallest HTML bounding box that contains all of the information required to infer the answer to a question. These long answers can be paragraphs, lists, list items, tables, or table rows. While the candidates can be inferred directly from the HTML or token sequence, we also include a list of long answer candidates for convenience. Each candidate is defined in terms of offsets into both the HTML and the document tokens. As with all other annotations, start offsets are inclusive and end offsets are exclusive.
\n\n\n\"long_answer_candidates\": [\n { \"start_byte\": 32, \"end_byte\": 106, \"start_token\": 5, \"end_token\": 22, \"top_level\": true },\n { \"start_byte\": 65, \"end_byte\": 102, \"start_token\": 13, \"end_token\": 21, \"top_level\": false },\n\nIn this example, you can see that the second long answer candidate is contained within the first. We do not disallow nested long answer candidates, we just ask annotators to find the smallest candidate containing all of the information required to infer the answer to the question. However, we do observe that 95% of all long answers (including all paragraph answers) are not nested below any other candidates. Since we believe that some users may want to start by only considering non-overlapping candidates, we include a boolean flag top_level that identifies whether a candidate is nested below another (top_level = False) or not (top_level = True). Please be aware that this flag is only included for convenience and it is not related to the task definition in any way. For more information about the distribution of long answer types, please see the data statistics section below.