{
  "id": 119363,
  "title": "Requesting clarifications on submissions",
  "url": "/competitions/tensorflow2-question-answering/discussion/119363",
  "author_name": "",
  "post_date": "2019-11-28T08:58:11.241101600Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I have the following questions:</p>\n\n<ol>\n<li><p>Is it okay to select a shorter span within the annotated long answer?\nFor example, consider -1706790511507651062 from the training dataset. The annotated long answer is 932:1489 but inside this is a non-top-level candidate 1147:1159. Is it okay to select the latter as the answer? The annotated short answer for this example is 1155:1157. </p></li>\n<li><p>Will private test data have long answer candidates <code>long_answer_candidates</code>?\nI'm assuming this is the case but it will be good if someone can confirm.</p></li>\n<li><p>How to submit short answers when there multiple spans?\nAn example of this is 8510065051410209672 in the training dataset. The question is \"who wrote the song all i ask by adele\". The answer has four spans: Adele (289:290), Bruno Mars (291:293), Philip Lawrence (294:296), Christopher Brody Brown (297:300).\nI believe submitting only one short answer is not enough. Does this mean that submission.csv will have two instances of 8510065051410209672_short?</p></li>\n</ol>",
  "messages": [
    {
      "id": "683266",
      "postDate": "11/28/2019 08:58:11",
      "content": "<p>I have the following questions:</p>\n\n<ol>\n<li><p>Is it okay to select a shorter span within the annotated long answer?\nFor example, consider -1706790511507651062 from the training dataset. The annotated long answer is 932:1489 but inside this is a non-top-level candidate 1147:1159. Is it okay to select the latter as the answer? The annotated short answer for this example is 1155:1157. </p></li>\n<li><p>Will private test data have long answer candidates <code>long_answer_candidates</code>?\nI'm assuming this is the case but it will be good if someone can confirm.</p></li>\n<li><p>How to submit short answers when there multiple spans?\nAn example of this is 8510065051410209672 in the training dataset. The question is \"who wrote the song all i ask by adele\". The answer has four spans: Adele (289:290), Bruno Mars (291:293), Philip Lawrence (294:296), Christopher Brody Brown (297:300).\nI believe submitting only one short answer is not enough. Does this mean that submission.csv will have two instances of 8510065051410209672_short?</p></li>\n</ol>",
      "rawMarkdown": "I have the following questions:\n\n1. Is it okay to select a shorter span within the annotated long answer?\nFor example, consider -1706790511507651062 from the training dataset. The annotated long answer is 932:1489 but inside this is a non-top-level candidate 1147:1159. Is it okay to select the latter as the answer? The annotated short answer for this example is 1155:1157. \n\n2. Will private test data have long answer candidates `long_answer_candidates`?\nI'm assuming this is the case but it will be good if someone can confirm.\n\n3. How to submit short answers when there multiple spans?\nAn example of this is 8510065051410209672 in the training dataset. The question is \"who wrote the song all i ask by adele\". The answer has four spans: Adele (289:290), Bruno Mars (291:293), Philip Lawrence (294:296), Christopher Brody Brown (297:300).\nI believe submitting only one short answer is not enough. Does this mean that submission.csv will have two instances of 8510065051410209672_short?",
      "votes": null
    },
    {
      "id": "683367",
      "postDate": "11/28/2019 10:54:46",
      "content": "<p>I think all were already answered:\n1. You need to select exactly correct long answer, it does not matter how they are nested\n2. Yes (if not our scripts would have failed to submit)\n3. Unless something changes, we should predict one short answer max, and if this answer matches any of the annotated short answers, we are good.</p>",
      "rawMarkdown": "I think all were already answered:\n1. You need to select exactly correct long answer, it does not matter how they are nested\n2. Yes (if not our scripts would have failed to submit)\n3. Unless something changes, we should predict one short answer max, and if this answer matches any of the annotated short answers, we are good.",
      "votes": null
    },
    {
      "id": "683402",
      "postDate": "11/28/2019 11:51:53",
      "content": "<p>Thanks.</p>",
      "rawMarkdown": "Thanks.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 683367,
      "author_name": "lopuhin",
      "author_url": "",
      "post_date": "11/28/2019 10:54:46",
      "content": "<p>I think all were already answered:\n1. You need to select exactly correct long answer, it does not matter how they are nested\n2. Yes (if not our scripts would have failed to submit)\n3. Unless something changes, we should predict one short answer max, and if this answer matches any of the annotated short answers, we are good.</p>",
      "votes": null,
      "replies": [
        {
          "id": 683402,
          "author_name": "arvindpdmn",
          "author_url": "",
          "post_date": "11/28/2019 11:51:53",
          "content": "<p>Thanks.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "683266": "I have the following questions:\n\n1. Is it okay to select a shorter span within the annotated long answer?\nFor example, consider -1706790511507651062 from the training dataset. The annotated long answer is 932:1489 but inside this is a non-top-level candidate 1147:1159. Is it okay to select the latter as the answer? The annotated short answer for this example is 1155:1157. \n\n2. Will private test data have long answer candidates `long_answer_candidates`?\nI'm assuming this is the case but it will be good if someone can confirm.\n\n3. How to submit short answers when there multiple spans?\nAn example of this is 8510065051410209672 in the training dataset. The question is \"who wrote the song all i ask by adele\". The answer has four spans: Adele (289:290), Bruno Mars (291:293), Philip Lawrence (294:296), Christopher Brody Brown (297:300).\nI believe submitting only one short answer is not enough. Does this mean that submission.csv will have two instances of 8510065051410209672_short?",
    "683367": "I think all were already answered:\n1. You need to select exactly correct long answer, it does not matter how they are nested\n2. Yes (if not our scripts would have failed to submit)\n3. Unless something changes, we should predict one short answer max, and if this answer matches any of the annotated short answers, we are good.",
    "683402": "Thanks."
  },
  "source": "meta"
}