{
  "id": 115048,
  "title": "Confuse about the kernel running time",
  "url": "/competitions/tensorflow2-question-answering/discussion/115048",
  "author_name": "",
  "post_date": "2019-10-31T00:35:28.997441700Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>It took me 30 minutes to run Bert-baseline model in my GPU-kernel, but after I submitted, the system spent way longer time (nearly 2 hours) to re-run my kernel. I am so confused. Is it because they use a larger unseen test set for evaluation? </p>",
  "messages": [
    {
      "id": "661991",
      "postDate": "10/31/2019 00:35:28",
      "content": "<p>It took me 30 minutes to run Bert-baseline model in my GPU-kernel, but after I submitted, the system spent way longer time (nearly 2 hours) to re-run my kernel. I am so confused. Is it because they use a larger unseen test set for evaluation? </p>",
      "rawMarkdown": "It took me 30 minutes to run Bert-baseline model in my GPU-kernel, but after I submitted, the system spent way longer time (nearly 2 hours) to re-run my kernel. I am so confused. Is it because they use a larger unseen test set for evaluation?",
      "votes": null
    },
    {
      "id": "662130",
      "postDate": "10/31/2019 05:54:01",
      "content": "<p>This is a code only competition. (:</p>",
      "rawMarkdown": "This is a code only competition. (:",
      "votes": null
    },
    {
      "id": "662595",
      "postDate": "10/31/2019 18:22:12",
      "content": "<p>Yes, the private test set is roughly 10x the size of the public test set (so roughly 2500 question-answer pairs). So you should expect the submission to require some factor longer to run.</p>",
      "rawMarkdown": "Yes, the private test set is roughly 10x the size of the public test set (so roughly 2500 question-answer pairs). So you should expect the submission to require some factor longer to run.",
      "votes": null
    },
    {
      "id": "662631",
      "postDate": "10/31/2019 19:20:51",
      "content": "<p>In addition to the longer runtime, my notebook returned a submission scoring error, but without any further descriptions. In other words, I have no idea what went wrong. It wasn't the runtime because this was a simple baseline that only takes to run for 7 minutes on the public test set.</p>",
      "rawMarkdown": "In addition to the longer runtime, my notebook returned a submission scoring error, but without any further descriptions. In other words, I have no idea what went wrong. It wasn't the runtime because this was a simple baseline that only takes to run for 7 minutes on the public test set.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 662130,
      "author_name": "qhd0081",
      "author_url": "",
      "post_date": "10/31/2019 05:54:01",
      "content": "<p>This is a code only competition. (:</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 662595,
      "author_name": "juliaelliott",
      "author_url": "",
      "post_date": "10/31/2019 18:22:12",
      "content": "<p>Yes, the private test set is roughly 10x the size of the public test set (so roughly 2500 question-answer pairs). So you should expect the submission to require some factor longer to run.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 662631,
      "author_name": "fschilder",
      "author_url": "",
      "post_date": "10/31/2019 19:20:51",
      "content": "<p>In addition to the longer runtime, my notebook returned a submission scoring error, but without any further descriptions. In other words, I have no idea what went wrong. It wasn't the runtime because this was a simple baseline that only takes to run for 7 minutes on the public test set.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "661991": "It took me 30 minutes to run Bert-baseline model in my GPU-kernel, but after I submitted, the system spent way longer time (nearly 2 hours) to re-run my kernel. I am so confused. Is it because they use a larger unseen test set for evaluation?",
    "662130": "This is a code only competition. (:",
    "662595": "Yes, the private test set is roughly 10x the size of the public test set (so roughly 2500 question-answer pairs). So you should expect the submission to require some factor longer to run.",
    "662631": "In addition to the longer runtime, my notebook returned a submission scoring error, but without any further descriptions. In other words, I have no idea what went wrong. It wasn't the runtime because this was a simple baseline that only takes to run for 7 minutes on the public test set."
  },
  "source": "meta"
}