{
  "id": 122976,
  "title": "What does score < 1.5 mean?",
  "url": "/competitions/tensorflow2-question-answering/discussion/122976",
  "author_name": "",
  "post_date": "2019-12-24T00:31:50.533581300Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi there,\nI'm reading through <a href=\"https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers\">https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers</a> and don't quite understand the following where they check score is less than 1.5.\nWhere is the 1.5 value coming from? Is it just heuristic hyper parameter? \n```\ndef create_long_answer(entry):</p>\n\n<pre><code>answer = []\n\nif entry['answer_type'] == 0:\n    return ''\n\nelif entry[\"long_answer_score\"] &lt; 1.5:\n    return \"\"\n\nelif entry[\"long_answer\"][\"start_token\"] &gt; -1:\n    answer.append(str(entry[\"long_answer\"][\"start_token\"]) + \":\" + str(entry[\"long_answer\"][\"end_token\"]))\n    return \" \".join(answer)\n</code></pre>\n\n<p>```</p>\n\n<p>I'm getting negative score with my model, so most of my predictions are filtered out here :(</p>",
  "messages": [
    {
      "id": "701817",
      "postDate": "12/24/2019 00:31:50",
      "content": "<p>Hi there,\nI'm reading through <a href=\"https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers\">https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers</a> and don't quite understand the following where they check score is less than 1.5.\nWhere is the 1.5 value coming from? Is it just heuristic hyper parameter? \n```\ndef create_long_answer(entry):</p>\n\n<pre><code>answer = []\n\nif entry['answer_type'] == 0:\n    return ''\n\nelif entry[\"long_answer_score\"] &lt; 1.5:\n    return \"\"\n\nelif entry[\"long_answer\"][\"start_token\"] &gt; -1:\n    answer.append(str(entry[\"long_answer\"][\"start_token\"]) + \":\" + str(entry[\"long_answer\"][\"end_token\"]))\n    return \" \".join(answer)\n</code></pre>\n\n<p>```</p>\n\n<p>I'm getting negative score with my model, so most of my predictions are filtered out here :(</p>",
      "rawMarkdown": "Hi there,\nI'm reading through https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers and don't quite understand the following where they check score is less than 1.5.\nWhere is the 1.5 value coming from? Is it just heuristic hyper parameter? \n```\ndef create_long_answer(entry):\n    \n    answer = []\n    \n    if entry['answer_type'] == 0:\n        return ''\n    \n    elif entry[\"long_answer_score\"] &lt; 1.5:\n        return \"\"\n\n    elif entry[\"long_answer\"][\"start_token\"] &gt; -1:\n        answer.append(str(entry[\"long_answer\"][\"start_token\"]) + \":\" + str(entry[\"long_answer\"][\"end_token\"]))\n        return \" \".join(answer)\n```\n\nI'm getting negative score with my model, so most of my predictions are filtered out here :(",
      "votes": null
    },
    {
      "id": "701912",
      "postDate": "12/24/2019 03:33:07",
      "content": "<p><a href=\"/higepon\">@higepon</a> This is the confidence level below which no prediction will be made. The evaluation metric is micro-F1 loss. Hence, in cases where the GT is no prediction, to get a high score our model should also submit no prediction. This hyperparameter lets us adjust how tight or loose this condition is.</p>",
      "rawMarkdown": "higepon This is the confidence level below which no prediction will be made. The evaluation metric is micro-F1 loss. Hence, in cases where the GT is no prediction, to get a high score our model should also submit no prediction. This hyperparameter lets us adjust how tight or loose this condition is.",
      "votes": null
    },
    {
      "id": "701933",
      "postDate": "12/24/2019 04:27:06",
      "content": "<p>Thank you Rohit. Your explanation makes senes to me. Thanks for the clarification!</p>",
      "rawMarkdown": "Thank you Rohit. Your explanation makes senes to me. Thanks for the clarification!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 701912,
      "author_name": "rohitagarwal",
      "author_url": "",
      "post_date": "12/24/2019 03:33:07",
      "content": "<p><a href=\"/higepon\">@higepon</a> This is the confidence level below which no prediction will be made. The evaluation metric is micro-F1 loss. Hence, in cases where the GT is no prediction, to get a high score our model should also submit no prediction. This hyperparameter lets us adjust how tight or loose this condition is.</p>",
      "votes": null,
      "replies": [
        {
          "id": 701933,
          "author_name": "higepon",
          "author_url": "",
          "post_date": "12/24/2019 04:27:06",
          "content": "<p>Thank you Rohit. Your explanation makes senes to me. Thanks for the clarification!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "701817": "Hi there,\nI'm reading through https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers and don't quite understand the following where they check score is less than 1.5.\nWhere is the 1.5 value coming from? Is it just heuristic hyper parameter? \n```\ndef create_long_answer(entry):\n    \n    answer = []\n    \n    if entry['answer_type'] == 0:\n        return ''\n    \n    elif entry[\"long_answer_score\"] &lt; 1.5:\n        return \"\"\n\n    elif entry[\"long_answer\"][\"start_token\"] &gt; -1:\n        answer.append(str(entry[\"long_answer\"][\"start_token\"]) + \":\" + str(entry[\"long_answer\"][\"end_token\"]))\n        return \" \".join(answer)\n```\n\nI'm getting negative score with my model, so most of my predictions are filtered out here :(",
    "701912": "higepon This is the confidence level below which no prediction will be made. The evaluation metric is micro-F1 loss. Hence, in cases where the GT is no prediction, to get a high score our model should also submit no prediction. This hyperparameter lets us adjust how tight or loose this condition is.",
    "701933": "Thank you Rohit. Your explanation makes senes to me. Thanks for the clarification!"
  },
  "source": "meta"
}