{
  "id": 119134,
  "title": "Evaluation Score Do not Change",
  "url": "/competitions/tensorflow2-question-answering/discussion/119134",
  "author_name": "",
  "post_date": "2019-11-26T19:19:10.009024700Z",
  "votes": 5,
  "comment_count": 4,
  "views": 0,
  "content": "<p>This <a href=\"https://www.kaggle.com/mmmarchetti/tensorflow-2-0-edited\">notebook</a> is an edition of <a href=\"https://www.kaggle.com/prokaj/bert-joint-baseline-notebook/notebook\">bert joint baseline notebook</a>. With some modifications, it was possible to slightly improve the code and get the YES / NO answers and leave the unknowns blank.\nBut LB scoring is not changing, not for less or for more, so I am making the code public to see if anyone in the community can better understand.  No need to share code, just knowledge.</p>",
  "messages": [
    {
      "id": "681996",
      "postDate": "11/26/2019 19:19:10",
      "content": "<p>This <a href=\"https://www.kaggle.com/mmmarchetti/tensorflow-2-0-edited\">notebook</a> is an edition of <a href=\"https://www.kaggle.com/prokaj/bert-joint-baseline-notebook/notebook\">bert joint baseline notebook</a>. With some modifications, it was possible to slightly improve the code and get the YES / NO answers and leave the unknowns blank.\nBut LB scoring is not changing, not for less or for more, so I am making the code public to see if anyone in the community can better understand.  No need to share code, just knowledge.</p>",
      "rawMarkdown": "This [notebook](https://www.kaggle.com/mmmarchetti/tensorflow-2-0-edited) is an edition of [bert joint baseline notebook](https://www.kaggle.com/prokaj/bert-joint-baseline-notebook/notebook). With some modifications, it was possible to slightly improve the code and get the YES / NO answers and leave the unknowns blank.\nBut LB scoring is not changing, not for less or for more, so I am making the code public to see if anyone in the community can better understand.  No need to share code, just knowledge.",
      "votes": null
    },
    {
      "id": "682040",
      "postDate": "11/26/2019 20:44:10",
      "content": "<p>Thanks <a href=\"/mmmarchetti\">@mmmarchetti</a> </p>",
      "rawMarkdown": "Thanks @mmmarchetti",
      "votes": null
    },
    {
      "id": "682208",
      "postDate": "11/27/2019 04:59:12",
      "content": "<p><a href=\"/mmmarchetti\">@mmmarchetti</a> If the score is not changing that means earlier also you were predicting a wrong answer for that example and now with <code>YES/NO</code> is also wrong answer. That's what I understood from the LB metric, correct me if I am wrong.</p>",
      "rawMarkdown": "mmmarchetti If the score is not changing that means earlier also you were predicting a wrong answer for that example and now with `YES/NO` is also wrong answer. That's what I understood from the LB metric, correct me if I am wrong.",
      "votes": null
    },
    {
      "id": "682405",
      "postDate": "11/27/2019 12:00:40",
      "content": "<p>You're welcome.</p>",
      "rawMarkdown": "You're welcome.",
      "votes": null
    },
    {
      "id": "682420",
      "postDate": "11/27/2019 12:19:10",
      "content": "<p>Now that you mention it, it makes sense. If previously I was predicting wrongly and just substituting for another wrong answer, the score tends not to change.\nIf we just take this into account, it means that the changes affected only the answers that were already wrong.</p>",
      "rawMarkdown": "Now that you mention it, it makes sense. If previously I was predicting wrongly and just substituting for another wrong answer, the score tends not to change.\nIf we just take this into account, it means that the changes affected only the answers that were already wrong.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 682040,
      "author_name": "ymcdull",
      "author_url": "",
      "post_date": "11/26/2019 20:44:10",
      "content": "<p>Thanks <a href=\"/mmmarchetti\">@mmmarchetti</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 682405,
          "author_name": "mmmarchetti",
          "author_url": "",
          "post_date": "11/27/2019 12:00:40",
          "content": "<p>You're welcome.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 682208,
      "author_name": "axel81",
      "author_url": "",
      "post_date": "11/27/2019 04:59:12",
      "content": "<p><a href=\"/mmmarchetti\">@mmmarchetti</a> If the score is not changing that means earlier also you were predicting a wrong answer for that example and now with <code>YES/NO</code> is also wrong answer. That's what I understood from the LB metric, correct me if I am wrong.</p>",
      "votes": null,
      "replies": [
        {
          "id": 682420,
          "author_name": "mmmarchetti",
          "author_url": "",
          "post_date": "11/27/2019 12:19:10",
          "content": "<p>Now that you mention it, it makes sense. If previously I was predicting wrongly and just substituting for another wrong answer, the score tends not to change.\nIf we just take this into account, it means that the changes affected only the answers that were already wrong.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "681996": "This [notebook](https://www.kaggle.com/mmmarchetti/tensorflow-2-0-edited) is an edition of [bert joint baseline notebook](https://www.kaggle.com/prokaj/bert-joint-baseline-notebook/notebook). With some modifications, it was possible to slightly improve the code and get the YES / NO answers and leave the unknowns blank.\nBut LB scoring is not changing, not for less or for more, so I am making the code public to see if anyone in the community can better understand.  No need to share code, just knowledge.",
    "682040": "Thanks @mmmarchetti",
    "682208": "mmmarchetti If the score is not changing that means earlier also you were predicting a wrong answer for that example and now with `YES/NO` is also wrong answer. That's what I understood from the LB metric, correct me if I am wrong.",
    "682405": "You're welcome.",
    "682420": "Now that you mention it, it makes sense. If previously I was predicting wrongly and just substituting for another wrong answer, the score tends not to change.\nIf we just take this into account, it means that the changes affected only the answers that were already wrong."
  },
  "source": "meta"
}