{
  "id": 127501,
  "title": "Evaluation Score",
  "url": "/competitions/pku-autonomous-driving/discussion/127501",
  "author_name": "",
  "post_date": "2020-01-24T09:55:03.063482900Z",
  "votes": 3,
  "comment_count": 1,
  "views": 0,
  "content": "<p>We noticed something strange with our evaluation scores. Our score significantly dropped when predictions could contain <code>roll</code> values smaller than <code>-pi</code> or larger than <code>pi</code>. Using the exact same predictions, but with <code>roll</code> in the <code>-pi ... pi</code> range (by adding or subtracting <code>2*pi</code>) our score improved significantly. Could this be a bug in the evaluation score calculations?</p>",
  "messages": [
    {
      "id": "727991",
      "postDate": "01/24/2020 09:55:03",
      "content": "<p>We noticed something strange with our evaluation scores. Our score significantly dropped when predictions could contain <code>roll</code> values smaller than <code>-pi</code> or larger than <code>pi</code>. Using the exact same predictions, but with <code>roll</code> in the <code>-pi ... pi</code> range (by adding or subtracting <code>2*pi</code>) our score improved significantly. Could this be a bug in the evaluation score calculations?</p>",
      "rawMarkdown": "We noticed something strange with our evaluation scores. Our score significantly dropped when predictions could contain `roll` values smaller than `-pi` or larger than `pi`. Using the exact same predictions, but with `roll` in the `-pi ... pi` range (by adding or subtracting `2*pi`) our score improved significantly. Could this be a bug in the evaluation score calculations?",
      "votes": null
    },
    {
      "id": "729599",
      "postDate": "01/26/2020 12:34:56",
      "content": "<p>yes, roll must be in the -pi to +pi range or you will get very low scores. One of the flaws in the metric ...</p>",
      "rawMarkdown": "yes, roll must be in the -pi to +pi range or you will get very low scores. One of the flaws in the metric ...",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 729599,
      "author_name": "ilu000",
      "author_url": "",
      "post_date": "01/26/2020 12:34:56",
      "content": "<p>yes, roll must be in the -pi to +pi range or you will get very low scores. One of the flaws in the metric ...</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "727991": "We noticed something strange with our evaluation scores. Our score significantly dropped when predictions could contain `roll` values smaller than `-pi` or larger than `pi`. Using the exact same predictions, but with `roll` in the `-pi ... pi` range (by adding or subtracting `2*pi`) our score improved significantly. Could this be a bug in the evaluation score calculations?",
    "729599": "yes, roll must be in the -pi to +pi range or you will get very low scores. One of the flaws in the metric ..."
  },
  "source": "meta"
}