{
  "id": 125979,
  "title": "Bug in Quaternion-based Evaluation Code",
  "url": "/competitions/pku-autonomous-driving/discussion/125979",
  "author_name": "",
  "post_date": "2020-01-14T22:34:54.795753900Z",
  "votes": 15,
  "comment_count": 4,
  "views": 0,
  "content": "<p>The following code is given in the Overview/Evaluation section </p>\n\n<p>```\npublic static double RotationDistance(Object3D o1, Object3D o2)\n    {\n        Quaternion q1 = Quaternion.CreateFromYawPitchRoll(o1.yaw, o1.pitch, \n             o1.roll);\n        Quaternion q2 = Quaternion.CreateFromYawPitchRoll(o2.yaw, o2.pitch, \n             o2.roll);\n        Quaternion diff = Quaternion.Normalize(q1) * \n             Quaternion.Inverse(Quaternion.Normalize(q2));</p>\n\n<pre><code>    diff.W = Math.Clamp(diff.W, -1.0f, 1.0f);\n    return Object3D.RadianToDegree( Math.Acos(diff.W) );\n}\n\npublic static double TranslationDistance(Object3D o1, Object3D o2)\n{\n    var dx = o1.x - o2.x;\n    var dy = o1.y - o2.y;\n    var dz = o1.z - o2.z;\n\n    return Math.Sqrt(dx * dx + dy * dy + dz * dz);\n}\n</code></pre>\n\n<p>```</p>\n\n<p>First of all, a multiplication by 2 is missing: the first component (W) of a quaternion is equal to cos(angle/2), so it should say <code>2*Math.Acos(diff.W)</code>.</p>\n\n<p>Second, Math.Acos will give values between 0 and pi (NOT -pi/2 and pi/2), so RotationDistance may be close to 360 degrees if the prediction error in the angle is very small in a certain direction.</p>\n\n<p>This would hugely penalize very good predictions. Therefore, the last line should be <code>return Object3D.RadianToDegree(Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi));</code></p>\n\n<p>If the code is used in production as it is posted, it makes the scores (at least the aspect of the orientation-estimation) essentially <em>random</em>.</p>",
  "messages": [
    {
      "id": "718880",
      "postDate": "01/14/2020 22:34:54",
      "content": "<p>The following code is given in the Overview/Evaluation section </p>\n\n<p>```\npublic static double RotationDistance(Object3D o1, Object3D o2)\n    {\n        Quaternion q1 = Quaternion.CreateFromYawPitchRoll(o1.yaw, o1.pitch, \n             o1.roll);\n        Quaternion q2 = Quaternion.CreateFromYawPitchRoll(o2.yaw, o2.pitch, \n             o2.roll);\n        Quaternion diff = Quaternion.Normalize(q1) * \n             Quaternion.Inverse(Quaternion.Normalize(q2));</p>\n\n<pre><code>    diff.W = Math.Clamp(diff.W, -1.0f, 1.0f);\n    return Object3D.RadianToDegree( Math.Acos(diff.W) );\n}\n\npublic static double TranslationDistance(Object3D o1, Object3D o2)\n{\n    var dx = o1.x - o2.x;\n    var dy = o1.y - o2.y;\n    var dz = o1.z - o2.z;\n\n    return Math.Sqrt(dx * dx + dy * dy + dz * dz);\n}\n</code></pre>\n\n<p>```</p>\n\n<p>First of all, a multiplication by 2 is missing: the first component (W) of a quaternion is equal to cos(angle/2), so it should say <code>2*Math.Acos(diff.W)</code>.</p>\n\n<p>Second, Math.Acos will give values between 0 and pi (NOT -pi/2 and pi/2), so RotationDistance may be close to 360 degrees if the prediction error in the angle is very small in a certain direction.</p>\n\n<p>This would hugely penalize very good predictions. Therefore, the last line should be <code>return Object3D.RadianToDegree(Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi));</code></p>\n\n<p>If the code is used in production as it is posted, it makes the scores (at least the aspect of the orientation-estimation) essentially <em>random</em>.</p>",
      "rawMarkdown": "The following code is given in the Overview/Evaluation section \n\n```\npublic static double RotationDistance(Object3D o1, Object3D o2)\n    {\n        Quaternion q1 = Quaternion.CreateFromYawPitchRoll(o1.yaw, o1.pitch, \n             o1.roll);\n        Quaternion q2 = Quaternion.CreateFromYawPitchRoll(o2.yaw, o2.pitch, \n             o2.roll);\n        Quaternion diff = Quaternion.Normalize(q1) * \n             Quaternion.Inverse(Quaternion.Normalize(q2));\n\n        diff.W = Math.Clamp(diff.W, -1.0f, 1.0f);\n        return Object3D.RadianToDegree( Math.Acos(diff.W) );\n    }\n\n    public static double TranslationDistance(Object3D o1, Object3D o2)\n    {\n        var dx = o1.x - o2.x;\n        var dy = o1.y - o2.y;\n        var dz = o1.z - o2.z;\n\n        return Math.Sqrt(dx * dx + dy * dy + dz * dz);\n    }\n```\n\nFirst of all, a multiplication by 2 is missing: the first component (W) of a quaternion is equal to cos(angle/2), so it should say `2*Math.Acos(diff.W)`.\n\nSecond, Math.Acos will give values between 0 and pi (NOT -pi/2 and pi/2), so RotationDistance may be close to 360 degrees if the prediction error in the angle is very small in a certain direction.\n\nThis would hugely penalize very good predictions. Therefore, the last line should be `return Object3D.RadianToDegree(Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi));`\n\nIf the code is used in production as it is posted, it makes the scores (at least the aspect of the orientation-estimation) essentially *random*.",
      "votes": null
    },
    {
      "id": "720427",
      "postDate": "01/16/2020 12:24:19",
      "content": "<p>I hope you are right :D It would be funny...</p>",
      "rawMarkdown": "I hope you are right :D It would be funny...",
      "votes": null
    },
    {
      "id": "720465",
      "postDate": "01/16/2020 13:08:26",
      "content": "<p>It seems that my decision to stop solving this contest 1 week ago was correct </p>",
      "rawMarkdown": "It seems that my decision to stop solving this contest 1 week ago was correct",
      "votes": null
    },
    {
      "id": "721390",
      "postDate": "01/17/2020 10:37:59",
      "content": "<p>It seems like you have a small mistake in the last formula and it should be <code>Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi)</code></p>",
      "rawMarkdown": "It seems like you have a small mistake in the last formula and it should be `Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi)`",
      "votes": null
    },
    {
      "id": "723904",
      "postDate": "01/20/2020 15:33:39",
      "content": "<p>Thanks, I replaced it in my post.</p>",
      "rawMarkdown": "Thanks, I replaced it in my post.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 720427,
      "author_name": "dvorobiev",
      "author_url": "",
      "post_date": "01/16/2020 12:24:19",
      "content": "<p>I hope you are right :D It would be funny...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 720465,
      "author_name": "dukhovnik",
      "author_url": "",
      "post_date": "01/16/2020 13:08:26",
      "content": "<p>It seems that my decision to stop solving this contest 1 week ago was correct </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 721390,
      "author_name": "cortwave",
      "author_url": "",
      "post_date": "01/17/2020 10:37:59",
      "content": "<p>It seems like you have a small mistake in the last formula and it should be <code>Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi)</code></p>",
      "votes": null,
      "replies": [
        {
          "id": 723904,
          "author_name": "isarandi",
          "author_url": "",
          "post_date": "01/20/2020 15:33:39",
          "content": "<p>Thanks, I replaced it in my post.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "718880": "The following code is given in the Overview/Evaluation section \n\n```\npublic static double RotationDistance(Object3D o1, Object3D o2)\n    {\n        Quaternion q1 = Quaternion.CreateFromYawPitchRoll(o1.yaw, o1.pitch, \n             o1.roll);\n        Quaternion q2 = Quaternion.CreateFromYawPitchRoll(o2.yaw, o2.pitch, \n             o2.roll);\n        Quaternion diff = Quaternion.Normalize(q1) * \n             Quaternion.Inverse(Quaternion.Normalize(q2));\n\n        diff.W = Math.Clamp(diff.W, -1.0f, 1.0f);\n        return Object3D.RadianToDegree( Math.Acos(diff.W) );\n    }\n\n    public static double TranslationDistance(Object3D o1, Object3D o2)\n    {\n        var dx = o1.x - o2.x;\n        var dy = o1.y - o2.y;\n        var dz = o1.z - o2.z;\n\n        return Math.Sqrt(dx * dx + dy * dy + dz * dz);\n    }\n```\n\nFirst of all, a multiplication by 2 is missing: the first component (W) of a quaternion is equal to cos(angle/2), so it should say `2*Math.Acos(diff.W)`.\n\nSecond, Math.Acos will give values between 0 and pi (NOT -pi/2 and pi/2), so RotationDistance may be close to 360 degrees if the prediction error in the angle is very small in a certain direction.\n\nThis would hugely penalize very good predictions. Therefore, the last line should be `return Object3D.RadianToDegree(Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi));`\n\nIf the code is used in production as it is posted, it makes the scores (at least the aspect of the orientation-estimation) essentially *random*.",
    "720427": "I hope you are right :D It would be funny...",
    "720465": "It seems that my decision to stop solving this contest 1 week ago was correct",
    "721390": "It seems like you have a small mistake in the last formula and it should be `Math.Pi - Math.Abs(2*Math.Acos(diff.W)-Math.Pi)`",
    "723904": "Thanks, I replaced it in my post."
  },
  "source": "meta"
}