{
  "id": 196107,
  "title": "Understanding Scoring metrics",
  "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/196107",
  "author_name": "",
  "post_date": "2020-11-09T12:20:22.371827900Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I am kind of lost understanding of the scoring calculations. Can anyone help me understand them with a beginner-friendly perspective? </p>",
  "messages": [
    {
      "id": "1073303",
      "postDate": "11/09/2020 12:20:22",
      "content": "<p>I am kind of lost understanding of the scoring calculations. Can anyone help me understand them with a beginner-friendly perspective? </p>",
      "rawMarkdown": "I am kind of lost understanding of the scoring calculations. Can anyone help me understand them with a beginner-friendly perspective?",
      "votes": null
    },
    {
      "id": "1073637",
      "postDate": "11/09/2020 18:20:39",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F3665f7d701b62b803c0102b51f47d533%2FSelection_043.png?generation=1604946037906933&amp;alt=media\" alt=\"\"></p>\n<p>in summary,</p>\n<ul>\n<li>when there is multi-mode, gives highest confidence to the prediction with lowest l2-loss trajectory</li>\n</ul>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F3665f7d701b62b803c0102b51f47d533%2FSelection_043.png?generation=1604946037906933&alt=media)\n\nin summary,\n- when there is multi-mode, gives highest confidence to the prediction with lowest l2-loss trajectory",
      "votes": null
    },
    {
      "id": "1073727",
      "postDate": "11/09/2020 21:26:42",
      "content": "<p>Thank you so much! I guess I got lost understanding it step by step. I will remember to directly see the final form of the metric.   I was actually trying to understand the line with the arrow.<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5433316%2F9c8766107152044c2728f52205aa4620%2Floss.jpg?generation=1604956729943288&amp;alt=media\" alt=\"\"> </p>\n<p>But, I totally understand what you said. How does one understand those in-between derivations without knowing what each term means? Like, what are those N's . And, its mentioned that \" We assume the ground truth positions to be modelled by a mixture of multi-dimensional independent Normal distributions over time, yielding the likelihood\" . Here, I was confused about why they have put x1…t and y1…t in the numerator.</p>\n<p>After thinking about it now, I guess what they mean is.. We will be predicting our trajectory. And, our trajectory will be an n-dimensional independent normal distribution over time. In our pdf, what's the likelihood of the actual trajectory! </p>\n<p>Am I thinking it right?</p>",
      "rawMarkdown": "Thank you so much! I guess I got lost understanding it step by step. I will remember to directly see the final form of the metric.   I was actually trying to understand the line with the arrow.![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5433316%2F9c8766107152044c2728f52205aa4620%2Floss.jpg?generation=1604956729943288&alt=media) \n\nBut, I totally understand what you said. How does one understand those in-between derivations without knowing what each term means? Like, what are those N's . And, its mentioned that \" We assume the ground truth positions to be modelled by a mixture of multi-dimensional independent Normal distributions over time, yielding the likelihood\" . Here, I was confused about why they have put x1...t and y1...t in the numerator.\n\nAfter thinking about it now, I guess what they mean is.. We will be predicting our trajectory. And, our trajectory will be an n-dimensional independent normal distribution over time. In our pdf, what's the likelihood of the actual trajectory! \n\nAm I thinking it right?",
      "votes": null
    },
    {
      "id": "1078405",
      "postDate": "11/14/2020 18:32:37",
      "content": "<p>N is the normal distribution. N(x_pred|x_true) = exp(-(x_pred - x_true)^2/2) at single time step<br>\nAnd same for y. This assumption is how you get mean square error as the loss function for most of the machine learning problem.<br>\n(Though I think there is some (unimportant) normalization factor that we miss from this formula.)</p>",
      "rawMarkdown": "N is the normal distribution. N(x_pred|x_true) = exp(-(x_pred - x_true)^2/2) at single time step\nAnd same for y. This assumption is how you get mean square error as the loss function for most of the machine learning problem.\n(Though I think there is some (unimportant) normalization factor that we miss from this formula.)",
      "votes": null
    },
    {
      "id": "1078765",
      "postDate": "11/15/2020 09:28:38",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fdab91ba68edc714d8f10a44f4de36e16%2FSelection_187.png?generation=1605432500208528&amp;alt=media\" alt=\"\"></p>\n<p>Lecture 10.2 — Mixtures of Experts [Neural Networks for Machine Learning]</p>\n<p>Lecture from the course Neural Networks for Machine Learning, as taught by Geoffrey Hinton (University of Toronto) on Coursera in 2012. </p>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fdab91ba68edc714d8f10a44f4de36e16%2FSelection_187.png?generation=1605432500208528&alt=media)\n\nLecture 10.2 — Mixtures of Experts [Neural Networks for Machine Learning]\n\nLecture from the course Neural Networks for Machine Learning, as taught by Geoffrey Hinton (University of Toronto) on Coursera in 2012.",
      "votes": null
    },
    {
      "id": "1078772",
      "postDate": "11/15/2020 09:34:27",
      "content": "<p>Thank you!</p>",
      "rawMarkdown": "Thank you!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1073637,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "11/09/2020 18:20:39",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F3665f7d701b62b803c0102b51f47d533%2FSelection_043.png?generation=1604946037906933&amp;alt=media\" alt=\"\"></p>\n<p>in summary,</p>\n<ul>\n<li>when there is multi-mode, gives highest confidence to the prediction with lowest l2-loss trajectory</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 1073727,
          "author_name": "rawwar",
          "author_url": "",
          "post_date": "11/09/2020 21:26:42",
          "content": "<p>Thank you so much! I guess I got lost understanding it step by step. I will remember to directly see the final form of the metric.   I was actually trying to understand the line with the arrow.<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5433316%2F9c8766107152044c2728f52205aa4620%2Floss.jpg?generation=1604956729943288&amp;alt=media\" alt=\"\"> </p>\n<p>But, I totally understand what you said. How does one understand those in-between derivations without knowing what each term means? Like, what are those N's . And, its mentioned that \" We assume the ground truth positions to be modelled by a mixture of multi-dimensional independent Normal distributions over time, yielding the likelihood\" . Here, I was confused about why they have put x1…t and y1…t in the numerator.</p>\n<p>After thinking about it now, I guess what they mean is.. We will be predicting our trajectory. And, our trajectory will be an n-dimensional independent normal distribution over time. In our pdf, what's the likelihood of the actual trajectory! </p>\n<p>Am I thinking it right?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1078405,
          "author_name": "louis925",
          "author_url": "",
          "post_date": "11/14/2020 18:32:37",
          "content": "<p>N is the normal distribution. N(x_pred|x_true) = exp(-(x_pred - x_true)^2/2) at single time step<br>\nAnd same for y. This assumption is how you get mean square error as the loss function for most of the machine learning problem.<br>\n(Though I think there is some (unimportant) normalization factor that we miss from this formula.)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1078765,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "11/15/2020 09:28:38",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fdab91ba68edc714d8f10a44f4de36e16%2FSelection_187.png?generation=1605432500208528&amp;alt=media\" alt=\"\"></p>\n<p>Lecture 10.2 — Mixtures of Experts [Neural Networks for Machine Learning]</p>\n<p>Lecture from the course Neural Networks for Machine Learning, as taught by Geoffrey Hinton (University of Toronto) on Coursera in 2012. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1078772,
          "author_name": "rawwar",
          "author_url": "",
          "post_date": "11/15/2020 09:34:27",
          "content": "<p>Thank you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1073303": "I am kind of lost understanding of the scoring calculations. Can anyone help me understand them with a beginner-friendly perspective?",
    "1073637": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F3665f7d701b62b803c0102b51f47d533%2FSelection_043.png?generation=1604946037906933&alt=media)\n\nin summary,\n- when there is multi-mode, gives highest confidence to the prediction with lowest l2-loss trajectory",
    "1073727": "Thank you so much! I guess I got lost understanding it step by step. I will remember to directly see the final form of the metric.   I was actually trying to understand the line with the arrow.![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5433316%2F9c8766107152044c2728f52205aa4620%2Floss.jpg?generation=1604956729943288&alt=media) \n\nBut, I totally understand what you said. How does one understand those in-between derivations without knowing what each term means? Like, what are those N's . And, its mentioned that \" We assume the ground truth positions to be modelled by a mixture of multi-dimensional independent Normal distributions over time, yielding the likelihood\" . Here, I was confused about why they have put x1...t and y1...t in the numerator.\n\nAfter thinking about it now, I guess what they mean is.. We will be predicting our trajectory. And, our trajectory will be an n-dimensional independent normal distribution over time. In our pdf, what's the likelihood of the actual trajectory! \n\nAm I thinking it right?",
    "1078405": "N is the normal distribution. N(x_pred|x_true) = exp(-(x_pred - x_true)^2/2) at single time step\nAnd same for y. This assumption is how you get mean square error as the loss function for most of the machine learning problem.\n(Though I think there is some (unimportant) normalization factor that we miss from this formula.)",
    "1078765": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fdab91ba68edc714d8f10a44f4de36e16%2FSelection_187.png?generation=1605432500208528&alt=media)\n\nLecture 10.2 — Mixtures of Experts [Neural Networks for Machine Learning]\n\nLecture from the course Neural Networks for Machine Learning, as taught by Geoffrey Hinton (University of Toronto) on Coursera in 2012.",
    "1078772": "Thank you!"
  },
  "source": "meta"
}