{
  "id": 191932,
  "title": "What about the ground truth of the modes of agents?",
  "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/191932",
  "author_name": "Yannik",
  "post_date": "2020-10-19T12:55:30.958000",
  "votes": 0,
  "comment_count": 6,
  "views": 0,
  "content": "<p>When we are training the multi-mode <a href=\"https://www.kaggle.com/huanvo/lyft-complete-train-and-prediction-pipeline\" target=\"_blank\">baseline</a>, we got <strong>confidences</strong> as a part of the output. However, we have no true label in the data, as shown in the following screenshot,  for the backpropagation. How can we adjust the multi-mode network to predict the mode of agents without any true label? Are the confidences we predict is totally meaningless?<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3722352%2Fb8b77b30ad92549cd6131090347821c2%2F1401603111927_.pic_hd.jpg?generation=1603112114960662&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": 1053886,
      "postDate": "2020-10-19T13:07:17.090Z",
      "content": "<p><strong>target_positions</strong>  : The coordinates (in agent reference system) of the AV in the future. Unit is meters<br>\n<a href=\"https://github.com/lyft/l5kit/blob/03eb9e037d23940a134e27c9f124021e18982020/data_format.md\" target=\"_blank\">https://github.com/lyft/l5kit/blob/03eb9e037d23940a134e27c9f124021e18982020/data_format.md</a></p>",
      "rawMarkdown": "**target_positions**  : The coordinates (in agent reference system) of the AV in the future. Unit is meters\nhttps://github.com/lyft/l5kit/blob/03eb9e037d23940a134e27c9f124021e18982020/data_format.md",
      "votes": 1,
      "replies": [
        {
          "id": 1053894,
          "postDate": "2020-10-19T13:17:08.930Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 1053883,
      "postDate": "2020-10-19T13:02:41.233Z",
      "content": "<p><code>target_positions</code> and <code>target_availabilities</code> are the labels you're looking for</p>",
      "rawMarkdown": "``target_positions`` and ``target_availabilities`` are the labels you're looking for",
      "votes": 1,
      "replies": [
        {
          "id": 1053892,
          "postDate": "2020-10-19T13:14:21.933Z",
          "content": "<p>So we do not actually need to predict the mode of agents like what the baseline I cite above? </p>",
          "rawMarkdown": "So we do not actually need to predict the mode of agents like what the baseline I cite above? "
        },
        {
          "id": 1053897,
          "postDate": "2020-10-19T13:19:45.153Z",
          "content": "<p>You can choose whether to predict multiple modes, or just a single mode. Multiple modes make sense in this context as that's the format in which we are being measured. </p>\n<p>You predict multiple modes, but the <a href=\"https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/overview/evaluation\" target=\"_blank\">negative log likelihood metric</a> measures these against a single truth.  See <a href=\"https://github.com/lyft/l5kit/blob/20ab033c01610d711c3d36e1963ecec86e8b85b6/l5kit/l5kit/evaluation/metrics.py#L4\" target=\"_blank\">here</a></p>\n<p><a href=\"https://www.kaggle.com/corochann/lyft-training-with-multi-mode-confidence\" target=\"_blank\">This</a> notebook is a good starting point.</p>",
          "rawMarkdown": "You can choose whether to predict multiple modes, or just a single mode. Multiple modes make sense in this context as that's the format in which we are being measured. \n\nYou predict multiple modes, but the [negative log likelihood metric](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/overview/evaluation) measures these against a single truth.  See [here](https://github.com/lyft/l5kit/blob/20ab033c01610d711c3d36e1963ecec86e8b85b6/l5kit/l5kit/evaluation/metrics.py#L4)\n\n[This](https://www.kaggle.com/corochann/lyft-training-with-multi-mode-confidence) notebook is a good starting point.",
          "votes": 4
        },
        {
          "id": 1053899,
          "postDate": "2020-10-19T13:26:39.123Z",
          "content": "<p>It seems that I misunderstood what the <code>conf_0,1,2</code> actually used for… Thank you so much.</p>",
          "rawMarkdown": "It seems that I misunderstood what the `conf_0,1,2` actually used for... Thank you so much.",
          "votes": 1
        }
      ]
    },
    {
      "id": 1053879,
      "postDate": "2020-10-19T12:55:30.957Z",
      "content": "<p>When we are training the multi-mode <a href=\"https://www.kaggle.com/huanvo/lyft-complete-train-and-prediction-pipeline\" target=\"_blank\">baseline</a>, we got <strong>confidences</strong> as a part of the output. However, we have no true label in the data, as shown in the following screenshot,  for the backpropagation. How can we adjust the multi-mode network to predict the mode of agents without any true label? Are the confidences we predict is totally meaningless?<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3722352%2Fb8b77b30ad92549cd6131090347821c2%2F1401603111927_.pic_hd.jpg?generation=1603112114960662&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "When we are training the multi-mode [baseline](https://www.kaggle.com/huanvo/lyft-complete-train-and-prediction-pipeline), we got **confidences** as a part of the output. However, we have no true label in the data, as shown in the following screenshot,  for the backpropagation. How can we adjust the multi-mode network to predict the mode of agents without any true label? Are the confidences we predict is totally meaningless?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3722352%2Fb8b77b30ad92549cd6131090347821c2%2F1401603111927_.pic_hd.jpg?generation=1603112114960662&alt=media)\n"
    }
  ],
  "comments": [
    {
      "id": 1053886,
      "author_name": "SeshuRaju 🧘‍♂️",
      "author_url": "",
      "post_date": "2020-10-19T13:07:17.090000",
      "content": "<p><strong>target_positions</strong>  : The coordinates (in agent reference system) of the AV in the future. Unit is meters<br>\n<a href=\"https://github.com/lyft/l5kit/blob/03eb9e037d23940a134e27c9f124021e18982020/data_format.md\" target=\"_blank\">https://github.com/lyft/l5kit/blob/03eb9e037d23940a134e27c9f124021e18982020/data_format.md</a></p>",
      "votes": 1,
      "replies": [
        {
          "id": 1053894,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-10-19T13:17:08.930000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1053883,
      "author_name": "fergusoci",
      "author_url": "",
      "post_date": "2020-10-19T13:02:41.233000",
      "content": "<p><code>target_positions</code> and <code>target_availabilities</code> are the labels you're looking for</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1053892,
          "author_name": "Yannik",
          "author_url": "",
          "post_date": "2020-10-19T13:14:21.933000",
          "content": "<p>So we do not actually need to predict the mode of agents like what the baseline I cite above? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1053897,
          "author_name": "fergusoci",
          "author_url": "",
          "post_date": "2020-10-19T13:19:45.153000",
          "content": "<p>You can choose whether to predict multiple modes, or just a single mode. Multiple modes make sense in this context as that's the format in which we are being measured. </p>\n<p>You predict multiple modes, but the <a href=\"https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/overview/evaluation\" target=\"_blank\">negative log likelihood metric</a> measures these against a single truth.  See <a href=\"https://github.com/lyft/l5kit/blob/20ab033c01610d711c3d36e1963ecec86e8b85b6/l5kit/l5kit/evaluation/metrics.py#L4\" target=\"_blank\">here</a></p>\n<p><a href=\"https://www.kaggle.com/corochann/lyft-training-with-multi-mode-confidence\" target=\"_blank\">This</a> notebook is a good starting point.</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1053899,
          "author_name": "Yannik",
          "author_url": "",
          "post_date": "2020-10-19T13:26:39.123000",
          "content": "<p>It seems that I misunderstood what the <code>conf_0,1,2</code> actually used for… Thank you so much.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1053886": "**target_positions**  : The coordinates (in agent reference system) of the AV in the future. Unit is meters\nhttps://github.com/lyft/l5kit/blob/03eb9e037d23940a134e27c9f124021e18982020/data_format.md",
    "1053883": "``target_positions`` and ``target_availabilities`` are the labels you're looking for",
    "1053879": "When we are training the multi-mode [baseline](https://www.kaggle.com/huanvo/lyft-complete-train-and-prediction-pipeline), we got **confidences** as a part of the output. However, we have no true label in the data, as shown in the following screenshot,  for the backpropagation. How can we adjust the multi-mode network to predict the mode of agents without any true label? Are the confidences we predict is totally meaningless?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3722352%2Fb8b77b30ad92549cd6131090347821c2%2F1401603111927_.pic_hd.jpg?generation=1603112114960662&alt=media)\n"
  }
}