{
  "id": 32483,
  "title": "Evaluation Criteria",
  "url": "/competitions/noaa-fisheries-steller-sea-lion-population-count/discussion/32483",
  "author_name": "",
  "post_date": "2017-05-04T02:59:14.960311400Z",
  "votes": 2,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I am confused by the evaluation criteria, since I have come up with two possible way to evaluate our predicted result with RMSE. </p>\n\n<p>Set the amount of picture as M, and we are asked to count the amount of K different kinds of sea lions.</p>\n\n<p>First possible way:\nCalculate the RMSE of each picture and then calculate the average of the RMSE of all the pictures.</p>\n\n<p>Second possible way:\nCalculate the RMSE of all the predicted result.</p>\n\n<p>Please take a look at the attached images of two equations to have a clearer understanding. \nThanks for any answers in advance!</p>",
  "messages": [
    {
      "id": "180084",
      "postDate": "05/04/2017 02:59:14",
      "content": "<p>I am confused by the evaluation criteria, since I have come up with two possible way to evaluate our predicted result with RMSE. </p>\n\n<p>Set the amount of picture as M, and we are asked to count the amount of K different kinds of sea lions.</p>\n\n<p>First possible way:\nCalculate the RMSE of each picture and then calculate the average of the RMSE of all the pictures.</p>\n\n<p>Second possible way:\nCalculate the RMSE of all the predicted result.</p>\n\n<p>Please take a look at the attached images of two equations to have a clearer understanding. \nThanks for any answers in advance!</p>",
      "rawMarkdown": "I am confused by the evaluation criteria, since I have come up with two possible way to evaluate our predicted result with RMSE. \n\nSet the amount of picture as M, and we are asked to count the amount of K different kinds of sea lions.\n\nFirst possible way:\nCalculate the RMSE of each picture and then calculate the average of the RMSE of all the pictures.\n\nSecond possible way:\nCalculate the RMSE of all the predicted result.\n\nPlease take a look at the attached images of two equations to have a clearer understanding. \nThanks for any answers in advance!",
      "votes": null
    },
    {
      "id": "180170",
      "postDate": "05/04/2017 10:06:45",
      "content": "<p>I think that the evaluation section is very clear about this: <a href=\"https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count#evaluation\">https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count#evaluation</a> \"Your submissions will be evaluated by their RMSE from the human-labelled ground truth, averaged over the columns.\" - that means you need to substitute K with M in your first formula - so you first compute RMSE for the first column (adult_males), then for the second, etc., and then average them. So the inner sum under the square root must be over pictures (M), not over classes (K).</p>",
      "rawMarkdown": "I think that the evaluation section is very clear about this: https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count#evaluation \"Your submissions will be evaluated by their RMSE from the human-labelled ground truth, averaged over the columns.\" - that means you need to substitute K with M in your first formula - so you first compute RMSE for the first column (adult_males), then for the second, etc., and then average them. So the inner sum under the square root must be over pictures (M), not over classes (K).",
      "votes": null
    },
    {
      "id": "180286",
      "postDate": "05/04/2017 18:48:39",
      "content": "<p>WOW. Thank you for the post.</p>\n\n<p>I actually compute the wrong rmse locally, which is like version 2 but km are under the square root.</p>",
      "rawMarkdown": "WOW. Thank you for the post.\n\nI actually compute the wrong rmse locally, which is like version 2 but km are under the square root.",
      "votes": null
    },
    {
      "id": "180290",
      "postDate": "05/04/2017 18:58:31",
      "content": "<p>Thank Konstantin for answering my question! It is clear and easy to understand. Thank you very much!</p>",
      "rawMarkdown": "Thank Konstantin for answering my question! It is clear and easy to understand. Thank you very much!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 180170,
      "author_name": "lopuhin",
      "author_url": "",
      "post_date": "05/04/2017 10:06:45",
      "content": "<p>I think that the evaluation section is very clear about this: <a href=\"https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count#evaluation\">https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count#evaluation</a> \"Your submissions will be evaluated by their RMSE from the human-labelled ground truth, averaged over the columns.\" - that means you need to substitute K with M in your first formula - so you first compute RMSE for the first column (adult_males), then for the second, etc., and then average them. So the inner sum under the square root must be over pictures (M), not over classes (K).</p>",
      "votes": null,
      "replies": [
        {
          "id": 180290,
          "author_name": "yuchunchien",
          "author_url": "",
          "post_date": "05/04/2017 18:58:31",
          "content": "<p>Thank Konstantin for answering my question! It is clear and easy to understand. Thank you very much!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 180286,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "05/04/2017 18:48:39",
      "content": "<p>WOW. Thank you for the post.</p>\n\n<p>I actually compute the wrong rmse locally, which is like version 2 but km are under the square root.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "180084": "I am confused by the evaluation criteria, since I have come up with two possible way to evaluate our predicted result with RMSE. \n\nSet the amount of picture as M, and we are asked to count the amount of K different kinds of sea lions.\n\nFirst possible way:\nCalculate the RMSE of each picture and then calculate the average of the RMSE of all the pictures.\n\nSecond possible way:\nCalculate the RMSE of all the predicted result.\n\nPlease take a look at the attached images of two equations to have a clearer understanding. \nThanks for any answers in advance!",
    "180170": "I think that the evaluation section is very clear about this: https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count#evaluation \"Your submissions will be evaluated by their RMSE from the human-labelled ground truth, averaged over the columns.\" - that means you need to substitute K with M in your first formula - so you first compute RMSE for the first column (adult_males), then for the second, etc., and then average them. So the inner sum under the square root must be over pictures (M), not over classes (K).",
    "180286": "WOW. Thank you for the post.\n\nI actually compute the wrong rmse locally, which is like version 2 but km are under the square root.",
    "180290": "Thank Konstantin for answering my question! It is clear and easy to understand. Thank you very much!"
  },
  "source": "meta"
}