{
  "id": 507737,
  "title": "Concerns Regarding the Fairness of Ranking",
  "url": "/competitions/cvpr-metafood-3d-food-reconstruction-challenge/discussion/507737",
  "author_name": "",
  "post_date": "2024-05-27T03:37:06.774991100Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hello, I am honored to have advanced to Phase Two of the competition, but I have some concerns about the ranking system. I noticed in the latest announcement in the discussion forum that the final rankings will be determined by averaging the rankings from two phases. However, the initial announcement at the beginning of the competition stated that the final rankings would combine scores across all 20 items from both phases. Given that Phase One only considered five items, I believe that averaging the rankings may not be a fair approach.</p>",
  "messages": [
    {
      "id": "2838446",
      "postDate": "05/27/2024 03:37:06",
      "content": "<p>Hello, I am honored to have advanced to Phase Two of the competition, but I have some concerns about the ranking system. I noticed in the latest announcement in the discussion forum that the final rankings will be determined by averaging the rankings from two phases. However, the initial announcement at the beginning of the competition stated that the final rankings would combine scores across all 20 items from both phases. Given that Phase One only considered five items, I believe that averaging the rankings may not be a fair approach.</p>",
      "rawMarkdown": "Hello, I am honored to have advanced to Phase Two of the competition, but I have some concerns about the ranking system. I noticed in the latest announcement in the discussion forum that the final rankings will be determined by averaging the rankings from two phases. However, the initial announcement at the beginning of the competition stated that the final rankings would combine scores across all 20 items from both phases. Given that Phase One only considered five items, I believe that averaging the rankings may not be a fair approach.",
      "votes": null
    },
    {
      "id": "2839465",
      "postDate": "05/27/2024 14:35:12",
      "content": "<p>Sorry for the confusion. We will recalculate the ranking of phase 1 based on 20 items.</p>\n<p>Here is how we will calculate.<br>\nFor phase 1 ranking calculation, we will combine your best public submission of 15 items and your best private submissions of 5 items and recalculate the phase 1 ranking based on all 20 items. </p>\n<p>For phase 2, you only have a one-time chance to submit all your 20 models. We will check to make sure the volume of all submitted models exist in all your past submissions based on the provided \"challenge_volume_calculation.py\" script. Your final evaluation will be based on your submitted 20 models. </p>\n<p>Thanks for bring this issue up.</p>",
      "rawMarkdown": "Sorry for the confusion. We will recalculate the ranking of phase 1 based on 20 items.\n\nHere is how we will calculate.\nFor phase 1 ranking calculation, we will combine your best public submission of 15 items and your best private submissions of 5 items and recalculate the phase 1 ranking based on all 20 items. \n\nFor phase 2, you only have a one-time chance to submit all your 20 models. We will check to make sure the volume of all submitted models exist in all your past submissions based on the provided \"challenge_volume_calculation.py\" script. Your final evaluation will be based on your submitted 20 models. \n\nThanks for bring this issue up.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2839465,
      "author_name": "metafoodcvpr",
      "author_url": "",
      "post_date": "05/27/2024 14:35:12",
      "content": "<p>Sorry for the confusion. We will recalculate the ranking of phase 1 based on 20 items.</p>\n<p>Here is how we will calculate.<br>\nFor phase 1 ranking calculation, we will combine your best public submission of 15 items and your best private submissions of 5 items and recalculate the phase 1 ranking based on all 20 items. </p>\n<p>For phase 2, you only have a one-time chance to submit all your 20 models. We will check to make sure the volume of all submitted models exist in all your past submissions based on the provided \"challenge_volume_calculation.py\" script. Your final evaluation will be based on your submitted 20 models. </p>\n<p>Thanks for bring this issue up.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2838446": "Hello, I am honored to have advanced to Phase Two of the competition, but I have some concerns about the ranking system. I noticed in the latest announcement in the discussion forum that the final rankings will be determined by averaging the rankings from two phases. However, the initial announcement at the beginning of the competition stated that the final rankings would combine scores across all 20 items from both phases. Given that Phase One only considered five items, I believe that averaging the rankings may not be a fair approach.",
    "2839465": "Sorry for the confusion. We will recalculate the ranking of phase 1 based on 20 items.\n\nHere is how we will calculate.\nFor phase 1 ranking calculation, we will combine your best public submission of 15 items and your best private submissions of 5 items and recalculate the phase 1 ranking based on all 20 items. \n\nFor phase 2, you only have a one-time chance to submit all your 20 models. We will check to make sure the volume of all submitted models exist in all your past submissions based on the provided \"challenge_volume_calculation.py\" script. Your final evaluation will be based on your submitted 20 models. \n\nThanks for bring this issue up."
  },
  "source": "meta"
}