{
  "id": 674651,
  "title": "Clarification on Leaderboard MAE values (>1.0) and scoring behavior",
  "url": "/competitions/automatic-lens-correction/discussion/674651",
  "author_name": "",
  "post_date": "2026-02-21T11:31:01.632624600Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>I had a question about interpreting the current public leaderboard scores.</p>\n<p>According to the evaluation description, submissions are evaluated using MAE between per-image scores (0.0–1.0) and the perfect score (1.0). Mathematically, this suggests that valid MAE values should lie in the range [0.0, 1.0], regardless of the number of test images.</p>\n<p>However, the current leaderboard shows some entries with MAE values significantly greater than 1.0 (e.g., 6+, 20+).</p>\n<p>Could the hosts or participants clarify:</p>\n<p>Whether MAE &gt; 1.0 indicates a submission or formatting failure (e.g., missing images, naming mismatch, resolution issues)?</p>\n<p>Or whether the external scoring service applies additional penalties that are then reflected directly in the Kaggle MAE?</p>\n<p>What the recommended way is to sanity-check a submission to ensure it’s considered valid (i.e., stays within the expected MAE range)?</p>\n<p>This clarification would help participants better interpret the leaderboard and debug their submission pipelines.</p>\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "3408822",
      "postDate": "02/21/2026 11:31:01",
      "content": "<p>Hi everyone,</p>\n<p>I had a question about interpreting the current public leaderboard scores.</p>\n<p>According to the evaluation description, submissions are evaluated using MAE between per-image scores (0.0–1.0) and the perfect score (1.0). Mathematically, this suggests that valid MAE values should lie in the range [0.0, 1.0], regardless of the number of test images.</p>\n<p>However, the current leaderboard shows some entries with MAE values significantly greater than 1.0 (e.g., 6+, 20+).</p>\n<p>Could the hosts or participants clarify:</p>\n<p>Whether MAE &gt; 1.0 indicates a submission or formatting failure (e.g., missing images, naming mismatch, resolution issues)?</p>\n<p>Or whether the external scoring service applies additional penalties that are then reflected directly in the Kaggle MAE?</p>\n<p>What the recommended way is to sanity-check a submission to ensure it’s considered valid (i.e., stays within the expected MAE range)?</p>\n<p>This clarification would help participants better interpret the leaderboard and debug their submission pipelines.</p>\n<p>Thanks!</p>",
      "rawMarkdown": "Hi everyone,\n\nI had a question about interpreting the current public leaderboard scores.\n\nAccording to the evaluation description, submissions are evaluated using MAE between per-image scores (0.0–1.0) and the perfect score (1.0). Mathematically, this suggests that valid MAE values should lie in the range [0.0, 1.0], regardless of the number of test images.\n\nHowever, the current leaderboard shows some entries with MAE values significantly greater than 1.0 (e.g., 6+, 20+).\n\nCould the hosts or participants clarify:\n\nWhether MAE > 1.0 indicates a submission or formatting failure (e.g., missing images, naming mismatch, resolution issues)?\n\nOr whether the external scoring service applies additional penalties that are then reflected directly in the Kaggle MAE?\n\nWhat the recommended way is to sanity-check a submission to ensure it’s considered valid (i.e., stays within the expected MAE range)?\n\nThis clarification would help participants better interpret the leaderboard and debug their submission pipelines.\n\nThanks!",
      "votes": null
    },
    {
      "id": "3409026",
      "postDate": "02/21/2026 21:34:35",
      "content": "<p>My interpretation is that each submission has a MAE from [0, 1] based on the rubric. The MAE is used to determine the score from [0, 100] (more better since baseline scores 0). The leaderboard is based on the mean of these scores I believe</p>",
      "rawMarkdown": "My interpretation is that each submission has a MAE from [0, 1] based on the rubric. The MAE is used to determine the score from [0, 100] (more better since baseline scores 0). The leaderboard is based on the mean of these scores I believe",
      "votes": null
    },
    {
      "id": "3409137",
      "postDate": "02/22/2026 07:52:15",
      "content": "<p>maximum of 100 is possible?</p>",
      "rawMarkdown": "maximum of 100 is possible?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3409026,
      "author_name": "lanc33llis",
      "author_url": "",
      "post_date": "02/21/2026 21:34:35",
      "content": "<p>My interpretation is that each submission has a MAE from [0, 1] based on the rubric. The MAE is used to determine the score from [0, 100] (more better since baseline scores 0). The leaderboard is based on the mean of these scores I believe</p>",
      "votes": null,
      "replies": [
        {
          "id": 3409137,
          "author_name": "kunalinfinite",
          "author_url": "",
          "post_date": "02/22/2026 07:52:15",
          "content": "<p>maximum of 100 is possible?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3408822": "Hi everyone,\n\nI had a question about interpreting the current public leaderboard scores.\n\nAccording to the evaluation description, submissions are evaluated using MAE between per-image scores (0.0–1.0) and the perfect score (1.0). Mathematically, this suggests that valid MAE values should lie in the range [0.0, 1.0], regardless of the number of test images.\n\nHowever, the current leaderboard shows some entries with MAE values significantly greater than 1.0 (e.g., 6+, 20+).\n\nCould the hosts or participants clarify:\n\nWhether MAE > 1.0 indicates a submission or formatting failure (e.g., missing images, naming mismatch, resolution issues)?\n\nOr whether the external scoring service applies additional penalties that are then reflected directly in the Kaggle MAE?\n\nWhat the recommended way is to sanity-check a submission to ensure it’s considered valid (i.e., stays within the expected MAE range)?\n\nThis clarification would help participants better interpret the leaderboard and debug their submission pipelines.\n\nThanks!",
    "3409026": "My interpretation is that each submission has a MAE from [0, 1] based on the rubric. The MAE is used to determine the score from [0, 100] (more better since baseline scores 0). The leaderboard is based on the mean of these scores I believe",
    "3409137": "maximum of 100 is possible?"
  },
  "source": "meta"
}