{
  "id": 138492,
  "title": "Possible discrepancy between test scores and train scores?",
  "url": "/competitions/herbarium-2020-fgvc7/discussion/138492",
  "author_name": "",
  "post_date": "2020-03-25T06:32:35.611002800Z",
  "votes": 6,
  "comment_count": 3,
  "views": 0,
  "content": "<p><a href=\"/kcgoogle\">@kcgoogle</a> \nWhen i am looking at the test F1 scores, they seem to be very very low, F1_macro ~1e-4\nThe gap between my train-set F1 score (~0.6) and test-set F1 score (6e-5) is very big.</p>\n\n<p>Are you sure the test scores are calculated correctly, and that the test-set ids have the same order as the train-set ids? </p>\n\n<p>Thanks,\nTal</p>",
  "messages": [
    {
      "id": "785546",
      "postDate": "03/25/2020 06:32:35",
      "content": "<p><a href=\"/kcgoogle\">@kcgoogle</a> \nWhen i am looking at the test F1 scores, they seem to be very very low, F1_macro ~1e-4\nThe gap between my train-set F1 score (~0.6) and test-set F1 score (6e-5) is very big.</p>\n\n<p>Are you sure the test scores are calculated correctly, and that the test-set ids have the same order as the train-set ids? </p>\n\n<p>Thanks,\nTal</p>",
      "rawMarkdown": "kcgoogle \nWhen i am looking at the test F1 scores, they seem to be very very low, F1_macro ~1e-4\nThe gap between my train-set F1 score (~0.6) and test-set F1 score (6e-5) is very big.\n\n\nAre you sure the test scores are calculated correctly, and that the test-set ids have the same order as the train-set ids? \n\nThanks,\nTal",
      "votes": null
    },
    {
      "id": "786807",
      "postDate": "03/26/2020 08:13:19",
      "content": "<p>Hi Tal,</p>\n\n<p>We're checking this currently and will be able to give an update within the next few days. Thanks for your feedback and  patience!</p>\n\n<p>Regards,\nKiat</p>",
      "rawMarkdown": "Hi Tal,\n\nWe're checking this currently and will be able to give an update within the next few days. Thanks for your feedback and  patience!\n\nRegards,\nKiat",
      "votes": null
    },
    {
      "id": "788554",
      "postDate": "03/27/2020 19:07:43",
      "content": "<p>We've discovered an issue with the solution mapping file, and will be uploading a new solution mapping file shortly. Thanks for your patience!</p>",
      "rawMarkdown": "We've discovered an issue with the solution mapping file, and will be uploading a new solution mapping file shortly. Thanks for your patience!",
      "votes": null
    },
    {
      "id": "788758",
      "postDate": "03/28/2020 01:27:25",
      "content": "<p>The new solution mapping file has been uploaded! Submissions have been rescored and the leaderboard updated accordingly. Thanks Tal for raising this, and sorry for the trouble!</p>",
      "rawMarkdown": "The new solution mapping file has been uploaded! Submissions have been rescored and the leaderboard updated accordingly. Thanks Tal for raising this, and sorry for the trouble!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 786807,
      "author_name": "kcgoogle",
      "author_url": "",
      "post_date": "03/26/2020 08:13:19",
      "content": "<p>Hi Tal,</p>\n\n<p>We're checking this currently and will be able to give an update within the next few days. Thanks for your feedback and  patience!</p>\n\n<p>Regards,\nKiat</p>",
      "votes": null,
      "replies": [
        {
          "id": 788554,
          "author_name": "kcgoogle",
          "author_url": "",
          "post_date": "03/27/2020 19:07:43",
          "content": "<p>We've discovered an issue with the solution mapping file, and will be uploading a new solution mapping file shortly. Thanks for your patience!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 788758,
          "author_name": "kcgoogle",
          "author_url": "",
          "post_date": "03/28/2020 01:27:25",
          "content": "<p>The new solution mapping file has been uploaded! Submissions have been rescored and the leaderboard updated accordingly. Thanks Tal for raising this, and sorry for the trouble!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "785546": "kcgoogle \nWhen i am looking at the test F1 scores, they seem to be very very low, F1_macro ~1e-4\nThe gap between my train-set F1 score (~0.6) and test-set F1 score (6e-5) is very big.\n\n\nAre you sure the test scores are calculated correctly, and that the test-set ids have the same order as the train-set ids? \n\nThanks,\nTal",
    "786807": "Hi Tal,\n\nWe're checking this currently and will be able to give an update within the next few days. Thanks for your feedback and  patience!\n\nRegards,\nKiat",
    "788554": "We've discovered an issue with the solution mapping file, and will be uploading a new solution mapping file shortly. Thanks for your patience!",
    "788758": "The new solution mapping file has been uploaded! Submissions have been rescored and the leaderboard updated accordingly. Thanks Tal for raising this, and sorry for the trouble!"
  },
  "source": "meta"
}