{
  "id": 324580,
  "title": "Is there a good way to check out the details of the test?",
  "url": "/competitions/birdclef-2022/discussion/324580",
  "author_name": "",
  "post_date": "2022-05-12T10:21:03.534758500Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I can only see the score I submitted, but I can't see which cases affected my score.</p>",
  "messages": [
    {
      "id": "1785689",
      "postDate": "05/12/2022 10:21:03",
      "content": "<p>I can only see the score I submitted, but I can't see which cases affected my score.</p>",
      "rawMarkdown": "I can only see the score I submitted, but I can't see which cases affected my score.",
      "votes": null
    },
    {
      "id": "1785792",
      "postDate": "05/12/2022 12:23:38",
      "content": "<p>You can do the following:</p>\n<ul>\n<li>submit your submission, get your score</li>\n<li>submit the same submission but set all birds in [\"bird_a\", \"bird_b\", … ] to 0, get your score</li>\n</ul>\n<p>The difference between your scores correspond to the uplift given by [\"bird_a\", \"bird_b\", … ].</p>\n<p>Note that for one bird class, the maximum uplift you can expect is 0.5 / num_birds ~ 0.024.</p>\n<p>So in order to get the details for each class you'll need 21 submissions &gt; 4 days of submissions.</p>\n<p>If you look at batches of species you'll get the summed uplift for those classes and you won't know which one brings good or bad uplift.</p>",
      "rawMarkdown": "You can do the following:\n- submit your submission, get your score\n- submit the same submission but set all birds in [\"bird_a\", \"bird_b\", ... ] to 0, get your score\n\nThe difference between your scores correspond to the uplift given by [\"bird_a\", \"bird_b\", ... ].\n\nNote that for one bird class, the maximum uplift you can expect is 0.5 / num_birds ~ 0.024.\n\nSo in order to get the details for each class you'll need 21 submissions > 4 days of submissions.\n\nIf you look at batches of species you'll get the summed uplift for those classes and you won't know which one brings good or bad uplift.",
      "votes": null
    },
    {
      "id": "1786245",
      "postDate": "05/12/2022 17:58:50",
      "content": "<p>You can actually get twice the uplift, if you go from your model predictions to \"1 - your model prediction\" (potentially 100% -&gt; 0% accuracy). But given the low LB precision, it may be quite hard to learn much - depends on what you are after.</p>",
      "rawMarkdown": "You can actually get twice the uplift, if you go from your model predictions to \"1 - your model prediction\" (potentially 100% -> 0% accuracy). But given the low LB precision, it may be quite hard to learn much - depends on what you are after.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1785792,
      "author_name": "optimo",
      "author_url": "",
      "post_date": "05/12/2022 12:23:38",
      "content": "<p>You can do the following:</p>\n<ul>\n<li>submit your submission, get your score</li>\n<li>submit the same submission but set all birds in [\"bird_a\", \"bird_b\", … ] to 0, get your score</li>\n</ul>\n<p>The difference between your scores correspond to the uplift given by [\"bird_a\", \"bird_b\", … ].</p>\n<p>Note that for one bird class, the maximum uplift you can expect is 0.5 / num_birds ~ 0.024.</p>\n<p>So in order to get the details for each class you'll need 21 submissions &gt; 4 days of submissions.</p>\n<p>If you look at batches of species you'll get the summed uplift for those classes and you won't know which one brings good or bad uplift.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1786245,
          "author_name": "motloch",
          "author_url": "",
          "post_date": "05/12/2022 17:58:50",
          "content": "<p>You can actually get twice the uplift, if you go from your model predictions to \"1 - your model prediction\" (potentially 100% -&gt; 0% accuracy). But given the low LB precision, it may be quite hard to learn much - depends on what you are after.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1785689": "I can only see the score I submitted, but I can't see which cases affected my score.",
    "1785792": "You can do the following:\n- submit your submission, get your score\n- submit the same submission but set all birds in [\"bird_a\", \"bird_b\", ... ] to 0, get your score\n\nThe difference between your scores correspond to the uplift given by [\"bird_a\", \"bird_b\", ... ].\n\nNote that for one bird class, the maximum uplift you can expect is 0.5 / num_birds ~ 0.024.\n\nSo in order to get the details for each class you'll need 21 submissions > 4 days of submissions.\n\nIf you look at batches of species you'll get the summed uplift for those classes and you won't know which one brings good or bad uplift.",
    "1786245": "You can actually get twice the uplift, if you go from your model predictions to \"1 - your model prediction\" (potentially 100% -> 0% accuracy). But given the low LB precision, it may be quite hard to learn much - depends on what you are after."
  },
  "source": "meta"
}