{
  "id": 147153,
  "title": "Understanding the LB",
  "url": "/competitions/alaska2-image-steganalysis/discussion/147153",
  "author_name": "",
  "post_date": "2020-04-29T17:07:34.195998700Z",
  "votes": 6,
  "comment_count": 1,
  "views": 0,
  "content": "<ul>\n<li>Test size: 5000 images.</li>\n<li>The public LB is calculated with approximately 20% of the test data = 1000 images.</li>\n<li>A <a href=\"https://www.kaggle.com/outrunner/random-seed-1\">random submission</a> (literally) by <a href=\"/outrunner\">@outrunner</a> scores <code>0.885</code></li>\n<li><a href=\"https://www.kaggle.com/xhlulu/alaska2-efficientnet-on-tpus\">Basic EfficientNet B3</a> trained on a subset by <a href=\"/xhlulu\">@xhlulu</a> scores <code>0.990</code> . Loss and accuracy aren't really good.</li>\n</ul>\n\n<p>LB 2 days after it started: <code>0.995</code>\n<a href=\"/remicogranne\">@remicogranne</a> is this \"normal\"? are you considering a metric change (maybe F1)? or as proposed at <a href=\"https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/146989\">Discrepancy between accuracy and the competition metric</a> and <a href=\"https://www.kaggle.com/maxjeblick/alaska2-efficientnet-on-tpus-competition-metric\">Alaska2: EfficientNet on TPUs competition Metric</a> by <a href=\"/maxjeblick\">@maxjeblick</a> </p>\n\n<ul>\n<li>this is not a code competition, considering such LB, the metric and easy reverse engineering...</li>\n</ul>",
  "messages": [
    {
      "id": "826486",
      "postDate": "04/29/2020 17:07:34",
      "content": "<ul>\n<li>Test size: 5000 images.</li>\n<li>The public LB is calculated with approximately 20% of the test data = 1000 images.</li>\n<li>A <a href=\"https://www.kaggle.com/outrunner/random-seed-1\">random submission</a> (literally) by <a href=\"/outrunner\">@outrunner</a> scores <code>0.885</code></li>\n<li><a href=\"https://www.kaggle.com/xhlulu/alaska2-efficientnet-on-tpus\">Basic EfficientNet B3</a> trained on a subset by <a href=\"/xhlulu\">@xhlulu</a> scores <code>0.990</code> . Loss and accuracy aren't really good.</li>\n</ul>\n\n<p>LB 2 days after it started: <code>0.995</code>\n<a href=\"/remicogranne\">@remicogranne</a> is this \"normal\"? are you considering a metric change (maybe F1)? or as proposed at <a href=\"https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/146989\">Discrepancy between accuracy and the competition metric</a> and <a href=\"https://www.kaggle.com/maxjeblick/alaska2-efficientnet-on-tpus-competition-metric\">Alaska2: EfficientNet on TPUs competition Metric</a> by <a href=\"/maxjeblick\">@maxjeblick</a> </p>\n\n<ul>\n<li>this is not a code competition, considering such LB, the metric and easy reverse engineering...</li>\n</ul>",
      "rawMarkdown": "Test size: 5000 images.\n- The public LB is calculated with approximately 20% of the test data = 1000 images.\n-  A [random submission](https://www.kaggle.com/outrunner/random-seed-1) (literally) by @outrunner scores `0.885`\n- [Basic EfficientNet B3](https://www.kaggle.com/xhlulu/alaska2-efficientnet-on-tpus) trained on a subset by @xhlulu scores `0.990` . Loss and accuracy aren't really good.\n\nLB 2 days after it started: `0.995`\n@remicogranne is this \"normal\"? are you considering a metric change (maybe F1)? or as proposed at [Discrepancy between accuracy and the competition metric](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/146989) and [Alaska2: EfficientNet on TPUs competition Metric](https://www.kaggle.com/maxjeblick/alaska2-efficientnet-on-tpus-competition-metric) by @maxjeblick \n\n+ this is not a code competition, considering such LB, the metric and easy reverse engineering...",
      "votes": null
    },
    {
      "id": "826699",
      "postDate": "04/29/2020 19:42:28",
      "content": "<p>You are absolutely right.\nWhile it is very important for us to push the community to focus on low-false alarm, which has simply almost never been studied ...  the evaluation brings too much confusion.\nWe will change the evaluation metric\nI apologize for the confusion</p>",
      "rawMarkdown": "You are absolutely right.\nWhile it is very important for us to push the community to focus on low-false alarm, which has simply almost never been studied ...  the evaluation brings too much confusion.\nWe will change the evaluation metric\nI apologize for the confusion",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 826699,
      "author_name": "remicogranne",
      "author_url": "",
      "post_date": "04/29/2020 19:42:28",
      "content": "<p>You are absolutely right.\nWhile it is very important for us to push the community to focus on low-false alarm, which has simply almost never been studied ...  the evaluation brings too much confusion.\nWe will change the evaluation metric\nI apologize for the confusion</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "826486": "Test size: 5000 images.\n- The public LB is calculated with approximately 20% of the test data = 1000 images.\n-  A [random submission](https://www.kaggle.com/outrunner/random-seed-1) (literally) by @outrunner scores `0.885`\n- [Basic EfficientNet B3](https://www.kaggle.com/xhlulu/alaska2-efficientnet-on-tpus) trained on a subset by @xhlulu scores `0.990` . Loss and accuracy aren't really good.\n\nLB 2 days after it started: `0.995`\n@remicogranne is this \"normal\"? are you considering a metric change (maybe F1)? or as proposed at [Discrepancy between accuracy and the competition metric](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/146989) and [Alaska2: EfficientNet on TPUs competition Metric](https://www.kaggle.com/maxjeblick/alaska2-efficientnet-on-tpus-competition-metric) by @maxjeblick \n\n+ this is not a code competition, considering such LB, the metric and easy reverse engineering...",
    "826699": "You are absolutely right.\nWhile it is very important for us to push the community to focus on low-false alarm, which has simply almost never been studied ...  the evaluation brings too much confusion.\nWe will change the evaluation metric\nI apologize for the confusion"
  },
  "source": "meta"
}