{
  "id": 77308,
  "title": "Private LB rank is quite different from Public LB rank",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/77308",
  "author_name": "",
  "post_date": "2019-01-11T11:10:29.199246400Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>F1 score gap between public and private make sense because of data leak. But big gap of LB rank is quite wired.\n<strong>Assume all of us use leak data</strong>, if public test set has similar data distribution with private test set, we should have similar LB rank.\nSo, maybe data distribution between public test set and private test set are quite different? If so, I guess we can do nothing about this.</p>",
  "messages": [
    {
      "id": "454283",
      "postDate": "01/11/2019 11:10:29",
      "content": "<p>F1 score gap between public and private make sense because of data leak. But big gap of LB rank is quite wired.\n<strong>Assume all of us use leak data</strong>, if public test set has similar data distribution with private test set, we should have similar LB rank.\nSo, maybe data distribution between public test set and private test set are quite different? If so, I guess we can do nothing about this.</p>",
      "rawMarkdown": "F1 score gap between public and private make sense because of data leak. But big gap of LB rank is quite wired.\n**Assume all of us use leak data**, if public test set has similar data distribution with private test set, we should have similar LB rank.\nSo, maybe data distribution between public test set and private test set are quite different? If so, I guess we can do nothing about this.",
      "votes": null
    },
    {
      "id": "454351",
      "postDate": "01/11/2019 13:42:12",
      "content": "<p>What you mean by big gap of LB rank? Do you mean shake ups? </p>\n\n<p>I personally see that public board and private board are pretty correlated at least for my model. </p>",
      "rawMarkdown": "What you mean by big gap of LB rank? Do you mean shake ups? \n\nI personally see that public board and private board are pretty correlated at least for my model.",
      "votes": null
    },
    {
      "id": "454371",
      "postDate": "01/11/2019 14:42:04",
      "content": "<p>We have the same gap between private and public score for ~top-20 of our models.</p>",
      "rawMarkdown": "We have the same gap between private and public score for ~top-20 of our models.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 454351,
      "author_name": "ryanzhang",
      "author_url": "",
      "post_date": "01/11/2019 13:42:12",
      "content": "<p>What you mean by big gap of LB rank? Do you mean shake ups? </p>\n\n<p>I personally see that public board and private board are pretty correlated at least for my model. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 454371,
      "author_name": "hokmund",
      "author_url": "",
      "post_date": "01/11/2019 14:42:04",
      "content": "<p>We have the same gap between private and public score for ~top-20 of our models.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "454283": "F1 score gap between public and private make sense because of data leak. But big gap of LB rank is quite wired.\n**Assume all of us use leak data**, if public test set has similar data distribution with private test set, we should have similar LB rank.\nSo, maybe data distribution between public test set and private test set are quite different? If so, I guess we can do nothing about this.",
    "454351": "What you mean by big gap of LB rank? Do you mean shake ups? \n\nI personally see that public board and private board are pretty correlated at least for my model.",
    "454371": "We have the same gap between private and public score for ~top-20 of our models."
  },
  "source": "meta"
}