{
  "id": 107913,
  "title": "Huge Huge Huge Shake-up!",
  "url": "/competitions/aptos2019-blindness-detection/discussion/107913",
  "author_name": "",
  "post_date": "2019-09-08T00:02:35.187223200Z",
  "votes": 3,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Wow I didn't expect the private LB to be so inconsistent with the public LB. I think we all had wrong performance metric :v</p>",
  "messages": [
    {
      "id": "620744",
      "postDate": "09/08/2019 00:02:35",
      "content": "<p>Wow I didn't expect the private LB to be so inconsistent with the public LB. I think we all had wrong performance metric :v</p>",
      "rawMarkdown": "Wow I didn't expect the private LB to be so inconsistent with the public LB. I think we all had wrong performance metric :v",
      "votes": null
    },
    {
      "id": "620751",
      "postDate": "09/08/2019 00:08:34",
      "content": "<p>I am very surprised that the top private scores are so much higher that the public scores. The public topped out at .86 but the private are .93. </p>",
      "rawMarkdown": "I am very surprised that the top private scores are so much higher that the public scores. The public topped out at .86 but the private are .93.",
      "votes": null
    },
    {
      "id": "620760",
      "postDate": "09/08/2019 00:17:59",
      "content": "<p>My sample size is quite small, 13 subs in total and only 5 I recorded local cv scores. But my local cv scores are within .005 to their private board scores.</p>",
      "rawMarkdown": "My sample size is quite small, 13 subs in total and only 5 I recorded local cv scores. But my local cv scores are within .005 to their private board scores.",
      "votes": null
    },
    {
      "id": "620769",
      "postDate": "09/08/2019 00:25:52",
      "content": "<p>Surprisingly I saw one of my past submission with public score of 76.7 but somehow managed to achieve 92.1 on private score (which is pretty damn close to my best model). </p>",
      "rawMarkdown": "Surprisingly I saw one of my past submission with public score of 76.7 but somehow managed to achieve 92.1 on private score (which is pretty damn close to my best model).",
      "votes": null
    },
    {
      "id": "620781",
      "postDate": "09/08/2019 00:50:34",
      "content": "<p>Much larger data, completely different target dists</p>",
      "rawMarkdown": "Much larger data, completely different target dists",
      "votes": null
    },
    {
      "id": "621273",
      "postDate": "09/08/2019 11:51:06",
      "content": "<p>I didn't believe in so huge shakeup. Private dataset is so different from public dataset. Really surprised!</p>",
      "rawMarkdown": "I didn't believe in so huge shakeup. Private dataset is so different from public dataset. Really surprised!",
      "votes": null
    },
    {
      "id": "621573",
      "postDate": "09/08/2019 17:13:25",
      "content": "<p>Wouldn't that result in lower scores though ? Unless a lot of people had models that generalized really well </p>",
      "rawMarkdown": "Wouldn't that result in lower scores though ? Unless a lot of people had models that generalized really well",
      "votes": null
    },
    {
      "id": "621582",
      "postDate": "09/08/2019 17:15:57",
      "content": "<p>My best public scoring model is not my best private scoring model. Happily, I didn't select it.  The model I put the most work into was my best private scoring model, which I take some happiness in. Public 73.3 Private 88.9</p>",
      "rawMarkdown": "My best public scoring model is not my best private scoring model. Happily, I didn't select it.  The model I put the most work into was my best private scoring model, which I take some happiness in. Public 73.3 Private 88.9",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 620751,
      "author_name": "chrisfs",
      "author_url": "",
      "post_date": "09/08/2019 00:08:34",
      "content": "<p>I am very surprised that the top private scores are so much higher that the public scores. The public topped out at .86 but the private are .93. </p>",
      "votes": null,
      "replies": [
        {
          "id": 620781,
          "author_name": "philippsinger",
          "author_url": "",
          "post_date": "09/08/2019 00:50:34",
          "content": "<p>Much larger data, completely different target dists</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 621573,
          "author_name": "chrisfs",
          "author_url": "",
          "post_date": "09/08/2019 17:13:25",
          "content": "<p>Wouldn't that result in lower scores though ? Unless a lot of people had models that generalized really well </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 620760,
      "author_name": "ryanzhang",
      "author_url": "",
      "post_date": "09/08/2019 00:17:59",
      "content": "<p>My sample size is quite small, 13 subs in total and only 5 I recorded local cv scores. But my local cv scores are within .005 to their private board scores.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 620769,
      "author_name": "quandapro",
      "author_url": "",
      "post_date": "09/08/2019 00:25:52",
      "content": "<p>Surprisingly I saw one of my past submission with public score of 76.7 but somehow managed to achieve 92.1 on private score (which is pretty damn close to my best model). </p>",
      "votes": null,
      "replies": [
        {
          "id": 621582,
          "author_name": "chrisfs",
          "author_url": "",
          "post_date": "09/08/2019 17:15:57",
          "content": "<p>My best public scoring model is not my best private scoring model. Happily, I didn't select it.  The model I put the most work into was my best private scoring model, which I take some happiness in. Public 73.3 Private 88.9</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 621273,
      "author_name": "demonplus",
      "author_url": "",
      "post_date": "09/08/2019 11:51:06",
      "content": "<p>I didn't believe in so huge shakeup. Private dataset is so different from public dataset. Really surprised!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "620744": "Wow I didn't expect the private LB to be so inconsistent with the public LB. I think we all had wrong performance metric :v",
    "620751": "I am very surprised that the top private scores are so much higher that the public scores. The public topped out at .86 but the private are .93.",
    "620760": "My sample size is quite small, 13 subs in total and only 5 I recorded local cv scores. But my local cv scores are within .005 to their private board scores.",
    "620769": "Surprisingly I saw one of my past submission with public score of 76.7 but somehow managed to achieve 92.1 on private score (which is pretty damn close to my best model).",
    "620781": "Much larger data, completely different target dists",
    "621273": "I didn't believe in so huge shakeup. Private dataset is so different from public dataset. Really surprised!",
    "621573": "Wouldn't that result in lower scores though ? Unless a lot of people had models that generalized really well",
    "621582": "My best public scoring model is not my best private scoring model. Happily, I didn't select it.  The model I put the most work into was my best private scoring model, which I take some happiness in. Public 73.3 Private 88.9"
  },
  "source": "meta"
}