Any feelings about what will occur when the private LB will be released ?
\nIf I were to guess, I'd say that we may not face any important changes when going from public to private, since everything looks rather well correlated between CV and LB scores.
\nAny thoughts about it ?
", "messages": [ { "id": "1516521", "postDate": "09/18/2021 11:56:27", "content": "Any feelings about what will occur when the private LB will be released ?
\nIf I were to guess, I'd say that we may not face any important changes when going from public to private, since everything looks rather well correlated between CV and LB scores.
\nAny thoughts about it ?
", "rawMarkdown": "Any feelings about what will occur when the private LB will be released ?\n\nIf I were to guess, I'd say that we may not face any important changes when going from public to private, since everything looks rather well correlated between CV and LB scores.\n\nAny thoughts about it ?", "votes": null }, { "id": "1516524", "postDate": "09/18/2021 12:07:14", "content": "No big shakeups, but even small ones would wreck chaos on a tightly packed silver/bronze leaderboard.
", "rawMarkdown": "No big shakeups, but even small ones would wreck chaos on a tightly packed silver/bronze leaderboard.", "votes": null }, { "id": "1516640", "postDate": "09/18/2021 14:54:00", "content": "I tested some ensembles in the public LB. If assuming most people around the bronze range are using the random'ish ensemble in the public notebooks (e.g., my current best public LB). I expect relatively big shakeups in the bronze range.
\nHowever, the top will have almost no shake-ups as I am having super stable CV-LB correlation for single models.
", "rawMarkdown": "I tested some ensembles in the public LB. If assuming most people around the bronze range are using the random'ish ensemble in the public notebooks (e.g., my current best public LB). I expect relatively big shakeups in the bronze range.\n\nHowever, the top will have almost no shake-ups as I am having super stable CV-LB correlation for single models.", "votes": null }, { "id": "1516665", "postDate": "09/18/2021 15:30:42", "content": "I guess top 20 or more in the LB will stay there. Bronze zone seems very unstable so a lot of shakeup is expected!
", "rawMarkdown": "I guess top 20 or more in the LB will stay there. Bronze zone seems very unstable so a lot of shakeup is expected!", "votes": null }, { "id": "1517290", "postDate": "09/19/2021 12:16:10", "content": "The LB seems to be very tightly packed, which suggests than even a small shake-up in terms of scores may have a bigger effect on placings and medals. Most people will have used ensembling, which often involves chasing the best public LB score, at least to some extent. However, the public LB test set is fairly large, ~38000 IDs [edited to correct this number], which suggests less shake-up.
", "rawMarkdown": "The LB seems to be very tightly packed, which suggests than even a small shake-up in terms of scores may have a bigger effect on placings and medals. Most people will have used ensembling, which often involves chasing the best public LB score, at least to some extent. However, the public LB test set is fairly large, ~38000 IDs [edited to correct this number], which suggests less shake-up.", "votes": null }, { "id": "1559724", "postDate": "10/27/2021 07:09:36", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
No big shakeups, but even small ones would wreck chaos on a tightly packed silver/bronze leaderboard.
", "votes": null, "replies": [] }, { "id": 1516640, "author_name": "scaomath", "author_url": "", "post_date": "09/18/2021 14:54:00", "content": "I tested some ensembles in the public LB. If assuming most people around the bronze range are using the random'ish ensemble in the public notebooks (e.g., my current best public LB). I expect relatively big shakeups in the bronze range.
\nHowever, the top will have almost no shake-ups as I am having super stable CV-LB correlation for single models.
", "votes": null, "replies": [] }, { "id": 1516665, "author_name": "yassinealouini", "author_url": "", "post_date": "09/18/2021 15:30:42", "content": "I guess top 20 or more in the LB will stay there. Bronze zone seems very unstable so a lot of shakeup is expected!
", "votes": null, "replies": [] }, { "id": 1517290, "author_name": "jbomitchell", "author_url": "", "post_date": "09/19/2021 12:16:10", "content": "The LB seems to be very tightly packed, which suggests than even a small shake-up in terms of scores may have a bigger effect on placings and medals. Most people will have used ensembling, which often involves chasing the best public LB score, at least to some extent. However, the public LB test set is fairly large, ~38000 IDs [edited to correct this number], which suggests less shake-up.
", "votes": null, "replies": [] }, { "id": 1559724, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 07:09:36", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris