but since they were in the bottom 50% of your public LB submissions and often not your best CV submissions, you didn't submit any of them. Instead you submitted your 1st and 4th best models based on the public LB, and…… 4th ranks higher than 1st on the private LB, but….was only your 11th best private LB model overall. I wasn't surprised at my 4th best public LB model leap frogging the first, nor was I surprised at scores significantly different on the LB, but, tossing away a bronze is going to hurt for a bit.
\nAll that being said, moving up 139 places on the Private LB and being in the top 15% on my first try is nothing to sneeze at and I think I'm easily twice the Data Scientist I was before I started this competition.
\nStill… I may need a moment.
", "messages": [ { "id": "1912779", "postDate": "08/25/2022 01:01:55", "content": "but since they were in the bottom 50% of your public LB submissions and often not your best CV submissions, you didn't submit any of them. Instead you submitted your 1st and 4th best models based on the public LB, and…… 4th ranks higher than 1st on the private LB, but….was only your 11th best private LB model overall. I wasn't surprised at my 4th best public LB model leap frogging the first, nor was I surprised at scores significantly different on the LB, but, tossing away a bronze is going to hurt for a bit.
\nAll that being said, moving up 139 places on the Private LB and being in the top 15% on my first try is nothing to sneeze at and I think I'm easily twice the Data Scientist I was before I started this competition.
\nStill… I may need a moment.
", "rawMarkdown": "but since they were in the bottom 50% of your public LB submissions and often not your best CV submissions, you didn't submit any of them. Instead you submitted your 1st and 4th best models based on the public LB, and...... 4th ranks higher than 1st on the private LB, but....was only your 11th best private LB model overall. I wasn't surprised at my 4th best public LB model leap frogging the first, nor was I surprised at scores significantly different on the LB, but, tossing away a bronze is going to hurt for a bit. \n\nAll that being said, moving up 139 places on the Private LB and being in the top 15% on my first try is nothing to sneeze at and I think I'm easily twice the Data Scientist I was before I started this competition. \n\n Still... I may need a moment.", "votes": null }, { "id": "1912851", "postDate": "08/25/2022 02:09:27", "content": "You are in good company. I feel confident in saying that most competitors didn't select their absolute best models. This was a weird metric and it was difficult to get a handle on it. I ensembled a group of models that had smaller CV-LB difference, instead of a group that had CV-LB difference >0.001. The latter group ended up being better, albeit by a tiny margin (0.00003).
", "rawMarkdown": "You are in good company. I feel confident in saying that most competitors didn't select their absolute best models. This was a weird metric and it was difficult to get a handle on it. I ensembled a group of models that had smaller CV-LB difference, instead of a group that had CV-LB difference >0.001. The latter group ended up being better, albeit by a tiny margin (0.00003).", "votes": null }, { "id": "1912929", "postDate": "08/25/2022 03:49:37", "content": "Weird metric and difficult to create quality criteria for the best models to submit, because if I had chosen any of higher scoring models it would've been by chance, funny enough two of top five were just single models, no ensembles, nothing special, in fact, they were lower scoring single models that I consider leaving out of the ensembles. Go figure.
", "rawMarkdown": "Weird metric and difficult to create quality criteria for the best models to submit, because if I had chosen any of higher scoring models it would've been by chance, funny enough two of top five were just single models, no ensembles, nothing special, in fact, they were lower scoring single models that I consider leaving out of the ensembles. Go figure.", "votes": null } ], "comments": [ { "id": 1912851, "author_name": "tilii7", "author_url": "", "post_date": "08/25/2022 02:09:27", "content": "You are in good company. I feel confident in saying that most competitors didn't select their absolute best models. This was a weird metric and it was difficult to get a handle on it. I ensembled a group of models that had smaller CV-LB difference, instead of a group that had CV-LB difference >0.001. The latter group ended up being better, albeit by a tiny margin (0.00003).
", "votes": null, "replies": [ { "id": 1912929, "author_name": "markhamlee", "author_url": "", "post_date": "08/25/2022 03:49:37", "content": "Weird metric and difficult to create quality criteria for the best models to submit, because if I had chosen any of higher scoring models it would've been by chance, funny enough two of top five were just single models, no ensembles, nothing special, in fact, they were lower scoring single models that I consider leaving out of the ensembles. Go figure.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1912779": "but since they were in the bottom 50% of your public LB submissions and often not your best CV submissions, you didn't submit any of them. Instead you submitted your 1st and 4th best models based on the public LB, and...... 4th ranks higher than 1st on the private LB, but....was only your 11th best private LB model overall. I wasn't surprised at my 4th best public LB model leap frogging the first, nor was I surprised at scores significantly different on the LB, but, tossing away a bronze is going to hurt for a bit. \n\nAll that being said, moving up 139 places on the Private LB and being in the top 15% on my first try is nothing to sneeze at and I think I'm easily twice the Data Scientist I was before I started this competition. \n\n Still... I may need a moment.", "1912851": "You are in good company. I feel confident in saying that most competitors didn't select their absolute best models. This was a weird metric and it was difficult to get a handle on it. I ensembled a group of models that had smaller CV-LB difference, instead of a group that had CV-LB difference >0.001. The latter group ended up being better, albeit by a tiny margin (0.00003).", "1912929": "Weird metric and difficult to create quality criteria for the best models to submit, because if I had chosen any of higher scoring models it would've been by chance, funny enough two of top five were just single models, no ensembles, nothing special, in fact, they were lower scoring single models that I consider leaving out of the ensembles. Go figure." }, "source": "meta" }