I remember D&L got 0.04015 score and lead the board several hours before the end of competition. The final check moved them down by 2 positions but leave Owen with the same score and give him a win.
\n\nThe obvious difference between them is the number of submissions (131 for D&L and 20 for Owen). Doesn't that mean that you are at risk of over-fitting when making many submissions and relying on it? What is the optimal rate of submission?
\n\nI made only few submissions but my final score is also 20 point less then before. Has anybody improved on final check?
", "messages": [ { "id": "87382", "postDate": "07/29/2015 03:53:04", "content": "I remember D&L got 0.04015 score and lead the board several hours before the end of competition. The final check moved them down by 2 positions but leave Owen with the same score and give him a win.
\n\nThe obvious difference between them is the number of submissions (131 for D&L and 20 for Owen). Doesn't that mean that you are at risk of over-fitting when making many submissions and relying on it? What is the optimal rate of submission?
\n\nI made only few submissions but my final score is also 20 point less then before. Has anybody improved on final check?
", "rawMarkdown": "I remember D&L got 0.04015 score and lead the board several hours before the end of competition. The final check moved them down by 2 positions but leave Owen with the same score and give him a win.\r\n\r\nThe obvious difference between them is the number of submissions (131 for D&L and 20 for Owen). Doesn't that mean that you are at risk of over-fitting when making many submissions and relying on it? What is the optimal rate of submission?\r\n\r\nI made only few submissions but my final score is also 20 point less then before. Has anybody improved on final check?", "votes": null }, { "id": "87400", "postDate": "07/29/2015 10:28:02", "content": "Relying on public LB is bad, but in this case we didn't overfit. Owen beat us on both public and private datasets fair and square, and we losing one position to gzs_Iceberg was a stroke of luck for him and bad luck for us.\nWe did check things on validation, and our best score on the public lb was our best score on the private one. Increments on public LB reflected increments on the private one. That kind of small shuffle was expected for scores that were so close. Happens in every competition.
", "rawMarkdown": "Relying on public LB is bad, but in this case we didn't overfit. Owen beat us on both public and private datasets fair and square, and we losing one position to gzs_Iceberg was a stroke of luck for him and bad luck for us.\r\nWe did check things on validation, and our best score on the public lb was our best score on the private one. Increments on public LB reflected increments on the private one. That kind of small shuffle was expected for scores that were so close. Happens in every competition.", "votes": null } ], "comments": [ { "id": 87400, "author_name": "leustagos", "author_url": "", "post_date": "07/29/2015 10:28:02", "content": "Relying on public LB is bad, but in this case we didn't overfit. Owen beat us on both public and private datasets fair and square, and we losing one position to gzs_Iceberg was a stroke of luck for him and bad luck for us.\nWe did check things on validation, and our best score on the public lb was our best score on the private one. Increments on public LB reflected increments on the private one. That kind of small shuffle was expected for scores that were so close. Happens in every competition.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "87382": "I remember D&L got 0.04015 score and lead the board several hours before the end of competition. The final check moved them down by 2 positions but leave Owen with the same score and give him a win.\r\n\r\nThe obvious difference between them is the number of submissions (131 for D&L and 20 for Owen). Doesn't that mean that you are at risk of over-fitting when making many submissions and relying on it? What is the optimal rate of submission?\r\n\r\nI made only few submissions but my final score is also 20 point less then before. Has anybody improved on final check?", "87400": "Relying on public LB is bad, but in this case we didn't overfit. Owen beat us on both public and private datasets fair and square, and we losing one position to gzs_Iceberg was a stroke of luck for him and bad luck for us.\r\nWe did check things on validation, and our best score on the public lb was our best score on the private one. Increments on public LB reflected increments on the private one. That kind of small shuffle was expected for scores that were so close. Happens in every competition." }, "source": "meta" }