I submitted two kernels that happened to generate exactly the same result (see below, copied and pasted from two output submissions.csv side by side). But they received different LB scores (0.9423 v.s. 0.9576).\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F47373%2F2fe5b090862b4c9832bdb9ecdf2f726c%2FScreen%20Shot%202020-02-17%20at%208.46.02%20PM.png?generation=1581990382541414&alt=media\")
\nMy understanding is that LB score is calculated from submission.csv (that can be downloaded from the \"Output\" part of the Kernel). Has my understanding been wrong? If not, I don't understand how these two got different scores. Has anyone experienced this, or can shed some light on a possible cause? Thanks for the help.
I submitted two kernels that happened to generate exactly the same result (see below, copied and pasted from two output submissions.csv side by side). But they received different LB scores (0.9423 v.s. 0.9576).\n\nMy understanding is that LB score is calculated from submission.csv (that can be downloaded from the \"Output\" part of the Kernel). Has my understanding been wrong? If not, I don't understand how these two got different scores. Has anyone experienced this, or can shed some light on a possible cause? Thanks for the help.
I think it's more than just the submission.csv. I update several same submission.csv, but the results are different. Although it seems very strange to me too.
", "rawMarkdown": "I think it's more than just the submission.csv. I update several same submission.csv, but the results are different. Although it seems very strange to me too.", "votes": null }, { "id": "748817", "postDate": "02/18/2020 02:50:25", "content": "Please Look at \"Data Description\".\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1310979%2F66689a52df3553fdaa60c6f858ecbc4a%2F.PNG?generation=1581994118454306&alt=media\")
You are only viewing 36 rows (predictions). The actual public test submission is about 100,000 rows and the actual private test submission is about 100,000 rows.
", "rawMarkdown": "You are only viewing 36 rows (predictions). The actual public test submission is about 100,000 rows and the actual private test submission is about 100,000 rows.", "votes": null }, { "id": "748826", "postDate": "02/18/2020 03:08:09", "content": "Awkward mistake 😅.\nThanks very much for the replies.
", "rawMarkdown": "Awkward mistake 😅.\nThanks very much for the replies.", "votes": null }, { "id": "748893", "postDate": "02/18/2020 05:18:18", "content": "No worries. I was confused about the same thing myself in the beginning.
", "rawMarkdown": "No worries. I was confused about the same thing myself in the beginning.", "votes": null }, { "id": "749206", "postDate": "02/18/2020 12:42:29", "content": "@zhangyang If you are using the same model and getting different LB, do check if you have called eval(). I have made this mistake several times myself: some operations such as dropout and shake-shake behave stochastically.
", "rawMarkdown": "zhangyang If you are using the same model and getting different LB, do check if you have called eval(). I have made this mistake several times myself: some operations such as dropout and shake-shake behave stochastically.", "votes": null }, { "id": "749820", "postDate": "02/18/2020 23:21:35", "content": "Thanks for the suggestion!
", "rawMarkdown": "Thanks for the suggestion!", "votes": null } ], "comments": [ { "id": 748816, "author_name": "yuanlin08", "author_url": "", "post_date": "02/18/2020 02:50:03", "content": "I think it's more than just the submission.csv. I update several same submission.csv, but the results are different. Although it seems very strange to me too.
", "votes": null, "replies": [] }, { "id": 748817, "author_name": "kyosato", "author_url": "", "post_date": "02/18/2020 02:50:25", "content": "Please Look at \"Data Description\".\n
You are only viewing 36 rows (predictions). The actual public test submission is about 100,000 rows and the actual private test submission is about 100,000 rows.
", "votes": null, "replies": [] }, { "id": 748826, "author_name": "zhangyang", "author_url": "", "post_date": "02/18/2020 03:08:09", "content": "Awkward mistake 😅.\nThanks very much for the replies.
", "votes": null, "replies": [ { "id": 748893, "author_name": "cdeotte", "author_url": "", "post_date": "02/18/2020 05:18:18", "content": "No worries. I was confused about the same thing myself in the beginning.
", "votes": null, "replies": [] } ] }, { "id": 749206, "author_name": "roguekk007", "author_url": "", "post_date": "02/18/2020 12:42:29", "content": "@zhangyang If you are using the same model and getting different LB, do check if you have called eval(). I have made this mistake several times myself: some operations such as dropout and shake-shake behave stochastically.
", "votes": null, "replies": [ { "id": 749820, "author_name": "zhangyang", "author_url": "", "post_date": "02/18/2020 23:21:35", "content": "Thanks for the suggestion!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "748790": "I submitted two kernels that happened to generate exactly the same result (see below, copied and pasted from two output submissions.csv side by side). But they received different LB scores (0.9423 v.s. 0.9576).\n\nMy understanding is that LB score is calculated from submission.csv (that can be downloaded from the \"Output\" part of the Kernel). Has my understanding been wrong? If not, I don't understand how these two got different scores. Has anyone experienced this, or can shed some light on a possible cause? Thanks for the help.", "748816": "I think it's more than just the submission.csv. I update several same submission.csv, but the results are different. Although it seems very strange to me too.", "748817": "Please Look at \"Data Description\".\n", "748819": "You are only viewing 36 rows (predictions). The actual public test submission is about 100,000 rows and the actual private test submission is about 100,000 rows.", "748826": "Awkward mistake 😅.\nThanks very much for the replies.", "748893": "No worries. I was confused about the same thing myself in the beginning.", "749206": "zhangyang If you are using the same model and getting different LB, do check if you have called eval(). I have made this mistake several times myself: some operations such as dropout and shake-shake behave stochastically.", "749820": "Thanks for the suggestion!" }, "source": "meta" }