I tried the method that has been discussed in the discussion.
\nhttps://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357
\nApparently, you can see the value of private score by following these steps.
The attached image is a screenshot taken with the above method.
", "messages": [ { "id": 1131159, "postDate": "2020-12-29T15:13:06.987Z", "content": "I tried the method that has been discussed in the discussion.
\nhttps://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357
\nApparently, you can see the value of private score by following these steps.
The attached image is a screenshot taken with the above method.
", "rawMarkdown": "I tried the method that has been discussed in the discussion.\n\nhttps://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357\nApparently, you can see the value of private score by following these steps.\n\n1. Go to Leaderboard\n1. open Devtool -> Network\n1. XHR only->select getUser request\n1. Preview -> result -> team, maybe you can see your public/private score and ranking(It seems to display the value of the currently selected submission.)\n\nThe attached image is a screenshot taken with the above method.", "votes": 15 }, { "id": 1131491, "postDate": "2020-12-29T18:54:47.360Z", "content": "It will be interesting how Kaggle will react to this major issue.
\nIt is hard to change the private entire datasets to all active competition. What can be done is to truncate it, remove a percentage of data but keeping the same labels distribution.
\nThis will not fix the leaking problem, some people already seen what is working and what not directly on the private leaderboard, but this action will at least change the private score for whomever seen it
When I changed selected submissions, the private leader board score was changed at the same time.
\nThis means we can check all the scores for private leader board.
\nSo I think that private dataset must be updated if possible.
Hi, please see this statement from our team regarding the leaderboard bug.
", "rawMarkdown": "Hi, please see [this statement](https://www.kaggle.com/private-leaderboard-bug-dec-2020) from our team regarding the leaderboard bug.", "votes": 1, "replies": [ { "id": 1132969, "postDate": "2020-12-30T20:35:22.887Z", "content": "Thanks for the statement release @juliaelliott. Will Kaggle staff take actions to alter the private datasets of the ongoing competitions ?
", "rawMarkdown": "Thanks for the statement release @juliaelliott. Will Kaggle staff take actions to alter the private datasets of the ongoing competitions ?" }, { "id": 1132973, "postDate": "2020-12-30T20:47:02.577Z", "content": "@vladvdv We are in contact with the hosts of currently live competitions to determine if and what any actions will be.
", "rawMarkdown": "@vladvdv We are in contact with the hosts of currently live competitions to determine if and what any actions will be." } ] }, { "id": 1131284, "postDate": "2020-12-29T16:31:27.453Z", "content": "The bug is fixed and trust your CV :)
", "rawMarkdown": "The bug is fixed and trust your CV :) ", "votes": 1 }, { "id": 1131273, "postDate": "2020-12-29T16:19:06.440Z", "content": "The bug is fixed now
\nPrivate LB is no longer accessible
Wow! After so many years and numerous competition shake ups, can't believe the private LB score was just there all the time.
", "rawMarkdown": "Wow! After so many years and numerous competition shake ups, can't believe the private LB score was just there all the time.", "votes": 2 }, { "id": 1132119, "postDate": "2020-12-30T07:14:12.153Z", "content": "LB 89.46 PB 89.28, which means the LB is reliable, so does the cv score. turst your local cv:D
", "rawMarkdown": "LB 89.46 PB 89.28, which means the LB is reliable, so does the cv score. turst your local cv:D", "votes": 2, "replies": [ { "id": 1132255, "postDate": "2020-12-30T09:10:15.793Z", "rawMarkdown": "", "votes": 3, "isDeleted": true }, { "id": 1132484, "postDate": "2020-12-30T12:43:51.683Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1132506, "postDate": "2020-12-30T12:58:56.090Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1131978, "postDate": "2020-12-30T05:18:53.707Z", "content": "The bug has been fixed. 😄Now jut have to trust our CV.
", "rawMarkdown": "The bug has been fixed. 😄Now jut have to trust our CV." }, { "id": 1131864, "postDate": "2020-12-30T03:29:16.443Z", "content": "I am so surprised to hear about these operations, never expected… and it's too late for me to see those results, but can we draw a basic conclusion that we can really trust our CV and LB, and dataset in PB also has noise.
", "rawMarkdown": "I am so surprised to hear about these operations, never expected... and it's too late for me to see those results, but can we draw a basic conclusion that we can really trust our CV and LB, and dataset in PB also has noise." } ], "comments": [ { "id": 1131491, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-12-29T18:54:47.360000", "content": "It will be interesting how Kaggle will react to this major issue.
\nIt is hard to change the private entire datasets to all active competition. What can be done is to truncate it, remove a percentage of data but keeping the same labels distribution.
\nThis will not fix the leaking problem, some people already seen what is working and what not directly on the private leaderboard, but this action will at least change the private score for whomever seen it
When I changed selected submissions, the private leader board score was changed at the same time.
\nThis means we can check all the scores for private leader board.
\nSo I think that private dataset must be updated if possible.
Hi, please see this statement from our team regarding the leaderboard bug.
", "votes": 1, "replies": [ { "id": 1132969, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-12-30T20:35:22.887000", "content": "Thanks for the statement release @juliaelliott. Will Kaggle staff take actions to alter the private datasets of the ongoing competitions ?
", "votes": 0, "replies": [] }, { "id": 1132973, "author_name": "Julia Elliott", "author_url": "", "post_date": "2020-12-30T20:47:02.577000", "content": "@vladvdv We are in contact with the hosts of currently live competitions to determine if and what any actions will be.
", "votes": 0, "replies": [] } ] }, { "id": 1131284, "author_name": "Atanas Atanasov", "author_url": "", "post_date": "2020-12-29T16:31:27.453000", "content": "The bug is fixed and trust your CV :)
", "votes": 1, "replies": [] }, { "id": 1131273, "author_name": "Debarshi Chanda", "author_url": "", "post_date": "2020-12-29T16:19:06.440000", "content": "The bug is fixed now
\nPrivate LB is no longer accessible
Wow! After so many years and numerous competition shake ups, can't believe the private LB score was just there all the time.
", "votes": 2, "replies": [] }, { "id": 1132119, "author_name": "Benlei Cui", "author_url": "", "post_date": "2020-12-30T07:14:12.153000", "content": "LB 89.46 PB 89.28, which means the LB is reliable, so does the cv score. turst your local cv:D
", "votes": 2, "replies": [ { "id": 1132255, "author_name": "", "author_url": "", "post_date": "2020-12-30T09:10:15.793000", "content": "", "votes": 3, "replies": [] }, { "id": 1132484, "author_name": "", "author_url": "", "post_date": "2020-12-30T12:43:51.683000", "content": "", "votes": 0, "replies": [] }, { "id": 1132506, "author_name": "", "author_url": "", "post_date": "2020-12-30T12:58:56.090000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1131978, "author_name": "Shivam Johri", "author_url": "", "post_date": "2020-12-30T05:18:53.707000", "content": "The bug has been fixed. 😄Now jut have to trust our CV.
", "votes": 0, "replies": [] }, { "id": 1131864, "author_name": "Yi Wu", "author_url": "", "post_date": "2020-12-30T03:29:16.443000", "content": "I am so surprised to hear about these operations, never expected… and it's too late for me to see those results, but can we draw a basic conclusion that we can really trust our CV and LB, and dataset in PB also has noise.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1131159": "I tried the method that has been discussed in the discussion.\n\nhttps://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357\nApparently, you can see the value of private score by following these steps.\n\n1. Go to Leaderboard\n1. open Devtool -> Network\n1. XHR only->select getUser request\n1. Preview -> result -> team, maybe you can see your public/private score and ranking(It seems to display the value of the currently selected submission.)\n\nThe attached image is a screenshot taken with the above method.", "1131491": "It will be interesting how Kaggle will react to this major issue.\nIt is hard to change the private entire datasets to all active competition. What can be done is to truncate it, remove a percentage of data but keeping the same labels distribution. \nThis will not fix the leaking problem, some people already seen what is working and what not directly on the private leaderboard, but this action will at least change the private score for whomever seen it", "1131235": "When I changed selected submissions, the private leader board score was changed at the same time.\nThis means we can check all the scores for private leader board.\nSo I think that private dataset must be updated if possible.", "1132963": "Hi, please see [this statement](https://www.kaggle.com/private-leaderboard-bug-dec-2020) from our team regarding the leaderboard bug.", "1131284": "The bug is fixed and trust your CV :) ", "1131273": "The bug is fixed now\nPrivate LB is no longer accessible", "1133335": "Wow! After so many years and numerous competition shake ups, can't believe the private LB score was just there all the time.", "1132119": "LB 89.46 PB 89.28, which means the LB is reliable, so does the cv score. turst your local cv:D", "1131978": "The bug has been fixed. 😄Now jut have to trust our CV.", "1131864": "I am so surprised to hear about these operations, never expected... and it's too late for me to see those results, but can we draw a basic conclusion that we can really trust our CV and LB, and dataset in PB also has noise." } }