It seems that the Efficiency LB has been finally updated with the private scores.
\nHowever, there is still one issue: LB includes some late submissions. Those can be easily tracked from the psp-efficiency-lb-sanity-check notebook.
\n@philculliton could you please take a look? Thanks in advance.
\nUPD: The issue has been promptly fixed by the Kaggle team.
", "messages": [ { "id": 2340972, "postDate": "2023-07-11T19:39:22.717Z", "content": "It seems that the Efficiency LB has been finally updated with the private scores.
\nHowever, there is still one issue: LB includes some late submissions. Those can be easily tracked from the psp-efficiency-lb-sanity-check notebook.
\n@philculliton could you please take a look? Thanks in advance.
\nUPD: The issue has been promptly fixed by the Kaggle team.
", "rawMarkdown": "It seems that [the Efficiency LB ](https://www.kaggle.com/code/philculliton/student-performance-efficiency-leaderboard/) has been finally updated with the private scores.\n\nHowever, there is still one issue: LB includes some late submissions. Those can be easily tracked from the [psp-efficiency-lb-sanity-check](https://www.kaggle.com/code/kononenko/psp-efficiency-lb-sanity-check) notebook.\n\n@philculliton could you please take a look? Thanks in advance.\n\n**UPD:** The issue has been promptly fixed by the Kaggle team.", "votes": 5 }, { "id": 2340977, "postDate": "2023-07-11T19:45:13.003Z", "content": "@kononenko Thank you, we're looking into it right now!
", "rawMarkdown": "@kononenko Thank you, we're looking into it right now!", "votes": 3, "replies": [ { "id": 2340992, "postDate": "2023-07-11T19:53:46.283Z", "content": "@philculliton, great, thanks for the quick follow-up!
", "rawMarkdown": "@philculliton, great, thanks for the quick follow-up!", "replies": [ { "id": 2341028, "postDate": "2023-07-11T20:15:25.973Z", "content": "@kononenko Sure, thank you for the report! There was a minor glitch in the efficiency LB calculation, it's been updated and appears to be corrected.
", "rawMarkdown": "@kononenko Sure, thank you for the report! There was a minor glitch in the efficiency LB calculation, it's been updated and appears to be corrected.", "votes": 2 }, { "id": 2341043, "postDate": "2023-07-11T20:25:19.763Z", "content": "Yep, no late submissions anymore 👍
", "rawMarkdown": "Yep, no late submissions anymore 👍" } ] } ] }, { "id": 2341000, "postDate": "2023-07-11T19:58:41.240Z", "content": "I am new to Kaggle competitions, I am wondering whether if there is medals also for the efficiency leaderboard or no, anybody have a clue?
", "rawMarkdown": "I am new to Kaggle competitions, I am wondering whether if there is medals also for the efficiency leaderboard or no, anybody have a clue? ", "votes": 1, "replies": [ { "id": 2341052, "postDate": "2023-07-11T20:30:35.540Z", "content": "What I saw was a proposition to award efficiency medals. It was greatly supported by the community, but not sure there was any official response.
\n@philculliton may be you could comment on that? Thanks again.
", "rawMarkdown": "What I saw was a [proposition](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384541) to award efficiency medals. It was greatly supported by the community, but not sure there was any official response.\n\n@philculliton may be you could comment on that? Thanks again.", "votes": 1, "replies": [ { "id": 2341122, "postDate": "2023-07-11T21:39:05.383Z", "content": "Hi! We don't currently have the infrastructure to award medals based on efficiency LBs. That's a great suggestion, though, and while they currently are separate secondary leaderboards, we'll continue to pursue formats where the main leaderboard metric incorporates more than just accuracy!
", "rawMarkdown": "Hi! We don't currently have the infrastructure to award medals based on efficiency LBs. That's a great suggestion, though, and while they currently are separate secondary leaderboards, we'll continue to pursue formats where the main leaderboard metric incorporates more than just accuracy!", "votes": 2 } ] } ] }, { "id": 2341182, "postDate": "2023-07-11T23:41:01.073Z", "content": "I'm sorry for being annoying, @philculliton, but I've just tried to figure out what is the value of Benchmark in the efficiency formula and it feels like it is not constant per submission.
+ lb/(lb - lb/)\n -\n \n \n \n \n ... \n \n \n \n \n \nLength: , dtype: float64\nAm I missing something?
", "rawMarkdown": "I'm sorry for being annoying, @philculliton, but I've just tried to figure out what is the value of `Benchmark` in [the efficiency formula](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/overview/efficiency-prize-evaluation) and it feels like it is not constant per submission.\n\n```\n0.705 + lb[\"PrivateScore\"]/(lb[\"EfficiencyScore\"] - lb[\"ScoringTime\"]/32400)\n```\n\n```\n0 -0.000113\n1 0.000021\n2 0.000101\n3 0.000730\n4 0.000350\n ... \n2013 0.705000\n2014 0.000807\n2015 0.000574\n2016 0.000462\n2017 0.000254\nLength: 2018, dtype: float64\n```\n\nAm I missing something?", "replies": [ { "id": 2341283, "postDate": "2023-07-12T02:29:16.247Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 2340977, "author_name": "Phil Culliton", "author_url": "", "post_date": "2023-07-11T19:45:13.003000", "content": "@kononenko Thank you, we're looking into it right now!
", "votes": 3, "replies": [ { "id": 2340992, "author_name": "Oleksiy Kononenko", "author_url": "", "post_date": "2023-07-11T19:53:46.283000", "content": "@philculliton, great, thanks for the quick follow-up!
", "votes": 0, "replies": [ { "id": 2341028, "author_name": "Phil Culliton", "author_url": "", "post_date": "2023-07-11T20:15:25.973000", "content": "@kononenko Sure, thank you for the report! There was a minor glitch in the efficiency LB calculation, it's been updated and appears to be corrected.
", "votes": 2, "replies": [] }, { "id": 2341043, "author_name": "Oleksiy Kononenko", "author_url": "", "post_date": "2023-07-11T20:25:19.763000", "content": "Yep, no late submissions anymore 👍
", "votes": 0, "replies": [] } ] } ] }, { "id": 2341000, "author_name": "Belal Emad", "author_url": "", "post_date": "2023-07-11T19:58:41.240000", "content": "I am new to Kaggle competitions, I am wondering whether if there is medals also for the efficiency leaderboard or no, anybody have a clue?
", "votes": 1, "replies": [ { "id": 2341052, "author_name": "Oleksiy Kononenko", "author_url": "", "post_date": "2023-07-11T20:30:35.540000", "content": "What I saw was a proposition to award efficiency medals. It was greatly supported by the community, but not sure there was any official response.
\n@philculliton may be you could comment on that? Thanks again.
", "votes": 1, "replies": [ { "id": 2341122, "author_name": "Phil Culliton", "author_url": "", "post_date": "2023-07-11T21:39:05.383000", "content": "Hi! We don't currently have the infrastructure to award medals based on efficiency LBs. That's a great suggestion, though, and while they currently are separate secondary leaderboards, we'll continue to pursue formats where the main leaderboard metric incorporates more than just accuracy!
", "votes": 2, "replies": [] } ] } ] }, { "id": 2341182, "author_name": "Oleksiy Kononenko", "author_url": "", "post_date": "2023-07-11T23:41:01.073000", "content": "I'm sorry for being annoying, @philculliton, but I've just tried to figure out what is the value of Benchmark in the efficiency formula and it feels like it is not constant per submission.
+ lb/(lb - lb/)\n -\n \n \n \n \n ... \n \n \n \n \n \nLength: , dtype: float64\nAm I missing something?
", "votes": 0, "replies": [ { "id": 2341283, "author_name": "", "author_url": "", "post_date": "2023-07-12T02:29:16.247000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2340972": "It seems that [the Efficiency LB ](https://www.kaggle.com/code/philculliton/student-performance-efficiency-leaderboard/) has been finally updated with the private scores.\n\nHowever, there is still one issue: LB includes some late submissions. Those can be easily tracked from the [psp-efficiency-lb-sanity-check](https://www.kaggle.com/code/kononenko/psp-efficiency-lb-sanity-check) notebook.\n\n@philculliton could you please take a look? Thanks in advance.\n\n**UPD:** The issue has been promptly fixed by the Kaggle team.", "2340977": "@kononenko Thank you, we're looking into it right now!", "2341000": "I am new to Kaggle competitions, I am wondering whether if there is medals also for the efficiency leaderboard or no, anybody have a clue? ", "2341182": "I'm sorry for being annoying, @philculliton, but I've just tried to figure out what is the value of `Benchmark` in [the efficiency formula](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/overview/efficiency-prize-evaluation) and it feels like it is not constant per submission.\n\n```\n0.705 + lb[\"PrivateScore\"]/(lb[\"EfficiencyScore\"] - lb[\"ScoringTime\"]/32400)\n```\n\n```\n0 -0.000113\n1 0.000021\n2 0.000101\n3 0.000730\n4 0.000350\n ... \n2013 0.705000\n2014 0.000807\n2015 0.000574\n2016 0.000462\n2017 0.000254\nLength: 2018, dtype: float64\n```\n\nAm I missing something?" } }