Now that the embargo is over, I see that the final results are listed in user profiles, but it does not appear that the points or overall user ranking has changed to reflect this. Is this a glitch, or is there normally a delay between announcing winners\r\n and updating user rankings?
", "messages": [ { "id": "22165", "postDate": "04/04/2013 16:26:14", "content": "Now that the embargo is over, I see that the final results are listed in user profiles, but it does not appear that the points or overall user ranking has changed to reflect this. Is this a glitch, or is there normally a delay between announcing winners\r\n and updating user rankings?
", "rawMarkdown": "", "votes": null }, { "id": "22175", "postDate": "04/04/2013 18:08:34", "content": "This issue should be resolved now. Sorry for the delay and errors.
", "rawMarkdown": "", "votes": null }, { "id": "22180", "postDate": "04/04/2013 19:08:49", "content": "In my case only the profile (competion count) was updated, but zero change in ranking or points that I could observe...
", "rawMarkdown": "", "votes": null }, { "id": "22227", "postDate": "04/05/2013 10:10:56", "content": "This was my first competition on kaggle but I don't see any points or ranking yet (/anymore*)
\r\n*a short while after the submission deadline points and ranking where visible
\r\n
\r\n
It would be helpful if a Kaggle Admin could reply, so we could could distinguish among the following cases:
\r\n1) "We acknowledge that there is still a problem, we are working on it, and there is no need to bother us any more about it. We will let you know when it's fixed."
\r\n2) "We don't consider this important enough to investigate. We have other priorities, and we'll get to this when we can."
\r\n3) "We only check the forums every other day. We just heard about this, and we will look into it again."
\r\n4) "You guys are wrong. We already fixed this, and if you think otherwise, speak up and provide evidence."
", "rawMarkdown": "", "votes": null }, { "id": "22306", "postDate": "04/06/2013 02:29:15", "content": "The answer is #1.
\r\nAs you may have noticed, the profiles have gone through some updates recently and there have been a few kinks. Thanks for letting us know about this problem, and we'll try to get it resolved as soon as we can. We do think this is important to investigate,\r\n but we are also responsible for other priorities, so unfortunately, the final resolution may not come as quickly as we woud all like. (In other words, yes let us know about problems but be understanding when we can't fix them instantaneously. :) )
", "rawMarkdown": "", "votes": null }, { "id": "22447", "postDate": "04/08/2013 23:33:31", "content": "I wanted to provide an update on this situation.
\r\nFollowing an investigation into reputation leakage after GE Flight Quest deadline we discovered two issues in the reputation system relating to multi-part competitions with lowest-is-best metrics, and to calculations made during periods where the final leaderboard\r\n has been withheld (due to the decay factor in the ranking algorithm).
\r\nAs a result, we slightly changed the points algorithm (to anchor all decay calculations at the most recent competition deadline that has been finalised and make point calculation stable over time) and replaced about half the reputation manager code to implement\r\n the decay change (and resolve it with recent addition of Highest Result and Novice/Kaggler/Master tiering).
\r\nWe deployed the new reputation code last week, and it resolved the leakage and ranking correctness issues.
\r\nUnfortunately the extra correctness made the reputation sweep a bit slower, pushed it over our timeouts for production background jobs and as a result the new more robust reputation code was not completing.
\r\nA much faster version has now been pushed to production and is functioning correctly.
\r\nI believe this resolves all outstanding issues with points and rankings. If you are seeing any further issues, please let me know.
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 22175, "author_name": "joycenv", "author_url": "", "post_date": "04/04/2013 18:08:34", "content": "This issue should be resolved now. Sorry for the delay and errors.
", "votes": null, "replies": [] }, { "id": 22180, "author_name": "konradb", "author_url": "", "post_date": "04/04/2013 19:08:49", "content": "In my case only the profile (competion count) was updated, but zero change in ranking or points that I could observe...
", "votes": null, "replies": [] }, { "id": 22227, "author_name": "julesvanligtenberg", "author_url": "", "post_date": "04/05/2013 10:10:56", "content": "This was my first competition on kaggle but I don't see any points or ranking yet (/anymore*)
\r\n*a short while after the submission deadline points and ranking where visible
\r\n
\r\n
It would be helpful if a Kaggle Admin could reply, so we could could distinguish among the following cases:
\r\n1) "We acknowledge that there is still a problem, we are working on it, and there is no need to bother us any more about it. We will let you know when it's fixed."
\r\n2) "We don't consider this important enough to investigate. We have other priorities, and we'll get to this when we can."
\r\n3) "We only check the forums every other day. We just heard about this, and we will look into it again."
\r\n4) "You guys are wrong. We already fixed this, and if you think otherwise, speak up and provide evidence."
", "votes": null, "replies": [] }, { "id": 22306, "author_name": "joycenv", "author_url": "", "post_date": "04/06/2013 02:29:15", "content": "The answer is #1.
\r\nAs you may have noticed, the profiles have gone through some updates recently and there have been a few kinks. Thanks for letting us know about this problem, and we'll try to get it resolved as soon as we can. We do think this is important to investigate,\r\n but we are also responsible for other priorities, so unfortunately, the final resolution may not come as quickly as we woud all like. (In other words, yes let us know about problems but be understanding when we can't fix them instantaneously. :) )
", "votes": null, "replies": [] }, { "id": 22447, "author_name": "adamkennedy", "author_url": "", "post_date": "04/08/2013 23:33:31", "content": "I wanted to provide an update on this situation.
\r\nFollowing an investigation into reputation leakage after GE Flight Quest deadline we discovered two issues in the reputation system relating to multi-part competitions with lowest-is-best metrics, and to calculations made during periods where the final leaderboard\r\n has been withheld (due to the decay factor in the ranking algorithm).
\r\nAs a result, we slightly changed the points algorithm (to anchor all decay calculations at the most recent competition deadline that has been finalised and make point calculation stable over time) and replaced about half the reputation manager code to implement\r\n the decay change (and resolve it with recent addition of Highest Result and Novice/Kaggler/Master tiering).
\r\nWe deployed the new reputation code last week, and it resolved the leakage and ranking correctness issues.
\r\nUnfortunately the extra correctness made the reputation sweep a bit slower, pushed it over our timeouts for production background jobs and as a result the new more robust reputation code was not completing.
\r\nA much faster version has now been pushed to production and is functioning correctly.
\r\nI believe this resolves all outstanding issues with points and rankings. If you are seeing any further issues, please let me know.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "22165": "", "22175": "", "22180": "", "22227": "", "22286": "", "22306": "", "22447": "" }, "source": "meta" }