As @giuliasavorgnan identified, a handful of the experiments shared the same plate layouts.
\n\nI'll be updating the scoring system soon to ignore the affected experiments, then recalculating the leaderboard. You may see some odd effects while these updates are underway, but the disruption should be brief and we don't expect material changes once the process is complete.
\n\nApologies for this, and thanks again to Guilia Savorgnan for flagging the issue.
\n\nUpdate: the rescore is now complete.
", "messages": [ { "id": "598457", "postDate": "08/13/2019 15:42:14", "content": "As @giuliasavorgnan identified, a handful of the experiments shared the same plate layouts.
\n\nI'll be updating the scoring system soon to ignore the affected experiments, then recalculating the leaderboard. You may see some odd effects while these updates are underway, but the disruption should be brief and we don't expect material changes once the process is complete.
\n\nApologies for this, and thanks again to Guilia Savorgnan for flagging the issue.
\n\nUpdate: the rescore is now complete.
", "rawMarkdown": "As @giuliasavorgnan identified, a handful of the experiments [shared the same plate layouts](https://www.kaggle.com/c/recursion-cellular-image-classification/discussion/102905#593925). \n\nI'll be updating the scoring system soon to ignore the affected experiments, then recalculating the leaderboard. You may see some odd effects while these updates are underway, but the disruption should be brief and we don't expect material changes once the process is complete. \n\nApologies for this, and thanks again to Guilia Savorgnan for flagging the issue.\n\n\n*Update*: the rescore is now complete.", "votes": null }, { "id": "598460", "postDate": "08/13/2019 15:46:41", "content": "If these new changes put me in Gold medal zone from the bottom of LB, I won't mind at all. 😊
", "rawMarkdown": "If these new changes put me in Gold medal zone from the bottom of LB, I won't mind at all. 😊", "votes": null }, { "id": "598502", "postDate": "08/13/2019 16:46:27", "content": "Hi @sohier , can you please specify which experiments exactly are moving into the ignore list? Does the submission file format change in anyway? Thanks!
", "rawMarkdown": "Hi @sohier , can you please specify which experiments exactly are moving into the ignore list? Does the submission file format change in anyway? Thanks!", "votes": null }, { "id": "598508", "postDate": "08/13/2019 16:55:30", "content": "The submission file format does not change at all.
\n\nAs a rule, we avoid commenting on the composition of the test set.
", "rawMarkdown": "The submission file format does not change at all.\n\nAs a rule, we avoid commenting on the composition of the test set.", "votes": null }, { "id": "598570", "postDate": "08/13/2019 18:26:32", "content": "hi Sohier, thanks for notifying. I wonder are the experiments removed from the final private LB score too. thanks.
", "rawMarkdown": "hi Sohier, thanks for notifying. I wonder are the experiments removed from the final private LB score too. thanks.", "votes": null }, { "id": "598578", "postDate": "08/13/2019 18:39:08", "content": "The ignored rows are removed from both the public and private LB scores.
", "rawMarkdown": "The ignored rows are removed from both the public and private LB scores.", "votes": null }, { "id": "627780", "postDate": "09/16/2019 11:11:58", "content": "I was not working on this competition while this dynamic happened. Did anyone recall their public board scores changed when this happened? If the scores changed, the delta contains information, Kaggle needs to let us be able to trace the delta. If Kaggle just overwrites the scores, you are potentially giving a subset of people an advantage.
", "rawMarkdown": "I was not working on this competition while this dynamic happened. Did anyone recall their public board scores changed when this happened? If the scores changed, the delta contains information, Kaggle needs to let us be able to trace the delta. If Kaggle just overwrites the scores, you are potentially giving a subset of people an advantage.", "votes": null }, { "id": "627822", "postDate": "09/16/2019 12:28:46", "content": "I noticed that the public LB scores did not change.
", "rawMarkdown": "I noticed that the public LB scores did not change.", "votes": null }, { "id": "628027", "postDate": "09/16/2019 17:57:28", "content": "Thanks! I guess HUVEC-18 is semi-officially moved to everybody's training set. Wonder how many stuff like this is still in the data.
", "rawMarkdown": "Thanks! I guess HUVEC-18 is semi-officially moved to everybody's training set. Wonder how many stuff like this is still in the data.", "votes": null }, { "id": "628393", "postDate": "09/17/2019 09:05:41", "content": "thanks
", "rawMarkdown": "thanks", "votes": null }, { "id": "628428", "postDate": "09/17/2019 10:28:26", "content": "a comment on the original thread says HUVEC-18 would be in the private LB, so removing it wouldn't change the public LB
", "rawMarkdown": "a [comment on the original thread](https://www.kaggle.com/c/recursion-cellular-image-classification/discussion/102905#594004) says HUVEC-18 would be in the private LB, so removing it wouldn't change the public LB", "votes": null }, { "id": "628437", "postDate": "09/17/2019 10:53:25", "content": "I am not happy with Kaggle's comment on this
\n\n> The ignored rows are removed from both the public and private LB scores.
\n\nHe phrased it like there were other affected stuff from public board data set been removed.
\n\nIf there are leak in the data, the worst thing is to make it more confusing. Please, when there is leak been exposed, don't try to cover up and keep us play the guessing game.
", "rawMarkdown": "I am not happy with Kaggle's comment on this \n\n> The ignored rows are removed from both the public and private LB scores.\n\nHe phrased it like there were other affected stuff from public board data set been removed. \n\nIf there are leak in the data, the worst thing is to make it more confusing. Please, when there is leak been exposed, don't try to cover up and keep us play the guessing game.", "votes": null }, { "id": "628448", "postDate": "09/17/2019 11:11:46", "content": "@ryanzhang , my initial thought was same as yours, but then Sohier commented
\n\n\n\n\nAs a rule, we avoid commenting on the composition of the test set.
\n
and now I think they have a solid set of rules in place, and they stick to it. And these rules do make sense, although it seems confusing at first, at least there is a clear logic behind every sentence that he wrote.
", "rawMarkdown": "ryanzhang , my initial thought was same as yours, but then Sohier commented\n\n> As a rule, we avoid commenting on the composition of the test set.\n\nand now I think they have a solid set of rules in place, and they stick to it. And these rules do make sense, although it seems confusing at first, at least there is a clear logic behind every sentence that he wrote.", "votes": null }, { "id": "628514", "postDate": "09/17/2019 13:04:19", "content": "I am fine with that rule in regular situations. When there are leaks, I do hope they can be crystal clear about it. As a casual kaggler, I want to use my limited time to do actual modeling than leak digging.
", "rawMarkdown": "I am fine with that rule in regular situations. When there are leaks, I do hope they can be crystal clear about it. As a casual kaggler, I want to use my limited time to do actual modeling than leak digging.", "votes": null }, { "id": "633650", "postDate": "09/25/2019 08:26:46", "content": "Thanks for notifying!
", "rawMarkdown": "Thanks for notifying!", "votes": null } ], "comments": [ { "id": 598460, "author_name": "cyberia", "author_url": "", "post_date": "08/13/2019 15:46:41", "content": "If these new changes put me in Gold medal zone from the bottom of LB, I won't mind at all. 😊
", "votes": null, "replies": [] }, { "id": 598502, "author_name": "zaharch", "author_url": "", "post_date": "08/13/2019 16:46:27", "content": "Hi @sohier , can you please specify which experiments exactly are moving into the ignore list? Does the submission file format change in anyway? Thanks!
", "votes": null, "replies": [ { "id": 598508, "author_name": "sohier", "author_url": "", "post_date": "08/13/2019 16:55:30", "content": "The submission file format does not change at all.
\n\nAs a rule, we avoid commenting on the composition of the test set.
", "votes": null, "replies": [] } ] }, { "id": 598570, "author_name": "wjshenggggg", "author_url": "", "post_date": "08/13/2019 18:26:32", "content": "hi Sohier, thanks for notifying. I wonder are the experiments removed from the final private LB score too. thanks.
", "votes": null, "replies": [ { "id": 598578, "author_name": "sohier", "author_url": "", "post_date": "08/13/2019 18:39:08", "content": "The ignored rows are removed from both the public and private LB scores.
", "votes": null, "replies": [] } ] }, { "id": 627780, "author_name": "ryanzhang", "author_url": "", "post_date": "09/16/2019 11:11:58", "content": "I was not working on this competition while this dynamic happened. Did anyone recall their public board scores changed when this happened? If the scores changed, the delta contains information, Kaggle needs to let us be able to trace the delta. If Kaggle just overwrites the scores, you are potentially giving a subset of people an advantage.
", "votes": null, "replies": [ { "id": 627822, "author_name": "zaharch", "author_url": "", "post_date": "09/16/2019 12:28:46", "content": "I noticed that the public LB scores did not change.
", "votes": null, "replies": [] }, { "id": 628027, "author_name": "ryanzhang", "author_url": "", "post_date": "09/16/2019 17:57:28", "content": "Thanks! I guess HUVEC-18 is semi-officially moved to everybody's training set. Wonder how many stuff like this is still in the data.
", "votes": null, "replies": [] }, { "id": 628428, "author_name": "hmendonca", "author_url": "", "post_date": "09/17/2019 10:28:26", "content": "a comment on the original thread says HUVEC-18 would be in the private LB, so removing it wouldn't change the public LB
", "votes": null, "replies": [] }, { "id": 628437, "author_name": "ryanzhang", "author_url": "", "post_date": "09/17/2019 10:53:25", "content": "I am not happy with Kaggle's comment on this
\n\n> The ignored rows are removed from both the public and private LB scores.
\n\nHe phrased it like there were other affected stuff from public board data set been removed.
\n\nIf there are leak in the data, the worst thing is to make it more confusing. Please, when there is leak been exposed, don't try to cover up and keep us play the guessing game.
", "votes": null, "replies": [] }, { "id": 628448, "author_name": "zaharch", "author_url": "", "post_date": "09/17/2019 11:11:46", "content": "@ryanzhang , my initial thought was same as yours, but then Sohier commented
\n\n\n\n\nAs a rule, we avoid commenting on the composition of the test set.
\n
and now I think they have a solid set of rules in place, and they stick to it. And these rules do make sense, although it seems confusing at first, at least there is a clear logic behind every sentence that he wrote.
", "votes": null, "replies": [] }, { "id": 628514, "author_name": "ryanzhang", "author_url": "", "post_date": "09/17/2019 13:04:19", "content": "I am fine with that rule in regular situations. When there are leaks, I do hope they can be crystal clear about it. As a casual kaggler, I want to use my limited time to do actual modeling than leak digging.
", "votes": null, "replies": [] } ] }, { "id": 628393, "author_name": "crazyai89757", "author_url": "", "post_date": "09/17/2019 09:05:41", "content": "thanks
", "votes": null, "replies": [] }, { "id": 633650, "author_name": "pajari", "author_url": "", "post_date": "09/25/2019 08:26:46", "content": "Thanks for notifying!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "598457": "As @giuliasavorgnan identified, a handful of the experiments [shared the same plate layouts](https://www.kaggle.com/c/recursion-cellular-image-classification/discussion/102905#593925). \n\nI'll be updating the scoring system soon to ignore the affected experiments, then recalculating the leaderboard. You may see some odd effects while these updates are underway, but the disruption should be brief and we don't expect material changes once the process is complete. \n\nApologies for this, and thanks again to Guilia Savorgnan for flagging the issue.\n\n\n*Update*: the rescore is now complete.", "598460": "If these new changes put me in Gold medal zone from the bottom of LB, I won't mind at all. 😊", "598502": "Hi @sohier , can you please specify which experiments exactly are moving into the ignore list? Does the submission file format change in anyway? Thanks!", "598508": "The submission file format does not change at all.\n\nAs a rule, we avoid commenting on the composition of the test set.", "598570": "hi Sohier, thanks for notifying. I wonder are the experiments removed from the final private LB score too. thanks.", "598578": "The ignored rows are removed from both the public and private LB scores.", "627780": "I was not working on this competition while this dynamic happened. Did anyone recall their public board scores changed when this happened? If the scores changed, the delta contains information, Kaggle needs to let us be able to trace the delta. If Kaggle just overwrites the scores, you are potentially giving a subset of people an advantage.", "627822": "I noticed that the public LB scores did not change.", "628027": "Thanks! I guess HUVEC-18 is semi-officially moved to everybody's training set. Wonder how many stuff like this is still in the data.", "628393": "thanks", "628428": "a [comment on the original thread](https://www.kaggle.com/c/recursion-cellular-image-classification/discussion/102905#594004) says HUVEC-18 would be in the private LB, so removing it wouldn't change the public LB", "628437": "I am not happy with Kaggle's comment on this \n\n> The ignored rows are removed from both the public and private LB scores.\n\nHe phrased it like there were other affected stuff from public board data set been removed. \n\nIf there are leak in the data, the worst thing is to make it more confusing. Please, when there is leak been exposed, don't try to cover up and keep us play the guessing game.", "628448": "ryanzhang , my initial thought was same as yours, but then Sohier commented\n\n> As a rule, we avoid commenting on the composition of the test set.\n\nand now I think they have a solid set of rules in place, and they stick to it. And these rules do make sense, although it seems confusing at first, at least there is a clear logic behind every sentence that he wrote.", "628514": "I am fine with that rule in regular situations. When there are leaks, I do hope they can be crystal clear about it. As a casual kaggler, I want to use my limited time to do actual modeling than leak digging.", "633650": "Thanks for notifying!" }, "source": "meta" }