I stopped working on this competition about a month ago due to personal reasons but I just want to say that there is leakage from public HPA v18 to Private LB. Looking at my submissions, I got 0.031 on private LB from a leakage file I made early on (no training at all).\nOrganizers told us private LB was clean but it is not true :(
", "messages": [ { "id": "454308", "postDate": "01/11/2019 12:04:23", "content": "I stopped working on this competition about a month ago due to personal reasons but I just want to say that there is leakage from public HPA v18 to Private LB. Looking at my submissions, I got 0.031 on private LB from a leakage file I made early on (no training at all).\nOrganizers told us private LB was clean but it is not true :(
", "rawMarkdown": "I stopped working on this competition about a month ago due to personal reasons but I just want to say that there is leakage from public HPA v18 to Private LB. Looking at my submissions, I got 0.031 on private LB from a leakage file I made early on (no training at all).\nOrganizers told us private LB was clean but it is not true :(", "votes": null }, { "id": "454322", "postDate": "01/11/2019 12:55:52", "content": "I tried with late submission and found the same. 259 leak did affect private lb too
", "rawMarkdown": "I tried with late submission and found the same. 259 leak did affect private lb too", "votes": null }, { "id": "454407", "postDate": "01/11/2019 15:42:49", "content": "I just did an experiment on the leakage: use the leakage labels to replace the different labels in our best submission. There is no difference on both public and private leaderboard. It is consistently with the reports from other people, as the model performance improves, the leakage has less impact.
", "rawMarkdown": "I just did an experiment on the leakage: use the leakage labels to replace the different labels in our best submission. There is no difference on both public and private leaderboard. It is consistently with the reports from other people, as the model performance improves, the leakage has less impact.", "votes": null }, { "id": "454475", "postDate": "01/11/2019 17:37:30", "content": "based on our result, submit our best submission with leakage lower both the private LB and public LB.
", "rawMarkdown": "based on our result, submit our best submission with leakage lower both the private LB and public LB.", "votes": null }, { "id": "454667", "postDate": "01/12/2019 00:27:40", "content": "That just proved that the leak is still in the private lb split. OP is not talking about the leak labels been correct or not, he is just saying that they remained in private.
", "rawMarkdown": "That just proved that the leak is still in the private lb split. OP is not talking about the leak labels been correct or not, he is just saying that they remained in private.", "votes": null }, { "id": "454681", "postDate": "01/12/2019 01:40:22", "content": "I think it depends on original results and how your model is performing. I guess for top rank kagglers, adding leak may not improve much.
", "rawMarkdown": "I think it depends on original results and how your model is performing. I guess for top rank kagglers, adding leak may not improve much.", "votes": null } ], "comments": [ { "id": 454322, "author_name": "ryanzhang", "author_url": "", "post_date": "01/11/2019 12:55:52", "content": "I tried with late submission and found the same. 259 leak did affect private lb too
", "votes": null, "replies": [] }, { "id": 454407, "author_name": "naivelamb", "author_url": "", "post_date": "01/11/2019 15:42:49", "content": "I just did an experiment on the leakage: use the leakage labels to replace the different labels in our best submission. There is no difference on both public and private leaderboard. It is consistently with the reports from other people, as the model performance improves, the leakage has less impact.
", "votes": null, "replies": [] }, { "id": 454475, "author_name": "manyfoldcv", "author_url": "", "post_date": "01/11/2019 17:37:30", "content": "based on our result, submit our best submission with leakage lower both the private LB and public LB.
", "votes": null, "replies": [ { "id": 454667, "author_name": "ryanzhang", "author_url": "", "post_date": "01/12/2019 00:27:40", "content": "That just proved that the leak is still in the private lb split. OP is not talking about the leak labels been correct or not, he is just saying that they remained in private.
", "votes": null, "replies": [] } ] }, { "id": 454681, "author_name": "zjucor", "author_url": "", "post_date": "01/12/2019 01:40:22", "content": "I think it depends on original results and how your model is performing. I guess for top rank kagglers, adding leak may not improve much.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "454308": "I stopped working on this competition about a month ago due to personal reasons but I just want to say that there is leakage from public HPA v18 to Private LB. Looking at my submissions, I got 0.031 on private LB from a leakage file I made early on (no training at all).\nOrganizers told us private LB was clean but it is not true :(", "454322": "I tried with late submission and found the same. 259 leak did affect private lb too", "454407": "I just did an experiment on the leakage: use the leakage labels to replace the different labels in our best submission. There is no difference on both public and private leaderboard. It is consistently with the reports from other people, as the model performance improves, the leakage has less impact.", "454475": "based on our result, submit our best submission with leakage lower both the private LB and public LB.", "454667": "That just proved that the leak is still in the private lb split. OP is not talking about the leak labels been correct or not, he is just saying that they remained in private.", "454681": "I think it depends on original results and how your model is performing. I guess for top rank kagglers, adding leak may not improve much." }, "source": "meta" }