![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2Ffb7a70b406f149e112ac2479e13adbea%2Fmsg5588674617-1300.jpg?generation=1667263041135945&alt=media\")
Dear Kagglers,
\nIt was supposed to be a nice post of us training more than 100+ models with awesome ensemble and last hour decision which submission to choose, but alas!
\n
It seems Kaggle algorithm or whatever Halloween witchcraft decided to remove us as cheaters. I have sent a dispute, and hope it will be resolved. Meanwhile, do I need to post the solution, how do you think?
\nThanks to Neptune.ai team for providing us with enough training logging hours. I was skeptical about it in the beginning, but the platform happened to be extremely important part of the \"success\".
\nP.s. I was really serious about this month competition. Our Neptune.ai experiment log supports it:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F8b5057dbc5daea3c4df3ac564fe5770d%2FExperiment%20history.png?generation=1667263901000763&alt=media\")
P.s.s. your upvote will help us to resolve the drama faster.
", "messages": [ { "id": 2011944, "postDate": "2022-11-01T00:38:51.660Z", "content": "Dear Kagglers,
\nIt was supposed to be a nice post of us training more than 100+ models with awesome ensemble and last hour decision which submission to choose, but alas!
\n
It seems Kaggle algorithm or whatever Halloween witchcraft decided to remove us as cheaters. I have sent a dispute, and hope it will be resolved. Meanwhile, do I need to post the solution, how do you think?
\nThanks to Neptune.ai team for providing us with enough training logging hours. I was skeptical about it in the beginning, but the platform happened to be extremely important part of the \"success\".
\nP.s. I was really serious about this month competition. Our Neptune.ai experiment log supports it:
\n
P.s.s. your upvote will help us to resolve the drama faster.
", "rawMarkdown": "Dear Kagglers, \n\nIt was supposed to be a nice post of us training more than 100+ models with awesome ensemble and last hour decision which submission to choose, but alas!\n\n\n\nIt seems Kaggle algorithm or whatever Halloween witchcraft decided to remove us as cheaters. I have sent a dispute, and hope it will be resolved. Meanwhile, do I need to post the solution, how do you think?\n\nThanks to [Neptune.ai](https://neptune.ai/) team for providing us with enough training logging hours. I was skeptical about it in the beginning, but the platform happened to be extremely important part of the \"success\".\n\nP.s. I was really serious about this month competition. Our Neptune.ai experiment log supports it:\n\n\n\nP.s.s. your upvote will help us to resolve the drama faster.", "votes": 18 }, { "id": 2012038, "postDate": "2022-11-01T03:50:27.223Z", "content": "Compliance will review the request when they receive it. And just a general reminder that falsifying information on a compliance request (e.g., stating there was no cheating where this is overwhelming evidence that there was) will almost always result in an account ban.
", "rawMarkdown": "Compliance will review the request when they receive it. And just a general reminder that falsifying information on a compliance request (e.g., stating there was no cheating where this is overwhelming evidence that there was) will almost always result in an account ban. ", "votes": 5, "replies": [ { "id": 2012608, "postDate": "2022-11-01T09:35:05.820Z", "content": "Hi, @inversion ,I'm new to kaggle. I would like to know when the account is closed due to cheating, will the account still be unblocked? :D
", "rawMarkdown": "Hi, @inversion ,I'm new to kaggle. I would like to know when the account is closed due to cheating, will the account still be unblocked? :D", "votes": -1, "isDeleted": true } ] }, { "id": 2056635, "postDate": "2022-12-06T10:09:33.603Z", "content": "Hi, @sergiosaharovskiy , @samuelcortinhas , @inversion , @dster ,
\nIs it possible to have a follow-up about why Sergey & Sam's solution was withdrawn from the leatherboard?
\nI am curious about how you could cheat in such a competition (except if you can access the private leaderboard test set).
If you do not disclose this information, could you explain why you choose to do so?
", "rawMarkdown": "Hi, @sergiosaharovskiy , @samuelcortinhas , @inversion , @dster , \n\nIs it possible to have a follow-up about why Sergey & Sam's solution was withdrawn from the leatherboard?\nI am curious about how you could cheat in such a competition (except if you can access the private leaderboard test set).\n\nIf you do not disclose this information, could you explain why you choose to do so?", "votes": 1, "replies": [ { "id": 2057020, "postDate": "2022-12-06T17:14:38.713Z", "content": "Hey Adrien, we generally do not want to litigate leaderboard removals in public, we have a Compliance program to privately discuss any removals which are contested and consider reinstatement. Accessing the test set labels is one way to cheat, but there are other ways to violate the rules of the competition which can be grounds for removal.
", "rawMarkdown": "Hey Adrien, we generally do not want to litigate leaderboard removals in public, we have a Compliance program to privately discuss any removals which are contested and consider reinstatement. Accessing the test set labels is one way to cheat, but there are other ways to violate the rules of the competition which can be grounds for removal.", "votes": 2 } ] }, { "id": 2014525, "postDate": "2022-11-02T16:12:07.583Z", "content": "Congrats @sergiosaharovskiy and good luck getting the issue resolved
\nI wanted to ask how you obtained your predictions for the 100+model ensemble. Were your predictions saved to neptune.ai or did you save the models and rerun them?
\nThe process of how you did this would really help me understand and improve, congrats again
", "rawMarkdown": "Congrats @sergiosaharovskiy and good luck getting the issue resolved\n\nI wanted to ask how you obtained your predictions for the 100+model ensemble. Were your predictions saved to neptune.ai or did you save the models and rerun them? \n\nThe process of how you did this would really help me understand and improve, congrats again ", "votes": 1, "replies": [ { "id": 2014785, "postDate": "2022-11-02T20:47:33.940Z", "content": "@slythe I used a callback every 10 epoch to save the model. The submission file was generated after the training. If the model showed good performance during the training and good stats during the inference (classifications report, confusion matrix, histograms), I saved it as f'{exp_num}_{neptune_run_num}.pth.
\nIn case if the model were lost, the results could have been reproduced, because the seed was set manually, as follows:
():\n random.seed(x)\n np.random.seed(x)\n torch.manual_seed(x)\n torch.backends.cudnn.deterministic = \n torch.backends.cudnn.benchmark = \n torch.cuda.is_available(): torch.cuda.manual_seed_all(x)\nFantastic thank you @sergiosaharovskiy .This helps alot
\nMy initial idea was to save the submission results for every run and use that for ensembling
Hello @sergiosaharovskiy good to see the usage of Neptune AI, I hope this gets resolved
", "rawMarkdown": "Hello @sergiosaharovskiy good to see the usage of Neptune AI, I hope this gets resolved", "votes": 2, "replies": [ { "id": 2011956, "postDate": "2022-11-01T00:59:31.217Z", "content": "Yep @cv13j0 but you probably mean @sergiosaharovskiy to tag :)
", "rawMarkdown": "Yep @cv13j0 but you probably mean @sergiosaharovskiy to tag :)", "votes": 2 }, { "id": 2015688, "postDate": "2022-11-03T13:13:53.637Z", "content": "Thanks, @alexryzhkov, for the heads up. I fixed my post.
", "rawMarkdown": "Thanks, @alexryzhkov, for the heads up. I fixed my post." } ] }, { "id": 2011949, "postDate": "2022-11-01T00:45:09.090Z", "content": "@sergiosaharovskiy yep, I can't believe my eyes after another LB update - they have deleted you from the public LB as well. Something strange is going on..
\nI think that we need @dster here to clarify the situation π
", "rawMarkdown": "@sergiosaharovskiy yep, I can't believe my eyes after another LB update - they have deleted you from the public LB as well. Something strange is going on..\n\nI think that we need @dster here to clarify the situation π", "votes": 2 }, { "id": 2011947, "postDate": "2022-11-01T00:43:20.727Z", "content": "I was surprised not to see you on the LB, but probably kaggle will resolve. Congratulations, you deserved it!
", "rawMarkdown": "I was surprised not to see you on the LB, but probably kaggle will resolve. Congratulations, you deserved it!", "votes": 2 } ], "comments": [ { "id": 2012038, "author_name": "inversion", "author_url": "", "post_date": "2022-11-01T03:50:27.223000", "content": "Compliance will review the request when they receive it. And just a general reminder that falsifying information on a compliance request (e.g., stating there was no cheating where this is overwhelming evidence that there was) will almost always result in an account ban.
", "votes": 5, "replies": [ { "id": 2012608, "author_name": "", "author_url": "", "post_date": "2022-11-01T09:35:05.820000", "content": "Hi, @inversion ,I'm new to kaggle. I would like to know when the account is closed due to cheating, will the account still be unblocked? :D
", "votes": -1, "replies": [] } ] }, { "id": 2056635, "author_name": "Adrien Pacifico", "author_url": "", "post_date": "2022-12-06T10:09:33.603000", "content": "Hi, @sergiosaharovskiy , @samuelcortinhas , @inversion , @dster ,
\nIs it possible to have a follow-up about why Sergey & Sam's solution was withdrawn from the leatherboard?
\nI am curious about how you could cheat in such a competition (except if you can access the private leaderboard test set).
If you do not disclose this information, could you explain why you choose to do so?
", "votes": 1, "replies": [ { "id": 2057020, "author_name": "DJ Sterling", "author_url": "", "post_date": "2022-12-06T17:14:38.713000", "content": "Hey Adrien, we generally do not want to litigate leaderboard removals in public, we have a Compliance program to privately discuss any removals which are contested and consider reinstatement. Accessing the test set labels is one way to cheat, but there are other ways to violate the rules of the competition which can be grounds for removal.
", "votes": 2, "replies": [] } ] }, { "id": 2014525, "author_name": "Andrew Schleiss", "author_url": "", "post_date": "2022-11-02T16:12:07.583000", "content": "Congrats @sergiosaharovskiy and good luck getting the issue resolved
\nI wanted to ask how you obtained your predictions for the 100+model ensemble. Were your predictions saved to neptune.ai or did you save the models and rerun them?
\nThe process of how you did this would really help me understand and improve, congrats again
", "votes": 1, "replies": [ { "id": 2014785, "author_name": "SSS", "author_url": "", "post_date": "2022-11-02T20:47:33.940000", "content": "@slythe I used a callback every 10 epoch to save the model. The submission file was generated after the training. If the model showed good performance during the training and good stats during the inference (classifications report, confusion matrix, histograms), I saved it as f'{exp_num}_{neptune_run_num}.pth.
\nIn case if the model were lost, the results could have been reproduced, because the seed was set manually, as follows:
():\n random.seed(x)\n np.random.seed(x)\n torch.manual_seed(x)\n torch.backends.cudnn.deterministic = \n torch.backends.cudnn.benchmark = \n torch.cuda.is_available(): torch.cuda.manual_seed_all(x)\nFantastic thank you @sergiosaharovskiy .This helps alot
\nMy initial idea was to save the submission results for every run and use that for ensembling
Hello @sergiosaharovskiy good to see the usage of Neptune AI, I hope this gets resolved
", "votes": 2, "replies": [ { "id": 2011956, "author_name": "Alexander Ryzhkov", "author_url": "", "post_date": "2022-11-01T00:59:31.217000", "content": "Yep @cv13j0 but you probably mean @sergiosaharovskiy to tag :)
", "votes": 2, "replies": [] }, { "id": 2015688, "author_name": "C4rl05/V", "author_url": "", "post_date": "2022-11-03T13:13:53.637000", "content": "Thanks, @alexryzhkov, for the heads up. I fixed my post.
", "votes": 0, "replies": [] } ] }, { "id": 2011949, "author_name": "Alexander Ryzhkov", "author_url": "", "post_date": "2022-11-01T00:45:09.090000", "content": "@sergiosaharovskiy yep, I can't believe my eyes after another LB update - they have deleted you from the public LB as well. Something strange is going on..
\nI think that we need @dster here to clarify the situation π
", "votes": 2, "replies": [] }, { "id": 2011947, "author_name": "Jose CΓ‘liz", "author_url": "", "post_date": "2022-11-01T00:43:20.727000", "content": "I was surprised not to see you on the LB, but probably kaggle will resolve. Congratulations, you deserved it!
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "2011944": "Dear Kagglers, \n\nIt was supposed to be a nice post of us training more than 100+ models with awesome ensemble and last hour decision which submission to choose, but alas!\n\n\n\nIt seems Kaggle algorithm or whatever Halloween witchcraft decided to remove us as cheaters. I have sent a dispute, and hope it will be resolved. Meanwhile, do I need to post the solution, how do you think?\n\nThanks to [Neptune.ai](https://neptune.ai/) team for providing us with enough training logging hours. I was skeptical about it in the beginning, but the platform happened to be extremely important part of the \"success\".\n\nP.s. I was really serious about this month competition. Our Neptune.ai experiment log supports it:\n\n\n\nP.s.s. your upvote will help us to resolve the drama faster.", "2012038": "Compliance will review the request when they receive it. And just a general reminder that falsifying information on a compliance request (e.g., stating there was no cheating where this is overwhelming evidence that there was) will almost always result in an account ban. ", "2056635": "Hi, @sergiosaharovskiy , @samuelcortinhas , @inversion , @dster , \n\nIs it possible to have a follow-up about why Sergey & Sam's solution was withdrawn from the leatherboard?\nI am curious about how you could cheat in such a competition (except if you can access the private leaderboard test set).\n\nIf you do not disclose this information, could you explain why you choose to do so?", "2014525": "Congrats @sergiosaharovskiy and good luck getting the issue resolved\n\nI wanted to ask how you obtained your predictions for the 100+model ensemble. Were your predictions saved to neptune.ai or did you save the models and rerun them? \n\nThe process of how you did this would really help me understand and improve, congrats again ", "2011953": "Hello @sergiosaharovskiy good to see the usage of Neptune AI, I hope this gets resolved", "2011949": "@sergiosaharovskiy yep, I can't believe my eyes after another LB update - they have deleted you from the public LB as well. Something strange is going on..\n\nI think that we need @dster here to clarify the situation π", "2011947": "I was surprised not to see you on the LB, but probably kaggle will resolve. Congratulations, you deserved it!" } }