As you may have already seen, it's currently possible to bypass the API's protections and achieve a perfect score. I want to thank @piupiupiu for bringing this to our attention promptly.
\nI am partway through testing a rough draft of the updates that should resolve this issue and hope to deploy them during PST business hours tomorrow.
Once the updates are deployed, all notebooks published during the relevant time window will be rerun with the updates. This will erase any impacts to the leaderboard from the vulnerability.
\nThank you for your patience; I'll provide another update when the update is live.
\nEdit: new status posted here.
", "messages": [ { "id": "1073105", "postDate": "11/09/2020 06:06:44", "content": "As you may have already seen, it's currently possible to bypass the API's protections and achieve a perfect score. I want to thank @piupiupiu for bringing this to our attention promptly.
\nI am partway through testing a rough draft of the updates that should resolve this issue and hope to deploy them during PST business hours tomorrow.
Once the updates are deployed, all notebooks published during the relevant time window will be rerun with the updates. This will erase any impacts to the leaderboard from the vulnerability.
\nThank you for your patience; I'll provide another update when the update is live.
\nEdit: new status posted here.
", "rawMarkdown": "As you may have already seen, [it's currently possible to bypass the API's protections](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/195815) and achieve a perfect score. I want to thank @piupiupiu for bringing this to our attention promptly.\nI am partway through testing a rough draft of the updates that should resolve this issue and hope to deploy them during PST business hours tomorrow.\n\nOnce the updates are deployed, all notebooks published during the relevant time window will be rerun with the updates. This will erase any impacts to the leaderboard from the vulnerability.\n\nThank you for your patience; I'll provide another update when the update is live.\n\nEdit: new [status posted here](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/196210).", "votes": null }, { "id": "1073261", "postDate": "11/09/2020 11:04:21", "content": "hello , we have submission issues
\nall our submissions display submission scor error , our notebooks submitted always produce a correct csv file
\noften when we try a previous successful submissions we still get the same issues
Is it possible to make a submission in the meantime?
", "rawMarkdown": "Is it possible to make a submission in the meantime?", "votes": null }, { "id": "1073536", "postDate": "11/09/2020 15:47:07", "content": "I suppose that is the reason we cannot submit ?
", "rawMarkdown": "I suppose that is the reason we cannot submit ?", "votes": null }, { "id": "1073747", "postDate": "11/09/2020 22:11:40", "content": "So it's not just me, was looking here for this problem
", "rawMarkdown": "So it's not just me, was looking here for this problem", "votes": null }, { "id": "1074198", "postDate": "11/10/2020 11:31:50", "content": "What error do you get? Notebook timeout?
", "rawMarkdown": "What error do you get? Notebook timeout?", "votes": null }, { "id": "1074201", "postDate": "11/10/2020 11:41:40", "content": "I get Submission Error. I do not have timeout as I train my model separately and then I upload it in a database. I did not have timeout issues, but memory issues, when training. I had to deal with it.
I do also seperate both, somehow I still get notebook timeout. I will investigate further, thank you.
", "rawMarkdown": "I do also seperate both, somehow I still get notebook timeout. I will investigate further, thank you.", "votes": null }, { "id": "1074312", "postDate": "11/10/2020 14:06:34", "content": "It's instant submission score error. By the run time to get the error I would say the notebook completes a single loop in the iter_test
", "rawMarkdown": "It's instant submission score error. By the run time to get the error I would say the notebook completes a single loop in the iter_test", "votes": null }, { "id": "1085048", "postDate": "11/20/2020 16:22:38", "content": "Do you have solved the submission error?
\nI have met the same problem.
yes , its ok
\nit was a runing time problem
\nyou have to ensure a runing time under the 9 hours , test included
hello , we have submission issues
\nall our submissions display submission scor error , our notebooks submitted always produce a correct csv file
\noften when we try a previous successful submissions we still get the same issues
Do you have solved the submission error?
\nI have met the same problem.
yes , its ok
\nit was a runing time problem
\nyou have to ensure a runing time under the 9 hours , test included
Is it possible to make a submission in the meantime?
", "votes": null, "replies": [] }, { "id": 1073536, "author_name": "catadanna", "author_url": "", "post_date": "11/09/2020 15:47:07", "content": "I suppose that is the reason we cannot submit ?
", "votes": null, "replies": [ { "id": 1073747, "author_name": "iuryck", "author_url": "", "post_date": "11/09/2020 22:11:40", "content": "So it's not just me, was looking here for this problem
", "votes": null, "replies": [] }, { "id": 1074198, "author_name": "aliabdin1", "author_url": "", "post_date": "11/10/2020 11:31:50", "content": "What error do you get? Notebook timeout?
", "votes": null, "replies": [] }, { "id": 1074201, "author_name": "catadanna", "author_url": "", "post_date": "11/10/2020 11:41:40", "content": "I get Submission Error. I do not have timeout as I train my model separately and then I upload it in a database. I did not have timeout issues, but memory issues, when training. I had to deal with it.
I do also seperate both, somehow I still get notebook timeout. I will investigate further, thank you.
", "votes": null, "replies": [] }, { "id": 1074312, "author_name": "iuryck", "author_url": "", "post_date": "11/10/2020 14:06:34", "content": "It's instant submission score error. By the run time to get the error I would say the notebook completes a single loop in the iter_test
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1073105": "As you may have already seen, [it's currently possible to bypass the API's protections](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/195815) and achieve a perfect score. I want to thank @piupiupiu for bringing this to our attention promptly.\nI am partway through testing a rough draft of the updates that should resolve this issue and hope to deploy them during PST business hours tomorrow.\n\nOnce the updates are deployed, all notebooks published during the relevant time window will be rerun with the updates. This will erase any impacts to the leaderboard from the vulnerability.\n\nThank you for your patience; I'll provide another update when the update is live.\n\nEdit: new [status posted here](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/196210).", "1073261": "hello , we have submission issues \nall our submissions display submission scor error , our notebooks submitted always produce a correct csv file \noften when we try a previous successful submissions we still get the same issues", "1073320": "Is it possible to make a submission in the meantime?", "1073536": "I suppose that is the reason we cannot submit ?", "1073747": "So it's not just me, was looking here for this problem", "1074198": "What error do you get? Notebook timeout?", "1074201": "I get `Submission Error`. I do not have timeout as I train my model separately and then I upload it in a database. I did not have timeout issues, but memory issues, when training. I had to deal with it.", "1074213": "I do also seperate both, somehow I still get notebook timeout. I will investigate further, thank you.", "1074312": "It's instant submission score error. By the run time to get the error I would say the notebook completes a single loop in the iter_test", "1085048": "Do you have solved the submission error?\nI have met the same problem.", "1085193": "yes , its ok \nit was a runing time problem \nyou have to ensure a runing time under the 9 hours , test included" }, "source": "meta" }