\nPlz help in this regard. Why my all of the submissions got rejected. 😥
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F153367%2F85044455ab0e5e9021227b23d1c0e04f%2FScreenshot_20210108-112234.jpg?generation=1610086941040024&alt=media\")
Dear Fellow Kagglers,
\nThank you for participating in Riiid AIEd Challenge 2020! We hope you had as much fun competing as we did organizing it. As the competition draws to a close, we want to hear your feedback so that we can keep improving our competition.
\nSurvey is now closed! Thank you for all participants who completed the survey. Your inputs are super valuable!
\nOnce again, thank you for taking this giant leap together with us by participating in this year’s challenge. Happy Kaggling!
\nOn behalf of Riiid,
\nTim & Jin (Challenge Organizers)
Dear Fellow Kagglers,
\nThank you for participating in Riiid AIEd Challenge 2020! We hope you had as much fun competing as we did organizing it. As the competition draws to a close, we want to hear your feedback so that we can keep improving our competition.
\nSurvey is now closed! Thank you for all participants who completed the survey. Your inputs are super valuable!
\nOnce again, thank you for taking this giant leap together with us by participating in this year’s challenge. Happy Kaggling!
\nOn behalf of Riiid,
\nTim & Jin (Challenge Organizers)
Permission denied:
\n\"This form can only be viewed by users in the owner's organization.
\nTry contacting the owner of the form if you think this is a mistake.\"
", "rawMarkdown": "Permission denied:\n\n\"This form can only be viewed by users in the owner's organization.\n\nTry contacting the owner of the form if you think this is a mistake.\"", "votes": null }, { "id": "1142235", "postDate": "01/07/2021 08:44:16", "content": "Thank you for pointing this out! Please try again with the updated link!
", "rawMarkdown": "Thank you for pointing this out! Please try again with the updated link!", "votes": null }, { "id": "1142239", "postDate": "01/07/2021 08:46:55", "content": "It works now. Thanks for this amazing competition👍.
", "rawMarkdown": "It works now. Thanks for this amazing competition👍.", "votes": null }, { "id": "1142289", "postDate": "01/07/2021 09:43:37", "content": "I both enjoyed very much this competition and was also very frustrated by the submission api. The subject is very interesting and working on the Saint+ model was also very interesting, I learned a lot. However, the submission API dosn't give meaningfull error messages and it could be so much better. For example, you could check that each predict calls provided all the necessary row_ids and yield instantly a clear error message if not. You could check for duplicates in row_id, for finiteness of answered_correctly, etc.
\nI had a lot of the time, errors were not clear at all (submission score error or timeout) that could have been very easy to fix if you had done this kind of error handling. Hopefully I solved all my errors in the last day but it was a pain.
I think so too.
\nI didn't no what type of error occurred in submission. I just saw after 9 hours and couldn't see error message.
Overall the competition had been really fun.
\nJust one input on the submission errors. The errors not being displayed is a headache. I believe it could've been simply resolved by providing us with a validation dataset that is in the same format as hidden test data and available during training time. So we can run it as sample inference for smoothing out the inference bugs and improving the pipeline. Leaving us more time to work on the actual model itself.
", "rawMarkdown": "Overall the competition had been really fun.\n\nJust one input on the submission errors. The errors not being displayed is a headache. I believe it could've been simply resolved by providing us with a validation dataset that is in the same format as hidden test data and available during training time. So we can run it as sample inference for smoothing out the inference bugs and improving the pipeline. Leaving us more time to work on the actual model itself.", "votes": null }, { "id": "1142385", "postDate": "01/07/2021 10:49:58", "content": "my wooden PC plays the game so hard
", "rawMarkdown": "my wooden PC plays the game so hard", "votes": null }, { "id": "1142475", "postDate": "01/07/2021 12:07:32", "content": "@hoonpyotimjeon
\nI think you'll probably get more survey submissions if you pin the discussion
Great competition.
\nThis was my first NN code and also my first Kaggle competition (if you exclude the fork I submitted for Covid19 mRNA couple of months back). In fact it was my first programming experience after a long long time :)
\nI had lot of troubles with my ID till last week and did not get required support from Kaggle as they do not seem to look at individual issues but only attend to group issues.
\nHowever this competition itself was wonderfully organized and planned!
", "rawMarkdown": "Great competition. \n\nThis was my first NN code and also my first Kaggle competition (if you exclude the fork I submitted for Covid19 mRNA couple of months back). In fact it was my first programming experience after a long long time :) \n\nI had lot of troubles with my ID till last week and did not get required support from Kaggle as they do not seem to look at individual issues but only attend to group issues. \n\nHowever this competition itself was wonderfully organized and planned!", "votes": null }, { "id": "1143488", "postDate": "01/07/2021 23:32:04", "content": "Thank you for hosting great competition! I had a great time for three months :D
", "rawMarkdown": "Thank you for hosting great competition! I had a great time for three months :D", "votes": null }, { "id": "1143507", "postDate": "01/07/2021 23:53:26", "content": "Thanks for the competition
", "rawMarkdown": "Thanks for the competition", "votes": null }, { "id": "1143518", "postDate": "01/08/2021 00:02:56", "content": "Thank you for a fun competition.
\nThis was the best competition I have ever participated in.
The volume and quality of the data, the challenges, and the design of the competition were all very sophisticated. In particular, the appropriateness of the constraints on inference (time series API, limited RAM, inference time) was a good reference for other competitions.
\nIf Riiid ever holds a competition on a related task, I'll be sure to join in !!!
", "rawMarkdown": "Thank you for a fun competition.\nThis was the best competition I have ever participated in.\n\nThe volume and quality of the data, the challenges, and the design of the competition were all very sophisticated. In particular, the appropriateness of the constraints on inference (time series API, limited RAM, inference time) was a good reference for other competitions.\n\nIf Riiid ever holds a competition on a related task, I'll be sure to join in !!!", "votes": null }, { "id": "1143657", "postDate": "01/08/2021 01:48:04", "content": "My first competition and it was both fun and educational. Thank you so much!
", "rawMarkdown": "My first competition and it was both fun and educational. Thank you so much!", "votes": null }, { "id": "1143727", "postDate": "01/08/2021 03:07:00", "content": "This competition was very fun and informative.
\nThank you for great competition.
I submitted predictions only for questions (where content_type_id = 0) but my submissions got rejected.
\nIt shows \"Submissions Scoring Error\". I have attached the screenshot of my submission page.
\nThe last line of my submitting code is as follows:
\nenv.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])
\nPlease let me know of any mistake in the above line (of code). I am unable to get score for my submissions because of this = Submission Scoring Error.
\nPlz help in this regard. Why my all of the submissions got rejected. 😥
When I ran my model on the given example_test.csv, it gave right results (without any Nan or NULL values). Could any one help, it appears my all efforts are in vain if I am unable to get score for any of my submission ?
", "rawMarkdown": "When I ran my model on the given example_test.csv, it gave right results (without any Nan or NULL values). Could any one help, it appears my all efforts are in vain if I am unable to get score for any of my submission ?", "votes": null }, { "id": "1145263", "postDate": "01/09/2021 03:03:25", "content": "Thanks, Organisers. It was a great competition that came with a lot of challenges like 100M large dataset, Kaggle 16GB RAM limit and 9 hours submission limit. The time series API during inference was also a very new experience. This competition was really close to the real-world challenges in Data Science industry.
\nAs many have mentioned the only problem in the competition was being not able to debug the \"Submission Error\". Sometimes it was very irritating to not able to see the error after submission running for hours. Having some prompt of the stderr would have saved us days. Though in retrospective it forced us to be more careful about our code and look for all corner cases before submitting.
\nAll in all a great competition. Thanks for organising it.
", "rawMarkdown": "Thanks, Organisers. It was a great competition that came with a lot of challenges like 100M large dataset, Kaggle 16GB RAM limit and 9 hours submission limit. The time series API during inference was also a very new experience. This competition was really close to the real-world challenges in Data Science industry. \n\nAs many have mentioned the only problem in the competition was being not able to debug the \"**Submission Error**\". Sometimes it was very irritating to not able to see the error after submission running for hours. Having some prompt of the stderr would have saved us **days**. Though in retrospective it forced us to be more careful about our code and look for all corner cases before submitting.\n\nAll in all a great competition. Thanks for organising it.", "votes": null }, { "id": "1148530", "postDate": "01/11/2021 07:41:18", "content": "@hoonpyotimjeon , Thank you for this great competition, I enjoyed it a lot, even a bit too much obsessed!
\nWhen would be the next AIEd Challenge hosted by Riiid?
", "rawMarkdown": "hoonpyotimjeon , Thank you for this great competition, I enjoyed it a lot, even a bit too much obsessed!\n\nWhen would be the next AIEd Challenge hosted by Riiid?", "votes": null }, { "id": "1148761", "postDate": "01/11/2021 11:19:28", "content": "Thanks for the competition!
\nFor me, the format was equal parts exciting and frustrating. Debugging notebooks without any feedback was very painful. But at the same time, it's interesting to participate in a competition that makes you think about the engineering aspects of your solution as well as the data science ones. I particularly liked that, due to the constraints, you can't simply use an ensemble of a gazillion different models, and that the task is hard enough that the top teams are not fighting for scores around 0.99.
", "rawMarkdown": "Thanks for the competition!\n\nFor me, the format was equal parts exciting and frustrating. Debugging notebooks without any feedback was very painful. But at the same time, it's interesting to participate in a competition that makes you think about the engineering aspects of your solution as well as the data science ones. I particularly liked that, due to the constraints, you can't simply use an ensemble of a gazillion different models, and that the task is hard enough that the top teams are not fighting for scores around 0.99.", "votes": null } ], "comments": [ { "id": 1142226, "author_name": "buumoo", "author_url": "", "post_date": "01/07/2021 08:37:45", "content": "Permission denied:
\n\"This form can only be viewed by users in the owner's organization.
\nTry contacting the owner of the form if you think this is a mistake.\"
", "votes": null, "replies": [ { "id": 1142235, "author_name": "hoonpyotimjeon", "author_url": "", "post_date": "01/07/2021 08:44:16", "content": "Thank you for pointing this out! Please try again with the updated link!
", "votes": null, "replies": [] }, { "id": 1142239, "author_name": "buumoo", "author_url": "", "post_date": "01/07/2021 08:46:55", "content": "It works now. Thanks for this amazing competition👍.
", "votes": null, "replies": [] } ] }, { "id": 1142289, "author_name": "rodolphelampe", "author_url": "", "post_date": "01/07/2021 09:43:37", "content": "I both enjoyed very much this competition and was also very frustrated by the submission api. The subject is very interesting and working on the Saint+ model was also very interesting, I learned a lot. However, the submission API dosn't give meaningfull error messages and it could be so much better. For example, you could check that each predict calls provided all the necessary row_ids and yield instantly a clear error message if not. You could check for duplicates in row_id, for finiteness of answered_correctly, etc.
\nI had a lot of the time, errors were not clear at all (submission score error or timeout) that could have been very easy to fix if you had done this kind of error handling. Hopefully I solved all my errors in the last day but it was a pain.
I think so too.
\nI didn't no what type of error occurred in submission. I just saw after 9 hours and couldn't see error message.
Overall the competition had been really fun.
\nJust one input on the submission errors. The errors not being displayed is a headache. I believe it could've been simply resolved by providing us with a validation dataset that is in the same format as hidden test data and available during training time. So we can run it as sample inference for smoothing out the inference bugs and improving the pipeline. Leaving us more time to work on the actual model itself.
", "votes": null, "replies": [] }, { "id": 1142385, "author_name": "southsakura", "author_url": "", "post_date": "01/07/2021 10:49:58", "content": "my wooden PC plays the game so hard
", "votes": null, "replies": [] }, { "id": 1142475, "author_name": "abdurrafae", "author_url": "", "post_date": "01/07/2021 12:07:32", "content": "@hoonpyotimjeon
\nI think you'll probably get more survey submissions if you pin the discussion
Great competition.
\nThis was my first NN code and also my first Kaggle competition (if you exclude the fork I submitted for Covid19 mRNA couple of months back). In fact it was my first programming experience after a long long time :)
\nI had lot of troubles with my ID till last week and did not get required support from Kaggle as they do not seem to look at individual issues but only attend to group issues.
\nHowever this competition itself was wonderfully organized and planned!
", "votes": null, "replies": [] }, { "id": 1143488, "author_name": "kurupical", "author_url": "", "post_date": "01/07/2021 23:32:04", "content": "Thank you for hosting great competition! I had a great time for three months :D
", "votes": null, "replies": [] }, { "id": 1143507, "author_name": "kuraji", "author_url": "", "post_date": "01/07/2021 23:53:26", "content": "Thanks for the competition
", "votes": null, "replies": [] }, { "id": 1143518, "author_name": "haradataman", "author_url": "", "post_date": "01/08/2021 00:02:56", "content": "Thank you for a fun competition.
\nThis was the best competition I have ever participated in.
The volume and quality of the data, the challenges, and the design of the competition were all very sophisticated. In particular, the appropriateness of the constraints on inference (time series API, limited RAM, inference time) was a good reference for other competitions.
\nIf Riiid ever holds a competition on a related task, I'll be sure to join in !!!
", "votes": null, "replies": [] }, { "id": 1143657, "author_name": "thomasliptay", "author_url": "", "post_date": "01/08/2021 01:48:04", "content": "My first competition and it was both fun and educational. Thank you so much!
", "votes": null, "replies": [] }, { "id": 1143727, "author_name": "japandata509", "author_url": "", "post_date": "01/08/2021 03:07:00", "content": "This competition was very fun and informative.
\nThank you for great competition.
I submitted predictions only for questions (where content_type_id = 0) but my submissions got rejected.
\nIt shows \"Submissions Scoring Error\". I have attached the screenshot of my submission page.
\nThe last line of my submitting code is as follows:
\nenv.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])
\nPlease let me know of any mistake in the above line (of code). I am unable to get score for my submissions because of this = Submission Scoring Error.
\nPlz help in this regard. Why my all of the submissions got rejected. 😥
When I ran my model on the given example_test.csv, it gave right results (without any Nan or NULL values). Could any one help, it appears my all efforts are in vain if I am unable to get score for any of my submission ?
", "votes": null, "replies": [] } ] }, { "id": 1145263, "author_name": "lplenka", "author_url": "", "post_date": "01/09/2021 03:03:25", "content": "Thanks, Organisers. It was a great competition that came with a lot of challenges like 100M large dataset, Kaggle 16GB RAM limit and 9 hours submission limit. The time series API during inference was also a very new experience. This competition was really close to the real-world challenges in Data Science industry.
\nAs many have mentioned the only problem in the competition was being not able to debug the \"Submission Error\". Sometimes it was very irritating to not able to see the error after submission running for hours. Having some prompt of the stderr would have saved us days. Though in retrospective it forced us to be more careful about our code and look for all corner cases before submitting.
\nAll in all a great competition. Thanks for organising it.
", "votes": null, "replies": [] }, { "id": 1148530, "author_name": "yihdarshieh", "author_url": "", "post_date": "01/11/2021 07:41:18", "content": "@hoonpyotimjeon , Thank you for this great competition, I enjoyed it a lot, even a bit too much obsessed!
\nWhen would be the next AIEd Challenge hosted by Riiid?
", "votes": null, "replies": [] }, { "id": 1148761, "author_name": "lisskor", "author_url": "", "post_date": "01/11/2021 11:19:28", "content": "Thanks for the competition!
\nFor me, the format was equal parts exciting and frustrating. Debugging notebooks without any feedback was very painful. But at the same time, it's interesting to participate in a competition that makes you think about the engineering aspects of your solution as well as the data science ones. I particularly liked that, due to the constraints, you can't simply use an ensemble of a gazillion different models, and that the task is hard enough that the top teams are not fighting for scores around 0.99.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1142218": "Dear Fellow Kagglers,\n\nThank you for participating in Riiid AIEd Challenge 2020! We hope you had as much fun competing as we did organizing it. As the competition draws to a close, we want to hear your feedback so that we can keep improving our competition. ~~In your free time, please complete this quick post-challenge survey and let us know your thoughts by 11:59PM UTC on Jan 14, 2021!~~ \n\n**Survey is now closed! Thank you for all participants who completed the survey. Your inputs are super valuable!**\n\nOnce again, thank you for taking this giant leap together with us by participating in this year’s challenge. Happy Kaggling!\n\nOn behalf of Riiid,\nTim & Jin (Challenge Organizers)", "1142226": "Permission denied:\n\n\"This form can only be viewed by users in the owner's organization.\n\nTry contacting the owner of the form if you think this is a mistake.\"", "1142235": "Thank you for pointing this out! Please try again with the updated link!", "1142239": "It works now. Thanks for this amazing competition👍.", "1142289": "I both enjoyed very much this competition and was also very frustrated by the submission api. The subject is very interesting and working on the Saint+ model was also very interesting, I learned a lot. However, the submission API dosn't give meaningfull error messages and it could be so much better. For example, you could check that each predict calls provided all the necessary row_ids and yield instantly a clear error message if not. You could check for duplicates in row_id, for finiteness of answered_correctly, etc. \nI had a lot of the time, errors were not clear at all (submission score error or timeout) that could have been very easy to fix if you had done this kind of error handling. Hopefully I solved all my errors in the last day but it was a pain.", "1142355": "I think so too. \nI didn't no what type of error occurred in submission. I just saw after 9 hours and couldn't see error message.", "1142381": "Overall the competition had been really fun.\n\nJust one input on the submission errors. The errors not being displayed is a headache. I believe it could've been simply resolved by providing us with a validation dataset that is in the same format as hidden test data and available during training time. So we can run it as sample inference for smoothing out the inference bugs and improving the pipeline. Leaving us more time to work on the actual model itself.", "1142385": "my wooden PC plays the game so hard", "1142475": "hoonpyotimjeon \nI think you'll probably get more survey submissions if you pin the discussion", "1142823": "Great competition. \n\nThis was my first NN code and also my first Kaggle competition (if you exclude the fork I submitted for Covid19 mRNA couple of months back). In fact it was my first programming experience after a long long time :) \n\nI had lot of troubles with my ID till last week and did not get required support from Kaggle as they do not seem to look at individual issues but only attend to group issues. \n\nHowever this competition itself was wonderfully organized and planned!", "1143488": "Thank you for hosting great competition! I had a great time for three months :D", "1143507": "Thanks for the competition", "1143518": "Thank you for a fun competition.\nThis was the best competition I have ever participated in.\n\nThe volume and quality of the data, the challenges, and the design of the competition were all very sophisticated. In particular, the appropriateness of the constraints on inference (time series API, limited RAM, inference time) was a good reference for other competitions.\n\nIf Riiid ever holds a competition on a related task, I'll be sure to join in !!!", "1143657": "My first competition and it was both fun and educational. Thank you so much!", "1143727": "This competition was very fun and informative.\nThank you for great competition.", "1143930": "I submitted predictions only for questions (where content_type_id = 0) but my submissions got rejected.\n\nIt shows \"Submissions Scoring Error\". I have attached the screenshot of my submission page.\n\nThe last line of my submitting code is as follows:\n\nenv.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])\n\nPlease let me know of any mistake in the above line (of code). I am unable to get score for my submissions because of this = Submission Scoring Error.\nPlz help in this regard. Why my all of the submissions got rejected. 😥 ", "1143950": "When I ran my model on the given example_test.csv, it gave right results (without any Nan or NULL values). Could any one help, it appears my all efforts are in vain if I am unable to get score for any of my submission ?", "1145263": "Thanks, Organisers. It was a great competition that came with a lot of challenges like 100M large dataset, Kaggle 16GB RAM limit and 9 hours submission limit. The time series API during inference was also a very new experience. This competition was really close to the real-world challenges in Data Science industry. \n\nAs many have mentioned the only problem in the competition was being not able to debug the \"**Submission Error**\". Sometimes it was very irritating to not able to see the error after submission running for hours. Having some prompt of the stderr would have saved us **days**. Though in retrospective it forced us to be more careful about our code and look for all corner cases before submitting.\n\nAll in all a great competition. Thanks for organising it.", "1148530": "hoonpyotimjeon , Thank you for this great competition, I enjoyed it a lot, even a bit too much obsessed!\n\nWhen would be the next AIEd Challenge hosted by Riiid?", "1148761": "Thanks for the competition!\n\nFor me, the format was equal parts exciting and frustrating. Debugging notebooks without any feedback was very painful. But at the same time, it's interesting to participate in a competition that makes you think about the engineering aspects of your solution as well as the data science ones. I particularly liked that, due to the constraints, you can't simply use an ensemble of a gazillion different models, and that the task is hard enough that the top teams are not fighting for scores around 0.99." }, "source": "meta" }