\n\n“That's one small step for man, one giant leap for mankind.” - Neil Armstrong
\n
Welcome to the inaugural Riiid AIEd Challenge! We are excited to collaborate with the best data scientists from around the world to imagine post-COVID education with AI. We hope that this challenge opens doors for everyone to get excited about the field of AI Education!
\nWe, as an AI tutor startup, believe that AI technology has a great potential to improve education. In the meantime, we also understand that there are many challenges ahead of us in this shared journey to develop the field of AI Education. Such challenges span a wide spectrum from the technological to the ethical, including, but not limited to, explainability, privacy, potential bias and equity in learning.
\nDespite all these difficulties, we are hopeful that this challenge will serve as an important first step to invite free and constructive discussion on the future of AI Education. In particular, we ask all of you to be part of the collective intelligence to address these important challenges.
\nIn this vein, we are excited to announce that we are also sponsoring an academic workshop at AAAI-2021 Conference. Prize-winning teams will be invited to present their models at the AAAI-2021 Workshop on AI Education - Imagining Post-COVID Education with AI - in February 2021. All competition participants are welcome to submit their write-ups to the workshop. The first deadline for submissions is November 9, so the earlier you submit the better!
\nWe will also be uploading some materials that could potentially be helpful for you (you are by no means required to use these resources). If there are any additional resources that you find helpful, please feel free to share them on this discussion forum.
\nOnce again, thank you for taking this giant leap together with us by participating in this year’s challenge. Happy Kaggling!
\nOn behalf of Riiid,
\nTim & Jin (Challenge Organizers)
\n\n“That's one small step for man, one giant leap for mankind.” - Neil Armstrong
\n
Welcome to the inaugural Riiid AIEd Challenge! We are excited to collaborate with the best data scientists from around the world to imagine post-COVID education with AI. We hope that this challenge opens doors for everyone to get excited about the field of AI Education!
\nWe, as an AI tutor startup, believe that AI technology has a great potential to improve education. In the meantime, we also understand that there are many challenges ahead of us in this shared journey to develop the field of AI Education. Such challenges span a wide spectrum from the technological to the ethical, including, but not limited to, explainability, privacy, potential bias and equity in learning.
\nDespite all these difficulties, we are hopeful that this challenge will serve as an important first step to invite free and constructive discussion on the future of AI Education. In particular, we ask all of you to be part of the collective intelligence to address these important challenges.
\nIn this vein, we are excited to announce that we are also sponsoring an academic workshop at AAAI-2021 Conference. Prize-winning teams will be invited to present their models at the AAAI-2021 Workshop on AI Education - Imagining Post-COVID Education with AI - in February 2021. All competition participants are welcome to submit their write-ups to the workshop. The first deadline for submissions is November 9, so the earlier you submit the better!
\nWe will also be uploading some materials that could potentially be helpful for you (you are by no means required to use these resources). If there are any additional resources that you find helpful, please feel free to share them on this discussion forum.
\nOnce again, thank you for taking this giant leap together with us by participating in this year’s challenge. Happy Kaggling!
\nOn behalf of Riiid,
\nTim & Jin (Challenge Organizers)
Thanks for organizing this competition!. Where will these resources be shared?
", "rawMarkdown": "Thanks for organizing this competition!. Where will these resources be shared?", "votes": 1 }, { "id": 1048343, "postDate": "2020-10-13T12:03:47.693Z", "content": "Thanks for the post. Where will these resources be shared?
", "rawMarkdown": "Thanks for the post. Where will these resources be shared? ", "votes": 1, "replies": [ { "id": 1048906, "postDate": "2020-10-13T23:32:25.600Z", "content": "@abhimanyud We will be uploading a couple of resources early next week!
", "rawMarkdown": "@abhimanyud We will be uploading a couple of resources early next week!", "votes": 4, "replies": [ { "id": 1049364, "postDate": "2020-10-14T11:19:26.147Z", "content": "Thanks! :)
", "rawMarkdown": "Thanks! :)", "votes": 1 } ] } ] }, { "id": 1096122, "postDate": "2020-11-30T08:23:06.710Z", "content": "Competition looks very exciting!
", "rawMarkdown": "Competition looks very exciting!" }, { "id": 1095115, "postDate": "2020-11-29T08:58:44.637Z", "content": "This competition seems exciting!
", "rawMarkdown": "This competition seems exciting!" }, { "id": 1055541, "postDate": "2020-10-20T23:16:47.057Z", "content": "Hi Tim: Please help me. I am stuck at the end of https://www.kaggle.com/sohier/competition-api-detailed-introduction
\nHow do I move on from that introduction to analyzing the hidden data? Participants tell me that is easy to do, but do not tell me how to do it. Please ….
", "rawMarkdown": "Hi Tim: Please help me. I am stuck at the end of https://www.kaggle.com/sohier/competition-api-detailed-introduction\n\nHow do I move on from that introduction to analyzing the hidden data? Participants tell me that is easy to do, but do not tell me how to do it. Please ....", "replies": [ { "id": 1055895, "postDate": "2020-10-21T07:50:28.377Z", "content": "You cannot analyze the hidden test data. That is only accessible and used when you submit a notebook for prediction and scoring.
\nYou are supposed to analyze the train data (and a little bit from the example test data) to get an understanding of how the data will be.
", "rawMarkdown": "You cannot analyze the hidden test data. That is only accessible and used when you submit a notebook for prediction and scoring.\n\nYou are supposed to analyze the train data (and a little bit from the example test data) to get an understanding of how the data will be." }, { "id": 1055916, "postDate": "2020-10-21T08:16:36.380Z", "content": "Thank you, thank you! \"You cannot analyze the hidden test data. That is only accessible and used when you submit a notebook for prediction and scoring.\"
\nSo now I will look more closely at all the Notebooks that are listed :-)
", "rawMarkdown": "Thank you, thank you! \"You cannot analyze the hidden test data. That is only accessible and used when you submit a notebook for prediction and scoring.\"\n\nSo now I will look more closely at all the Notebooks that are listed :-)" }, { "id": 1094745, "postDate": "2020-11-28T21:46:36.703Z", "content": "Hey, @rohanrao
\nI'm trying to make sure I understand this correctly. In Competition API Detailed Introduction it is stated that the next batch yielded by the iter_test function contains the user responses of the previous group (and whether the user answered the question correctly).
\nAre we allowed to incorporate this new information into our model (since we now know the results)? I thought yes because I'm not sure why it would be there otherwise but perhaps I'm wrong.
\nMuch appreciated!
", "rawMarkdown": "Hey, @rohanrao \n\nI'm trying to make sure I understand this correctly. In Competition API Detailed Introduction it is stated that the next batch yielded by the iter_test function contains the user responses of the previous group (and whether the user answered the question correctly).\n\nAre we allowed to incorporate this new information into our model (since we now know the results)? I thought yes because I'm not sure why it would be there otherwise but perhaps I'm wrong.\n\nMuch appreciated!", "votes": 1 } ] }, { "id": 1053241, "postDate": "2020-10-18T18:45:05.037Z", "content": "Looks Exciting!
", "rawMarkdown": "Looks Exciting!" }, { "id": 1064640, "postDate": "2020-10-30T11:15:19.437Z", "content": "Is this competition worth our time?
\n@hoonpyotimjeon
\nWe are provided with ~100million rows of train data, and you expect to pick the best model by evaluating it with just ~2.5% of the data? I believe the winner will not necessarily have the best model but just a model that predicts the test set best, Is this what you are looking for?
Imagine I provided you with gold's historical prices for the last 52yrs and tasked you to predict 2020 gold prices do you think the best model will be the best at predicting gold price movement? I don't know how the test set is picked and regardless of how it is picked it is too small in my opinion, this is supposed to be a scientific process not a gamble.
\nIf you look at the current CV-LB scores, the best model (CV) also predicts the test set best (LB).
\nThere is nothing wrong in how the test set is picked since it mimics how and how much of inference is required when the model is put into production.
\nSame for the use-case of predicting gold too… you just need the model that predicts best for 2020 (and refresh the model year-on-year). You can always choose to look at only the last 4yrs data (instead of 52yrs) in which case the ratio of test data changes from 1/53 to 1/5.
", "rawMarkdown": "If you look at the current CV-LB scores, the best model (CV) also predicts the test set best (LB).\n\nThere is nothing wrong in how the test set is picked since it mimics how and how much of inference is required when the model is put into production.\n\nSame for the use-case of predicting gold too... you just need the model that predicts best for 2020 (and refresh the model year-on-year). You can always choose to look at only the last 4yrs data (instead of 52yrs) in which case the ratio of test data changes from 1/53 to 1/5.", "votes": 1, "replies": [ { "id": 1064720, "postDate": "2020-10-30T12:56:57.620Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1064722, "postDate": "2020-10-30T12:58:52.520Z", "content": "I may need loads of data to model a student's performance in Mathematics (Content Doesn't change much overtime) which may not be true for Political Science(May change overtime) so if the test set is small the best model will poorly predict Mathematics over time, which is not true if the test set was large enough.
\nBut Hey I be wrong here!
May be I was wrong here but now someone has a perfect score in the LB, this points me to my point the test set is too small.
", "rawMarkdown": "May be I was wrong here but now someone has a perfect score in the LB, this points me to my point the test set is too small. ", "votes": -1, "isDeleted": true } ] }, { "id": 1048865, "postDate": "2020-10-13T21:46:26.890Z", "content": "Thanks for organizing this competition!
", "rawMarkdown": "Thanks for organizing this competition! ", "votes": 2 }, { "id": 1065030, "postDate": "2020-10-30T19:11:50Z", "content": "thanks for the competition!
", "rawMarkdown": "thanks for the competition!" }, { "id": 1050888, "postDate": "2020-10-15T20:51:10.390Z", "content": "Thanks, for the competition!!
", "rawMarkdown": "Thanks, for the competition!! " } ], "comments": [ { "id": 1060805, "author_name": "ManjotSinghDhillon", "author_url": "", "post_date": "2020-10-26T14:50:54.167000", "content": "Thanks for organizing this competition!. Where will these resources be shared?
", "votes": 1, "replies": [] }, { "id": 1048343, "author_name": "Abhimanyu Dikshit", "author_url": "", "post_date": "2020-10-13T12:03:47.693000", "content": "Thanks for the post. Where will these resources be shared?
", "votes": 1, "replies": [ { "id": 1048906, "author_name": "Hoon Pyo (Tim) Jeon", "author_url": "", "post_date": "2020-10-13T23:32:25.600000", "content": "@abhimanyud We will be uploading a couple of resources early next week!
", "votes": 4, "replies": [ { "id": 1049364, "author_name": "Abhimanyu Dikshit", "author_url": "", "post_date": "2020-10-14T11:19:26.147000", "content": "Thanks! :)
", "votes": 1, "replies": [] } ] } ] }, { "id": 1096122, "author_name": "manish k", "author_url": "", "post_date": "2020-11-30T08:23:06.710000", "content": "Competition looks very exciting!
", "votes": 0, "replies": [] }, { "id": 1095115, "author_name": "PRAVEEN B", "author_url": "", "post_date": "2020-11-29T08:58:44.637000", "content": "This competition seems exciting!
", "votes": 0, "replies": [] }, { "id": 1055541, "author_name": "Mike L.", "author_url": "", "post_date": "2020-10-20T23:16:47.057000", "content": "Hi Tim: Please help me. I am stuck at the end of https://www.kaggle.com/sohier/competition-api-detailed-introduction
\nHow do I move on from that introduction to analyzing the hidden data? Participants tell me that is easy to do, but do not tell me how to do it. Please ….
", "votes": 0, "replies": [ { "id": 1055895, "author_name": "Vopani", "author_url": "", "post_date": "2020-10-21T07:50:28.377000", "content": "You cannot analyze the hidden test data. That is only accessible and used when you submit a notebook for prediction and scoring.
\nYou are supposed to analyze the train data (and a little bit from the example test data) to get an understanding of how the data will be.
", "votes": 0, "replies": [] }, { "id": 1055916, "author_name": "Mike L.", "author_url": "", "post_date": "2020-10-21T08:16:36.380000", "content": "Thank you, thank you! \"You cannot analyze the hidden test data. That is only accessible and used when you submit a notebook for prediction and scoring.\"
\nSo now I will look more closely at all the Notebooks that are listed :-)
", "votes": 0, "replies": [] }, { "id": 1094745, "author_name": "Patrick Brazil", "author_url": "", "post_date": "2020-11-28T21:46:36.703000", "content": "Hey, @rohanrao
\nI'm trying to make sure I understand this correctly. In Competition API Detailed Introduction it is stated that the next batch yielded by the iter_test function contains the user responses of the previous group (and whether the user answered the question correctly).
\nAre we allowed to incorporate this new information into our model (since we now know the results)? I thought yes because I'm not sure why it would be there otherwise but perhaps I'm wrong.
\nMuch appreciated!
", "votes": 1, "replies": [] } ] }, { "id": 1053241, "author_name": "Javier del Villar", "author_url": "", "post_date": "2020-10-18T18:45:05.037000", "content": "Looks Exciting!
", "votes": 0, "replies": [] }, { "id": 1064640, "author_name": "", "author_url": "", "post_date": "2020-10-30T11:15:19.437000", "content": "Is this competition worth our time?
\n@hoonpyotimjeon
\nWe are provided with ~100million rows of train data, and you expect to pick the best model by evaluating it with just ~2.5% of the data? I believe the winner will not necessarily have the best model but just a model that predicts the test set best, Is this what you are looking for?
Imagine I provided you with gold's historical prices for the last 52yrs and tasked you to predict 2020 gold prices do you think the best model will be the best at predicting gold price movement? I don't know how the test set is picked and regardless of how it is picked it is too small in my opinion, this is supposed to be a scientific process not a gamble.
\nIf you look at the current CV-LB scores, the best model (CV) also predicts the test set best (LB).
\nThere is nothing wrong in how the test set is picked since it mimics how and how much of inference is required when the model is put into production.
\nSame for the use-case of predicting gold too… you just need the model that predicts best for 2020 (and refresh the model year-on-year). You can always choose to look at only the last 4yrs data (instead of 52yrs) in which case the ratio of test data changes from 1/53 to 1/5.
", "votes": 1, "replies": [ { "id": 1064720, "author_name": "", "author_url": "", "post_date": "2020-10-30T12:56:57.620000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1064722, "author_name": "", "author_url": "", "post_date": "2020-10-30T12:58:52.520000", "content": "I may need loads of data to model a student's performance in Mathematics (Content Doesn't change much overtime) which may not be true for Political Science(May change overtime) so if the test set is small the best model will poorly predict Mathematics over time, which is not true if the test set was large enough.
\nBut Hey I be wrong here!
May be I was wrong here but now someone has a perfect score in the LB, this points me to my point the test set is too small.
", "votes": -1, "replies": [] } ] }, { "id": 1048865, "author_name": "LGreig", "author_url": "", "post_date": "2020-10-13T21:46:26.890000", "content": "Thanks for organizing this competition!
", "votes": 2, "replies": [] }, { "id": 1065030, "author_name": "Rodrigo Leitão", "author_url": "", "post_date": "2020-10-30T19:11:50", "content": "thanks for the competition!
", "votes": 0, "replies": [] }, { "id": 1050888, "author_name": "Avinash Kumar", "author_url": "", "post_date": "2020-10-15T20:51:10.390000", "content": "Thanks, for the competition!!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1048024": "> “That's one small step for man, one giant leap for mankind.” - Neil Armstrong\n\nWelcome to the inaugural Riiid AIEd Challenge! We are excited to collaborate with the best data scientists from around the world to imagine post-COVID education with AI. We hope that this challenge opens doors for everyone to get excited about the field of AI Education!\n\nWe, as an AI tutor startup, believe that AI technology has a great potential to improve education. In the meantime, we also understand that there are many challenges ahead of us in this shared journey to develop the field of AI Education. Such challenges span a wide spectrum from the technological to the ethical, including, but not limited to, explainability, privacy, potential bias and equity in learning. \n\nDespite all these difficulties, we are hopeful that this challenge will serve as an important first step to invite free and constructive discussion on the future of AI Education. In particular, we ask all of you to be part of the collective intelligence to address these important challenges. \n\nIn this vein, we are excited to announce that we are also sponsoring an academic workshop at AAAI-2021 Conference. Prize-winning teams will be invited to present their models at the AAAI-2021 Workshop on AI Education - [Imagining Post-COVID Education with AI](https://sites.google.com/view/tipce-2021/home?authuser=1) - in February 2021. All competition participants are welcome to submit their write-ups to the workshop. The first deadline for submissions is November 9, so the earlier you submit the better!\n\nWe will also be uploading some materials that could potentially be helpful for you (you are by no means required to use these resources). If there are any additional resources that you find helpful, please feel free to share them on this discussion forum.\n\nOnce again, thank you for taking this giant leap together with us by participating in this year’s challenge. Happy Kaggling!\n\nOn behalf of Riiid,\nTim & Jin (Challenge Organizers)", "1060805": "Thanks for organizing this competition!. Where will these resources be shared?", "1048343": "Thanks for the post. Where will these resources be shared? ", "1096122": "Competition looks very exciting!", "1095115": "This competition seems exciting!", "1055541": "Hi Tim: Please help me. I am stuck at the end of https://www.kaggle.com/sohier/competition-api-detailed-introduction\n\nHow do I move on from that introduction to analyzing the hidden data? Participants tell me that is easy to do, but do not tell me how to do it. Please ....", "1053241": "Looks Exciting!", "1064640": "**Is this competition worth our time?**\n@hoonpyotimjeon \nWe are provided with ~100million rows of train data, and you expect to pick the best model by evaluating it with just ~2.5% of the data? I believe the winner will not necessarily have the best model but just a model that predicts the test set best, Is this what you are looking for?\n\nImagine I provided you with gold's historical prices for the last 52yrs and tasked you to predict 2020 gold prices do you think the best model will be the best at predicting gold price movement? I don't know how the test set is picked and regardless of how it is picked it is too small in **my opinion**, this is supposed to be a scientific process not a gamble.\n\n- I kindly ask that you make the competition worth our time, increase the test set to a reasonable size (>=10millions or ~ 10% of train data).\n ", "1048865": "Thanks for organizing this competition! ", "1065030": "thanks for the competition!", "1050888": "Thanks, for the competition!! " } }