{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# Riiid! Answer Correctness Prediction\n## Introduction\nIn this competition you will predict which questions each student is able to answer correctly. You will loop through a series of batches of questions. Once you make that prediction, you can move on to the next batch.\n\nThis competition is different from most Kaggle Competitions in that:\n* You can only submit from Kaggle Notebooks\n* You must use our custom **`riiideducation`** Python module.  The purpose of this module is to control the flow of information to ensure that you are not using future data to make predictions.  If you do not use this module properly, your code may fail.\n\n## In this Starter Notebook, we'll show how to use the **`riiideducation`** module to get the test features and make predictions.\n## TL;DR: End-to-End Usage Example\n```\nimport riiideducation\nenv = riiideducation.make_env()\n\n# Training data is in the competition dataset as usual\ntrain_df = pd.read_csv('/kaggle/input/riiideducation/train.csv', low_memory=False)\ntrain_my_model(train_df)\niter_test = env.iter_test()\nfor (test_df, sample_prediction_df) in iter_test:\n    test_df['answered_correctly'] = 0.5\n    env.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])```\nNote that `train_my_model` is a function you need to write for the above example to work."},{"metadata":{},"cell_type":"markdown","source":"## In-depth Introduction\nFirst let's import the module and create an environment."},{"metadata":{"trusted":true},"cell_type":"code","source":"import riiideducation\nimport pandas as pd\n\n# You can only call make_env() once, so don't lose it!\nenv = riiideducation.make_env()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training data is in the competition dataset as usual\nIt's larger than will fit in memory with default settings, so we'll specify more efficient datatypes and only load a subset of the data for now."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/riiid-test-answer-prediction/train.csv', low_memory=False, nrows=10**5, \n                       dtype={'row_id': 'int64', 'timestamp': 'int64', 'user_id': 'int32', 'content_id': 'int16', 'content_type_id': 'int8',\n                              'task_container_id': 'int16', 'user_answer': 'int8', 'answered_correctly': 'int8', 'prior_question_elapsed_time': 'float32', \n                             'prior_question_had_explanation': 'boolean',\n                             }\n                      )\ntrain_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## `iter_test` function\n\nGenerator which loops through each batch of questions in the test set. You have direct access to the example test rows for your convenience, but your code will only be able to get rows from the real test set via the API. Once you call **`predict`** you can continue on to the next batch.\n\nYields:\n* While there are more batch(es) and `predict` was called successfully since the last yield, yields a tuple of:\n    * `test_df`: DataFrame with the test features for the next batch, and user responses for the previous batch.\n    * `sample_prediction_df`: DataFrame with an example prediction.  Intended to be filled in and passed back to the `predict` function.\n* If `predict` has not been called successfully since the last yield, prints an error and yields `None`."},{"metadata":{"trusted":true},"cell_type":"code","source":"# You can only iterate through a result from `env.iter_test()` once\n# so be careful not to lose it once you start iterating.\niter_test = env.iter_test()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's get the data for the first test batch and check it out."},{"metadata":{"trusted":true},"cell_type":"code","source":"(test_df, sample_prediction_df) = next(iter_test)\ntest_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_prediction_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Note that we'll get an error if we try to continue on to the next batch without making our predictions for the current batch."},{"metadata":{"trusted":true},"cell_type":"code","source":"next(iter_test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### **`predict`** function\nStores your predictions for the current batch.  Expects the same format as you saw in `sample_prediction_df` returned from the `iter_test` generator.\n\nArgs:\n* `predictions_df`: DataFrame which must have the same format as `sample_prediction_df`.\n\nThis function will raise an Exception if not called after a successful iteration of the `iter_test` generator."},{"metadata":{},"cell_type":"markdown","source":"Let's make a dummy prediction using the sample provided by `iter_test`."},{"metadata":{"trusted":true},"cell_type":"code","source":"env.predict(sample_prediction_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Main Loop\nLet's loop through all the remaining batches in the test set generator and make the default prediction for each.  The `iter_test` generator will simply stop returning values once you've reached the end.\n\nWhen writing your own Notebooks, be sure to write robust code that makes as few assumptions about the `iter_test`/`predict` loop as possible.  For example, the test set contains question IDs that have not been previously observed in train.\n\nYou may assume that the structure of `sample_prediction_df` will not change in this competition.\n\n**The lecture rows in `test_df` should not be submitted.**"},{"metadata":{"trusted":true},"cell_type":"code","source":"for (test_df, sample_prediction_df) in iter_test:\n    test_df['answered_correctly'] = 0.5\n    env.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## Restart the Notebook to run your code again\nIn order to combat cheating, you are only allowed to call `make_env` or iterate through `iter_test` once per Notebook run.  However, while you're iterating on your model it's reasonable to try something out, change the model a bit, and try it again.  Unfortunately, if you try to simply re-run the code, or even refresh the browser page, you'll still be running on the same Notebook execution session you had been running before, and the `riideducation` module will still throw errors.  To get around this, you need to explicitly restart your Notebook execution session, which you can do by **clicking \"Run\"->\"Restart Session\"** in the Notebook Editor's menu bar at the top."},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}