{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# Riiid! Answer Correctness Prediction"},{"metadata":{},"cell_type":"markdown","source":"## In-depth Introduction\n\n> **timestamp**: (int64) the time in milliseconds between this user interaction and the first event completion from that user.\n\n> content_type_id: (int8) 0 if the event was a question being posed to the user, 1 if the event was the user watching a lecture.\n\n> task_container_id : (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a task_container_id.\n\n> user_answer : (int8) the user's answer to the question, if any. Read -1 as null, for lectures\n\n> answered_correctly : (int8) if the user responded correctly. Read -1 as null, for lectures.\n\n> prior_question_elapsed_time : (float32) The average time in milliseconds it took a user to answer each question in the previous question bundle, ignoring any lectures in between. Is null for a user's first question bundle or lecture. Note that the time is the average time a user took to solve each question in the previous bundle.\n\n> prior_question_had_explanation: (bool) Whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle, ignoring any lectures in between. The value is shared across a single question bundle, and is null for a user's first question bundle or lecture. Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback."},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import riiideducation\nimport pandas as pd\nimport numpy as np\nimport tensorflow as tf\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport lightgbm as lgb\nfrom scipy.stats import pearsonr\nfrom lightgbm import LGBMClassifier\nfrom sklearn.metrics import roc_auc_score\nimport pandas_profiling\nfrom sklearn.model_selection import train_test_split, cross_val_score, RepeatedStratifiedKFold\nfrom sklearn.preprocessing import LabelEncoder\nimport plotly_express as px\nfrom collections import Counter\nfrom catboost import CatBoostClassifier\nimport shap\nsns.set_style(style=\"whitegrid\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training data is in the competition dataset as usual\nIt's larger than will fit in memory with default settings, so we'll specify more efficient datatypes and only load a subset of the data for now."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/riiid-test-answer-prediction/train.csv', low_memory=False, nrows=10**5, \n                       dtype={'row_id': 'int64', 'timestamp': 'int64', 'user_id': 'int32', 'content_id': 'int16', 'content_type_id': 'int8',\n                              'task_container_id': 'int16', 'user_answer': 'int8', 'answered_correctly': 'int8', 'prior_question_elapsed_time': 'float32', \n                             'prior_question_had_explanation': 'boolean',\n                             }\n                      )\ntrain_df.drop(\"row_id\", axis=1, inplace=True)\nquestions = pd.read_csv(\"/kaggle/input/riiid-test-answer-prediction/questions.csv\")\nlectures = pd.read_csv(\"/kaggle/input/riiid-test-answer-prediction/lectures.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data Visualization "},{"metadata":{},"cell_type":"markdown","source":"### Training set"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.scatter(train_df[\"prior_question_elapsed_time\"])\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(11, 7), nrows=2, ncols=2)\nsns.countplot(train_df[\"user_answer\"], ax=ax[0,0])\nsns.countplot(train_df[\"answered_correctly\"], ax=ax[0, 1])\nsns.countplot(train_df[\"prior_question_had_explanation\"], ax=ax[1, 0])\nsns.countplot(train_df[\"content_type_id\"],ax=ax[1, 1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.barplot(x=train_df[\"prior_question_had_explanation\"], y=train_df[\"prior_question_elapsed_time\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.barplot(x=train_df[\"prior_question_had_explanation\"], y=train_df[\"answered_correctly\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Questions"},{"metadata":{"trusted":true},"cell_type":"code","source":"questions","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tag = questions[\"tags\"].str.split(\" \", expand = True)\ntag.columns = ['tag1','tag2','tag3','tag4','tag5','tag6']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tag.fillna(0, inplace=True)\ntag = tag.astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"questions['tags'] = questions['tags'].astype(str)\nfrom sklearn.preprocessing import LabelEncoder\nle = LabelEncoder()\nquestions[\"labels\"] = le.fit_transform(questions[\"tags\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"questions","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"questions =  pd.concat([questions,tag],axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"questions","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"questions[\"tag\"] = questions[\"tags\"].astype(str).str.split()\ntags = []\nfor i in questions.tag:\n    for j in i:\n        tags.append(j)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tag_count = Counter(tags)\ntag = list(tag_count.keys())\ncount = list(tag_count.values())\ntag_counts = pd.DataFrame(data={\"tag\":tag, \"count\":count})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.bar(tag_counts, y='tag', x='count', orientation='h', width=800, height=900)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"questions[\"part\"].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.merge(train_df[train_df['content_type_id']==0],\n                              questions, \n                              how='left', \n                              left_on='content_id', \n                              right_on='question_id')\ntrain_df.drop([\"question_id\", \"bundle_id\", \"tags\", \"tag\"], axis=1, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Feature Engineering"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check for missing value\ntrain_df.isna().sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fill the null values in elapsed time column with the mean of elapsed time\ntrain_df[\"prior_question_elapsed_time\"].fillna(train_df[\"prior_question_elapsed_time\"].mean(), inplace=True)\ntrain_df[\"prior_question_elapsed_time\"] = train_df[\"prior_question_elapsed_time\"] / train_df[\"prior_question_elapsed_time\"].mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"prior_question_had_explanation\"] = train_df[\"prior_question_had_explanation\"].fillna(value=False).astype(bool)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# transform last column \ntrain_df[\"prior_question_had_explanation\"] = train_df[\"prior_question_had_explanation\"].map({True:1, False:0})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"part\"] = train_df[\"part\"].map({1:6, 2:25, 3:39, 4:30, 5:30, 6:16, 7:54})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Drop rows with content type id = lecture\ntrain_df.drop(train_df[train_df[\"answered_correctly\"] == -1].index, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"content_id = train_df.groupby('content_id')\ngrouped_answer = content_id.agg({ 'answered_correctly': [np.mean, np.std, np.median, np.cumsum, 'count', \"sum\", \"skew\"]}).copy()\ngrouped_answer.columns = [\"Content_Mean\", \"content_Std\", \"content_Median\", 'content_cumsum', 'question_asked', \"Sum_by_content\", \"skew_content\"]\ngrouped_answer.index.names = ['content_id']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"user_id = train_df.groupby('user_id')\ngrouped_user_id = user_id.agg({'answered_correctly':[np.mean, np.std, np.median, np.cumsum, 'count', \"sum\", \"skew\" ]}).copy()\ngrouped_user_id.columns = [\"user_Mean\", \"user_Std\", \"user_Median\", 'user_cumsum', \"question_answered\", \"Sum_by_user\", \"skew_user\"]\ngrouped_user_id.index.names = ['user_id']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"part_id = train_df.groupby('user_id')\npart_user_id = part_id.agg({\"part\":[np.mean, \"sum\"]}).copy()\npart_user_id.columns = [\"Mean_questions\", \"Total_questions\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"task_container = train_df.groupby(\"task_container_id\")\ntask_id = task_container.agg({'answered_correctly':[np.mean, np.std, np.cumsum, 'count', 'sum', \"skew\"]})\ntask_id.columns = [\"task_Mean\", \"task_std\", 'task_cumsum', 'count_by_task', \"Sum_by_task\", \"skew_task\"]\ntask_id.index.names = ['task_container_id']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = train_df.merge(grouped_answer, how='left', on='content_id')\ntrain_df = train_df.merge(grouped_user_id, how='left', on='user_id')\ntrain_df = train_df.merge(part_user_id, how=\"left\", on=\"user_id\")\ntrain_df = train_df.merge(task_id, how='left', on=\"task_container_id\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"user_correctness\"] = train_df[\"Sum_by_user\"] / train_df[\"question_answered\"]\ntrain_df[\"user_uncorrectness\"] = 1 - train_df[\"user_correctness\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"content_correctness\"] = train_df[\"Sum_by_content\"] / train_df[\"question_asked\"]\ntrain_df[\"content_uncorrectness\"] = 1 - train_df[\"Sum_by_content\"] / train_df[\"question_asked\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"task_correctness\"] = train_df[\"Sum_by_task\"] / train_df[\"count_by_task\"]\ntrain_df[\"task_uncorrectness\"] = 1 - train_df[\"task_correctness\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"timestamp\"] = train_df[\"timestamp\"] / train_df[\"timestamp\"].mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Lag feature"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"lag_1\"] = train_df[\"timestamp\"].shift(1)\ntrain_df[\"lag_2\"] = train_df[\"timestamp\"].shift(2)\ntrain_df[\"lag_3\"] = train_df[\"timestamp\"].shift(3)\ntrain_df[\"lag_4\"] = train_df[\"timestamp\"].shift(4)\ntrain_df[\"lag_5\"] = train_df[\"timestamp\"].shift(5)\ntrain_df[\"lag_6\"] = train_df[\"timestamp\"].shift(6)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from statsmodels.graphics.tsaplots import plot_acf, plot_pacf\nplot_acf(train_df[\"timestamp\"], lags=10)\nplot_pacf(train_df[\"timestamp\"], lags=10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Sliding window"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"rolling_mean\"] = train_df[\"timestamp\"].rolling(window=6).mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df[\"expanding_mean\"] = train_df[\"timestamp\"].expanding(2).mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.fillna(0, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.drop(\"content_type_id\", axis=1, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"features = ['timestamp',     \n            'prior_question_elapsed_time',\n            'Content_Mean',\n            'content_Std',  \n            'question_asked', \n            'Sum_by_content',\n            'skew_content', \n            'user_Mean',\n            'question_answered', \n            'skew_user', \n            'Mean_questions',\n            'task_Mean', \n            'task_std',\n            'skew_task',\n            'content_uncorrectness',\n            'task_uncorrectness','lag_2', \n            'rolling_mean', \n            'expanding_mean',\n            'content_cumsum',\n            'user_cumsum',\n            'task_cumsum']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(features)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(10, 10))\ntrain_corr = train_df[features].corr()\nsns.heatmap(train_corr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Modelling"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.reset_index(drop=True)\nX = train_df.drop(\"answered_correctly\", axis=1)\ny = train_df[\"answered_correctly\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"x_train, x_test, y_train, y_test = train_test_split(X[features], y, test_size=0.3, random_state=42)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Lightgbm"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = lgb.Dataset(x_train, label=y_train)\ntest = lgb.Dataset(x_test, label=y_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"params= {\n    'objective': 'binary',\n    'seed': 42,\n    'metric': 'auc',\n    'learning_rate': 0.001,\n    'max_bin': 1500,\n    'num_leaves': 80 \n    }\n    \nmodel_lgb = lgb.train(\n        params, \n        train, \n        num_boost_round=5000, \n        valid_sets=[train, test], \n        early_stopping_rounds=50, \n        verbose_eval=50,\n        feature_name = features\n        )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"roc_auc_score(y_test, model_lgb.predict(x_test))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lgb.plot_importance(model_lgb)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nexplainer = shap.TreeExplainer(model_lgb)\nshap_values = explainer.shap_values(X[features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"shap.summary_plot(shap_values, X[features])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### LightGBM Classifier"},{"metadata":{"trusted":true},"cell_type":"code","source":"params = {\n    'num_leaves': 10, \n    'n_estimators': 100, \n    'min_data_in_leaf': 10, \n    'max_depth': 5, \n    'lambda': 0.0, \n    'feature_fraction': 1.0\n}\n\nmodel_clf = LGBMClassifier(**params)\nmodel_clf.fit(x_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3)\nscores = cross_val_score(model_clf, X, y, scoring=\"accuracy\", cv=cv)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(scores.mean())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"roc_auc_score(y_test, model_clf.predict_proba(x_test)[:, 1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lgb.plot_importance(model_clf)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"explainer = shap.TreeExplainer(model_clf)\nshap_values = explainer.shap_values(X[features])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"shap.summary_plot(shap_values, X[features])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Hyperparameter Tuning using optuna"},{"metadata":{"trusted":true},"cell_type":"code","source":"import optuna","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# def create_model(trial):\n#     num_leaves = trial.suggest_int('num_leaves', 10, 100)\n#     n_estimators = trial.suggest_int(\"n_estimators\", 100, 3000)\n#     min_data_in_leaf = trial.suggest_int(\"min_data_in_leaf\", 5, 100)\n#     learning_rate = trial.suggest_uniform(\"learning_rate\", 0.0001, 0.99)\n#     bagging_fraction = trial.suggest_uniform(\"bagging_fraction\", 0.0001, 1)\n#     feature_fraction = trial.suggest_uniform(\"feature_fraction\", 0.0001, 1)\n#     max_depth = trial.suggest_int(\"max_depth\", 5, 20)\n    \n#     model = LGBMClassifier(num_leaves=num_leaves,\n#                            n_estimators=n_estimators,\n#                            learning_rate=learning_rate,\n#                            bagging_fraction=bagging_fraction,\n#                            feature_fraction= feature_fraction,\n#                            min_data_in_leaf=min_data_in_leaf,\n#                            max_depth=max_depth)\n    \n#     return model\n\n# def objective(trial):\n#     model = create_model(trial)\n# #     model = lgb.train(\n# #         params, \n# #         train, \n# #         num_boost_round=2500, \n# #         valid_sets=[train, test], \n# #         early_stopping_rounds=20, \n# #         verbose_eval=50,\n# #         feature_name = features\n# #         )\n#     model.fit(x_train, y_train)\n#     y_pred = model.predict_proba(x_test)[:, 1]\n#     score = roc_auc_score(y_test, y_pred)\n#     return score\n\n# study = optuna.create_study(direction=\"maximize\")\n# study.optimize(objective, n_trials=50)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Umcomment for hyper paramterer tuning\n\n# import optuna.integration.lightgbm as lgbm\n# def objective(trial):\n#     param = {\n#         'objective': 'binary',\n#         'metric': 'auc',\n#         'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10.0),\n#         'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10.0),\n#         'num_leaves': trial.suggest_int('num_leaves', 2, 256),\n#         'feature_fraction': trial.suggest_uniform('feature_fraction', 0.4, 1.0),\n#         'bagging_fraction': trial.suggest_uniform('bagging_fraction', 0.4, 1.0),\n#         'bagging_freq': trial.suggest_int('bagging_freq', 1, 7),\n#         'min_child_samples': trial.suggest_int('min_child_samples', 5, 100)\n#     }\n \n#     gbm = lgbm.train(param, train, valid_sets=[train, test] ,early_stopping_rounds=20)\n#     preds = gbm.predict(x_test)\n#     accuracy = roc_auc_score(y_test, preds)\n#     return accuracy\n \n# study = optuna.create_study(direction='maximize')\n# study.optimize(objective, n_trials=5)\n \n# print('Number of finished trials:', len(study.trials))\n# print('Best trial:', study.best_trial.params)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# param = study.best_params\n# param","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Hyperparameters tuned for Lightgbm Classifier"},{"metadata":{"trusted":true},"cell_type":"code","source":"param_lgb = {\n  'objective': 'binary',\n  'seed': 42,\n  'metric': 'auc',\n  'learning_rate': 0.001,\n  'max_bin': 1500,\n 'lambda_l1': 5.793574585607526,\n 'lambda_l2': 0.5835207139734166,\n 'num_leaves': 183,\n 'feature_fraction': 0.6392575149246273,\n 'bagging_fraction': 0.7925551905109522,\n 'bagging_freq': 6,\n 'min_child_samples': 43\n}\n\nmodel_lgb = lgb.train(\n        param_lgb, \n        train, \n        num_boost_round=5000, \n        valid_sets=[train, test], \n        early_stopping_rounds=50, \n        verbose_eval=50,\n        feature_name = features\n        )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lgb.plot_importance(model_lgb)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"params = {'num_leaves': 10,\n 'n_estimators': 437,\n 'min_data_in_leaf': 22,\n 'learning_rate': 0.051498290819129294,\n 'bagging_fraction': 0.1604505638073827,\n 'feature_fraction': 0.8585127657321616,\n 'max_depth': 11}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model_lgbm_clf = LGBMClassifier(**params)\nmodel_lgbm_clf.fit(x_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"roc_auc_score(y_test, model_lgbm_clf.predict_proba(x_test)[:, 1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lgb.plot_importance(model_lgbm_clf)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"env = riiideducation.make_env()\niter_test = env.iter_test()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for (test_df, sample_prediction_df) in iter_test:\n    test_df = pd.merge(test_df[test_df['content_type_id']==0],\n                              questions, \n                              how='left', \n                              left_on='content_id', \n                              right_on='question_id')\n    test_df[\"prior_question_elapsed_time\"].fillna(test_df[\"prior_question_elapsed_time\"].mean(), inplace=True)\n    test_df[\"prior_question_elapsed_time\"] = test_df[\"prior_question_elapsed_time\"] / train_df[\"prior_question_elapsed_time\"].mean()\n    test_df[\"prior_question_had_explanation\"] = test_df[\"prior_question_had_explanation\"].fillna(value=True).astype(bool)\n    test_df[\"prior_question_had_explanation\"] = test_df[\"prior_question_had_explanation\"].map({True:1, False:0})\n    test_df = test_df.merge(grouped_answer, how='left', on='content_id')\n    test_df = test_df.merge(grouped_user_id, how='left', on='user_id')\n    test_df = test_df.merge(part_user_id, how=\"left\", on=\"user_id\")\n    test_df = test_df.merge(task_id, how='left', on=\"task_container_id\")\n    \n    test_df[\"user_correctness\"] = test_df[\"Sum_by_user\"] / test_df[\"question_answered\"]\n    test_df[\"user_uncorrectness\"] = test_df[\"question_answered\"] - test_df[\"Sum_by_user\"]\n    \n    test_df[\"content_correctness\"] = test_df[\"Sum_by_content\"] / test_df[\"question_asked\"]\n    test_df[\"content_uncorrectness\"] = 1 - test_df[\"Sum_by_content\"] / test_df[\"question_asked\"]\n    \n    test_df[\"task_correctness\"] = test_df[\"Sum_by_task\"] / test_df[\"count_by_task\"]\n    test_df[\"task_uncorrectness\"] = 1 - test_df[\"task_correctness\"]\n    \n    # lag feature\n    test_df[\"lag_1\"] = test_df[\"timestamp\"].shift(1)\n    test_df[\"lag_2\"] = test_df[\"timestamp\"].shift(2)\n    test_df[\"lag_3\"] = test_df[\"timestamp\"].shift(3)\n    test_df[\"lag_4\"] = test_df[\"timestamp\"].shift(4)\n    test_df[\"lag_5\"] = test_df[\"timestamp\"].shift(5)\n    test_df[\"lag_6\"] = test_df[\"timestamp\"].shift(6)\n    \n    # Sliding window\n    test_df[\"rolling_mean\"] = test_df[\"timestamp\"].rolling(window=6).mean()\n    test_df[\"expanding_mean\"] = test_df[\"timestamp\"].expanding(2).mean()\n    \n    test_df.fillna(0, inplace=True)\n    \n    test_df['answered_correctly'] = model_lgb.predict(test_df[features])\n    env.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}