{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction / Setup\n\nThis kernel demonstrates online/incremental learning on the Riiid! Answer Correctness Prediction.\n\nIncremental learning is especially beneficial for timeseries, as the model is adapted each time a datapoint is added. Instead of pre-training on a large batch of data and deploying a static model to perform inference for the rest of its life, we can deploy a untrained model and let it train as data flows through it. This makes it easy to modify and limits computing power needed, plus the model is able to adapt to slowly changing data.\n\nWhile this approach will likely result in a lower score than using the entire train set to pre-train a static model (given the same level of feature engineering), it seems like the practically better way to handle this challenge - especially if we consider applicability of the result to the actual data in the competition organizer's system."},{"metadata":{"trusted":true},"cell_type":"code","source":"# import some typical packages and fix random seed\n\nimport numpy as np\nimport pandas as pd\n\nimport time\n\nfrom tqdm.notebook import tqdm\n\nfrom sklearn.metrics import roc_auc_score\n\nRANDOM_SEED = 42\nnp.random.seed(RANDOM_SEED)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We'll need to install creme from its wheel file as internet has to be turned off for submission kernels. Creme is a package explicitly written for online/incremental learning and can be found here:\n\nhttps://creme-ml.github.io/\n\nhttps://github.com/creme-ml/creme/"},{"metadata":{"trusted":true},"cell_type":"code","source":"# install creme from wheel\n!pip install ../input/cremewheels/creme-0.6.1-cp37-cp37m-manylinux1_x86_64.whl","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Defining datatypes for loading train.csv as suggested in the official intro kernel."},{"metadata":{"trusted":true},"cell_type":"code","source":"# datatypes for loading train.csv\ntrain_dtypes = {'row_id': 'int64',\n                'timestamp': 'int64',\n                'user_id': 'int32',\n                'content_id': 'int16',\n                'content_type_id': 'int8',\n                'task_container_id': 'int16',\n                'user_answer': 'int8',\n                'answered_correctly': 'int8',\n                'prior_question_elapsed_time': 'float32',\n                'prior_question_had_explanation': 'boolean'}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Set up data handling, define features\n\nWe want to store some answering history to help with our predictions, so we'll go with two dataframes - one for question-specific statistics and one for user-specific statistics. On the most basic level we want to at least know for each question how often it was attempted and how often it was answered correctly, and for each user how many questions were attempted and answered correctly."},{"metadata":{"trusted":true},"cell_type":"code","source":"# dataframe keeping question-specific statistics and features\nquestions = pd.read_csv('/kaggle/input/riiid-test-answer-prediction/questions.csv')\nquestions.drop(columns=['question_id', 'bundle_id', 'correct_answer', 'tags', 'part'], inplace=True)\nquestions['num_answers'] = 0.\nquestions['num_correct'] = 0.\n\n#----------------------------------------------------------------------------------------------------\n\n# dataframe keeping user-specific statistics and features\nusers = pd.DataFrame(columns = ['num_answers', 'num_correct'], dtype=np.float)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"questions.sample()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"users","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This function allows use to add information to these two dataframes. Given one or more rows from the test or train dataset with known answering outcome, it increases the attempt / correct answers counters both in the <code>questions</code> and <code>users</code> dataframes. We can always decide to store other information later on."},{"metadata":{"trusted":true},"cell_type":"code","source":"def add_data(df):   \n    # ---------------------------------------------------------------------------------------\n    gb = df.groupby(by=['user_id', 'answered_correctly']).count().max(axis=1)\n    # add users which aren't recorded yet\n    for uid in np.setdiff1d(gb.index.levels[0], users.index):\n        users.at[uid] = 0\n    # add question counts [user]\n    for uid in gb.index.levels[0]:\n        users.loc[uid, 'num_answers'] += gb.loc[uid].sum()\n        if 1 in gb[uid].index.get_level_values(0):\n            users.loc[uid, 'num_correct'] += gb.loc[uid].loc[1]\n\n    # ---------------------------------------------------------------------------------------\n    # add question counts [question]\n    gb = df.groupby(by=['content_id']).count().max(axis=1)\n    questions['num_answers'] = questions['num_answers'].add(gb.reindex(questions.index, fill_value=0))\n    \n    gb = df[df['answered_correctly'] == 1].groupby(by=['content_id']).count().max(axis=1)\n    questions['num_correct'] = questions['num_correct'].add(gb.reindex(questions.index, fill_value=0))\n    return","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This function generates features using the dataframe given to it (train or test) and the current status of the <code>users</code> and <code>questions</code> tables. It returns a list of the label names and a feature array for the given batch."},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_features(df):\n    uid_list = df['user_id'].values\n    qid_list = df['content_id'].values\n\n    for uid in np.setdiff1d(np.unique(uid_list), users.index):\n        users.at[uid] = 0\n    q_selec = questions.reindex(df['content_id'].values).copy()\n    u_selec = users.reindex(df['user_id'].values).copy()\n\n    # create features\n    df['mean_q_correctness'] = q_selec['num_correct'].divide(q_selec['num_answers']).values\n    df['mean_u_correctness'] = u_selec['num_correct'].divide(u_selec['num_answers']).values\n    \n    # fill invalid values\n    fill_dict = {\n        'mean_q_correctness': 0.5,\n        'mean_u_correctness': 0.5,    \n    }\n    for col in [*fill_dict]:\n        df[col].fillna(value=fill_dict[col], inplace=True)\n        \n    # return only columns we want to use for predicting\n    use_cols = ['mean_q_correctness', 'mean_u_correctness']\n            \n    return use_cols, df[use_cols].values.astype(np.float)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Define models\n\nNow that the data handling is set up, let's have a look at models which we want to use for prediction. Here we'll use a Pytorch multilayer perceptron (MLP) and a model using the creme package. We can quickly add more models without having to do any noticeable modifications in the code further below as long as we stick to a certain format.\n\nHere I built each model to have three utility functions:\n* <code>predict(X)</code> uses the model in its current state to predict an output <code>y_pred</code> for the input X\n* <code>fit(X, y)</code> fits a new batch of data to the model\n* <code>iterate_batch(X, y)</code> is simply a combination of the two, first obtaining a prediction for <code>X</code> before updating the model using the true <code>y</code> values given and returning the predicted output <code>y_pred</code>.\n\nThe first two functions are used for test data, where we first need to predict on a dataset and submit the prediction before we obtain the true outcome with which we can update our models. For train data we can simply use <code>iterate_batch()</code> to perform both steps in one."},{"metadata":{},"cell_type":"markdown","source":"## Pytorch MLP\n\nSimple MLP with three hidden layers and ReLu activations"},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom torch.utils.data import Dataset\nfrom torch.utils.data import DataLoader\n\nfrom torch.optim import SGD, Adam\n\ntorch.manual_seed(RANDOM_SEED)\n\ndevice = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n\nclass RiiidNet(nn.Module):\n    \n    def __init__(self, NFEATURES, NHIDDEN):\n        super(RiiidNet, self).__init__()\n        self.NFEATURES = NFEATURES\n        self.fc1 = nn.Linear(NFEATURES, NHIDDEN)\n        self.fc2 = nn.Linear(NHIDDEN, NHIDDEN)\n        self.fc3 = nn.Linear(NHIDDEN, NHIDDEN)\n        self.fc4 = nn.Linear(NHIDDEN, NHIDDEN)\n        self.fc5 = nn.Linear(NHIDDEN, 2)\n\n    def forward(self, x):\n        x = x.view(-1, self.NFEATURES)\n        x = self.fc1(x)\n        x = F.relu(x)\n        x = self.fc2(x)\n        x = F.relu(x)\n        x = self.fc3(x)\n        x = F.relu(x)\n        x = self.fc4(x)\n        x = F.relu(x)\n        x = self.fc5(x)\n        x = torch.softmax(x, dim=1)\n        return x\n    \nclass MLPModel():\n    \n    def __init__(self, NFEATURES, NHIDDEN, device):\n        self.model = RiiidNet(NFEATURES, NHIDDEN).to(device)\n        self.optimiser = Adam(self.model.parameters(), lr=0.002)\n        self.device = device\n        \n    def predict(self, X):\n        with torch.no_grad():\n            # prepare data\n            X = torch.tensor(X, dtype=torch.float).to(self.device)\n            # forward pass\n            output = self.model(X)\n        return output[:,1].numpy()\n    \n    def fit(self, X, y):\n        self.iterate_batch(X, y)\n        return\n    \n    def iterate_batch(self, X, y):\n        # prepare data\n        X = torch.tensor(X, dtype=torch.float).to(self.device)\n        y = torch.tensor(y, dtype=torch.float).to(self.device)\n        y_full = torch.zeros(len(y), 2).to(self.device)\n        y_full[np.where(y.numpy()),1] = 1\n\n        # zero gradients\n        self.model.zero_grad()\n        # forward pass\n        output = self.model(X)\n\n        # calculate loss\n        loss = F.binary_cross_entropy(output, y_full)\n        # backward pass\n        loss.backward()\n        # update parameters\n        self.optimiser.step()\n        \n        return output[:,1].detach().numpy()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Creme\n\nHere we use the logistic regression model of the creme package. Predictions and model updates can be performed either one by one (<code>predict_proba_one</code> and <code>fit_one</code>) or in batches (<code>predict_proba_many</code> and <code>fit_many</code>). However, when I tried with the <code>_many</code> functions my ROC area under curve scores nosedived quite badly - so I'm not using this for now (might not be working perfectly yet, creme is pretty new) but keep the code for demonstration / completeness."},{"metadata":{"trusted":true},"cell_type":"code","source":"from creme import linear_model\nfrom creme import optim\n\nclass CremeModel():\n    \n    def __init__(self):\n        self.optimizer = optim.SGD(lr=0.01)\n        self.model = linear_model.LogisticRegression(self.optimizer)\n             \n    def predict(self, X, many=False):\n        if many:\n            # predict using dataframe\n            X_df = pd.DataFrame(data=X)\n            y_pred = self.model.predict_proba_many(X_df).loc[:,True].values\n        else:\n            # allocate prediction list\n            y_pred = np.zeros((X.shape[0]))\n            # make numeric keys for dict\n            tmp_names = [str(i) for i in range(X.shape[1])]\n            for i in range(len(y_pred)):\n                # create data dict\n                X_dict = dict(zip(tmp_names, list(X[i])))\n                # predict\n                y_pred[i] = self.model.predict_proba_one(X_dict)[True]\n        return y_pred\n    \n    def fit(self, X, y, many=False):\n        if many:\n            # prepare data\n            X_df = pd.DataFrame(data=X)\n            y_df = pd.Series(data=y)\n            # train / update\n            self.model.fit_many(X_df, y_df)\n        else:\n            # make numeric keys for dict\n            tmp_names = [str(i) for i in range(X.shape[1])]\n            for i in range(len(y)):\n                # create data dict\n                X_dict = dict(zip(tmp_names, list(X[i])))\n                # train / update\n                self.model.fit_one(X_dict, y[i])\n        return \n    \n    def iterate_batch(self, X, y, many=False):\n        y_pred = self.predict(X)\n        self.fit(X,y)\n        return y_pred","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Perform incremental training\n\nNow we just set up these models and feed some batches of train data through it. I'm stopping after 300 batches of 10,000 rows to keep the waiting time down."},{"metadata":{"trusted":true},"cell_type":"code","source":"mlp = MLPModel(2, 32, device)\nlrg = CremeModel()\n\nmodels = {\n    'MLP': mlp,\n    'LRG': lrg,\n}\n\nmetrics = {}\nfor tag in [*models] + ['mean']:\n    metrics[tag] = {}\n    metrics[tag]['accuracy'] = []\n    metrics[tag]['roc_auc'] = []\n\ntrainfile = '/kaggle/input/riiid-test-answer-prediction/train.csv'\n\nfor n, train_part in enumerate(pd.read_csv(trainfile, chunksize=10**4, dtype=train_dtypes, iterator=True)):\n    # discard lecture rows\n    train_part = train_part[train_part['content_type_id']==0]\n    \n    # get features\n    label_names, X = get_features(train_part.copy())\n    # get targets\n    y = train_part['answered_correctly'].values.astype(np.float)\n    \n    # iterate all models\n    preds = {}\n    for tag in [*models]:\n        preds[tag] = models[tag].iterate_batch(X, y)\n\n    # get mean of all predictions\n    preds['mean'] = np.mean([preds[tag] for tag in [*models]], axis=0)\n    \n    # calculate metrics\n    for tag in [*models] + ['mean']:\n        acc = ((preds[tag] > 0.5).astype(float) == y).sum() / len(y)\n        roc_auc = roc_auc_score(y, preds[tag])\n        metrics[tag]['accuracy'].append(acc)\n        metrics[tag]['roc_auc'].append(roc_auc)\n        \n        if tag == 'mean':\n            print('Chunk: {} \\t Accuracy: {:0.3f} \\t ROC AUC: {:0.3f}'.format(\n                n, acc, roc_auc))\n            \n    add_data(train_part.copy())\n    \n    if n > 500:\n        break","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here come plots of accuracy and score..."},{"metadata":{"trusted":true},"cell_type":"code","source":"from scipy.signal import convolve\n\ndef graphfilter(x, N=3):\n    filt = np.ones((N)) / N\n    x_full = np.concatenate([\n        np.full((int((N-1)/2)), x[0]),\n        x,\n        np.full((int((N-1)/2)), x[-1])])\n    return convolve(x, filt, mode='valid')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nplt.figure(figsize=(9,6))\nplt.plot(graphfilter(metrics['mean']['accuracy']), label='Accuracy')\nplt.plot(graphfilter(metrics['mean']['roc_auc']), label='ROC AUC')\nplt.xlabel('Chunk number')\nplt.ylabel('Accuracy / ROC AUC')\nplt.ylim([0.4,0.9])\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nplt.figure(figsize=(9,6))\nfor tag in [*models] + ['mean']:\n    plt.plot(graphfilter(metrics[tag]['roc_auc']), label=tag)\nplt.xlabel('Chunk number')\nplt.ylabel('ROC AUC')\nplt.ylim([0.4,0.9])\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And the mean ROC area under curve from the train data. Note that while this is train data, this score is calculated entirely on unseen data - each batch is predicted before the model is updated with it."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Mean ROC AUC:', np.mean(metrics['mean']['roc_auc']))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Test evaluation\n\nFor completeness' sake, we feed the test data through as well."},{"metadata":{"trusted":true},"cell_type":"code","source":"import riiideducation\nenv = riiideducation.make_env()\niter_test = env.iter_test()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"prev_test = None\ncount = 0\n\nfor (test_df, sample_prediction_df) in iter_test:\n    \n    if (not prev_test is None):\n        # take answers returned and update data with previous test set\n        prev_answers = np.array(eval(test_df.iloc[0]['prior_group_answers_correct']), dtype=np.int)\n        prev_test['answered_correctly'] = prev_answers\n        prev_test = prev_test[prev_test['content_type_id'] == 0]\n        add_data(prev_test.copy())\n\n        # iterate models with previous data\n        for tag in [*models]:\n            models[tag].fit(X, prev_test['answered_correctly'].values)\n\n        # calculate metrics if possible\n        try:\n            acc = ((preds['mean'] > 0.5).astype(float) == prev_answers).sum() / len(prev_answers)\n            roc_auc = roc_auc_score(prev_answers, preds[tag])\n            print('Group: {} \\t Accuracy: {:0.3f} \\t ROC AUC: {:0.3f}'.format(count, acc, roc_auc))\n        except:\n            print('Group: {} \\t Could not calculate metrics'.format(count))\n    \n    # copy new test_df into prev_test\n    prev_test = test_df.copy()\n    \n    # remove lecture rows in test data\n    test_df = test_df[test_df['content_type_id'] == 0]\n    \n    # get features\n    label_names, X = get_features(test_df.copy())\n    \n    # predict all models\n    preds = {}\n    for tag in [*models]:\n        preds[tag] = models[tag].predict(X)\n\n    # get mean of all predictions\n    preds['mean'] = np.mean([preds[tag] for tag in [*models]], axis=0)\n    \n    # submit predictions\n    test_df['answered_correctly'] = preds['mean']\n    env.predict(test_df.loc[:,['row_id', 'answered_correctly']])\n    \n    count += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Feel free to comment if you have any questions/suggestions and/or leave an upvote!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}