{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This notebook has been copied from [Isamu's original.](https://www.kaggle.com/code/yamsam/recbole-gru4rec-sample-code) I have added additional notes to enhance my own understanding of the Recbole library.\n\nRecBole is a unified, comprehensive and efficient framework developed based on PyTorch. It aims to help the researchers to reproduce and develop recommendation models.\n\n[Source](https://recbole.io/docs/index.html)","metadata":{}},{"cell_type":"code","source":"!pip install polars","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-01-16T20:36:29.627842Z","iopub.execute_input":"2023-01-16T20:36:29.628315Z","iopub.status.idle":"2023-01-16T20:36:43.693546Z","shell.execute_reply.started":"2023-01-16T20:36:29.628273Z","shell.execute_reply":"2023-01-16T20:36:43.692254Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install recbole","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-01-16T20:36:43.696180Z","iopub.execute_input":"2023-01-16T20:36:43.697320Z","iopub.status.idle":"2023-01-16T20:36:55.164904Z","shell.execute_reply.started":"2023-01-16T20:36:43.697273Z","shell.execute_reply":"2023-01-16T20:36:55.163637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tqdm\nimport polars as pl\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport random\nimport os \nimport h5py\nimport sys\nimport gc\nfrom collections import defaultdict\nfrom matplotlib import pyplot as plt\nimport pyarrow.parquet as pq","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:36:55.168838Z","iopub.execute_input":"2023-01-16T20:36:55.169172Z","iopub.status.idle":"2023-01-16T20:36:56.008893Z","shell.execute_reply.started":"2023-01-16T20:36:55.169137Z","shell.execute_reply":"2023-01-16T20:36:56.007761Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 1. Create atomic file","metadata":{}},{"cell_type":"markdown","source":"In order to characterize most forms of the input data required by different recommendation tasks, RecBole designs an input data format called Atomic Files. All the input data should be converted into Atomic Files format. Besides, we design a data structure called Interaction to provides a unified internal data representation for different recommendation algorithms.\n\n[Source](https://recbole.io/docs/user_guide/data_intro.html)\n\nEach atomic file can be viewed as a m x n table, where n is the number of features and m-1 is the number of data records(one line for header).\n\n[Source](https://recbole.io/docs/user_guide/data/atomic_files.html)","metadata":{}},{"cell_type":"code","source":"train = pl.read_parquet('/kaggle/input/otto-train-and-test-data-for-local-validation/test.parquet')\ntest = pl.read_parquet('/kaggle/input/otto-full-optimized-memory-footprint/test.parquet')\nlen(train)","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:36:56.011646Z","iopub.execute_input":"2023-01-16T20:36:56.012083Z","iopub.status.idle":"2023-01-16T20:36:57.341150Z","shell.execute_reply.started":"2023-01-16T20:36:56.012020Z","shell.execute_reply":"2023-01-16T20:36:57.339974Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pl.concat([train, test])\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:37:08.661227Z","iopub.execute_input":"2023-01-16T20:37:08.662234Z","iopub.status.idle":"2023-01-16T20:37:08.826741Z","shell.execute_reply.started":"2023-01-16T20:37:08.662190Z","shell.execute_reply":"2023-01-16T20:37:08.825453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df.sort(['session', 'aid', 'ts'])\ndf = df.rename({'session': 'session:token', 'aid': 'aid:token', 'ts': 'ts:float'})","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:37:09.702616Z","iopub.execute_input":"2023-01-16T20:37:09.702998Z","iopub.status.idle":"2023-01-16T20:37:11.183085Z","shell.execute_reply.started":"2023-01-16T20:37:09.702963Z","shell.execute_reply":"2023-01-16T20:37:11.181980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The overall data flow is described below. Below, we save the data as atomic files in a working directory.","metadata":{}},{"cell_type":"markdown","source":"![Image](https://recbole.io/docs/_images/data_flow_en.png)","metadata":{}},{"cell_type":"code","source":"!mkdir /kaggle/working/recbox_data\ndf['session:token', 'aid:token', 'ts:float'].write_csv('/kaggle/working/recbox_data/recbox_data.inter', sep='\\t')\n\ndel df, train, test\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:37:14.543353Z","iopub.execute_input":"2023-01-16T20:37:14.544311Z","iopub.status.idle":"2023-01-16T20:37:19.268669Z","shell.execute_reply.started":"2023-01-16T20:37:14.544274Z","shell.execute_reply":"2023-01-16T20:37:19.267655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Create dataset and configuration to train the model\n\nRecBole is able to config different parameters for controlling the experiment setup (e.g., data processing, data splitting, training and evaluation). The users can select the settings according to their own requirements.The users can write their parameters into a dict, and input it into the configuration module.\n\n[Source](https://recbole.io/docs/user_guide/config_settings.html)","metadata":{}},{"cell_type":"code","source":"import logging\nfrom logging import getLogger\nfrom recbole.config import Config\nfrom recbole.data import create_dataset, data_preparation\nfrom recbole.model.sequential_recommender import GRU4Rec\nfrom recbole.trainer import Trainer\nfrom recbole.utils import init_seed, init_logger\n\nfrom recbole.utils.case_study import full_sort_topk","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:38:18.625417Z","iopub.execute_input":"2023-01-16T20:38:18.626122Z","iopub.status.idle":"2023-01-16T20:38:21.079388Z","shell.execute_reply.started":"2023-01-16T20:38:18.626084Z","shell.execute_reply":"2023-01-16T20:38:21.078430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MAX_ITEM = 20\n\nparameter_dict = {\n    'data_path': '/kaggle/working/',# The path of input dataset.\n    'USER_ID_FIELD': 'session',\n    'ITEM_ID_FIELD': 'aid',\n    'TIME_FIELD': 'ts',\n    'user_inter_num_interval': \"[5,Inf)\",# Users whose number of interactions is in the interval will be retained.\n    'item_inter_num_interval': \"[5,Inf)\",# Items whose number of interactions is in the interval will be retained.\n    'load_col': {'inter': ['session', 'aid', 'ts']},\n    'train_neg_sample_args': None,\n    'epochs': 1,# Defaults to 300\n    'stopping_step':3,# The threshold for validation-based early stopping. Defaults to 10.\n    'eval_batch_size': 1024,# Defaults to 4096.\n    #'train_batch_size': 1024,\n    #'enable_amp':True,\n    'MAX_ITEM_LIST_LENGTH': MAX_ITEM,# Maximum length of each generated sequence. Defaults to 50.\n    'eval_args': {\n        'split': {'RS': [9, 1, 0]},\n        'group_by': 'user', # the data will be grouped by the column of USER_ID_FIELD and split in user dimension. \n        'order': 'TO', # sort the data by the column of TIME_FIELD in ascending order and the split them in this order.\n        'mode': 'full' # full means evaluating the model on the set of all items.\n                }\n            }","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:38:22.126959Z","iopub.execute_input":"2023-01-16T20:38:22.128216Z","iopub.status.idle":"2023-01-16T20:38:22.135255Z","shell.execute_reply.started":"2023-01-16T20:38:22.128176Z","shell.execute_reply":"2023-01-16T20:38:22.134142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Below code is directly copied from the [usage documentation.](https://recbole.io/docs/user_guide/usage/use_modules.html)","metadata":{}},{"cell_type":"code","source":"config = Config(model='GRU4Rec', dataset='recbox_data', config_dict=parameter_dict)\n\n# init random seed\ninit_seed(config['seed'], config['reproducibility'])\n\n# logger initialization\ninit_logger(config)\nlogger = getLogger()\n\n# Create handlers\nc_handler = logging.StreamHandler()\nc_handler.setLevel(logging.INFO)\nlogger.addHandler(c_handler)\n\n# write config info into log\nlogger.info(config)","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:55:20.632039Z","iopub.execute_input":"2023-01-16T20:55:20.632475Z","iopub.status.idle":"2023-01-16T20:55:21.094121Z","shell.execute_reply.started":"2023-01-16T20:55:20.632437Z","shell.execute_reply":"2023-01-16T20:55:21.093218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Filtering the data files according to the parameters indicated in the configuration.\ndataset = create_dataset(config)\nlogger.info(dataset)","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:55:28.151064Z","iopub.execute_input":"2023-01-16T20:55:28.151479Z","iopub.status.idle":"2023-01-16T20:58:21.166543Z","shell.execute_reply.started":"2023-01-16T20:55:28.151442Z","shell.execute_reply":"2023-01-16T20:58:21.165641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# dataset splitting\ntrain_data, valid_data, test_data = data_preparation(config, dataset)","metadata":{"execution":{"iopub.status.busy":"2023-01-16T20:41:14.254703Z","iopub.execute_input":"2023-01-16T20:41:14.255151Z","iopub.status.idle":"2023-01-16T20:43:14.490394Z","shell.execute_reply.started":"2023-01-16T20:41:14.255109Z","shell.execute_reply":"2023-01-16T20:43:14.489017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n# model loading and initialization\nmodel = GRU4Rec(config, train_data.dataset).to(config['device'])\nlogger.info(model)\n\n# trainer loading and initialization\ntrainer = Trainer(config, model)\n\n# model training\nbest_valid_score, best_valid_result = trainer.fit(train_data, valid_data)\n#best_valid_score, best_valid_result = trainer.fit(train_data)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-01-16T21:00:24.526029Z","iopub.execute_input":"2023-01-16T21:00:24.526485Z","iopub.status.idle":"2023-01-16T21:15:31.976524Z","shell.execute_reply.started":"2023-01-16T21:00:24.526447Z","shell.execute_reply":"2023-01-16T21:15:31.975571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import gc\ndel trainer, train_data, valid_data, test_data","metadata":{"execution":{"iopub.status.busy":"2023-01-16T21:17:07.411245Z","iopub.execute_input":"2023-01-16T21:17:07.411906Z","iopub.status.idle":"2023-01-16T21:17:07.416626Z","shell.execute_reply.started":"2023-01-16T21:17:07.411868Z","shell.execute_reply":"2023-01-16T21:17:07.415596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-16T21:17:08.229354Z","iopub.execute_input":"2023-01-16T21:17:08.230061Z","iopub.status.idle":"2023-01-16T21:17:09.056615Z","shell.execute_reply.started":"2023-01-16T21:17:08.230026Z","shell.execute_reply":"2023-01-16T21:17:09.055461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Create recommendation result from trained model\n\nThe original notebook took the code from [this article.](https://qiita.com/fufufukakaka/items/e03df3a7299b2b8f99cf)","metadata":{}},{"cell_type":"code","source":"from typing import List, Tuple\nimport numpy as np\nimport torch\n\nfrom pydantic import BaseModel\nfrom recbole.data import create_dataset\nfrom recbole.data.dataset.sequential_dataset import SequentialDataset\nfrom recbole.data.interaction import Interaction\nfrom recbole.model.sequential_recommender.sine import SINE\nfrom recbole.utils import get_model, init_seed","metadata":{"execution":{"iopub.status.busy":"2023-01-16T21:17:11.276311Z","iopub.execute_input":"2023-01-16T21:17:11.276749Z","iopub.status.idle":"2023-01-16T21:17:11.421038Z","shell.execute_reply.started":"2023-01-16T21:17:11.276707Z","shell.execute_reply":"2023-01-16T21:17:11.419938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ItemHistory(BaseModel):\n    sequence: List[str]\n    topk: int\n\nclass RecommendedItems(BaseModel):\n    score_list: List[float]\n    item_list: List[str]\n\n\ndef pred_user_to_item(item_history: ItemHistory):\n    item_history_dict = item_history.dict()\n    item_sequence = item_history_dict[\"sequence\"]\n    item_length = len(item_sequence)\n    pad_length = MAX_ITEM  # pre-defined by recbole\n    \n    '''\n    First, we need to use token2id() to convert external user id \n    into internal user id.\n    Then, we create a 0 padded tensor to pass into the interaction object. \n    The number of 0s depends on the length of the original item list. \n    If there are 4 items, then its padded with 16 0s so that the total \n    length is 20, which is what we want to predict.\n    '''\n    padded_item_sequence = torch.nn.functional.pad(\n        torch.tensor(dataset.token2id(dataset.iid_field, item_sequence)),\n        (0, pad_length - item_length),\n        \"constant\",\n        0,\n    )\n    \n    '''To perform prediction, we need to create the sequence in this\n    interaction object.'''\n    \n    input_interaction = Interaction(\n        {\n            \"aid_list\": padded_item_sequence.reshape(1, -1),\n            \"item_length\": torch.tensor([item_length]),\n        }\n    )\n    \n    '''\n    In full_sort_predict, first we pass the sequence forward in the model to get the next article.\n    This forward pass gives us an embedding. We multiple this embedding with the embedding space \n    learnt by the model. This matrix multiplication gives us a single score for each item. The higher \n    the score, the closer that article is to the predicted embedding. \n    '''\n    scores = model.full_sort_predict(input_interaction.to(model.device))\n    scores = scores.view(-1, dataset.item_num)\n    scores[:, 0] = -np.inf  # pad item score -> -inf\n    \n    '''Top 20 scores and items are selected using torch.topk.'''\n    topk_score, topk_iid_list = torch.topk(scores, item_history_dict[\"topk\"])\n    predicted_score_list = topk_score.tolist()[0]\n    \n    '''Predicted items need to be translated back into original article IDs \n    using dataset.id2token.'''\n    predicted_item_list = dataset.id2token(\n        dataset.iid_field, topk_iid_list.tolist()\n    ).tolist()\n\n    recommended_items = {\n        \"score_list\": predicted_score_list,\n        \"item_list\": predicted_item_list,\n    }\n    return recommended_items","metadata":{"execution":{"iopub.status.busy":"2023-01-16T21:17:15.468010Z","iopub.execute_input":"2023-01-16T21:17:15.469180Z","iopub.status.idle":"2023-01-16T21:17:15.486532Z","shell.execute_reply.started":"2023-01-16T21:17:15.469131Z","shell.execute_reply":"2023-01-16T21:17:15.485364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test = pl.read_parquet('../input/otto-full-optimized-memory-footprint/test.parquet')\nsession_types = ['clicks', 'carts', 'orders']\ntest_session_AIDs = test.to_pandas().reset_index(drop=True).groupby('session')['aid'].apply(list)\ntest_session_types = test.to_pandas().reset_index(drop=True).groupby('session')['type'].apply(list)\n\ndel test\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-16T21:17:26.644131Z","iopub.execute_input":"2023-01-16T21:17:26.644511Z","iopub.status.idle":"2023-01-16T21:18:26.377215Z","shell.execute_reply.started":"2023-01-16T21:17:26.644478Z","shell.execute_reply":"2023-01-16T21:18:26.375682Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels = []\n\ntype_weight_multipliers = {0: 1, 1: 6, 2: 3}\ni = 0\nfor AIDs, types in zip(test_session_AIDs, test_session_types):\n    if len(AIDs) >= 20:\n        # if we have enough aids (over equals 20) we don't need to look for candidates! we just use the old logic\n        weights=np.logspace(0.1,1,len(AIDs),base=2, endpoint=True)-1\n        aids_temp=defaultdict(lambda: 0)\n        for aid,w,t in zip(AIDs,weights,types): \n            aids_temp[aid]+= w * type_weight_multipliers[t]\n            \n        sorted_aids=[k for k, v in sorted(aids_temp.items(), key=lambda item: -item[1])]\n        labels.append(sorted_aids[:20])\n    else:\n        AIDs = list(dict.fromkeys(AIDs))\n        item = ItemHistory(sequence=AIDs, topk=20)\n        try:\n            nns = [ int(v) for v in pred_user_to_item(item)['item_list']]\n        except:\n            nns = []\n\n        for word in nns:\n            if len(AIDs) == 20:\n                break\n            if int(word) not in AIDs:\n                AIDs.append(word)\n\n        labels.append(AIDs[:20])\n    i += 1\n    if i%50000==0:\n        print (i)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-01-12T17:29:05.129186Z","iopub.execute_input":"2023-01-12T17:29:05.129803Z","iopub.status.idle":"2023-01-12T20:29:36.548945Z","shell.execute_reply.started":"2023-01-12T17:29:05.129764Z","shell.execute_reply":"2023-01-12T20:29:36.548012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_user_to_item(item)['item_list']","metadata":{"execution":{"iopub.status.busy":"2023-01-12T20:29:36.550333Z","iopub.execute_input":"2023-01-12T20:29:36.550681Z","iopub.status.idle":"2023-01-12T20:29:36.570245Z","shell.execute_reply.started":"2023-01-12T20:29:36.550647Z","shell.execute_reply":"2023-01-12T20:29:36.569228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_as_strings = [' '.join([str(l) for l in lls]) for lls in labels]\npredictions = pd.DataFrame(data={'session_type': test_session_AIDs.index, 'labels': labels_as_strings})","metadata":{"execution":{"iopub.status.busy":"2023-01-12T20:29:36.571530Z","iopub.execute_input":"2023-01-12T20:29:36.571986Z","iopub.status.idle":"2023-01-12T20:29:43.805813Z","shell.execute_reply.started":"2023-01-12T20:29:36.571934Z","shell.execute_reply":"2023-01-12T20:29:43.804850Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_as_strings = [' '.join([str(l) for l in lls]) for lls in labels]\n\npredictions = pd.DataFrame(data={'session_type': test_session_AIDs.index, 'labels': labels_as_strings})\n\nprediction_dfs = []\n\nfor st in session_types:\n    modified_predictions = predictions.copy()\n    modified_predictions.session_type = modified_predictions.session_type.astype('str') + f'_{st}'\n    prediction_dfs.append(modified_predictions)\n\nsubmission = pd.concat(prediction_dfs).reset_index(drop=True)\nsubmission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T20:29:43.807323Z","iopub.execute_input":"2023-01-12T20:29:43.807687Z","iopub.status.idle":"2023-01-12T20:30:07.484948Z","shell.execute_reply.started":"2023-01-12T20:29:43.807640Z","shell.execute_reply":"2023-01-12T20:30:07.484022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}