{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Prediction Notebook\n\n## This notebook is a work in progress. I'll update it when I'll get some free time. If you are intereseted in something in particular leave a comment and I'll try to answer as soon as possible.\n\n\nIn this notebook I use the models trained to make inference on the test set.\n\nMy last submission was made of 3 LGBM classifier one for each interaction type.\n\nThe prediction on the test set would take too much memory so the prediction is done in 10 folds.\n\nThe LGBM classifiers combine the score of many models and features to make predictions.\n\nThe model I use uses candidates from 3 groups of models:\nThe models used in the first group are the 3 candidate selectors from an improved version of Co-visitation matrix starting from this notebook [notebook](https://www.kaggle.com/code/carnozhao/otto-fast-cpu-end-to-end-pipeline) by [carnozhao](https://www.kaggle.com/carnozhao), a tuned version of Matrix Factorization from the [notebook](https://www.kaggle.com/code/cpmpml/matrix-factorization-with-gpu) by [cpmpml](https://www.kaggle.com/cpmpml), a slightly changed version of W2V model form the [notebook](https://www.kaggle.com/code/radek1/word2vec-how-to-training-and-submission) by [radek1](https://www.kaggle.com/radek1). In addition to these models I tried also to reverse the order of each session for Co-visitation.\nThe second group uses a set of candidate selector based purely on the history of each session, to each item we give a weight and we sum the weight for each item in the session, I used different weighting for each type of interaction but also a weighting based on the temporal distance from the last seen item in the session.\nThe last group in the end was only a single model using RecBole, in particular I used SRGNN model using the code from this [notebook](https://www.kaggle.com/code/yamsam/recbole-gru4rec-sample-code) by [yamsam](https://www.kaggle.com/yamsam) as a starting point.\n\nThe features used are fairly easy ones, for each candidate how many times it already appeared in the session for each type of interaction and the sum of this 3 counts, the length of the session (number of items) and the temporal length of the session (just now I'm thinking that I forgot to also add the count of unique items seen in the session).\n\nOther features I used are Target Encodings trained on the week before the test set and target encoding trained on the test week ( for training the models I used the Target encodings calculated on the week before the validation week and encoding calculated on the validation week itself).\n\n# Everything has been trained on kaggle!!!\nI have over 50 Notebooks combining model validation, model training and inference. \n\nI struggled a lot for some parts, an example is SRGNN: Inference would take more than 24h on CPU and 10h on GPU. My solution was training on GPU (2h) saving the model and making predictions on 7 different notebooks each for a split of the dataset, in this way the time to get the result was around 4-5h and the GPU time used was minimal.\n\nI'll try polishing as much as I can and share to the community as many notebook as possible.\n\nThis was a hard but fun competition, I hope everyone enjoyed it. \n\nLet's keep kaggling!","metadata":{}},{"cell_type":"code","source":"splits=10\nDEBUG = False","metadata":{"execution":{"iopub.status.busy":"2023-01-27T07:56:20.133437Z","iopub.execute_input":"2023-01-27T07:56:20.134711Z","iopub.status.idle":"2023-01-27T07:56:36.048679Z","shell.execute_reply.started":"2023-01-27T07:56:20.134575Z","shell.execute_reply":"2023-01-27T07:56:36.047134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd \n\nfrom functools import reduce\nimport os\nimport gc\nfrom sklearn.model_selection import train_test_split\nimport lightgbm\nimport logging\nimport sys\n\n\nfrom tqdm.auto import tqdm\n\nFORMAT = '%(asctime)s - %(levelname)s - %(message)s'\n\n\ntry:\n logger\nexcept:\n logger=logging.getLogger(\"experiment_logger\")\n formatter = logging.Formatter(FORMAT)\n consoleHandler = logging.StreamHandler(sys.stdout)\n consoleHandler.setFormatter(formatter)\n logger.addHandler(consoleHandler)\n logger.setLevel(logging.DEBUG)\n logger.debug(\"debugging\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-01-27T07:56:36.050871Z","iopub.execute_input":"2023-01-27T07:56:36.051777Z","iopub.status.idle":"2023-01-27T07:56:37.794159Z","shell.execute_reply.started":"2023-01-27T07:56:36.051733Z","shell.execute_reply":"2023-01-27T07:56:37.792792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_sessions_all=pd.read_parquet(\"/kaggle/input/otto-full-optimized-memory-footprint/test.parquet\")\nsessions_full=test_sessions_all[\"session\"].drop_duplicates().reset_index(drop=True)\nif DEBUG:\n sessions_full=sessions_full[-10:]","metadata":{"execution":{"iopub.status.busy":"2023-01-27T07:56:37.795867Z","iopub.execute_input":"2023-01-27T07:56:37.796235Z","iopub.status.idle":"2023-01-27T07:56:38.886232Z","shell.execute_reply.started":"2023-01-27T07:56:37.796201Z","shell.execute_reply":"2023-01-27T07:56:38.885061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Combine score of candidates for each candidate selector with various features\ndef prepare_split(split,splits):\n assert split