{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Implement Baseline with Most Popular Items and Local Validation template\n\nThis notebook implements a very naive baseline with a solid local validation strategy. My goal was not to achieve high accuracy, but to provide a template to have a good starting point with best practices for your further experiments. So all you need to do is to copy & edit this notebook and start kaggling!\n\nMain idea of the baseline is to use most frequently bought items over a fixed window prior to the test set. I show how to optimize for the best window length, using cross-validation in a time-series manner. This provides an example of implementing local validation strategy with statistically significant results, together with the evaluation metric used for this competition. \n\nIt also implements:\n* [trick to reduce transactions dataframe's overall memory](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308635)\n* [solid local validation strategy](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308919)\n* [evaluation metric MAP@K](https://raw.githubusercontent.com/benhamner/Metrics/master/Python/ml_metrics/average_precision.py)","metadata":{}},{"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport gc\nimport random\n\nfrom pathlib import Path\ndata_path = Path('/kaggle/input/h-and-m-personalized-fashion-recommendations/')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-03-25T09:20:43.674273Z","iopub.execute_input":"2022-03-25T09:20:43.674794Z","iopub.status.idle":"2022-03-25T09:20:43.680212Z","shell.execute_reply.started":"2022-03-25T09:20:43.674741Z","shell.execute_reply":"2022-03-25T09:20:43.67942Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Read input data","metadata":{}},{"cell_type":"code","source":"transactions = pd.read_csv(\n    data_path / 'transactions_train.csv',\n    # set dtype or pandas will drop the leading '0' and convert to int\n    dtype={'article_id': str} \n)\n\ntransactions = transactions[['t_dat','customer_id','article_id']]\ntransactions['t_dat'] = pd.to_datetime(transactions['t_dat'])\ntransactions['customer_id'] = transactions['customer_id'].apply(lambda x: int(x[-16:],16) ).astype('int64')\ntransactions['article_id'] = transactions['article_id'].astype('int32')\ntransactions.sort_values(by=['t_dat', 'customer_id'], inplace=True)\n\n_ = gc.collect()\n\nprint(transactions['t_dat'].min(), transactions['t_dat'].max())\nprint(\"Num of rows in training data:\", transactions.shape[0])\ntransactions.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-25T08:17:07.400063Z","iopub.execute_input":"2022-03-25T08:17:07.400451Z","iopub.status.idle":"2022-03-25T08:18:33.914676Z","shell.execute_reply.started":"2022-03-25T08:17:07.400412Z","shell.execute_reply":"2022-03-25T08:18:33.913623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Evaluation Metric MAP@k\n\nFrom https://raw.githubusercontent.com/benhamner/Metrics/master/Python/ml_metrics/average_precision.py","metadata":{}},{"cell_type":"code","source":"def apk(actual, predicted, k=10):\n    \"\"\"\n    Computes the average precision at k.\n\n    This function computes the average prescision at k between two lists of\n    items.\n\n    Parameters\n    ----------\n    actual : list\n             A list of elements that are to be predicted (order doesn't matter)\n    predicted : list\n                A list of predicted elements (order does matter)\n    k : int, optional\n        The maximum number of predicted elements\n\n    Returns\n    -------\n    score : double\n            The average precision at k over the input lists\n\n    \"\"\"\n    if len(predicted)>k:\n        predicted = predicted[:k]\n\n    score = 0.0\n    num_hits = 0.0\n\n    for i,p in enumerate(predicted):\n        if p in actual and p not in predicted[:i]:\n            num_hits += 1.0\n            score += num_hits / (i+1.0)\n\n    if not actual:\n        return 0.0\n\n    return score / min(len(actual), k)\n\ndef mapk(actual, predicted, k=10):\n    \"\"\"\n    Computes the mean average precision at k.\n\n    This function computes the mean average prescision at k between two lists\n    of lists of items.\n\n    Parameters\n    ----------\n    actual : list\n             A list of lists of elements that are to be predicted \n             (order doesn't matter in the lists)\n    predicted : list\n                A list of lists of predicted elements\n                (order matters in the lists)\n    k : int, optional\n        The maximum number of predicted elements\n\n    Returns\n    -------\n    score : double\n            The mean average precision at k over the input lists\n\n    \"\"\"\n    return np.mean([apk(a,p,k) for a,p in zip(actual, predicted)])","metadata":{"execution":{"iopub.status.busy":"2022-03-25T08:37:30.740399Z","iopub.execute_input":"2022-03-25T08:37:30.740772Z","iopub.status.idle":"2022-03-25T08:37:30.748838Z","shell.execute_reply.started":"2022-03-25T08:37:30.740735Z","shell.execute_reply":"2022-03-25T08:37:30.747963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Baseline - Recommend 12 most popular articles of last X weeks\n\nRecommend most popular articles (popular being most frequently bought) of last X weeks before the start of the test period. I'll use different values to test the best performing one such as 1, 2, 3, 4, 5, and 10 weeks. Validation period will be the following 7-day window after last day of training data.","metadata":{}},{"cell_type":"code","source":"class BaselineMostPopularArticles():\n    \n    def __init__(self, k=12):\n        self._k_most_popular_items = []\n        self._k = k\n    \n    def fit(self, X: pd.DataFrame):\n        \"\"\"\n            X is a DataFrame with purchase granularity (date, user, item) i.e. columns \n                ['t_dat', 'customer_id', 'article_id', ...].\n            To fit the baseline model, it counts the total amount of times an article has been purchased\n            over all the dataframe, and stores the k most popular items in memory.\n        \"\"\"\n        self._k_most_popular_items = X['article_id'].value_counts()[:self._k].index.tolist()\n    \n    def predict(self, for_submission=False):\n        if not for_submission: \n            return self._k_most_popular_items\n        else: \n            return \" \".join(['0' + str(item) for item in self._k_most_popular_items])","metadata":{"execution":{"iopub.status.busy":"2022-03-25T10:04:19.280958Z","iopub.execute_input":"2022-03-25T10:04:19.281272Z","iopub.status.idle":"2022-03-25T10:04:19.288412Z","shell.execute_reply.started":"2022-03-25T10:04:19.281239Z","shell.execute_reply":"2022-03-25T10:04:19.287752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Local Validation strategy\n\nDepending on the amount of weeks used for training data, the cross-validation strategy should follow a traditional time-series train-test split over 5 different periods. For example, for a 2 week training window:\n\n1. Fold 1 -> Weeks 1, 2 for training and Week 3 for validation\n2. Fold 2 -> Weeks 2, 3 for training and Week 4 for validation\n3. Fold 3 -> Weeks 3, 4 for training and Week 5 for validation\n4. ...\n\nand so on. One might want to use different seasons in the folds e.g. 4 folds in each different season, winter, spring, summer and autumn. Or having a considerable gap between each fold, to cover as much possibly different test distributions as possible. ","metadata":{}},{"cell_type":"code","source":"# extract week number without using repeating weeks, week 0 is last week of dataset\ntransactions[\"week\"] = (transactions[\"t_dat\"].max() - transactions[\"t_dat\"]).dt.days // 7","metadata":{"execution":{"iopub.status.busy":"2022-03-25T08:53:59.550375Z","iopub.execute_input":"2022-03-25T08:53:59.550678Z","iopub.status.idle":"2022-03-25T08:54:09.416452Z","shell.execute_reply.started":"2022-03-25T08:53:59.550648Z","shell.execute_reply":"2022-03-25T08:54:09.415894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Run the experiments!","metadata":{}},{"cell_type":"code","source":"WEEK_HISTS = [1, 2, 3, 4, 5, 10] # the training window length for each experiment\nN_FOLDS = 5 # number of folds for cross-validation\nWEEK_NUMBERS = sorted(transactions['week'].unique().tolist())[1:] # 1: to avoid taking week 0\nSTART_WEEK_FOLD = [1, 5, 10, 15, 20] # valid week for fold i will be START_WEEK_FOLD[i] - 1\n\nVERBOSE = 0 # 0 will show only aggregated results from cross-validation, 1 will show results in each fold\n\nprint(f\"nfolds={N_FOLDS} start_week_folds={START_WEEK_FOLD} \\n\")\n\nfor week_hist in WEEK_HISTS:\n\n    print(f\"Experiment: week_hist={week_hist}\")\n    metrics = []\n    for fold, start_week in enumerate(START_WEEK_FOLD):\n        \n        week_valid = [start_week - 1]\n        week_train = list(range(start_week, start_week + week_hist))\n        \n        train = transactions[transactions['week'].isin(week_train)].copy()\n        valid = transactions[transactions['week'].isin(week_valid)].copy()\n        \n        ## TEST YOUR MODEL HERE\n        \n        baseline = BaselineMostPopularArticles(k=12)\n        baseline.fit(train)\n\n        valid_grouped = valid.groupby(['customer_id'])['article_id'].apply(list).reset_index()\n        valid_grouped['article_preds'] = valid_grouped.apply(lambda x: baseline.predict(), axis=1)\n        \n        loss_at_fold = mapk(valid_grouped['article_id'], valid_grouped['article_preds'], k=12)\n        metrics.append(loss_at_fold)\n        \n        if VERBOSE == 1:\n            print(f\"\\tFold={fold}, week_train={week_train}, week_valid={week_valid}\")\n            print(\"\\tTrain window:\", train['t_dat'].min(), train['t_dat'].max())\n            print(\"\\tValid window:\", valid['t_dat'].min(), valid['t_dat'].max())\n            print(\"\\tMAP@12 = \", loss_at_fold)\n            print()\n    \n    print(\"Results MAP@12 => \", np.mean(metrics))\n    print()","metadata":{"execution":{"iopub.status.busy":"2022-03-25T09:40:03.69158Z","iopub.execute_input":"2022-03-25T09:40:03.6924Z","iopub.status.idle":"2022-03-25T09:41:03.712912Z","shell.execute_reply.started":"2022-03-25T09:40:03.692351Z","shell.execute_reply":"2022-03-25T09:41:03.712108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Best result for our naive baseline is to use last week's most popular items. Let's then create a submission with this model:\n\n## Generate Submission (0.0071 LB score)","metadata":{}},{"cell_type":"code","source":"train = transactions[transactions['week'] == 0].copy()\nbaseline = BaselineMostPopularArticles(k=12)\nbaseline.fit(train)\n\nsubmission = pd.read_csv(data_path / 'sample_submission.csv')\nsubmission['prediction'] = baseline.predict(for_submission=True)\nsubmission.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-25T10:04:59.724653Z","iopub.execute_input":"2022-03-25T10:04:59.725382Z","iopub.status.idle":"2022-03-25T10:04:59.807022Z","shell.execute_reply.started":"2022-03-25T10:04:59.725345Z","shell.execute_reply":"2022-03-25T10:04:59.806205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2022-03-25T10:05:46.263946Z","iopub.execute_input":"2022-03-25T10:05:46.264727Z","iopub.status.idle":"2022-03-25T10:05:52.676495Z","shell.execute_reply.started":"2022-03-25T10:05:46.264681Z","shell.execute_reply":"2022-03-25T10:05:52.675474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Next steps \n\nIn this notebook, I have shown how to get started in this competition with a simple baseline and a thorough cross-validation strategy to test your experiments. My recommendation on how to use this notebook and getting started with your next experiments would be:\n\n* Run this notebook and get familiar with the process, data, and prediction style.\n* As one of the bottlenecks of execution time for this notebook is to read input data, I recommend saving the transactions dataframe (after reducing memory) as parquet file, to speed up loading time later on. You may use `to_parquet()` function from pandas for that.\n* The goal should be to have an end-to-end pipeline with which you can test experiments as fast as possible.\n\n**How to improve from here?**\n\nCheck out these notebooks implementing baselines from other kagglers. These helped me out get a better picture on the data format and how to deal with the dataset:\n\n* [Recommend Items Purchased Together LB Score 0.021](https://www.kaggle.com/code/cdeotte/recommend-items-purchased-together-0-021/notebook)\n* [Time is our Best Friend v2](https://www.kaggle.com/code/hengzheng/time-is-our-best-friend-v2/notebook)\n* [This discussion](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/307288) is worth reading to get an understanding on how to approach this competition.\n\nMy ideas on how to improve from here are:\n\n* Implement classic Item-based Collaborative Filtering approaches. Can we model similar users and similar articles solely based on purchase history?\n* Integrate Customer and Item metadata in the model assuming it might help with the cold start problem (customers who bought zero (or few) article in the training dataset.\n* How can we adapt the model to include time sensitive data? \n* [Advanced] use NLP and Computer Vision pretrained models to map article description and images into latent vector spaces to enhance item metadata.\n\n# Thanks for reading!\n\nIf you've gotten this far, congrats! I hope you enjoyed this notebook. If so, please upvote and comment your thoughts. Good luck and have fun!","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}