{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":31254,"databundleVersionId":3103714,"sourceType":"competition"},{"sourceId":3205676,"sourceType":"datasetVersion","datasetId":1945136}],"dockerImageVersionId":30163,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Recommend Items Frequently Purchased Together\nThis notebook demonstrates how recommending items that are frequently purchased together is effective. The current best scoring public notebook [here][1] recommends to customers those customers' last purchases and scores public LB 0.020. In this notebook here, we will begin with that idea and add recommending items that are frequently purchased together with a customers' previous purchaes. This notebook improves the LB and scores LB 0.021. This notebook's strategy is as follows:\n* recommend items previously purchased [idea here][1]\n* recommend items that are bought together with previous purchases [idea here][2]\n* recommend popular items [idea here][1]\n\n[1]: https://www.kaggle.com/hengzheng/time-is-our-best-friend-v2\n[2]: https://www.kaggle.com/cdeotte/customers-who-bought-this-frequently-buy-this","metadata":{}},{"cell_type":"markdown","source":"# RAPIDS cuDF\nWe will use RAPIDS cuDF for fast dataframe operations","metadata":{}},{"cell_type":"code","source":"import cudf\nprint('RAPIDS version',cudf.__version__)","metadata":{"execution":{"iopub.status.busy":"2024-07-10T09:44:46.845134Z","iopub.execute_input":"2024-07-10T09:44:46.845819Z","iopub.status.idle":"2024-07-10T09:44:50.328092Z","shell.execute_reply.started":"2024-07-10T09:44:46.845726Z","shell.execute_reply":"2024-07-10T09:44:50.327273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Transactions, Reduce Memory\nDiscussion about reducing memory is [here][1]\n\n[1]: https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308635","metadata":{}},{"cell_type":"code","source":"train = cudf.read_csv('../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv')\ntrain['customer_id'] = train['customer_id'].str[-16:].str.hex_to_int().astype('int64')\ntrain['article_id'] = train.article_id.astype('int32')\ntrain.t_dat = cudf.to_datetime(train.t_dat)\ntrain = train[['t_dat', 'customer_id', 'article_id']]\ntrain.to_parquet('train.pqt', index=False)\nprint(train.shape)\ntrain.head()\n","metadata":{"execution":{"iopub.status.busy":"2024-07-10T09:44:50.330096Z","iopub.execute_input":"2024-07-10T09:44:50.330476Z","iopub.status.idle":"2024-07-10T09:45:32.276803Z","shell.execute_reply.started":"2024-07-10T09:44:50.330433Z","shell.execute_reply":"2024-07-10T09:45:32.276020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Find Each Customer's Last Week of Purchases\nOur final predictions will have the row order from of our dataframe. Each row of our dataframe will be a prediction. We will create the `predictionstring` later by `train.groupby('customer_id').article_id.sum()`. Since `article_id` is a string, when we groupby sum, it will concatenate all the customer predictions into a single string. It will also create the string in the order of the dataframe. So as we proceed in this notebook, we will order the dataframe how we want our predictions ordered.","metadata":{}},{"cell_type":"code","source":"# Find the most recent purchase date and corresponding article for each customer\ntmp = train.groupby('customer_id').t_dat.idxmax()\nlast_purchases = train.loc[tmp].reset_index(drop=True)\nlast_purchases = last_purchases[['customer_id', 'article_id']]\nlast_purchases.columns = ['customer_id', 'last_article_id']\n","metadata":{"execution":{"iopub.status.busy":"2024-07-10T09:45:32.278005Z","iopub.execute_input":"2024-07-10T09:45:32.278268Z","iopub.status.idle":"2024-07-10T09:45:32.406387Z","shell.execute_reply.started":"2024-07-10T09:45:32.278232Z","shell.execute_reply":"2024-07-10T09:45:32.405739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# (1) Load Pairs and Recommend Items Purchased Together\nNote that many operations in cuDF will shuffle the order of the dataframe rows. Therefore we need to sort afterward because we want the most often previously purchased items first. Because this will be the order of our predictons. Since we sort by `ct` and then `t_dat` will will recommend items that have been purchased more frequently first followed by items purchased more recently second.","metadata":{}},{"cell_type":"code","source":"import pandas as pd, numpy as np\n\n# Convert to pandas for using map with dictionary\nlast_purchases = last_purchases.to_pandas()\npairs = np.load('../input/hmitempairs/pairs_cudf.npy', allow_pickle=True).item()\nlast_purchases['recommended_article_id'] = last_purchases['last_article_id'].map(pairs)\n\n# Keep only non-null recommendations\nrecommendations = last_purchases[['customer_id', 'recommended_article_id']].dropna()\nrecommendations = recommendations.drop_duplicates(['customer_id', 'recommended_article_id'])\nrecommendations = recommendations.rename({'recommended_article_id': 'article_id'}, axis=1)\n","metadata":{"execution":{"iopub.status.busy":"2024-07-10T09:45:32.407964Z","iopub.execute_input":"2024-07-10T09:45:32.408198Z","iopub.status.idle":"2024-07-10T09:45:32.921983Z","shell.execute_reply.started":"2024-07-10T09:45:32.408169Z","shell.execute_reply":"2024-07-10T09:45:32.921153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# (2)  Convert Recommendations into Single String\nIn my notebook [here][1], we compute a dictionary of items frequently purchased together. We will load and use that dictionary below. Note that we use the command `drop_duplicates` so that we don't recommend an item that the user has already bought and we have already recommended above. We will need to use Pandas for some commands because RAPIDS cuDF doesn't have two conveinent commands, (1) create new column from dictionary map of another column (2) groupby aggregate strings sum.\n\nWe concatenate these rows after the rows containing customers' previous purchases. Therefore we will recommend previous items first and then items purchased together second. Note the trick to convert a column of int32 into a prediction string (using groupby agg str sum) is from notebook [here][2]\n\n[1]: https://www.kaggle.com/cdeotte/customers-who-bought-this-frequently-buy-this\n[2]: https://www.kaggle.com/hiroshisakiyama/recommending-items-recently-bought","metadata":{}},{"cell_type":"code","source":"# Convert back to cudf\nrecommendations = cudf.DataFrame(recommendations)\n\n# Convert article_id to string\nrecommendations['article_id'] = ' 0' + recommendations['article_id'].astype('str')\n\n# Perform the groupby operation using cudf\ngrouped = recommendations.groupby('customer_id').agg({'article_id': 'collect'}).reset_index()\n\n# Convert the grouped result to pandas\ngrouped_pandas = grouped.to_pandas()\n\n# Apply the string concatenation in pandas\ngrouped_pandas['prediction'] = grouped_pandas['article_id'].apply(lambda x: ' '.join(x))\n\n# Select necessary columns\ngrouped_pandas = grouped_pandas[['customer_id', 'prediction']]\n\n# Convert back to cudf\npreds = cudf.DataFrame(grouped_pandas)\n\n# Display the first few rows of preds\npreds.head()","metadata":{"execution":{"iopub.status.busy":"2024-07-10T09:48:17.000068Z","iopub.execute_input":"2024-07-10T09:48:17.000395Z","iopub.status.idle":"2024-07-10T09:48:19.820216Z","shell.execute_reply.started":"2024-07-10T09:48:17.000361Z","shell.execute_reply":"2024-07-10T09:48:19.819531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Recommend Last Week's Most Popular Items","metadata":{}},{"cell_type":"code","source":"train = cudf.read_parquet('train.pqt')\ntrain.t_dat = cudf.to_datetime(train.t_dat)\ntrain = train.loc[train.t_dat >= cudf.to_datetime('2020-09-16')]\ntop12 = ' 0' + ' 0'.join(train.article_id.value_counts().to_pandas().index.astype('str')[:12])\nprint(\"Last week's top 12 popular items:\")\nprint(top12)\n","metadata":{"execution":{"iopub.status.busy":"2024-07-10T09:48:26.519944Z","iopub.execute_input":"2024-07-10T09:48:26.520696Z","iopub.status.idle":"2024-07-10T09:48:26.770662Z","shell.execute_reply.started":"2024-07-10T09:48:26.520656Z","shell.execute_reply":"2024-07-10T09:48:26.769884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Write Submission CSV\nWe will merge our predictions onto `sample_submission.csv` and submit to Kaggle.","metadata":{}},{"cell_type":"code","source":"sub = cudf.read_csv('../input/h-and-m-personalized-fashion-recommendations/sample_submission.csv')\nsub = sub[['customer_id']]\nsub['customer_id_2'] = sub['customer_id'].str[-16:].str.hex_to_int().astype('int64')\nsub = sub.merge(preds.rename({'customer_id': 'customer_id_2'}, axis=1), on='customer_id_2', how='left').fillna('')\ndel sub['customer_id_2']\nsub.prediction = sub.prediction + top12\nsub.prediction = sub.prediction.str.strip()\nsub.prediction = sub.prediction.str[:131]\nsub.to_csv(f'submission.csv', index=False)\nsub.head()\n","metadata":{"execution":{"iopub.status.busy":"2024-07-10T09:48:33.290527Z","iopub.execute_input":"2024-07-10T09:48:33.291352Z","iopub.status.idle":"2024-07-10T09:48:37.360314Z","shell.execute_reply.started":"2024-07-10T09:48:33.291299Z","shell.execute_reply":"2024-07-10T09:48:37.359474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}