{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<!-- \n\nimport numpy as np \nimport pandas as pd \n\n\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename)) -->\n\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"markdown","source":"# H&M Personalized Fashion Recommendations","metadata":{}},{"cell_type":"markdown","source":"**Analysing the data and developing product recommendations based on data from previous transactions. For each customer_id observed in the training data, predicted up to 12 labels for the article_id, which is the predicted items a customer will buy in the next 7-day period after the training time period.**\n\nThis notebook demonstrates how recommending items that are frequently purchased together is effective.  This notebook's strategy is as follows:\n\n* recommend items previously purchased idea here\n* recommend items that are bought together with previous purchases idea here\n* recommend popular items","metadata":{}},{"cell_type":"markdown","source":"# Table of Content","metadata":{}},{"cell_type":"markdown","source":"**1. Loading the dataset:** Load the data and import the libraries.\n\n**2. Data Cleaning:**\n*   Deleting redundant columns.\n*   Renaming the columns.\n*   Dropping duplicates.\n*   Cleaning individual columns.\n*   Remove the NaN values from the dataset\n\n**3. Data Visualization:** Using plots to find relations between the features.\n   1.    Articles Data : \n*       Product Types Per Product Group\n* Articles count per each product Group\n* Number of Articles per each Product Type\n* Number of articles per each index name\n* The garments grouped by index:\n* Number of Articles per each Perceived Colour Value Name\n* Number of Articles per each  Colour group\n   2. Customers Data: \n*           Age distribution of customers\n* Distribution of Club member status\n* Distribution of Fashion News Frequency\n   3. Transactions data\n* Price Outliers\n* Mean Price of Each Product Group Name\n\n**4. Items recommendations based on data from previous transactions**\n*  Each Customer's Last Week of Purchases\n*  Recommend Most Often Previously Purchased Items\n*  Recommend Items Purchased Together\n*  Recommend Last Week's Most Popular Items\n*  Write Submission CSV","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"# 1. Loading the dataset","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:10:37.542125Z","iopub.execute_input":"2022-03-11T07:10:37.543051Z","iopub.status.idle":"2022-03-11T07:10:38.828966Z","shell.execute_reply.started":"2022-03-11T07:10:37.542749Z","shell.execute_reply":"2022-03-11T07:10:38.827822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles=pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/articles.csv')\ncustomers=pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/customers.csv')\ntransactions=pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:11:38.961248Z","iopub.execute_input":"2022-03-11T07:11:38.961575Z","iopub.status.idle":"2022-03-11T07:12:53.864717Z","shell.execute_reply.started":"2022-03-11T07:11:38.961538Z","shell.execute_reply":"2022-03-11T07:12:53.863758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:36:05.35372Z","iopub.execute_input":"2022-03-11T06:36:05.354251Z","iopub.status.idle":"2022-03-11T06:36:05.414407Z","shell.execute_reply.started":"2022-03-11T06:36:05.354212Z","shell.execute_reply":"2022-03-11T06:36:05.41358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles.info()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:36:08.942718Z","iopub.execute_input":"2022-03-11T06:36:08.943992Z","iopub.status.idle":"2022-03-11T06:36:09.148949Z","shell.execute_reply.started":"2022-03-11T06:36:08.943928Z","shell.execute_reply":"2022-03-11T06:36:09.147914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Data Cleaning","metadata":{}},{"cell_type":"code","source":"articles.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:36:13.85972Z","iopub.execute_input":"2022-03-11T06:36:13.860743Z","iopub.status.idle":"2022-03-11T06:36:14.03486Z","shell.execute_reply.started":"2022-03-11T06:36:13.860688Z","shell.execute_reply":"2022-03-11T06:36:14.033881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles.nunique()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:36:28.793763Z","iopub.execute_input":"2022-03-11T06:36:28.794043Z","iopub.status.idle":"2022-03-11T06:36:28.967937Z","shell.execute_reply.started":"2022-03-11T06:36:28.794014Z","shell.execute_reply":"2022-03-11T06:36:28.967235Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for c in articles.columns:\n    if not 'no' in c and not 'id' in c and not 'code' in c:\n        n=articles[c].nunique()\n        print(f'number of unique {c} is: {n}')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:36:31.783723Z","iopub.execute_input":"2022-03-11T06:36:31.784061Z","iopub.status.idle":"2022-03-11T06:36:31.926364Z","shell.execute_reply.started":"2022-03-11T06:36:31.784028Z","shell.execute_reply":"2022-03-11T06:36:31.925524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Articles Data : \n\n**Product Types Per Product Group**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(12,5))\ntemp = articles.groupby([\"product_group_name\"])[\"product_type_name\"].nunique()\narticles_temp_df = pd.DataFrame({'Product Group': temp.index,\n                   'Product Types': temp.values\n                  })\narticles_temp_df=articles_temp_df.sort_values('Product Types',ascending=False)\nsns.barplot(y='Product Group',x='Product Types',data=articles_temp_df)\nplt.title('NUmber of Product Types per Product group')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:36:34.449697Z","iopub.execute_input":"2022-03-11T06:36:34.450446Z","iopub.status.idle":"2022-03-11T06:36:34.885084Z","shell.execute_reply.started":"2022-03-11T06:36:34.450412Z","shell.execute_reply":"2022-03-11T06:36:34.884288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**And the product group-product structure. Accessories are really various, the most numerious: bags, earrings and hats. However, trousers prevail.**","metadata":{}},{"cell_type":"code","source":"pd.options.display.max_rows = None\narticles.groupby(by=['product_group_name','product_type_name'])['article_id'].count()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:36:42.132455Z","iopub.execute_input":"2022-03-11T06:36:42.132926Z","iopub.status.idle":"2022-03-11T06:36:42.170522Z","shell.execute_reply.started":"2022-03-11T06:36:42.132869Z","shell.execute_reply":"2022-03-11T06:36:42.169978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Articles count per each product group**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10,5))\ntemp=articles.groupby(by='product_group_name')['article_id'].count()\ntemp_articles_df=pd.DataFrame({'Product Group':temp.index,'Articles':temp.values})\ntemp_articles_df=temp_articles_df.sort_values('Articles',ascending=False)\ns=sns.barplot(x='Product Group',y='Articles',data=temp_articles_df)\ns.set_xticklabels(s.get_xticklabels(),rotation=90)\nplt.title('Number of Articles per each Product Group')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:42:10.512161Z","iopub.execute_input":"2022-03-11T06:42:10.51256Z","iopub.status.idle":"2022-03-11T06:42:10.881248Z","shell.execute_reply.started":"2022-03-11T06:42:10.512529Z","shell.execute_reply":"2022-03-11T06:42:10.880084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Number of Articles per each Product Type**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\ntemp=articles.groupby(by='product_type_name')['article_id'].count()\ntemp_articles_df=pd.DataFrame({'Product Type':temp.index,'Articles':temp.values})\ntemp_articles_df=temp_articles_df.sort_values('Articles',ascending=False)[:50]\ns=sns.barplot(x='Product Type',y='Articles',data=temp_articles_df)\ns.set_xticklabels(s.get_xticklabels(),rotation=90)\nplt.title('Number of Articles per each Product Type ')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:43:15.866352Z","iopub.execute_input":"2022-03-11T06:43:15.866642Z","iopub.status.idle":"2022-03-11T06:43:16.760805Z","shell.execute_reply.started":"2022-03-11T06:43:15.866613Z","shell.execute_reply":"2022-03-11T06:43:16.759783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles.groupby(by=['index_group_name','index_name'])['article_id'].count()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:46:09.367154Z","iopub.execute_input":"2022-03-11T06:46:09.367476Z","iopub.status.idle":"2022-03-11T06:46:09.402916Z","shell.execute_reply.started":"2022-03-11T06:46:09.367447Z","shell.execute_reply":"2022-03-11T06:46:09.402182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Number of articles per each index name**","metadata":{}},{"cell_type":"code","source":"sns.countplot(y='index_name',data=articles)\nplt.xlabel('Count by index name')\nplt.ylabel('Index Name')\nplt.title('Number of articles per each index name')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:46:12.424169Z","iopub.execute_input":"2022-03-11T06:46:12.424524Z","iopub.status.idle":"2022-03-11T06:46:12.808888Z","shell.execute_reply.started":"2022-03-11T06:46:12.424486Z","shell.execute_reply":"2022-03-11T06:46:12.807883Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"\n**The garments grouped by index: Jersey fancy is the most frequent garment, especially for women and children. The next by number is accessories, many various accessories with low price.**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,8))\nsns.histplot(y='garment_group_name',data=articles,hue='index_name',multiple=\"stack\")\nplt.xlabel('Count by Garment Group Name')\nplt.ylabel('Garment Group Name')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:46:48.532851Z","iopub.execute_input":"2022-03-11T06:46:48.53323Z","iopub.status.idle":"2022-03-11T06:46:50.066293Z","shell.execute_reply.started":"2022-03-11T06:46:48.533187Z","shell.execute_reply":"2022-03-11T06:46:50.065505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Number of Articles per each Perceived Colour Value Name**","metadata":{}},{"cell_type":"code","source":"temp = articles.groupby([\"perceived_colour_value_name\"])[\"article_id\"].nunique()\ndf = pd.DataFrame({'Perceived Colour Value Name': temp.index,\n                   'Articles': temp.values\n                  })\ndf = df.sort_values(['Articles'], ascending=False)\nplt.figure(figsize = (8,6))\nplt.title(f'Number of Articles per each Perceived Colour Value Name')\nsns.set_color_codes(\"pastel\")\ns = sns.barplot(x = 'Perceived Colour Value Name', y=\"Articles\", data=df)\ns.set_xticklabels(s.get_xticklabels(),rotation=90)\nlocs, labels = plt.xticks()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:50:02.056761Z","iopub.execute_input":"2022-03-11T06:50:02.057149Z","iopub.status.idle":"2022-03-11T06:50:02.339376Z","shell.execute_reply.started":"2022-03-11T06:50:02.057103Z","shell.execute_reply":"2022-03-11T06:50:02.338361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Number of Articles per each Perceived Colour Master Name**","metadata":{}},{"cell_type":"code","source":"temp = articles.groupby([\"perceived_colour_master_name\"])[\"article_id\"].nunique()\ndf = pd.DataFrame({'Perceived Colour Master Name': temp.index,\n                   'Articles': temp.values\n                  })\ndf = df.sort_values(['Articles'], ascending=False)\nplt.figure(figsize = (12,6))\nplt.title(f'Number of Articles per each Perceived Colour Master Name')\nsns.set_color_codes(\"pastel\")\ns = sns.barplot(x = 'Perceived Colour Master Name', y=\"Articles\", data=df)\ns.set_xticklabels(s.get_xticklabels(),rotation=90)\nlocs, labels = plt.xticks()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:50:29.940441Z","iopub.execute_input":"2022-03-11T06:50:29.94079Z","iopub.status.idle":"2022-03-11T06:50:30.304838Z","shell.execute_reply.started":"2022-03-11T06:50:29.940756Z","shell.execute_reply":"2022-03-11T06:50:30.303645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = articles.groupby([\"colour_group_name\"])[\"article_id\"].nunique()\ndf = pd.DataFrame({'Color Group Name': temp.index,\n                   'Articles': temp.values\n                  })\ndf = df.sort_values(['Articles'], ascending=False)\nplt.figure(figsize = (12,6))\nplt.title(f'Number of Articles per each  Colour group')\nsns.set_color_codes(\"pastel\")\ns = sns.barplot(x = 'Color Group Name', y=\"Articles\", data=df)\ns.set_xticklabels(s.get_xticklabels(),rotation=90)\nlocs, labels = plt.xticks()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:51:18.31926Z","iopub.execute_input":"2022-03-11T06:51:18.319604Z","iopub.status.idle":"2022-03-11T06:51:19.196203Z","shell.execute_reply.started":"2022-03-11T06:51:18.319569Z","shell.execute_reply":"2022-03-11T06:51:19.195092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Customers Data","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"customers.head()\n","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:52:05.793968Z","iopub.execute_input":"2022-03-11T06:52:05.794793Z","iopub.status.idle":"2022-03-11T06:52:05.809756Z","shell.execute_reply.started":"2022-03-11T06:52:05.79474Z","shell.execute_reply":"2022-03-11T06:52:05.809142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers.info()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:52:07.88659Z","iopub.execute_input":"2022-03-11T06:52:07.887323Z","iopub.status.idle":"2022-03-11T06:52:08.697492Z","shell.execute_reply.started":"2022-03-11T06:52:07.887278Z","shell.execute_reply":"2022-03-11T06:52:08.696697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:52:11.226698Z","iopub.execute_input":"2022-03-11T06:52:11.227045Z","iopub.status.idle":"2022-03-11T06:52:11.874397Z","shell.execute_reply.started":"2022-03-11T06:52:11.227008Z","shell.execute_reply":"2022-03-11T06:52:11.873286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**There are no duplicates in customers**","metadata":{}},{"cell_type":"code","source":"customers.shape[0]- customers['customer_id'].nunique()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:52:14.126235Z","iopub.execute_input":"2022-03-11T06:52:14.126575Z","iopub.status.idle":"2022-03-11T06:52:14.913915Z","shell.execute_reply.started":"2022-03-11T06:52:14.126539Z","shell.execute_reply":"2022-03-11T06:52:14.913046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Age Distribution**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10,5))\nsns.histplot(x='age',data=customers,bins=40)\nplt.title('Age Distribution of Customers')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:52:16.224348Z","iopub.execute_input":"2022-03-11T06:52:16.224632Z","iopub.status.idle":"2022-03-11T06:52:16.744213Z","shell.execute_reply.started":"2022-03-11T06:52:16.224599Z","shell.execute_reply":"2022-03-11T06:52:16.743159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Distribution of Club member status**","metadata":{}},{"cell_type":"code","source":"sns.countplot(x='club_member_status',data=customers)\nplt.xlabel('Distribution of Club member status')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:52:25.290857Z","iopub.execute_input":"2022-03-11T06:52:25.291555Z","iopub.status.idle":"2022-03-11T06:52:27.173288Z","shell.execute_reply.started":"2022-03-11T06:52:25.291519Z","shell.execute_reply":"2022-03-11T06:52:27.172607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers['fashion_news_frequency'].unique()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:55:05.618226Z","iopub.execute_input":"2022-03-11T06:55:05.618836Z","iopub.status.idle":"2022-03-11T06:55:05.774992Z","shell.execute_reply.started":"2022-03-11T06:55:05.618787Z","shell.execute_reply":"2022-03-11T06:55:05.773829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers.loc[~customers['fashion_news_frequency'].isin(['Regularly', 'Monthly']), 'fashion_news_frequency']='None'","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:55:07.507641Z","iopub.execute_input":"2022-03-11T06:55:07.507911Z","iopub.status.idle":"2022-03-11T06:55:07.657025Z","shell.execute_reply.started":"2022-03-11T06:55:07.507877Z","shell.execute_reply":"2022-03-11T06:55:07.65603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers['fashion_news_frequency'].unique()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:55:10.74216Z","iopub.execute_input":"2022-03-11T06:55:10.742469Z","iopub.status.idle":"2022-03-11T06:55:10.851849Z","shell.execute_reply.started":"2022-03-11T06:55:10.742439Z","shell.execute_reply":"2022-03-11T06:55:10.850993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pie_data=customers[['customer_id','fashion_news_frequency']].groupby(by='fashion_news_frequency').count()\npie_data","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:55:13.219639Z","iopub.execute_input":"2022-03-11T06:55:13.219912Z","iopub.status.idle":"2022-03-11T06:55:13.623275Z","shell.execute_reply.started":"2022-03-11T06:55:13.21988Z","shell.execute_reply":"2022-03-11T06:55:13.622515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Distribution of Fashion News Frequency**","metadata":{}},{"cell_type":"code","source":"\ncolors = sns.color_palette('colorblind')\nf,ax=plt.subplots(figsize=(10,5))\nax.pie(pie_data.customer_id,labels=pie_data.index,colors=colors)\nplt.title('Distribution of Fashion News Frequency')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T06:55:16.279214Z","iopub.execute_input":"2022-03-11T06:55:16.279674Z","iopub.status.idle":"2022-03-11T06:55:16.407098Z","shell.execute_reply.started":"2022-03-11T06:55:16.279626Z","shell.execute_reply":"2022-03-11T06:55:16.406186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Transactions Data","metadata":{}},{"cell_type":"code","source":"transactions.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:13:22.587728Z","iopub.execute_input":"2022-03-11T07:13:22.588120Z","iopub.status.idle":"2022-03-11T07:13:22.615951Z","shell.execute_reply.started":"2022-03-11T07:13:22.588060Z","shell.execute_reply":"2022-03-11T07:13:22.615146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.set_option('display.float_format','{:.4f}'.format)\ntransactions['price'].describe()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:13:24.836569Z","iopub.execute_input":"2022-03-11T07:13:24.837049Z","iopub.status.idle":"2022-03-11T07:13:26.019098Z","shell.execute_reply.started":"2022-03-11T07:13:24.836993Z","shell.execute_reply":"2022-03-11T07:13:26.018266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Price Outliers**","metadata":{}},{"cell_type":"code","source":"sns.boxplot(x='price',data=transactions)\nplt.xlabel('Price outliers')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:13:28.191279Z","iopub.execute_input":"2022-03-11T07:13:28.191583Z","iopub.status.idle":"2022-03-11T07:13:35.488986Z","shell.execute_reply.started":"2022-03-11T07:13:28.191548Z","shell.execute_reply":"2022-03-11T07:13:35.488146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tran_temp=transactions.groupby(by='customer_id').count()\n","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:13:40.307784Z","iopub.execute_input":"2022-03-11T07:13:40.308242Z","iopub.status.idle":"2022-03-11T07:13:58.665118Z","shell.execute_reply.started":"2022-03-11T07:13:40.308202Z","shell.execute_reply":"2022-03-11T07:13:58.664323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tran_temp.sort_values('price',ascending=False)['price'][:10]","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:14:05.291524Z","iopub.execute_input":"2022-03-11T07:14:05.291900Z","iopub.status.idle":"2022-03-11T07:14:05.965231Z","shell.execute_reply.started":"2022-03-11T07:14:05.291862Z","shell.execute_reply":"2022-03-11T07:14:05.964310Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles_main=articles[['article_id', 'prod_name', 'product_type_name', 'product_group_name', 'index_name']]","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:14:10.764850Z","iopub.execute_input":"2022-03-11T07:14:10.765561Z","iopub.status.idle":"2022-03-11T07:14:10.777380Z","shell.execute_reply.started":"2022-03-11T07:14:10.765513Z","shell.execute_reply":"2022-03-11T07:14:10.776228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles_transactions_merge=transactions[['customer_id', 'article_id', 'price', 't_dat']].merge(articles_main,on='article_id',how='left')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:14:13.521269Z","iopub.execute_input":"2022-03-11T07:14:13.522388Z","iopub.status.idle":"2022-03-11T07:14:25.003152Z","shell.execute_reply.started":"2022-03-11T07:14:13.522326Z","shell.execute_reply":"2022-03-11T07:14:25.002045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles_transactions_merge.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:14:33.483863Z","iopub.execute_input":"2022-03-11T07:14:33.484226Z","iopub.status.idle":"2022-03-11T07:14:33.500548Z","shell.execute_reply.started":"2022-03-11T07:14:33.484192Z","shell.execute_reply":"2022-03-11T07:14:33.499883Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"# f, ax = plt.subplots(figsize=(25,15))\n# ax.set_xlabel('price outliers')\n# ax.set_ylabel('index name')\n# ax=sns.boxplot(x='price', y='product_group_name',data=articles_transactions_merge)\n# ax.xaxis.set_tick_params(labelsize=22)\n# ax.yaxis.set_tick_params(labelsize=22)\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-11T07:48:15.169176Z","iopub.execute_input":"2022-03-11T07:48:15.169993Z","iopub.status.idle":"2022-03-11T07:48:15.173460Z","shell.execute_reply.started":"2022-03-11T07:48:15.169949Z","shell.execute_reply":"2022-03-11T07:48:15.172794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"# f, ax = plt.subplots(figsize=(25,15))\n# ax.set_xlabel('price outliers')\n# ax.set_ylabel('index name')\n# ax=sns.boxplot(x='price', y='index_name',data=articles_transactions_merge)\n# ax.xaxis.set_tick_params(labelsize=22)\n# ax.yaxis.set_tick_params(labelsize=22)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-11T07:48:17.903724Z","iopub.execute_input":"2022-03-11T07:48:17.904083Z","iopub.status.idle":"2022-03-11T07:48:17.908232Z","shell.execute_reply.started":"2022-03-11T07:48:17.904034Z","shell.execute_reply":"2022-03-11T07:48:17.907380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Mean Price of Each Product Group Name**","metadata":{}},{"cell_type":"code","source":"# mean_price=articles_transactions_merge[['index_name','price']].groupby(by='product_group_name').mean()\n# sns.barplot(x=mean_price.price,y=mean_price.index,color='orange')\n# plt.xlabel('Price')\n# plt.ylabel('Index Name')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:48:32.342836Z","iopub.execute_input":"2022-03-11T07:48:32.343476Z","iopub.status.idle":"2022-03-11T07:48:32.347895Z","shell.execute_reply.started":"2022-03-11T07:48:32.343427Z","shell.execute_reply":"2022-03-11T07:48:32.347044Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mean_price=articles_transactions_merge[['product_group_name','price']].groupby(by='product_group_name').mean().sort_values(by='price',ascending=False)\nsns.barplot(x=mean_price.price,y=mean_price.index,color='green')\nplt.xlabel('Price')\nplt.ylabel('Product Group Name')","metadata":{"execution":{"iopub.status.busy":"2022-03-11T07:49:03.448098Z","iopub.execute_input":"2022-03-11T07:49:03.448810Z","iopub.status.idle":"2022-03-11T07:49:08.711856Z","shell.execute_reply.started":"2022-03-11T07:49:03.448752Z","shell.execute_reply":"2022-03-11T07:49:08.711040Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Recommend Items Purchased Together ","metadata":{}},{"cell_type":"code","source":"\ntrain = pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv')\ntrain['article_id'] = train.article_id.astype('int32')\ntrain.t_dat = pd.to_datetime(train.t_dat)\ntrain = train[['t_dat','customer_id','article_id']]\ntrain.to_parquet('train.pqt',index=False)\nprint( train.shape )\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T04:43:43.753243Z","iopub.execute_input":"2022-03-11T04:43:43.753576Z","iopub.status.idle":"2022-03-11T04:45:19.372633Z","shell.execute_reply.started":"2022-03-11T04:43:43.753527Z","shell.execute_reply":"2022-03-11T04:45:19.371503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Last Week Purchases of each customer ","metadata":{}},{"cell_type":"code","source":"tmp = train.groupby('customer_id').t_dat.max().reset_index()\ntmp.columns = ['customer_id','max_dat']\ntrain = train.merge(tmp,on=['customer_id'],how='left')\ntrain['diff_dat'] = (train.max_dat - train.t_dat).dt.days\ntrain = train.loc[train['diff_dat']<=6]\nprint('Train shape:',train.shape)\ntrain.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-11T04:45:50.130288Z","iopub.execute_input":"2022-03-11T04:45:50.130769Z","iopub.status.idle":"2022-03-11T04:46:28.853323Z","shell.execute_reply.started":"2022-03-11T04:45:50.130717Z","shell.execute_reply":"2022-03-11T04:46:28.852002Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Recommend Most Often Previously Purchased Items\n","metadata":{}},{"cell_type":"code","source":"tmp = train.groupby(['customer_id','article_id'])['t_dat'].agg('count').reset_index()\ntmp.columns = ['customer_id','article_id','ct']\ntrain = train.merge(tmp,on=['customer_id','article_id'],how='left')\ntrain = train.sort_values(['ct','t_dat'],ascending=False)\ntrain = train.drop_duplicates(['customer_id','article_id'])\ntrain = train.sort_values(['ct','t_dat'],ascending=False)\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T04:46:32.438139Z","iopub.execute_input":"2022-03-11T04:46:32.438909Z","iopub.status.idle":"2022-03-11T04:46:52.840099Z","shell.execute_reply.started":"2022-03-11T04:46:32.438874Z","shell.execute_reply":"2022-03-11T04:46:52.839468Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Recommend Items Purchased Together","metadata":{}},{"cell_type":"code","source":"import pandas as pd, numpy as np\npairs = np.load('../input/pairs-cudfnpy/pairs_cudf.npy',allow_pickle=True).item()\ntrain['article_id2'] = train.article_id.map(pairs)","metadata":{"execution":{"iopub.status.busy":"2022-03-11T05:03:12.162123Z","iopub.execute_input":"2022-03-11T05:03:12.162439Z","iopub.status.idle":"2022-03-11T05:03:12.485664Z","shell.execute_reply.started":"2022-03-11T05:03:12.162407Z","shell.execute_reply":"2022-03-11T05:03:12.48444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# code in  pairs_cudf.npy file\n\n\n# df = pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv')\n# print('Transactions shape',df.shape)\n# display( df.head() )\n\n# # REDUCE MEMORY OF DATAFRAME\n# df = df[['customer_id','article_id']]\n# df.customer_id = df.customer_id.str[-16:].str.hex_to_int().astype('int64')\n# df.article_id = df.article_id.astype('int32')\n# _ = gc.collect()\n\n# vc = df.article_id.value_counts()\n# pairs = {}\n# for j,i in enumerate(vc.index.values[1000:1032]):\n#     #if j%10==0: print(j,', ',end='')\n#     USERS = df.loc[df.article_id==i.item(),'customer_id'].unique()\n#     vc2 = df.loc[(df.customer_id.isin(USERS))&(df.article_id!=i.item()),'article_id'].value_counts()\n#     pairs[i.item()] = [vc2.index[0], vc2.index[1], vc2.index[2]]\n# train['article_id2'] = train.article_id.map(pairs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-11T05:14:34.008018Z","iopub.execute_input":"2022-03-11T05:14:34.008352Z","iopub.status.idle":"2022-03-11T05:14:34.013498Z","shell.execute_reply.started":"2022-03-11T05:14:34.008317Z","shell.execute_reply":"2022-03-11T05:14:34.012503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# RECOMMENDATION OF PAIRED ITEMS\ntrain2 = train[['customer_id','article_id2']].copy()\ntrain2 = train2.loc[train2.article_id2.notnull()]\ntrain2 = train2.drop_duplicates(['customer_id','article_id2'])\ntrain2 = train2.rename({'article_id2':'article_id'},axis=1)","metadata":{"execution":{"iopub.status.busy":"2022-03-11T05:19:31.401125Z","iopub.execute_input":"2022-03-11T05:19:31.401471Z","iopub.status.idle":"2022-03-11T05:19:35.736163Z","shell.execute_reply.started":"2022-03-11T05:19:31.401436Z","shell.execute_reply":"2022-03-11T05:19:35.735182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CONCATENATE PAIRED ITEM RECOMMENDATION AFTER PREVIOUS PURCHASED RECOMMENDATIONS\ntrain = train[['customer_id','article_id']]\ntrain = pd.concat([train,train2],axis=0,ignore_index=True)\ntrain.article_id = train.article_id.astype('int32')\ntrain = train.drop_duplicates(['customer_id','article_id'])","metadata":{"execution":{"iopub.status.busy":"2022-03-11T05:19:43.052322Z","iopub.execute_input":"2022-03-11T05:19:43.053133Z","iopub.status.idle":"2022-03-11T05:19:49.144985Z","shell.execute_reply.started":"2022-03-11T05:19:43.053088Z","shell.execute_reply":"2022-03-11T05:19:49.143964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CONVERT RECOMMENDATIONS INTO SINGLE STRING\ntrain.article_id = ' 0' + train.article_id.astype('str')\npreds = pd.DataFrame( train.groupby('customer_id').article_id.sum().reset_index() )\npreds.columns = ['customer_id','prediction']\npreds.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-11T05:21:24.329432Z","iopub.execute_input":"2022-03-11T05:21:24.329791Z","iopub.status.idle":"2022-03-11T05:21:46.607025Z","shell.execute_reply.started":"2022-03-11T05:21:24.329758Z","shell.execute_reply":"2022-03-11T05:21:46.605841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Recommend Last Week's Most Popular Items","metadata":{}},{"cell_type":"code","source":"train = pd.read_parquet('train.pqt')\ntrain.t_dat = pd.to_datetime(train.t_dat)\ntrain = train.loc[train.t_dat >= pd.to_datetime('2020-09-16')]\ntop12 = ' 0' + ' 0'.join(train.article_id.value_counts().index.astype('str')[:12])\nprint(\"Last week's top 12 popular items:\")\nprint( top12 )","metadata":{"execution":{"iopub.status.busy":"2022-03-11T05:26:23.11574Z","iopub.execute_input":"2022-03-11T05:26:23.116752Z","iopub.status.idle":"2022-03-11T05:26:40.030651Z","shell.execute_reply.started":"2022-03-11T05:26:23.116698Z","shell.execute_reply":"2022-03-11T05:26:40.029629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Output the Predictions into CSV","metadata":{}},{"cell_type":"code","source":"sub = pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/sample_submission.csv')\nsub = sub[['customer_id']]\nsub = sub.merge(preds,on='customer_id', how='left').fillna('')\nsub.prediction = sub.prediction + top12\nsub.prediction = sub.prediction.str.strip()\nsub.prediction = sub.prediction.str[:131]\nsub.to_csv(f'submission.csv',index=False)\nsub.head()\n","metadata":{"execution":{"iopub.status.busy":"2022-03-11T05:38:55.649195Z","iopub.execute_input":"2022-03-11T05:38:55.649573Z","iopub.status.idle":"2022-03-11T05:39:20.080697Z","shell.execute_reply.started":"2022-03-11T05:38:55.649511Z","shell.execute_reply":"2022-03-11T05:39:20.079987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}}]}