","metadata":{}},{"cell_type":"markdown","source":"In this notebook, I develop product recommendations based on data from previous H&M transactions, as well as from customer and product meta data. The available meta data spans from simple data, such as garment type and customer age, to text data from product descriptions, to image data from garment images.","metadata":{}},{"cell_type":"markdown","source":"
Throughout this notebook, the plots have been made using plotly, as it provides efficient interactive plots, implying better analysis. Sit back, toggle the sidebar, and enjoy!
","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom tabulate import tabulate\nfrom plotly.offline import plot, iplot, init_notebook_mode\nimport plotly.express as px\nimport plotly.graph_objects as go\ninit_notebook_mode(connected=True)\nfrom datetime import datetime","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-03-16T11:26:08.785485Z","iopub.execute_input":"2022-03-16T11:26:08.785815Z","iopub.status.idle":"2022-03-16T11:26:11.386177Z","shell.execute_reply.started":"2022-03-16T11:26:08.785782Z","shell.execute_reply":"2022-03-16T11:26:11.385236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
1.1 | Looking at the Data
\n
\n","metadata":{}},{"cell_type":"code","source":"l = os.listdir('/kaggle/input/h-and-m-personalized-fashion-recommendations/')\nprint(f\"Folders: {l}\")","metadata":{"execution":{"iopub.status.busy":"2022-03-16T11:24:16.428153Z","iopub.execute_input":"2022-03-16T11:24:16.428591Z","iopub.status.idle":"2022-03-16T11:24:16.441167Z","shell.execute_reply.started":"2022-03-16T11:24:16.428559Z","shell.execute_reply":"2022-03-16T11:24:16.43994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First let us focus on the three kinds of csv's available, `articles.csv`, `transactions_train.csv` and `customers.csv`.","metadata":{}},{"cell_type":"code","source":"articles = pd.read_csv('/kaggle/input/h-and-m-personalized-fashion-recommendations/articles.csv')\narticles.info()","metadata":{"execution":{"iopub.status.busy":"2022-03-16T11:26:18.075801Z","iopub.execute_input":"2022-03-16T11:26:18.077020Z","iopub.status.idle":"2022-03-16T11:26:19.260351Z","shell.execute_reply.started":"2022-03-16T11:26:18.076955Z","shell.execute_reply":"2022-03-16T11:26:19.259529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-16T11:24:17.441048Z","iopub.execute_input":"2022-03-16T11:24:17.441302Z","iopub.status.idle":"2022-03-16T11:24:17.469745Z","shell.execute_reply.started":"2022-03-16T11:24:17.441273Z","shell.execute_reply":"2022-03-16T11:24:17.468725Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers = pd.read_csv('/kaggle/input/h-and-m-personalized-fashion-recommendations/customers.csv')\ncustomers.info()","metadata":{"execution":{"iopub.status.busy":"2022-03-16T11:26:21.944897Z","iopub.execute_input":"2022-03-16T11:26:21.945209Z","iopub.status.idle":"2022-03-16T11:26:28.488105Z","shell.execute_reply.started":"2022-03-16T11:26:21.945179Z","shell.execute_reply":"2022-03-16T11:26:28.486334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-16T11:24:22.796264Z","iopub.execute_input":"2022-03-16T11:24:22.796484Z","iopub.status.idle":"2022-03-16T11:24:22.818959Z","shell.execute_reply.started":"2022-03-16T11:24:22.796454Z","shell.execute_reply":"2022-03-16T11:24:22.817878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transactions = pd.read_csv('/kaggle/input/h-and-m-personalized-fashion-recommendations/transactions_train.csv')\ntransactions.info()","metadata":{"execution":{"iopub.status.busy":"2022-03-16T11:26:32.405215Z","iopub.execute_input":"2022-03-16T11:26:32.405536Z","iopub.status.idle":"2022-03-16T11:27:42.890130Z","shell.execute_reply.started":"2022-03-16T11:26:32.405488Z","shell.execute_reply":"2022-03-16T11:27:42.889079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transactions.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
1.2 | Missing Values
\n
","metadata":{}},{"cell_type":"code","source":"def missing_values(data):\n total = data.isnull().sum().sort_values(ascending = False)\n percent = (data.isnull().sum()/data.isnull().count()*100).sort_values(ascending = False)\n return pd.concat([total, percent], axis=1, keys=['Number of Missing Values', 'Percentage'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Articles","metadata":{}},{"cell_type":"code","source":"articles_missing=missing_values(articles)\narticles_missing.loc[articles_missing['Percentage']>0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Customers","metadata":{}},{"cell_type":"code","source":"customers_missing=missing_values(customers)\ncustomers_missing.loc[customers_missing['Percentage']>0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Transactions","metadata":{}},{"cell_type":"code","source":"transactions_missing=missing_values(transactions)\ntransactions_missing.loc[transactions_missing['Percentage']>0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Based on the above, we can see that:\n* In the articles dataframe, the only missing data is for *detail_desc*, the detailed description of the article.\n* In the customers dataframe,a significant amount of data is present in *FN* and *Active*.\n* There is no missing data in transactions dataframe.","metadata":{}},{"cell_type":"markdown","source":"# 2 | Data Visualization","metadata":{}},{"cell_type":"markdown","source":"# **2.1 Articles**","metadata":{}},{"cell_type":"code","source":"fig = px.pie(articles, values='article_id', \n title='Distribution by Index Group Name',\n names='index_group_name',\n color_discrete_sequence=px.colors.sequential.RdBu,\n hover_data=['index_group_name'],\n labels={'index_group_name':'Index Group Name'},\n height=450)\nfig.show()","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Ladieswear comprises of the maximum artciles, followed by Baby/Children on a close second.","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(articles, x='garment_group_name',color=\"index_group_name\", \n title=\"Index Group Name per Garment Group Name\",\n color_discrete_sequence=px.colors.sequential.Agsunset,\n height=600)\nfig.show()\n","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"*P.s. For some reason I had a hard time plotting this stacked plot, after numerous efforts, I owe the succes to [this blog by Vaclav Dekanovsky.](https://towardsdatascience.com/histograms-with-plotly-express-complete-guide-d483656c5ad7)*","metadata":{}},{"cell_type":"markdown","source":"* The maximum garments sold are of the group Jersey Fancy, followed by Accessories.\n* Around 40% of Jersey Fancy comprises of those of Baby/Children.\n* Around 70% of the shirts sold are in Menswear.\n* Group Divided comprises of almost 50% of the Dresses Ladies.\n","metadata":{}},{"cell_type":"code","source":"df1 = articles.groupby([\"section_name\"]).count().reset_index()\n\nfig = px.bar(df1,\n x=articles.groupby([\"section_name\"]).size(),\n y=\"section_name\",\n color='section_name',\n title='Distribution by Section Name',\n hover_data=['section_name'],\n text_auto='.2s',\n labels={'section_name':'Section Name',\"x\":\"Count\"},\n orientation='h',\n color_discrete_sequence=px.colors.diverging.Temps, \n height=1000)\nfig.update_traces(textfont_size=11, textangle=0, textposition=\"outside\", cliponaxis=False)\nfig.update_layout(xaxis_title = 'Count')\nfig.show()","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* The highest number of articles belong to the section Womens Everyday Collection, followed by Divided collection, and next Baby Essentials and Complements.\n* The least belong to 'Men Other' group.\n* There are almost 3 times Womens shoes than Men. \n* The highest number of articles in Menswear belong to 'Men Underwear', followed by 'Mens Suits and Tailoring.","metadata":{}},{"cell_type":"code","source":"df4= articles.groupby([\"index_name\"])[\"article_id\"].nunique()\ndf4 = pd.DataFrame({'IndexName': df4.index,\n 'Articles': df4.values\n })\nlabels=df4['IndexName']\nvalues=df4['Articles']\nfig = px.pie(labels, values = values, hole = 0.35,\n names = labels,\n title = 'Distribution by Index Name',\n color_discrete_sequence =px.colors.cyclical.mygbm\n )\nfig.show()","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df5 = articles.groupby([\"perceived_colour_master_name\"]).count().reset_index()\ncolors = ['#F5F5DC','#000000','#023e8a','#168aad','#7f5539','#90be6d','#b7b7a4','#606c38','#9d4edd','#b7b7a4','#9e2a2b','#f77f00','#ffafcc','#d00000','#34a0a4','#3e1f47','#ffffff','#fcbf49','#dddf00','#9e0059'] \nfig = px.bar(df5,\n y=articles.groupby([\"perceived_colour_master_name\"]).size(),\n x=\"perceived_colour_master_name\",\n color='perceived_colour_master_name',\n hover_data=['perceived_colour_master_name'],\n text_auto='.2s',\n color_discrete_sequence =colors,\n title='Distribution by Percieved Color Master Name',\n labels={'perceived_colour_master_name':'Percieved Color Master Name'})\nfig.update_traces(textfont_size=11, textangle=0, textposition=\"outside\", cliponaxis=False)\nfig.update_layout(yaxis_title = 'Count')\nfig.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-03-16T12:05:00.454952Z","iopub.execute_input":"2022-03-16T12:05:00.455415Z","iopub.status.idle":"2022-03-16T12:05:00.768753Z","shell.execute_reply.started":"2022-03-16T12:05:00.455376Z","shell.execute_reply":"2022-03-16T12:05:00.766927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **2.2 Customers**","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(customers, \n x=\"age\", \n range_x=[\"0\",\"100\"],\n title=\"Age Distribution\",\n height=450,\n color_discrete_sequence =px.colors.cyclical.IceFire\n )\nfig.update_layout(bargap=0.2)\nfig.show()","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df6 = customers.groupby([\"age\"]).count().reset_index()\nage_groups = pd.cut(customers[\"age\"], bins=[0, 20, 40, 60, 80,100])\nfig = px.histogram(age_groups.astype('str'), \n x=\"age\", \n title=\"Age Distribution in Groups\",\n height=350,\n text_auto=True,\n color_discrete_sequence =px.colors.sequential.Jet\n )\n\nfig.show()","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* The highest number of customers prevail from ages 20-40. This is about 52% of the total customers.","metadata":{}},{"cell_type":"markdown","source":"# **2.3 Transactions**","metadata":{}},{"cell_type":"code","source":"cus_transactions = transactions.groupby('customer_id').count()\nprint(\"Top 10 Customers by Number of Items Purchased: \")\ncus_transactions.sort_values(by='price', ascending=False)['price'][:10]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"merged_at = articles[['article_id', 'prod_name', 'product_type_name', 'product_group_name', 'index_name']]\nmerged_at = transactions.merge(merged_at, on='article_id', how='left')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df7 = merged_at[['index_name', 'price']].groupby('index_name').sum()\nfig = px.bar(df7,\n x=df7.price,\n y=df7.index,\n title='Sales per Index Name',\n text_auto='.2s',\n orientation='h',\n color_discrete_sequence=px.colors.diverging.Temps, \n height=500,\n labels={'t_dat':'Transaction date', 'price':'Sales'})\nfig.update_traces(textfont_size=11, textangle=0, textposition=\"outside\", cliponaxis=False)\nfig.update_layout(xaxis_title = 'Total Sale',yaxis_title ='Index Name')\nfig.show()","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df8 = transactions.groupby([\"t_dat\"])[\"price\"].sum().reset_index()\ndf8[\"t_dat\"] = df8[\"t_dat\"].apply(lambda x: datetime.strptime(x, '%Y-%m-%d'))\n\nfig = px.line(df8,\n x=df8[\"t_dat\"],\n y=df8[\"price\"],\n title='Sales Amount per Transaction date',\n color_discrete_sequence=px.colors.diverging.Portland, \n height=400,\n labels={'t_dat':'Transaction date', 'price':'Sales'})\nfig.update_layout(xaxis_title = 'Transaction Date',yaxis_title ='Total Sales Amount')\nfig.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-03-16T11:27:52.024997Z","iopub.execute_input":"2022-03-16T11:27:52.025494Z","iopub.status.idle":"2022-03-16T11:27:56.093223Z","shell.execute_reply.started":"2022-03-16T11:27:52.025460Z","shell.execute_reply":"2022-03-16T11:27:56.092416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* The highest sales was made on September 28, 2019.","metadata":{}},{"cell_type":"code","source":"cust_age= pd.DataFrame(customers, columns = ['customer_id','age'])\ntotal_tran = pd.merge(cust_age,transactions, how='right', on='customer_id')\ndatanew= pd.DataFrame(total_tran, columns = ['price','t_dat','sales_channel_id','article_id'])\ndfp = datanew.groupby([\"t_dat\", \"sales_channel_id\"])[\"price\"].sum().reset_index()\n\nfig_bar = px.line(dfp, x=\"t_dat\", y=\"price\", color=\"sales_channel_id\",\n animation_frame=\"sales_channel_id\", animation_group=\"t_dat\", \n title='Sales Amount per Sales Channel',\n color_discrete_sequence=px.colors.diverging.Portland,\n labels={'sales_channel_id': 'Sales channel ID','t_dat':'Transaction date', 'price':'Sales'})\nfig_bar.update_yaxes(showgrid=False),\nfig_bar.update_layout(\n hovermode=\"x unified\",\n xaxis_tickangle=360,\n xaxis_title='Sales Channel', yaxis_title=\"Sales Amount\",\n legend=dict(orientation=\"h\", yanchor=\"bottom\", y=1.02, xanchor=\"right\", x=1)\n )\nfig_bar.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-16T12:03:43.708633Z","iopub.execute_input":"2022-03-16T12:03:43.709648Z","iopub.status.idle":"2022-03-16T12:04:15.807599Z","shell.execute_reply.started":"2022-03-16T12:03:43.709611Z","shell.execute_reply":"2022-03-16T12:04:15.806622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.histogram(dfp, \n y=\"price\", \n x=\"sales_channel_id\",\n color_discrete_sequence =['#fe6d73'],\n barmode=\"group\", \n animation_frame=\"t_dat\", \n title='Day Wise Sales per Sales Channel',\n range_x=[0,3]\n)\nfig.update_layout(xaxis_title='Sales Channel', yaxis_title=\"Sales Amount\",bargap=0.5\n )\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-16T12:08:38.978788Z","iopub.execute_input":"2022-03-16T12:08:38.979103Z","iopub.status.idle":"2022-03-16T12:08:42.009462Z","shell.execute_reply.started":"2022-03-16T12:08:38.979059Z","shell.execute_reply":"2022-03-16T12:08:42.008212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**I will soon be updating the notebook with prediction section as well. Do let me know your reviews and suggestions.**\n\n**Until next time!**","metadata":{}},{"cell_type":"markdown","source":">\"**And, when you can’t go back, you have to worry only about the best way of moving forward.**\"\n *- Paulo Coelho, The Alchemist*","metadata":{}}]}
