{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## **
EDA
Exploratory Data Analysis. H&M
**","metadata":{}},{"cell_type":"markdown","source":"# **Table of Content**\n* [**1. First steps**](#First) \n* [**2. Articles**](#Articles) \n* [**3. Customers**](#Customers) \n* [**4. Transactions**](#Transactions) \n* [**5. Images with description and price**](#Images) \n","metadata":{}},{"cell_type":"markdown","source":"## **Intro**\n\n**The competition is dedicated to the product recomendations (H&M) **\n\n**Here we have different kinds of data that help us to get good recomendations: **\n\n๐Ÿ“ธ `images` - images of every article_id\n\n๐Ÿ™‹ `articles` - detailed metadata of every article_id\n\n๐Ÿ‘” `customers` - detailed metadata of every customer_id\n\n๐Ÿงพ `transactions_train` - purchases with details","metadata":{}},{"cell_type":"markdown","source":"## **1. First steps**","metadata":{}},{"cell_type":"markdown","source":"[**Go to Table of Content**](#Content)","metadata":{}},{"cell_type":"markdown","source":"** The first step as always: load the data :)**","metadata":{}},{"cell_type":"code","source":"# data analysis and wrangling\nimport pandas as pd\nimport numpy as np\n\n# visualization\nimport seaborn as sns\nfrom matplotlib import pyplot as plt\nfrom tqdm.notebook import tqdm","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:49:41.989182Z","iopub.execute_input":"2022-03-09T22:49:41.989637Z","iopub.status.idle":"2022-03-09T22:49:43.114392Z","shell.execute_reply.started":"2022-03-09T22:49:41.989498Z","shell.execute_reply":"2022-03-09T22:49:43.113628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"The Python Pandas packages helps us work with our datasets. We start by acquiring the training and testing datasets into Pandas DataFrames. We also combine these datasets to run certain operations on both datasets together.","metadata":{}},{"cell_type":"code","source":"articles = pd.read_csv(\"../input/h-and-m-personalized-fashion-recommendations/articles.csv\")\ncustomers = pd.read_csv(\"../input/h-and-m-personalized-fashion-recommendations/customers.csv\")\ntransactions = pd.read_csv(\"../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:49:43.115943Z","iopub.execute_input":"2022-03-09T22:49:43.116323Z","iopub.status.idle":"2022-03-09T22:51:06.587219Z","shell.execute_reply.started":"2022-03-09T22:49:43.116291Z","shell.execute_reply":"2022-03-09T22:51:06.585891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Let's look at the tables and try to get some outcomes about data inside.**","metadata":{}},{"cell_type":"markdown","source":"## **2. Articles**","metadata":{}},{"cell_type":"markdown","source":"[**Go to Table of Content**](#Content)","metadata":{}},{"cell_type":"markdown","source":"** This table contains all h&m articles with details such as a type of product, a color, a product group and other features.** \n** Article data description: **\n\n> `article_id` **: A unique identifier of every article.** \n> `product_code`, `prod_name` **: A unique identifier of every product and its name (not the same).** \n> `product_type`, `product_type_name` **: The group of product_code and its name** \n> `graphical_appearance_no`, `graphical_appearance_name` **: The group of graphics and its name** \n> `colour_group_code`, `colour_group_name` **: The group of color and its name** \n> `perceived_colour_value_id`, `perceived_colour_value_name`, `perceived_colour_master_id`, `perceived_colour_master_name` **: The added color info** \n> `department_no`, `department_name`: **: A unique identifier of every dep and its name** \n> `index_code`, `index_name`: **: A unique identifier of every index and its name** \n> `index_group_no`, `index_group_name`: **: A group of indeces and its name** \n> `section_no`, `section_name`: **: A unique identifier of every section and its name** \n> `garment_group_no`, `garment_group_name`: **: A unique identifier of every garment and its name** \n> `detail_desc`: **: Details** ","metadata":{}},{"cell_type":"code","source":"# preview the data\narticles.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:58:25.234263Z","iopub.execute_input":"2022-03-09T22:58:25.234658Z","iopub.status.idle":"2022-03-09T22:58:25.26779Z","shell.execute_reply.started":"2022-03-09T22:58:25.234623Z","shell.execute_reply":"2022-03-09T22:58:25.266871Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# preview the data\narticles.tail()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:58:52.281243Z","iopub.execute_input":"2022-03-09T22:58:52.281571Z","iopub.status.idle":"2022-03-09T22:58:52.312761Z","shell.execute_reply.started":"2022-03-09T22:58:52.281535Z","shell.execute_reply":"2022-03-09T22:58:52.311733Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles.describe","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:06.670657Z","iopub.execute_input":"2022-03-09T22:51:06.670891Z","iopub.status.idle":"2022-03-09T22:51:06.895494Z","shell.execute_reply.started":"2022-03-09T22:51:06.670861Z","shell.execute_reply":"2022-03-09T22:51:06.894611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Ladieswear accounts for a significant part of all dresses. Sportswear has the least portion.**","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(figsize=(15, 7))\nax = sns.histplot(data=articles, y='index_name', color='red')\nax.set_xlabel('count by index name')\nax.set_ylabel('index name')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:06.896742Z","iopub.execute_input":"2022-03-09T22:51:06.896948Z","iopub.status.idle":"2022-03-09T22:51:07.430525Z","shell.execute_reply.started":"2022-03-09T22:51:06.89692Z","shell.execute_reply":"2022-03-09T22:51:07.429488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** The garments grouped by index: Jersey fancy is the most frequent garment, especially for women and children. The next by number is accessories, many various accessories with low price.**","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(figsize=(15, 7))\nax = sns.histplot(data=articles, y='garment_group_name', color='orange', hue='index_group_name', multiple=\"stack\")\nax.set_xlabel('count by garment group')\nax.set_ylabel('garment group')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:07.432146Z","iopub.execute_input":"2022-03-09T22:51:07.432443Z","iopub.status.idle":"2022-03-09T22:51:08.50226Z","shell.execute_reply.started":"2022-03-09T22:51:07.432392Z","shell.execute_reply":"2022-03-09T22:51:08.50114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Now, pay attention to index group-index structure. Ladieswear and Children/Baby have subgroups.**","metadata":{}},{"cell_type":"code","source":"articles.groupby(['index_group_name', 'index_name']).count()['article_id']","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:08.504226Z","iopub.execute_input":"2022-03-09T22:51:08.504614Z","iopub.status.idle":"2022-03-09T22:51:08.677281Z","shell.execute_reply.started":"2022-03-09T22:51:08.504568Z","shell.execute_reply":"2022-03-09T22:51:08.676489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** And look at the product group-product structure. Accessories are really various, the most numerious: bags, earrings and hats. However, trousers prevail.**","metadata":{}},{"cell_type":"code","source":"pd.options.display.max_rows = None\narticles.groupby(['product_group_name', 'product_type_name']).count()['article_id']","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:08.678863Z","iopub.execute_input":"2022-03-09T22:51:08.679096Z","iopub.status.idle":"2022-03-09T22:51:08.8514Z","shell.execute_reply.started":"2022-03-09T22:51:08.679065Z","shell.execute_reply":"2022-03-09T22:51:08.850351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** And the table with number of unique values in columns:**","metadata":{}},{"cell_type":"code","source":"for col in articles.columns:\n if not 'no' in col and not 'code' in col and not 'id' in col:\n un_n = articles[col].nunique()\n print(f'n of unique {col}: {un_n}')","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:08.852883Z","iopub.execute_input":"2022-03-09T22:51:08.853943Z","iopub.status.idle":"2022-03-09T22:51:09.004009Z","shell.execute_reply.started":"2022-03-09T22:51:08.853889Z","shell.execute_reply":"2022-03-09T22:51:09.003213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **3. Customers**","metadata":{}},{"cell_type":"markdown","source":"[**Go to Table of Content**](#Content)","metadata":{}},{"cell_type":"markdown","source":"** Customers data description: **\n\n> `customer_id` **: A unique identifier of every customer** \n> `FN` **: 1 or missed ** \n> `Active` **: 1 or missed** \n> `club_member_status` **: Status in club** \n> `fashion_news_frequency` **: How often H&M may send news to customer** \n> `age` **: The current age** \n> `postal_code` **: Postal code of customer** ","metadata":{}},{"cell_type":"code","source":"pd.options.display.max_rows = 50\ncustomers.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:09.007183Z","iopub.execute_input":"2022-03-09T22:51:09.007684Z","iopub.status.idle":"2022-03-09T22:51:09.025933Z","shell.execute_reply.started":"2022-03-09T22:51:09.007637Z","shell.execute_reply":"2022-03-09T22:51:09.025147Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** There are no duplicates in ** `customers`","metadata":{}},{"cell_type":"code","source":"customers.shape[0] - customers['customer_id'].nunique()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:09.027455Z","iopub.execute_input":"2022-03-09T22:51:09.027807Z","iopub.status.idle":"2022-03-09T22:51:09.795788Z","shell.execute_reply.started":"2022-03-09T22:51:09.027759Z","shell.execute_reply":"2022-03-09T22:51:09.794768Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Here we have abnormal number of customers by one postal code. One has 120303, it might be encoded nan adress or smth like a huge distribution center, or pickup.**","metadata":{}},{"cell_type":"code","source":"data_postal = customers.groupby('postal_code', as_index=False).count().sort_values('customer_id', ascending=False)\ndata_postal.head(25)","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:09.797408Z","iopub.execute_input":"2022-03-09T22:51:09.797825Z","iopub.status.idle":"2022-03-09T22:51:11.91924Z","shell.execute_reply.started":"2022-03-09T22:51:09.797779Z","shell.execute_reply":"2022-03-09T22:51:11.918556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Ages, club_member_status are different, like customer_ids.**","metadata":{}},{"cell_type":"code","source":"customers[customers['postal_code']=='2c29ae653a9282cce4151bd87643c907644e09541abc28ae87dea0d1f6603b1c'].head(5)","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:11.920404Z","iopub.execute_input":"2022-03-09T22:51:11.920988Z","iopub.status.idle":"2022-03-09T22:51:12.221057Z","shell.execute_reply.started":"2022-03-09T22:51:11.920951Z","shell.execute_reply":"2022-03-09T22:51:12.220113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** The most common age is about 21-23**","metadata":{}},{"cell_type":"code","source":"import seaborn as sns\nfrom matplotlib import pyplot as plt\nsns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\nax = sns.histplot(data=customers, x='age', bins=50, color='orange')\nax.set_xlabel('Distribution of the customers age')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:12.222722Z","iopub.execute_input":"2022-03-09T22:51:12.223221Z","iopub.status.idle":"2022-03-09T22:51:12.931366Z","shell.execute_reply.started":"2022-03-09T22:51:12.223175Z","shell.execute_reply":"2022-03-09T22:51:12.930403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Status in H&M club. Almost every customer has an active club status, some of them begin to activate it (pre-create). A tiny part of customers abandoned the club.**","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\nax = sns.histplot(data=customers, x='club_member_status', color='orange')\nax.set_xlabel('Distribution of club member status')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:12.932582Z","iopub.execute_input":"2022-03-09T22:51:12.932842Z","iopub.status.idle":"2022-03-09T22:51:14.85648Z","shell.execute_reply.started":"2022-03-09T22:51:12.932811Z","shell.execute_reply":"2022-03-09T22:51:14.855462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Here we have three types for NO DATA. Let's unite these values.**","metadata":{}},{"cell_type":"code","source":"customers['fashion_news_frequency'].unique()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:14.858129Z","iopub.execute_input":"2022-03-09T22:51:14.858395Z","iopub.status.idle":"2022-03-09T22:51:14.995174Z","shell.execute_reply.started":"2022-03-09T22:51:14.858326Z","shell.execute_reply":"2022-03-09T22:51:14.993898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customers.loc[~customers['fashion_news_frequency'].isin(['Regularly', 'Monthly']), 'fashion_news_frequency'] = 'None'\ncustomers['fashion_news_frequency'].unique()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:14.99657Z","iopub.execute_input":"2022-03-09T22:51:14.996821Z","iopub.status.idle":"2022-03-09T22:51:15.192485Z","shell.execute_reply.started":"2022-03-09T22:51:14.996786Z","shell.execute_reply":"2022-03-09T22:51:15.191492Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pie_data = customers[['customer_id', 'fashion_news_frequency']].groupby('fashion_news_frequency').count()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:15.194571Z","iopub.execute_input":"2022-03-09T22:51:15.195307Z","iopub.status.idle":"2022-03-09T22:51:15.530496Z","shell.execute_reply.started":"2022-03-09T22:51:15.195253Z","shell.execute_reply":"2022-03-09T22:51:15.529754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Customers prefer not to get any messages about the current news.**","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\n# ax = sns.histplot(data=customers, x='fashion_news_frequency', color='orange')\n# ax = sns.pie(data=customers, x='fashion_news_frequency', color='orange')\ncolors = sns.color_palette('pastel')\nax.pie(pie_data.customer_id, labels=pie_data.index, colors = colors)\nax.set_facecolor('lightgrey')\nax.set_xlabel('Distribution of fashion news frequency')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:15.532029Z","iopub.execute_input":"2022-03-09T22:51:15.53257Z","iopub.status.idle":"2022-03-09T22:51:15.687329Z","shell.execute_reply.started":"2022-03-09T22:51:15.532521Z","shell.execute_reply":"2022-03-09T22:51:15.686143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **4. Transactions**","metadata":{}},{"cell_type":"markdown","source":"[**Go to Table of Content**](#Content)","metadata":{}},{"cell_type":"markdown","source":"** Transactions data description: **\n\n> `t_dat` **: A unique identifier of every customer** \n> `customer_id` **: A unique identifier of every customer ** **(in ** `customers` ** table)** \n> `article_id` **: A unique identifier of every article** **(in ** `articles` ** table)** \n> `price` **: Price of purchase** \n> `sales_channel_id` **: 1 or 2** ","metadata":{}},{"cell_type":"code","source":"transactions.head(50)","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:15.68912Z","iopub.execute_input":"2022-03-09T22:51:15.690184Z","iopub.status.idle":"2022-03-09T22:51:15.725133Z","shell.execute_reply.started":"2022-03-09T22:51:15.690126Z","shell.execute_reply":"2022-03-09T22:51:15.724053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Here we see outliers for price. **","metadata":{}},{"cell_type":"code","source":"pd.set_option('display.float_format', '{:.4f}'.format)\ntransactions.describe()['price']","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:15.726808Z","iopub.execute_input":"2022-03-09T22:51:15.727844Z","iopub.status.idle":"2022-03-09T22:51:18.748674Z","shell.execute_reply.started":"2022-03-09T22:51:15.727785Z","shell.execute_reply":"2022-03-09T22:51:18.747779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transactions.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:18.749774Z","iopub.execute_input":"2022-03-09T22:51:18.75179Z","iopub.status.idle":"2022-03-09T22:51:18.764247Z","shell.execute_reply.started":"2022-03-09T22:51:18.751751Z","shell.execute_reply":"2022-03-09T22:51:18.763195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\nax = sns.boxplot(data=transactions, x='price', color='orange')\nax.set_xlabel('Price outliers')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:18.765622Z","iopub.execute_input":"2022-03-09T22:51:18.765848Z","iopub.status.idle":"2022-03-09T22:51:23.309958Z","shell.execute_reply.started":"2022-03-09T22:51:18.765816Z","shell.execute_reply":"2022-03-09T22:51:23.309066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Top 10 customers by num of transactions. **","metadata":{}},{"cell_type":"code","source":"transactions_byid = transactions.groupby('customer_id').count()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:23.311345Z","iopub.execute_input":"2022-03-09T22:51:23.311697Z","iopub.status.idle":"2022-03-09T22:51:40.540984Z","shell.execute_reply.started":"2022-03-09T22:51:23.311594Z","shell.execute_reply":"2022-03-09T22:51:40.539967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transactions_byid.sort_values(by='price', ascending=False)['price'][:10]","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:40.542697Z","iopub.execute_input":"2022-03-09T22:51:40.542985Z","iopub.status.idle":"2022-03-09T22:51:41.140697Z","shell.execute_reply.started":"2022-03-09T22:51:40.54295Z","shell.execute_reply":"2022-03-09T22:51:41.139655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** However, comparing prices inside groups is more accurate, because accessories and trousers prices may vary largerly. **","metadata":{}},{"cell_type":"markdown","source":"** Get subset from articles and merge it to transactions. **","metadata":{}},{"cell_type":"code","source":"articles_for_merge = articles[['article_id', 'prod_name', 'product_type_name', 'product_group_name', 'index_name']]","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:41.142225Z","iopub.execute_input":"2022-03-09T22:51:41.142723Z","iopub.status.idle":"2022-03-09T22:51:41.151536Z","shell.execute_reply.started":"2022-03-09T22:51:41.142682Z","shell.execute_reply":"2022-03-09T22:51:41.150504Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articles_for_merge = transactions[['customer_id', 'article_id', 'price', 't_dat']].merge(articles_for_merge, on='article_id', how='left')","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:41.153197Z","iopub.execute_input":"2022-03-09T22:51:41.153476Z","iopub.status.idle":"2022-03-09T22:51:57.498598Z","shell.execute_reply.started":"2022-03-09T22:51:41.153406Z","shell.execute_reply":"2022-03-09T22:51:57.497363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Here we see outliers for group name prices. Lower/Upper/Full body have a huge price variance. I guess it could be like some unique collections, relative to casual ones. Some high price articles even belong to accessories group.**","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(25,18))\nax = sns.boxplot(data=articles_for_merge, x='price', y='product_group_name')\nax.set_xlabel('Price outliers', fontsize=22)\nax.set_ylabel('Index names', fontsize=22)\nax.xaxis.set_tick_params(labelsize=22)\nax.yaxis.set_tick_params(labelsize=22)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:51:57.502879Z","iopub.execute_input":"2022-03-09T22:51:57.503149Z","iopub.status.idle":"2022-03-09T22:52:25.556904Z","shell.execute_reply.started":"2022-03-09T22:51:57.503116Z","shell.execute_reply":"2022-03-09T22:52:25.555885Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Then look at boxplot prices according to accessories product group and find the reasons of high prices inside group.**\n\n** The largest outliers can be found among bags, which is logical enough. In addition, scarves and other accessories have articles with prices highly contrasting to the rest of garments.**","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(25,18))\n_ = articles_for_merge[articles_for_merge['product_group_name'] == 'Accessories']\nax = sns.boxplot(data=_, x='price', y='product_type_name')\nax.set_xlabel('Price outliers', fontsize=22)\nax.set_ylabel('Index names', fontsize=22)\nax.xaxis.set_tick_params(labelsize=22)\nax.yaxis.set_tick_params(labelsize=22)\ndel _\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:52:25.55861Z","iopub.execute_input":"2022-03-09T22:52:25.559302Z","iopub.status.idle":"2022-03-09T22:52:40.106451Z","shell.execute_reply.started":"2022-03-09T22:52:25.559262Z","shell.execute_reply":"2022-03-09T22:52:40.105441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** The index with the highest mean price is Ladieswear. With the lowest - children. **","metadata":{}},{"cell_type":"code","source":"articles_index = articles_for_merge[['index_name', 'price']].groupby('index_name').mean()\nsns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\nax = sns.barplot(x=articles_index.price, y=articles_index.index, color='orange', alpha=0.8)\nax.set_xlabel('Price by index')\nax.set_ylabel('Index')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:52:40.107651Z","iopub.execute_input":"2022-03-09T22:52:40.107957Z","iopub.status.idle":"2022-03-09T22:52:44.210593Z","shell.execute_reply.started":"2022-03-09T22:52:40.107924Z","shell.execute_reply":"2022-03-09T22:52:44.209704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Stationery has the lowest mean price, the highest - shoes. **","metadata":{}},{"cell_type":"code","source":"articles_index = articles_for_merge[['product_group_name', 'price']].groupby('product_group_name').mean()\nsns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\nax = sns.barplot(x=articles_index.price, y=articles_index.index, color='orange', alpha=0.8)\nax.set_xlabel('Price by product group')\nax.set_ylabel('Product group')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:52:44.211968Z","iopub.execute_input":"2022-03-09T22:52:44.212225Z","iopub.status.idle":"2022-03-09T22:52:48.532266Z","shell.execute_reply.started":"2022-03-09T22:52:44.212194Z","shell.execute_reply":"2022-03-09T22:52:48.53148Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Now check the mean price change in time for top 5 product groups by mean price: **\n>`Shoes` \n>`Garment Full body` \n>`Bags` \n>`Garment Lower body` \n>`Underwear/nightwear` ","metadata":{}},{"cell_type":"code","source":"articles_for_merge['t_dat'] = pd.to_datetime(articles_for_merge['t_dat'])","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:52:48.53394Z","iopub.execute_input":"2022-03-09T22:52:48.534482Z","iopub.status.idle":"2022-03-09T22:52:56.478053Z","shell.execute_reply.started":"2022-03-09T22:52:48.534432Z","shell.execute_reply":"2022-03-09T22:52:56.477108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"product_list = ['Shoes', 'Garment Full body', 'Bags', 'Garment Lower body', 'Underwear/nightwear']\ncolors = ['cadetblue', 'orange', 'mediumspringgreen', 'tomato', 'lightseagreen']\nk = 0\nf, ax = plt.subplots(3, 2, figsize=(20, 15))\nfor i in range(3):\n for j in range(2):\n try:\n product = product_list[k]\n articles_for_merge_product = articles_for_merge[articles_for_merge.product_group_name == product_list[k]]\n series_mean = articles_for_merge_product[['t_dat', 'price']].groupby(pd.Grouper(key=\"t_dat\", freq='M')).mean().fillna(0)\n series_std = articles_for_merge_product[['t_dat', 'price']].groupby(pd.Grouper(key=\"t_dat\", freq='M')).std().fillna(0)\n ax[i, j].plot(series_mean, linewidth=4, color=colors[k])\n ax[i, j].fill_between(series_mean.index, (series_mean.values-2*series_std.values).ravel(), \n (series_mean.values+2*series_std.values).ravel(), color=colors[k], alpha=.1)\n ax[i, j].set_title(f'Mean {product_list[k]} price in time')\n ax[i, j].set_xlabel('month')\n ax[i, j].set_xlabel(f'{product_list[k]}')\n k += 1\n except IndexError:\n ax[i, j].set_visible(False)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:52:56.479362Z","iopub.execute_input":"2022-03-09T22:52:56.479628Z","iopub.status.idle":"2022-03-09T22:53:20.268075Z","shell.execute_reply.started":"2022-03-09T22:52:56.479597Z","shell.execute_reply":"2022-03-09T22:53:20.26665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **5. Images with description and price**","metadata":{}},{"cell_type":"markdown","source":"[**Go to Table of Content**](#Content)","metadata":{}},{"cell_type":"markdown","source":"** Let's check the last purchases by max price and by min price **","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport matplotlib.image as mpimg","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:53:20.269389Z","iopub.execute_input":"2022-03-09T22:53:20.269726Z","iopub.status.idle":"2022-03-09T22:53:20.275687Z","shell.execute_reply.started":"2022-03-09T22:53:20.26969Z","shell.execute_reply":"2022-03-09T22:53:20.274338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"max_price_ids = transactions[transactions.t_dat==transactions.t_dat.max()].sort_values('price', ascending=False).iloc[:5][['article_id', 'price']]\nmin_price_ids = transactions[transactions.t_dat==transactions.t_dat.min()].sort_values('price', ascending=True).iloc[:5][['article_id', 'price']]","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:53:20.277136Z","iopub.execute_input":"2022-03-09T22:53:20.277401Z","iopub.status.idle":"2022-03-09T22:53:36.75899Z","shell.execute_reply.started":"2022-03-09T22:53:20.277369Z","shell.execute_reply":"2022-03-09T22:53:36.758027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Photos with description and price (top 5 max) **","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(1, 5, figsize=(20,10))\ni = 0\nfor _, data in max_price_ids.iterrows():\n desc = articles[articles['article_id'] == data['article_id']]['detail_desc'].iloc[0]\n desc_list = desc.split(' ')\n for j, elem in enumerate(desc_list):\n if j > 0 and j % 5 == 0:\n desc_list[j] = desc_list[j] + '\\n'\n desc = ' '.join(desc_list)\n img = mpimg.imread(f'../input/h-and-m-personalized-fashion-recommendations/images/0{str(data.article_id)[:2]}/0{int(data.article_id)}.jpg')\n ax[i].imshow(img)\n ax[i].set_title(f'price: {data.price:.2f}')\n ax[i].set_xticks([], [])\n ax[i].set_yticks([], [])\n ax[i].grid(False)\n ax[i].set_xlabel(desc, fontsize=10)\n i += 1\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:53:36.760288Z","iopub.execute_input":"2022-03-09T22:53:36.760666Z","iopub.status.idle":"2022-03-09T22:53:38.64245Z","shell.execute_reply.started":"2022-03-09T22:53:36.760631Z","shell.execute_reply":"2022-03-09T22:53:38.641702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"** Photos with description and price (top 5 min) **","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(1, 5, figsize=(20,10))\ni = 0\nfor _, data in min_price_ids.iterrows():\n desc = articles[articles['article_id'] == data['article_id']]['detail_desc'].iloc[0]\n desc_list = desc.split(' ')\n for j, elem in enumerate(desc_list):\n if j > 0 and j % 4 == 0:\n desc_list[j] = desc_list[j] + '\\n'\n desc = ' '.join(desc_list)\n img = mpimg.imread(f'../input/h-and-m-personalized-fashion-recommendations/images/0{str(data.article_id)[:2]}/0{int(data.article_id)}.jpg')\n ax[i].imshow(img)\n ax[i].set_title(f'price: {data.price:.4f}')\n ax[i].set_xlabel(desc, fontsize=10)\n ax[i].set_xticks([], [])\n ax[i].set_yticks([], [])\n ax[i].grid(False)\n i += 1\nplt.axis('off')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-09T22:53:38.643858Z","iopub.execute_input":"2022-03-09T22:53:38.644312Z","iopub.status.idle":"2022-03-09T22:53:40.383578Z","shell.execute_reply.started":"2022-03-09T22:53:38.644279Z","shell.execute_reply":"2022-03-09T22:53:40.382526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **
Thanks for reading! If you find this notebook useful or interesting, please, support with an upvote :)
**","metadata":{}}]}