{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.0.5"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# H&M EDA","metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle"}},{"cell_type":"code","source":"library(data.table)\nlibrary(ggplot2)","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:45:22.802368Z","iopub.execute_input":"2022-02-15T22:45:22.805128Z","iopub.status.idle":"2022-02-15T22:45:23.261658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Introduction\n\nIn this report, we will explore the data set from [H&M Competition](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/overview).","metadata":{}},{"cell_type":"markdown","source":"### Data Analysis","metadata":{}},{"cell_type":"code","source":"DAT_DIR <- '../input/h-and-m-personalized-fashion-recommendations'","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:45:43.549103Z","iopub.execute_input":"2022-02-15T22:45:43.597298Z","iopub.status.idle":"2022-02-15T22:45:43.610222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's first explore the articles file. A quick peek shows the article id, product code, product name, product type and product type name. There are more than 105K items. ","metadata":{}},{"cell_type":"code","source":"article_dt <- fread(file.path(DAT_DIR, 'articles.csv'))\narticle_dt","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:45:48.911878Z","iopub.execute_input":"2022-02-15T22:45:48.914117Z","iopub.status.idle":"2022-02-15T22:45:50.136537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We summarize the number of the product types and find there are still 132 product types. Therefore, we will show the size of each groups by the number of products within.","metadata":{}},{"cell_type":"code","source":"article_prod_types <- unique(article_dt[, .(product_type_no, product_type_name)])\narticle_prod_types","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:46:08.792359Z","iopub.execute_input":"2022-02-15T22:46:08.794365Z","iopub.status.idle":"2022-02-15T22:46:08.877294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see Trousers and Dress categories host most of the items while the some of the categories have very limited items.","metadata":{}},{"cell_type":"code","source":"article_dt[, .N, by=product_type_name][order(-N)]","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:46:15.018015Z","iopub.execute_input":"2022-02-15T22:46:15.020009Z","iopub.status.idle":"2022-02-15T22:46:15.095290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The customer data set contains about 1.4 million customers. In addition to the customer ID, we also have member status, activity indicator, fashion news frequency, age, and (hashed) postal code.","metadata":{}},{"cell_type":"code","source":"customer_dt <- fread(file.path(DAT_DIR, 'customers.csv'))\ncustomer_dt","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:46:22.161805Z","iopub.execute_input":"2022-02-15T22:46:22.163795Z","iopub.status.idle":"2022-02-15T22:46:30.128118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ggplot(customer_dt[, .(Active_ind=ifelse(is.na(Active), 0, 1))], aes(x=Active_ind)) +\n  geom_bar() +\n  labs(x=\"Active\")","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:48:37.535890Z","iopub.execute_input":"2022-02-15T22:48:37.538018Z","iopub.status.idle":"2022-02-15T22:48:40.963431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ggplot(customer_dt, aes(x=club_member_status)) +\n  geom_bar() +\n  labs(x=\"Club Member Status\")","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:49:00.750304Z","iopub.execute_input":"2022-02-15T22:49:00.752349Z","iopub.status.idle":"2022-02-15T22:49:02.316076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ggplot(customer_dt, aes(x=fashion_news_frequency)) +\n  geom_bar() +\n  labs(x=\"Fashion News Frequency\")","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:49:18.212195Z","iopub.execute_input":"2022-02-15T22:49:18.214195Z","iopub.status.idle":"2022-02-15T22:49:19.779083Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A review on the transaction file shows the transaction date, customer id, article id, price, and sales channel. A total of 31.8 million transactions are included.","metadata":{}},{"cell_type":"code","source":"train_txn <- fread(file.path(DAT_DIR, 'transactions_train.csv'))\ntrain_txn","metadata":{"execution":{"iopub.status.busy":"2022-02-15T22:49:26.204090Z","iopub.execute_input":"2022-02-15T22:49:26.206109Z","iopub.status.idle":"2022-02-15T22:50:04.061361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_txn_daily <- train_txn[, .N, by=t_dat]\n\nggplot(train_txn_daily, aes(x=t_dat, y=N)) +\n  geom_line() +\n  labs(x=\"Transaction Date\")","metadata":{"execution":{"iopub.status.busy":"2022-02-15T23:04:15.379457Z","iopub.execute_input":"2022-02-15T23:04:15.381506Z","iopub.status.idle":"2022-02-15T23:04:16.243669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The training transaction data set shows transactions from 09/2018 to 09/2020, or 2 years. The sale volume is quite dynamic as well. Let's find out the top sale dates.","metadata":{}},{"cell_type":"code","source":"train_txn_daily[order(-N)]","metadata":{"execution":{"iopub.status.busy":"2022-02-15T23:04:20.112041Z","iopub.execute_input":"2022-02-15T23:04:20.113367Z","iopub.status.idle":"2022-02-15T23:04:20.153386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It seems some (possible) holidays are driving the sales.","metadata":{}},{"cell_type":"code","source":"train_txn_channel <- train_txn[, .N, by=sales_channel_id]\nggplot(train_txn_channel, aes(x=factor(sales_channel_id), y=N)) +\n  geom_bar(stat='identity') +\n  labs(x=\"Channel\")","metadata":{"execution":{"iopub.status.busy":"2022-02-15T23:04:27.419113Z","iopub.execute_input":"2022-02-15T23:04:27.420466Z","iopub.status.idle":"2022-02-15T23:04:28.769700Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It seems only two channels are included and Channel 2 is significantly more than channel 1.","metadata":{}},{"cell_type":"code","source":"train_txn[, .(min_price=min(price), max_price=max(price))]","metadata":{"execution":{"iopub.status.busy":"2022-02-15T23:04:32.819066Z","iopub.execute_input":"2022-02-15T23:04:32.820552Z","iopub.status.idle":"2022-02-15T23:04:32.917897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Most of the items have price in 0.01-0.1 bucket, while price range of 0.001 to 0.01 and 0.1 to 1 are quite popular as well. ","metadata":{}},{"cell_type":"code","source":"breaks <- c(0, 1e-4, 1e-3, 1e-2, 1e-1, 1)\nlabels <- c('<1e-5', '<1e-3', '<1e-2', '<1e-1', '<1')\ntrain_txn[, price_grp:=cut(price, breaks=breaks, labels=labels)]\ntrain_txn_price <- train_txn[, .N, by=price_grp]\nggplot(train_txn_price, aes(x=price_grp, y=N)) +\n  geom_bar(stat='identity') +\n  labs(x=\"price group\")","metadata":{"execution":{"iopub.status.busy":"2022-02-15T23:04:51.220095Z","iopub.execute_input":"2022-02-15T23:04:51.221957Z","iopub.status.idle":"2022-02-15T23:04:56.630753Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}