{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-03-13T06:42:14.545443Z","iopub.execute_input":"2022-03-13T06:42:14.545783Z","iopub.status.idle":"2022-03-13T06:42:14.575055Z","shell.execute_reply.started":"2022-03-13T06:42:14.5457Z","shell.execute_reply":"2022-03-13T06:42:14.574261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# I. Overview\n## 1. Problem\nIn recent time, we are reported that deep model(deep learning, tree model,...) get better performance than traditional model in recommendation. But by viewing [best score notebooks](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/code?competitionId=31254&sortBy=scoreDescending), I am surprised that traditional approaches (trending, most purchased items for each group, ...) outperform deep model in this data. I also tried to create some deep models in [this notebook](https://www.kaggle.com/astrung/recbole-lstm-sequential-for-recomendation-tutorial) and [this notebook](https://www.kaggle.com/astrung/lstm-sequential-modelwith-item-features-tutorial), but it still get lower score than traditional model. Is it weird ? \n\nI think our data may has something which is different from others data in publications, so i start this notebook to investigate the problem. This is my hypothesis: our data has too many customers which theirs preference can not predicted from past data, or data for each user is too small for deep model.\n\nBy EDA on customers, we can see that most of users in test data are cold start user, or user who is inactive in a long time, so their data isn't enough, or doesn't reflect their interest correctly:\n\n* Inactive users: In our test data(1371980 users), **509256 users(37% of all user) have been inactive for a year**-they stop buying from before 2020. Beside, **373171 users(27% of all user) have been inactive for 3 months**(Sep, Aug, July) or more in 2020. In other words, **882427(64% of all user) users have been inactive in our test data**, they have gave up in a long time, and then they reappear in our test set. In most scenarios, customers give up on a system because their interest/priority factors have been changed, so their past data isn't enough to inference their desire now. Example: I gave up on H&M a year ago because i have changed my fashion style, but now i have seen a sale-off/advertising campaign/hot trending in H&M, so i comeback. With this type of user, my advice is avoiding use their past data correctly for recommendation. You should use items in sale-off/advertising campaign/hot trending, because they are likely the reason they come back.The more time they disappeared in past data, the more challenged correct prediction for their interest. \n\n* Cold-start users: In recommendation, cold-start users are some users who don't have past data, or their past data is too small for inference correct recommendation. In this notebook, I demonstrate that most of users have very small transaction data. In detail, in 862724 users who have transactions in 2020, 547161 users(63% all users) have number transactions < 10. In other words, we have too many cold-start users in test data, and deep models don't work with this type of user. Deep model only works with users who have big number of transactions, not cold-start users or inactive users.\n\n* Another challenge is low frequent in transaction. In average, each user will buy only 4 items in a month/1 item in a week. so we need to predict unique correct item in one week test data. It is very challenged, but it is ussual in practice.\n\n## 2.Solution\nDeep model only works with active/non cold start users. But in our test data, only there are only 9% users who satisfy this condition. So can we give up deep model ?\n\n**In this case, my advice is using hybrid approach. You should use sale-off/advertising campaign/hot trending for 92% users who is inactive/cold-start in test data, then you can use deep model with remaining users**. We should only use deep models for right use cases.By combining deep model into general approach, i got some higher score than origin general approach. After cleaning my notebook, I will publish it as soon as posible. \n\n# Dataset for anyone who want to use directly inactive/cold start user\nI already publish inactive/cold-start users in this customer metadata dataset, so you can use inactive/cold-start information directly for your hybrid approach:\n* https://www.kaggle.com/astrung/hm-customer-metadata\n\nIf you want find more ideal about hybrid approach, you can check comments in my thread:\nhttps://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/312653\n\nI also create a dataset for items, which extract sale-off/advertising campaign from transaction data. You can get sale-off information from this data for your general recommendation. If you want to find more information about article and campaign, please check my following notebook:\n* sale-off item dataset: https://www.kaggle.com/astrung/hm-article-capaign\n* sale-off item notebook: https://www.kaggle.com/astrung/eda-extract-campaign-from-transactions\n\n","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(r'../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv')\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:42:14.57682Z","iopub.execute_input":"2022-03-13T06:42:14.577589Z","iopub.status.idle":"2022-03-13T06:43:22.654025Z","shell.execute_reply.started":"2022-03-13T06:42:14.577548Z","shell.execute_reply":"2022-03-13T06:43:22.653175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['t_dat'] = pd.to_datetime(df['t_dat'], format=\"%Y-%m-%d\")\ndf['month'] = df['t_dat'].dt.strftime('%m')\ndf['year'] = df['t_dat'].dt.strftime('%Y')\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:43:22.655383Z","iopub.execute_input":"2022-03-13T06:43:22.6577Z","iopub.status.idle":"2022-03-13T06:51:09.786254Z","shell.execute_reply.started":"2022-03-13T06:43:22.657641Z","shell.execute_reply":"2022-03-13T06:51:09.785101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df[df['year'] == '2020']\ndf.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:09.788978Z","iopub.execute_input":"2022-03-13T06:51:09.789251Z","iopub.status.idle":"2022-03-13T06:51:21.077278Z","shell.execute_reply.started":"2022-03-13T06:51:09.789221Z","shell.execute_reply":"2022-03-13T06:51:21.076662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test_user = pd.read_csv(r'../input/h-and-m-personalized-fashion-recommendations/sample_submission.csv')\ndf_test_user.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:21.078774Z","iopub.execute_input":"2022-03-13T06:51:21.079213Z","iopub.status.idle":"2022-03-13T06:51:26.140591Z","shell.execute_reply.started":"2022-03-13T06:51:21.079172Z","shell.execute_reply":"2022-03-13T06:51:26.139423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Find inactive user","metadata":{}},{"cell_type":"markdown","source":"First, let count number of transaction in each month for all users","metadata":{}},{"cell_type":"code","source":"df_month_avg_item_per_u = df.groupby(['customer_id', 'month'])['price'].count().unstack().reset_index()\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:26.142264Z","iopub.execute_input":"2022-03-13T06:51:26.14255Z","iopub.status.idle":"2022-03-13T06:51:34.916239Z","shell.execute_reply.started":"2022-03-13T06:51:26.142518Z","shell.execute_reply":"2022-03-13T06:51:34.915301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then merge with test data. Test data has more rows than our transaction data. It means we have some users who don't have any transactions in 2020 in test data. Let check how many users like it","metadata":{}},{"cell_type":"code","source":"df_month_avg_item_per_u = pd.merge(df_month_avg_item_per_u, df_test_user[['customer_id']], on='customer_id', how='outer')\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:34.918118Z","iopub.execute_input":"2022-03-13T06:51:34.918467Z","iopub.status.idle":"2022-03-13T06:51:37.228019Z","shell.execute_reply.started":"2022-03-13T06:51:34.918424Z","shell.execute_reply":"2022-03-13T06:51:37.226848Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_month_avg_item_per_u['num_missing_months'] = df_month_avg_item_per_u.isnull().sum(axis=1)\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:37.229509Z","iopub.execute_input":"2022-03-13T06:51:37.229775Z","iopub.status.idle":"2022-03-13T06:51:37.563766Z","shell.execute_reply.started":"2022-03-13T06:51:37.229745Z","shell.execute_reply":"2022-03-13T06:51:37.562679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**num_missing_months=9 means users don't have any transactions in 2020(9 months of 2020). There is 37% users with this condition in test data**","metadata":{}},{"cell_type":"code","source":"num_missing_year = len(df_month_avg_item_per_u[df_month_avg_item_per_u['num_missing_months'] == 9])\nprint(num_missing_year)\nprint(num_missing_year/len(df_test_user))","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:37.56516Z","iopub.execute_input":"2022-03-13T06:51:37.565404Z","iopub.status.idle":"2022-03-13T06:51:37.647179Z","shell.execute_reply.started":"2022-03-13T06:51:37.565375Z","shell.execute_reply":"2022-03-13T06:51:37.645913Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_month_avg_item_per_u = df_month_avg_item_per_u.fillna(0)\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:37.650577Z","iopub.execute_input":"2022-03-13T06:51:37.650869Z","iopub.status.idle":"2022-03-13T06:51:38.100097Z","shell.execute_reply.started":"2022-03-13T06:51:37.650841Z","shell.execute_reply":"2022-03-13T06:51:38.098917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Inactive users with more than 3 consecutive months will be still masked as 3**","metadata":{}},{"cell_type":"code","source":"def cal_inactive_months(x):\n    if x['09'] > 0:\n        return 0\n    elif x['09'] == 0 and x['08'] > 0:\n        return 1\n    elif x['09'] == 0 and x['08'] == 0 and x['07'] > 0:\n        return 2\n    elif x['09'] == 0 and x['08'] == 0 and x['07'] == 0:\n        return 3\n    else:\n        return 4\n\ndf_month_avg_item_per_u['lastest_inactive_months'] = df_month_avg_item_per_u[\n    df_month_avg_item_per_u.columns.difference(['customer_id', 'num_missing_months'])].apply(\n    lambda x: cal_inactive_months(x), axis=1)\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:51:38.101439Z","iopub.execute_input":"2022-03-13T06:51:38.101672Z","iopub.status.idle":"2022-03-13T06:52:43.811993Z","shell.execute_reply.started":"2022-03-13T06:51:38.101644Z","shell.execute_reply":"2022-03-13T06:52:43.810922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**In below cell, we see that 50% of users disappers in 8 or more months before reactive. It is another challenge in our data**","metadata":{}},{"cell_type":"code","source":"print(df_month_avg_item_per_u.num_missing_months.value_counts())\nprint(df_month_avg_item_per_u.num_missing_months.describe())\ndf_month_avg_item_per_u.num_missing_months.hist()","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:43.813831Z","iopub.execute_input":"2022-03-13T06:52:43.814159Z","iopub.status.idle":"2022-03-13T06:52:44.202701Z","shell.execute_reply.started":"2022-03-13T06:52:43.814119Z","shell.execute_reply":"2022-03-13T06:52:44.202015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***In following cell, we see that 64% of users disappeared in recent 3 months (Sep, Aug, July) before reappear in testdata. ***","metadata":{}},{"cell_type":"code","source":"print(df_month_avg_item_per_u.lastest_inactive_months.value_counts())\nprint(df_month_avg_item_per_u.lastest_inactive_months.describe())\ndf_month_avg_item_per_u.lastest_inactive_months.hist()","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:44.203772Z","iopub.execute_input":"2022-03-13T06:52:44.204762Z","iopub.status.idle":"2022-03-13T06:52:44.69147Z","shell.execute_reply.started":"2022-03-13T06:52:44.204722Z","shell.execute_reply":"2022-03-13T06:52:44.690637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Missing 3 months\")\nnum_missing_3months = len(df_month_avg_item_per_u[df_month_avg_item_per_u['lastest_inactive_months'] == 3])\nprint(num_missing_3months)\nprint(num_missing_3months/len(df_test_user))\nprint(\"Missing 2 months\")\nnum_missing_2months = len(df_month_avg_item_per_u[df_month_avg_item_per_u['lastest_inactive_months'] == 2])\nprint(num_missing_2months)\nprint(num_missing_2months/len(df_test_user))\nprint(\"Missing 1 months\")\nnum_missing_1months = len(df_month_avg_item_per_u[df_month_avg_item_per_u['lastest_inactive_months'] == 1])\nprint(num_missing_1months)\nprint(num_missing_1months/len(df_test_user))","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:44.692707Z","iopub.execute_input":"2022-03-13T06:52:44.692942Z","iopub.status.idle":"2022-03-13T06:52:44.941481Z","shell.execute_reply.started":"2022-03-13T06:52:44.692915Z","shell.execute_reply":"2022-03-13T06:52:44.940234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Create a dataframe for inactive user, in order to merge with other information about user","metadata":{}},{"cell_type":"code","source":"df_month_avg_item_per_u['active_status'] = 'active'\ndf_month_avg_item_per_u.loc[(df_month_avg_item_per_u.num_missing_months == 9),'active_status']='inactive_in_year'\ndf_month_avg_item_per_u.loc[(df_month_avg_item_per_u.num_missing_months < 9) &\n                            (df_month_avg_item_per_u.lastest_inactive_months == 3),\n                            'active_status']='inactive_in_3_months_or_more'\ndf_month_avg_item_per_u.loc[\n    (df_month_avg_item_per_u.lastest_inactive_months == 2),'active_status']='inactive_in_2_months'\ndf_month_avg_item_per_u.loc[\n    (df_month_avg_item_per_u.lastest_inactive_months == 1),'active_status']='inactive_in_1_month'\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:44.943052Z","iopub.execute_input":"2022-03-13T06:52:44.943372Z","iopub.status.idle":"2022-03-13T06:52:45.11986Z","shell.execute_reply.started":"2022-03-13T06:52:44.943331Z","shell.execute_reply":"2022-03-13T06:52:45.118867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_active_user = df_month_avg_item_per_u[['customer_id', 'num_missing_months', 'lastest_inactive_months', 'active_status']].copy()\ndf_active_user","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:45.121181Z","iopub.execute_input":"2022-03-13T06:52:45.121434Z","iopub.status.idle":"2022-03-13T06:52:45.49755Z","shell.execute_reply.started":"2022-03-13T06:52:45.121386Z","shell.execute_reply":"2022-03-13T06:52:45.4964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Find coldstart customer","metadata":{}},{"cell_type":"markdown","source":"**First, count number of transaction. We will mask users with number of transactions <= 10 are cold start user. They are users with too small data for correct recommendation**","metadata":{}},{"cell_type":"code","source":"df_avg_item_per_u = df.groupby(['customer_id'])['price'].count().reset_index()\ndf_avg_item_per_u.columns = ['customer_id', 'num_transactions']\ndf_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:45.499396Z","iopub.execute_input":"2022-03-13T06:52:45.499797Z","iopub.status.idle":"2022-03-13T06:52:50.447929Z","shell.execute_reply.started":"2022-03-13T06:52:45.499766Z","shell.execute_reply":"2022-03-13T06:52:50.44685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In test data, we have some users who dont have any transactions in 2020. Let add it into our dataframe, and label their number of transaction as 0","metadata":{}},{"cell_type":"code","source":"df_avg_item_per_u = pd.merge(df_avg_item_per_u, df_test_user[['customer_id']], on='customer_id', how='outer')\ndf_avg_item_per_u = df_avg_item_per_u.fillna(0)\ndf_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:50.449205Z","iopub.execute_input":"2022-03-13T06:52:50.449445Z","iopub.status.idle":"2022-03-13T06:52:52.39967Z","shell.execute_reply.started":"2022-03-13T06:52:50.449402Z","shell.execute_reply":"2022-03-13T06:52:52.399063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**In below plot, we see that most of users have small number of transactions**","metadata":{}},{"cell_type":"code","source":"df_avg_item_per_u.num_transactions.hist(bins=100)\nplt.show()\nplt.close()\ndf_avg_item_per_u.boxplot('num_transactions')\nplt.show()\nplt.close()","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:52.400848Z","iopub.execute_input":"2022-03-13T06:52:52.40129Z","iopub.status.idle":"2022-03-13T06:52:56.080717Z","shell.execute_reply.started":"2022-03-13T06:52:52.401251Z","shell.execute_reply":"2022-03-13T06:52:56.079478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_avg_item_per_u.num_transactions.value_counts(bins=[-1, 0, 10, 100, 1000])","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:56.083732Z","iopub.execute_input":"2022-03-13T06:52:56.084109Z","iopub.status.idle":"2022-03-13T06:52:56.157606Z","shell.execute_reply.started":"2022-03-13T06:52:56.084062Z","shell.execute_reply":"2022-03-13T06:52:56.156453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_avg_item_per_u.num_transactions.describe()","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:56.159589Z","iopub.execute_input":"2022-03-13T06:52:56.159965Z","iopub.status.idle":"2022-03-13T06:52:56.22694Z","shell.execute_reply.started":"2022-03-13T06:52:56.15992Z","shell.execute_reply":"2022-03-13T06:52:56.225829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**we mask users with num transaction < 10 as cold start user**","metadata":{}},{"cell_type":"code","source":"df_avg_item_per_u['cold_start_status'] = 'cold_start'\ndf_avg_item_per_u.loc[(df_avg_item_per_u.num_transactions >= 10),'cold_start_status']='non_cold_start'\ndf_coldstart_user = df_avg_item_per_u.copy()\ndf_coldstart_user","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:56.228635Z","iopub.execute_input":"2022-03-13T06:52:56.228913Z","iopub.status.idle":"2022-03-13T06:52:56.483843Z","shell.execute_reply.started":"2022-03-13T06:52:56.22888Z","shell.execute_reply":"2022-03-13T06:52:56.482737Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Find about frequent transaction of user in month ","metadata":{}},{"cell_type":"code","source":"df_month_avg_item_per_u = df.groupby(['customer_id', 'month'])['price'].count().unstack().reset_index()\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:52:56.485321Z","iopub.execute_input":"2022-03-13T06:52:56.485559Z","iopub.status.idle":"2022-03-13T06:53:05.030662Z","shell.execute_reply.started":"2022-03-13T06:52:56.485533Z","shell.execute_reply":"2022-03-13T06:53:05.0294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def find_active_month(x):\n    float_x = x.values[1:].astype(float)\n    return float_x[~np.isnan(float_x)]\ndf_month_avg_item_per_u['transactions_in_active_month'] = df_month_avg_item_per_u.apply(\n    lambda x: find_active_month(x), axis=1)\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:53:05.032181Z","iopub.execute_input":"2022-03-13T06:53:05.032516Z","iopub.status.idle":"2022-03-13T06:53:19.148464Z","shell.execute_reply.started":"2022-03-13T06:53:05.032443Z","shell.execute_reply":"2022-03-13T06:53:19.147868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_month_avg_item_per_u['mean_transactions_in_active_month'] = df_month_avg_item_per_u.apply(\n    lambda x: x['transactions_in_active_month'].mean(), axis=1)\ndf_month_avg_item_per_u","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:53:19.149562Z","iopub.execute_input":"2022-03-13T06:53:19.149881Z","iopub.status.idle":"2022-03-13T06:53:42.540492Z","shell.execute_reply.started":"2022-03-13T06:53:19.149855Z","shell.execute_reply":"2022-03-13T06:53:42.539529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In average, each user only buy 4 items in a month/1 item in a week. It is another challenge","metadata":{}},{"cell_type":"code","source":"print(df_month_avg_item_per_u.mean_transactions_in_active_month.describe())\ndf_month_avg_item_per_u.mean_transactions_in_active_month.hist(bins=100)","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:53:42.541712Z","iopub.execute_input":"2022-03-13T06:53:42.541942Z","iopub.status.idle":"2022-03-13T06:53:42.995258Z","shell.execute_reply.started":"2022-03-13T06:53:42.541914Z","shell.execute_reply":"2022-03-13T06:53:42.99439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Create dataframe for all metadata for user: active status/cold start status","metadata":{}},{"cell_type":"code","source":"df_transaction_frequent = df_month_avg_item_per_u[['customer_id', 'mean_transactions_in_active_month']].copy()\ndf_transaction_frequent","metadata":{"execution":{"iopub.status.busy":"2022-03-13T06:53:42.996918Z","iopub.execute_input":"2022-03-13T06:53:42.997431Z","iopub.status.idle":"2022-03-13T06:53:43.120255Z","shell.execute_reply.started":"2022-03-13T06:53:42.997369Z","shell.execute_reply":"2022-03-13T06:53:43.119572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"result = pd.merge(df_active_user, df_coldstart_user, on='customer_id', how='outer')\nresult = pd.merge(result, df_transaction_frequent, on='customer_id', how='outer')\nresult","metadata":{"execution":{"iopub.status.busy":"2022-03-13T07:41:13.961775Z","iopub.execute_input":"2022-03-13T07:41:13.962047Z","iopub.status.idle":"2022-03-13T07:41:17.640057Z","shell.execute_reply.started":"2022-03-13T07:41:13.96202Z","shell.execute_reply":"2022-03-13T07:41:17.638945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"result[(result.active_status == 'active') & (result.cold_start_status == 'non_cold_start')].shape","metadata":{"execution":{"iopub.status.busy":"2022-03-13T07:41:20.864096Z","iopub.execute_input":"2022-03-13T07:41:20.864436Z","iopub.status.idle":"2022-03-13T07:41:21.560203Z","shell.execute_reply.started":"2022-03-13T07:41:20.864384Z","shell.execute_reply":"2022-03-13T07:41:21.559478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"result.to_csv('metadata_customer_id.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-03-13T08:08:08.522127Z","iopub.execute_input":"2022-03-13T08:08:08.522478Z","iopub.status.idle":"2022-03-13T08:08:19.066518Z","shell.execute_reply.started":"2022-03-13T08:08:08.522446Z","shell.execute_reply":"2022-03-13T08:08:19.065545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"result.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-13T07:40:30.169197Z","iopub.execute_input":"2022-03-13T07:40:30.169513Z","iopub.status.idle":"2022-03-13T07:40:30.176173Z","shell.execute_reply.started":"2022-03-13T07:40:30.169481Z","shell.execute_reply":"2022-03-13T07:40:30.175511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"121843/1371980","metadata":{"execution":{"iopub.status.busy":"2022-03-13T07:41:46.020861Z","iopub.execute_input":"2022-03-13T07:41:46.021221Z","iopub.status.idle":"2022-03-13T07:41:46.028213Z","shell.execute_reply.started":"2022-03-13T07:41:46.021184Z","shell.execute_reply":"2022-03-13T07:41:46.027364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}