\nAn internationally diversified payments corporation is American Express. They are the biggest payment card issuer in the world, giving clients access to goods, information, and experiences that improve lives and foster economic success.\nThe goal of this competition is to forecast, based on a customer's monthly customer profile, the likelihood that they won't pay off their credit card bill in the future. You'll use machine learning to predict credit default in this challenge. You will use an industrial size data set specifically to create a machine learning model that competes with the one currently being used in production. Datasets used for training, validation, and testing include anonymised consumer profiles and time-series behavioral data. You are allowed to experiment with any method to produce the most potent model,
","metadata":{}},{"cell_type":"markdown","source":"The objective of this competition is to predict the probability that a customer does not pay back their credit card balance amount in the future based on their monthly customer profile.
","metadata":{}},{"cell_type":"markdown","source":"\nThe dataset contains aggregated profile features for each customer at each statement date. Features are anonymized and normalized, and fall into the following general categories:
\n\n- `D_*` = Delinquency variables\n- `S_*` = Spend variables\n- `P_*` = Payment variables\n- `B_*` = Balance variables\n- `R_*` = Risk variables\n\n\n With the following features being categorical:`B_30`,`B_38`,`D_114`,`D_116`,`D_117`,`D_120`,`D_126`,`D_63`,`D_64`, `D_66`,`D_68\n
\n\nYour task is to predict, for each customer_ID, the probability of a future payment default (target = 1).
","metadata":{}},{"cell_type":"markdown","source":"[Back to Top]","metadata":{"execution":{"iopub.execute_input":"2022-07-29T18:14:06.195456Z","iopub.status.busy":"2022-07-29T18:14:06.194955Z","iopub.status.idle":"2022-07-29T18:14:06.227006Z","shell.execute_reply":"2022-07-29T18:14:06.225688Z","shell.execute_reply.started":"2022-07-29T18:14:06.195361Z"}}},{"cell_type":"markdown","source":"# 03.Importing Libraries & Packages","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport plotly.graph_objects as go\nimport plotly.express as px\nfrom itertools import cycle\n\nimport warnings, gc\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:46:51.275708Z","iopub.execute_input":"2022-07-30T09:46:51.276268Z","iopub.status.idle":"2022-07-30T09:46:51.284048Z","shell.execute_reply.started":"2022-07-30T09:46:51.276222Z","shell.execute_reply":"2022-07-30T09:46:51.282817Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Back to Top]","metadata":{"execution":{"iopub.execute_input":"2022-07-29T18:14:06.195456Z","iopub.status.busy":"2022-07-29T18:14:06.194955Z","iopub.status.idle":"2022-07-29T18:14:06.227006Z","shell.execute_reply":"2022-07-29T18:14:06.225688Z","shell.execute_reply.started":"2022-07-29T18:14:06.195361Z"}}},{"cell_type":"markdown","source":"# 04.Importing & Reading the Datasets","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_feather('../input/amexfeather/train_data.ftr')\ndf_train = df_train.groupby('customer_ID').tail(1).set_index('customer_ID')\n\ndf_test = pd.read_feather('../input/amexfeather/test_data.ftr')\ndf_test = df_test.groupby('customer_ID').tail(1).set_index('customer_ID')\n\ndf_subm = pd.read_csv(\"../input/amex-default-prediction/sample_submission.csv\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:46:51.286598Z","iopub.execute_input":"2022-07-30T09:46:51.287409Z","iopub.status.idle":"2022-07-30T09:47:56.220335Z","shell.execute_reply.started":"2022-07-30T09:46:51.287371Z","shell.execute_reply":"2022-07-30T09:47:56.212616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Custom Color Palette 🎨\ncustom_colors = [\"#ffe7ba\",\"#70d6ff\",\"#ff4d6d\",\"#8338ec\",\"#00ff00\"]\ncustom_Palette = sns.set_palette(sns.color_palette(custom_colors))\nsns.palplot(sns.color_palette(custom_colors),size=1.2)\nplt.tick_params(axis='both', labelsize=0, length = 0)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.227780Z","iopub.execute_input":"2022-07-30T09:47:56.228855Z","iopub.status.idle":"2022-07-30T09:47:56.368639Z","shell.execute_reply.started":"2022-07-30T09:47:56.228812Z","shell.execute_reply":"2022-07-30T09:47:56.367751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_row_count, df_train_column_count=df_train.shape\nprint('Total amount of rows (Train):', df_train_row_count)\nprint('Total amount of columns (Train):', df_train_column_count)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.374147Z","iopub.execute_input":"2022-07-30T09:47:56.375560Z","iopub.status.idle":"2022-07-30T09:47:56.384654Z","shell.execute_reply.started":"2022-07-30T09:47:56.375519Z","shell.execute_reply":"2022-07-30T09:47:56.383556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test_row_count, df_test_column_count=df_test.shape\nprint('Total amount of rows (Test):', df_test_row_count)\nprint('Total amount of columns (Test):', df_test_column_count)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.387825Z","iopub.execute_input":"2022-07-30T09:47:56.388888Z","iopub.status.idle":"2022-07-30T09:47:56.404130Z","shell.execute_reply.started":"2022-07-30T09:47:56.388849Z","shell.execute_reply":"2022-07-30T09:47:56.402492Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['S_2'] = pd.to_datetime(df_train['S_2'])\ndf_test['S_2'] = pd.to_datetime(df_test['S_2'])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:47:56.407819Z","iopub.execute_input":"2022-07-30T09:47:56.410382Z","iopub.status.idle":"2022-07-30T09:47:56.528827Z","shell.execute_reply.started":"2022-07-30T09:47:56.410337Z","shell.execute_reply":"2022-07-30T09:47:56.527804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Duration of Train Date: ', df_train['S_2'].min(), df_train['S_2'].max())\nprint('Duration of Test Date: ', df_test['S_2'].min(), df_test['S_2'].max())","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-07-30T09:47:56.530515Z","iopub.execute_input":"2022-07-30T09:47:56.531202Z","iopub.status.idle":"2022-07-30T09:47:56.555225Z","shell.execute_reply.started":"2022-07-30T09:47:56.531161Z","shell.execute_reply":"2022-07-30T09:47:56.554184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.559351Z","iopub.execute_input":"2022-07-30T09:47:56.559860Z","iopub.status.idle":"2022-07-30T09:47:56.597362Z","shell.execute_reply.started":"2022-07-30T09:47:56.559788Z","shell.execute_reply":"2022-07-30T09:47:56.596213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.599094Z","iopub.execute_input":"2022-07-30T09:47:56.599776Z","iopub.status.idle":"2022-07-30T09:47:56.641464Z","shell.execute_reply.started":"2022-07-30T09:47:56.599735Z","shell.execute_reply":"2022-07-30T09:47:56.639247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cat_cols = ['B_30', 'B_38', 'D_114', 'D_116', 'D_117', 'D_120', 'D_126', 'D_63', 'D_64', 'D_66', 'D_68']","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.649422Z","iopub.execute_input":"2022-07-30T09:47:56.651654Z","iopub.status.idle":"2022-07-30T09:47:56.658376Z","shell.execute_reply.started":"2022-07-30T09:47:56.651615Z","shell.execute_reply":"2022-07-30T09:47:56.657447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"feat_Delinquency = [c for c in df_train.columns if c.startswith('D_')]\nfeat_Spend = [c for c in df_train.columns if c.startswith('S_')]\nfeat_Payment = [c for c in df_train.columns if c.startswith('P_')]\nfeat_Balance = [c for c in df_train.columns if c.startswith('B_')]\nfeat_Risk = [c for c in df_train.columns if c.startswith('R_')]\nprint(f'Total amount of Delinquency variables: {len(feat_Delinquency)}')\nprint(f'Total amount of Expenses variables: {len(feat_Spend)}')\nprint(f'Total amount of Payment variables: {len(feat_Payment)}')\nprint(f'Total amount of Balance variables: {len(feat_Balance)}')\nprint(f'Total amount of Risk variables: {len(feat_Risk)}')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:47:56.662995Z","iopub.execute_input":"2022-07-30T09:47:56.665977Z","iopub.status.idle":"2022-07-30T09:47:56.681230Z","shell.execute_reply.started":"2022-07-30T09:47:56.665930Z","shell.execute_reply":"2022-07-30T09:47:56.679298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Back to Top]","metadata":{"execution":{"iopub.execute_input":"2022-07-29T18:14:06.195456Z","iopub.status.busy":"2022-07-29T18:14:06.194955Z","iopub.status.idle":"2022-07-29T18:14:06.227006Z","shell.execute_reply":"2022-07-29T18:14:06.225688Z","shell.execute_reply.started":"2022-07-29T18:14:06.195361Z"}}},{"cell_type":"markdown","source":"## 05.EDA","metadata":{}},{"cell_type":"code","source":"labels=['Delinquency', 'Spend','Payment','Balance','Risk']\nvalues= [len(feat_Delinquency), len(feat_Spend),len(feat_Payment), len(feat_Balance),len(feat_Risk)]","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.686088Z","iopub.execute_input":"2022-07-30T09:47:56.688823Z","iopub.status.idle":"2022-07-30T09:47:56.695880Z","shell.execute_reply.started":"2022-07-30T09:47:56.688785Z","shell.execute_reply":"2022-07-30T09:47:56.694844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig_1 = go.Figure()\nfig_1.add_trace(go.Pie(values = values,labels = labels,hole = 0.6, \n hoverinfo ='label+percent'))\nfig_1.update_traces(textfont_size = 12, hoverinfo ='label+percent',textinfo ='label', \n showlegend = False,marker = dict(colors =[\"#70d6ff\",\"#ff9770\"]),\n title = dict(text = 'Distribution of Feature')) \nfig_1.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.701080Z","iopub.execute_input":"2022-07-30T09:47:56.701760Z","iopub.status.idle":"2022-07-30T09:47:56.747888Z","shell.execute_reply.started":"2022-07-30T09:47:56.701725Z","shell.execute_reply":"2022-07-30T09:47:56.746695Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:56.751818Z","iopub.execute_input":"2022-07-30T09:47:56.752330Z","iopub.status.idle":"2022-07-30T09:47:57.150027Z","shell.execute_reply.started":"2022-07-30T09:47:56.752298Z","shell.execute_reply":"2022-07-30T09:47:57.149083Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sum(df_train.isna().sum())","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:47:57.151302Z","iopub.execute_input":"2022-07-30T09:47:57.151667Z","iopub.status.idle":"2022-07-30T09:47:57.521437Z","shell.execute_reply.started":"2022-07-30T09:47:57.151630Z","shell.execute_reply":"2022-07-30T09:47:57.520418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"background_color = 'white'\nmissing = pd.DataFrame(columns = ['% Missing Data'],data = df_train.isnull().sum()/len(df_train))\nfig = plt.figure(figsize = (20, 60),facecolor=background_color)\ngs = fig.add_gridspec(1, 2)\ngs.update(wspace = 0.5, hspace = 0.5)\nax0 = fig.add_subplot(gs[0, 0])\nfor s in [\"right\", \"top\",\"bottom\",\"left\"]:\n ax0.spines[s].set_visible(False)\nsns.heatmap(missing,cbar = False,annot = True,fmt =\".2%\", linewidths = 2,cmap = custom_colors,vmax = 1, ax = ax0)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:47:57.522765Z","iopub.execute_input":"2022-07-30T09:47:57.523737Z","iopub.status.idle":"2022-07-30T09:48:00.425065Z","shell.execute_reply.started":"2022-07-30T09:47:57.523696Z","shell.execute_reply":"2022-07-30T09:48:00.424186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"target_class = pd.DataFrame({'count': df_train.target.value_counts(),\n 'percentage': df_train['target'].value_counts() / df_train.shape[0] * 100\n})","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:48:00.426348Z","iopub.execute_input":"2022-07-30T09:48:00.426985Z","iopub.status.idle":"2022-07-30T09:48:00.442220Z","shell.execute_reply.started":"2022-07-30T09:48:00.426922Z","shell.execute_reply":"2022-07-30T09:48:00.441380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"target_class ","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:48:00.443699Z","iopub.execute_input":"2022-07-30T09:48:00.444155Z","iopub.status.idle":"2022-07-30T09:48:00.458331Z","shell.execute_reply.started":"2022-07-30T09:48:00.444103Z","shell.execute_reply":"2022-07-30T09:48:00.457404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\nfig = go.Figure()\nfig.add_trace(go.Pie(values = target_class['count'],labels = target_class.index,hole = 0.6, \n hoverinfo ='label+percent'))\nfig.update_traces(textfont_size = 12, hoverinfo ='label+percent',textinfo ='label', \n showlegend = False,marker = dict(colors =[\"#9acd32\",\"#87ceeb\"]),\n title = dict(text = 'Distribution of a Target Variable')) \nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:48:00.461683Z","iopub.execute_input":"2022-07-30T09:48:00.462439Z","iopub.status.idle":"2022-07-30T09:48:00.479958Z","shell.execute_reply.started":"2022-07-30T09:48:00.462402Z","shell.execute_reply":"2022-07-30T09:48:00.478195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"stat_plot = df_train.reset_index().groupby('S_2')['customer_ID'].nunique().reset_index()\nfig = go.Figure()\nfig.add_trace(go.Scatter(x = stat_plot['S_2'], y = stat_plot['customer_ID']))\nfig.update_layout(title=\"Customer Presences\", width = 700, height = 500,xaxis_title ='Presencs Date',\n paper_bgcolor='rgb(0,0,0,0)',plot_bgcolor='rgb(0,0,0,0)') \nfig['data'][0]['line']['color']=\"#8338ec\"\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:48:00.482412Z","iopub.execute_input":"2022-07-30T09:48:00.483230Z","iopub.status.idle":"2022-07-30T09:48:00.842077Z","shell.execute_reply.started":"2022-07-30T09:48:00.483191Z","shell.execute_reply":"2022-07-30T09:48:00.841174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gc.collect()","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-07-30T09:48:00.843559Z","iopub.execute_input":"2022-07-30T09:48:00.844142Z","iopub.status.idle":"2022-07-30T09:48:01.356675Z","shell.execute_reply.started":"2022-07-30T09:48:00.844089Z","shell.execute_reply":"2022-07-30T09:48:01.355529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del_cols = [c for c in df_train.columns if (c.startswith(('D','t'))) & (c not in cat_cols)]\ndf_del = df_train[del_cols]\nspd_cols = [c for c in df_train.columns if (c.startswith(('S','t'))) & (c not in cat_cols)]\ndf_spd = df_train[spd_cols]\npay_cols = [c for c in df_train.columns if (c.startswith(('P','t'))) & (c not in cat_cols)]\ndf_pay = df_train[pay_cols]\nbal_cols = [c for c in df_train.columns if (c.startswith(('B','t'))) & (c not in cat_cols)]\ndf_bal = df_train[bal_cols]\nris_cols = [c for c in df_train.columns if (c.startswith(('R','t'))) & (c not in cat_cols)]\ndf_ris = df_train[ris_cols]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:48:01.358193Z","iopub.execute_input":"2022-07-30T09:48:01.359007Z","iopub.status.idle":"2022-07-30T09:48:01.691169Z","shell.execute_reply.started":"2022-07-30T09:48:01.358965Z","shell.execute_reply":"2022-07-30T09:48:01.690174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(29, 3, figsize = (35,150))\nfor i, ax in enumerate(axes.reshape(-1)):\n if i < len(del_cols) - 1:\n sns.kdeplot(x = del_cols[i], hue='target', data = df_del, fill = True, ax = ax, palette =[\"#ffb6c1\",\"#8338ec\"])\n ax.tick_params()\n ax.xaxis.get_label()\n ax.set_ylabel('')\nfig.suptitle('Distribution among Delinquency Variables', fontsize = 35, x = 0.5, y = 1)\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:48:01.692830Z","iopub.execute_input":"2022-07-30T09:48:01.693202Z","iopub.status.idle":"2022-07-30T09:50:38.637298Z","shell.execute_reply.started":"2022-07-30T09:48:01.693162Z","shell.execute_reply":"2022-07-30T09:50:38.636360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize =(11,11))\ncorr = df_del.corr()\nmask = np.triu(np.ones_like(corr, dtype = bool))\nsns.heatmap(corr, mask = mask, robust = True, center = 0,square = True, linewidths =.6, cmap = custom_colors)\nplt.title('Correlation of Delinquency Variables')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:50:38.638585Z","iopub.execute_input":"2022-07-30T09:50:38.639151Z","iopub.status.idle":"2022-07-30T09:50:47.143011Z","shell.execute_reply.started":"2022-07-30T09:50:38.639100Z","shell.execute_reply":"2022-07-30T09:50:47.142059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(8, 3, figsize = (16,18))\nfig.suptitle('Distribution among Spend Variables', fontsize = 15, x = 0.5, y = 1)\nfor i, ax in enumerate(axes.reshape(-1)):\n if i < len(spd_cols) - 1:\n sns.kdeplot(x = spd_cols[i], hue ='target', data = df_spd, fill = True, ax = ax, palette =[\"#ffb6c1\",\"#8338ec\"])\n ax.tick_params()\n ax.xaxis.get_label()\n ax.set_ylabel('')\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:50:47.144437Z","iopub.execute_input":"2022-07-30T09:50:47.145025Z","iopub.status.idle":"2022-07-30T09:51:34.116981Z","shell.execute_reply.started":"2022-07-30T09:50:47.144982Z","shell.execute_reply":"2022-07-30T09:51:34.116071Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"S_cols = [c for c in df_train.columns if (c.startswith(('S')))]\ndf_S = df_train[S_cols]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:51:34.118371Z","iopub.execute_input":"2022-07-30T09:51:34.119306Z","iopub.status.idle":"2022-07-30T09:51:34.156907Z","shell.execute_reply.started":"2022-07-30T09:51:34.119268Z","shell.execute_reply":"2022-07-30T09:51:34.155977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (11,11))\ncorr = df_S.corr()\nmask = np.triu(np.ones_like(corr, dtype=bool))\nsns.heatmap(corr, mask = mask, robust = True, center = 0,square = True, linewidths = .6, cmap = custom_colors)\nplt.title('Correlation of Spend Variable')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:51:34.158621Z","iopub.execute_input":"2022-07-30T09:51:34.159332Z","iopub.status.idle":"2022-07-30T09:51:35.253427Z","shell.execute_reply.started":"2022-07-30T09:51:34.159293Z","shell.execute_reply":"2022-07-30T09:51:35.252504Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(1, 3, figsize = (12,4))\nfig.suptitle('Distribution among Payment Variables',fontsize = 15)\nfor i, ax in enumerate(axes.reshape(-1)):\n if i < len(pay_cols) - 1:\n sns.kdeplot(x = pay_cols[i], hue ='target', data = df_pay, fill = True, ax = ax, palette =[\"#ffb6c1\",\"#8338ec\"])\n ax.tick_params()\n ax.xaxis.get_label()\n ax.set_ylabel('')\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:51:35.259333Z","iopub.execute_input":"2022-07-30T09:51:35.259616Z","iopub.status.idle":"2022-07-30T09:51:42.072788Z","shell.execute_reply.started":"2022-07-30T09:51:35.259589Z","shell.execute_reply":"2022-07-30T09:51:42.071762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"P_cols = [c for c in df_train.columns if (c.startswith(('P')))]\ndf_P = df_train[P_cols]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:51:42.074250Z","iopub.execute_input":"2022-07-30T09:51:42.074958Z","iopub.status.idle":"2022-07-30T09:51:42.082456Z","shell.execute_reply.started":"2022-07-30T09:51:42.074914Z","shell.execute_reply":"2022-07-30T09:51:42.081373Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (6,6))\ncorr = df_P.corr()\nmask = np.triu(np.ones_like(corr, dtype = bool))\nsns.heatmap(corr, mask = mask, robust = True, center = 0,square = True, linewidths = .6, cmap = custom_colors)\nplt.title('Correlation of Payment Variables')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:51:42.083852Z","iopub.execute_input":"2022-07-30T09:51:42.084231Z","iopub.status.idle":"2022-07-30T09:51:42.306071Z","shell.execute_reply.started":"2022-07-30T09:51:42.084193Z","shell.execute_reply":"2022-07-30T09:51:42.305047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(10, 4, figsize = (15,24))\nfig.suptitle('Distribution among Balance Variables',fontsize = 15, x = 0.5, y = 1)\nfor i, ax in enumerate(axes.reshape(-1)):\n if i < len(bal_cols) - 1:\n sns.kdeplot(x = bal_cols[i], hue ='target', data = df_bal, fill = True, ax = ax, palette =[\"#ffb6c1\",\"#8338ec\"])\n ax.tick_params()\n ax.xaxis.get_label()\n ax.set_ylabel('')\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:51:42.307578Z","iopub.execute_input":"2022-07-30T09:51:42.307915Z","iopub.status.idle":"2022-07-30T09:52:59.638885Z","shell.execute_reply.started":"2022-07-30T09:51:42.307878Z","shell.execute_reply":"2022-07-30T09:52:59.637963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"B_cols = [c for c in df_train.columns if (c.startswith(('B')))]\ndf_B = df_train[B_cols]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:52:59.640416Z","iopub.execute_input":"2022-07-30T09:52:59.640974Z","iopub.status.idle":"2022-07-30T09:52:59.707568Z","shell.execute_reply.started":"2022-07-30T09:52:59.640938Z","shell.execute_reply":"2022-07-30T09:52:59.706545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (11,11))\ncorr = df_B.corr()\nmask = np.triu(np.ones_like(corr, dtype = bool))\nsns.heatmap(corr, mask = mask, robust=True, center = 0,square = True, linewidths =.6, cmap = custom_colors)\nplt.title('Correlation of Balance Variables')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:52:59.709105Z","iopub.execute_input":"2022-07-30T09:52:59.709740Z","iopub.status.idle":"2022-07-30T09:53:01.979019Z","shell.execute_reply.started":"2022-07-30T09:52:59.709701Z","shell.execute_reply":"2022-07-30T09:53:01.978056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(10, 3, figsize = (18,23))\nfig.suptitle('Distribution among Risk Variables',fontsize=15, x = 0.5, y = 1)\nfor i, ax in enumerate(axes.reshape(-1)):\n if i < len(ris_cols) - 1:\n sns.kdeplot(x = ris_cols[i], hue ='target', data = df_ris, fill = True, ax = ax, palette =[\"#ffb6c1\",\"#8338ec\"])\n ax.tick_params()\n ax.xaxis.get_label()\n ax.set_ylabel('')\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:53:01.980610Z","iopub.execute_input":"2022-07-30T09:53:01.981214Z","iopub.status.idle":"2022-07-30T09:54:01.653974Z","shell.execute_reply.started":"2022-07-30T09:53:01.981175Z","shell.execute_reply":"2022-07-30T09:54:01.653053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"R_cols = [c for c in df_train.columns if (c.startswith(('R')))]\ndf_R = df_train[R_cols]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T09:54:01.655500Z","iopub.execute_input":"2022-07-30T09:54:01.655887Z","iopub.status.idle":"2022-07-30T09:54:01.689403Z","shell.execute_reply.started":"2022-07-30T09:54:01.655830Z","shell.execute_reply":"2022-07-30T09:54:01.688483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(11,11))\ncorr = df_R.corr()\nmask = np.triu(np.ones_like(corr, dtype=bool))\nsns.heatmap(corr, mask = mask, robust = True, center = 0, square = True, linewidths =.6, cmap = custom_colors)\nplt.title('Correlation of Risk Variables')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T09:54:01.690925Z","iopub.execute_input":"2022-07-30T09:54:01.691310Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"palette = cycle([\"#ffe7ba\",\"#70d6ff\",\"#ff4d6d\",\"#8338ec\",\"#00ff00\"])\ntarg = df_train.corrwith(df_train['target'], axis=0)\nval = [str(round(v ,1) *100) + '%' for v in targ.values]\nfig = go.Figure()\nfig.add_trace(go.Bar(y=targ.index, x= targ.values, orientation='h',text = val, marker_color = next(palette)))\nfig.update_layout(title = \"Correlation of variables with Target\",width = 750, height = 3500,\n paper_bgcolor='rgb(0,0,0,0)',plot_bgcolor='rgb(0,0,0,0)')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gc.collect()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Back to Top]","metadata":{"execution":{"iopub.execute_input":"2022-07-29T18:14:06.195456Z","iopub.status.busy":"2022-07-29T18:14:06.194955Z","iopub.status.idle":"2022-07-29T18:14:06.227006Z","shell.execute_reply":"2022-07-29T18:14:06.225688Z","shell.execute_reply.started":"2022-07-29T18:14:06.195361Z"}}},{"cell_type":"markdown","source":"# 06.Model Training","metadata":{}},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nlab_enc = LabelEncoder()\nfor cat_feat in cat_cols:\n df_train[cat_feat] = lab_enc.fit_transform(df_train[cat_feat])\n df_test[cat_feat] = lab_enc.transform(df_test[cat_feat])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Reference: https://www.kaggle.com/code/inversion/amex-competition-metric-python/notebook\ndef amex_metric(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n\n def top_four_percent_captured(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n df = (pd.concat([y_true, y_pred], axis='columns')\n .sort_values('prediction', ascending=False))\n df['weight'] = df['target'].apply(lambda x: 20 if x==0 else 1)\n four_pct_cutoff = int(0.04 * df['weight'].sum())\n df['weight_cumsum'] = df['weight'].cumsum()\n df_cutoff = df.loc[df['weight_cumsum'] <= four_pct_cutoff]\n return (df_cutoff['target'] == 1).sum() / (df['target'] == 1).sum()\n \n def weighted_gini(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n df = (pd.concat([y_true, y_pred], axis='columns')\n .sort_values('prediction', ascending=False))\n df['weight'] = df['target'].apply(lambda x: 20 if x==0 else 1)\n df['random'] = (df['weight'] / df['weight'].sum()).cumsum()\n total_pos = (df['target'] * df['weight']).sum()\n df['cum_pos_found'] = (df['target'] * df['weight']).cumsum()\n df['lorentz'] = df['cum_pos_found'] / total_pos\n df['gini'] = (df['lorentz'] - df['random']) * df['weight']\n return df['gini'].sum()\n\n def normalized_weighted_gini(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n y_true_pred = y_true.rename(columns={'target': 'prediction'})\n return weighted_gini(y_true, y_pred) / weighted_gini(y_true, y_true_pred)\n\n g = normalized_weighted_gini(y_true, y_pred)\n d = top_four_percent_captured(y_true, y_pred)\n\n return 0.5 * (g + d)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define dataset\nX = df_train.drop('target', axis=1)\ny = df_train['target']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\n# creating dataset split for prediction\nX_train, X_test , y_train , y_test = train_test_split(X,y,test_size=0.2,random_state=42) # 80-20 split\n\n# Checking split \nprint('X_train:', X_train.shape)\nprint('y_train:', y_train.shape)\nprint('X_test:', X_test.shape)\nprint('y_test:', y_test.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from catboost import CatBoostClassifier\nclf = CatBoostClassifier(iterations = 3000, random_state = 42, nan_mode ='Min',task_type =\"GPU\")\nclf.fit(X_train, y_train, eval_set = [(X_test, y_test)], cat_features=cat_cols, verbose = 100)\npreds = clf.predict_proba(X_test)[:, 1]","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_preds = clf.predict_proba(df_test)[:, 1]\ny_preds","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_subm[\"prediction\"] = y_preds\ndf_subm.to_csv('submission_catb.csv', index=False)\ndf_subm","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Back to Top]","metadata":{"execution":{"iopub.execute_input":"2022-07-29T18:14:06.195456Z","iopub.status.busy":"2022-07-29T18:14:06.194955Z","iopub.status.idle":"2022-07-29T18:14:06.227006Z","shell.execute_reply":"2022-07-29T18:14:06.225688Z","shell.execute_reply.started":"2022-07-29T18:14:06.195361Z"}}},{"cell_type":"markdown","source":"# **Thank You!**","metadata":{}}]}