{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n## Table Of Contents","metadata":{}},{"cell_type":"markdown","source":"\n\n[1. Loading Dataset 🔃](#1)\n\n[2. Basic Overview 📺](#2)\n\n[3. Exploratory Data Analysis ✨](#3)\n\n* [Insights About Customers](#3.1)\n \n* [Deliquency Variables](#3.2)\n \n* [Balance Variables](#3.3) \n\n* [Spend Variables](#3.4)\n\n* [Risk Viriables](#3.5)\n\n[4. Modelling 🐱🏍](#4)\n\n[5. Test Data Prediction](#5)\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"### About Competition","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"Read Overview of Competition from Here \n\n[Overview Of Competition](https://www.kaggle.com/competitions/amex-default-prediction/overview)","metadata":{}},{"cell_type":"markdown","source":"The objective of this competition is to predict the probability that a customer does not pay back their credit card balance amount in the future based on their monthly customer profile. The target binary variable is calculated by observing 18 months performance window after the latest credit card statement, and if the customer does not pay due amount in 120 days after their latest statement date it is considered a default event.\n\nThe dataset contains aggregated profile features for each customer at each statement date. Features are anonymized and normalized, and fall into the following general categories:\n\n D_* = Delinquency variables\n S_* = Spend variables\n P_* = Payment variables\n B_* = Balance variables\n R_* = Risk variables","metadata":{}},{"cell_type":"markdown","source":"\n### 1. Loading Dataset 🔃","metadata":{}},{"cell_type":"markdown","source":"1) Dataset provided by AMEX is too big more than 50GB .. It cannot fit into memory \n\n2) Used dataset published by @Munum , who converted .csv file to .ftr(feather file) \n\n3) Real Dataset Size :\n\n Train_data.csv : 16.38 GB\n Test_data.csv : 33.82 GB\n Train_Lables.csv : 30.75 MB\n Sample_submission.csv : 61.95 MB\n \n \n 4) Converted feather file size :\n \n Train_data.ftr : 1.73 GB\n Test_data.ftr : 3.55 GB\n \n ","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport gc","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:39:38.758045Z","iopub.execute_input":"2022-07-17T06:39:38.759847Z","iopub.status.idle":"2022-07-17T06:39:39.841552Z","shell.execute_reply.started":"2022-07-17T06:39:38.759729Z","shell.execute_reply":"2022-07-17T06:39:39.840572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_labels = pd.read_csv(\"../input/amex-default-prediction/train_labels.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:39:58.961052Z","iopub.execute_input":"2022-07-17T06:39:58.961488Z","iopub.status.idle":"2022-07-17T06:39:59.974613Z","shell.execute_reply.started":"2022-07-17T06:39:58.961456Z","shell.execute_reply":"2022-07-17T06:39:59.973777Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_labels.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:40:03.022840Z","iopub.execute_input":"2022-07-17T06:40:03.023258Z","iopub.status.idle":"2022-07-17T06:40:03.047166Z","shell.execute_reply.started":"2022-07-17T06:40:03.023226Z","shell.execute_reply":"2022-07-17T06:40:03.045183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_labels['target'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-07-16T06:59:32.672812Z","iopub.execute_input":"2022-07-16T06:59:32.673318Z","iopub.status.idle":"2022-07-16T06:59:32.691211Z","shell.execute_reply.started":"2022-07-16T06:59:32.673273Z","shell.execute_reply":"2022-07-16T06:59:32.690077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"1) Real dataset ","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_feather(\"../input/amexfeather/train_data.ftr\") #Reading train file ==> feather file\n#df_test = pd.read_feather(\"../input/amexfeather/test_data.ftr\") #Reading test file ==> feather file\nsample = pd.read_csv(\"../input/amex-default-prediction/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:40:55.008440Z","iopub.execute_input":"2022-07-17T06:40:55.008824Z","iopub.status.idle":"2022-07-17T06:41:16.348421Z","shell.execute_reply.started":"2022-07-17T06:40:55.008796Z","shell.execute_reply":"2022-07-17T06:41:16.347382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-16T09:23:11.615901Z","iopub.execute_input":"2022-07-16T09:23:11.616366Z","iopub.status.idle":"2022-07-16T09:23:11.622389Z","shell.execute_reply.started":"2022-07-16T09:23:11.616330Z","shell.execute_reply":"2022-07-16T09:23:11.621612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#First Submission preporcessing based on Feather file\n\n#df_train1 = df_train.groupby('customer_ID').tail(1).set_index('customer_ID') #Getting tail row of each customer(1st Submission)\n\n#df_test1 = df_test.groupby('customer_ID').tail(1).set_index('customer_ID') #Getting tail row of each customer(1st Submission)\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Data about one customer\n\ndf_train[df_train['customer_ID']=='0000099d6bd597052cdcda90ffabf56573fe9d7c79be5fbac11a8ed792feb62a']","metadata":{"execution":{"iopub.status.busy":"2022-07-10T17:32:51.359542Z","iopub.execute_input":"2022-07-10T17:32:51.360202Z","iopub.status.idle":"2022-07-10T17:32:51.744815Z","shell.execute_reply.started":"2022-07-10T17:32:51.360157Z","shell.execute_reply":"2022-07-10T17:32:51.743878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\n#df_train = df_train.merge(train_labels , how='left',on = 'customer_ID') #Merging train_labels with training dats\n\n#df_train.drop(['target_x'],axis=1,inplace=True) #Removing 'target_x'\n#df_train.rename(columns = {'target_y':'target'},inplace=True) #Renaming 'target_y' to 'target'","metadata":{"execution":{"iopub.status.busy":"2022-07-10T08:06:17.500690Z","iopub.execute_input":"2022-07-10T08:06:17.501220Z","iopub.status.idle":"2022-07-10T08:09:04.003045Z","shell.execute_reply.started":"2022-07-10T08:06:17.501182Z","shell.execute_reply":"2022-07-10T08:09:04.001721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Basic Overview 📺","metadata":{}},{"cell_type":"code","source":"print(\"***** Shape of training data set is *****\",df_train.shape)\nprint(\"***** Shape of test data set is *****\",df_test.shape)\nprint(\"***** Shape of training labels *****\",train_labels.shape)","metadata":{"execution":{"iopub.status.busy":"2022-07-10T08:09:56.519483Z","iopub.execute_input":"2022-07-10T08:09:56.520034Z","iopub.status.idle":"2022-07-10T08:09:56.529187Z","shell.execute_reply.started":"2022-07-10T08:09:56.519991Z","shell.execute_reply":"2022-07-10T08:09:56.527640Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-14T13:08:46.064745Z","iopub.execute_input":"2022-07-14T13:08:46.065523Z","iopub.status.idle":"2022-07-14T13:08:46.096783Z","shell.execute_reply.started":"2022-07-14T13:08:46.065476Z","shell.execute_reply":"2022-07-14T13:08:46.095689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.info(max_cols=200 ,show_counts = True )","metadata":{"execution":{"iopub.status.busy":"2022-07-10T06:44:50.805313Z","iopub.execute_input":"2022-07-10T06:44:50.806562Z","iopub.status.idle":"2022-07-10T06:44:57.161044Z","shell.execute_reply.started":"2022-07-10T06:44:50.806505Z","shell.execute_reply":"2022-07-10T06:44:57.159866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"null= pd.DataFrame(df_train.isnull().sum(),columns=['null_count'])\nnull['percentage_of_null'] = round(((null['null_count']/len(df_train))*100) , 2)","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:48:59.114322Z","iopub.execute_input":"2022-07-09T17:48:59.114762Z","iopub.status.idle":"2022-07-09T17:49:04.292603Z","shell.execute_reply.started":"2022-07-09T17:48:59.114726Z","shell.execute_reply":"2022-07-09T17:49:04.291873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"null.sort_values(by='percentage_of_null',ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:49:48.272523Z","iopub.execute_input":"2022-07-09T17:49:48.273269Z","iopub.status.idle":"2022-07-09T17:49:48.292517Z","shell.execute_reply.started":"2022-07-09T17:49:48.273226Z","shell.execute_reply":"2022-07-09T17:49:48.291889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"**** Total number of values in training dataset ***** \" ,df_train.isnull().sum().sum())","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:49:52.493633Z","iopub.execute_input":"2022-07-09T17:49:52.494084Z","iopub.status.idle":"2022-07-09T17:49:57.758466Z","shell.execute_reply.started":"2022-07-09T17:49:52.494048Z","shell.execute_reply":"2022-07-09T17:49:57.757458Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"D = 0\nS = 0\nP = 0\nB = 0\nR = 0\n\nfor i in list(df_train.columns):\n if i.startswith(\"D_\"):\n D+=1\n if i.startswith(\"S_\"):\n S+=1\n if i.startswith(\"P_\"):\n P+=1\n if i.startswith(\"B_\"):\n B+=1\n if i.startswith(\"R_\"):\n R+=1\n \n\nprint(\"***** Number of Delinquency Variables ***** \",D)\nprint(\"***** Number of Spend Variables ***** \",S)\nprint(\"***** Number of Payment Variables ***** \",P)\nprint(\"***** Number of Balance Variables ***** \",B)\nprint(\"***** Number of Risk Variables ***** \",R)","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:50:11.468263Z","iopub.execute_input":"2022-07-09T17:50:11.468633Z","iopub.status.idle":"2022-07-09T17:50:11.475594Z","shell.execute_reply.started":"2022-07-09T17:50:11.468604Z","shell.execute_reply":"2022-07-09T17:50:11.474682Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n## Exploratory Data Analysis ✨","metadata":{}},{"cell_type":"markdown","source":"1) Data is Imbalanced i.e. 75% is mapped to value 0 and remaining 25% is mapped to value 1\n\n2) There are many variables which has null values greater than 99% , some of varaibles are 'D_87 , D_88 , D_108\n","metadata":{}},{"cell_type":"code","source":"def without_hue(data,feature,ax):\n \n total=float(len(data))\n bars_plot=ax.patches\n \n for bars in bars_plot:\n percentage = '{:.1f}%'.format(100 * bars.get_height()/total)\n x = bars.get_x() + bars.get_width()/2.0\n y = bars.get_height()\n ax.text(x, y,(percentage,bars.get_height()),ha='center',fontweight='bold',fontsize=10)","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:50:23.397297Z","iopub.execute_input":"2022-07-09T17:50:23.397699Z","iopub.status.idle":"2022-07-09T17:50:23.403158Z","shell.execute_reply.started":"2022-07-09T17:50:23.397669Z","shell.execute_reply":"2022-07-09T17:50:23.402315Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Distribution of Target Value","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(8,5))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='x',color='gray')\n\na=sns.countplot(data=df_train,x='target')\nwithout_hue(df_train,'target',a)","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:50:31.025653Z","iopub.execute_input":"2022-07-09T17:50:31.026042Z","iopub.status.idle":"2022-07-09T17:50:31.724246Z","shell.execute_reply.started":"2022-07-09T17:50:31.026012Z","shell.execute_reply":"2022-07-09T17:50:31.723407Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Null Value Percentage of Variables","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(15,40))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='x',color='gray')\n\nnull = null[null['percentage_of_null']>0.0]\nnull=null.sort_values(by='percentage_of_null',ascending=False)\nsns.barplot(data=null,y=null.index,x=null['percentage_of_null'])\ndel null","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:50:35.851229Z","iopub.execute_input":"2022-07-09T17:50:35.851656Z","iopub.status.idle":"2022-07-09T17:50:37.258244Z","shell.execute_reply.started":"2022-07-09T17:50:35.851606Z","shell.execute_reply":"2022-07-09T17:50:37.255698Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Insights about customers","metadata":{}},{"cell_type":"markdown","source":"1) There are more than 350k customers who have 13 number of statments in training dataset\n\n2) Last statement of customers in training dataset is in month of March 2018\n\n3) For test Dataset , ther are more than 800k customers who have 13 number of statements\n\n4) Last statment of customers in test dataset varies from March 2019 to October 2019","metadata":{}},{"cell_type":"markdown","source":"#### Train Dataset","metadata":{}},{"cell_type":"code","source":"df_customers = df_train.groupby('customer_ID')['customer_ID'].count()\n\n\ndf_customers = pd.DataFrame(df_customers)\ndf_customers.rename(columns = {'customer_ID':'Count'},inplace=True)\n\ndf_customers.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:51:35.677899Z","iopub.execute_input":"2022-07-09T17:51:35.679107Z","iopub.status.idle":"2022-07-09T17:51:37.097164Z","shell.execute_reply.started":"2022-07-09T17:51:35.679051Z","shell.execute_reply":"2022-07-09T17:51:37.096286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(8,5))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='y',color='gray')\n\nsns.countplot(data=df_customers,x='Count')\nax.text(2,420000,\"Customer Statement's Count Distribution\",font='bold')\nplt.show()\n\ndel df_customers","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:51:39.821762Z","iopub.execute_input":"2022-07-09T17:51:39.822659Z","iopub.status.idle":"2022-07-09T17:51:40.195466Z","shell.execute_reply.started":"2022-07-09T17:51:39.822601Z","shell.execute_reply":"2022-07-09T17:51:40.194683Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_statement = df_train.groupby('customer_ID')['S_2'].max()\n\ndf_statement = pd.DataFrame(df_statement)\n\nprint(df_statement.shape)\n\ndf_statement.head()\n ","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:51:44.808542Z","iopub.execute_input":"2022-07-09T17:51:44.808937Z","iopub.status.idle":"2022-07-09T17:51:45.975080Z","shell.execute_reply.started":"2022-07-09T17:51:44.808907Z","shell.execute_reply":"2022-07-09T17:51:45.974304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(20,5))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='y',color='gray')\n\na=sns.countplot(data=df_statement,x=df_statement['S_2'])\n\nax.text(10,24000,\"Customer's Last Date Statement's Count Distribution\",font='bold')\nx_dates = df_statement['S_2'].dt.strftime('%Y-%m-%d').sort_values().unique()\nax.set_xticklabels(labels=x_dates, rotation=45, ha='right')\nplt.show()\n\ndel df_statement","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:51:48.767083Z","iopub.execute_input":"2022-07-09T17:51:48.767462Z","iopub.status.idle":"2022-07-09T17:52:00.443071Z","shell.execute_reply.started":"2022-07-09T17:51:48.767432Z","shell.execute_reply":"2022-07-09T17:52:00.442046Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Test Dataset","metadata":{}},{"cell_type":"code","source":"df_customers = df_test.groupby('customer_ID')['customer_ID'].count()\n\n\ndf_customers = pd.DataFrame(df_customers)\ndf_customers.rename(columns = {'customer_ID':'Count'},inplace=True)\n\ndf_customers.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:52:04.604460Z","iopub.execute_input":"2022-07-09T17:52:04.604884Z","iopub.status.idle":"2022-07-09T17:52:07.588342Z","shell.execute_reply.started":"2022-07-09T17:52:04.604845Z","shell.execute_reply":"2022-07-09T17:52:07.587696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(8,5))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='y',color='gray')\n\nsns.countplot(data=df_customers,x='Count')\nax.text(2,820000,\"Customer Statement's Count Distribution Test Dataset\",font='bold')\nplt.show()\n\ndel df_customers","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:52:13.878744Z","iopub.execute_input":"2022-07-09T17:52:13.879889Z","iopub.status.idle":"2022-07-09T17:52:14.459171Z","shell.execute_reply.started":"2022-07-09T17:52:13.879845Z","shell.execute_reply":"2022-07-09T17:52:14.458216Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_statement = df_test.groupby('customer_ID')['S_2'].max()\n\ndf_statement = pd.DataFrame(df_statement)\n\nprint(df_statement.shape)\n\ndf_statement.head()\n","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:52:18.837567Z","iopub.execute_input":"2022-07-09T17:52:18.837997Z","iopub.status.idle":"2022-07-09T17:52:21.370851Z","shell.execute_reply.started":"2022-07-09T17:52:18.837962Z","shell.execute_reply":"2022-07-09T17:52:21.369868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(20,5))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='y',color='gray')\n\na=sns.countplot(data=df_statement,x=df_statement['S_2'])\n\nax.text(10,32000,\"Customer's Last Date Statement's Count Distribution Test Dataset\",font='bold')\nx_dates = df_statement['S_2'].dt.strftime('%Y-%m-%d').sort_values().unique()\nax.set_xticklabels(labels=x_dates, rotation=45, ha='right')\nplt.show()\n\ndel df_statement","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:52:25.725032Z","iopub.execute_input":"2022-07-09T17:52:25.725800Z","iopub.status.idle":"2022-07-09T17:52:52.040425Z","shell.execute_reply.started":"2022-07-09T17:52:25.725763Z","shell.execute_reply":"2022-07-09T17:52:52.039329Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Delinquency Variables","metadata":{}},{"cell_type":"markdown","source":"1) There are 96 deliquency variables in dataset \n\n2) Out of 96 , 87 variables are continuous and other 9 are categorical\n\n3) After observing correlation between deliquency variables , there are some variables which are highly correlated to each other i.e. value greater than 0.90\n\n4) Some highly correlated features are :-\n\n 1) D_75 and D_74 : 0.987\n 2) D_62 and D_77 : 0.99\n 3) D_74 and D_58 : 0.92\n ","metadata":{}},{"cell_type":"markdown","source":"#### Continuous Variables","metadata":{}},{"cell_type":"markdown","source":"##### Distribution of Continuous Deliquency Variables","metadata":{}},{"cell_type":"code","source":"cat = ['D_114', 'D_116', 'D_117', 'D_120', 'D_126', 'D_63', 'D_64', 'D_66', 'D_68']\n\ndf_train_Dcols = [d for d in list(df_train.columns) if d.startswith(\"D_\")]\n\ndf_train_Dcols = [i for i in df_train_Dcols if i not in cat ]\nncols=10\nnrows=10\nfig,ax = plt.subplots(nrows=nrows,ncols=ncols,figsize=(20,20))\n\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor i in range(0,ncols):\n for j in range(0,nrows):\n ax[i][j].set_facecolor('#F2EDD7FF')\n ax[i][j].spines['top'].set_visible(False)\n ax[i][j].spines['right'].set_visible(False)\n ax[i][j].spines['left'].set_visible(False)\n ax[i][j].grid(linestyle=\"--\",axis='y',color='gray')\n \n if(int(str(i)+str(j))>86):\n ax[i][j].text(0.5,0.5,\"No Data Available\")\n \n else:\n sns.kdeplot(data=df_train,x=df_train_Dcols[int(str(i)+str(j))],ax=ax[i][j],hue='target',fill=True)\n \n ","metadata":{"execution":{"iopub.status.busy":"2022-07-09T17:54:08.285516Z","iopub.execute_input":"2022-07-09T17:54:08.285901Z","iopub.status.idle":"2022-07-09T18:19:11.128091Z","shell.execute_reply.started":"2022-07-09T17:54:08.285871Z","shell.execute_reply":"2022-07-09T18:19:11.127088Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Categorical Variables","metadata":{}},{"cell_type":"code","source":"cat = ['D_114', 'D_116', 'D_117', 'D_120', 'D_126', 'D_63', 'D_64', 'D_66', 'D_68']\n\nncols=5\nnrows=2\nfig,ax = plt.subplots(nrows=nrows,ncols=ncols,figsize=(25,10))\n\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor i in range(0,nrows):\n for j in range(0,ncols):\n #print(i,j)\n ax[i][j].set_facecolor('#F2EDD7FF')\n ax[i][j].spines['top'].set_visible(False)\n ax[i][j].spines['right'].set_visible(False)\n ax[i][j].spines['left'].set_visible(False)\n ax[i][j].grid(linestyle=\"--\",axis='y',color='gray')\n \n \n\nsns.countplot(data=df_train,x='D_114',ax=ax[0][0],hue='target')\nsns.countplot(data=df_train,x='D_116',ax=ax[0][1],hue='target')\nsns.countplot(data=df_train,x='D_117',ax=ax[0][2],hue='target')\nsns.countplot(data=df_train,x='D_120',ax=ax[0][3],hue='target')\nsns.countplot(data=df_train,x='D_126',ax=ax[0][4],hue='target')\nsns.countplot(data=df_train,x='D_63',ax=ax[1][0],hue='target')\nsns.countplot(data=df_train,x='D_64',ax=ax[1][1],hue='target')\nsns.countplot(data=df_train,x='D_66',ax=ax[1][2],hue='target')\nsns.countplot(data=df_train,x='D_68',ax=ax[1][3],hue='target')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-09T18:55:17.459024Z","iopub.execute_input":"2022-07-09T18:55:17.459987Z","iopub.status.idle":"2022-07-09T18:55:21.951509Z","shell.execute_reply.started":"2022-07-09T18:55:17.459937Z","shell.execute_reply":"2022-07-09T18:55:21.950445Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Correlation between Deliquency Variables","metadata":{}},{"cell_type":"code","source":"cat = ['D_114', 'D_116', 'D_117', 'D_120', 'D_126', 'D_63', 'D_64', 'D_66', 'D_68']\n\ndf_train_Dcols = [d for d in list(df_train.columns) if d.startswith(\"D_\") and d not in cat]\n\ndf_train_Ddata = df_train[df_train_Dcols]\n\ndf_train_Ddata.corr().style.background_gradient(cmap='YlOrRd')\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T06:45:18.867216Z","iopub.execute_input":"2022-07-10T06:45:18.867795Z","iopub.status.idle":"2022-07-10T06:47:00.983614Z","shell.execute_reply.started":"2022-07-10T06:45:18.867750Z","shell.execute_reply":"2022-07-10T06:47:00.982109Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Spend Variables","metadata":{}},{"cell_type":"markdown","source":"1) There are 22 Spend Variabled\n\n2) All 22 spend variables are continuous \n\n3) Some Spend having correlation more than 0.90 \n\n 1) S_7 and S_3 : 0.904\n \n 2) S_22 and S_24 : 0.95","metadata":{}},{"cell_type":"markdown","source":"##### Distribution of Continuous Spend Variables","metadata":{}},{"cell_type":"code","source":"df_train_Scols = [d for d in list(df_train.columns) if d.startswith(\"S_\")]\n\n#df_train_Dcols = [i for i in df_train_Dcols if i not in cat ]\n\nncols=10\nnrows=3\nfig,ax = plt.subplots(nrows=nrows,ncols=ncols,figsize=(20,10))\n\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor i in range(0,nrows):\n for j in range(0,ncols):\n ax[i][j].set_facecolor('#F2EDD7FF')\n ax[i][j].spines['top'].set_visible(False)\n ax[i][j].spines['right'].set_visible(False)\n ax[i][j].spines['left'].set_visible(False)\n ax[i][j].grid(linestyle=\"--\",axis='y',color='gray')\n \n if(int(str(i)+str(j))>21):\n ax[i][j].text(0.5,0.5,\"No Data Available\")\n \n else:\n sns.kdeplot(data=df_train,x=df_train_Scols[int(str(i)+str(j))],ax=ax[i][j],hue='target',fill=True)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T06:47:35.963787Z","iopub.execute_input":"2022-07-10T06:47:35.964278Z","iopub.status.idle":"2022-07-10T06:55:35.967022Z","shell.execute_reply.started":"2022-07-10T06:47:35.964232Z","shell.execute_reply":"2022-07-10T06:55:35.965846Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Correlation between Spend Varaibles","metadata":{}},{"cell_type":"code","source":"df_train_Scols = [d for d in list(df_train.columns) if d.startswith(\"S_\")]\n\ndf_train_Sdata = df_train[df_train_Scols]\n\ndf_train_Sdata.corr().style.background_gradient(cmap='YlOrRd')\n\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T07:00:27.333434Z","iopub.execute_input":"2022-07-10T07:00:27.333976Z","iopub.status.idle":"2022-07-10T07:00:37.410851Z","shell.execute_reply.started":"2022-07-10T07:00:27.333938Z","shell.execute_reply":"2022-07-10T07:00:37.409513Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Balance Variables","metadata":{}},{"cell_type":"markdown","source":"1) There are 40 balance variables \n\n2) Out of 40 , there are 38 variables which are continuous and remaining 2 are categorical\n\n3) Some balance variables have correlation more than 0.90 \n\n 1) B_1 and B_11 : 0.99\n 2) B_7 and B_23 : 0.99\n 3) B_1 and B_37 : 0.99 .... more","metadata":{}},{"cell_type":"markdown","source":"##### Distribution of Continuous Spend Variables","metadata":{}},{"cell_type":"code","source":"cat_B = ['B_30', 'B_38']\ndf_train_Bcols = [d for d in list(df_train.columns) if d.startswith(\"B_\")]\n\ndf_train_Bcols = [i for i in df_train_Bcols if i not in cat_B ]\n\nncols=10\nnrows=4\nfig,ax = plt.subplots(nrows=nrows,ncols=ncols,figsize=(20,15))\n\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor i in range(0,nrows):\n for j in range(0,ncols):\n ax[i][j].set_facecolor('#F2EDD7FF')\n ax[i][j].spines['top'].set_visible(False)\n ax[i][j].spines['right'].set_visible(False)\n ax[i][j].spines['left'].set_visible(False)\n ax[i][j].grid(linestyle=\"--\",axis='y',color='gray')\n \n if(int(str(i)+str(j))>37):\n ax[i][j].text(0.5,0.5,\"No Data Available\")\n \n else:\n sns.kdeplot(data=df_train,x=df_train_Bcols[int(str(i)+str(j))],ax=ax[i][j],hue='target',fill=True)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T07:16:50.813893Z","iopub.execute_input":"2022-07-10T07:16:50.814532Z","iopub.status.idle":"2022-07-10T07:29:54.720678Z","shell.execute_reply.started":"2022-07-10T07:16:50.814484Z","shell.execute_reply":"2022-07-10T07:29:54.719497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Categorical Variables","metadata":{}},{"cell_type":"code","source":"ncols=2\nnrows=1\nfig,ax = plt.subplots(nrows=nrows,ncols=ncols,figsize=(10,5))\n\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor j in range(0,ncols):\n #print(i,j)\n ax[j].set_facecolor('#F2EDD7FF')\n ax[j].spines['top'].set_visible(False)\n ax[j].spines['right'].set_visible(False)\n ax[j].spines['left'].set_visible(False)\n ax[j].grid(linestyle=\"--\",axis='y',color='gray')\n \n \n\nsns.countplot(data=df_train,x='B_30',ax=ax[0],hue='target')\nsns.countplot(data=df_train,x='B_38',ax=ax[1],hue='target')\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T07:30:06.014505Z","iopub.execute_input":"2022-07-10T07:30:06.015200Z","iopub.status.idle":"2022-07-10T07:30:07.934040Z","shell.execute_reply.started":"2022-07-10T07:30:06.015139Z","shell.execute_reply":"2022-07-10T07:30:07.932847Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Correlation between Balance Variables","metadata":{}},{"cell_type":"code","source":"cat_B = ['B_30', 'B_38']\n\ndf_train_Bcols = [d for d in list(df_train.columns) if d.startswith(\"B_\") and d not in cat_B]\n\ndf_train_Bdata = df_train[df_train_Bcols]\n\ndf_train_Bdata.corr().style.background_gradient(cmap='YlOrRd')\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T07:30:11.733101Z","iopub.execute_input":"2022-07-10T07:30:11.733596Z","iopub.status.idle":"2022-07-10T07:30:36.538606Z","shell.execute_reply.started":"2022-07-10T07:30:11.733558Z","shell.execute_reply":"2022-07-10T07:30:36.537779Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Risk Variables","metadata":{}},{"cell_type":"markdown","source":"1) There are 28 Risk variables in which all are continuous\n\n2) No Risk variables correlation is greater than 0.90","metadata":{}},{"cell_type":"markdown","source":"##### Distribution of Continuous Risk Variables","metadata":{}},{"cell_type":"code","source":"df_train_Rcols = [d for d in list(df_train.columns) if d.startswith(\"R_\")]\n\n#df_train_Bcols = [i for i in df_train_Bcols if i not in cat_B ]\n\nncols=10\nnrows=3\nfig,ax = plt.subplots(nrows=nrows,ncols=ncols,figsize=(20,15))\n\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor i in range(0,nrows):\n for j in range(0,ncols):\n ax[i][j].set_facecolor('#F2EDD7FF')\n ax[i][j].spines['top'].set_visible(False)\n ax[i][j].spines['right'].set_visible(False)\n ax[i][j].spines['left'].set_visible(False)\n ax[i][j].grid(linestyle=\"--\",axis='y',color='gray')\n \n if(int(str(i)+str(j))>27):\n ax[i][j].text(0.5,0.5,\"No Data Available\")\n \n else:\n sns.kdeplot(data=df_train,x=df_train_Rcols[int(str(i)+str(j))],ax=ax[i][j],hue='target',fill=True)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T07:30:47.821980Z","iopub.execute_input":"2022-07-10T07:30:47.822650Z","iopub.status.idle":"2022-07-10T07:40:28.077805Z","shell.execute_reply.started":"2022-07-10T07:30:47.822595Z","shell.execute_reply":"2022-07-10T07:40:28.076764Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Correlation between Risk Variables","metadata":{}},{"cell_type":"code","source":"df_train_Rcols = [d for d in list(df_train.columns) if d.startswith(\"R_\")]\n\ndf_train_Rdata = df_train[df_train_Rcols]\n\ndf_train_Rdata.corr().style.background_gradient(cmap='YlOrRd')\n","metadata":{"execution":{"iopub.status.busy":"2022-07-10T07:40:41.565736Z","iopub.execute_input":"2022-07-10T07:40:41.566201Z","iopub.status.idle":"2022-07-10T07:40:57.182196Z","shell.execute_reply.started":"2022-07-10T07:40:41.566159Z","shell.execute_reply":"2022-07-10T07:40:57.181249Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Correlation between Target and Variables ","metadata":{}},{"cell_type":"markdown","source":"No variable is very much correlated to target","metadata":{}},{"cell_type":"code","source":"corr= pd.DataFrame(df_train.corrwith(df_train['target'],axis=0))\ncorr.style.background_gradient(cmap='YlOrRd')\n\n#df_train.corr().style.background_gradient(cmap='YlOrRd')","metadata":{"execution":{"iopub.status.busy":"2022-07-10T07:41:16.810488Z","iopub.execute_input":"2022-07-10T07:41:16.811616Z","iopub.status.idle":"2022-07-10T07:41:38.794654Z","shell.execute_reply.started":"2022-07-10T07:41:16.811564Z","shell.execute_reply":"2022-07-10T07:41:38.793515Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Modelling 🐱🏍","metadata":{}},{"cell_type":"markdown","source":"1) Using LightGBM Algorithm for Modelling \n\n2) Reason for using LightGBM \n\n 2.1) It can handle null values\n 2.2) It is affected by high correlation between variables\n 2.3) It can handle outliers \n 2.4) It is not affected by scale of features , so need to scale variables","metadata":{}},{"cell_type":"code","source":"from lightgbm import LGBMClassifier , early_stopping , log_evaluation\nfrom sklearn.model_selection import StratifiedKFold\nfrom sklearn.metrics import roc_curve , accuracy_score , roc_auc_score\nfrom sklearn.preprocessing import LabelEncoder\n","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:44:32.346752Z","iopub.execute_input":"2022-07-17T06:44:32.347198Z","iopub.status.idle":"2022-07-17T06:44:32.352279Z","shell.execute_reply.started":"2022-07-17T06:44:32.347162Z","shell.execute_reply":"2022-07-17T06:44:32.351029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#df_train_par = pd.read_parquet(\"../input/amex-data-integer-dtypes-parquet-format/train.parquet\")\n#df_test_par = pd.read_parquet(\"../input/amex-data-integer-dtypes-parquet-format/test.parquet\")","metadata":{"execution":{"iopub.status.busy":"2022-07-14T09:03:48.233424Z","iopub.execute_input":"2022-07-14T09:03:48.233912Z","iopub.status.idle":"2022-07-14T09:05:03.260448Z","shell.execute_reply.started":"2022-07-14T09:03:48.233869Z","shell.execute_reply":"2022-07-14T09:05:03.256593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Getting categorical columns\ncat_columns = ['B_30', 'B_38', 'D_114', 'D_116', 'D_117', 'D_120', 'D_126', 'D_63', 'D_64', 'D_66', 'D_68']\n\n#Getting continuous columns\ncont_columns = [cols for cols in list(df_train.columns) if cols not in (cat_columns+['customer_ID','S_2','target'])]\n","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:44:36.044187Z","iopub.execute_input":"2022-07-17T06:44:36.045243Z","iopub.status.idle":"2022-07-17T06:44:36.050696Z","shell.execute_reply.started":"2022-07-17T06:44:36.045203Z","shell.execute_reply":"2022-07-17T06:44:36.049528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#df_train1 = df_train.groupby('customer_ID').tail(1).set_index('customer_ID') #Getting tail row of each customer(1st Submission)\n\n#df_test1 = df_test.groupby('customer_ID').tail(1).set_index('customer_ID') #Getting tail row of each customer(1st Submission)\n\n\n#Feature Engg ==> https://www.kaggle.com/code/cdeotte/xgboost-starter-0-793\n# ==> https://www.kaggle.com/code/thedevastator/lag-features-are-all-you-need/notebook\n\n\n#Training Data\n\n# 1) Applied groupby on 'customer_ID' , to get each customer's data . Also applied mean , last , first , \n#min and max aggregation functions to add new features inside dataset\n\ndf_cont_train = df_train.groupby('customer_ID')[cont_columns].agg(['mean','last','first'])\n\ndf_cont_train.columns = ['_'.join(x) for x in df_cont_train.columns ] #Joining column names in tuple form for ex. ('P2','mean') ==> P2_mean\n\ndf_cont_train.reset_index(inplace=True)\n\n# 2) Adding 'first' and 'last' lag feature and ratio feature\nfor col in df_cont_train.columns:\n #print('cols',col)\n if 'last' in col and col.replace('last','first') in df_cont_train:\n \n df_cont_train[col+'Diff_statement'] = df_cont_train[col]-df_cont_train[col.replace('last','first')]\n df_cont_train[col+'Diff_statement_ratio'] = df_cont_train[col]/df_cont_train[col.replace('last','first')]\n\n'''for col in df_cont_train.columns:\n if 'max' in col and col.replace('max','min') in df_cont_train:\n #min max ratio\n df_cont_train['min_max_ratio'] = df_cont_train[col]/df_cont_train[col.replace('max','min')]'''\n\n\ncols_fl = [i for i in list(df_cont_train.columns) if df_cont_train[i].dtypes == 'float64']\ncols_int = [i for i in list(df_cont_train.columns) if df_cont_train[i].dtypes == 'int64']\n\nfor col in cols_fl:\n df_cont_train[col] = df_cont_train[cols].astype(np.float32)\nfor col in cols_int:\n df_cont_train[col] = df_cont_train[cols].astype(np.float32)\n \n \n# 3) For Categorical Columns\ndf_cat_train = df_train.groupby('customer_ID')[cat_columns].agg(['count','last'])\n\ndf_cat_train.columns = ['_'.join(x) for x in df_cat_train.columns ] #Same as above\n\ndf_cat_train.reset_index(inplace=True)\n# 4) Merging continuous and categorical dataframes and train_labels\ndf_train = df_cont_train.merge(df_cat_train , how = 'inner' , on = 'customer_ID').merge(train_labels,how='inner',on='customer_ID')\n\ndel df_cat_train , df_cont_train , train_labels #Deleting dataframe for memory optimization\n\ngc.collect()\n\ndf_train.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:45:18.950654Z","iopub.execute_input":"2022-07-17T06:45:18.951027Z","iopub.status.idle":"2022-07-17T06:49:56.518993Z","shell.execute_reply.started":"2022-07-17T06:45:18.950999Z","shell.execute_reply":"2022-07-17T06:49:56.517805Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.drop(['customer_ID'],axis=1,inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:49:56.521194Z","iopub.execute_input":"2022-07-17T06:49:56.521874Z","iopub.status.idle":"2022-07-17T06:49:58.682514Z","shell.execute_reply.started":"2022-07-17T06:49:56.521829Z","shell.execute_reply":"2022-07-17T06:49:58.681647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Referred From ==> https://www.kaggle.com/code/kellibelcher/amex-default-prediction-eda-lgbm-baseline\n\ndef amex_metric(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n\n def top_four_percent_captured(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n df = (pd.concat([y_true, y_pred], axis='columns')\n .sort_values('prediction', ascending=False))\n df['weight'] = df['target'].apply(lambda x: 20 if x==0 else 1)\n four_pct_cutoff = int(0.04 * df['weight'].sum())\n df['weight_cumsum'] = df['weight'].cumsum()\n df_cutoff = df.loc[df['weight_cumsum'] <= four_pct_cutoff]\n return (df_cutoff['target'] == 1).sum() / (df['target'] == 1).sum()\n \n def weighted_gini(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n df = (pd.concat([y_true, y_pred], axis='columns')\n .sort_values('prediction', ascending=False))\n df['weight'] = df['target'].apply(lambda x: 20 if x==0 else 1)\n df['random'] = (df['weight'] / df['weight'].sum()).cumsum()\n total_pos = (df['target'] * df['weight']).sum()\n df['cum_pos_found'] = (df['target'] * df['weight']).cumsum()\n df['lorentz'] = df['cum_pos_found'] / total_pos\n df['gini'] = (df['lorentz'] - df['random']) * df['weight']\n return df['gini'].sum()\n\n def normalized_weighted_gini(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> float:\n y_true_pred = y_true.rename(columns={'target': 'prediction'})\n return weighted_gini(y_true, y_pred) / weighted_gini(y_true, y_true_pred)\n\n g = normalized_weighted_gini(y_true, y_pred)\n d = top_four_percent_captured(y_true, y_pred)\n\n return 0.5 * (g + d)","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:52:00.253885Z","iopub.execute_input":"2022-07-17T06:52:00.254789Z","iopub.status.idle":"2022-07-17T06:52:00.267209Z","shell.execute_reply.started":"2022-07-17T06:52:00.254744Z","shell.execute_reply":"2022-07-17T06:52:00.266080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sk = StratifiedKFold(n_splits=10,shuffle=True,random_state=21)\n\nY = df_train['target']\nX = df_train.drop(['target'],axis=1)\n \nlgbm_test_preds = []\n\nfor fold , (train_idx,test_idx) in enumerate(sk.split(X,Y)):\n \n print(\" ***** This is fold no. \",fold ,\" *****\")\n print()\n \n train_x,train_y = X.iloc[train_idx,:] , Y[train_idx]\n test_x,test_y = X.iloc[test_idx,:] , Y[test_idx]\n \n print(\"*** Shape of train_x shape ***\",train_x.shape)\n print(\"*** Shape of train_y shape ***\",train_y.shape)\n print(\"*** Shape of test_x shape ***\",test_x.shape)\n print(\"*** Shape of test_y shape ***\",test_y.shape)\n \n \n #Referred From ==> https://www.kaggle.com/code/kellibelcher/amex-default-prediction-eda-lgbm-baseline\n \n params = {'boosting_type': 'gbdt',\n 'n_estimators': 1000,\n 'num_leaves': 50,\n 'learning_rate': 0.05,\n 'colsample_bytree': 0.9,\n 'min_child_samples': 2000,\n 'max_bins': 500,\n 'reg_alpha': 2,\n 'objective': 'binary',\n 'random_state': 21}\n \n lgbm = LGBMClassifier(**params).fit(train_x, train_y, \n eval_set=[(train_x, train_y), (test_x, test_y)],\n callbacks=[early_stopping(200), log_evaluation(500)],\n eval_metric=['auc','binary_logloss'])\n \n \n lgbm_pred = lgbm.predict_proba(test_x)[:,1]\n \n y_pred = pd.DataFrame(data={'prediction':lgbm_pred})\n y_true = pd.DataFrame(data={'target':test_y.reset_index(drop=True)}) #drop current index of dataframe and replace it with increasing integers\n \n gini_score = amex_metric(y_true,y_pred)\n \n auc_score = roc_auc_score(test_y,lgbm_pred)\n \n \n \n print(\"**** Amex Metric **** \",gini_score)\n print(\"**** AUC Score ****\",auc_score)\n \n \n #lgbm_test_preds.append(lgbm.predict_proba(df_test1)[:,1])\n \n del train_x , train_y , test_x , test_y\n \n \ndel X , Y","metadata":{"execution":{"iopub.status.busy":"2022-07-17T06:52:04.828646Z","iopub.execute_input":"2022-07-17T06:52:04.829025Z","iopub.status.idle":"2022-07-17T09:47:28.279514Z","shell.execute_reply.started":"2022-07-17T06:52:04.828993Z","shell.execute_reply":"2022-07-17T09:47:28.275466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Test Data Prediction","metadata":{}},{"cell_type":"code","source":"#For Test dataset\n\ndef feat_engg_test(df_test):\n #print(df_test.columns)\n #Getting categorical columns\n cat_columns = ['B_30', 'B_38', 'D_114', 'D_116', 'D_117', 'D_120', 'D_126', 'D_63', 'D_64', 'D_66', 'D_68']\n\n #Getting continuous columns\n cont_columns = [cols for cols in list(df_test.columns) if cols not in (cat_columns+['customer_ID','S_2'])]\n\n \n df_cont_test = df_test.groupby('customer_ID')[cont_columns].agg(['mean','last'])\n #print(df_cont_test.columns)\n df_cont_test.columns = ['_'.join(x) for x in df_cont_test.columns ]\n \n df_cont_test.reset_index(inplace=True)\n\n for col in df_cont_test.columns:\n #print('cols',col)\n if 'last' in col and col.replace('last','first') in df_cont_test:\n #Lag feature and ratio\n df_cont_test[col+'Diff_statement'] = df_cont_test[col]-df_cont_test[col.replace('last','first')]\n df_cont_test[col+'Diff_statement_ratio'] = df_cont_test[col]/df_cont_test[col.replace('last','first')]\n\n '''for col in df_cont_test.columns:\n if 'max' in col and col.replace('max','min') in df_cont_test:\n #min max ratio\n df_cont_test['min_max_ratio'] = df_cont_test[col]/df_cont_test[col.replace('max','min')]'''\n\n cols_fl = [i for i in list(df_cont_test.columns) if df_cont_test[i].dtypes == 'float64']\n cols_int = [i for i in list(df_cont_test.columns) if df_cont_test[i].dtypes == 'int64']\n\n for col in cols_fl:\n df_cont_test[col] = df_cont_test[cols].astype(np.float32)\n for col in cols_int:\n df_cont_test[col] = df_cont_test[cols].astype(np.float32)\n\n\n\n # For Categorical Columns\n df_cat_test = df_test.groupby('customer_ID')[cat_columns].agg(['count','last'])\n\n df_cat_test.columns = ['_'.join(x) for x in df_cat_test.columns ]\n\n df_cat_test.reset_index(inplace=True)\n\n df_test = df_cont_test.merge(df_cat_test , how = 'inner' , on = 'customer_ID')\n\n del df_cat_test , df_cont_test\n \n gc.collect()\n print(df_test.shape)\n \n return(df_test)\n \n#df_test.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-17T09:47:28.286777Z","iopub.execute_input":"2022-07-17T09:47:28.288208Z","iopub.status.idle":"2022-07-17T09:47:28.309511Z","shell.execute_reply.started":"2022-07-17T09:47:28.288151Z","shell.execute_reply":"2022-07-17T09:47:28.308304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Getting rows and chunks from test dataset\ndef get_rows(customer,test,num_parts=10):\n \n chunk = len(customer)//num_parts #dividing into chunks\n \n rows = []\n\n for k in range(num_parts): #getting unique customer's chunk\n \n if (k==num_parts-1):\n cc = customer[k*chunk:]\n else:\n cc = customer[k*chunk:(k+1)*chunk]\n \n s = test.loc[test.customer_ID.isin(cc)].shape[0] #getting number of rows of customer's who's data present in test according to customer_ID provided in each chunk \n \n rows.append(s)\n \n return rows,chunk #number if rows for each chunk in test dataset , chunk size\n \n \ndf_test = pd.read_feather(\"../input/amexfeather/test_data.ftr\",columns= ['customer_ID','S_2'])\ncustomer_data = df_test[['customer_ID']].drop_duplicates().sort_index().values.flatten() #getting unique customer_ID in test dataset\n\nprint(\"***** Number of unique customers in test data *****\",len(customer_data))\nprint()\nprint(\"***** Unique customer's ID in test data *****\",customer_data)\n\nrows , chunk_cust = get_rows(customer_data,df_test[['customer_ID']],num_parts=10)\n \n \nprint(\"***** Number of rows present for each chunk in test data ****\",rows)\nprint()\nprint(\"***** Chunk size(except last one) *****\",chunk_cust)\n ","metadata":{"execution":{"iopub.status.busy":"2022-07-17T09:47:37.712580Z","iopub.execute_input":"2022-07-17T09:47:37.713357Z","iopub.status.idle":"2022-07-17T09:47:58.884855Z","shell.execute_reply.started":"2022-07-17T09:47:37.713313Z","shell.execute_reply":"2022-07-17T09:47:58.883782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_parts = 10\nskip_rows = 0\nskip_cust = 0\nlgbm_test_preds = []\nfor k in range(num_parts):\n \n print(\"**** Reading test data *****\")\n \n test = pd.read_feather(\"../input/amexfeather/test_data.ftr\")\n test = test.iloc[skip_rows:skip_rows+rows[k]] #Loading test data according to row size\n skip_rows+=rows[k]\n \n test = feat_engg_test(test) #feature engg on each part of test dataset\n test.drop(['customer_ID'],axis=1,inplace=True)\n \n '''if k==num_parts-1: #getting error for this\n test1 = test.loc[customer_data[skip_cust:]]\n else: \n test1 = test.loc[customer_data[skip_cust:skip_cust+chunk_cust]]\n skip_cust += chunk_cust'''\n \n print(test.shape)\n \n print(\"***** Predicting....... *****\")\n lgbm_test_preds.append(lgbm.predict_proba(test)[:,1])\n \n ","metadata":{"execution":{"iopub.status.busy":"2022-07-17T09:49:17.221233Z","iopub.execute_input":"2022-07-17T09:49:17.221771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample['prediction'] = np.concatenate(lgbm_test_preds)\n\nsample.to_csv('submission_3.csv',index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-16T19:37:45.146908Z","iopub.execute_input":"2022-07-16T19:37:45.147428Z","iopub.status.idle":"2022-07-16T19:37:45.159057Z","shell.execute_reply.started":"2022-07-16T19:37:45.147386Z","shell.execute_reply":"2022-07-16T19:37:45.157683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Thank You Scrolling this down on my Notebook , please let know if you have any inputs and provide your feedback too ☺\n\n### I will keep exploring and look forward increase my score ... so will keep updating my notebook ... Stay Tuned ⏩","metadata":{}}]}