{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <h1><center> ⭐️⭐️American Express - Default Prediction⭐️⭐️ </center></h1>\n\n<img src='https://www.underconsideration.com/brandnew/archives/american_express_logo_wordmark_detail.png'>\n\n## **Problem:**\n\nThe objective of this competition is to predict the probability that a customer does not pay back their credit card balance amount in the future based on their monthly customer profile. The target binary variable is calculated by observing 18 months performance window after the latest credit card statement, and if the customer does not pay due amount in 120 days after their latest statement date it is considered a default event.\n\nThe dataset contains aggregated profile features for each customer at each statement date. Features are anonymized and normalized, and fall into the following general categories:\n\n- D_* = Delinquency variables\n- S_* = Spend variables\n- P_* = Payment variables\n- B_* = Balance variables\n- R_* = Risk variables","metadata":{}},{"cell_type":"markdown","source":"## **This Competition is very large amount data - so i was using (.parquet format) of data**\n\n### ***Now I am trying in one of my favorite Fast.ai - Tabular Data***\n\n<img src='https://www.thebalancecareers.com/thmb/6VdCQZNeEcX8M5ysvhZJszDwZV8=/1185x320/filters:no_upscale():max_bytes(150000):strip_icc()/fastai-336cfe963d544d3aa71e195b8485d075.jpg'>\n \n## ***Steps:***\n\n**1. Import Necessary Library**\n\n**2. Try Fast.ai**","metadata":{}},{"cell_type":"markdown","source":"# **1. Import Necessary Library** ","metadata":{}},{"cell_type":"code","source":"from pathlib import Path\nimport numpy as np \nimport pandas as pd","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:44:45.802948Z","iopub.execute_input":"2022-05-28T13:44:45.803743Z","iopub.status.idle":"2022-05-28T13:44:45.826091Z","shell.execute_reply.started":"2022-05-28T13:44:45.803640Z","shell.execute_reply":"2022-05-28T13:44:45.825324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **2. Load the Data**","metadata":{}},{"cell_type":"code","source":"root = \"../input/amexfeather\"\ntrain = Path(root)/\"train_data.ftr\"\ntest = Path(root)/\"test_data.ftr\"\ntrain_label = pd.read_csv(\"../input/amex-default-prediction/train_labels.csv\")\nsample = pd.read_csv(\"../input/amex-default-prediction/sample_submission.csv\")\nprint('Shape of sample',sample.shape)","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:44:46.385150Z","iopub.execute_input":"2022-05-28T13:44:46.385850Z","iopub.status.idle":"2022-05-28T13:44:48.585617Z","shell.execute_reply.started":"2022-05-28T13:44:46.385811Z","shell.execute_reply":"2022-05-28T13:44:48.584623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Taking sample only 50_000 data**","metadata":{}},{"cell_type":"code","source":"df = pd.read_feather(train,use_threads=True).sample(50_000)\nprint('Shape of Train data',df.shape)\ndisplay(df.head())","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:44:48.587336Z","iopub.execute_input":"2022-05-28T13:44:48.588254Z","iopub.status.idle":"2022-05-28T13:45:07.733145Z","shell.execute_reply.started":"2022-05-28T13:44:48.588214Z","shell.execute_reply":"2022-05-28T13:45:07.732298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **EDA**","metadata":{}},{"cell_type":"code","source":"df.describe().T","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:45:07.734476Z","iopub.execute_input":"2022-05-28T13:45:07.735415Z","iopub.status.idle":"2022-05-28T13:45:09.340654Z","shell.execute_reply.started":"2022-05-28T13:45:07.735373Z","shell.execute_reply":"2022-05-28T13:45:09.339832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#identify the null data\npd.DataFrame(df.isnull().sum(), columns=['isnull'])","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:45:09.342765Z","iopub.execute_input":"2022-05-28T13:45:09.343147Z","iopub.status.idle":"2022-05-28T13:45:09.403865Z","shell.execute_reply.started":"2022-05-28T13:45:09.343109Z","shell.execute_reply":"2022-05-28T13:45:09.402966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df.duplicated().sum())","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:45:09.405405Z","iopub.execute_input":"2022-05-28T13:45:09.405776Z","iopub.status.idle":"2022-05-28T13:45:09.787586Z","shell.execute_reply.started":"2022-05-28T13:45:09.405740Z","shell.execute_reply":"2022-05-28T13:45:09.786743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df['target'].value_counts(normalize=True))\n","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:45:09.788999Z","iopub.execute_input":"2022-05-28T13:45:09.789520Z","iopub.status.idle":"2022-05-28T13:45:09.798138Z","shell.execute_reply.started":"2022-05-28T13:45:09.789479Z","shell.execute_reply":"2022-05-28T13:45:09.797247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Start with Fast.ai**","metadata":{}},{"cell_type":"code","source":"!pip install --upgrade fastai","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-05-28T13:45:09.799604Z","iopub.execute_input":"2022-05-28T13:45:09.801622Z","iopub.status.idle":"2022-05-28T13:45:20.515436Z","shell.execute_reply.started":"2022-05-28T13:45:09.801581Z","shell.execute_reply":"2022-05-28T13:45:20.514363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install fast_tabnet","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-05-28T13:45:20.517269Z","iopub.execute_input":"2022-05-28T13:45:20.517978Z","iopub.status.idle":"2022-05-28T13:45:30.216609Z","shell.execute_reply.started":"2022-05-28T13:45:20.517916Z","shell.execute_reply":"2022-05-28T13:45:30.215548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from fastai.tabular.all import *\nfrom fast_tabnet.core import *","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:45:30.219196Z","iopub.execute_input":"2022-05-28T13:45:30.220195Z","iopub.status.idle":"2022-05-28T13:45:32.873388Z","shell.execute_reply.started":"2022-05-28T13:45:30.220145Z","shell.execute_reply":"2022-05-28T13:45:32.872494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Feature Selection**","metadata":{}},{"cell_type":"code","source":"dep_var  = 'target'\ncont_names = ['P_2',\n     'D_39',\n     'B_1',\n     'B_2',\n     'R_1',\n     'S_3',\n     'D_41',\n     'B_3',\n     'D_42',\n     'D_43',\n     'D_44',\n     'B_4',\n     'D_45',\n     'B_5',\n     'R_2',\n     'D_46',\n     'D_47',\n     'D_48',\n     'D_49',\n     'B_6',\n     'B_7',\n     'B_8',\n     'D_50',\n     'D_51',\n     'B_9',\n     'R_3',\n     'D_52',\n     'P_3',\n     'B_10',\n     'D_53',\n     'S_5',\n     'B_11',\n     'S_6',\n     'D_54',\n     'R_4',\n     'S_7',\n     'B_12',\n     'S_8',\n     'D_55',\n     'D_56',\n     'B_13',\n     'R_5',\n     'D_58',\n     'S_9',\n     'B_14',\n     'D_59',\n     'D_60',\n     'D_61',\n     'B_15',\n     'S_11',\n     'D_62',\n     'D_65',\n     'B_16',\n     'B_17',\n     'B_18',\n     'B_19',\n     'B_20',\n     'R_6',\n     'S_13',\n     'B_21',\n     'D_69',\n     'B_22',\n     'D_70',\n     'D_71',\n     'D_72',\n     'S_15',\n     'B_23',\n     'D_73',\n     'P_4',\n     'D_74',\n     'D_75',\n     'D_76',\n     'B_24',\n     'R_7',\n     'D_77',\n     'B_25',\n     'B_26',\n     'D_78',\n     'D_79',\n     'R_8',\n     'R_9',\n     'S_16',\n     'D_80',\n     'R_10',\n     'R_11',\n     'B_27',\n     'D_81',\n     'D_82',\n     'S_17',\n     'R_12',\n     'B_28',\n     'R_13',\n     'D_83',\n     'R_14',\n     'R_15',\n     'D_84',\n     'R_16',\n     'B_29',\n     'S_18',\n     'D_86',\n     'D_87',\n     'R_17',\n     'R_18',\n     'D_88',\n     'B_31',\n     'S_19',\n     'R_19',\n     'B_32',\n     'S_20',\n     'R_20',\n     'R_21',\n     'B_33',\n     'D_89',\n     'R_22',\n     'R_23',\n     'D_91',\n     'D_92',\n     'D_93',\n     'D_94',\n     'R_24',\n     'R_25',\n     'D_96',\n     'S_22',\n     'S_23',\n     'S_24',\n     'S_25',\n     'S_26',\n     'D_102',\n     'D_103',\n     'D_104',\n     'D_105',\n     'D_106',\n     'D_107',\n     'B_36',\n     'B_37',\n     'R_26',\n     'R_27',\n     'D_108',\n     'D_109',\n     'D_110',\n     'D_111',\n     'B_39',\n     'D_112',\n     'B_40',\n     'S_27',\n     'D_113',\n     'D_115',\n     'D_118',\n     'D_119',\n     'D_121',\n     'D_122',\n     'D_123',\n     'D_124',\n     'D_125',\n     'D_127',\n     'D_128',\n     'D_129',\n     'B_41',\n     'B_42',\n     'D_130',\n     'D_131',\n     'D_132',\n     'D_133',\n     'R_28',\n     'D_134',\n     'D_135',\n     'D_136',\n     'D_137',\n     'D_138',\n     'D_139',\n     'D_140',\n     'D_141',\n     'D_142',\n     'D_143',\n     'D_144',\n     'D_145',\n    ]\n\ncat_names = ['B_30', 'B_38', 'D_114', 'D_116', 'D_117', 'D_120', 'D_126', 'D_63', 'D_64', 'D_66', 'D_68'] \nprocs = [Categorify,FillMissing,Normalize]","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:56:45.535378Z","iopub.execute_input":"2022-05-28T13:56:45.535788Z","iopub.status.idle":"2022-05-28T13:56:45.574974Z","shell.execute_reply.started":"2022-05-28T13:56:45.535753Z","shell.execute_reply":"2022-05-28T13:56:45.573992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Random sample\nsplit_sample = np.random.choice(df.shape[0],200)","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:56:46.874463Z","iopub.execute_input":"2022-05-28T13:56:46.875207Z","iopub.status.idle":"2022-05-28T13:56:46.883790Z","shell.execute_reply.started":"2022-05-28T13:56:46.875167Z","shell.execute_reply":"2022-05-28T13:56:46.882996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Build the Model**","metadata":{}},{"cell_type":"code","source":"\ndls = TabularDataLoaders.from_df(df,root,procs,cat_names,cont_names,y_names=dep_var,valid_idx=split_sample,bs=64,y_block=CategoryBlock)","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:56:49.120413Z","iopub.execute_input":"2022-05-28T13:56:49.121320Z","iopub.status.idle":"2022-05-28T13:56:59.159213Z","shell.execute_reply.started":"2022-05-28T13:56:49.121266Z","shell.execute_reply":"2022-05-28T13:56:59.158123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn = tabular_learner(dls, model_dir=\"/tmp/model/\", metrics=[accuracy]).to_fp16()","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:56:59.161417Z","iopub.execute_input":"2022-05-28T13:56:59.161847Z","iopub.status.idle":"2022-05-28T13:56:59.258020Z","shell.execute_reply.started":"2022-05-28T13:56:59.161805Z","shell.execute_reply":"2022-05-28T13:56:59.256886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls.valid.show_batch()","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:56:59.260841Z","iopub.execute_input":"2022-05-28T13:56:59.261734Z","iopub.status.idle":"2022-05-28T13:56:59.544513Z","shell.execute_reply.started":"2022-05-28T13:56:59.261684Z","shell.execute_reply":"2022-05-28T13:56:59.543639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.lr_find()\n","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:56:59.546636Z","iopub.execute_input":"2022-05-28T13:56:59.547164Z","iopub.status.idle":"2022-05-28T13:57:03.584573Z","shell.execute_reply.started":"2022-05-28T13:56:59.547121Z","shell.execute_reply":"2022-05-28T13:57:03.583779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fine_tune(4,1e-3)\n","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:57:03.585895Z","iopub.execute_input":"2022-05-28T13:57:03.586782Z","iopub.status.idle":"2022-05-28T13:59:25.664629Z","shell.execute_reply.started":"2022-05-28T13:57:03.586742Z","shell.execute_reply":"2022-05-28T13:59:25.663805Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Confusion Matrix**","metadata":{}},{"cell_type":"code","source":"interp = ClassificationInterpretation.from_learner(learn)\ninterp.plot_confusion_matrix(figsize=(5,5), dpi=60)","metadata":{"execution":{"iopub.status.busy":"2022-05-28T13:59:25.666321Z","iopub.execute_input":"2022-05-28T13:59:25.666825Z","iopub.status.idle":"2022-05-28T13:59:25.937131Z","shell.execute_reply.started":"2022-05-28T13:59:25.666774Z","shell.execute_reply":"2022-05-28T13:59:25.935783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## ***⭐️⭐️Thankyou for visiting guys⭐️⭐️***\n\nReference:\n\n1. https://www.kaggle.com/competitions/amex-default-prediction/discussion/327612\n2. https://www.kaggle.com/code/venkatkumar001/fast-ai-1","metadata":{}}]}