{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Overview\nThis was my second Kaggle competition driven project. My primary objective here was to try to be able to be able to quickly assess and run through a large and complex dataset in a quick and dirty way. This was one where I wanted to skim the surface area as fast as possible, while providing enough skeletal framework that I could come back later and flesh things out more for more accurate predictions.\n\n## Learning objectives\n- Learn and deepen my understanding around classifiers.\n- Test out an approach where I use a very limited number of features, as opposed to all features.\n- See if I could be more concise in preprocessing.\n- Try to focus more time and energy on experimentation with machine learning models.\n\n## Takeaways\n- The Home Credit dataset is huge, with over 100 columns in the main dataset, plus about a half dozen other datasets. I wanted to test to see if focusing on a narrower set of features (if for no other reason than personal comprehension) would still yield interesting enough results. The models ended up yielding very mediocre results across the board, so clearly that was wrong. However, given the investment of time I wanted to give towards this exercise, I think the effort led to the learning objectives.\n- This was already a 4 year old (and closed) competition by the time I got around to working with the dataset. There was already a very rich catalog of Kaggle discussions and publicly shared notebooks to read through. There were some very excellent ones that both summarized the salient points and went into details on very useful data science techniques. Spending time reading through those proved very helpful for my personal education.\n- Clearly, there is a lot of room for improvement. I feel good about the flexibility in the notebook, such that coming back through and adding more capabilities should be possible without having to start all over again. A few ideas for improvement:\n    - Use more / *all* of the features in the training dataset, as well as the related datasets.\n    - Learn and experiment more around hyperparemeter tuning for the ML models.\n    - Take the time to see how the K Nearest Neighbor Imputer may work to improve the effectiveness of the model.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport scipy as stats\nfrom sklearn.experimental import enable_iterative_imputer\nfrom sklearn.impute import KNNImputer, IterativeImputer\nfrom sklearn.preprocessing import RobustScaler, MinMaxScaler, MaxAbsScaler, StandardScaler\nfrom sklearn.model_selection import train_test_split\nfrom imblearn.over_sampling import SMOTE\nfrom sklearn.metrics import confusion_matrix, precision_score, recall_score, roc_auc_score\n\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.neural_network import MLPClassifier\nfrom xgboost import XGBClassifier\nfrom lightgbm import LGBMClassifier\n\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"id":"tE6T5J05U1Be","execution":{"iopub.status.busy":"2022-07-21T12:37:46.565310Z","iopub.execute_input":"2022-07-21T12:37:46.565786Z","iopub.status.idle":"2022-07-21T12:37:49.591910Z","shell.execute_reply.started":"2022-07-21T12:37:46.565694Z","shell.execute_reply":"2022-07-21T12:37:49.590741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv('../input/home-credit-default-risk/application_train.csv')\ntest = pd.read_csv('../input/home-credit-default-risk/application_test.csv')","metadata":{"id":"h7D_3C6CYc2v","execution":{"iopub.status.busy":"2022-07-21T12:37:49.594047Z","iopub.execute_input":"2022-07-21T12:37:49.595136Z","iopub.status.idle":"2022-07-21T12:37:56.830051Z","shell.execute_reply.started":"2022-07-21T12:37:49.595088Z","shell.execute_reply":"2022-07-21T12:37:56.828866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Analysis\nThe main takeaway here is that the dataset is *huge*. In a professional / production context, obviously there is no choice but to dive in with gusto and attempt to wrangle all of the features and develop new ones. However, for the purposes of this exercise, I wanted to do things as quick and dirty as possible. I looked at the highest correlation features (by absolute value, since negative correlation still has impact), and isolated about a dozen features to work with.\n\nAgain, I recognize that this is nowhere near \"good enough\" for production. However, for the purposes of learning and testing new skills, this is fine.","metadata":{"id":"wtc_9SHFVAXY"}},{"cell_type":"code","source":"train.keys()","metadata":{"id":"gqgG201cYrs6","outputId":"41149bda-cec0-480e-ddfc-8f0124c7056e","execution":{"iopub.status.busy":"2022-07-21T12:37:56.831436Z","iopub.execute_input":"2022-07-21T12:37:56.831764Z","iopub.status.idle":"2022-07-21T12:37:56.839394Z","shell.execute_reply.started":"2022-07-21T12:37:56.831734Z","shell.execute_reply":"2022-07-21T12:37:56.838273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"corr_rank = train.corr()['TARGET'].sort_values()","metadata":{"id":"DTo4qd-oYrqU","execution":{"iopub.status.busy":"2022-07-21T12:37:56.842276Z","iopub.execute_input":"2022-07-21T12:37:56.842678Z","iopub.status.idle":"2022-07-21T12:38:07.878421Z","shell.execute_reply.started":"2022-07-21T12:37:56.842641Z","shell.execute_reply":"2022-07-21T12:38:07.877432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'm scanning for what are the most correlated (positive and negative) features to the target variable.","metadata":{}},{"cell_type":"code","source":"corr_rank.head(n=15)","metadata":{"id":"ZCxplTnIYrnL","outputId":"62b8a1ea-3f8d-4228-87ff-ae272b7cd2af","execution":{"iopub.status.busy":"2022-07-21T12:38:07.879846Z","iopub.execute_input":"2022-07-21T12:38:07.880280Z","iopub.status.idle":"2022-07-21T12:38:07.890557Z","shell.execute_reply.started":"2022-07-21T12:38:07.880240Z","shell.execute_reply":"2022-07-21T12:38:07.889363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"corr_rank.tail(n=15)","metadata":{"id":"hRUAwU95ZaBF","outputId":"f489182b-043d-458c-f399-100531a8c320","execution":{"iopub.status.busy":"2022-07-21T12:38:07.891718Z","iopub.execute_input":"2022-07-21T12:38:07.892130Z","iopub.status.idle":"2022-07-21T12:38:07.907861Z","shell.execute_reply.started":"2022-07-21T12:38:07.892102Z","shell.execute_reply":"2022-07-21T12:38:07.906799Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Feature Selection\n* Make life simple on myself, just select the most correlated features. For now, I'm doing it manually, but could come back and write up a script which could pick up the features automatically based on some variable threshold. ","metadata":{"id":"1S7jQrMfVAOi"}},{"cell_type":"code","source":"train = train[['TARGET', 'EXT_SOURCE_3', 'EXT_SOURCE_1', 'EXT_SOURCE_2', 'DAYS_EMPLOYED', 'DAYS_BIRTH', 'REGION_RATING_CLIENT_W_CITY', 'REGION_RATING_CLIENT', 'DAYS_LAST_PHONE_CHANGE', 'DAYS_ID_PUBLISH', 'DEF_30_CNT_SOCIAL_CIRCLE']]","metadata":{"id":"CnSSMsNbZ6PA","execution":{"iopub.status.busy":"2022-07-21T12:38:07.909377Z","iopub.execute_input":"2022-07-21T12:38:07.910344Z","iopub.status.idle":"2022-07-21T12:38:07.938124Z","shell.execute_reply.started":"2022-07-21T12:38:07.910303Z","shell.execute_reply":"2022-07-21T12:38:07.937307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train","metadata":{"id":"zk-SZoQLa3S2","outputId":"52f66a53-c227-4a61-f92c-ffee64e90d0d","execution":{"iopub.status.busy":"2022-07-21T12:38:07.939309Z","iopub.execute_input":"2022-07-21T12:38:07.939808Z","iopub.status.idle":"2022-07-21T12:38:07.964338Z","shell.execute_reply.started":"2022-07-21T12:38:07.939777Z","shell.execute_reply":"2022-07-21T12:38:07.963584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test = test[['EXT_SOURCE_3', 'EXT_SOURCE_1', 'EXT_SOURCE_2', 'DAYS_EMPLOYED', 'DAYS_BIRTH', 'REGION_RATING_CLIENT_W_CITY', 'REGION_RATING_CLIENT', 'DAYS_LAST_PHONE_CHANGE', 'DAYS_ID_PUBLISH', 'DEF_30_CNT_SOCIAL_CIRCLE']]","metadata":{"id":"70pQo3ukbWiW","execution":{"iopub.status.busy":"2022-07-21T12:38:07.965541Z","iopub.execute_input":"2022-07-21T12:38:07.965994Z","iopub.status.idle":"2022-07-21T12:38:07.973608Z","shell.execute_reply.started":"2022-07-21T12:38:07.965966Z","shell.execute_reply":"2022-07-21T12:38:07.972722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test","metadata":{"id":"sfHi3E2RbjJi","outputId":"73a7c58c-ab42-4dd7-909b-6e2b7a7e15d1","execution":{"iopub.status.busy":"2022-07-21T12:38:07.978018Z","iopub.execute_input":"2022-07-21T12:38:07.978546Z","iopub.status.idle":"2022-07-21T12:38:08.001106Z","shell.execute_reply.started":"2022-07-21T12:38:07.978512Z","shell.execute_reply":"2022-07-21T12:38:07.999900Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Stacking the notebooks\nThis is a practice I've picked up from my class. The goal here is to be able to do all nulls filling, transformations, scaling, and encoding on a consistent basis across both datasets.\n\nHowever, I do note that this does not appear to be a common practice in other notebooks on Kaggle. It is something to do a bit of side research on.","metadata":{"id":"woOtXxQOYn8E"}},{"cell_type":"code","source":"train['TRAINING'] = True\ntest['TRAINING'] = False","metadata":{"id":"v4J9HcuXYrAH","outputId":"8d32bde4-13c0-4a99-d13f-8f05498a7327","execution":{"iopub.status.busy":"2022-07-21T12:38:08.003337Z","iopub.execute_input":"2022-07-21T12:38:08.004402Z","iopub.status.idle":"2022-07-21T12:38:08.013194Z","shell.execute_reply.started":"2022-07-21T12:38:08.004328Z","shell.execute_reply":"2022-07-21T12:38:08.012030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y = train.TARGET","metadata":{"id":"y9sGbHWNYnNl","execution":{"iopub.status.busy":"2022-07-21T12:38:08.014873Z","iopub.execute_input":"2022-07-21T12:38:08.015391Z","iopub.status.idle":"2022-07-21T12:38:08.024012Z","shell.execute_reply.started":"2022-07-21T12:38:08.015329Z","shell.execute_reply":"2022-07-21T12:38:08.022432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y","metadata":{"id":"UNafXabscZ0T","outputId":"e7dac0b7-9e4e-41b9-9f50-f7213f249e67","execution":{"iopub.status.busy":"2022-07-21T12:38:08.025611Z","iopub.execute_input":"2022-07-21T12:38:08.026565Z","iopub.status.idle":"2022-07-21T12:38:08.042062Z","shell.execute_reply.started":"2022-07-21T12:38:08.026525Z","shell.execute_reply":"2022-07-21T12:38:08.041135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.drop(columns = 'TARGET', inplace=True)","metadata":{"id":"o04Aw5kEeQrJ","execution":{"iopub.status.busy":"2022-07-21T12:38:08.043474Z","iopub.execute_input":"2022-07-21T12:38:08.046641Z","iopub.status.idle":"2022-07-21T12:38:08.059462Z","shell.execute_reply.started":"2022-07-21T12:38:08.046597Z","shell.execute_reply":"2022-07-21T12:38:08.058441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x = pd.concat([train, test], axis = 0)","metadata":{"id":"sdOFA3uSeW8K","execution":{"iopub.status.busy":"2022-07-21T12:38:08.060720Z","iopub.execute_input":"2022-07-21T12:38:08.061206Z","iopub.status.idle":"2022-07-21T12:38:08.082466Z","shell.execute_reply.started":"2022-07-21T12:38:08.061175Z","shell.execute_reply":"2022-07-21T12:38:08.080634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x.DAYS_EMPLOYED.describe()","metadata":{"id":"39o3LvrFiwwu","outputId":"a153024d-bae6-4fa3-b3d2-68b4dee1325a","execution":{"iopub.status.busy":"2022-07-21T12:38:08.083943Z","iopub.execute_input":"2022-07-21T12:38:08.085108Z","iopub.status.idle":"2022-07-21T12:38:08.118766Z","shell.execute_reply.started":"2022-07-21T12:38:08.085040Z","shell.execute_reply":"2022-07-21T12:38:08.117631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x['DAYS_EMPLOYED'] = x['DAYS_EMPLOYED'].apply(lambda x: np.nan if x == 365243 else x)","metadata":{"id":"2vg9TUP_i8LP","execution":{"iopub.status.busy":"2022-07-21T12:38:08.120530Z","iopub.execute_input":"2022-07-21T12:38:08.121794Z","iopub.status.idle":"2022-07-21T12:38:08.346760Z","shell.execute_reply.started":"2022-07-21T12:38:08.121748Z","shell.execute_reply":"2022-07-21T12:38:08.345745Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Clean up","metadata":{"id":"NEbzQncfiuKh"}},{"cell_type":"code","source":"x.hist(figsize = (15,10), bins = 20);","metadata":{"id":"MCIi-lvJsEt7","outputId":"d834efad-4ebf-4c09-9a1f-6d444af16f02","execution":{"iopub.status.busy":"2022-07-21T12:38:08.348404Z","iopub.execute_input":"2022-07-21T12:38:08.348714Z","iopub.status.idle":"2022-07-21T12:38:10.072002Z","shell.execute_reply.started":"2022-07-21T12:38:08.348685Z","shell.execute_reply":"2022-07-21T12:38:10.070828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x.nunique()","metadata":{"id":"xBQbmb_VmIHU","outputId":"bbbd7807-5888-4f35-8511-a465a2e1496e","execution":{"iopub.status.busy":"2022-07-21T12:38:10.073529Z","iopub.execute_input":"2022-07-21T12:38:10.073850Z","iopub.status.idle":"2022-07-21T12:38:10.142428Z","shell.execute_reply.started":"2022-07-21T12:38:10.073823Z","shell.execute_reply":"2022-07-21T12:38:10.141329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Transformations","metadata":{"id":"Jh0cHMgerskm"}},{"cell_type":"code","source":"skew_list = x.skew()","metadata":{"id":"IKMbZcBssC3R","execution":{"iopub.status.busy":"2022-07-21T12:38:10.144046Z","iopub.execute_input":"2022-07-21T12:38:10.144767Z","iopub.status.idle":"2022-07-21T12:38:10.199551Z","shell.execute_reply.started":"2022-07-21T12:38:10.144729Z","shell.execute_reply":"2022-07-21T12:38:10.198408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"skew_list","metadata":{"id":"PA4F-g8_iZfH","outputId":"5c055704-6c22-48b4-8078-55938d655ae9","execution":{"iopub.status.busy":"2022-07-21T12:38:10.201122Z","iopub.execute_input":"2022-07-21T12:38:10.201491Z","iopub.status.idle":"2022-07-21T12:38:10.209678Z","shell.execute_reply.started":"2022-07-21T12:38:10.201460Z","shell.execute_reply":"2022-07-21T12:38:10.208341Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Scaling","metadata":{"id":"3dtuhMJZU_9U"}},{"cell_type":"code","source":"cols_to_scale = x.select_dtypes(exclude = ['object', 'bool']).columns.values.tolist()\ncols_to_scale","metadata":{"id":"r64A9fPdhWhg","outputId":"58c87d06-e1ca-418d-d5e5-051560e9f769","execution":{"iopub.status.busy":"2022-07-21T12:38:10.211262Z","iopub.execute_input":"2022-07-21T12:38:10.211613Z","iopub.status.idle":"2022-07-21T12:38:10.231889Z","shell.execute_reply.started":"2022-07-21T12:38:10.211581Z","shell.execute_reply":"2022-07-21T12:38:10.231004Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"robust_scaler = RobustScaler().fit(x[cols_to_scale])\nstandard_scaler = StandardScaler().fit(x[cols_to_scale])\nminmax_scaler = MinMaxScaler().fit(x[cols_to_scale])","metadata":{"id":"2X5QCeHJgxlH","execution":{"iopub.status.busy":"2022-07-21T12:38:10.232829Z","iopub.execute_input":"2022-07-21T12:38:10.233140Z","iopub.status.idle":"2022-07-21T12:38:10.521108Z","shell.execute_reply.started":"2022-07-21T12:38:10.233111Z","shell.execute_reply":"2022-07-21T12:38:10.519591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x[cols_to_scale] = robust_scaler.transform(x[cols_to_scale])","metadata":{"id":"456dT_JLhHPb","execution":{"iopub.status.busy":"2022-07-21T12:38:10.522763Z","iopub.execute_input":"2022-07-21T12:38:10.523233Z","iopub.status.idle":"2022-07-21T12:38:10.577250Z","shell.execute_reply.started":"2022-07-21T12:38:10.523187Z","shell.execute_reply":"2022-07-21T12:38:10.576420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x","metadata":{"id":"j-A9p071f7Oq","outputId":"a7ef2fd6-f816-474a-d09f-acaeeeee57b5","execution":{"iopub.status.busy":"2022-07-21T12:38:10.578439Z","iopub.execute_input":"2022-07-21T12:38:10.579533Z","iopub.status.idle":"2022-07-21T12:38:10.603831Z","shell.execute_reply.started":"2022-07-21T12:38:10.579500Z","shell.execute_reply":"2022-07-21T12:38:10.602979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Encoding","metadata":{"id":"vP1wZ8ZLVAFN"}},{"cell_type":"code","source":"x = pd.get_dummies(x)","metadata":{"id":"bnN54ey3ftBV","execution":{"iopub.status.busy":"2022-07-21T12:38:10.605477Z","iopub.execute_input":"2022-07-21T12:38:10.606189Z","iopub.status.idle":"2022-07-21T12:38:10.645632Z","shell.execute_reply.started":"2022-07-21T12:38:10.606143Z","shell.execute_reply":"2022-07-21T12:38:10.644322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Imputing Nulls","metadata":{"id":"Zj0kylJnU_xF"}},{"cell_type":"markdown","source":"I initially wanted to use a KNNImputer for filling in null values. However, the amount of time it was taking to execute the algorithm was too much. Therefore, I fell back on using the Iterative Imputer approach.","metadata":{}},{"cell_type":"code","source":"# impute_n = 3\n# imputer = KNNImputer(n_neighbors = impute_n)","metadata":{"id":"TgBTdJWgiA7n","execution":{"iopub.status.busy":"2022-07-21T12:38:10.647054Z","iopub.execute_input":"2022-07-21T12:38:10.647811Z","iopub.status.idle":"2022-07-21T12:38:10.652180Z","shell.execute_reply.started":"2022-07-21T12:38:10.647775Z","shell.execute_reply":"2022-07-21T12:38:10.651410Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"imputer = IterativeImputer()","metadata":{"id":"7ullGr8duPvf","execution":{"iopub.status.busy":"2022-07-21T12:38:10.660037Z","iopub.execute_input":"2022-07-21T12:38:10.661001Z","iopub.status.idle":"2022-07-21T12:38:10.675096Z","shell.execute_reply.started":"2022-07-21T12:38:10.660958Z","shell.execute_reply":"2022-07-21T12:38:10.673889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x.isna().sum()","metadata":{"id":"G8jx5blIiW70","outputId":"ff0fa7bf-06d1-4375-ab99-0f03f92c467e","execution":{"iopub.status.busy":"2022-07-21T12:38:10.676693Z","iopub.execute_input":"2022-07-21T12:38:10.677215Z","iopub.status.idle":"2022-07-21T12:38:10.704739Z","shell.execute_reply.started":"2022-07-21T12:38:10.677180Z","shell.execute_reply":"2022-07-21T12:38:10.703574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x = pd.DataFrame(imputer.fit_transform(x), columns = x.columns)","metadata":{"id":"FTPbcBDUuXcP","execution":{"iopub.status.busy":"2022-07-21T12:38:10.706078Z","iopub.execute_input":"2022-07-21T12:38:10.706493Z","iopub.status.idle":"2022-07-21T12:38:23.520546Z","shell.execute_reply.started":"2022-07-21T12:38:10.706463Z","shell.execute_reply":"2022-07-21T12:38:23.519196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x","metadata":{"id":"pw_mKEPuuj7_","outputId":"b5214c55-c34d-4526-fba5-14d3f99335c1","execution":{"iopub.status.busy":"2022-07-21T12:38:23.525908Z","iopub.execute_input":"2022-07-21T12:38:23.526275Z","iopub.status.idle":"2022-07-21T12:38:23.552697Z","shell.execute_reply.started":"2022-07-21T12:38:23.526243Z","shell.execute_reply":"2022-07-21T12:38:23.551528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x.isna().sum().sum()","metadata":{"id":"z3uHtWmupvKT","outputId":"5ad1ceb7-ac4d-4ee5-a04b-3feb20738d3c","execution":{"iopub.status.busy":"2022-07-21T12:38:23.554563Z","iopub.execute_input":"2022-07-21T12:38:23.555079Z","iopub.status.idle":"2022-07-21T12:38:23.571318Z","shell.execute_reply.started":"2022-07-21T12:38:23.555032Z","shell.execute_reply":"2022-07-21T12:38:23.570301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Unstacking the train and test sets","metadata":{"id":"8hwgp4S7u-Z9"}},{"cell_type":"code","source":"train = x[x['TRAINING'] == True]","metadata":{"id":"SElgEw8lu-HV","execution":{"iopub.status.busy":"2022-07-21T12:38:23.572789Z","iopub.execute_input":"2022-07-21T12:38:23.573409Z","iopub.status.idle":"2022-07-21T12:38:23.589937Z","shell.execute_reply.started":"2022-07-21T12:38:23.573346Z","shell.execute_reply":"2022-07-21T12:38:23.588744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test = x[x['TRAINING'] == False]","metadata":{"id":"o_IAKEQsvOZv","execution":{"iopub.status.busy":"2022-07-21T12:38:23.591281Z","iopub.execute_input":"2022-07-21T12:38:23.591666Z","iopub.status.idle":"2022-07-21T12:38:23.600411Z","shell.execute_reply.started":"2022-07-21T12:38:23.591634Z","shell.execute_reply":"2022-07-21T12:38:23.599328Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.drop(columns = 'TRAINING', inplace=True)\ntest.drop(columns = 'TRAINING', inplace=True)","metadata":{"id":"TVyAcba11J3H","outputId":"4e9f1908-fd7d-4b24-de97-e256281daf28","execution":{"iopub.status.busy":"2022-07-21T12:38:23.602047Z","iopub.execute_input":"2022-07-21T12:38:23.602478Z","iopub.status.idle":"2022-07-21T12:38:23.623515Z","shell.execute_reply.started":"2022-07-21T12:38:23.602447Z","shell.execute_reply":"2022-07-21T12:38:23.622631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# SMOTE\nBecause the target has been very unbalanced, I used SMOTE in order to synthetically create more datapoints for the model to train on.","metadata":{"id":"40_73lmIiEh9"}},{"cell_type":"code","source":"smote = SMOTE()","metadata":{"id":"xt5QO8n0tpac","execution":{"iopub.status.busy":"2022-07-21T12:38:23.624686Z","iopub.execute_input":"2022-07-21T12:38:23.625153Z","iopub.status.idle":"2022-07-21T12:38:23.629343Z","shell.execute_reply.started":"2022-07-21T12:38:23.625125Z","shell.execute_reply":"2022-07-21T12:38:23.628519Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x_smote, y_smote = smote.fit_resample(train, y)","metadata":{"id":"FZET9fnavWNW","execution":{"iopub.status.busy":"2022-07-21T12:38:23.630672Z","iopub.execute_input":"2022-07-21T12:38:23.631178Z","iopub.status.idle":"2022-07-21T12:38:25.980935Z","shell.execute_reply.started":"2022-07-21T12:38:23.631147Z","shell.execute_reply":"2022-07-21T12:38:25.979948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training Data","metadata":{"id":"BaKowKDvXAHE"}},{"cell_type":"code","source":"xtrain, xtest, ytrain, ytest = train_test_split(x_smote, y_smote, test_size = .3, random_state = 7)","metadata":{"id":"Kn9vvJZwvgmn","execution":{"iopub.status.busy":"2022-07-21T12:38:25.981978Z","iopub.execute_input":"2022-07-21T12:38:25.983035Z","iopub.status.idle":"2022-07-21T12:38:26.111308Z","shell.execute_reply.started":"2022-07-21T12:38:25.982998Z","shell.execute_reply":"2022-07-21T12:38:26.110118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# KNN Classifier","metadata":{"id":"i_-nAlgivxab"}},{"cell_type":"code","source":"n = 7\nknn = KNeighborsClassifier(n_neighbors = n).fit(xtrain, ytrain)","metadata":{"id":"nHJ8n9HZvzuU","execution":{"iopub.status.busy":"2022-07-21T12:38:26.112971Z","iopub.execute_input":"2022-07-21T12:38:26.113328Z","iopub.status.idle":"2022-07-21T12:38:27.160609Z","shell.execute_reply.started":"2022-07-21T12:38:26.113295Z","shell.execute_reply":"2022-07-21T12:38:27.159374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn_predict = knn.predict(xtest)","metadata":{"id":"h3wvmuwtwzdc","execution":{"iopub.status.busy":"2022-07-21T12:38:27.161855Z","iopub.execute_input":"2022-07-21T12:38:27.162174Z","iopub.status.idle":"2022-07-21T12:39:53.459523Z","shell.execute_reply.started":"2022-07-21T12:38:27.162147Z","shell.execute_reply":"2022-07-21T12:39:53.457994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn_score = knn.score(xtest, ytest)\nknn_score","metadata":{"id":"SfVpDZIiwG00","outputId":"df4d3ba1-a8f9-4dff-cfa5-e6f9fdc938d5","execution":{"iopub.status.busy":"2022-07-21T12:39:53.462204Z","iopub.execute_input":"2022-07-21T12:39:53.462780Z","iopub.status.idle":"2022-07-21T12:41:11.510113Z","shell.execute_reply.started":"2022-07-21T12:39:53.462725Z","shell.execute_reply":"2022-07-21T12:41:11.508998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn_recall = recall_score(ytest, knn_predict)\nknn_precision = precision_score(ytest, knn_predict)","metadata":{"id":"ZcczwjIOwsw0","execution":{"iopub.status.busy":"2022-07-21T12:41:11.512455Z","iopub.execute_input":"2022-07-21T12:41:11.513230Z","iopub.status.idle":"2022-07-21T12:41:11.682851Z","shell.execute_reply.started":"2022-07-21T12:41:11.513179Z","shell.execute_reply":"2022-07-21T12:41:11.682005Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn_score, knn_recall, knn_precision","metadata":{"id":"3HZAyNfkxUDF","outputId":"9fd09142-6802-4090-c0a6-6b703c3ac401","execution":{"iopub.status.busy":"2022-07-21T12:41:11.684119Z","iopub.execute_input":"2022-07-21T12:41:11.685133Z","iopub.status.idle":"2022-07-21T12:41:11.692054Z","shell.execute_reply.started":"2022-07-21T12:41:11.685098Z","shell.execute_reply":"2022-07-21T12:41:11.690842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn_rocauc = roc_auc_score(ytest, knn.predict_proba(xtest)[:,1])","metadata":{"id":"cwgyh-pK4QMG","execution":{"iopub.status.busy":"2022-07-21T12:41:11.694389Z","iopub.execute_input":"2022-07-21T12:41:11.695258Z","iopub.status.idle":"2022-07-21T12:42:23.819137Z","shell.execute_reply.started":"2022-07-21T12:41:11.695212Z","shell.execute_reply":"2022-07-21T12:42:23.817790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn_rocauc","metadata":{"id":"tekSWp3A5Nad","outputId":"3a9ffbdd-d6df-4c34-8252-a1fa455a95c5","execution":{"iopub.status.busy":"2022-07-21T12:42:23.820634Z","iopub.execute_input":"2022-07-21T12:42:23.821067Z","iopub.status.idle":"2022-07-21T12:42:23.828254Z","shell.execute_reply.started":"2022-07-21T12:42:23.821032Z","shell.execute_reply":"2022-07-21T12:42:23.827232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Logistic Regression","metadata":{"id":"ly05RtOBxW8U"}},{"cell_type":"code","source":"log = LogisticRegression().fit(xtrain, ytrain)","metadata":{"id":"ggOR7AaQxYvQ","execution":{"iopub.status.busy":"2022-07-21T12:42:23.829641Z","iopub.execute_input":"2022-07-21T12:42:23.829987Z","iopub.status.idle":"2022-07-21T12:42:24.748395Z","shell.execute_reply.started":"2022-07-21T12:42:23.829957Z","shell.execute_reply":"2022-07-21T12:42:24.747040Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"log_predict = log.predict(xtest)","metadata":{"id":"07jDKpkvxdDG","execution":{"iopub.status.busy":"2022-07-21T12:42:24.750717Z","iopub.execute_input":"2022-07-21T12:42:24.751764Z","iopub.status.idle":"2022-07-21T12:42:24.767452Z","shell.execute_reply.started":"2022-07-21T12:42:24.751709Z","shell.execute_reply":"2022-07-21T12:42:24.765569Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"log_score = log.score(xtest, ytest)\nlog_recall = recall_score(ytest, log_predict)\nlog_precision = precision_score(ytest, log_predict)","metadata":{"id":"FIw2qpBcxiIO","execution":{"iopub.status.busy":"2022-07-21T12:42:24.770698Z","iopub.execute_input":"2022-07-21T12:42:24.771802Z","iopub.status.idle":"2022-07-21T12:42:25.015718Z","shell.execute_reply.started":"2022-07-21T12:42:24.771746Z","shell.execute_reply":"2022-07-21T12:42:25.014623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"log_score, log_recall, log_precision","metadata":{"id":"WITb1OevxtuZ","outputId":"bbe049d1-9013-4906-ba57-70163e2c572a","execution":{"iopub.status.busy":"2022-07-21T12:42:25.017340Z","iopub.execute_input":"2022-07-21T12:42:25.018050Z","iopub.status.idle":"2022-07-21T12:42:25.024614Z","shell.execute_reply.started":"2022-07-21T12:42:25.018016Z","shell.execute_reply":"2022-07-21T12:42:25.023578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"log_rocauc = roc_auc_score(ytest, log.predict_proba(xtest)[:,1])","metadata":{"id":"Ubs-GhHW43E5","execution":{"iopub.status.busy":"2022-07-21T12:42:25.026180Z","iopub.execute_input":"2022-07-21T12:42:25.026651Z","iopub.status.idle":"2022-07-21T12:42:25.165269Z","shell.execute_reply.started":"2022-07-21T12:42:25.026617Z","shell.execute_reply":"2022-07-21T12:42:25.163382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"log_rocauc","metadata":{"id":"83JdFJIb46hS","outputId":"08d8273e-665e-45cf-ed00-119311d62a50","execution":{"iopub.status.busy":"2022-07-21T12:42:25.166933Z","iopub.execute_input":"2022-07-21T12:42:25.168040Z","iopub.status.idle":"2022-07-21T12:42:25.179149Z","shell.execute_reply.started":"2022-07-21T12:42:25.167990Z","shell.execute_reply":"2022-07-21T12:42:25.177755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# LGBM Classifier","metadata":{"id":"aHfRWKpfx-_r"}},{"cell_type":"code","source":"lgbm = LGBMClassifier()\nlgbm.fit(xtrain, ytrain)","metadata":{"id":"QrzhqTlDx-uH","outputId":"7f5d9ebb-ac34-4d2c-cb42-9fc533c37af8","execution":{"iopub.status.busy":"2022-07-21T12:42:25.181478Z","iopub.execute_input":"2022-07-21T12:42:25.182656Z","iopub.status.idle":"2022-07-21T12:42:27.208816Z","shell.execute_reply.started":"2022-07-21T12:42:25.182610Z","shell.execute_reply":"2022-07-21T12:42:27.207873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lgbm_predict = lgbm.predict(xtest)","metadata":{"id":"A_pukqlZyK9S","execution":{"iopub.status.busy":"2022-07-21T12:42:27.213178Z","iopub.execute_input":"2022-07-21T12:42:27.214914Z","iopub.status.idle":"2022-07-21T12:42:27.640235Z","shell.execute_reply.started":"2022-07-21T12:42:27.214875Z","shell.execute_reply":"2022-07-21T12:42:27.639305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lgbm_score = lgbm.score(xtest, ytest)\nlgbm_recall = recall_score(ytest, lgbm_predict)\nlgbm_precision = precision_score(ytest, lgbm_predict)","metadata":{"id":"NwrEM65fyde_","execution":{"iopub.status.busy":"2022-07-21T12:42:27.644851Z","iopub.execute_input":"2022-07-21T12:42:27.646077Z","iopub.status.idle":"2022-07-21T12:42:28.259871Z","shell.execute_reply.started":"2022-07-21T12:42:27.646028Z","shell.execute_reply":"2022-07-21T12:42:28.258613Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lgbm_score, lgbm_recall, lgbm_precision","metadata":{"id":"SdwW28MVymon","outputId":"9d258261-8145-40a5-e04e-193b9599dd72","execution":{"iopub.status.busy":"2022-07-21T12:42:28.261341Z","iopub.execute_input":"2022-07-21T12:42:28.261711Z","iopub.status.idle":"2022-07-21T12:42:28.269505Z","shell.execute_reply.started":"2022-07-21T12:42:28.261679Z","shell.execute_reply":"2022-07-21T12:42:28.268203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lgbm_rocauc = roc_auc_score(ytest, lgbm.predict_proba(xtest)[:,1])","metadata":{"id":"H--hpL1c5XRk","execution":{"iopub.status.busy":"2022-07-21T12:42:28.270833Z","iopub.execute_input":"2022-07-21T12:42:28.271196Z","iopub.status.idle":"2022-07-21T12:42:28.762892Z","shell.execute_reply.started":"2022-07-21T12:42:28.271168Z","shell.execute_reply":"2022-07-21T12:42:28.762046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lgbm_rocauc","metadata":{"id":"X-BKbNcW5cXD","outputId":"6ff14885-a6f3-4805-c85b-839e059d3218","execution":{"iopub.status.busy":"2022-07-21T12:42:28.764274Z","iopub.execute_input":"2022-07-21T12:42:28.764860Z","iopub.status.idle":"2022-07-21T12:42:28.770404Z","shell.execute_reply.started":"2022-07-21T12:42:28.764826Z","shell.execute_reply":"2022-07-21T12:42:28.769568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# MLPClassifier","metadata":{"id":"pYLcx6zzrRvk"}},{"cell_type":"code","source":"mlp = MLPClassifier(hidden_layer_sizes=(100, 20), early_stopping = True, max_iter = 200)","metadata":{"id":"grBg7tvvsGnC","execution":{"iopub.status.busy":"2022-07-21T12:42:28.771904Z","iopub.execute_input":"2022-07-21T12:42:28.772228Z","iopub.status.idle":"2022-07-21T12:42:28.801377Z","shell.execute_reply.started":"2022-07-21T12:42:28.772193Z","shell.execute_reply":"2022-07-21T12:42:28.800208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mlp.fit(xtrain, ytrain)","metadata":{"id":"knlqeApfsdp3","outputId":"17c21d56-356c-4375-d081-0a4f3a02cfe4","execution":{"iopub.status.busy":"2022-07-21T12:42:28.802996Z","iopub.execute_input":"2022-07-21T12:42:28.803628Z","iopub.status.idle":"2022-07-21T12:46:14.666969Z","shell.execute_reply.started":"2022-07-21T12:42:28.803593Z","shell.execute_reply":"2022-07-21T12:46:14.665711Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mlp_predict = mlp.predict(xtest)","metadata":{"id":"eguKqF7Ks2sy","execution":{"iopub.status.busy":"2022-07-21T12:46:14.668742Z","iopub.execute_input":"2022-07-21T12:46:14.669634Z","iopub.status.idle":"2022-07-21T12:46:15.523688Z","shell.execute_reply.started":"2022-07-21T12:46:14.669590Z","shell.execute_reply":"2022-07-21T12:46:15.522322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mlp_score = mlp.score(xtest, ytest)\nmlp_recall = recall_score(ytest, mlp_predict)\nmlp_precision = precision_score(ytest, mlp_predict)","metadata":{"id":"toK9iCi2s7xm","execution":{"iopub.status.busy":"2022-07-21T12:46:15.525807Z","iopub.execute_input":"2022-07-21T12:46:15.526653Z","iopub.status.idle":"2022-07-21T12:46:16.601836Z","shell.execute_reply.started":"2022-07-21T12:46:15.526608Z","shell.execute_reply":"2022-07-21T12:46:16.600653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mlp_score, mlp_recall, mlp_precision","metadata":{"id":"YaBDLblytLad","outputId":"141f505b-ede3-4ac1-f1ba-b043d3a1de7b","execution":{"iopub.status.busy":"2022-07-21T12:46:16.603105Z","iopub.execute_input":"2022-07-21T12:46:16.603465Z","iopub.status.idle":"2022-07-21T12:46:16.609882Z","shell.execute_reply.started":"2022-07-21T12:46:16.603435Z","shell.execute_reply":"2022-07-21T12:46:16.608810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mlp_rocauc = roc_auc_score(ytest, mlp.predict_proba(xtest)[:,1])","metadata":{"id":"PJfHtq3StZ92","execution":{"iopub.status.busy":"2022-07-21T12:46:16.611217Z","iopub.execute_input":"2022-07-21T12:46:16.611550Z","iopub.status.idle":"2022-07-21T12:46:17.571595Z","shell.execute_reply.started":"2022-07-21T12:46:16.611522Z","shell.execute_reply":"2022-07-21T12:46:17.570215Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mlp_rocauc","metadata":{"id":"JEGGfn_KteCE","outputId":"a959e232-957b-4fea-83fa-893461ec6763","execution":{"iopub.status.busy":"2022-07-21T12:46:17.573727Z","iopub.execute_input":"2022-07-21T12:46:17.574963Z","iopub.status.idle":"2022-07-21T12:46:17.581054Z","shell.execute_reply.started":"2022-07-21T12:46:17.574927Z","shell.execute_reply":"2022-07-21T12:46:17.579939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Summarized results","metadata":{"id":"gfHcPXCFyuUD"}},{"cell_type":"code","source":"print('{:.4f}'.format(knn_score), '- KNN Accuracy w/', n, 'neighbors')\nprint('{:.4f}'.format(knn_recall), '- KNN Recall w/', n, 'neighbors')\nprint('{:.4f}'.format(knn_precision), '- KNN Precision w/', n, 'neighbors')\nprint('{:.4f}'.format(knn_rocauc), '- KNN ROC AUC w/', n, 'neighbors')\nprint()\nprint('{:.4f}'.format(log_score), '- Log Accuracy')\nprint('{:.4f}'.format(log_recall), '- Log Recall')\nprint('{:.4f}'.format(log_precision), '- Log Precision')\nprint('{:.4f}'.format(log_rocauc), '- Log ROC AUC')\nprint()\nprint('{:.4f}'.format(lgbm_score), '- LGBM Accuracy')\nprint('{:.4f}'.format(lgbm_recall), '- LGBM Recall')\nprint('{:.4f}'.format(lgbm_precision), '- LGBM Precision')\nprint('{:.4f}'.format(lgbm_rocauc), '- LGBM ROC AUC')\nprint()\nprint('{:.4f}'.format(mlp_score), '- MLP Accuracy')\nprint('{:.4f}'.format(mlp_recall), '- MLP Recall')\nprint('{:.4f}'.format(mlp_precision), '- MLP Precision')\nprint('{:.4f}'.format(mlp_rocauc), '- MLP ROC AUC')","metadata":{"id":"Rq6Sqz7cyvZC","outputId":"18b82abc-1b1b-4560-c404-2b3fe667d924","execution":{"iopub.status.busy":"2022-07-21T12:46:17.582521Z","iopub.execute_input":"2022-07-21T12:46:17.582873Z","iopub.status.idle":"2022-07-21T12:46:17.595699Z","shell.execute_reply.started":"2022-07-21T12:46:17.582844Z","shell.execute_reply":"2022-07-21T12:46:17.594567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predicting","metadata":{"id":"h_Rp7wes01Nh"}},{"cell_type":"code","source":"submission = pd.read_csv('../input/home-credit-default-risk/application_test.csv')","metadata":{"id":"SjC_D0FqujdH","execution":{"iopub.status.busy":"2022-07-21T12:48:40.414091Z","iopub.execute_input":"2022-07-21T12:48:40.414524Z","iopub.status.idle":"2022-07-21T12:48:41.070926Z","shell.execute_reply.started":"2022-07-21T12:48:40.414491Z","shell.execute_reply":"2022-07-21T12:48:41.069776Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## KNN","metadata":{"id":"NOIW6un1emU0"}},{"cell_type":"code","source":"submit_knn = knn.predict(test)\nsubmission_clean = submission[['SK_ID_CURR']]\nsubmission_clean['TARGET'] = submit_knn\nsubmission_clean.to_csv('./submission (KNN)', index=False)","metadata":{"id":"N3I_Qc9o2Kek","outputId":"41076873-0560-4ed9-bb6f-332938c34f16","execution":{"iopub.status.busy":"2022-07-21T12:48:48.938125Z","iopub.execute_input":"2022-07-21T12:48:48.938661Z","iopub.status.idle":"2022-07-21T12:49:19.389452Z","shell.execute_reply.started":"2022-07-21T12:48:48.938616Z","shell.execute_reply":"2022-07-21T12:49:19.388098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Logistic Regression","metadata":{"id":"hjO1V8J2en_l"}},{"cell_type":"code","source":"submit_log = log.predict(test)\nsubmission_clean = submission[['SK_ID_CURR']]\nsubmission_clean['TARGET'] = submit_log\nsubmission_clean.to_csv('./submission (logistic)', index=False)","metadata":{"id":"4Zd1JI5EeyTU","outputId":"c70f3ee9-8dda-42f5-b560-715aead5e268","execution":{"iopub.status.busy":"2022-07-21T12:49:30.393813Z","iopub.execute_input":"2022-07-21T12:49:30.394198Z","iopub.status.idle":"2022-07-21T12:49:30.547878Z","shell.execute_reply.started":"2022-07-21T12:49:30.394168Z","shell.execute_reply":"2022-07-21T12:49:30.546445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## LGB","metadata":{"id":"01W87_f6en2C"}},{"cell_type":"code","source":"submit_lgbm = lgbm.predict(test)\nsubmission_clean = submission[['SK_ID_CURR']]\nsubmission_clean['TARGET'] = submit_lgbm\nsubmission_clean.to_csv('./submission (LGBM)', index=False)","metadata":{"id":"SWy1rJi5e7-i","outputId":"7f970705-cef4-45be-98e8-ed8c479ec1d0","execution":{"iopub.status.busy":"2022-07-21T12:49:37.284715Z","iopub.execute_input":"2022-07-21T12:49:37.285146Z","iopub.status.idle":"2022-07-21T12:49:37.498743Z","shell.execute_reply.started":"2022-07-21T12:49:37.285109Z","shell.execute_reply":"2022-07-21T12:49:37.497630Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## MLP","metadata":{"id":"-qWWuj4KuKDM"}},{"cell_type":"code","source":"submit_mlp = mlp.predict(test)\nsubmission_clean = submission[['SK_ID_CURR']]\nsubmission_clean['TARGET'] = submit_mlp\nsubmission_clean.to_csv('./submission (MLP)', index=False)","metadata":{"id":"Hf1OpKfJuK_l","outputId":"363eb07d-4228-40b4-a041-9f23985f29f8","execution":{"iopub.status.busy":"2022-07-21T12:49:39.773246Z","iopub.execute_input":"2022-07-21T12:49:39.773650Z","iopub.status.idle":"2022-07-21T12:49:40.173876Z","shell.execute_reply.started":"2022-07-21T12:49:39.773621Z","shell.execute_reply":"2022-07-21T12:49:40.172524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Blend","metadata":{"id":"uJK41RdQfnlZ"}},{"cell_type":"code","source":"submission_clean = submission[['SK_ID_CURR']]\nsubmission_clean['TARGET_KNN'] = submit_knn\nsubmission_clean['TARGET_LOG'] = submit_log\nsubmission_clean['TARGET_LGBM'] = submit_lgbm\nsubmission_clean['TARGET_MLP'] = submit_mlp","metadata":{"id":"VwEZMy_LfoTo","outputId":"5359bba7-2d07-41a5-f032-cc2e62cb311c","execution":{"iopub.status.busy":"2022-07-21T12:49:43.188182Z","iopub.execute_input":"2022-07-21T12:49:43.188573Z","iopub.status.idle":"2022-07-21T12:49:43.200226Z","shell.execute_reply.started":"2022-07-21T12:49:43.188538Z","shell.execute_reply":"2022-07-21T12:49:43.198921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_clean['TARGET'] = submission_clean[['TARGET_KNN', 'TARGET_LOG', 'TARGET_LGBM', 'TARGET_MLP']].mean(axis=1)\nsubmission_clean['TARGET'] = submission_clean['TARGET'].round(decimals =0)\nsubmission_clean.drop(columns = ['TARGET_KNN', 'TARGET_LOG', 'TARGET_LGBM', 'TARGET_MLP'], inplace=True)\nsubmission_clean.to_csv('./submission(blend)', index=False)","metadata":{"id":"u6FVED9Bf1B2","outputId":"5c1d51d2-f6c0-4d37-ddb1-90d67e9cf9c2","execution":{"iopub.status.busy":"2022-07-21T12:50:04.067342Z","iopub.execute_input":"2022-07-21T12:50:04.067728Z","iopub.status.idle":"2022-07-21T12:50:04.101693Z","shell.execute_reply.started":"2022-07-21T12:50:04.067699Z","shell.execute_reply":"2022-07-21T12:50:04.100533Z"},"trusted":true},"execution_count":null,"outputs":[]}]}