{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b4c8dc59d34e24cdcdf85c14a4cc54b27559a953"},"cell_type":"markdown","source":"#  Introduction\nThis EDA is taking these 4 steps.  \nStep1 : Overview the data  \nStep2 : See the relationship between...   \n <span>　</span>a) \"AMT_INCOME_TOTAL\" & repayment or non-repayment rate   \n <span>　</span>b) \"OCCUPATION_TYPE\" & repayment or non-repayment rate  \nStep3 : Picking up other features that will help to predict custmer's repayment ability  \nStep4 : Plot non-repayment rate for all selected features\n  \nIn Step1, overview the data to check what kind of data in there and if there is any missing values. Then, define non-repayment rate from target data.  \nIn Step2, I forcus on \"AMT_INCOME_TOATL\" & \"OCCUPATION_TYPE\" features because, generaly thinking, these features seem to relate to custmer's repayment ability. It's easy to imagine that low income cause of default so I expect the tendecy which is \"if income gets lower then non-repayment rate becomes higher and if income gets higher then non-repayment rate becomes lower\". I will test this \"theory\" in step2. Then I will check \"OCCUPATION_TYPE\"  because I believe it relates to income level and also occupation infomation can be reflected personal feature more than income.   \nSo, I will check if \"AMT_INCOME_TOATL\" & \"OCCUPATION_TYPE\"  can help to predict custmer's repayment ability or not in Step2.   \nIn Step3,  I will pick up other important features with using random forest analysis.  \nIn Step4, Plot non-repayment rate for all selected fesatures  "},{"metadata":{"_uuid":"bda18cd75cf393ac770ba0df055b78d8e0245371"},"cell_type":"markdown","source":"# Step1.  Overview the data\noverview the data to check what kind of data in there and  if there is any missing values. Then define non-repayment rate from target data."},{"metadata":{"_uuid":"ecc21ba26c9ab21b545f616046e17c180038c579"},"cell_type":"markdown","source":">### Load the data & take a look what kind of data in there"},{"metadata":{"trusted":true,"_uuid":"9956d540a921ae904dfc1a28e1d2c9cddd7d9476"},"cell_type":"code","source":"import seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c360be119f3be3a690992b4ed5112d6e840dd23b","_kg_hide-output":false},"cell_type":"code","source":"data_application_train = pd.read_csv(\"../input/application_train.csv\")\npd.options.display.max_columns = len(data_application_train.columns)\ndata_application_train.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2bf36922358d9987fb4a4aaee0fd0f7a0728fd56"},"cell_type":"markdown","source":"> ### Check missing data\nPlot missing data rate with \"missingno\" module and calculate missing rate for \"AMT_INCOME_TOATL\" & \"OCCUPATION_TYPE\""},{"metadata":{"trusted":true,"_uuid":"8b17b837a9542a2ef5600204e7ea783cfe300dca"},"cell_type":"code","source":"import missingno as msno\nmsno.bar(data_application_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"37c49b32a1210d13c8f14a657a38819361151360"},"cell_type":"code","source":"total = len(data_application_train)\nsum_income = data_application_train[\"AMT_INCOME_TOTAL\"].isnull().sum()\nsum_occupation = data_application_train[\"OCCUPATION_TYPE\"].isnull().sum()\n\nprint(\"AMT_INCOME_TOTAL  num of missing data={}　 missing data rate={:.2f}[%]\".format(sum_income, 100*sum_income/total))\nprint(\"OCCUPATION_TYPE   num of missing data={}　 missing data rate={:.2f}[%]\".format(sum_occupation, 100*sum_occupation/total))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"61e6e5defbd6a76e374d9db167ab40d4b17fdc22"},"cell_type":"markdown","source":">### Define non-repayment rate from target data\n\"Home_credit_columns_description.csv\" says target data is defined as    \n*\"1 - client with payment difficulties: he/she had late payment more than X days on at least one of the first Y installments of the loan in our sample, 0 - all other cases\"*  \nHere, I intoroduce \"non-repayment rate\" and define it as  $\\frac {sum(1)}  {(sum(1) + sum (0))}$ .   If I calculate \"non-repayment rate\" across the whole data, then result is 8:07[%] ( It also can be calculated by mean of data because target data is 0 & 1 )"},{"metadata":{"trusted":true,"_uuid":"40a76a0e360f3cc28081ee49146dd1e1eaf0b144"},"cell_type":"code","source":"sns.countplot(x=\"TARGET\", data=data_application_train)\nplt.title(\"Data volume in each class('Target')\")\nplt.show()\n\ncount_1 = (data_application_train[\"TARGET\"] == 1).sum()\ncount_0 = (data_application_train[\"TARGET\"] == 0).sum()\nprint(\"Non-repayment rate={:.2f}[%]\".format((count_1/(count_0+count_1))*100))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"01aeecb96209e37c5b72fd0c191297121afc5f68"},"cell_type":"markdown","source":"# Step2.  See the relationship between...   \n **<span>　</span>a) \"AMT_INCOME_TOTAL\" & Non-repayment rate**  \n **<span>　</span>b) \"OCCUPATION_TYPE\" & Non-repayment rate**"},{"metadata":{"_uuid":"fbf392d9441670cfaaf81097f47a71f329b4f37f"},"cell_type":"markdown","source":"###  See the relationship between \"AMT_INCOME_TOTAL\" and  Non-repayment rate\nTest the theory \"low income = high non-repayment rate\" & \"high income = low non-repayment rate\".  \n**Method**\n1. Benning income data\n2. Calculate Non-repayment rate in each the bins\n3. Plot \"Binned income\" vs \"Non-repayment\""},{"metadata":{"trusted":true,"_uuid":"dbd6bbdd09396b2a7bcc58c1e40b73f904e04d87"},"cell_type":"code","source":"#Extract require data\ndata_to_use = data_application_train[[\"TARGET\",\"AMT_INCOME_TOTAL\",\"OCCUPATION_TYPE\"]]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e9ffb328957331647d01c55ddf071b2af78ad71d"},"cell_type":"markdown","source":"To decide bin width, checking \"AMT_INCOME_TOTAL\" distribution below.   \nFrom the distribution, we understand we can't use max & min value to decide bin width becasue there are outliers in the data. So I'm going to use quartile value to decide bin width."},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"608dca016109aff9d91776baf048be4174017633"},"cell_type":"code","source":"sns.violinplot(y='AMT_INCOME_TOTAL', data=data_to_use)\nplt.ylabel(\"Income\")\nplt.title(\"Income distribution\")\nplt.show()\ndata_to_use.describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"503135acd4331d99f7eede4358b089d89dd38623"},"cell_type":"markdown","source":"Now binning data by quartile value"},{"metadata":{"trusted":true,"_uuid":"988e510add1fbdec5d953018114c7cf1acf8ac7c"},"cell_type":"code","source":"def binning_data(data_source, col_name, binned_col_name, num_of_bin=10):\n    \"\"\"\n    Binning the specified column data\n    & add the new column that has bin label to original data\n    \n    parameter\n    --------------\n    data_source : Pandas dataframe\n    col_name : string \n    binned_col_name : string \n    num_of_bin : int\n    \n    return\n    --------------\n    data_source : pandas dataframe\n    \n    \"\"\"\n    #Check quartile value\n    data_info = data_source.describe()\n    bin_min = data_info.loc[\"25%\", col_name]\n    bin_max = data_info.loc[\"75%\", col_name]\n    IQR = bin_max - bin_min\n    #Use interquartile value to decide bin_min & max value\n    bin_min = bin_min-IQR*1.5 if bin_min > IQR*1.5 else 0\n    bin_max += IQR*1.5\n    \n    bin_width = (int)((bin_max - bin_min) / num_of_bin)\n    #bins = [value for value in gen_range(bin_min, bin_max, bin_width )]\n    bins = [value for value in range((int)(bin_min), (int)(bin_max), bin_width )]\n    labels = [i for i in range(0, len(bins)-1)]\n    binned_label = pd.cut(data_source[col_name], bins=bins, labels=labels)\n    data_source[binned_col_name] = binned_label\n    \n    for i in range(0, len(bins)-1):\n        print(\"range label={} : range={:.5f}~{:.5f}\".format(i, bins[i], bins[i+1]) )\n    \n    return data_source","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f77c249febe8bb4db8111a16e847400c4bfbb82f"},"cell_type":"code","source":"#Binning data\ndata_to_use = binning_data(data_to_use, \"AMT_INCOME_TOTAL\", \"BINNED_AMT_INCOME_TOTAL\", 8)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"91909f12adee01c4cc620e2083fb24b005f66985"},"cell_type":"markdown","source":"1. Income distribution after benning"},{"metadata":{"trusted":true,"_uuid":"ddcafa2423ba5bfe80a4210fb6be90e77a54cca8"},"cell_type":"code","source":"#Check distribution\nincome_destribution_w_income_range = data_to_use.groupby(\"BINNED_AMT_INCOME_TOTAL\", as_index=False).count()\nplt.bar(income_destribution_w_income_range[\"BINNED_AMT_INCOME_TOTAL\"]\n        , income_destribution_w_income_range[\"TARGET\"])\nplt.xlabel('Low<- Income range label ->High)')\nplt.ylabel(\"count\")\nplt.title(\"Income distribution\")\nfor x, y in zip(income_destribution_w_income_range[\"BINNED_AMT_INCOME_TOTAL\"]\n                , income_destribution_w_income_range[\"TARGET\"]):\n    plt.text(x, y, y, ha='center', va='bottom')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c5c472aa9e717c3ea427ca60d24c61aac20a26c1"},"cell_type":"markdown","source":"Finally, calculate non-repayment rate in each bins & plot them"},{"metadata":{"trusted":true,"_uuid":"3a7354f86bfdec7e6468fc3afb7d6714c5969f7b"},"cell_type":"code","source":"#Cal non-repayment rate\nnon_repayment_rate_w_income_range = data_to_use.groupby(\"BINNED_AMT_INCOME_TOTAL\", as_index=False).mean()\nnon_repayment_rate_w_income_range[\"TARGET\"] *= 100 #% notation\n\n#Bar plot\nplt.figure(figsize=(10,6))\nplt.bar(non_repayment_rate_w_income_range[\"BINNED_AMT_INCOME_TOTAL\"]\n        , non_repayment_rate_w_income_range[\"TARGET\"], color=\"Blue\")\nplt.xlabel('Low<- Income range label ->High')\nplt.ylabel('non-payment rate[%]')\nplt.ylim(0,10)\nplt.title(\"non-payment rate\")\n\nfor x, y in zip(non_repayment_rate_w_income_range[\"BINNED_AMT_INCOME_TOTAL\"]\n                , non_repayment_rate_w_income_range[\"TARGET\"]):\n    plt.text( x, y, str(\"{:.2f}\").format(y), ha='center', va='bottom')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3862c2fde4ffd7c4d27c25f931aa6ecd34c3b032"},"cell_type":"markdown","source":"####  Conclusion of  \"AMT_INCOME_TOTAL\" vs  \"Non-repayment rate\" analysis\n1. What I expected is \"low income => high non-repayment rate\" . However no significant differernce is observed among Range0 to 4 (Across low income to middle).  \n2. Highest non-repayment rate is observed in Range2, which is considered average income range.\n3. \"high income => low non-repayment rate\" seems right. Range6 & 7 show lower non-repayment rate than other\n4. Non-repayment rate difference between higest & lowest is 2.4%. It is not the big difference as I expected.    \n5. From this analysis,  I can say...  \n\"Low income isn't the convincing reason of loan rejection\" or \"Home credit may have careful loan assessment for low income customers (this is why we don't see significant non-repayment rate differernce in the result)\"  \n6. However, becasue no significant non-repayment rate difference is observed among the income ranges,  I can't conclude that \"AMT_INCOME_TOTAL\" is important feature. I will test  \"AMT_INCOME_TOTAL\" importance with Random forest analysis in Step3"},{"metadata":{"_uuid":"fe453469f55701df8cc493d8a20ba4a2f4f60b46"},"cell_type":"markdown","source":"###  See the relationship between \"OCCUPATION_TYPE\" and  Non-repayment rate\n**Method**\n1. Calculate non-repayment rate in each \"OCCUPATION_TYPE\"\n2. Plot \"OCCUPATION_TYPE vs \"Non-repayment\""},{"metadata":{"trusted":true,"_uuid":"7e880ff627080cc5a3a07e81f0a0425f86fed17f"},"cell_type":"code","source":"#Cal non-repayment rate for each occupation type\nnon_repayment_rate_w_occupation = data_to_use.groupby(\"OCCUPATION_TYPE\", as_index=False).mean()\nnon_repayment_rate_w_occupation[\"TARGET\"] *= 100\n#non_repayment_rate_w_occupation[\"OCCUPATION_TYPE_LABEL\"] = [i for i in range(0, len(non_repayment_rate_w_occupation))]\nprint(non_repayment_rate_w_occupation)\n\n#Bar plot\nplt.figure(figsize=(30,12))\nplt.bar(non_repayment_rate_w_occupation.index, non_repayment_rate_w_occupation[\"TARGET\"], color=\"Blue\")\nplt.xlabel(\"OCCUPATION_TYPE\")\nplt.ylabel(\"Non-payment rate[%]\")\nplt.title(\"Non-payment rate\")\n\nfor x, y in zip(non_repayment_rate_w_occupation.index, non_repayment_rate_w_occupation[\"TARGET\"]):\n    plt.text(x, y, str(\"{:.2f}\").format(y), ha='center', va='bottom')\n\nplt.xticks(non_repayment_rate_w_occupation.index, non_repayment_rate_w_occupation[\"OCCUPATION_TYPE\"])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"061f2336484f4d26569cc34a88fb39ec1825272a"},"cell_type":"markdown","source":"####  Conclusion of  \"OCCUPATION_TYPE\" vs  \"Non-repayment rate\" analysis\n1. High non-repayment rate is observed in Labors. Espacilly low-skill labors marks highest rate 17.15%.   \n2. Non-repayment rate tends to high in the job that doesn't require high-skill, high-education like waiter, drivers, security staff etc...  Education history may also be a important feature. \n3. Accountant has Lowest non-repayment rate 4.53%. I suppose they are good at finacial planning & control because they are accountant.   \n4. As a result, we see siginificant non-repayment rate difference among occupation type so I conclude that ”OCCUPATION_TYPE” can be a important feature."},{"metadata":{"_uuid":"230335974cc7ae2b52f487f494b030d98867583f"},"cell_type":"markdown","source":"###  Conclusion of  Step2\n1. \"AMT_INCOME_TOTAL\" will be tested with random forest analysis in Step3 to find out if it's important feature or not  \n2. ”OCCUPATION_TYPE” can be a important feature  \n3. Generally thinking, income relate to occupation type. However non-repayment rate difference is observed well among occupation type rather than among income range.  Thus the features that describe customer's social role, resposibility and surrounding environment seems to be more important feature than income. "},{"metadata":{"trusted":true,"_uuid":"0cd4365a27a0f0ccbec0e6ea8dcabd0d353083c9"},"cell_type":"markdown","source":"# Step3.  Picking up other features that will help to predict custmer's repayment ability    \n\n**method**\n1. Drop the feature that have much missing data\n2. Encoding object tyep data\n3. Calculate importance with Random forest analysis"},{"metadata":{"_uuid":"2cb588bfb0677180ced2e519f113c5178405373d"},"cell_type":"markdown","source":"1. Drop the feature that have much missing data  \nIf missing data rate is over 40%, then drop those features from this analysis.   (I will run same analysis without dropping missing data later and compare the result. )"},{"metadata":{"trusted":true,"_uuid":"5351d3e2765b85a898bf2fac496c692e06dbd7da"},"cell_type":"code","source":"#1.Drop high missing value data\n#Set drop criteria as 40%. If missing value rate is over 40%, then drop those features.\nREDUCTION_CRITERIA_FOR_MISSING_DATA = 40 #[%] \ntotal = len(data_application_train)\ncol_name = data_application_train.columns\nmissing_data_rate = [100*data_application_train[col_name[i]].isnull().sum() / total for i in range(0, len(col_name))]\n\nfor i in range(0, len(col_name)):\n    if REDUCTION_CRITERIA_FOR_MISSING_DATA < missing_data_rate[i]:\n        del data_application_train[col_name[i]]\n\nmsno.bar(data_application_train)\nprint(\"Num of Featurs: Before reduction {} => After Reduction {}\".format(len(col_name), len(data_application_train.columns.values)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3e20591da2494a2e231b383991d804772b53e262"},"cell_type":"markdown","source":"Encoding object tyep data  \nAll object type data are categorical variable (nominal scale) then replace those value, include \"nan\" , to any number to run Random forest analysis in next."},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"c24166223a8daf28fb057163a37e000ebd916d50"},"cell_type":"code","source":"object_data_application_train = data_application_train.select_dtypes(['object'])\nobject_data_application_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"35e165f263e0f3f34e5db38794bb6e1309da559e"},"cell_type":"code","source":"#Replace object type data to number\nobject_col_name = object_data_application_train.columns.values\narray_object_to_int = np.array([object_data_application_train[object_col_name[i]].unique() for i in range(0,len(object_col_name))])\n\nfor i in range(0,len(object_col_name)):\n    labels, uniques = pd.factorize(data_application_train[object_col_name[i]])\n    data_application_train[object_col_name[i]] = labels   \n\n#Repalce \"NAN\" to -1    \ndata_application_train = data_application_train.replace(np.nan, -1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"617240086985c8ae5a922faa289db46dda739e53"},"cell_type":"markdown","source":"Run Random forest analysis"},{"metadata":{"trusted":true,"_uuid":"dfac3e21ad410e61ff61679a33c275ef257028e8"},"cell_type":"code","source":"from sklearn.ensemble import RandomForestClassifier\nmodel = RandomForestClassifier()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"39db4c39e277cb37095efb07edf6a2deae2cc794"},"cell_type":"code","source":"#Drop SK_ID_CURR\ndel data_application_train[\"SK_ID_CURR\"]\n#Drop OCCUPATION_TYPE (Already done in Step2)\ndel data_application_train[\"OCCUPATION_TYPE\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c1e4ae1284fc70bcd01ea3f429b8d572f46939fc"},"cell_type":"code","source":"target = data_application_train['TARGET']\nfeature = data_application_train.iloc[:,1:len(data_application_train.columns)]\nmodel.fit(feature, target)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"119b9a426a715a77d39c93ad806b62e77047bc52"},"cell_type":"code","source":"rank = np.argsort(-model.feature_importances_)\nf, ax = plt.subplots(figsize=(11, 11)) \nsns.barplot(x=model.feature_importances_[rank], y=data_application_train.columns.values[rank], orient='h')\nax.set_xlabel(\"Importance\")\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f62ba64a3e0363beeb336e93d2038525bb445cdb"},"cell_type":"code","source":"#Pick up top10 important feature\nfor i in range(0, 10):\n    print(\"Rank{} => {} (Importance={:.3f})\".format(i+1, \n                                                    data_application_train.columns.values[rank[i]], \n                                                    model.feature_importances_[rank[i]]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7e6ff40b88a8772262b24bac63554cf6fe822340"},"cell_type":"markdown","source":"Run Random forest analysis without dropping missing values"},{"metadata":{"trusted":true,"_uuid":"51bdab922b0025c08e700226afee191143994769"},"cell_type":"code","source":"train_data =  pd.read_csv(\"../input/application_train.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b67b05153c69b0b6570c0d2ff74359bea0939e03"},"cell_type":"code","source":"#Replace object type data to number\nobject_train_data = train_data.select_dtypes(['object'])\nobject_col_name = object_train_data.columns.values\narray_object_to_int = np.array([object_train_data[object_col_name[i]].unique() for i in range(0,len(object_col_name))])\n\nfor i in range(0,len(object_col_name)):\n    labels, uniques = pd.factorize(train_data[object_col_name[i]])\n    train_data[object_col_name[i]] = labels   \n\n#Repalce \"NAN\" to -1    \ntrain_data = train_data.replace(np.nan, -1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4af1ba1f2f4dd622e93cfe98e5a9c4884eaf2a3f"},"cell_type":"code","source":"#Drop SK_ID_CURR\ndel train_data[\"SK_ID_CURR\"]\n#Drop OCCUPATION_TYPE (Already done in Step2)\ndel train_data[\"OCCUPATION_TYPE\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b5a465b443c60d75fafe6f772f4f2c74afcee371"},"cell_type":"code","source":"target = train_data['TARGET']\nfeature = train_data.iloc[:,1:len(train_data.columns)]\nmodel.fit(feature, target)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5ef1087fd332eae923bf4f272292d78ca26b5852"},"cell_type":"code","source":"rank = np.argsort(-model.feature_importances_)\nf, ax = plt.subplots(figsize=(15, 15)) \nsns.barplot(x=model.feature_importances_[rank], y=train_data.columns.values[rank], orient='h')\nax.set_xlabel(\"Importance\")\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7b7d67f2eb6fb77033814fdf228fe1a50c7f4ff8"},"cell_type":"code","source":"for i in range(0, 10):\n    print(\"Rank{} => {} (Importance={:.3f})\".format(i+1, train_data.columns.values[rank[i]], model.feature_importances_[rank[i]]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"742e10a2fe97afb5698a71e5a39c201feae1609c"},"cell_type":"markdown","source":"Comparing Random forest analysis result w/ dropping missing values case and w/o dropping missing value case\n\n| Ranking | w/o missing value | w/ missing value |\n|:-----------:|:-----------:|:------------:|\n| 1 | **ORGANIZATION_TYPE (Importance=0.077)**  | **EXT_SOURCE_1 (Importance=0.058)**  |\n| 2 | EXT_SOURCE_2 (Importance=0.058)  | EXT_SOURCE_2 (Importance=0.044) |\n| 3 | REGION_POPULATION_RELATIVE (Importance=0.058)  | DAYS_EMPLOYED (Importance=0.039)  |\n| 4 | DAYS_REGISTRATION (Importance=0.056)  | DAYS_REGISTRATION (Importance=0.039) |\n| 5 | DAYS_EMPLOYED (Importance=0.055) | REGION_POPULATION_RELATIVE (Importance=0.039) |\n| 6 | AMT_CREDIT (Importance=0.051)  | DAYS_BIRTH (Importance=0.035) |\n| 7 | DEF_60_CNT_SOCIAL_CIRCLE (Importance=0.050) | AMT_CREDIT (Importance=0.034) |\n| 8 | DAYS_BIRTH (Importance=0.049) | DEF_60_CNT_SOCIAL_CIRCLE (Importance=0.033) |\n| 9 | AMT_INCOME_TOTAL (Importance=0.046) | AMT_INCOME_TOTAL (Importance=0.032) |\n| 10 | **CNT_CHILDREN (Importance=0.042)** | **AMT_ANNUITY (Importance=0.029)** |"},{"metadata":{"_uuid":"3fec1f0fb245a6113011f43dcbf597edbbb3f9e4"},"cell_type":"markdown","source":"Importance is changed however 8 common features are in the both top10 rank.  \"ORGANIZATION_TYPE\" and \"EXT_SOURCE_1\" are switched between w/ dropping and w/o dropping.  Also \"CNT_CHILDREN\" and \"AMT_AN NUITY\" are switched.  I dropped \"EXT_SOURCE_1\" and  \"AMT_AN NUITY\" in first Random forest analysis becasue they have over 40% missing value. However, from 2nd Random forest analysis, it turns out that actually they are important features even they have missing values.   \n\n### Conclusion of Step3\nAs a result, 12 festures below are picked up.\n\n| No| Selected feature | \n|:-----------:|:-----------:|\n| 1 | ORGANIZATION_TYPE  |\n| 2 | EXT_SOURCE_1  |\n| 3 | EXT_SOURCE_2 |\n| 4 | REGION_POPULATION_RELATIVE |\n| 5 | DAYS_REGISTRATION  |\n| 6 | DAYS_EMPLOYED |\n| 7 | AMT_CREDIT |\n| 8 | DEF_60_CNT_SOCIAL_CIRCLE  |\n| 9 | DAYS_BIRTH  |\n| 10 | AMT_INCOME_TOTAL |\n| 11 | CNT_CHILDREN |\n| 12 |AMT_ANNUITY |"},{"metadata":{"_uuid":"496f593cab8a7e645b9eb8034d873b567fab078c"},"cell_type":"markdown","source":"# Plot non-repayment rate for all selected features  \nAs a result, some plots show strong relationship between selected feature and non-repayment rate. For example \"Organization type\". We can see significant non-repayment differece among parameters as same as \"occupation type\". So it seems customer's surronding environment is a key to pridect the repayment ability.   \nAnd also \"EXT_SOURCE_1 & 2\". I need to study about these features more. However these show the strong trend which lower value of \"EXT_SOURCE_1 & 2\" shows higher non-repayment rate and higher value of \"EXT_SOURCE_1 & 2\" shows lower non-repayment rate. "},{"metadata":{"trusted":true,"_uuid":"1a6701aed0a279cec54c826df78574e6bbb21ae8"},"cell_type":"code","source":"data =  pd.read_csv(\"../input/application_train.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5f949e3cc655e9ee575256a2ca9d9caca05165ac"},"cell_type":"code","source":"def plot_non_repayment_rate(data_source, col_name, flag_rename_x_label=False):\n    \"\"\"\n    Plot non-repayment rate. (Xaxis=Col_name, Yaxis=non-repayment rate)\n    \n    parameter\n    --------------\n    data_source : Pandas dataframe\n    col_name : string \n    flag_rename_x_label : bool\n    \n    return\n    --------------\n    None\n    \"\"\"\n    \n    #Cal non-repayment rate\n    non_repayment_rate = data_source.groupby(col_name, as_index=False).mean()\n    non_repayment_rate[\"TARGET\"] *= 100\n    #print(non_repayment_rate)\n    \n    #Bar plot\n    plt.figure(figsize=(25,10))\n    plt.bar(non_repayment_rate.index, non_repayment_rate[\"TARGET\"], color=\"Blue\")\n    plt.xlabel(col_name, fontsize=18)\n    plt.ylabel(\"Non-payment rate[%]\", fontsize=18)\n    plt.title(\"Non-payment rate\")\n    plt.tight_layout()\n    \n    for x, y in zip(non_repayment_rate.index, non_repayment_rate[\"TARGET\"]):\n        plt.text(x, y, str(\"{:.2f}\").format(y), ha='center', va='bottom')\n        if flag_rename_x_label == True:\n            plt.xticks(non_repayment_rate.index, non_repayment_rate[col_name])\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"868d5dd9adc8a379ba9dd7cd60fc253330f4ab62"},"cell_type":"code","source":"def gen_range(value_start, value_end, value_step):\n    \"\"\"\n    Extend range() so it can handle float type data\n    ----------------\n    Parameter\n        value_start:float\n        value_end:float\n        value_step:float\n    ----------------\n    ----------------\n    Return\n    　　value:float\n    ----------------\n    \"\"\"\n    value = value_start\n    while value+value_step < value_end:\n     yield value\n     value += value_step","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c1d75f6340fe2958fcb07dcb22ae734eac659484"},"cell_type":"code","source":"def binning_data2(data_source, col_name, binned_col_name, num_of_bin=10):\n    \"\"\"\n    parameter\n    --------------\n    data_source : Pandas dataframe\n    col_name : string \n    binned_col_name : string \n    num_of_bin : int\n    \n    return\n    --------------\n    data_source : pandas dataframe\n    \n    \"\"\"\n    #Use interquartile value to decide bin_min & max value\n    data_info = data_source.describe()\n    bin_min = data_info.loc[\"25%\", col_name]\n    bin_max = data_info.loc[\"75%\", col_name]\n    IQR = bin_max - bin_min\n    bin_min = bin_min-IQR*1.5 if bin_min > IQR*1.5 else 0\n    bin_max += IQR*1.5\n    \n    bin_width = (bin_max - bin_min) / num_of_bin\n    bins = [value for value in gen_range(bin_min, bin_max, bin_width )]\n    labels = [i for i in range(0, len(bins)-1)]\n    binned_label = pd.cut(data_source[col_name], bins=bins, labels=labels)\n    data_source[binned_col_name] = binned_label\n\n    for i in range(0, len(bins)-1):\n        print(\"range label={} : range={:.5f}~{:.5f}\".format(i, bins[i], bins[i+1]) )\n    \n    return data_source","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"36fad138f74c36dc38f46de02a2bbfcd104774f8"},"cell_type":"code","source":"#ORGANIZATION_TYPE\nplot_non_repayment_rate(data, 'ORGANIZATION_TYPE', False)\ntmp = data[\"ORGANIZATION_TYPE\"].unique()\nfor i in range(0, len(tmp)):\n    print(\"{}:{}\".format(i, tmp[i]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9c2117c056280ffb35488d93789da8f48672e8c1"},"cell_type":"code","source":"#EXT_SOURCE_1\nplot_non_repayment_rate(binning_data2(data, 'EXT_SOURCE_1', 'BIN_EXT_SOURCE_1', 8), 'BIN_EXT_SOURCE_1')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"281fd4e1c62a06acdae2aa547be539d1d2f11262"},"cell_type":"code","source":"#EXT_SOURCE_2\nplot_non_repayment_rate(binning_data2(data, 'EXT_SOURCE_2', 'BIN_EXT_SOURCE_2', 8), 'BIN_EXT_SOURCE_2')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ce242320283e93241c9f245ed8be25531c407e25"},"cell_type":"code","source":"#REGION_POPULATION_RELATIVE\nplot_non_repayment_rate(binning_data2(data, 'REGION_POPULATION_RELATIVE', 'BIN_REGION_POPULATION_RELATIVE', 8), 'BIN_REGION_POPULATION_RELATIVE')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f2b30754db7b591f01a6cec9e74cdbc4146bf684"},"cell_type":"code","source":"#AMT_CREDIT\nplot_non_repayment_rate(binning_data(data, 'AMT_CREDIT', 'BIN_AMT_CREDIT', 8), 'BIN_AMT_CREDIT')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4722e9df5ae2a4ec23073f21f2727fd18777c953"},"cell_type":"code","source":"#DEF_60_CNT_SOCIAL_CIRCLE\nplot_non_repayment_rate(data, 'DEF_60_CNT_SOCIAL_CIRCLE', True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4a05e29efb1d68059334097ca834bf1f3dbf7199"},"cell_type":"code","source":"#CNT_CHILDREN\nplot_non_repayment_rate(data, 'CNT_CHILDREN', True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4ba53369b20937e79b9de92a72f12a4e88c4087b"},"cell_type":"code","source":"#AMT_ANNUITY\nplot_non_repayment_rate(binning_data2(data, 'AMT_ANNUITY', 'BIN_AMT_ANNUITY', 8), 'BIN_AMT_ANNUITY')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"917307ce18e8196afab18867ec7b2f427aaabd60"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}