{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Script para generar la solución del Tercer Benchmark de la Competencia\n\n## Si no presentaste aún tu primera solución, tenes la oportunidad de hacerlo en pocos Clicks!\n\n**Hola! **  \n  \nEste Script es un Ejemplo de Procesamiento de los Datos, Modelado y Generación de una Solución.\n\nAgregamos una pequeña explicación de lo que se hace en cada paso para ayudar a los que están comenzando ahora\n"},{"metadata":{},"cell_type":"markdown","source":"### Importamos las librerías que vamos a utilizar"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import pandas as pd\nfrom lightgbm import LGBMClassifier\nfrom sklearn.metrics import roc_auc_score\nfrom sklearn.model_selection import KFold\nimport re","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Lectura de las Bases\n\nObservamos los datos que tenemos disponibles en https://www.kaggle.com/c/interbank20/data\n\n\n#### En este caso, vamos a descartar la base de censo, dado que empeora lo resultados. Este tipo de comportamiento es inusual pero no inesperado. Merecería un estudio detallado para saber con más precisión por qué esto ocurre, pero a priori, parecería ser que las variables del censo son relevantes (ergo, el modelo las usa) pero, por su antigüedad, son menos relevantes para el período de test (probablemente por no estar ya vigentes). Por lo tanto, estaríamos frente a una perturbación de la distribución del input, lo que perjudica al modelo que depende de las variables alteradas."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nrcc_train = pd.read_csv(\"/kaggle/input/interbank20/rcc_train.csv\")\nse_train = pd.read_csv(\"/kaggle/input/interbank20/se_train.csv\", index_col=\"key_value\")\nsunat_train = pd.read_csv(\"/kaggle/input/interbank20/sunat_train.csv\")\ny_train = pd.read_csv(\"/kaggle/input/interbank20/y_train.csv\", index_col=\"key_value\").target\n\nrcc_test= pd.read_csv(\"/kaggle/input/interbank20/rcc_test.csv\")\nse_test= pd.read_csv(\"/kaggle/input/interbank20/se_test.csv\", index_col=\"key_value\")\nsunat_test= pd.read_csv(\"/kaggle/input/interbank20/sunat_test.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Vamos a trabajar ahora con la base de **RCC**:\n\nEl principal problema que tiene esta base es su estructura temporal, que consiste de múltiples series de tiempo, una por cada producto en cada banco. \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"rcc_train[(rcc_train.key_value == 4) & (rcc_train.cod_instit_financiera == 33)].sort_values(\"codmes\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"rcc_train[(rcc_train.key_value == 4) & (rcc_train.cod_instit_financiera == 61)].sort_values(\"codmes\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nPrimero discretizamos los días de atraso para poder manipularla mejor."},{"metadata":{"trusted":true},"cell_type":"code","source":"#rcc_train = pd.read_csv(\"/kaggle/input/interbank20/rcc_train.csv\")\n#rcc_train[['codmes', 'key_value', 'condicion']].to_csv('condicion_ok.csv', encoding='UTF-8', sep='|')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nbins = [-1, 0, 10, 20, 30, 60, 90, 180, 360, 720, float(\"inf\")]\n#bins = [-1, 0, 10, float(\"inf\")]\nrcc_train[\"condicion\"] = pd.cut(rcc_train.condicion, bins)\nrcc_train[\"condicion\"] = rcc_train[\"condicion\"].cat.codes\nrcc_test[\"condicion\"] = pd.cut(rcc_test.condicion, bins)\nrcc_test[\"condicion\"] = rcc_test[\"condicion\"].cat.codes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#·pip install jenkspy","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# import jenkspy\n\n# breaks = jenkspy.jenks_breaks(rcc_train.condicion, nb_class=6)\n# breaks","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# import numpy as np","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### ¿Cómo podemos procesar rcc para extraer información útil?"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ndef makeCt(df, c, aggfunc=sum):\n    try:\n        ct = pd.crosstab(df.key_value, df[c].fillna(\"N/A\"), values=df.saldo, aggfunc=aggfunc)\n    except:\n        ct = pd.crosstab(df.key_value, df[c], values=df.saldo, aggfunc=aggfunc)\n    ct.columns = [f\"{c}_{aggfunc.__name__}_{v}\" for v in ct.columns]\n    return ct","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from scipy import stats\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntrain = []\ntest = []\naggfuncs = [len, sum, min, max, np.median, np.std] # esto corre ok  pd.Series.mode, pd.Series.nunique, stats.median_absolute_deviation,np.mean, \nfor c in rcc_train.drop([\"codmes\", \"key_value\", \"saldo\"], axis=1):\n    print(\"haciendo\", c)\n    train.extend([makeCt(rcc_train, c, aggfunc) for aggfunc in aggfuncs])\n    test.extend([makeCt(rcc_test, c, aggfunc) for aggfunc in aggfuncs])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import gc\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#%%time\n#pd.crosstab(rcc_train.key_value, rcc_train.condicion.fillna(\"N/A\"), values=rcc_train.saldo, aggfunc=\"mean\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#%%time\n#pd.crosstab(rcc_train.key_value, rcc_train.condicion.fillna(\"N/A\"), values=rcc_train.saldo, aggfunc=len)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\ndel rcc_train, rcc_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.concat(train, axis=1)\ntest = pd.concat(test, axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.dtypes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.dtypes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = train.astype(np.float32)\ntest = test.astype(np.float32)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.fillna(-1, inplace=True)\ntest.fillna(-1, inplace=True)\n# train.replace([np.inf, -np.inf, 'N/A'], -1, inplace=True)\n# test.replace([np.inf, -np.inf, 'N/A'], -1, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# for col in train.select_dtypes('object').columns:\n#     train[col] = pd.to_numeric(train[col]).astype('float64')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Sunat\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"sunat_train","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"En este caso, no es una serie de tiempo pero tenemos multiples filas por cada persona, dadas por la multiplicidad de rubros anotados"},{"metadata":{"trusted":true},"cell_type":"code","source":"#pd.crosstab(sunat_train.key_value, sunat_train.ciiu)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Incorporamos la Información adicional existente en las tablas socio económicas y del censo. Es un simple join porque ambas tienen key_value únicos\n#### Por el momento no incorporamos la información tributaria porque requiere un tratamiento más complejo que queda para futuras revisiones"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntrain = train.join(pd.crosstab(sunat_train.key_value, sunat_train.ciiu)).join(se_train)\ntest = test.join(pd.crosstab(sunat_test.key_value, sunat_test.ciiu)).join(se_test)\n\ndel sunat_train, se_train, sunat_test, se_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Por la naturaleza de las variables creadas, nos aseguramos que solo se utilicen variables existentes en ambos conjuntos de datos (train y test)"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nkeep_cols = list(set(train.columns).intersection(set(test.columns)))\ntrain = train[keep_cols]\ntest = test[keep_cols]\nlen(set(train.columns) - set(test.columns)) , len(set(test.columns) - set(train.columns))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntrain.columns = [str(c) for c in train.columns]\ntrain = train.rename(columns = lambda x:re.sub('[^A-Za-z0-9_-]+', '', x))\n\ntest.columns = [str(c) for c in test.columns]\ntest = test.rename(columns = lambda x:re.sub('[^A-Za-z0-9_-]+', '', x))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"folds = [train.index[t] for t, v in KFold(5, shuffle=True, random_state=42).split(train)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.fillna(-1, inplace=True)\ntest.fillna(-1, inplace=True)\n# train.replace([np.nan, np.inf, -np.inf], -1, inplace=True)\n# test.replace([np.nan, np.inf, -np.inf], -1, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# for col in train.select_dtypes('object').columns:\n#     train[col] = train[col].astype('float64', )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"canarios"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy.random as random\n\nrandom.seed(42)\n\n\ntrain['canario_random'] = random.rand(train.shape[0])\ntrain['canario_normal'] = random.normal(0, 3, train.shape[0])\ntrain['canario_binomial_50'] = np.random.binomial(10, 0.5, train.shape[0])\ntrain['canario_binomial_5'] = np.random.binomial(10, 0.05, train.shape[0])\ntrain['canario_skewed_50'] = skewnorm.rvs(50, loc=90, scale=50, size=train.shape[0])\ntrain['canario_skewed_menos50'] = skewnorm.rvs(-50, loc=90, scale=50, size=train.shape[0])\n\ntest['canario_random'] = random.rand(test.shape[0])\ntest['canario_normal'] = random.normal(0, 3, test.shape[0])\ntest['canario_binomial_50'] = np.random.binomial(10, 0.5, test.shape[0])\ntest['canario_binomial_5'] = np.random.binomial(10, 0.05, test.shape[0])\ntest['canario_skewed_50'] = skewnorm.rvs(50, loc=90, scale=50, size=test.shape[0])\ntest['canario_skewed_menos50'] = skewnorm.rvs(-50, loc=90, scale=50, size=test.shape[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learner.best_score_","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\nfolds = [train.index[t] for t, v in KFold(5, shuffle=True, random_state=42).split(train)]\ntest_probs = []\ntrain_probs = []\nfi = []\nfor i, idx in enumerate(folds):\n    print(\"*\"*10, i, \"*\"*10)\n    Xt = train.loc[idx]\n    yt = y_train.loc[Xt.index]\n\n    Xv = train.drop(Xt.index)\n    yv = y_train.loc[Xv.index]\n    \n    learner = LGBMClassifier(n_estimators=1000, random_state=42, min_child_samples = 500, feature_fraction =0.9, learning_rate =  0.03)\n    learner.fit(Xt, yt,  early_stopping_rounds=10, eval_metric=\"auc\",\n                eval_set=[(Xt, yt), (Xv, yv)], verbose=50)\n    test_probs.append(pd.Series(learner.predict_proba(test)[:, -1], index=test.index, name=\"fold_\" + str(i)))\n    train_probs.append(pd.Series(learner.predict_proba(Xv)[:, -1], index=Xv.index, name=\"probs\"))\n    fi.append(pd.Series(learner.feature_importances_ / learner.feature_importances_.sum(), index=Xt.columns))\n          \n#test_probs = pd.concat(test_probs, axis=1).mean(axis=1)\ntest_probs = pd.concat(test_probs, axis=1).max(axis=1)\ntrain_probs = pd.concat(train_probs)\nfi = pd.concat(fi, axis=1).mean(axis=1)\n\nprint(\"*\" * 21)\nprint(\"roc auc estimado: \", roc_auc_score(y_train, train_probs.loc[y_train.index]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_probs = pd.concat(train_probs)\nfi = pd.concat(fi, axis=1).mean(axis=1)\n\nprint(\"*\" * 21)\nprint(\"roc auc estimado: \", roc_auc_score(y_train, train_probs.loc[y_train.index]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fi.reset_index().to_excel('features_3.xlsx')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dropear estas columnas\n'canario_normal', 'canario_skewed_50'\nCOD_CLASIFICACION_DEUDOR_len_0\ncondicion_median_0\ntipo_credito_min_11\nlgr_vot\nsexo\nCOD_CLASIFICACION_DEUDOR_sum_0\ntipo_credito_std_10\ncondicion_std_0\nCOD_CLASIFICACION_DEUDOR_max_0\nflg_sin_email\ntipo_credito_median_10\nCOD_CLASIFICACION_DEUDOR_std_0\nrgn\ncondicion_max_0\ntipo_credito_sum_10\ncod_ocu\ncondicion_sum_0\nCOD_CLASIFICACION_DEUDOR_std_5\ncondicion_len_0\ncondicion_max_1\ncondicion_std_1\ntipo_credito_min_10\ncondicion_min_1\nCOD_CLASIFICACION_DEUDOR_median_1\nRIESGO_DIRECTO_min_-1\ncondicion_min_2\nRIESGO_DIRECTO_min_3\nctd_veh\nCOD_CLASIFICACION_DEUDOR_sum_1\nCOD_CLASIFICACION_DEUDOR_len_1\ncondicion_max_2\nCOD_CLASIFICACION_DEUDOR_min_1\nCOD_CLASIFICACION_DEUDOR_sum_5\nCOD_CLASIFICACION_DEUDOR_len_4\nCOD_CLASIFICACION_DEUDOR_std_1\ncondicion_sum_4\ncondicion_len_1\ncondicion_std_3\ncondicion_std_2\nCOD_CLASIFICACION_DEUDOR_min_4\n0\ncondicion_median_1\nCOD_CLASIFICACION_DEUDOR_len_2\ncondicion_std_4\ntipo_credito_len_9\n285\nCOD_CLASIFICACION_DEUDOR_min_0\ncondicion_max_3\nRIESGO_DIRECTO_max_2\ncondicion_len_9\ncondicion_min_3\nCOD_CLASIFICACION_DEUDOR_max_1\ncondicion_sum_1\nCOD_CLASIFICACION_DEUDOR_min_2\ncondicion_len_7\nCOD_CLASIFICACION_DEUDOR_sum_3\ncanario_binomial_50\nRIESGO_DIRECTO_std_2\nCOD_CLASIFICACION_DEUDOR_sum_2\nRIESGO_DIRECTO_len_2\ncondicion_min_0\nRIESGO_DIRECTO_min_2\ncondicion_median_3\ncondicion_len_6\ntipo_credito_len_99\ntipo_credito_min_99\ntipo_credito_std_99\ncondicion_sum_9\ntipo_credito_median_9\nest_cvl\n259\nCOD_CLASIFICACION_DEUDOR_len_3\ncondicion_median_2\ncondicion_sum_5\ncondicion_len_2\ncondicion_min_4\nCOD_CLASIFICACION_DEUDOR_min_5\n244\nRIESGO_DIRECTO_sum_2\nCOD_CLASIFICACION_DEUDOR_min_3\nRIESGO_DIRECTO_median_2\ncondicion_min_9\ncondicion_len_8\ncondicion_std_5\ncondicion_len_4\ntipo_credito_max_9\ncondicion_sum_2\ncondicion_sum_3\ncondicion_median_4\n184\nCOD_CLASIFICACION_DEUDOR_max_3\ncondicion_len_3\ncanario_binomial_5\nCOD_CLASIFICACION_DEUDOR_median_3\n233\ncondicion_median_9\ntipo_credito_sum_99\ncondicion_max_4\n260\nctd_hijos\ncondicion_sum_8\nCOD_CLASIFICACION_DEUDOR_max_2\n215\ncondicion_min_8\ntipo_credito_std_9\nCOD_CLASIFICACION_DEUDOR_std_3\ncondicion_std_9\n232\n173\nCOD_CLASIFICACION_DEUDOR_max_4\nCOD_CLASIFICACION_DEUDOR_std_2\ntipo_credito_len_13\ncondicion_min_7\ncondicion_max_5\ntipo_credito_min_9\ntipo_credito_max_99\ntipo_credito_min_13\n171\ncondicion_max_9\ntipo_credito_sum_9\nCOD_CLASIFICACION_DEUDOR_std_4\ntipo_credito_median_99\nCOD_CLASIFICACION_DEUDOR_median_2\ncondicion_min_6\ncondicion_std_6\ncondicion_min_5\n178\ntipo_credito_std_13\nRIESGO_DIRECTO_std_3\ncondicion_max_6\n182\ntipo_credito_max_13\nCOD_CLASIFICACION_DEUDOR_max_5\n254\n235\ncondicion_sum_6\nRIESGO_DIRECTO_len_3\ncondicion_len_5\ncondicion_median_8\ncondicion_median_6\n27\n240\ncondicion_std_7\n288\nCOD_CLASIFICACION_DEUDOR_median_5\n2\ncondicion_max_8\n157\nCOD_CLASIFICACION_DEUDOR_median_4\ntipo_credito_median_13\nCOD_CLASIFICACION_DEUDOR_sum_4\ntipo_credito_sum_13\ncondicion_median_5\n43\ncondicion_std_8\ncondicion_median_7\n187\n236\n170\ncondicion_sum_7\n90\nRIESGO_DIRECTO_max_3\n150\n174\nRIESGO_DIRECTO_median_3\n160\nRIESGO_DIRECTO_sum_3\n185\n167\n253\n216\n177\ncondicion_max_7\n137\n245\n163\n175\n202\n162\n238\n283\n3\n198\n156\n4\n5\n6\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20\n21\n22\n23\n24\nRIESGO_DIRECTO_sum_5\n26\n28\n29\n30\n31\n32\n33\n34\n35\n36\n37\n38\n39\n40\n41\n42\n44\n45\n46\n47\n48\n49\n50\n51\n52\n53\n54\n55\n56\n57\n58\n59\n60\n61\n62\n63\n64\n65\n66\n67\n68\n69\n70\n71\n72\n73\n74\n75\n76\n77\n78\n79\n80\n81\n82\n83\n84\n85\n86\n87\n88\n89\n91\n93\n94\n95\n96\n98\n100\n101\n102\n104\n103\n106\n105\n107\n108\n110\n111\n109\n113\n114\n115\n116\n117\n118\n119\n120\n121\n122\n123\n124\n125\n127\n128\n129\n130\n131\n133\n134\n135\n136\n138\n139\n140\n141\n142\n143\n144\n145\n146\n147\n148\n149\n151\n152\n153\n154\n155\n158\n159\n161\n164\n165\n166\n168\n169\n172\n176\n179\n180\n181\n183\n186\n188\n189\n190\n191\n192\n193\n194\n195\n196\n197\n199\n200\n201\n204\n205\n206\n207\n208\n209\n210\n211\n212\n213\n217\n214\n218\n219\n220\n221\n222\n223\n224\n225\n226\n227\n228\n229\n230\n231\n234\n239\n242\n241\n243\n246\n248\n249\n250\n251\n252\n255\n256\n257\n258\n261\n263\n264\n265\n266\n267\n268\n269\n270\n271\n272\n273\n274\n275\n276\n277\n278\n279\n280\n281\n282\n284\n286\n287\ntipo_credito_max_8\ntipo_credito_std_8\ntipo_credito_median_6\n112\ntipo_credito_std_6\ntipo_credito_sum_6\ntipo_credito_min_6\nRIESGO_DIRECTO_min_5\ntipo_credito_len_6\ntipo_credito_max_6\ntipo_credito_min_8\n237\n247\n262\nRIESGO_DIRECTO_len_5\ntipo_credito_median_8\ntipo_credito_len_8\nRIESGO_DIRECTO_median_5\ntipo_credito_sum_8\nRIESGO_DIRECTO_std_5\nRIESGO_DIRECTO_max_5\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"feature_imp = pd.DataFrame(sorted(zip(learner.feature_importances_,train.columns)), columns=['Value','Feature'])\nfeature_imp.to_excel('features_2.xlsx')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"fin de canarios"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import ParameterGrid\n\nparams = ParameterGrid({\"min_child_samples\": [250, 500, 1000],  \"feature_fraction\": [0.9], \"learning_rate\": [0.01, 0.003, 0.03]}) #\"boosting_type\": [\"gbdt\", \"goss\"], ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"best_score = 0\nbest_probs = []\nfor param in params:\n    test_probs = []\n    train_probs = []\n    p  = \"///\".join([f\"{k}={v}\" for k, v in param.items()])\n    print(\"*\"*10, p, \"*\"*10)\n    for i, idx in enumerate(folds):\n        Xt = train.loc[idx]\n        yt = y_train.loc[Xt.index]\n\n        Xv = train.drop(Xt.index)\n        yv = y_train.loc[Xv.index]\n\n        learner = LGBMClassifier(n_estimators=1000,  random_state=42, **param)\n        learner.fit(Xt, yt,  early_stopping_rounds=10, eval_metric=\"recall\",\n                    eval_set=[(Xt, yt), (Xv, yv)], verbose=False,\n                   callbacks=[learner.reset_parameter(learning_rate=learning_rate_005_decay_power_099)])\n        test_probs.append(pd.Series(learner.predict_proba(test)[:, -1], index=test.index, name=\"fold_\" + str(i)))\n        train_probs.append(pd.Series(learner.predict_proba(Xv)[:, -1], index=Xv.index, name=\"probs\"))\n\n    test_probs = pd.concat(test_probs, axis=1).mean(axis=1)\n    train_probs = pd.concat(train_probs)\n    score = roc_auc_score(y_train, train_probs.loc[y_train.index])\n    print(f\"roc auc estimado para {p}: {score}\")\n    if score > best_score:\n        print(\"*\"*10, f\"{p} es el nuevo mejor modelo\", \"*\"*10)\n        best_score = score\n        best_probs = test_probs","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"def learning_rate_005_decay_power_099(current_iter):\n    base_learning_rate = 0.05\n    lr = base_learning_rate  * np.power(.99, current_iter)\n    return lr if lr > 1e-3 else 1e-3\n\neste va en el train para aplicarlo en cada vuelta\ncallbacks=[lgb.reset_parameter(learning_rate=learning_rate_005_decay_power_099)]\n\nhttps://stackoverflow.com/questions/58197225/how-can-i-correctly-set-a-decaying-learning-rate-callback-passing-it-a-custom-fu"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.ensemble import AdaBoostClassifier","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"params_ = ParameterGrid({\"n_estimators\": [50, 150, 250, 500, 1000], \"learning_rate\": [0.03, 0.3, 0.003]})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"best_score = 0\nbest_probs = []\nfor param in params_:\n    test_probs = []\n    train_probs = []\n    p  = \"///\".join([f\"{k}={v}\" for k, v in param.items()])\n    print(\"*\"*10, p, \"*\"*10)\n    for i, idx in enumerate(folds):\n        Xt = train.loc[idx]\n        yt = y_train.loc[Xt.index]\n\n        Xv = train.drop(Xt.index)\n        yv = y_train.loc[Xv.index]\n\n        learner = AdaBoostClassifier(**param, random_state=42)\n        learner.fit(Xt, yt)\n        test_probs.append(pd.Series(learner.predict_proba(test)[:, -1], index=test.index, name=\"fold_\" + str(i)))\n        train_probs.append(pd.Series(learner.predict_proba(Xv)[:, -1], index=Xv.index, name=\"probs\"))\n\n    test_probs = pd.concat(test_probs, axis=1).mean(axis=1)\n    train_probs = pd.concat(train_probs)\n    score = roc_auc_score(y_train, train_probs.loc[y_train.index])\n    print(f\"roc auc estimado para {p}: {score}\")\n    if score > best_score:\n        print(\"*\"*10, f\"{p} es el nuevo mejor modelo\", \"*\"*10)\n        best_score = score\n        best_probs = test_probs","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# lista_cate = []\n# for col in train.columns:\n#     if train[col].nunique() < 50:\n#         train[col] = train[col].astype(str).astype('category')\n#         test[col] = test[col].astype(str).astype('category')\n#         lista_cate.append(col)\n        \n# train.columns = train.columns.astype(str)\n# test.columns = test.columns.astype(str) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# from sklearn.calibration import CalibratedClassifierCV\n# from catboost import CatBoostClassifier","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# from skopt import BayesSearchCV\n# from skopt.space import Real, Categorical, Integer\n# from skopt.plots import plot_objective, plot_histogram","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# from scipy.stats import randint as sp_randint\n# from scipy.stats import uniform as sp_uniform\n# from sklearn.model_selection import cross_val_score","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# estimator_ = CatBoostClassifier(random_seed=42,\n#                                  task_type='GPU',\n#                                 eval_metric = 'Recall',\n#                                 loss_function = 'Logloss',\n#                                 'auto_class_weights' : 'SqrtBalanced'\n#                                 'score_function': 'L2'\n#                                 **params_ \n#                                  )\n\n\n# cb_search = {\n#         'learning_rate' : [0.001, 0.01, 0.03],\n#         'min_child_samples' : Integer(1, 500),\n#         #'num_leaves' :  Integer(32, 500),\n#         'max_depth' : Integer(5, 15)\n#     }\n\n# optimizer = BayesSearchCV( estimator_,\n#                           cb_search,\n#                           n_iter = 50,\n#                           random_state = 42,\n#                           verbose = 2,\n#                           refit = True,\n#                           cv = 5,\n#                           scoring = 'roc_auc'\n#                           )\n\n            \n            \n\n# #print(optimizer.total_iterations)\n# optimizer.fit(X_train, y_train)\n\n# #print(\"Final result:\", optimizer.max)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# print(\"val. score: %s\" % optimizer.best_score_)\n# print(\"test score: %s\" % optimizer.score(X_test, y_test))\n# print(\"best params: %s\" % str(optimizer.best_params_))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# series = pd.value_counts(train.ctd_hijos)\n# mask = (series/series.sum() * 100).gt(1)\n# print(((series/series.sum() * (series/series.sum() * 100).gt(1) * 100)).sum())\n# To replace df['column'] use np.where I.e \n#df['column'] = np.where(df['column'].isin(series[mask].index),'Other',df['column'])\n#https://stackoverflow.com/questions/47418299/python-combining-low-frequency-factors-category-counts","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Guardado de las predicciones modelo para hacer la presentación\n\nFinalmente creamos el archivo CSV que podemos subir como nuestra Solución a la competencia"},{"metadata":{"trusted":true},"cell_type":"code","source":"best_probs.name = \"target\"\nbest_probs.to_csv(\"benchmark3.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}