{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":50160,"databundleVersionId":7921029,"sourceType":"competition"},{"sourceId":7487967,"sourceType":"datasetVersion","datasetId":3933894}],"dockerImageVersionId":30648,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"https://www.kaggle.com/code/jirkaborovec/credit-risk-eda-xgboost-depth-0-gpu","metadata":{}},{"cell_type":"code","source":"import warnings\n# warnings.simplefilter(\"ignore\")\nwarnings.simplefilter(\"ignore\", UserWarning)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-03-17T17:45:39.822734Z","iopub.execute_input":"2024-03-17T17:45:39.823315Z","iopub.status.idle":"2024-03-17T17:45:39.834683Z","shell.execute_reply.started":"2024-03-17T17:45:39.823285Z","shell.execute_reply":"2024-03-17T17:45:39.833722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os, glob\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom pathlib import Path\n\nPATH_DATASET = Path(\"/kaggle/input/home-credit-credit-risk-model-stability\")\nPATH_PARQUETS = PATH_DATASET / \"parquet_files\"\nPARQUETS_TRAIN = PATH_PARQUETS / \"train\"\nPARQUETS_TEST = PATH_PARQUETS / \"test\"\npd.set_option('display.max_columns', None)\npd.set_option('display.max_rows', None)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-03-17T17:45:39.836304Z","iopub.execute_input":"2024-03-17T17:45:39.836662Z","iopub.status.idle":"2024-03-17T17:45:41.913269Z","shell.execute_reply.started":"2024-03-17T17:45:39.836622Z","shell.execute_reply":"2024-03-17T17:45:41.912453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Explore the training data\n\n**Borrowed from the competition describtion:**\n\nTable Description\nThis dataset contains a large number of tables as a result of utilizing diverse data sources and the varying levels of data aggregation used while preparing the dataset. Note: All files listed below are found in both .csv and .parquet formats.\n\n### Depth values\n\n- **depth=0** - These are static features directly tied to a specific case_id.\n- **depth=1** - Each case_id has an associated historical record, indexed by num_group1.\n- **depth=2** - Each case_id has an associated historical record, indexed by both num_group1 and num_group2.\n\nYou can read more about Credit bureau (CB) here https://en.wikipedia.org/wiki/Credit_bureau.","metadata":{}},{"cell_type":"code","source":"!cat /kaggle/input/home-credit-credit-risk-model-stability/feature_definitions.csv","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T17:45:41.914477Z","iopub.execute_input":"2024-03-17T17:45:41.914926Z","iopub.status.idle":"2024-03-17T17:45:42.887502Z","shell.execute_reply.started":"2024-03-17T17:45:41.914899Z","shell.execute_reply":"2024-03-17T17:45:42.886233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Various predictors were transformed, therefore we have the following notation for similar groups of transformations\n\n- **P** - Transform DPD (Days past due)\n- **M** - Masking categories\n- **A** - Transform amount\n- **D** - Transform date\n- **T** - Unspecified Transform\n- **L** - Unspecified Transform\n\nPlease note that transformations within a group are denoted by a capital letter at the end of the predictor name (e.g., maxdbddpdtollast6m_4187119P). We hope that this will simplify the manipulation with predictors.","metadata":{}},{"cell_type":"code","source":"def _short_array(arr):\n    if len(arr) <= 5:\n        return repr(arr)\n    return f\"[{', '.join(map(str, arr[:2]))}, ..., {', '.join(map(str, arr[-2:]))}]\"\n\n# taking inpiration with column names from https://www.kaggle.com/code/greysky/home-credit-baseline\ndef convert_dtypes(df):\n    cols = []\n    for col, dt in dict(df.dtypes).items():\n        if col.startswith(\"for\"):\n            df[col] = df[col].fillna(0).astype(\"int16\")\n        elif \"num\" in col or \"cnt\" in col:\n            df[col] = df[col].fillna(0).astype(\"int32\")\n        elif col.startswith(\"pct\"):\n            df[col] = df[col].astype(\"float16\")\n        elif col[-1] in (\"A\", \"P\"):\n            df[col] = df[col].astype(\"float32\")\n        elif col[-1] in (\"D\", ):\n            df[col] = pd.to_datetime(df[col])\n        elif col[-1] in (\"M\", \"L\"):\n            if col[-1] == \"L\" and dt.name.startswith(\"int\"):\n                df[col] = df[col].astype(\"int32\")\n            elif col[-1] == \"L\" and dt.name.startswith(\"float\"):\n                df[col] = df[col].astype(\"float32\")\n            else:\n                uq = list(df[col].unique())\n                print(f'{col} -> #{len(uq)} -> {_short_array(uq)}')\n                df[col] = df[col].astype(\"category\")\n        if col[-1] in (\"A\", \"P\", \"M\", \"L\"):\n            cols.append(col)\n    return cols","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:45:42.890777Z","iopub.execute_input":"2024-03-17T17:45:42.891171Z","iopub.status.idle":"2024-03-17T17:45:42.903242Z","shell.execute_reply.started":"2024-03-17T17:45:42.891133Z","shell.execute_reply":"2024-03-17T17:45:42.902227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Base tables\n\nBase tables store the basic information about the observation and case_id. This is a unique identification of every observation and you need to use it to join the other tables to base tables.\n\ntrain_base.csv","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_parquet(PARQUETS_TRAIN / \"train_base.parquet\")\nprint(f\"size: {len(df_train)}\")\ndisplay(df_train.head())","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:45:42.904587Z","iopub.execute_input":"2024-03-17T17:45:42.905018Z","iopub.status.idle":"2024-03-17T17:45:43.316395Z","shell.execute_reply.started":"2024-03-17T17:45:42.904988Z","shell.execute_reply":"2024-03-17T17:45:43.315457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train[\"date_decision\"] = pd.to_datetime(df_train[\"date_decision\"]).dt.date\n# delete redundat cols\ndel df_train[\"MONTH\"], df_train[\"WEEK_NUM\"]","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:45:43.317503Z","iopub.execute_input":"2024-03-17T17:45:43.317761Z","iopub.status.idle":"2024-03-17T17:45:44.084123Z","shell.execute_reply.started":"2024-03-17T17:45:43.317738Z","shell.execute_reply":"2024-03-17T17:45:44.083120Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def merge_parquets(df, name, prefix=\"train\", folder=PATH_PARQUETS):\n    df_ = pd.concat(\n        [pd.read_parquet(p) for p in glob.glob(str(folder / prefix / f\"{prefix}_{name}*.parquet\"))],\n    )\n    if \"num_group1\" in df_.columns:\n        del df_[\"num_group1\"]\n    df_.drop_duplicates(inplace=True)\n    print(f\"{name} size: {len(df_)} with features: {len(df_.columns)}\")\n    display(df_.head())\n    cols = convert_dtypes(df_) + [\"case_id\"]\n    df_ = df_[cols]\n    if len(df_) > len(df_[\"case_id\"].unique()):\n        df_ = df_.groupby(['case_id'], as_index=False).first()\n    df = df.merge(df_, how=\"left\", on=\"case_id\")\n    print(f\"fused size: {len(df)}\")\n    return df","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:45:44.085339Z","iopub.execute_input":"2024-03-17T17:45:44.085657Z","iopub.status.idle":"2024-03-17T17:45:44.093424Z","shell.execute_reply.started":"2024-03-17T17:45:44.085631Z","shell.execute_reply":"2024-03-17T17:45:44.092386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 0\n\n### static_0 | Properties: depth=0, internal data source\n\n- train_static_0_0.csv\n- train_static_0_1.csv\n\n### static_cb_0 | Properties: depth=0, external data source\n\n- train_static_cb_0.csv","metadata":{}},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"static_0\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:49:56.665814Z","iopub.execute_input":"2024-03-17T17:49:56.666668Z","iopub.status.idle":"2024-03-17T17:50:29.643399Z","shell.execute_reply.started":"2024-03-17T17:49:56.666623Z","shell.execute_reply":"2024-03-17T17:50:29.642446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"static_cb_0\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:50:29.645010Z","iopub.execute_input":"2024-03-17T17:50:29.645312Z","iopub.status.idle":"2024-03-17T17:50:37.441057Z","shell.execute_reply.started":"2024-03-17T17:50:29.645288Z","shell.execute_reply":"2024-03-17T17:50:37.440106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 1\n\n### applprev_1 | Properties: depth=1, internal data source\n\n- train_applprev_1_0.csv\n- train_applprev_1_1.csv\n\n### other_1 | Properties: depth=1, internal data source\n\n- train_other_1.csv\n\n### tax_registry_a_1 | Properties: depth=1, external data source, Tax registry provider A\n\n- train_tax_registry_a_1.csv\n\n### tax_registry_b_1 | Properties: depth=1, external data source, Tax registry provider B\n\n- train_tax_registry_b_1.csv\n\n### tax_registry_c_1 | Properties: depth=1, external data source, Tax registry provider C\n\n- train_tax_registry_c_1.csv\n\n### credit_bureau_a_1 | Properties: depth=1, external data source, Credit bureau provider A\n\n- train_credit_bureau_a_1_0.csv\n- train_credit_bureau_a_1_1.csv\n- train_credit_bureau_a_1_2.csv\n- train_credit_bureau_a_1_3.csv\n\n### credit_bureau_b_1 | Properties: depth=1, external data source, Credit bureau provider B\n\n- train_credit_bureau_b_1.csv\n\n### deposit_1 | Properties: depth=1, internal data source\n\n- train_deposit_1.csv\n\n### person_1 | Properties: depth=1, internal data source\n\n- train_person_1.csv\n\n### debitcard_1 | Properties: depth=1, internal data source\n\n- train_debitcard_1.csv","metadata":{}},{"cell_type":"code","source":"# df_train = merge_tables(df_train, \"applprev_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:50:37.442331Z","iopub.execute_input":"2024-03-17T17:50:37.442697Z","iopub.status.idle":"2024-03-17T17:50:37.447665Z","shell.execute_reply.started":"2024-03-17T17:50:37.442665Z","shell.execute_reply":"2024-03-17T17:50:37.446491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df_train = merge_parquets(df_train, \"other_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:50:37.450104Z","iopub.execute_input":"2024-03-17T17:50:37.450359Z","iopub.status.idle":"2024-03-17T17:50:37.457821Z","shell.execute_reply.started":"2024-03-17T17:50:37.450337Z","shell.execute_reply":"2024-03-17T17:50:37.456997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df_train = merge_parquets(df_train, \"deposit_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:50:37.458792Z","iopub.execute_input":"2024-03-17T17:50:37.459016Z","iopub.status.idle":"2024-03-17T17:50:37.467824Z","shell.execute_reply.started":"2024-03-17T17:50:37.458997Z","shell.execute_reply":"2024-03-17T17:50:37.467028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df_train = merge_parquets(df_train, \"debitcard_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:50:37.468940Z","iopub.execute_input":"2024-03-17T17:50:37.469521Z","iopub.status.idle":"2024-03-17T17:50:37.477311Z","shell.execute_reply.started":"2024-03-17T17:50:37.469491Z","shell.execute_reply":"2024-03-17T17:50:37.476527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"person_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:50:37.478386Z","iopub.execute_input":"2024-03-17T17:50:37.478901Z","iopub.status.idle":"2024-03-17T17:51:01.088485Z","shell.execute_reply.started":"2024-03-17T17:50:37.478871Z","shell.execute_reply":"2024-03-17T17:51:01.087444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df_train = merge_parquets(df_train, \"tax_registry_a_1\")\n# df_train = merge_parquets(df_train, \"tax_registry_b_1\")\n# df_train = merge_parquets(df_train, \"tax_registry_c_1\")  # empty in test set","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:01.089921Z","iopub.execute_input":"2024-03-17T17:51:01.090370Z","iopub.status.idle":"2024-03-17T17:51:01.096348Z","shell.execute_reply.started":"2024-03-17T17:51:01.090336Z","shell.execute_reply":"2024-03-17T17:51:01.095384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# TODO","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:01.097465Z","iopub.execute_input":"2024-03-17T17:51:01.097773Z","iopub.status.idle":"2024-03-17T17:51:01.104345Z","shell.execute_reply.started":"2024-03-17T17:51:01.097750Z","shell.execute_reply":"2024-03-17T17:51:01.103609Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 2\n\n### applprev_2 | Properties: depth=2, internal data source\n\n- train_applprev_2.csv\n\n### person_2 | Properties: depth=2, internal data source\n\n- train_person_2.csv\n\n### redit_bureau_a_2 | Properties: depth=2, external data source, Credit bureau provider A\n\n- train_credit_bureau_a_2_0.csv\n- train_credit_bureau_a_2_1.csv\n- train_credit_bureau_a_2_2.csv\n- train_credit_bureau_a_2_3.csv\n- train_credit_bureau_a_2_4.csv\n- train_credit_bureau_a_2_5.csv\n- train_credit_bureau_a_2_6.csv\n- train_credit_bureau_a_2_7.csv\n- train_credit_bureau_a_2_8.csv\n- train_credit_bureau_a_2_9.csv\n- train_credit_bureau_a_2_10.csv\n\n### credit_bureau_b_2 | Properties: depth=2, external data source, Credit bureau provider B\n\n- train_credit_bureau_b_2.csv","metadata":{}},{"cell_type":"code","source":"# TODO","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:01.108229Z","iopub.execute_input":"2024-03-17T17:51:01.108508Z","iopub.status.idle":"2024-03-17T17:51:01.114552Z","shell.execute_reply.started":"2024-03-17T17:51:01.108486Z","shell.execute_reply":"2024-03-17T17:51:01.113709Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Brows the data","metadata":{}},{"cell_type":"code","source":"print(f\"data size: {len(df_train)}\")\nprint(f\"unique: {len(df_train['case_id'].unique())}\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:01.115593Z","iopub.execute_input":"2024-03-17T17:51:01.115855Z","iopub.status.idle":"2024-03-17T17:51:01.160468Z","shell.execute_reply.started":"2024-03-17T17:51:01.115833Z","shell.execute_reply":"2024-03-17T17:51:01.159677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T17:51:01.161575Z","iopub.execute_input":"2024-03-17T17:51:01.161867Z","iopub.status.idle":"2024-03-17T17:51:01.227610Z","shell.execute_reply.started":"2024-03-17T17:51:01.161842Z","shell.execute_reply":"2024-03-17T17:51:01.226783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"t = df_train.groupby('date_decision')['target'].value_counts(normalize=True).mul(100)\nt.unstack().plot.bar(stacked=True, figsize=(14, 2), legend=True)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:01.228649Z","iopub.execute_input":"2024-03-17T17:51:01.228893Z","iopub.status.idle":"2024-03-17T17:51:08.962363Z","shell.execute_reply.started":"2024-03-17T17:51:01.228871Z","shell.execute_reply":"2024-03-17T17:51:08.961467Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.groupby('date_decision')['target'].mean().plot(\n    figsize=(14, 2), grid=True,\n    xlabel=\"date of decision\",\n    ylabel=\"day mean / proxi ratio\",\n)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:08.963594Z","iopub.execute_input":"2024-03-17T17:51:08.963967Z","iopub.status.idle":"2024-03-17T17:51:09.573284Z","shell.execute_reply.started":"2024-03-17T17:51:08.963933Z","shell.execute_reply":"2024-03-17T17:51:09.572376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train.dtypes)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T17:51:09.574587Z","iopub.execute_input":"2024-03-17T17:51:09.574981Z","iopub.status.idle":"2024-03-17T17:51:09.589489Z","shell.execute_reply.started":"2024-03-17T17:51:09.574940Z","shell.execute_reply":"2024-03-17T17:51:09.588590Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert_dtypes(df_train)\n# display(df_train.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T17:51:09.590503Z","iopub.execute_input":"2024-03-17T17:51:09.590773Z","iopub.status.idle":"2024-03-17T17:51:09.600138Z","shell.execute_reply.started":"2024-03-17T17:51:09.590750Z","shell.execute_reply":"2024-03-17T17:51:09.599198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train simple XGBoost","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\ndf_train.replace([np.inf, -np.inf], np.nan, inplace=True)\ntrain_cols = [c for c in df_train.columns if c not in (\"case_id\", \"date_decision\", \"target\")]\nX_train, X_valid, y_train, y_valid = train_test_split(\n    df_train[train_cols], df_train['target'], test_size=0.2)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:09.601231Z","iopub.execute_input":"2024-03-17T17:51:09.603433Z","iopub.status.idle":"2024-03-17T17:51:13.055356Z","shell.execute_reply.started":"2024-03-17T17:51:09.603389Z","shell.execute_reply":"2024-03-17T17:51:13.054333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -U xgboost -f /kaggle/input/xgboost-python-package/ --no-index","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T17:51:13.056731Z","iopub.execute_input":"2024-03-17T17:51:13.057244Z","iopub.status.idle":"2024-03-17T17:51:25.940690Z","shell.execute_reply.started":"2024-03-17T17:51:13.057217Z","shell.execute_reply":"2024-03-17T17:51:25.939478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import xgboost as xgb\n\nprint(xgb.__version__)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:51:25.942162Z","iopub.execute_input":"2024-03-17T17:51:25.942488Z","iopub.status.idle":"2024-03-17T17:51:26.101093Z","shell.execute_reply.started":"2024-03-17T17:51:25.942459Z","shell.execute_reply":"2024-03-17T17:51:26.100151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = xgb.XGBClassifier(\n    device=\"cuda\",\n    objective='binary:logistic',\n    enable_categorical=True,\n    eval_metric='auc',\n    #learning_rate=0.05,\n    subsample=1,\n    colsample_bytree=1,\n    min_child_weight=1,\n    #gamma=0.7,\n    #reg_alpha=0.7,\n    max_depth=20,\n    n_estimators=800,\n    random_state=42,\n)\n\n# Training the model on the training data\nmodel.fit(\n    X_train, y_train,\n    eval_set=[(X_valid, y_valid)],\n    early_stopping_rounds=5,\n    verbose=True,\n)\n\nprint(model)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T17:51:26.102429Z","iopub.execute_input":"2024-03-17T17:51:26.103040Z","iopub.status.idle":"2024-03-17T17:52:36.735099Z","shell.execute_reply.started":"2024-03-17T17:51:26.103007Z","shell.execute_reply":"2024-03-17T17:52:36.734148Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_columns_dtypes = dict(df_train.dtypes)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T17:52:36.736387Z","iopub.execute_input":"2024-03-17T17:52:36.736771Z","iopub.status.idle":"2024-03-17T17:52:36.742961Z","shell.execute_reply.started":"2024-03-17T17:52:36.736739Z","shell.execute_reply":"2024-03-17T17:52:36.741962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ОТБОР ПРИЗНАКОВ","metadata":{}},{"cell_type":"code","source":"zero_importance_features = [feature for feature, importance in zip(X_train.columns, model.feature_importances_) if importance == 0]\nzero_importance_features","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:00:32.111160Z","iopub.execute_input":"2024-03-17T18:00:32.112078Z","iopub.status.idle":"2024-03-17T18:00:32.126121Z","shell.execute_reply.started":"2024-03-17T18:00:32.112047Z","shell.execute_reply":"2024-03-17T18:00:32.125122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.replace([np.inf, -np.inf], np.nan, inplace=True)\ntrain_cols = [c for c in df_train.columns if c not in [\"case_id\", \"date_decision\", \"target\"] + zero_importance_features]\nX_train, X_valid, y_train, y_valid = train_test_split(\n    df_train[train_cols], df_train['target'], test_size=0.2)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:00:42.236317Z","iopub.execute_input":"2024-03-17T18:00:42.236691Z","iopub.status.idle":"2024-03-17T18:00:45.177588Z","shell.execute_reply.started":"2024-03-17T18:00:42.236664Z","shell.execute_reply":"2024-03-17T18:00:45.176769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ПЕРЕБОР ГИПЕРПАРАМЕТРОВ","metadata":{}},{"cell_type":"code","source":"# from sklearn.model_selection import RandomizedSearchCV\n\n\n# # Задаем пространство поиска для гиперпараметров\n# param_grid = {\n#     'learning_rate': [0.01, 0.05, 0.1, 0.3],\n#     'gamma': [0.0, 0.1, 0.2, 0.3],\n#     'reg_alpha': [0.0, 0.1, 0.2, 0.3],\n#     'n_estimators': [100, 300, 500, 800],\n#     'subsample': [0.8, 1.0],\n#     'colsample_bytree': [0.8, 1.0],\n#     'min_child_weight': [1, 5, 10],\n#     'max_depth': [3, 6, 9, 12]\n# }\n\n# # Создаем модель XGBClassifier\n# model = xgb.XGBClassifier(\n#     device='cuda',\n#     objective='binary:logistic',\n#     enable_categorical=True,\n#     eval_metric='auc',\n#     random_state=42,\n#     verbose=False\n# )\n\n# # Используем RandomizedSearchCV для поиска лучших гиперпараметров\n# random_search = RandomizedSearchCV(model, param_distributions=param_grid, n_iter=10, scoring='roc_auc', cv=3, verbose=3, random_state=42)\n# random_search.fit(\n#     X_train, y_train,\n#     eval_set=[(X_valid, y_valid)],\n#     early_stopping_rounds=5,\n#     verbose=True,\n# )\n\n# # Выводим лучшие гиперпараметры\n# print(random_search.best_params_)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2024-03-17T18:00:49.063031Z","iopub.execute_input":"2024-03-17T18:00:49.063769Z","iopub.status.idle":"2024-03-17T18:00:49.068919Z","shell.execute_reply.started":"2024-03-17T18:00:49.063740Z","shell.execute_reply":"2024-03-17T18:00:49.067849Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = xgb.XGBClassifier(\n    device=\"cuda\",\n    objective='binary:logistic',\n    enable_categorical=True,\n    eval_metric='auc',\n    learning_rate=0.1,\n    subsample=0.8,\n    colsample_bytree=0.8,\n    min_child_weight=10,\n    gamma=0.3,\n    reg_alpha=0.3,\n    max_depth=12,\n    n_estimators=100,\n    random_state=42,\n)\n\n# Training the model on the training data\nmodel.fit(\n    X_train, y_train,\n    eval_set=[(X_valid, y_valid)],\n    early_stopping_rounds=5,\n    verbose=True,\n)\n\n# print(model)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:00:49.733273Z","iopub.execute_input":"2024-03-17T18:00:49.734149Z","iopub.status.idle":"2024-03-17T18:01:34.882357Z","shell.execute_reply.started":"2024-03-17T18:00:49.734104Z","shell.execute_reply":"2024-03-17T18:01:34.881385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"# Loading test data","metadata":{}},{"cell_type":"markdown","source":"## base data","metadata":{"execution":{"iopub.status.busy":"2024-02-07T11:36:19.157007Z","iopub.execute_input":"2024-02-07T11:36:19.157411Z","iopub.status.idle":"2024-02-07T11:36:19.166433Z","shell.execute_reply.started":"2024-02-07T11:36:19.157376Z","shell.execute_reply":"2024-02-07T11:36:19.165721Z"}}},{"cell_type":"code","source":"df_test = pd.read_parquet(PARQUETS_TEST / \"test_base.parquet\")\nprint(f\"size: {len(df_test)}\")\ndisplay(df_test.head())","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:36.853452Z","iopub.execute_input":"2024-03-17T18:07:36.854339Z","iopub.status.idle":"2024-03-17T18:07:36.874246Z","shell.execute_reply.started":"2024-03-17T18:07:36.854308Z","shell.execute_reply":"2024-03-17T18:07:36.873387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test[\"date_decision\"] = pd.to_datetime(df_test[\"date_decision\"]).dt.date\n# delete redundat cols\ndel df_test[\"MONTH\"], df_test[\"WEEK_NUM\"]","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:37.004690Z","iopub.execute_input":"2024-03-17T18:07:37.005284Z","iopub.status.idle":"2024-03-17T18:07:37.012057Z","shell.execute_reply.started":"2024-03-17T18:07:37.005260Z","shell.execute_reply":"2024-03-17T18:07:37.011108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 0","metadata":{}},{"cell_type":"code","source":"for name in [\"static_0\", \"static_cb_0\"]:\n    df_test = merge_parquets(df_test, name, \"test\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:37.339777Z","iopub.execute_input":"2024-03-17T18:07:37.340061Z","iopub.status.idle":"2024-03-17T18:07:37.796147Z","shell.execute_reply.started":"2024-03-17T18:07:37.340039Z","shell.execute_reply":"2024-03-17T18:07:37.795253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 1","metadata":{}},{"cell_type":"code","source":"# df_test = merge_parquets(df_test, \"applprev_1\", \"test\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:37.914589Z","iopub.execute_input":"2024-03-17T18:07:37.914884Z","iopub.status.idle":"2024-03-17T18:07:37.918909Z","shell.execute_reply.started":"2024-03-17T18:07:37.914860Z","shell.execute_reply":"2024-03-17T18:07:37.918004Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for name in [\n    # \"other_1\",\n    # \"deposit_1\",\n    # \"debitcard_1\",\n    \"person_1\",\n    # \"tax_registry_a_1\",\n    # \"tax_registry_b_1\",\n    # \"tax_registry_c_1\"\n]:\n    df_test = merge_parquets(df_test, name, \"test\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:38.094817Z","iopub.execute_input":"2024-03-17T18:07:38.095120Z","iopub.status.idle":"2024-03-17T18:07:38.169276Z","shell.execute_reply.started":"2024-03-17T18:07:38.095095Z","shell.execute_reply":"2024-03-17T18:07:38.168442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Overview","metadata":{}},{"cell_type":"code","source":"for col, dt in train_columns_dtypes.items():\n    if col not in train_cols:\n        continue\n    try:\n        df_test[col] = df_test[col].astype(dt)\n    except:\n        print(f\"failed converting {col} to {dt}\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:38.582200Z","iopub.execute_input":"2024-03-17T18:07:38.582565Z","iopub.status.idle":"2024-03-17T18:07:38.650917Z","shell.execute_reply.started":"2024-03-17T18:07:38.582537Z","shell.execute_reply":"2024-03-17T18:07:38.650003Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_test.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T18:07:38.968906Z","iopub.execute_input":"2024-03-17T18:07:38.969216Z","iopub.status.idle":"2024-03-17T18:07:39.033303Z","shell.execute_reply.started":"2024-03-17T18:07:38.969191Z","shell.execute_reply":"2024-03-17T18:07:39.032386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_test.dtypes)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T18:07:39.336646Z","iopub.execute_input":"2024-03-17T18:07:39.336937Z","iopub.status.idle":"2024-03-17T18:07:39.350150Z","shell.execute_reply.started":"2024-03-17T18:07:39.336914Z","shell.execute_reply":"2024-03-17T18:07:39.349207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert_dtypes(df_test)\n# display(df_test.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-17T18:07:39.870872Z","iopub.execute_input":"2024-03-17T18:07:39.871717Z","iopub.status.idle":"2024-03-17T18:07:39.875398Z","shell.execute_reply.started":"2024-03-17T18:07:39.871683Z","shell.execute_reply":"2024-03-17T18:07:39.874383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict and submit","metadata":{}},{"cell_type":"code","source":"!head /kaggle/input/home-credit-credit-risk-model-stability/sample_submission.csv","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:39.950310Z","iopub.execute_input":"2024-03-17T18:07:39.950813Z","iopub.status.idle":"2024-03-17T18:07:40.939595Z","shell.execute_reply.started":"2024-03-17T18:07:39.950785Z","shell.execute_reply":"2024-03-17T18:07:40.938384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.replace([np.inf, -np.inf], np.nan, inplace=True)\npreds_proba = model.predict_proba(df_test[train_cols])\n\ndf_test[\"score\"] = np.clip(preds_proba[:, 1], 0, 1)\ndisplay(df_test[[\"case_id\", \"date_decision\", \"score\"]].head(10).T)","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:40.942014Z","iopub.execute_input":"2024-03-17T18:07:40.942440Z","iopub.status.idle":"2024-03-17T18:07:41.126104Z","shell.execute_reply.started":"2024-03-17T18:07:40.942384Z","shell.execute_reply":"2024-03-17T18:07:41.125182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df_test[\"case_id\"] = df_test[\"case_id\"].astype(\"int32\")\n# df_test[\"score\"] = df_test[\"score\"].astype(\"float32\")","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:41.127728Z","iopub.execute_input":"2024-03-17T18:07:41.128033Z","iopub.status.idle":"2024-03-17T18:07:41.132374Z","shell.execute_reply.started":"2024-03-17T18:07:41.128007Z","shell.execute_reply":"2024-03-17T18:07:41.131344Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test[[\"case_id\", \"score\"]].to_csv(\"submission.csv\", float_format='%.3f', index=False)\n\n!head submission.csv","metadata":{"execution":{"iopub.status.busy":"2024-03-17T18:07:41.260102Z","iopub.execute_input":"2024-03-17T18:07:41.260471Z","iopub.status.idle":"2024-03-17T18:07:42.259435Z","shell.execute_reply.started":"2024-03-17T18:07:41.260439Z","shell.execute_reply":"2024-03-17T18:07:42.258280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}