{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":50160,"databundleVersionId":7921029,"sourceType":"competition"},{"sourceId":7487967,"sourceType":"datasetVersion","datasetId":3933894}],"dockerImageVersionId":30648,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Here I did a homework for CDM course in HSE University. I added depth=1 features to the initial notebook and tuned hyperparameters for xgboost with optuna library. As further work I can mention adding depth=2 features, combining them in a some way, may be some time series analysis, because one of the tasks in this competition is about stability of model during time. May be it tcan be useful in cases if we wantto find some seasonability or something like that.","metadata":{}},{"cell_type":"code","source":"import warnings\n# warnings.simplefilter(\"ignore\")\nwarnings.simplefilter(\"ignore\", UserWarning)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-03-15T11:10:44.844328Z","iopub.execute_input":"2024-03-15T11:10:44.845043Z","iopub.status.idle":"2024-03-15T11:10:44.849485Z","shell.execute_reply.started":"2024-03-15T11:10:44.845009Z","shell.execute_reply":"2024-03-15T11:10:44.848280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os, glob\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom pathlib import Path\n\nPATH_DATASET = Path(\"/kaggle/input/home-credit-credit-risk-model-stability\")\nPATH_PARQUETS = PATH_DATASET / \"parquet_files\"\nPARQUETS_TRAIN = PATH_PARQUETS / \"train\"\nPARQUETS_TEST = PATH_PARQUETS / \"test\"\npd.set_option('display.max_columns', None)\npd.set_option('display.max_rows', None)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-03-15T11:10:46.522678Z","iopub.execute_input":"2024-03-15T11:10:46.523533Z","iopub.status.idle":"2024-03-15T11:10:47.439287Z","shell.execute_reply.started":"2024-03-15T11:10:46.523498Z","shell.execute_reply":"2024-03-15T11:10:47.438466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Explore the training data\n\n**Borrowed from the competition describtion:**\n\nTable Description\nThis dataset contains a large number of tables as a result of utilizing diverse data sources and the varying levels of data aggregation used while preparing the dataset. Note: All files listed below are found in both .csv and .parquet formats.\n\n### Depth values\n\n- **depth=0** - These are static features directly tied to a specific case_id.\n- **depth=1** - Each case_id has an associated historical record, indexed by num_group1.\n- **depth=2** - Each case_id has an associated historical record, indexed by both num_group1 and num_group2.\n\nYou can read more about Credit bureau (CB) here https://en.wikipedia.org/wiki/Credit_bureau.","metadata":{}},{"cell_type":"code","source":"!cat /kaggle/input/home-credit-credit-risk-model-stability/feature_definitions.csv","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T11:10:47.686572Z","iopub.execute_input":"2024-03-15T11:10:47.687086Z","iopub.status.idle":"2024-03-15T11:10:48.629379Z","shell.execute_reply.started":"2024-03-15T11:10:47.687042Z","shell.execute_reply":"2024-03-15T11:10:48.628259Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Various predictors were transformed, therefore we have the following notation for similar groups of transformations\n\n- **P** - Transform DPD (Days past due)\n- **M** - Masking categories\n- **A** - Transform amount\n- **D** - Transform date\n- **T** - Unspecified Transform\n- **L** - Unspecified Transform\n\nPlease note that transformations within a group are denoted by a capital letter at the end of the predictor name (e.g., maxdbddpdtollast6m_4187119P). We hope that this will simplify the manipulation with predictors.","metadata":{}},{"cell_type":"code","source":"def _short_array(arr):\n    if len(arr) <= 5:\n        return repr(arr)\n    return f\"[{', '.join(map(str, arr[:2]))}, ..., {', '.join(map(str, arr[-2:]))}]\"\n\n# taking inpiration with column names from https://www.kaggle.com/code/greysky/home-credit-baseline\ndef convert_dtypes(df):\n    cols = []\n    for col, dt in dict(df.dtypes).items():\n        if col.startswith(\"for\"):\n            df[col] = df[col].fillna(0).astype(\"int16\")\n        elif \"num\" in col or \"cnt\" in col:\n            df[col] = df[col].fillna(0).astype(\"int32\")\n        elif col.startswith(\"pct\"):\n            df[col] = df[col].astype(\"float16\")\n        elif col[-1] in (\"A\", \"P\"):\n            df[col] = df[col].astype(\"float32\")\n        elif col[-1] in (\"D\", ):\n            df[col] = pd.to_datetime(df[col])\n        elif col[-1] in (\"M\", \"L\"):\n            if col[-1] == \"L\" and dt.name.startswith(\"int\"):\n                df[col] = df[col].astype(\"int32\")\n            elif col[-1] == \"L\" and dt.name.startswith(\"float\"):\n                df[col] = df[col].astype(\"float32\")\n            else:\n                uq = list(df[col].unique())\n                print(f'{col} -> #{len(uq)} -> {_short_array(uq)}')\n                df[col] = df[col].astype(\"category\")\n        if col[-1] in (\"A\", \"P\", \"M\", \"L\"):\n            cols.append(col)\n    return cols","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:10:48.770579Z","iopub.execute_input":"2024-03-15T11:10:48.770935Z","iopub.status.idle":"2024-03-15T11:10:48.783286Z","shell.execute_reply.started":"2024-03-15T11:10:48.770903Z","shell.execute_reply":"2024-03-15T11:10:48.782396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Base tables\n\nBase tables store the basic information about the observation and case_id. This is a unique identification of every observation and you need to use it to join the other tables to base tables.\n\ntrain_base.csv","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_parquet(PARQUETS_TRAIN / \"train_base.parquet\")\nprint(f\"size: {len(df_train)}\")\ndisplay(df_train.head())","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:10:49.671729Z","iopub.execute_input":"2024-03-15T11:10:49.672520Z","iopub.status.idle":"2024-03-15T11:10:50.055418Z","shell.execute_reply.started":"2024-03-15T11:10:49.672489Z","shell.execute_reply":"2024-03-15T11:10:50.054416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train[\"date_decision\"] = pd.to_datetime(df_train[\"date_decision\"]).dt.date\n# delete redundat cols\ndel df_train[\"MONTH\"], df_train[\"WEEK_NUM\"]","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:10:51.513090Z","iopub.execute_input":"2024-03-15T11:10:51.513457Z","iopub.status.idle":"2024-03-15T11:10:52.263141Z","shell.execute_reply.started":"2024-03-15T11:10:51.513430Z","shell.execute_reply":"2024-03-15T11:10:52.262361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def merge_parquets(df, name, prefix=\"train\", folder=PATH_PARQUETS):\n    df_ = pd.concat(\n        [pd.read_parquet(p) for p in glob.glob(str(folder / prefix / f\"{prefix}_{name}*.parquet\"))],\n    )\n    if \"num_group1\" in df_.columns:\n        del df_[\"num_group1\"]\n    df_.drop_duplicates(inplace=True)\n    print(f\"{name} size: {len(df_)} with features: {len(df_.columns)}\")\n    display(df_.head())\n    cols = convert_dtypes(df_) + [\"case_id\"]\n    df_ = df_[cols]\n    if len(df_) > len(df_[\"case_id\"].unique()):\n        df_ = df_.groupby(['case_id'], as_index=False).first()\n    df = df.merge(df_, how=\"left\", on=\"case_id\")\n    print(f\"fused size: {len(df)}\")\n    return df","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:10:52.264681Z","iopub.execute_input":"2024-03-15T11:10:52.264997Z","iopub.status.idle":"2024-03-15T11:10:52.272472Z","shell.execute_reply.started":"2024-03-15T11:10:52.264970Z","shell.execute_reply":"2024-03-15T11:10:52.271592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 0\n\n### static_0 | Properties: depth=0, internal data source\n\n- train_static_0_0.csv\n- train_static_0_1.csv\n\n### static_cb_0 | Properties: depth=0, external data source\n\n- train_static_cb_0.csv","metadata":{}},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"static_0\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:10:53.188092Z","iopub.execute_input":"2024-03-15T11:10:53.188939Z","iopub.status.idle":"2024-03-15T11:11:25.836988Z","shell.execute_reply.started":"2024-03-15T11:10:53.188909Z","shell.execute_reply":"2024-03-15T11:11:25.836023Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"static_cb_0\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:11:25.838887Z","iopub.execute_input":"2024-03-15T11:11:25.839625Z","iopub.status.idle":"2024-03-15T11:11:33.710320Z","shell.execute_reply.started":"2024-03-15T11:11:25.839589Z","shell.execute_reply":"2024-03-15T11:11:33.709378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 1\n\n### applprev_1 | Properties: depth=1, internal data source\n\n- train_applprev_1_0.csv\n- train_applprev_1_1.csv\n\n### other_1 | Properties: depth=1, internal data source\n\n- train_other_1.csv\n\n### tax_registry_a_1 | Properties: depth=1, external data source, Tax registry provider A\n\n- train_tax_registry_a_1.csv\n\n### tax_registry_b_1 | Properties: depth=1, external data source, Tax registry provider B\n\n- train_tax_registry_b_1.csv\n\n### tax_registry_c_1 | Properties: depth=1, external data source, Tax registry provider C\n\n- train_tax_registry_c_1.csv\n\n### credit_bureau_a_1 | Properties: depth=1, external data source, Credit bureau provider A\n\n- train_credit_bureau_a_1_0.csv\n- train_credit_bureau_a_1_1.csv\n- train_credit_bureau_a_1_2.csv\n- train_credit_bureau_a_1_3.csv\n\n### credit_bureau_b_1 | Properties: depth=1, external data source, Credit bureau provider B\n\n- train_credit_bureau_b_1.csv\n\n### deposit_1 | Properties: depth=1, internal data source\n\n- train_deposit_1.csv\n\n### person_1 | Properties: depth=1, internal data source\n\n- train_person_1.csv\n\n### debitcard_1 | Properties: depth=1, internal data source\n\n- train_debitcard_1.csv","metadata":{}},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"applprev_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:11:33.711642Z","iopub.execute_input":"2024-03-15T11:11:33.712009Z","iopub.status.idle":"2024-03-15T11:12:41.398011Z","shell.execute_reply.started":"2024-03-15T11:11:33.711976Z","shell.execute_reply":"2024-03-15T11:12:41.397082Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.shape","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:12:41.400847Z","iopub.execute_input":"2024-03-15T11:12:41.401274Z","iopub.status.idle":"2024-03-15T11:12:41.407000Z","shell.execute_reply.started":"2024-03-15T11:12:41.401245Z","shell.execute_reply":"2024-03-15T11:12:41.406191Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"other_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:12:41.408431Z","iopub.execute_input":"2024-03-15T11:12:41.408714Z","iopub.status.idle":"2024-03-15T11:12:42.165949Z","shell.execute_reply.started":"2024-03-15T11:12:41.408680Z","shell.execute_reply":"2024-03-15T11:12:42.165064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"deposit_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:12:42.167058Z","iopub.execute_input":"2024-03-15T11:12:42.167325Z","iopub.status.idle":"2024-03-15T11:12:42.957478Z","shell.execute_reply.started":"2024-03-15T11:12:42.167302Z","shell.execute_reply":"2024-03-15T11:12:42.956579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"debitcard_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:12:42.958473Z","iopub.execute_input":"2024-03-15T11:12:42.958724Z","iopub.status.idle":"2024-03-15T11:12:43.721381Z","shell.execute_reply.started":"2024-03-15T11:12:42.958702Z","shell.execute_reply":"2024-03-15T11:12:43.720441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = merge_parquets(df_train, \"person_1\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:12:43.722886Z","iopub.execute_input":"2024-03-15T11:12:43.723171Z","iopub.status.idle":"2024-03-15T11:13:07.594239Z","shell.execute_reply.started":"2024-03-15T11:12:43.723146Z","shell.execute_reply":"2024-03-15T11:13:07.593273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df_train = merge_parquets(df_train, \"tax_registry_a_1\")\n# df_train = merge_parquets(df_train, \"tax_registry_b_1\")\n# df_train = merge_parquets(df_train, \"tax_registry_c_1\")  # empty in test set","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:13:07.595631Z","iopub.execute_input":"2024-03-15T11:13:07.596273Z","iopub.status.idle":"2024-03-15T11:13:07.600262Z","shell.execute_reply.started":"2024-03-15T11:13:07.596236Z","shell.execute_reply":"2024-03-15T11:13:07.599331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.head()","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:13:07.603761Z","iopub.execute_input":"2024-03-15T11:13:07.604027Z","iopub.status.idle":"2024-03-15T11:13:07.784087Z","shell.execute_reply.started":"2024-03-15T11:13:07.604004Z","shell.execute_reply":"2024-03-15T11:13:07.783106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 2\n\n### applprev_2 | Properties: depth=2, internal data source\n\n- train_applprev_2.csv\n\n### person_2 | Properties: depth=2, internal data source\n\n- train_person_2.csv\n\n### redit_bureau_a_2 | Properties: depth=2, external data source, Credit bureau provider A\n\n- train_credit_bureau_a_2_0.csv\n- train_credit_bureau_a_2_1.csv\n- train_credit_bureau_a_2_2.csv\n- train_credit_bureau_a_2_3.csv\n- train_credit_bureau_a_2_4.csv\n- train_credit_bureau_a_2_5.csv\n- train_credit_bureau_a_2_6.csv\n- train_credit_bureau_a_2_7.csv\n- train_credit_bureau_a_2_8.csv\n- train_credit_bureau_a_2_9.csv\n- train_credit_bureau_a_2_10.csv\n\n### credit_bureau_b_2 | Properties: depth=2, external data source, Credit bureau provider B\n\n- train_credit_bureau_b_2.csv","metadata":{}},{"cell_type":"code","source":"# TODO","metadata":{"execution":{"iopub.status.busy":"2024-02-08T15:27:43.912721Z","iopub.execute_input":"2024-02-08T15:27:43.913079Z","iopub.status.idle":"2024-02-08T15:27:43.923304Z","shell.execute_reply.started":"2024-02-08T15:27:43.913053Z","shell.execute_reply":"2024-02-08T15:27:43.922363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Brows the data","metadata":{}},{"cell_type":"code","source":"print(f\"data size: {len(df_train)}\")\nprint(f\"unique: {len(df_train['case_id'].unique())}\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T10:08:46.204621Z","iopub.execute_input":"2024-03-15T10:08:46.205371Z","iopub.status.idle":"2024-03-15T10:08:46.249601Z","shell.execute_reply.started":"2024-03-15T10:08:46.205340Z","shell.execute_reply":"2024-03-15T10:08:46.248537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T10:08:51.695379Z","iopub.execute_input":"2024-03-15T10:08:51.695742Z","iopub.status.idle":"2024-03-15T10:08:51.780414Z","shell.execute_reply.started":"2024-03-15T10:08:51.695715Z","shell.execute_reply":"2024-03-15T10:08:51.779530Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"t = df_train.groupby('date_decision')['target'].value_counts(normalize=True).mul(100)\nt.unstack().plot.bar(stacked=True, figsize=(14, 2), legend=True)","metadata":{"execution":{"iopub.status.busy":"2024-03-15T10:09:11.137966Z","iopub.execute_input":"2024-03-15T10:09:11.138332Z","iopub.status.idle":"2024-03-15T10:09:19.501429Z","shell.execute_reply.started":"2024-03-15T10:09:11.138304Z","shell.execute_reply":"2024-03-15T10:09:19.500554Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.groupby('date_decision')['target'].mean().plot(\n    figsize=(14, 2), grid=True,\n    xlabel=\"date of decision\",\n    ylabel=\"day mean / proxi ratio\",\n)","metadata":{"execution":{"iopub.status.busy":"2024-02-08T15:27:53.230272Z","iopub.execute_input":"2024-02-08T15:27:53.23079Z","iopub.status.idle":"2024-02-08T15:27:53.939384Z","shell.execute_reply.started":"2024-02-08T15:27:53.230736Z","shell.execute_reply":"2024-02-08T15:27:53.938086Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train.dtypes)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T10:11:25.288935Z","iopub.execute_input":"2024-03-15T10:11:25.289745Z","iopub.status.idle":"2024-03-15T10:11:25.304977Z","shell.execute_reply.started":"2024-03-15T10:11:25.289712Z","shell.execute_reply":"2024-03-15T10:11:25.303947Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert_dtypes(df_train)\n# display(df_train.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-02-08T15:27:53.961932Z","iopub.execute_input":"2024-02-08T15:27:53.962357Z","iopub.status.idle":"2024-02-08T15:27:53.967631Z","shell.execute_reply.started":"2024-02-08T15:27:53.962318Z","shell.execute_reply":"2024-02-08T15:27:53.966331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train simple XGBoost","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\ndf_train.replace([np.inf, -np.inf], np.nan, inplace=True)\ntrain_cols = [c for c in df_train.columns if c not in (\"case_id\", \"date_decision\", \"target\")]\nX_train, X_valid, y_train, y_valid = train_test_split(\n    df_train[train_cols], df_train['target'], test_size=0.2)","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:13:07.785330Z","iopub.execute_input":"2024-03-15T11:13:07.785626Z","iopub.status.idle":"2024-03-15T11:13:12.014121Z","shell.execute_reply.started":"2024-03-15T11:13:07.785600Z","shell.execute_reply":"2024-03-15T11:13:12.013266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -U xgboost -f /kaggle/input/xgboost-python-package/ --no-index","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T10:12:47.684241Z","iopub.execute_input":"2024-03-15T10:12:47.684882Z","iopub.status.idle":"2024-03-15T10:13:00.757664Z","shell.execute_reply.started":"2024-03-15T10:12:47.684832Z","shell.execute_reply":"2024-03-15T10:13:00.756517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import xgboost as xgb\n\nprint(xgb.__version__)","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:13:12.015160Z","iopub.execute_input":"2024-03-15T11:13:12.015642Z","iopub.status.idle":"2024-03-15T11:13:12.177083Z","shell.execute_reply.started":"2024-03-15T11:13:12.015617Z","shell.execute_reply":"2024-03-15T11:13:12.176169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install optuna","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:13:12.178478Z","iopub.execute_input":"2024-03-15T11:13:12.179176Z","iopub.status.idle":"2024-03-15T11:13:45.079194Z","shell.execute_reply.started":"2024-03-15T11:13:12.179140Z","shell.execute_reply":"2024-03-15T11:13:45.078201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import optuna\n# from sklearn.metrics import roc_auc_score","metadata":{"execution":{"iopub.status.busy":"2024-03-15T11:13:45.080638Z","iopub.execute_input":"2024-03-15T11:13:45.080945Z","iopub.status.idle":"2024-03-15T11:13:46.233740Z","shell.execute_reply.started":"2024-03-15T11:13:45.080917Z","shell.execute_reply":"2024-03-15T11:13:46.232932Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def objective(trial):\n#     params = {\n#         \"device\": \"cuda\",\n#         \"objective\":'binary:logistic',\n#         \"enable_categorical\":True,\n#         \"eval_metric\":'auc',\n#         'learning_rate': trial.suggest_categorical('learning_rate', [0.008,0.01,0.012,0.014,0.016,0.018,0.02,0.05]),\n#         'n_estimators': trial.suggest_int('n_estimators', 100, 1000),\n#         'max_depth': trial.suggest_categorical('max_depth', [5,7,9,11,13,15,17]),\n#         \"subsample\":1,\n#         \"colsample_bytree\":1,\n#         \"min_child_weight\":1,\n        \n#     }\n    \n#     model = xgb.XGBClassifier(\n#         **params\n#     )\n\n#     # Training the model on the training data\n#     model.fit(\n#         X_train, y_train,\n#         eval_set=[(X_valid, y_valid)],\n#         early_stopping_rounds=5,\n#         verbose=False,\n#     )\n#     y_pred = model.predict(X_valid)\n#     return roc_auc_score(y_valid, y_pred)\n    ","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T11:13:46.234758Z","iopub.execute_input":"2024-03-15T11:13:46.235010Z","iopub.status.idle":"2024-03-15T11:13:46.242592Z","shell.execute_reply.started":"2024-03-15T11:13:46.234988Z","shell.execute_reply":"2024-03-15T11:13:46.241645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# study = optuna.create_study(direction='maximize')\n# study.optimize(objective, n_trials=10, show_progress_bar=True)\n# print('Number of finished trials:', len(study.trials))\n# print('Best trial:', study.best_trial.params)","metadata":{"execution":{"iopub.status.busy":"2024-03-15T12:34:16.492263Z","iopub.execute_input":"2024-03-15T12:34:16.492639Z","iopub.status.idle":"2024-03-15T13:13:04.980654Z","shell.execute_reply.started":"2024-03-15T12:34:16.492611Z","shell.execute_reply":"2024-03-15T13:13:04.979650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params = {\n        \"device\": \"cuda\",\n        \"objective\":'binary:logistic',\n        \"enable_categorical\":True,\n        \"eval_metric\":'auc',\n        'learning_rate': 0.05,\n        'n_estimators': 166,\n        'max_depth': 17,\n        \"subsample\":1,\n        \"colsample_bytree\":1,\n        \"min_child_weight\":1,\n        \n    }\n\nmodel = xgb.XGBClassifier(\n    **params\n)\n\n# Training the model on the training data\nmodel.fit(\n    X_train, y_train,\n    eval_set=[(X_valid, y_valid)],\n    early_stopping_rounds=5,\n    verbose=True,\n)\n","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:53:54.568362Z","iopub.execute_input":"2024-03-15T13:53:54.568731Z","iopub.status.idle":"2024-03-15T13:57:19.737052Z","shell.execute_reply.started":"2024-03-15T13:53:54.568704Z","shell.execute_reply":"2024-03-15T13:57:19.735928Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_columns_dtypes = dict(df_train.dtypes)\ndel df_train","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:57:58.564710Z","iopub.execute_input":"2024-03-15T13:57:58.565745Z","iopub.status.idle":"2024-03-15T13:57:58.602474Z","shell.execute_reply.started":"2024-03-15T13:57:58.565702Z","shell.execute_reply":"2024-03-15T13:57:58.601276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading test data","metadata":{}},{"cell_type":"markdown","source":"## base data","metadata":{"execution":{"iopub.status.busy":"2024-02-07T11:36:19.157007Z","iopub.execute_input":"2024-02-07T11:36:19.157411Z","iopub.status.idle":"2024-02-07T11:36:19.166433Z","shell.execute_reply.started":"2024-02-07T11:36:19.157376Z","shell.execute_reply":"2024-02-07T11:36:19.165721Z"}}},{"cell_type":"code","source":"df_test = pd.read_parquet(PARQUETS_TEST / \"test_base.parquet\")\nprint(f\"size: {len(df_test)}\")\ndisplay(df_test.head())","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:10.259306Z","iopub.execute_input":"2024-03-15T13:58:10.260027Z","iopub.status.idle":"2024-03-15T13:58:10.286555Z","shell.execute_reply.started":"2024-03-15T13:58:10.259997Z","shell.execute_reply":"2024-03-15T13:58:10.285614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test[\"date_decision\"] = pd.to_datetime(df_test[\"date_decision\"]).dt.date\n# delete redundat cols\ndel df_test[\"MONTH\"], df_test[\"WEEK_NUM\"]","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:10.723656Z","iopub.execute_input":"2024-03-15T13:58:10.724526Z","iopub.status.idle":"2024-03-15T13:58:10.731658Z","shell.execute_reply.started":"2024-03-15T13:58:10.724494Z","shell.execute_reply":"2024-03-15T13:58:10.730666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 0","metadata":{}},{"cell_type":"code","source":"for name in [\"static_0\", \"static_cb_0\"]:\n    df_test = merge_parquets(df_test, name, \"test\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:12.076239Z","iopub.execute_input":"2024-03-15T13:58:12.077082Z","iopub.status.idle":"2024-03-15T13:58:12.509916Z","shell.execute_reply.started":"2024-03-15T13:58:12.077051Z","shell.execute_reply":"2024-03-15T13:58:12.508793Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.shape","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:12.511521Z","iopub.execute_input":"2024-03-15T13:58:12.511800Z","iopub.status.idle":"2024-03-15T13:58:12.517657Z","shell.execute_reply.started":"2024-03-15T13:58:12.511775Z","shell.execute_reply":"2024-03-15T13:58:12.516670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Depth 1","metadata":{}},{"cell_type":"code","source":"df_test = merge_parquets(df_test, \"applprev_1\", \"test\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:13.456769Z","iopub.execute_input":"2024-03-15T13:58:13.457556Z","iopub.status.idle":"2024-03-15T13:58:13.567232Z","shell.execute_reply.started":"2024-03-15T13:58:13.457524Z","shell.execute_reply":"2024-03-15T13:58:13.566318Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for name in [\n    \"other_1\",\n    \"deposit_1\",\n    \"debitcard_1\",\n    \"person_1\",\n    # \"tax_registry_a_1\",\n    # \"tax_registry_b_1\",\n    # \"tax_registry_c_1\"\n]:\n    df_test = merge_parquets(df_test, name, \"test\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:14.813068Z","iopub.execute_input":"2024-03-15T13:58:14.813952Z","iopub.status.idle":"2024-03-15T13:58:14.964943Z","shell.execute_reply.started":"2024-03-15T13:58:14.813919Z","shell.execute_reply":"2024-03-15T13:58:14.964004Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.shape","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:16.994249Z","iopub.execute_input":"2024-03-15T13:58:16.995089Z","iopub.status.idle":"2024-03-15T13:58:17.000951Z","shell.execute_reply.started":"2024-03-15T13:58:16.995037Z","shell.execute_reply":"2024-03-15T13:58:16.999978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Overview","metadata":{}},{"cell_type":"code","source":"for col, dt in train_columns_dtypes.items():\n    if col not in train_cols:\n        continue\n    try:\n        df_test[col] = df_test[col].astype(dt)\n    except:\n        print(f\"failed converting {col} to {dt}\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:19.336288Z","iopub.execute_input":"2024-03-15T13:58:19.336660Z","iopub.status.idle":"2024-03-15T13:58:19.402224Z","shell.execute_reply.started":"2024-03-15T13:58:19.336632Z","shell.execute_reply":"2024-03-15T13:58:19.401254Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_test.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T13:58:30.608008Z","iopub.execute_input":"2024-03-15T13:58:30.608401Z","iopub.status.idle":"2024-03-15T13:58:30.671802Z","shell.execute_reply.started":"2024-03-15T13:58:30.608373Z","shell.execute_reply":"2024-03-15T13:58:30.670993Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_test.dtypes)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T13:58:31.989075Z","iopub.execute_input":"2024-03-15T13:58:31.989451Z","iopub.status.idle":"2024-03-15T13:58:32.004512Z","shell.execute_reply.started":"2024-03-15T13:58:31.989426Z","shell.execute_reply":"2024-03-15T13:58:32.003483Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert_dtypes(df_test)\n# display(df_test.head().T)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-15T13:58:33.055966Z","iopub.execute_input":"2024-03-15T13:58:33.056356Z","iopub.status.idle":"2024-03-15T13:58:33.060368Z","shell.execute_reply.started":"2024-03-15T13:58:33.056327Z","shell.execute_reply":"2024-03-15T13:58:33.059379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict and submit","metadata":{}},{"cell_type":"code","source":"!head /kaggle/input/home-credit-credit-risk-model-stability/sample_submission.csv","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:35.465490Z","iopub.execute_input":"2024-03-15T13:58:35.466378Z","iopub.status.idle":"2024-03-15T13:58:36.547740Z","shell.execute_reply.started":"2024-03-15T13:58:35.466345Z","shell.execute_reply":"2024-03-15T13:58:36.546606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.replace([np.inf, -np.inf], np.nan, inplace=True)\npreds_proba = model.predict_proba(df_test[train_cols])\n\ndf_test[\"score\"] = np.clip(preds_proba[:, 1], 0, 1)\ndisplay(df_test[[\"case_id\", \"date_decision\", \"score\"]].head(10).T)","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:36.550202Z","iopub.execute_input":"2024-03-15T13:58:36.550629Z","iopub.status.idle":"2024-03-15T13:58:36.815184Z","shell.execute_reply.started":"2024-03-15T13:58:36.550588Z","shell.execute_reply":"2024-03-15T13:58:36.814258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test[\"case_id\"] = df_test[\"case_id\"].astype(\"int32\")\ndf_test[\"score\"] = df_test[\"score\"].astype(\"float32\")","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:37.973329Z","iopub.execute_input":"2024-03-15T13:58:37.973685Z","iopub.status.idle":"2024-03-15T13:58:37.979151Z","shell.execute_reply.started":"2024-03-15T13:58:37.973660Z","shell.execute_reply":"2024-03-15T13:58:37.978238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test[[\"case_id\", \"score\"]].to_csv(\"submission.csv\", float_format='%.3f', index=False)\n\n!head submission.csv","metadata":{"execution":{"iopub.status.busy":"2024-03-15T13:58:40.385845Z","iopub.execute_input":"2024-03-15T13:58:40.386573Z","iopub.status.idle":"2024-03-15T13:58:41.461554Z","shell.execute_reply.started":"2024-03-15T13:58:40.386543Z","shell.execute_reply":"2024-03-15T13:58:41.460426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}