{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":50160,"databundleVersionId":7602123,"sourceType":"competition"}],"dockerImageVersionId":30646,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"In the competition [Home Credit - Credit Risk Model Stability](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability),\nwe are asked to predict which clients are more likely to default on their loans.","metadata":{}},{"cell_type":"markdown","source":"# Setup","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport polars as pl","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:15:39.209518Z","iopub.execute_input":"2024-02-09T02:15:39.210174Z","iopub.status.idle":"2024-02-09T02:15:39.554700Z","shell.execute_reply.started":"2024-02-09T02:15:39.210124Z","shell.execute_reply":"2024-02-09T02:15:39.553053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data overview","metadata":{}},{"cell_type":"markdown","source":"The competition dataset consists of a lot of tables.\nAll tables are in both csv and parquet formats.\nI prefer parquet.","metadata":{}},{"cell_type":"markdown","source":"## Base tables","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_base.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:29:10.515623Z","iopub.execute_input":"2024-02-09T02:29:10.516083Z","iopub.status.idle":"2024-02-09T02:29:11.645398Z","shell.execute_reply.started":"2024-02-09T02:29:10.516043Z","shell.execute_reply":"2024-02-09T02:29:11.644184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## static_0","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_static_0_*.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:29:26.355309Z","iopub.execute_input":"2024-02-09T02:29:26.355799Z","iopub.status.idle":"2024-02-09T02:29:27.475630Z","shell.execute_reply.started":"2024-02-09T02:29:26.355760Z","shell.execute_reply":"2024-02-09T02:29:27.473765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Please be aware that the same naming conventions apply to the test files. It's worth noting that some external data providers might not be available for future (test) evaluations, which is anticipated.","metadata":{}},{"cell_type":"markdown","source":"## static_cb_0","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_static_cb_0.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:29:42.534381Z","iopub.execute_input":"2024-02-09T02:29:42.534937Z","iopub.status.idle":"2024-02-09T02:29:43.641645Z","shell.execute_reply.started":"2024-02-09T02:29:42.534885Z","shell.execute_reply":"2024-02-09T02:29:43.639796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## applprev_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_applprev_1_*.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:29:58.690977Z","iopub.execute_input":"2024-02-09T02:29:58.691428Z","iopub.status.idle":"2024-02-09T02:29:59.816825Z","shell.execute_reply.started":"2024-02-09T02:29:58.691391Z","shell.execute_reply":"2024-02-09T02:29:59.815426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## other_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_other_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:30:13.925811Z","iopub.execute_input":"2024-02-09T02:30:13.926757Z","iopub.status.idle":"2024-02-09T02:30:15.041685Z","shell.execute_reply.started":"2024-02-09T02:30:13.926702Z","shell.execute_reply":"2024-02-09T02:30:15.039712Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## tax_registry_a_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_tax_registry_a_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:30:30.494663Z","iopub.execute_input":"2024-02-09T02:30:30.495114Z","iopub.status.idle":"2024-02-09T02:30:31.652826Z","shell.execute_reply.started":"2024-02-09T02:30:30.495071Z","shell.execute_reply":"2024-02-09T02:30:31.650931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## tax_registry_b_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_tax_registry_b_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:30:49.212544Z","iopub.execute_input":"2024-02-09T02:30:49.214063Z","iopub.status.idle":"2024-02-09T02:30:50.331725Z","shell.execute_reply.started":"2024-02-09T02:30:49.213986Z","shell.execute_reply":"2024-02-09T02:30:50.330323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## tax_registry_c_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_tax_registry_c_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:31:01.991120Z","iopub.execute_input":"2024-02-09T02:31:01.991667Z","iopub.status.idle":"2024-02-09T02:31:03.106420Z","shell.execute_reply.started":"2024-02-09T02:31:01.991618Z","shell.execute_reply":"2024-02-09T02:31:03.104984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## credit_bureau_a_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_credit_bureau_a_1_*.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:31:12.970636Z","iopub.execute_input":"2024-02-09T02:31:12.971256Z","iopub.status.idle":"2024-02-09T02:31:14.100892Z","shell.execute_reply.started":"2024-02-09T02:31:12.971208Z","shell.execute_reply":"2024-02-09T02:31:14.098880Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## credit_bureau_b_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_credit_bureau_b_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:31:27.095315Z","iopub.execute_input":"2024-02-09T02:31:27.095859Z","iopub.status.idle":"2024-02-09T02:31:28.221956Z","shell.execute_reply.started":"2024-02-09T02:31:27.095809Z","shell.execute_reply":"2024-02-09T02:31:28.220253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## deposit_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_deposit_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:31:44.080729Z","iopub.execute_input":"2024-02-09T02:31:44.081371Z","iopub.status.idle":"2024-02-09T02:31:45.232527Z","shell.execute_reply.started":"2024-02-09T02:31:44.081324Z","shell.execute_reply":"2024-02-09T02:31:45.231218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## person_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_person_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:31:56.615470Z","iopub.execute_input":"2024-02-09T02:31:56.616557Z","iopub.status.idle":"2024-02-09T02:31:57.723951Z","shell.execute_reply.started":"2024-02-09T02:31:56.616499Z","shell.execute_reply":"2024-02-09T02:31:57.722879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## debitcard_1","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_debitcard_1.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:32:08.111960Z","iopub.execute_input":"2024-02-09T02:32:08.112396Z","iopub.status.idle":"2024-02-09T02:32:09.220635Z","shell.execute_reply.started":"2024-02-09T02:32:08.112354Z","shell.execute_reply":"2024-02-09T02:32:09.219121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## applprev_2","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_applprev_2.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:32:20.034906Z","iopub.execute_input":"2024-02-09T02:32:20.035406Z","iopub.status.idle":"2024-02-09T02:32:21.205157Z","shell.execute_reply.started":"2024-02-09T02:32:20.035363Z","shell.execute_reply":"2024-02-09T02:32:21.203300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## person_2","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_person_2.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:32:30.451031Z","iopub.execute_input":"2024-02-09T02:32:30.451523Z","iopub.status.idle":"2024-02-09T02:32:31.607217Z","shell.execute_reply.started":"2024-02-09T02:32:30.451478Z","shell.execute_reply":"2024-02-09T02:32:31.605986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## credit_bureau_a_2","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_credit_bureau_a_2_*.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:32:39.841373Z","iopub.execute_input":"2024-02-09T02:32:39.841859Z","iopub.status.idle":"2024-02-09T02:32:41.003838Z","shell.execute_reply.started":"2024-02-09T02:32:39.841818Z","shell.execute_reply":"2024-02-09T02:32:41.001506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## credit_bureau_b_2","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/home-credit-credit-risk-model-stability/parquet_files/*/*_credit_bureau_b_2.parquet","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:32:59.164191Z","iopub.execute_input":"2024-02-09T02:32:59.165115Z","iopub.status.idle":"2024-02-09T02:33:00.273065Z","shell.execute_reply.started":"2024-02-09T02:32:59.165044Z","shell.execute_reply":"2024-02-09T02:33:00.271553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load dataset","metadata":{}},{"cell_type":"code","source":"df_train_base = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_base.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:38:47.350976Z","iopub.execute_input":"2024-02-09T02:38:47.351471Z","iopub.status.idle":"2024-02-09T02:38:47.439236Z","shell.execute_reply.started":"2024-02-09T02:38:47.351426Z","shell.execute_reply":"2024-02-09T02:38:47.437823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_base)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:39:15.418769Z","iopub.execute_input":"2024-02-09T02:39:15.419158Z","iopub.status.idle":"2024-02-09T02:39:15.432534Z","shell.execute_reply.started":"2024-02-09T02:39:15.419125Z","shell.execute_reply":"2024-02-09T02:39:15.430669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- `date_decision` is `str`. Should be casted to `date`.\n- `target` is the target. `1`: defaulted.","metadata":{}},{"cell_type":"code","source":"df_train_static_0_0 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_static_0_0.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:39:16.799630Z","iopub.execute_input":"2024-02-09T02:39:16.800154Z","iopub.status.idle":"2024-02-09T02:39:18.558891Z","shell.execute_reply.started":"2024-02-09T02:39:16.800116Z","shell.execute_reply":"2024-02-09T02:39:18.556812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_static_0_0)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:39:21.399987Z","iopub.execute_input":"2024-02-09T02:39:21.400426Z","iopub.status.idle":"2024-02-09T02:39:21.457431Z","shell.execute_reply.started":"2024-02-09T02:39:21.400392Z","shell.execute_reply":"2024-02-09T02:39:21.455668Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Many columns\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_static_cb_0 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_static_cb_0.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:41:35.198682Z","iopub.execute_input":"2024-02-09T02:41:35.199153Z","iopub.status.idle":"2024-02-09T02:41:36.105417Z","shell.execute_reply.started":"2024-02-09T02:41:35.199099Z","shell.execute_reply":"2024-02-09T02:41:36.104270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_static_cb_0)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:41:41.510057Z","iopub.execute_input":"2024-02-09T02:41:41.510682Z","iopub.status.idle":"2024-02-09T02:41:41.558380Z","shell.execute_reply.started":"2024-02-09T02:41:41.510645Z","shell.execute_reply":"2024-02-09T02:41:41.556747Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Date columns should be casted do `date`\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_applprev_1_0 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_applprev_1_0.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:43:09.137356Z","iopub.execute_input":"2024-02-09T02:43:09.137942Z","iopub.status.idle":"2024-02-09T02:43:11.104987Z","shell.execute_reply.started":"2024-02-09T02:43:09.137901Z","shell.execute_reply":"2024-02-09T02:43:11.103584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_applprev_1_0)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:43:16.065305Z","iopub.execute_input":"2024-02-09T02:43:16.065849Z","iopub.status.idle":"2024-02-09T02:43:16.105726Z","shell.execute_reply.started":"2024-02-09T02:43:16.065807Z","shell.execute_reply":"2024-02-09T02:43:16.104551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Date columns should be casted do `date`\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_other_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_other_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:44:11.978967Z","iopub.execute_input":"2024-02-09T02:44:11.979415Z","iopub.status.idle":"2024-02-09T02:44:12.009165Z","shell.execute_reply.started":"2024-02-09T02:44:11.979381Z","shell.execute_reply":"2024-02-09T02:44:12.008140Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_other_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:44:20.287902Z","iopub.execute_input":"2024-02-09T02:44:20.288298Z","iopub.status.idle":"2024-02-09T02:44:20.303526Z","shell.execute_reply.started":"2024-02-09T02:44:20.288269Z","shell.execute_reply":"2024-02-09T02:44:20.302057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_tax_registry_a_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_tax_registry_a_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:46:45.718399Z","iopub.execute_input":"2024-02-09T02:46:45.718855Z","iopub.status.idle":"2024-02-09T02:46:46.093284Z","shell.execute_reply.started":"2024-02-09T02:46:45.718822Z","shell.execute_reply":"2024-02-09T02:46:46.092098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_tax_registry_a_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:46:51.479225Z","iopub.execute_input":"2024-02-09T02:46:51.479680Z","iopub.status.idle":"2024-02-09T02:46:51.492856Z","shell.execute_reply.started":"2024-02-09T02:46:51.479646Z","shell.execute_reply":"2024-02-09T02:46:51.491304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Date columns should be casted to `date`","metadata":{}},{"cell_type":"code","source":"df_train_tax_registry_b_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_tax_registry_b_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:47:22.207609Z","iopub.execute_input":"2024-02-09T02:47:22.208950Z","iopub.status.idle":"2024-02-09T02:47:22.352389Z","shell.execute_reply.started":"2024-02-09T02:47:22.208900Z","shell.execute_reply":"2024-02-09T02:47:22.351479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_tax_registry_b_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:47:27.226718Z","iopub.execute_input":"2024-02-09T02:47:27.227123Z","iopub.status.idle":"2024-02-09T02:47:27.239842Z","shell.execute_reply.started":"2024-02-09T02:47:27.227091Z","shell.execute_reply":"2024-02-09T02:47:27.238204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_tax_registry_c_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_tax_registry_c_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:48:24.961603Z","iopub.execute_input":"2024-02-09T02:48:24.962034Z","iopub.status.idle":"2024-02-09T02:48:25.354727Z","shell.execute_reply.started":"2024-02-09T02:48:24.962001Z","shell.execute_reply":"2024-02-09T02:48:25.353767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_tax_registry_c_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:48:30.352831Z","iopub.execute_input":"2024-02-09T02:48:30.353246Z","iopub.status.idle":"2024-02-09T02:48:30.364454Z","shell.execute_reply.started":"2024-02-09T02:48:30.353214Z","shell.execute_reply":"2024-02-09T02:48:30.363134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_credit_bureau_a_1_0 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_credit_bureau_a_1_0.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:48:53.728674Z","iopub.execute_input":"2024-02-09T02:48:53.730298Z","iopub.status.idle":"2024-02-09T02:48:57.108629Z","shell.execute_reply.started":"2024-02-09T02:48:53.730233Z","shell.execute_reply":"2024-02-09T02:48:57.107466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_credit_bureau_a_1_0)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:49:01.266132Z","iopub.execute_input":"2024-02-09T02:49:01.266573Z","iopub.status.idle":"2024-02-09T02:49:01.322524Z","shell.execute_reply.started":"2024-02-09T02:49:01.266540Z","shell.execute_reply":"2024-02-09T02:49:01.320882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Data columns should be casted to `date`\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_credit_bureau_b_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_credit_bureau_b_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:50:39.010746Z","iopub.execute_input":"2024-02-09T02:50:39.011211Z","iopub.status.idle":"2024-02-09T02:50:39.089653Z","shell.execute_reply.started":"2024-02-09T02:50:39.011177Z","shell.execute_reply":"2024-02-09T02:50:39.088198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_credit_bureau_b_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:50:44.059621Z","iopub.execute_input":"2024-02-09T02:50:44.061242Z","iopub.status.idle":"2024-02-09T02:50:44.112892Z","shell.execute_reply.started":"2024-02-09T02:50:44.061170Z","shell.execute_reply":"2024-02-09T02:50:44.111067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Data columns should be casted to `date`\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_deposit_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_deposit_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:51:15.364647Z","iopub.execute_input":"2024-02-09T02:51:15.365124Z","iopub.status.idle":"2024-02-09T02:51:15.411306Z","shell.execute_reply.started":"2024-02-09T02:51:15.365084Z","shell.execute_reply":"2024-02-09T02:51:15.409946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_deposit_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:51:19.449791Z","iopub.execute_input":"2024-02-09T02:51:19.450437Z","iopub.status.idle":"2024-02-09T02:51:19.465010Z","shell.execute_reply.started":"2024-02-09T02:51:19.450390Z","shell.execute_reply":"2024-02-09T02:51:19.462954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Data columns should be casted to `date`\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_person_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_person_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:51:52.190769Z","iopub.execute_input":"2024-02-09T02:51:52.191358Z","iopub.status.idle":"2024-02-09T02:51:53.455183Z","shell.execute_reply.started":"2024-02-09T02:51:52.191315Z","shell.execute_reply":"2024-02-09T02:51:53.454211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_person_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:51:55.582963Z","iopub.execute_input":"2024-02-09T02:51:55.583337Z","iopub.status.idle":"2024-02-09T02:51:55.618533Z","shell.execute_reply.started":"2024-02-09T02:51:55.583308Z","shell.execute_reply":"2024-02-09T02:51:55.617382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Data columns should be casted to `date`\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_debitcard_1 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_debitcard_1.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:52:43.150188Z","iopub.execute_input":"2024-02-09T02:52:43.150609Z","iopub.status.idle":"2024-02-09T02:52:43.194371Z","shell.execute_reply.started":"2024-02-09T02:52:43.150578Z","shell.execute_reply":"2024-02-09T02:52:43.192244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_debitcard_1)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:52:48.951036Z","iopub.execute_input":"2024-02-09T02:52:48.951528Z","iopub.status.idle":"2024-02-09T02:52:48.967127Z","shell.execute_reply.started":"2024-02-09T02:52:48.951490Z","shell.execute_reply":"2024-02-09T02:52:48.966028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Data columns should be casted to `date`\n- Many missing values","metadata":{}},{"cell_type":"code","source":"df_train_applprev_2 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_applprev_2.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:54:40.770768Z","iopub.execute_input":"2024-02-09T02:54:40.771277Z","iopub.status.idle":"2024-02-09T02:54:42.086992Z","shell.execute_reply.started":"2024-02-09T02:54:40.771239Z","shell.execute_reply":"2024-02-09T02:54:42.085952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_applprev_2)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:54:44.694839Z","iopub.execute_input":"2024-02-09T02:54:44.695229Z","iopub.status.idle":"2024-02-09T02:54:44.709631Z","shell.execute_reply.started":"2024-02-09T02:54:44.695199Z","shell.execute_reply":"2024-02-09T02:54:44.707523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Many missing values\n- The columns are different from `applprev_1`","metadata":{}},{"cell_type":"code","source":"df_train_person_2 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_person_2.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:56:04.956428Z","iopub.execute_input":"2024-02-09T02:56:04.956991Z","iopub.status.idle":"2024-02-09T02:56:05.219918Z","shell.execute_reply.started":"2024-02-09T02:56:04.956948Z","shell.execute_reply":"2024-02-09T02:56:05.218756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_person_2)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:56:08.970718Z","iopub.execute_input":"2024-02-09T02:56:08.971175Z","iopub.status.idle":"2024-02-09T02:56:08.993933Z","shell.execute_reply.started":"2024-02-09T02:56:08.971140Z","shell.execute_reply":"2024-02-09T02:56:08.992405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Many missing values\n- The columns are different from `person_1`","metadata":{}},{"cell_type":"code","source":"df_train_credit_bureau_a_2_0 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_credit_bureau_a_2_0.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:57:07.078098Z","iopub.execute_input":"2024-02-09T02:57:07.078587Z","iopub.status.idle":"2024-02-09T02:57:08.134493Z","shell.execute_reply.started":"2024-02-09T02:57:07.078550Z","shell.execute_reply":"2024-02-09T02:57:08.133489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_credit_bureau_a_2_0)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:57:12.711785Z","iopub.execute_input":"2024-02-09T02:57:12.712280Z","iopub.status.idle":"2024-02-09T02:57:12.738298Z","shell.execute_reply.started":"2024-02-09T02:57:12.712242Z","shell.execute_reply":"2024-02-09T02:57:12.736520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Many missing values\n- The columns are different from `credit_bureau_a_1`","metadata":{}},{"cell_type":"code","source":"df_train_credit_bureau_b_2 = pl.read_parquet('/kaggle/input/home-credit-credit-risk-model-stability/parquet_files/train/train_credit_bureau_b_2.parquet')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:58:41.322369Z","iopub.execute_input":"2024-02-09T02:58:41.322878Z","iopub.status.idle":"2024-02-09T02:58:41.431839Z","shell.execute_reply.started":"2024-02-09T02:58:41.322841Z","shell.execute_reply":"2024-02-09T02:58:41.430691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train_credit_bureau_b_2)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T02:58:41.779512Z","iopub.execute_input":"2024-02-09T02:58:41.781127Z","iopub.status.idle":"2024-02-09T02:58:41.796324Z","shell.execute_reply.started":"2024-02-09T02:58:41.781073Z","shell.execute_reply":"2024-02-09T02:58:41.794520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Date columns should be casted to `date`\n- Many missing values\n- The columns are different from `credit_bureau_b_1`","metadata":{}},{"cell_type":"markdown","source":"> All other raw columns in the tables serve as predictors. Their definitions can be found in the file `feature_definitions.csv`. ","metadata":{}},{"cell_type":"code","source":"df_feature_definitions = pl.read_csv('/kaggle/input/home-credit-credit-risk-model-stability/feature_definitions.csv')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T03:00:57.916302Z","iopub.execute_input":"2024-02-09T03:00:57.916793Z","iopub.status.idle":"2024-02-09T03:00:58.092867Z","shell.execute_reply.started":"2024-02-09T03:00:57.916757Z","shell.execute_reply":"2024-02-09T03:00:58.091128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with pl.Config() as cfg:\n    cfg.set_fmt_str_lengths(150)\n    cfg.set_tbl_rows(-1)\n    display(df_feature_definitions)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T03:02:16.977121Z","iopub.execute_input":"2024-02-09T03:02:16.977739Z","iopub.status.idle":"2024-02-09T03:02:17.027574Z","shell.execute_reply.started":"2024-02-09T03:02:16.977680Z","shell.execute_reply":"2024-02-09T03:02:17.026412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Check the submission format.","metadata":{}},{"cell_type":"code","source":"df_sample_submission = pl.read_csv('/kaggle/input/home-credit-credit-risk-model-stability/sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2024-02-09T03:03:27.015003Z","iopub.execute_input":"2024-02-09T03:03:27.015414Z","iopub.status.idle":"2024-02-09T03:03:27.027100Z","shell.execute_reply.started":"2024-02-09T03:03:27.015384Z","shell.execute_reply":"2024-02-09T03:03:27.025325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_sample_submission)","metadata":{"execution":{"iopub.status.busy":"2024-02-09T03:03:31.646763Z","iopub.execute_input":"2024-02-09T03:03:31.647224Z","iopub.status.idle":"2024-02-09T03:03:31.657059Z","shell.execute_reply.started":"2024-02-09T03:03:31.647189Z","shell.execute_reply":"2024-02-09T03:03:31.655681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> For each case_id in the test set, you must predict a probability for the target score.","metadata":{}},{"cell_type":"markdown","source":"# Next steps","metadata":{}},{"cell_type":"markdown","source":"- EDA! There are a lot of data to understand.\n- Data cleaning. We have seen a lot of missing values and wrong dtypes.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}