{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":96164,"databundleVersionId":12993472,"sourceType":"competition"},{"sourceId":12462463,"sourceType":"datasetVersion","datasetId":7861436},{"sourceId":12477994,"sourceType":"datasetVersion","datasetId":7870836}],"dockerImageVersionId":31040,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"- 0.90038  v.03 - [XGB + Deep Learning Ensemble](https://www.kaggle.com/code/taylorsamarel/xgb-deep-learning-ensemble) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n- 0.88377  v.09 - [REBOOT: Use the lead values - IMPROVED](https://www.kaggle.com/code/taylorsamarel/reboot-use-the-lead-values-improved) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n- 0.89178  v.13 - [REBOOT: SGD CONVERGENCE ++](https://www.kaggle.com/code/taylorsamarel/reboot-sgd-convergence/notebook) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n- 0.86767  v.01 - [DRW-Ensembling-0.86767](https://www.kaggle.com/code/vishalpainjane/drw-ensemble-0-72837) - [Vishal Painjane](https://www.kaggle.com/vishalpainjane)\n- 0.83975  v.07 - [REBOOT: Use the lead values](https://www.kaggle.com/code/gromml/reboot-use-the-lead-values/notebook) - [gromml](https://www.kaggle.com/gromml)\n\n \n\n15-july-2025 - LB = [0.93794](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n","metadata":{}},{"cell_type":"markdown","source":"### Archive\n\n- 0.12744  v.10 - [Top Notebook + New Model CSV Ensemble](https://www.kaggle.com/code/migrantworkerdatahub/top-notebook-new-model-csv-ensemble) - [Migrant Worker Data Hub](https://www.kaggle.com/migrantworkerdatahub)\n- 0.12958  v.03 - [DRW - Ensemble of Ensemble of Ensemble](https://www.kaggle.com/code/yingjunmao/drw-ensemble-of-ensemble-of-ensemble) - [YingJunMao](https://www.kaggle.com/yingjunmao)\n- 0.12864  v.26 - [**5 subm.files** from page Ensemble_Envy / averaged_final_submissions -> iBlend](https://www.kaggle.com/code/taylorsamarel/ensemble-envy)\n\n3-july-2025 - LB = [0.12957](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n \n\n- 0.57383  v.01 - [All You Need is Random State 700](https://www.kaggle.com/code/shinchen93/all-you-need-is-random-state-700) - [ShinC](https://www.kaggle.com/shinchen93)\n- 0.13628  v.72 - [DRW Add New Feature](https://www.kaggle.com/code/seowoohyeon/drw-add-new-feature/output) - [seowoohyeon](https://www.kaggle.com/seowoohyeon)\n- 0.13439  v.26 - [Small Improvements](https://www.kaggle.com/code/migrantworkerdatahub/small-improvements/notebook?scriptVersionId=249322532) - [Migrant Worker Data Hub](https://www.kaggle.com/migrantworkerdatahub)\n\n7-july-2025 - LB = [0.57389](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n \n\n- 0.84629  v.1 - [Improved Unshuffle 700 - Time Series Style](https://www.kaggle.com/code/taylorsamarel/improved-unshuffle-700-time-series-style) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n- 0.84585  v.2 - [Add more lags](https://www.kaggle.com/code/dmitriych/add-more-lags) - [Dmitriy Ch](https://www.kaggle.com/dmitriych)\n- 0.84443  v.1 - [Add More Lags - Improvement - Random State 700](https://www.kaggle.com/code/taylorsamarel/add-more-lags-improvement-random-state-700/notebook) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n\n\n9-july-2025 - LB = [0.84940](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n\n \n\n- 0.85944  v.2 - [Add more lags](https://www.kaggle.com/code/dmitriych/add-more-lags) - [Dmitriy Ch](https://www.kaggle.com/dmitriych)\n- 0.85472  v.1 - [Add More Lags](https://www.kaggle.com/code/taylorsamarel/add-more-lags/notebook) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n- 0.84629  v.1 - [Improved Unshuffle 700 - Time Series Style](https://www.kaggle.com/code/taylorsamarel/improved-unshuffle-700-time-series-style) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n\n\n9-july-2025 - LB = [The train data has been modified by the competition copyright holder](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n\n \n\n- 0.82968  v.11 - [DRW - Crypto Market Prediction | Turkish gambit](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-turkish-gambit/) - [riri](https://www.kaggle.com/nina2025)\n- 0.73799  v.13 - [REBOOT THE REBOOT 2.0](https://www.kaggle.com/code/taylorsamarel/reboot-the-reboot-2-0) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n- 0.72837  v.01 - [DRW-Ensemble-0.72837](https://www.kaggle.com/code/vishalpainjane/drw-ensemble-0-72837) - [Vishal Painjane](https://www.kaggle.com/vishalpainjane)\n- 0.70871  v.01 - [Anyone can win on Public LB](https://www.kaggle.com/code/johndoe2011/anyone-can-win-on-public-lb) - [ZULQAR.](https://www.kaggle.com/johndoe2011)\n\n14-july-2025 - LB = [0.82734](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n\n \n\n\n- 0.89178  v.13 - [REBOOT: SGD CONVERGENCE ++](https://www.kaggle.com/code/taylorsamarel/reboot-sgd-convergence/notebook) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n- 0.86767  v.01 - [DRW-Ensembling-0.86767](https://www.kaggle.com/code/vishalpainjane/drw-ensemble-0-72837) - [Vishal Painjane](https://www.kaggle.com/vishalpainjane)\n- 0.83975  v.07 - [REBOOT: Use the lead values](https://www.kaggle.com/code/gromml/reboot-use-the-lead-values/notebook) - [gromml](https://www.kaggle.com/gromml)\n\n15-july-2025 - LB = [0.93598](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n \n\n- 0.90038  v.03 - [XGB + Deep Learning Ensemble](https://www.kaggle.com/code/taylorsamarel/xgb-deep-learning-ensemble) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n- 0.89178  v.13 - [REBOOT: SGD CONVERGENCE ++](https://www.kaggle.com/code/taylorsamarel/reboot-sgd-convergence/notebook) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n- 0.88377  v.09 - [REBOOT: Use the lead values - IMPROVED](https://www.kaggle.com/code/taylorsamarel/reboot-use-the-lead-values-improved) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n\n\n15-july-2025 - LB = [?](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n \n\n- 0.82968  v.11 - [DRW - Crypto Market Prediction | Turkish gambit](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-turkish-gambit/) - [riri](https://www.kaggle.com/nina2025)\n- 0.73799  v.13 - [REBOOT THE REBOOT 2.0](https://www.kaggle.com/code/taylorsamarel/reboot-the-reboot-2-0) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n- 0.72837  v.01 - [DRW-Ensemble-0.72837](https://www.kaggle.com/code/vishalpainjane/drw-ensemble-0-72837) - [Vishal Painjane](https://www.kaggle.com/vishalpainjane)\n- 0.70871  v.01 - [Anyone can win on Public LB](https://www.kaggle.com/code/johndoe2011/anyone-can-win-on-public-lb) - [ZULQAR.](https://www.kaggle.com/johndoe2011)\n\n14-july-2025 - LB = [0.82734](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n\n \n\n\n- 0.89178  v.13 - [REBOOT: SGD CONVERGENCE ++](https://www.kaggle.com/code/taylorsamarel/reboot-sgd-convergence/notebook) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamarel)\n- 0.86767  v.01 - [DRW-Ensembling-0.86767](https://www.kaggle.com/code/vishalpainjane/drw-ensemble-0-72837) - [Vishal Painjane](https://www.kaggle.com/vishalpainjane)\n- 0.83975  v.07 - [REBOOT: Use the lead values](https://www.kaggle.com/code/gromml/reboot-use-the-lead-values/notebook) - [gromml](https://www.kaggle.com/gromml)\n\n15-july-2025 - LB = [0.93598](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)\n\n \n\n- 0.90038  v.03 - [XGB + Deep Learning Ensemble](https://www.kaggle.com/code/taylorsamarel/xgb-deep-learning-ensemble) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n- 0.89178  v.13 - [REBOOT: SGD CONVERGENCE ++](https://www.kaggle.com/code/taylorsamarel/reboot-sgd-convergence/notebook) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n- 0.88377  v.09 - [REBOOT: Use the lead values - IMPROVED](https://www.kaggle.com/code/taylorsamarel/reboot-use-the-lead-values-improved) - [Taylor S. Amarel](https://www.kaggle.com/taylorsamare)\n\n\n15-july-2025 - LB = [0.90174](https://www.kaggle.com/code/nina2025/drw-crypto-market-prediction-iblend)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"jupyter":{"source_hidden":true}}},{"cell_type":"code","source":"import pandas as pd","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-15T14:32:51.670661Z","iopub.execute_input":"2025-07-15T14:32:51.670983Z","iopub.status.idle":"2025-07-15T14:32:53.996140Z","shell.execute_reply.started":"2025-07-15T14:32:51.670938Z","shell.execute_reply":"2025-07-15T14:32:53.995046Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def iBlend(path_to_ds, file_short_names, sls):\n\n    def tida(sls):\n        \n        def read_subm(sls,i):\n            tnm = sls[\"subm\"][i][\"name\"]\n            FiN = sls[\"path\"] + tnm + \".csv\"\n            return pd.read_csv(FiN).rename(columns={'target':tnm, sls[\"target\"]:tnm})\n        \n        dfs_subm = [read_subm(sls,i) for i in range(len(sls[\"subm\"]))]\n        df_subms = pd.merge(dfs_subm[0],  dfs_subm[1], on=['ID'])\n        \n        for i in range(2, len(sls[\"subm\"])): \n            df_subms = pd.merge(df_subms, dfs_subm[i], on=['ID'])\n            \n        cols = [col for col in df_subms.columns if col != \"ID\"]\n        short_name_cols = [c.replace(sls[\"prefix\"], '') for c in cols]\n        corrects = [wt for wt in sls[\"subwts\"]]\n        weights = [subm['weight'] for subm in sls[\"subm\"]]\n        \n        def alls(x, cs=cols):\n            tes = {c: x[c] for c in cs}.items()\n            subms_sorted = [\n              t[0].replace(sls[\"prefix\"], '')\n              for t in sorted(tes,key=lambda k:k[1],reverse=True if sls[\"sort\"]=='desc' else False)]\n            return subms_sorted\n        \n        def correct(x, cs=cols, w=weights, cw=corrects):\n            ic = [x['alls'].index(c) for c in short_name_cols]\n            cS = [x[cols[j]] * (w[j] + cw[ic[j]]) for j in range(len(cols))]\n            return sum(cS)\n        \n        df_subms['alls']        = df_subms.apply(lambda x: alls   (x), axis=1)\n        df_subms[sls[\"target\"]] = df_subms.apply(lambda x: correct(x), axis=1)\n        \n        schema_rename = { old_nc:new_shnc for old_nc, new_shnc in zip(cols, short_name_cols) }\n        \n        df_subms = df_subms.rename(columns=schema_rename)\n        df_subms = df_subms.rename(columns={sls[\"target\"]:\"ensemble\"})\n        \n        df_subms.insert(loc=1, column=' _ ', value=['   '] * sls[\"q_rows\"])\n        \n        df_subms[' _ '] = df_subms[' _ '].astype(str)\n        pd.set_option('display.max_rows',100)\n        pd.set_option('display.float_format', '{:.4f}'.format)\n        vcols = ['ID'] + [' _ '] + short_name_cols + [' _ '] + ['alls'] + [' _ '] + ['ensemble']\n        df_subms = df_subms[vcols]\n        display(df_subms.head(7))\n        pd.set_option('display.float_format', '{:.7f}'.format)\n        df_subms = df_subms.rename(columns={\"ensemble\":sls[\"target\"]})\n        \n        return df_subms\n        \n\n    sample_subm = pd.read_csv(path_to_ds + file_short_names[1] + \".csv\")\n\n    \n    def ensemble_tida(sls,submission=sample_subm):   \n        sls['sort'] = 'desc'\n        dfs = tida(sls)\n        dfD = dfs[['ID', sls['target']]]\n        dfD.to_csv(f'tida_desc.csv', index=False)\n        sls['sort'] = 'asc'\n        dfs = tida(sls)\n        dfA = dfs[['ID', sls['target']]]\n        dfA.to_csv(f'tida_asc.csv',  index=False)\n        target,d,a = sls['target'],sls['desc'],sls['asc']\n        submission[target] = dfD[target] * d + a * dfA[target]\n        return submission\n\n    submission = ensemble_tida(sls)\n    \n    return submission","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-15T14:32:58.632050Z","iopub.execute_input":"2025-07-15T14:32:58.632359Z","iopub.status.idle":"2025-07-15T14:32:58.648385Z","shell.execute_reply.started":"2025-07-15T14:32:58.632333Z","shell.execute_reply":"2025-07-15T14:32:58.647481Z"},"_kg_hide-input":false},"outputs":[],"execution_count":null},{"cell_type":"code","source":"path_to_ds ='/kaggle/input/15-juli-2025-drw/submission '\n\nfile_short_names = ['0.83975','0.86767','0.89178']\n\nparams = {\n      'path'  : path_to_ds,                                 \n      'sort'  : \"dynamic\",\n      'target': \"prediction\",\n      'q_rows': 538_150,\n      'prefix': \"subm_\",\n      'desc'  : 0.30,\n      'asc'   : 0.70,\n      'subwts': [+1.00, -0.40, -0.60],                          # LB = 0.93598\n      'subm'  : [\n        { 'name':file_short_names[0],'weight':0.40, },\n        { 'name':file_short_names[1],'weight':0.60, },\n        { 'name':file_short_names[2],'weight':1.00, },\n      ]\n    }\n\n\npath_to_ds ='/kaggle/input/15-juli-2025-drw/submission '\n\nfile_short_names = ['0.88377','0.89178','0.90038']\n\nparams = {\n      'path'  : path_to_ds,                                 \n      'sort'  : \"dynamic\",\n      'target': \"prediction\",\n      'q_rows': 538_150,\n      'prefix': \"subm_\",\n      'desc'  : 0.45,\n      'asc'   : 0.55,\n      'subwts': [+0.55, -0.20, -0.35],                          # LB = 0.89864\n      'subm'  : [\n        { 'name':file_short_names[0],'weight':0.27, },\n        { 'name':file_short_names[1],'weight':0.33, },\n        { 'name':file_short_names[2],'weight':0.40, },\n      ]\n    }\n\n\npath_to_ds ='/kaggle/input/15-juli-2025-drw/submission '\n\nfile_short_names = ['0.88377','0.89178','0.90038']\n\nparams = {\n      'path'  : path_to_ds,                                 \n      'sort'  : \"dynamic\",\n      'target': \"prediction\",\n      'q_rows': 538_150,\n      'prefix': \"subm_\",\n      'desc'  : 0.20,\n      'asc'   : 0.80,\n      'subwts': [+0.50, -0.20, -0.30],                          # LB = 0.90174\n      'subm'  : [\n        { 'name':file_short_names[0],'weight':0.20, },\n        { 'name':file_short_names[1],'weight':0.30, },\n        { 'name':file_short_names[2],'weight':0.50, },\n      ]\n    }\n\n\npath_to_ds ='/kaggle/input/15-juli-2025-drw/submission '\n\nfile_short_names = ['0.83975','0.86767','0.88377','0.89178','0.90038']\n\nparams = {\n      'path'  : path_to_ds,                                 \n      'sort'  : \"dynamic\",\n      'target': \"prediction\",\n      'q_rows': 538_150,\n      'prefix': \"subm_\",\n      'desc'  : 0.30,\n      'asc'   : 0.70,\n      'subwts': [+0.40, +0.05, -0.05,-0.15,-0.25],              # LB = 0.93794\n      'subm'  : [\n         { 'name':file_short_names[0],'weight':0.20, },\n         { 'name':file_short_names[1],'weight':0.30, },\n         { 'name':file_short_names[2],'weight':0.01, },\n         { 'name':file_short_names[3],'weight':0.47, },\n         { 'name':file_short_names[4],'weight':0.02, },\n      ]\n    }\n\npath_to_ds ='/kaggle/input/15-juli-2025-drw/submission '\n\nfile_short_names = ['0.83975','0.86767','0.88377','0.89178','0.90038']\n\nparams = {\n      'path'  : path_to_ds,                                 \n      'sort'  : \"dynamic\",\n      'target': \"prediction\",\n      'q_rows': 538_150,\n      'prefix': \"subm_\",\n      'desc'  : 0.30,\n      'asc'   : 0.70,\n      'subwts': [+0.20, +0.10, -0.05,-0.10,-0.15],              # LB = \n      'subm'  : [\n         { 'name':file_short_names[0],'weight':0.20, },\n         { 'name':file_short_names[1],'weight':0.20, },\n         { 'name':file_short_names[2],'weight':0.20, },\n         { 'name':file_short_names[3],'weight':0.20, },\n         { 'name':file_short_names[4],'weight':0.20, },\n      ]\n    }","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-15T15:06:04.787393Z","iopub.execute_input":"2025-07-15T15:06:04.787772Z","iopub.status.idle":"2025-07-15T15:06:04.798299Z","shell.execute_reply.started":"2025-07-15T15:06:04.787743Z","shell.execute_reply":"2025-07-15T15:06:04.797434Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df = iBlend ( path_to_ds, file_short_names, params )\n\ndf.to_csv('submission.csv', index=False)\n\ndisplay(df)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-15T15:06:11.875315Z","iopub.execute_input":"2025-07-15T15:06:11.875585Z","iopub.status.idle":"2025-07-15T15:07:02.200889Z","shell.execute_reply.started":"2025-07-15T15:06:11.875566Z","shell.execute_reply":"2025-07-15T15:07:02.199980Z"}},"outputs":[],"execution_count":null}]}