{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🤝 Intro","metadata":{}},{"cell_type":"markdown","source":"Ensemble Learning is the technique of training many Machine Learning models and aggregating their results via an aggregation metric.  \nToday we'll aggregate the results of the top performing notebooks via the arithmetic mean.\n\n$$ \\hat{y} = \\frac{1}{n}\\sum_{i=1}^n y_i $$","metadata":{}},{"cell_type":"markdown","source":"![wisdom of crowds](https://i.ytimg.com/vi/enMCFep1s-4/maxresdefault.jpg)","metadata":{}},{"cell_type":"code","source":"import pandas as pd","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:13:02.929826Z","iopub.execute_input":"2022-10-11T12:13:02.930258Z","iopub.status.idle":"2022-10-11T12:13:02.959586Z","shell.execute_reply.started":"2022-10-11T12:13:02.930173Z","shell.execute_reply":"2022-10-11T12:13:02.958303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 👑 Top Submissions","metadata":{}},{"cell_type":"markdown","source":"We'll be taking the top performing notebooks available right now (that aren't ensemble notebooks themselves) as our individual model submissions.  \nPlease check out their notebooks as well, they are quite well written and diverse!","metadata":{}},{"cell_type":"markdown","source":"## Model 1","metadata":{}},{"cell_type":"markdown","source":"Notebook link: https://www.kaggle.com/code/paddykb/tps-2022-10-fastai  \nShoutout [paddykb](https://www.kaggle.com/paddykb)!","metadata":{}},{"cell_type":"code","source":"sub1 = pd.read_csv('../input/tps-2022-10-fastai/model_fastai_v3.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:14:17.145676Z","iopub.execute_input":"2022-10-11T12:14:17.146102Z","iopub.status.idle":"2022-10-11T12:14:17.456925Z","shell.execute_reply.started":"2022-10-11T12:14:17.146069Z","shell.execute_reply":"2022-10-11T12:14:17.455600Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub1","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:14:17.517706Z","iopub.execute_input":"2022-10-11T12:14:17.518631Z","iopub.status.idle":"2022-10-11T12:14:17.534931Z","shell.execute_reply.started":"2022-10-11T12:14:17.518591Z","shell.execute_reply":"2022-10-11T12:14:17.533665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model 2","metadata":{}},{"cell_type":"markdown","source":"Notebook link: https://www.kaggle.com/code/ahmedelfazouan/tabular-playground-catb-inference  \nShoutout [ahmedelfazouan](https://www.kaggle.com/ahmedelfazouan)!","metadata":{}},{"cell_type":"code","source":"sub2 = pd.read_csv('../input/tabular-playground-catb-inference/submission.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:14:52.801059Z","iopub.execute_input":"2022-10-11T12:14:52.801436Z","iopub.status.idle":"2022-10-11T12:14:53.621745Z","shell.execute_reply.started":"2022-10-11T12:14:52.801407Z","shell.execute_reply":"2022-10-11T12:14:53.620460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model 3","metadata":{}},{"cell_type":"markdown","source":"Notebook link: https://www.kaggle.com/code/hasanbasriakcay/tpsoct22-insightful-eda-fe-modeling  \nShoutout [hasanbasriakcay](https://www.kaggle.com/hasanbasriakcay)!","metadata":{}},{"cell_type":"code","source":"sub3 = pd.read_csv('../input/tpsoct22-insightful-eda-fe-modeling/final_lgbm_5fold.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:14:53.733143Z","iopub.execute_input":"2022-10-11T12:14:53.733904Z","iopub.status.idle":"2022-10-11T12:14:54.426101Z","shell.execute_reply.started":"2022-10-11T12:14:53.733865Z","shell.execute_reply":"2022-10-11T12:14:54.424997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model 4","metadata":{}},{"cell_type":"markdown","source":"Notebook link: https://www.kaggle.com/code/chazzer/rocket-league-xgboost-feat-engineering-cv  \nShoutout [chazzer](https://www.kaggle.com/chazzer)! (me lol)","metadata":{}},{"cell_type":"code","source":"sub4 = pd.read_csv('../input/rocket-league-xgboost-feat-engineering-cv/submission.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:14:54.890591Z","iopub.execute_input":"2022-10-11T12:14:54.891833Z","iopub.status.idle":"2022-10-11T12:14:55.548558Z","shell.execute_reply.started":"2022-10-11T12:14:54.891785Z","shell.execute_reply":"2022-10-11T12:14:55.547162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model 5","metadata":{}},{"cell_type":"markdown","source":"Notebook link: https://www.kaggle.com/code/infrarosso/tps-oct-2022-eda-hybrid-model-ensemble  \nShoutout [infrarosso](https://www.kaggle.com/infrarosso)!","metadata":{}},{"cell_type":"code","source":"sub5 = pd.read_csv('../input/tps-oct-2022-eda-hybrid-model-ensemble/submission_lgbm.csv')","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:14:57.388835Z","iopub.execute_input":"2022-10-11T12:14:57.389213Z","iopub.status.idle":"2022-10-11T12:14:58.042456Z","shell.execute_reply.started":"2022-10-11T12:14:57.389181Z","shell.execute_reply":"2022-10-11T12:14:58.041620Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🧠 Aggregate","metadata":{}},{"cell_type":"code","source":"sub = pd.DataFrame(columns=[\"id\", \"team_A_scoring_within_10sec\", \"team_B_scoring_within_10sec\"])\n\nsub[\"id\"] = sub1[\"id\"]\n\nsub[\"team_A_scoring_within_10sec\"] = sub1[\"team_A_scoring_within_10sec\"] + sub2[\"team_A_scoring_within_10sec\"] + sub3[\"team_A_scoring_within_10sec\"]\nsub[\"team_A_scoring_within_10sec\"] += sub4[\"team_A_scoring_within_10sec\"] + sub5[\"team_A_scoring_within_10sec\"]\nsub[\"team_A_scoring_within_10sec\"] /= 5\n\nsub[\"team_B_scoring_within_10sec\"] = sub1[\"team_B_scoring_within_10sec\"] + sub2[\"team_B_scoring_within_10sec\"] + sub3[\"team_B_scoring_within_10sec\"]\nsub[\"team_B_scoring_within_10sec\"] += sub4[\"team_B_scoring_within_10sec\"] + sub5[\"team_B_scoring_within_10sec\"]\nsub[\"team_B_scoring_within_10sec\"] /= 5","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:20:02.414709Z","iopub.execute_input":"2022-10-11T12:20:02.415118Z","iopub.status.idle":"2022-10-11T12:20:02.603504Z","shell.execute_reply.started":"2022-10-11T12:20:02.415087Z","shell.execute_reply":"2022-10-11T12:20:02.602527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:20:11.480726Z","iopub.execute_input":"2022-10-11T12:20:11.481191Z","iopub.status.idle":"2022-10-11T12:20:11.496303Z","shell.execute_reply.started":"2022-10-11T12:20:11.481157Z","shell.execute_reply":"2022-10-11T12:20:11.495051Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 💌 Submission","metadata":{}},{"cell_type":"code","source":"sub.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-10-11T12:20:39.592966Z","iopub.execute_input":"2022-10-11T12:20:39.593480Z","iopub.status.idle":"2022-10-11T12:20:42.243502Z","shell.execute_reply.started":"2022-10-11T12:20:39.593434Z","shell.execute_reply":"2022-10-11T12:20:42.242620Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 👋 Conclusion","metadata":{}},{"cell_type":"markdown","source":"A simple technique, but a quite powerful one.  \nPlease visit all of the notebooks that were used for this ensemble!","metadata":{}}]}