{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Combining Lyft Multimode Models"},{"metadata":{},"cell_type":"markdown","source":"### <font color=\"red\">Notice</font>"},{"metadata":{},"cell_type":"markdown","source":"> <b><font color=\"blue\">This notebook is intended to help you combining your internal models, not for public test data probing ! Please use it wisely.</font></b>"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# The main idea"},{"metadata":{},"cell_type":"markdown","source":"At the early beginning of this competition, we found that <font color=\"blue\">combining solutions from different single-mode models considerably ameliorates results</font>. Furthermore, the combining process was very easy when dealing with **single-mode** models we just have to take 3 models, one for each channel, and tune their probabilities accordingly.\n\nThings become harder when talking about **multi-mode models** where there is no obvious order between diffrent models predictions. We come accross this order absence issue by sorting the 3 channels such that the one with the highest probability will be the first one for all the models. Then, the channel with the second highest proba will be the second channel for all the models; same for the third channel. Hence, we introduce a somehow consistent order which allow to combine different models.\n\nAs the whole process was slow with **pure pandas**, we switch to **numpy** to speed up  everything.\n\nLet's see it in practice !"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**In the rest of my code, I will be using [this public dataset](https://www.kaggle.com/kneroma/lyft-best-performing-public-kernels) for demo only**"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd, numpy as np\npd.options.display.max_columns=305","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"paths = [\n#     \"../input/lyft-best-performing-public-kernels/lyft-ensembling-raster-sizes.csv\", \n#     \"../input/lyft-best-performing-public-kernels/lyft-prediction-with-multi-mode-confidence.csv\",\n    \"../input/lyft-complete-train-and-prediction-pipeline/submission.csv\", #42.285\n    \"../input/lyft-multi-mode-448px-inference/submission.csv\", #33.953\n]\nweights = [0.2, 0.8]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"conf_cols = np.array([\"conf_0\", \"conf_1\", \"conf_2\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"xy_cols = [[],[],[]]\nfor i in range(50):\n    for j in range(3):\n        xy_cols[j].append(f\"coord_x{j}{i}\")\n        xy_cols[j].append(f\"coord_y{j}{i}\")\nxy_cols[0][:10]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"COLUMNS = [\"timestamp\", \"track_id\"] + list(conf_cols) + xy_cols[0] + xy_cols[1] + xy_cols[2]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Sorting"},{"metadata":{"trusted":true},"cell_type":"code","source":"def sort_df(df, sort_timestamp_track_id=True):\n    \n    conf_orders = np.argsort(-df[conf_cols].values,1)\n    XY = np.stack([df[xy_cols[0]].values,df[xy_cols[1]].values, df[xy_cols[2]].values], axis=1)\n    XY = XY[np.arange(len(XY))[:, None], conf_orders]\n\n    df2 = pd.DataFrame(columns = COLUMNS)\n    df2[\"timestamp\"] = df[\"timestamp\"].values\n    df2[\"track_id\"] = df[\"track_id\"].values\n    df2[xy_cols[0] + xy_cols[1] + xy_cols[2]] = XY.reshape(-1,300)\n    df2[conf_cols] = df[conf_cols].values[np.arange(len(df))[:, None], conf_orders]\n    \n    if sort_timestamp_track_id:\n        df2.sort_values([\"timestamp\", \"track_id\"], inplace=True)\n        df2.reset_index(inplace=True, drop=True)\n    return df2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Combining"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\ndf = None\nfor path,w in zip(paths,weights):\n    print(w, path)\n    temp = pd.read_csv(path)\n    temp = sort_df(temp)\n    temp[COLUMNS[5:]] *= w\n    if df is None:\n        df = temp\n    else:\n        df[COLUMNS[2:]] += temp[COLUMNS[2:]]\ndf[conf_cols] /= df[conf_cols].sum(1).values[:, None]\n\nsample = pd.read_csv(\"../input/lyft-motion-prediction-autonomous-vehicles/multi_mode_sample_submission.csv\")\n\ndf = sample[[\"timestamp\", \"track_id\"]].merge(df, on=[\"timestamp\", \"track_id\"])\nsample.shape, df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.to_csv(\"submission.csv\", index=False, float_format='%.6f')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}