This is a long and hard competition! I'd like to congratulate all the teams who stick to it till the end. I am looking forward to the sharing of winning solutions.
\nMy main goal of this competition is to experiment with Rapids.ai tools to accelerate the pre-processing and post-processing of deep learning models.
\nPre-processing:
\ndask to process raw data, convert them to dataframes and save them as parquets.dask-cudf to engineer features from many small parquets. cuml LabelEncoder, TargetEncoder, Nearest Neighbor and Xgboost to create simple models and explore the dataset.Model:
\nI built two RNNs using PyTorch Lightning:
delta x & y. similar to Olaf's approachPost-processing:
\nThe approach is to interleave cost minimization, snap to corridor and snap to grid and run them iteratively. Hyperparameters such as alpha in cost minimization, threshold in snap grid are tuned for each iteration. I rewrote these functions with cupy and cudf and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes. I didn't do the exact measurements but that could be 100x faster than the original implementation. The speedup is very important for searching for the best hyperparameters. The post-processing improves the score by 1.1~1.2, which is quite significant.
Other notes:
\nI didn't use the start/end points leak and the hand-labeled waypoints. I spent a lot of time creating an end-to-end neural network that incorporates cost minimization and snap to grids in the training process but it didn't go anywhere in the end. I'm looking forward to the approach of the 1st place team who successfully built an end-to-end model.
Unfortunately, I am way behind in my other workloads so I apologize that my solution and source code will be shared later when I find the time. I'll update this thread when it's done.
", "messages": [ { "id": "1312356", "postDate": "05/18/2021 02:19:51", "content": "This is a long and hard competition! I'd like to congratulate all the teams who stick to it till the end. I am looking forward to the sharing of winning solutions.
\nMy main goal of this competition is to experiment with Rapids.ai tools to accelerate the pre-processing and post-processing of deep learning models.
\nPre-processing:
\ndask to process raw data, convert them to dataframes and save them as parquets.dask-cudf to engineer features from many small parquets. cuml LabelEncoder, TargetEncoder, Nearest Neighbor and Xgboost to create simple models and explore the dataset.Model:
\nI built two RNNs using PyTorch Lightning:
delta x & y. similar to Olaf's approachPost-processing:
\nThe approach is to interleave cost minimization, snap to corridor and snap to grid and run them iteratively. Hyperparameters such as alpha in cost minimization, threshold in snap grid are tuned for each iteration. I rewrote these functions with cupy and cudf and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes. I didn't do the exact measurements but that could be 100x faster than the original implementation. The speedup is very important for searching for the best hyperparameters. The post-processing improves the score by 1.1~1.2, which is quite significant.
Other notes:
\nI didn't use the start/end points leak and the hand-labeled waypoints. I spent a lot of time creating an end-to-end neural network that incorporates cost minimization and snap to grids in the training process but it didn't go anywhere in the end. I'm looking forward to the approach of the 1st place team who successfully built an end-to-end model.
Unfortunately, I am way behind in my other workloads so I apologize that my solution and source code will be shared later when I find the time. I'll update this thread when it's done.
", "rawMarkdown": "This is a long and hard competition! I'd like to congratulate all the teams who stick to it till the end. I am looking forward to the sharing of winning solutions.\n\nMy main goal of this competition is to experiment with [Rapids.ai](https://rapids.ai/start.html) tools to accelerate the pre-processing and post-processing of deep learning models. \n\nPre-processing:\n- use `dask` to process raw data, convert them to dataframes and save them as `parquets`.\n- use `dask-cudf` to engineer features from many small `parquets`. \n- use `cuml LabelEncoder, TargetEncoder, Nearest Neighbor` and `Xgboost` to create simple models and explore the dataset.\n\nModel:\nI built two RNNs using PyTorch Lightning:\n- use wifi features to predict the waypoints directly. \n- use IMU features to predict the shift of waypoints, `delta x & y`. similar to [Olaf's approach](https://www.kaggle.com/c/indoor-location-navigation/discussion/239884)\n\nPost-processing:\nThe approach is to interleave [cost minimization](https://www.kaggle.com/saitodevel01/indoor-post-processing-by-cost-minimization), [snap to corridor] (https://www.kaggle.com/rafaelcartenet/scaled-floors-geojsons-new-dataset) and [snap to grid](https://www.kaggle.com/robikscube/indoor-navigation-snap-to-grid-post-processing) and run them iteratively. Hyperparameters such as `alpha` in `cost minimization`, `threshold` in `snap grid` are tuned for each iteration. I rewrote these functions with `cupy` and `cudf` and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes. I didn't do the exact measurements but that could be 100x faster than the original implementation. The speedup is very important for searching for the best hyperparameters. The post-processing improves the score by 1.1~1.2, which is quite significant.\n\nOther notes:\nI didn't use the start/end points leak and the hand-labeled waypoints. I spent a lot of time creating an end-to-end neural network that incorporates `cost minimization` and `snap to grids` in the training process but it didn't go anywhere in the end. I'm looking forward to the approach of the 1st place team who successfully built an end-to-end model.\n\nUnfortunately, I am way behind in my other workloads so I apologize that my solution and source code will be shared later when I find the time. I'll update this thread when it's done.", "votes": null }, { "id": "1314125", "postDate": "05/19/2021 01:03:11", "content": "Awesome work @jiweiliu - you were a strong competitor in the competition from day 1, it was fun seeing you consistently at the top of the leaderboard.
\n\n\nI rewrote these functions with cupy and cudf and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes.
\n
That is very impressive. Will you be sharing the code?
", "rawMarkdown": "Awesome work @jiweiliu - you were a strong competitor in the competition from day 1, it was fun seeing you consistently at the top of the leaderboard.\n\n> I rewrote these functions with cupy and cudf and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes.\n\nThat is very impressive. Will you be sharing the code?", "votes": null }, { "id": "1314538", "postDate": "05/19/2021 08:11:24", "content": "yes, I will. Please give me some time to clean the code (and I might be able to make it even faster) and measure the speedup. I'll update this thread :)
", "rawMarkdown": "yes, I will. Please give me some time to clean the code (and I might be able to make it even faster) and measure the speedup. I'll update this thread :)", "votes": null }, { "id": "1314541", "postDate": "05/19/2021 08:14:24", "content": "A quick tip is to avoid using python for loops as much as possible. For snap-to-grid we can calculate all-to-all distances using numpy/cupy broadcast in one shot. For cost-minimization, we can connect paths into one long sequence and set the beta at the boundaries to zero. I'd like to do some further optimization to remove for loops completely for these two functions.
Thank you again for posting the great kernel. It's elegant and so insightful.
", "rawMarkdown": "Thank you again for posting the great kernel. It's elegant and so insightful.", "votes": null } ], "comments": [ { "id": 1314125, "author_name": "robikscube", "author_url": "", "post_date": "05/19/2021 01:03:11", "content": "Awesome work @jiweiliu - you were a strong competitor in the competition from day 1, it was fun seeing you consistently at the top of the leaderboard.
\n\n\nI rewrote these functions with cupy and cudf and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes.
\n
That is very impressive. Will you be sharing the code?
", "votes": null, "replies": [ { "id": 1314538, "author_name": "jiweiliu", "author_url": "", "post_date": "05/19/2021 08:11:24", "content": "yes, I will. Please give me some time to clean the code (and I might be able to make it even faster) and measure the speedup. I'll update this thread :)
", "votes": null, "replies": [] }, { "id": 1314541, "author_name": "jiweiliu", "author_url": "", "post_date": "05/19/2021 08:14:24", "content": "A quick tip is to avoid using python for loops as much as possible. For snap-to-grid we can calculate all-to-all distances using numpy/cupy broadcast in one shot. For cost-minimization, we can connect paths into one long sequence and set the beta at the boundaries to zero. I'd like to do some further optimization to remove for loops completely for these two functions.
Thank you again for posting the great kernel. It's elegant and so insightful.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1312356": "This is a long and hard competition! I'd like to congratulate all the teams who stick to it till the end. I am looking forward to the sharing of winning solutions.\n\nMy main goal of this competition is to experiment with [Rapids.ai](https://rapids.ai/start.html) tools to accelerate the pre-processing and post-processing of deep learning models. \n\nPre-processing:\n- use `dask` to process raw data, convert them to dataframes and save them as `parquets`.\n- use `dask-cudf` to engineer features from many small `parquets`. \n- use `cuml LabelEncoder, TargetEncoder, Nearest Neighbor` and `Xgboost` to create simple models and explore the dataset.\n\nModel:\nI built two RNNs using PyTorch Lightning:\n- use wifi features to predict the waypoints directly. \n- use IMU features to predict the shift of waypoints, `delta x & y`. similar to [Olaf's approach](https://www.kaggle.com/c/indoor-location-navigation/discussion/239884)\n\nPost-processing:\nThe approach is to interleave [cost minimization](https://www.kaggle.com/saitodevel01/indoor-post-processing-by-cost-minimization), [snap to corridor] (https://www.kaggle.com/rafaelcartenet/scaled-floors-geojsons-new-dataset) and [snap to grid](https://www.kaggle.com/robikscube/indoor-navigation-snap-to-grid-post-processing) and run them iteratively. Hyperparameters such as `alpha` in `cost minimization`, `threshold` in `snap grid` are tuned for each iteration. I rewrote these functions with `cupy` and `cudf` and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes. I didn't do the exact measurements but that could be 100x faster than the original implementation. The speedup is very important for searching for the best hyperparameters. The post-processing improves the score by 1.1~1.2, which is quite significant.\n\nOther notes:\nI didn't use the start/end points leak and the hand-labeled waypoints. I spent a lot of time creating an end-to-end neural network that incorporates `cost minimization` and `snap to grids` in the training process but it didn't go anywhere in the end. I'm looking forward to the approach of the 1st place team who successfully built an end-to-end model.\n\nUnfortunately, I am way behind in my other workloads so I apologize that my solution and source code will be shared later when I find the time. I'll update this thread when it's done.", "1314125": "Awesome work @jiweiliu - you were a strong competitor in the competition from day 1, it was fun seeing you consistently at the top of the leaderboard.\n\n> I rewrote these functions with cupy and cudf and they were very fast. My best submission is done with 30 iterations and 90 post-processing functions in total, which only took less than 10 minutes.\n\nThat is very impressive. Will you be sharing the code?", "1314538": "yes, I will. Please give me some time to clean the code (and I might be able to make it even faster) and measure the speedup. I'll update this thread :)", "1314541": "A quick tip is to avoid using python `for` loops as much as possible. For snap-to-grid we can calculate all-to-all distances using numpy/cupy broadcast in one shot. For cost-minimization, we can connect paths into one long sequence and set the `beta` at the boundaries to zero. I'd like to do some further optimization to remove `for` loops completely for these two functions.", "1314544": "Thank you again for posting the great kernel. It's elegant and so insightful." }, "source": "meta" }