First and foremost, I would like to express my gratitude to the hosts who made significant efforts in organizing the competition, despite numerous challenges. Congratulations to all the winners! While my model does not come close to the top performers, I am sharing my solution, hoping that it can be of use to someone, as it is relatively simple.
\nThe central idea of my model revolves around the ensemble of raw log data processing using 1D-CNN and aggregated feature processing using GBDT. Individually, these models achieved CV=0.696 and Public LB=0.697, respectively. However, by ensembling them, I was able to improve the scores to CV=0.700, Public LB=0.700, and Private LB=0.700.
\nThe competition data provided consisted of gameplay logs, with several hundred to several thousand logs per session. Hence, I employed two modeling approaches: 1D-CNN, which directly extracts features from the temporal log sequences, and LightGBM, which utilizes aggregated features obtained through feature engineering. For 1D-CNN, I based my implementation on the public notebook by ABAOJIANG. As for feature engineering and LightGBM, I referred to the public notebook by ONELUX. I extend my gratitude to them for sharing their excellent notebooks.
\nRegarding 1D-CNN, I used the encoder part of the public notebook as the base. After performing feature extraction using 1D-CNN, I applied the Multi-Head Attention structure before conducting temporal aggregation. I utilized five input features: numerical features such as diff(elapsed_time) and log(elapsed_time), and categorical features such as event_name + name, room_fqid, and fqid + text_fqid.
\nFor LightGBM, I added several features to the ones presented in the public notebook, resulting in inputting over 2000 features. Most of the additional features were related to text_fqid, including total time spent displaying text for each fqid and the reading speed per word.
\nFurthermore, I combined a subset of these features (around 6) with 1D-CNN to create a new neural network model, which also had a positive effect (CV+0.001 approximately). I integrated these three models using linear regression-based stacking to generate the final predictions.
\nHowever, for the simplest questions (2, 3, 18), I did not perform any modeling and predicted all of them as 1. The inference time was cutting it close at around 9 hours (528 minutes), and I was quite nervous during the final submission lol.
\nHere is a list summarizing the experiments I conducted that did not yield successful results:
\nFirst and foremost, I would like to express my gratitude to the hosts who made significant efforts in organizing the competition, despite numerous challenges. Congratulations to all the winners! While my model does not come close to the top performers, I am sharing my solution, hoping that it can be of use to someone, as it is relatively simple.
\nThe central idea of my model revolves around the ensemble of raw log data processing using 1D-CNN and aggregated feature processing using GBDT. Individually, these models achieved CV=0.696 and Public LB=0.697, respectively. However, by ensembling them, I was able to improve the scores to CV=0.700, Public LB=0.700, and Private LB=0.700.
\nThe competition data provided consisted of gameplay logs, with several hundred to several thousand logs per session. Hence, I employed two modeling approaches: 1D-CNN, which directly extracts features from the temporal log sequences, and LightGBM, which utilizes aggregated features obtained through feature engineering. For 1D-CNN, I based my implementation on the public notebook by ABAOJIANG. As for feature engineering and LightGBM, I referred to the public notebook by ONELUX. I extend my gratitude to them for sharing their excellent notebooks.
\nRegarding 1D-CNN, I used the encoder part of the public notebook as the base. After performing feature extraction using 1D-CNN, I applied the Multi-Head Attention structure before conducting temporal aggregation. I utilized five input features: numerical features such as diff(elapsed_time) and log(elapsed_time), and categorical features such as event_name + name, room_fqid, and fqid + text_fqid.
\nFor LightGBM, I added several features to the ones presented in the public notebook, resulting in inputting over 2000 features. Most of the additional features were related to text_fqid, including total time spent displaying text for each fqid and the reading speed per word.
\nFurthermore, I combined a subset of these features (around 6) with 1D-CNN to create a new neural network model, which also had a positive effect (CV+0.001 approximately). I integrated these three models using linear regression-based stacking to generate the final predictions.
\nHowever, for the simplest questions (2, 3, 18), I did not perform any modeling and predicted all of them as 1. The inference time was cutting it close at around 9 hours (528 minutes), and I was quite nervous during the final submission lol.
\nHere is a list summarizing the experiments I conducted that did not yield successful results:
\nCongratulations @jinmiyashita on coming 54th 🎉🎉. Thanks for sharing ur approach and experience
", "rawMarkdown": "Congratulations @jinmiyashita on coming 54th 🎉🎉. Thanks for sharing ur approach and experience", "votes": null }, { "id": "2329629", "postDate": "07/04/2023 11:51:42", "content": "@jinmiyashita congrats with 54 place! Thanks for sharing this!
", "rawMarkdown": "jinmiyashita congrats with 54 place! Thanks for sharing this!", "votes": null }, { "id": "2330657", "postDate": "07/05/2023 06:10:29", "content": "Thank you!
", "rawMarkdown": "Thank you!", "votes": null }, { "id": "2330663", "postDate": "07/05/2023 06:11:57", "content": "Thank you so much!
", "rawMarkdown": "Thank you so much!", "votes": null } ], "comments": [ { "id": 2329244, "author_name": "swapnilchowdhury", "author_url": "", "post_date": "07/04/2023 06:57:34", "content": "Congratulations @jinmiyashita on coming 54th 🎉🎉. Thanks for sharing ur approach and experience
", "votes": null, "replies": [ { "id": 2330663, "author_name": "jinmiyashita", "author_url": "", "post_date": "07/05/2023 06:11:57", "content": "Thank you so much!
", "votes": null, "replies": [] } ] }, { "id": 2329629, "author_name": "serangu", "author_url": "", "post_date": "07/04/2023 11:51:42", "content": "@jinmiyashita congrats with 54 place! Thanks for sharing this!
", "votes": null, "replies": [ { "id": 2330657, "author_name": "jinmiyashita", "author_url": "", "post_date": "07/05/2023 06:10:29", "content": "Thank you!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2328816": "First and foremost, I would like to express my gratitude to the hosts who made significant efforts in organizing the competition, despite numerous challenges. Congratulations to all the winners! While my model does not come close to the top performers, I am sharing my solution, hoping that it can be of use to someone, as it is relatively simple.\n\n## Overview\nThe central idea of my model revolves around the ensemble of raw log data processing using 1D-CNN and aggregated feature processing using GBDT. Individually, these models achieved CV=0.696 and Public LB=0.697, respectively. However, by ensembling them, I was able to improve the scores to CV=0.700, Public LB=0.700, and Private LB=0.700.\n\n## Models\nThe competition data provided consisted of gameplay logs, with several hundred to several thousand logs per session. Hence, I employed two modeling approaches: 1D-CNN, which directly extracts features from the temporal log sequences, and LightGBM, which utilizes aggregated features obtained through feature engineering. For 1D-CNN, I based my implementation on the [public notebook by ABAOJIANG](https://www.kaggle.com/code/abaojiang/lb-0-694-tconv-with-4-features-training-part). As for feature engineering and LightGBM, I referred to the [public notebook by ONELUX](https://www.kaggle.com/code/leehomhuang/catboost-baseline-with-lots-features-inference). I extend my gratitude to them for sharing their excellent notebooks.\n\nRegarding 1D-CNN, I used the encoder part of the public notebook as the base. After performing feature extraction using 1D-CNN, I applied the Multi-Head Attention structure before conducting temporal aggregation. I utilized five input features: numerical features such as diff(elapsed_time) and log(elapsed_time), and categorical features such as event_name + name, room_fqid, and fqid + text_fqid.\n\nFor LightGBM, I added several features to the ones presented in the public notebook, resulting in inputting over 2000 features. Most of the additional features were related to text_fqid, including total time spent displaying text for each fqid and the reading speed per word.\n\nFurthermore, I combined a subset of these features (around 6) with 1D-CNN to create a new neural network model, which also had a positive effect (CV+0.001 approximately). I integrated these three models using linear regression-based stacking to generate the final predictions.\n\nHowever, for the simplest questions (2, 3, 18), I did not perform any modeling and predicted all of them as 1. The inference time was cutting it close at around 9 hours (528 minutes), and I was quite nervous during the final submission lol.\n\n## What Didn't Work\nHere is a list summarizing the experiments I conducted that did not yield successful results:\n- Building the encoder solely using Transformers\n- Utilizing state-of-the-art time series neural networks like [Patch TST](https://github.com/timeseriesAI/tsai/blob/main/tsai/models/PatchTST.py) or [SCINet](https://github.com/cure-lab/SCINet)\n- Including CatBoost in the ensemble\n- Handling all the problems with a single model\n- TabNet\n- Merging [additional data](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/412098)\n- DAE (Denoising Autoencoder)\n- Applying an anomaly detection model to the simplest questions (2, 3, 18)\n- Using LightGBM with the latent features of the neural network", "2329244": "Congratulations @jinmiyashita on coming 54th 🎉🎉. Thanks for sharing ur approach and experience", "2329629": "jinmiyashita congrats with 54 place! Thanks for sharing this!", "2330657": "Thank you!", "2330663": "Thank you so much!" }, "source": "meta" }