![\"event-aware-tconvb.png\"](\"https://i.postimg.cc/MT5H2Zg9/event-aware-tconvb.png\")
Hi everyone,
\nBefore diving into the manual feature engineering, I'm interested in seeing how far a DL-based model can go with a small set of features. After running experiments day and night, I temporarily can achieve CV 0.6914 and LB 0.694 with only four features, including:
\nelapsed_timeevent_namenameroom_fqidFirst, event_name is combined with name to form a new feature event_comb (i.e., event combination). Then, I treat the difference of elapsed_time as the only numeric feature and the others categorical. The motivation behind the scene is that I hope the model can capture event-aware temporal patterns; that is, each time difference value is enriched by the event information and where the event take places. Finally, the model achieves the performance summarized as follows:
| CV (GroupKFold with k=5) | \nHoldout (Released Old Test Set) | \nLB | \n
|---|---|---|
| 0.6914 | \n0.6911 | \n0.694 | \n
To be honest, I think I'm not good at model building and DL tuning, but I still want to share my experience and results I get so far. What's more, we should be cautious about the CV / LB gap to avoid overfitting on LB. I hope this can inspire you to build a more robust model!
\nThe inference part is here.
\nThe training part remains a work in progress and will be published within 2 ~ 3 days.
Thanks a lot!
", "messages": [ { "id": "2203292", "postDate": "03/30/2023 17:07:44", "content": "Hi everyone,
\nBefore diving into the manual feature engineering, I'm interested in seeing how far a DL-based model can go with a small set of features. After running experiments day and night, I temporarily can achieve CV 0.6914 and LB 0.694 with only four features, including:
\nelapsed_timeevent_namenameroom_fqidFirst, event_name is combined with name to form a new feature event_comb (i.e., event combination). Then, I treat the difference of elapsed_time as the only numeric feature and the others categorical. The motivation behind the scene is that I hope the model can capture event-aware temporal patterns; that is, each time difference value is enriched by the event information and where the event take places. Finally, the model achieves the performance summarized as follows:
| CV (GroupKFold with k=5) | \nHoldout (Released Old Test Set) | \nLB | \n
|---|---|---|
| 0.6914 | \n0.6911 | \n0.694 | \n
To be honest, I think I'm not good at model building and DL tuning, but I still want to share my experience and results I get so far. What's more, we should be cautious about the CV / LB gap to avoid overfitting on LB. I hope this can inspire you to build a more robust model!
\nThe inference part is here.
\nThe training part remains a work in progress and will be published within 2 ~ 3 days.
Thanks a lot!
", "rawMarkdown": "Hi everyone,\n\nBefore diving into the manual feature engineering, I'm interested in seeing how far a DL-based model can go with a small set of features. After running experiments day and night, I temporarily can achieve **CV 0.6914** and LB 0.694 with only four features, including:\n\n1. Difference of `elapsed_time`\n2. `event_name`\n3. `name`\n4. `room_fqid`\n\nFirst, `event_name` is combined with `name` to form a new feature `event_comb` (i.e., event combination). Then, I treat the difference of `elapsed_time` as the only numeric feature and the others categorical. The motivation behind the scene is that I hope the model can capture **event-aware temporal patterns**; that is, each time difference value is **enriched by the event information and where the event take places**. Finally, the model achieves the performance summarized as follows:\n\n| CV (GroupKFold with k=5) | Holdout (Released Old Test Set) | LB |\n| ------------------------ | ------------------------------- | ----- |\n| 0.6914 | 0.6911 | 0.694 |\n\nTo be honest, I think I'm not good at model building and DL tuning, but I still want to share my experience and results I get so far. What's more, we should be cautious about the CV / LB gap to avoid overfitting on LB. I hope this can inspire you to build a more robust model!\n\nThe inference part is [here](https://www.kaggle.com/code/abaojiang/lb-0-694-event-aware-tconv-with-only-4-features).\nThe training part remains a work in progress and will be published within 2 ~ 3 days.\n\nThanks a lot!", "votes": null }, { "id": "2203305", "postDate": "03/30/2023 17:27:48", "content": "Very nice. How long does inference take when you submit (roughly)?
", "rawMarkdown": "Very nice. How long does inference take when you submit (roughly)?", "votes": null }, { "id": "2203452", "postDate": "03/30/2023 20:04:05", "content": "Great DL model. Well done
", "rawMarkdown": "Great DL model. Well done", "votes": null }, { "id": "2203492", "postDate": "03/30/2023 21:21:10", "content": "Nice .. thanks will be good to add this if works into an ensemble with a large feature rich GBM 😸 .
", "rawMarkdown": "Nice .. thanks will be good to add this if works into an ensemble with a large feature rich GBM 😸 .", "votes": null }, { "id": "2203630", "postDate": "03/31/2023 02:00:43", "content": "Nice! Combining event_name and name might be a good approach. name does not provide many information alone.
Thanks for sharing! What NN type did you use (suppose I can't wait for the training notebook haha)?
", "rawMarkdown": "Thanks for sharing! What NN type did you use (suppose I can't wait for the training notebook haha)?", "votes": null }, { "id": "2203693", "postDate": "03/31/2023 03:56:14", "content": "Hi @cpmpml,
\nIt takes roughly 65 minutes to finish scoring (including data processing, simple feature engineering and model inference). And, there are 15 models in total (i.e., 5-fold models for each level_group).
Hope this helps, thanks a lot!
", "rawMarkdown": "Hi @cpmpml,\n\nIt takes roughly 65 minutes to finish scoring (including data processing, simple feature engineering and model inference). And, there are 15 models in total (*i.e.*, 5-fold models for each `level_group`). \n\nHope this helps, thanks a lot!", "votes": null }, { "id": "2203697", "postDate": "03/31/2023 03:58:19", "content": "Hi @cdeotte,
\nThanks for your appreciation, I'll add more features to see whether DL-based models can catch up with tree-based models or not!
", "rawMarkdown": "Hi @cdeotte,\n\nThanks for your appreciation, I'll add more features to see whether DL-based models can catch up with tree-based models or not!", "votes": null }, { "id": "2203703", "postDate": "03/31/2023 04:00:31", "content": "Hi @gauravbrills,
\nThanks for the comment. I'm also curious about how an ensemble of DL-based model and GBM can perform. Let's keep going!
", "rawMarkdown": "Hi @gauravbrills,\n\nThanks for the comment. I'm also curious about how an ensemble of DL-based model and GBM can perform. Let's keep going!", "votes": null }, { "id": "2203722", "postDate": "03/31/2023 04:25:29", "content": "Hi @chrisqiu,
\nI think name can provide information about the status of the corresponding event. Also, there exist only 19 combinations of event_name (11 unique values) and name (6 unique values), not 66. Hence, training only one embedding table with the combination might be a little bit better than training two separate ones.
I learned a lot from your notebook! Thanks for your sharing.
", "rawMarkdown": "Hi @chrisqiu,\n\nI think `name` can provide information about the **status** of the corresponding event. Also, there exist only 19 combinations of `event_name` (11 unique values) and `name` (6 unique values), not 66. Hence, training only one embedding table with the combination might be a little bit better than training two separate ones.\n\nI learned a lot from your notebook! Thanks for your sharing.", "votes": null }, { "id": "2203746", "postDate": "03/31/2023 05:03:35", "content": "Hi @hoangnguyen719,
\nI plot an overview of the model architecture shown as follows,
\n
Hope this can help you understand. I'll publish the training part asap😂. Thanks a lot!
", "rawMarkdown": "Hi @hoangnguyen719,\n\nI plot an overview of the model architecture shown as follows, \n\n[](https://postimg.cc/R634wmSt)\n\nHope this can help you understand. I'll publish the training part asap😂. Thanks a lot!", "votes": null }, { "id": "2205073", "postDate": "04/01/2023 08:15:18", "content": "hi, do the difference of the elapsed time here means between 2 sucessive events based on the index?
", "rawMarkdown": "hi, do the difference of the elapsed time here means between 2 sucessive events based on the index?", "votes": null }, { "id": "2205449", "postDate": "04/01/2023 15:10:48", "content": "Hi @narendra,
\nThat's true, just take the difference of elapsed_time between two successive events. However, please take care that index itself has some data errors confirmed by the competition host here. Thanks!
Hi @abaojiang,
\nThanks for sharing this notebook.
\nI am facing an issue. I am not able to get the pickle file. There is nothing at the path /kaggle/input/cat2code-v2/cat2code. Are others also facing the same issue? Could you please help me in finding out how to get the pickle file?
Very nice. How long does inference take when you submit (roughly)?
", "votes": null, "replies": [ { "id": 2203693, "author_name": "abaojiang", "author_url": "", "post_date": "03/31/2023 03:56:14", "content": "Hi @cpmpml,
\nIt takes roughly 65 minutes to finish scoring (including data processing, simple feature engineering and model inference). And, there are 15 models in total (i.e., 5-fold models for each level_group).
Hope this helps, thanks a lot!
", "votes": null, "replies": [] } ] }, { "id": 2203452, "author_name": "cdeotte", "author_url": "", "post_date": "03/30/2023 20:04:05", "content": "Great DL model. Well done
", "votes": null, "replies": [ { "id": 2203697, "author_name": "abaojiang", "author_url": "", "post_date": "03/31/2023 03:58:19", "content": "Hi @cdeotte,
\nThanks for your appreciation, I'll add more features to see whether DL-based models can catch up with tree-based models or not!
", "votes": null, "replies": [] } ] }, { "id": 2203492, "author_name": "gauravbrills", "author_url": "", "post_date": "03/30/2023 21:21:10", "content": "Nice .. thanks will be good to add this if works into an ensemble with a large feature rich GBM 😸 .
", "votes": null, "replies": [ { "id": 2203703, "author_name": "abaojiang", "author_url": "", "post_date": "03/31/2023 04:00:31", "content": "Hi @gauravbrills,
\nThanks for the comment. I'm also curious about how an ensemble of DL-based model and GBM can perform. Let's keep going!
", "votes": null, "replies": [] } ] }, { "id": 2203630, "author_name": "chrisqiu", "author_url": "", "post_date": "03/31/2023 02:00:43", "content": "Nice! Combining event_name and name might be a good approach. name does not provide many information alone.
Hi @chrisqiu,
\nI think name can provide information about the status of the corresponding event. Also, there exist only 19 combinations of event_name (11 unique values) and name (6 unique values), not 66. Hence, training only one embedding table with the combination might be a little bit better than training two separate ones.
I learned a lot from your notebook! Thanks for your sharing.
", "votes": null, "replies": [] } ] }, { "id": 2203667, "author_name": "hoangnguyen719", "author_url": "", "post_date": "03/31/2023 03:09:06", "content": "Thanks for sharing! What NN type did you use (suppose I can't wait for the training notebook haha)?
", "votes": null, "replies": [ { "id": 2203746, "author_name": "abaojiang", "author_url": "", "post_date": "03/31/2023 05:03:35", "content": "Hi @hoangnguyen719,
\nI plot an overview of the model architecture shown as follows,
\n
Hope this can help you understand. I'll publish the training part asap😂. Thanks a lot!
", "votes": null, "replies": [] } ] }, { "id": 2205073, "author_name": "narendra", "author_url": "", "post_date": "04/01/2023 08:15:18", "content": "hi, do the difference of the elapsed time here means between 2 sucessive events based on the index?
", "votes": null, "replies": [ { "id": 2205449, "author_name": "abaojiang", "author_url": "", "post_date": "04/01/2023 15:10:48", "content": "Hi @narendra,
\nThat's true, just take the difference of elapsed_time between two successive events. However, please take care that index itself has some data errors confirmed by the competition host here. Thanks!
Hi @abaojiang,
\nThanks for sharing this notebook.
\nI am facing an issue. I am not able to get the pickle file. There is nothing at the path /kaggle/input/cat2code-v2/cat2code. Are others also facing the same issue? Could you please help me in finding out how to get the pickle file?