Hi everyone,
\nAfter spending some time refactoring the local pipeline into readable notebook, I finally publish the training counterpart of the inference notebook released about 3 days ago.
\nIn this kernel, I implement a complete pipeline, including data cleaning and processing, simple feature engineering, and model training with the specified CV scheme (i.e., GroupKFold with k=5). After the process is done, output objects (e.g., best model checkpoints) can be downloaded and used in the inference part.
To be concrete, the experiment setup can be summarized as follows:
\nelapsed_time, event_name, name, and room_fqid.![\"2023-04-02-11-22-49.png\"](\"https://i.postimg.cc/90YNxtLQ/2023-04-02-11-22-49.png\")
BCEWithLogitsLoss is selected to be the loss criterion.Adam and CosineAnnealingWarmRestarts are used.GroupKFold with k=5 is adopted to be the CV scheme, and all models are trained with 100 epochs without early stopping.Also, I think there's still a long way to go. In the future, I'm going to extend the experiments as follows:
\nHope this quick summarization can help clarify the core setup of the experiment. If there's any mistake, please feel free to correct me. I appreciate you taking the time to look over the work and discuss with me.
", "messages": [ { "id": "2206499", "postDate": "04/02/2023 15:41:44", "content": "Hi everyone,
\nAfter spending some time refactoring the local pipeline into readable notebook, I finally publish the training counterpart of the inference notebook released about 3 days ago.
\nIn this kernel, I implement a complete pipeline, including data cleaning and processing, simple feature engineering, and model training with the specified CV scheme (i.e., GroupKFold with k=5). After the process is done, output objects (e.g., best model checkpoints) can be downloaded and used in the inference part.
To be concrete, the experiment setup can be summarized as follows:
\nelapsed_time, event_name, name, and room_fqid.BCEWithLogitsLoss is selected to be the loss criterion.Adam and CosineAnnealingWarmRestarts are used.GroupKFold with k=5 is adopted to be the CV scheme, and all models are trained with 100 epochs without early stopping.Also, I think there's still a long way to go. In the future, I'm going to extend the experiments as follows:
\nHope this quick summarization can help clarify the core setup of the experiment. If there's any mistake, please feel free to correct me. I appreciate you taking the time to look over the work and discuss with me.
", "rawMarkdown": "Hi everyone,\n\nAfter spending some time refactoring the local pipeline into readable notebook, I finally publish the [**training** counterpart](https://www.kaggle.com/code/abaojiang/lb-0-694-tconv-with-4-features-training-part) of the inference notebook released about 3 days ago.\n\nIn this kernel, I implement a complete pipeline, including data cleaning and processing, simple feature engineering, and model training with the specified CV scheme (i.e., `GroupKFold` with k=5). After the process is done, output objects (e.g., best model checkpoints) can be downloaded and used in the inference part.\n\nTo be concrete, the experiment setup can be summarized as follows:\n1. Only 4 features are taken into consideration, including `elapsed_time`, `event_name`, `name`, and `room_fqid`.\n2. The model architecture is mainly based on 1D-Conv and categorical embeddings (as illustrated in following figure).\n[](https://postimg.cc/vxDzT9RJ)\n3. `BCEWithLogitsLoss` is selected to be the loss criterion.\n4. `Adam` and `CosineAnnealingWarmRestarts` are used.\n5. `GroupKFold` with `k=5` is adopted to be the CV scheme, and all models are trained with 100 epochs without early stopping.\n\nAlso, I think there's still a long way to go. In the future, I'm going to extend the experiments as follows:\n1. Add more features (*e.g.,* text information).\n2. Try different model architectures.\n * I've tried a simple modification of the model architecture, [SAINT+](https://arxiv.org/abs/2010.12042), but still can't obtain satisfying performance.\n3. Add more training data (*i.e.,* released old test set).\n * As you can see in the training notebook, I use the old version of training set to train models.\n4. Ensemble with tree-based models.\n5. Do some hyperparameter tuning near the end of the competition.\n\nHope this quick summarization can help clarify the core setup of the experiment. If there's any mistake, please feel free to correct me. I appreciate you taking the time to look over the work and discuss with me.", "votes": null }, { "id": "2206676", "postDate": "04/02/2023 18:25:01", "content": "thank you so much for publishing this!
", "rawMarkdown": "thank you so much for publishing this!", "votes": null }, { "id": "2206884", "postDate": "04/03/2023 01:25:19", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!", "votes": null }, { "id": "2208273", "postDate": "04/03/2023 23:55:49", "content": "Thank you so much for sharing it!
", "rawMarkdown": "Thank you so much for sharing it!", "votes": null }, { "id": "2208931", "postDate": "04/04/2023 11:25:12", "content": "Thank you so much!! @abaojiang
\nThis is very helpful!
(Update, 0404): After I train models using the new training set (i.e., old training set with old test set), CV and LB both boost a little. The performance report is shown as follows:
\n| Training Data | \nCV (GroupKFold with k=5) | \nHoldout (Released Old Test Set) | \nLB | \n
|---|---|---|---|
| Old training set only | \n0.6914 | \n0.6911 | \n0.694 | \n
| Old training set with old test set | \n0.6931 (+0.0017) | \nX (Already act as training set) | \n0.695 | \n
The experimental result shows that adding more training data is beneficial, but not significant. Thanks!
", "rawMarkdown": "**(Update, 0404):** After I train models using the new training set (*i.e.,* old training set with old test set), CV and LB both boost a little. The performance report is shown as follows:\n\n|Training Data | CV (GroupKFold with k=5) | Holdout (Released Old Test Set) | LB |\n|------------------------| ------------------------ | ------------------------------- | ----- |\n|Old training set only| 0.6914 | 0.6911 | 0.694 |\n|Old training set with old test set| 0.6931 **(+0.0017)** | X (Already act as training set) | 0.695 |\n\nThe experimental result shows that adding more training data is beneficial, but not significant. Thanks!", "votes": null }, { "id": "2209225", "postDate": "04/04/2023 14:56:22", "content": "Thanks @cdeotte, I've learned a lot from you!
", "rawMarkdown": "Thanks @cdeotte, I've learned a lot from you!", "votes": null }, { "id": "2209228", "postDate": "04/04/2023 14:57:06", "content": "Thanks @hanaori 😊
", "rawMarkdown": "Thanks @hanaori 😊", "votes": null }, { "id": "2209230", "postDate": "04/04/2023 14:57:49", "content": "Thanks @simonveitner, glad it helps!
", "rawMarkdown": "Thanks @simonveitner, glad it helps!", "votes": null }, { "id": "2221704", "postDate": "04/14/2023 13:37:35", "content": "Thank you so much for sharing this!
", "rawMarkdown": "Thank you so much for sharing this!", "votes": null }, { "id": "2221749", "postDate": "04/14/2023 14:37:24", "content": "Excellent work done @abaojiang
", "rawMarkdown": "Excellent work done @abaojiang", "votes": null }, { "id": "2226654", "postDate": "04/19/2023 05:33:55", "content": "Thanks @abaojiang
", "rawMarkdown": "Thanks @abaojiang", "votes": null } ], "comments": [ { "id": 2206676, "author_name": "simonveitner", "author_url": "", "post_date": "04/02/2023 18:25:01", "content": "thank you so much for publishing this!
", "votes": null, "replies": [ { "id": 2209230, "author_name": "abaojiang", "author_url": "", "post_date": "04/04/2023 14:57:49", "content": "Thanks @simonveitner, glad it helps!
", "votes": null, "replies": [] } ] }, { "id": 2206884, "author_name": "cdeotte", "author_url": "", "post_date": "04/03/2023 01:25:19", "content": "Thanks for sharing!
", "votes": null, "replies": [ { "id": 2209225, "author_name": "abaojiang", "author_url": "", "post_date": "04/04/2023 14:56:22", "content": "Thanks @cdeotte, I've learned a lot from you!
", "votes": null, "replies": [] } ] }, { "id": 2208273, "author_name": "hanaori", "author_url": "", "post_date": "04/03/2023 23:55:49", "content": "Thank you so much for sharing it!
", "votes": null, "replies": [ { "id": 2209228, "author_name": "abaojiang", "author_url": "", "post_date": "04/04/2023 14:57:06", "content": "Thanks @hanaori 😊
", "votes": null, "replies": [] } ] }, { "id": 2208931, "author_name": "shashwatraman", "author_url": "", "post_date": "04/04/2023 11:25:12", "content": "Thank you so much!! @abaojiang
\nThis is very helpful!
(Update, 0404): After I train models using the new training set (i.e., old training set with old test set), CV and LB both boost a little. The performance report is shown as follows:
\n| Training Data | \nCV (GroupKFold with k=5) | \nHoldout (Released Old Test Set) | \nLB | \n
|---|---|---|---|
| Old training set only | \n0.6914 | \n0.6911 | \n0.694 | \n
| Old training set with old test set | \n0.6931 (+0.0017) | \nX (Already act as training set) | \n0.695 | \n
The experimental result shows that adding more training data is beneficial, but not significant. Thanks!
", "votes": null, "replies": [] }, { "id": 2221704, "author_name": "linchia6", "author_url": "", "post_date": "04/14/2023 13:37:35", "content": "Thank you so much for sharing this!
", "votes": null, "replies": [] }, { "id": 2221749, "author_name": "mohimaakter", "author_url": "", "post_date": "04/14/2023 14:37:24", "content": "Excellent work done @abaojiang
", "votes": null, "replies": [] }, { "id": 2226654, "author_name": "zainalinasir", "author_url": "", "post_date": "04/19/2023 05:33:55", "content": "Thanks @abaojiang
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2206499": "Hi everyone,\n\nAfter spending some time refactoring the local pipeline into readable notebook, I finally publish the [**training** counterpart](https://www.kaggle.com/code/abaojiang/lb-0-694-tconv-with-4-features-training-part) of the inference notebook released about 3 days ago.\n\nIn this kernel, I implement a complete pipeline, including data cleaning and processing, simple feature engineering, and model training with the specified CV scheme (i.e., `GroupKFold` with k=5). After the process is done, output objects (e.g., best model checkpoints) can be downloaded and used in the inference part.\n\nTo be concrete, the experiment setup can be summarized as follows:\n1. Only 4 features are taken into consideration, including `elapsed_time`, `event_name`, `name`, and `room_fqid`.\n2. The model architecture is mainly based on 1D-Conv and categorical embeddings (as illustrated in following figure).\n[](https://postimg.cc/vxDzT9RJ)\n3. `BCEWithLogitsLoss` is selected to be the loss criterion.\n4. `Adam` and `CosineAnnealingWarmRestarts` are used.\n5. `GroupKFold` with `k=5` is adopted to be the CV scheme, and all models are trained with 100 epochs without early stopping.\n\nAlso, I think there's still a long way to go. In the future, I'm going to extend the experiments as follows:\n1. Add more features (*e.g.,* text information).\n2. Try different model architectures.\n * I've tried a simple modification of the model architecture, [SAINT+](https://arxiv.org/abs/2010.12042), but still can't obtain satisfying performance.\n3. Add more training data (*i.e.,* released old test set).\n * As you can see in the training notebook, I use the old version of training set to train models.\n4. Ensemble with tree-based models.\n5. Do some hyperparameter tuning near the end of the competition.\n\nHope this quick summarization can help clarify the core setup of the experiment. If there's any mistake, please feel free to correct me. I appreciate you taking the time to look over the work and discuss with me.", "2206676": "thank you so much for publishing this!", "2206884": "Thanks for sharing!", "2208273": "Thank you so much for sharing it!", "2208931": "Thank you so much!! @abaojiang \nThis is very helpful!", "2209209": "**(Update, 0404):** After I train models using the new training set (*i.e.,* old training set with old test set), CV and LB both boost a little. The performance report is shown as follows:\n\n|Training Data | CV (GroupKFold with k=5) | Holdout (Released Old Test Set) | LB |\n|------------------------| ------------------------ | ------------------------------- | ----- |\n|Old training set only| 0.6914 | 0.6911 | 0.694 |\n|Old training set with old test set| 0.6931 **(+0.0017)** | X (Already act as training set) | 0.695 |\n\nThe experimental result shows that adding more training data is beneficial, but not significant. Thanks!", "2209225": "Thanks @cdeotte, I've learned a lot from you!", "2209228": "Thanks @hanaori 😊", "2209230": "Thanks @simonveitner, glad it helps!", "2221704": "Thank you so much for sharing this!", "2221749": "Excellent work done @abaojiang", "2226654": "Thanks @abaojiang" }, "source": "meta" }