Starter Notebook: Multi-Target Prediction Using CatBoost
\nIn this notebook, we demonstrate how to use CatBoost's MultiRegression to predict multiple targets. The goal is to build a baseline model that can be useful for Kaggle competitions.
\nMulti-Target Prediction Using CatBoost's MultiRegression
\nConverting PCIAT-PCIAT_Total Predictions to Sii Predictions
\nUtilizing Features from the Parquet File
\nWe plan to incorporate features from the parquet files to further improve the model's performance.
Pseudo-Labeling
\nWe aim to implement pseudo-labeling to improve performance when labeled data is scarce.
Feature Engineering
\nWe will explore feature combinations and transformations to enhance the model's accuracy.
This notebook serves as a useful baseline for multi-target prediction, especially for Kaggle participants. We plan to update it with improvements and new ideas in the future. Please feel free to share your feedback or suggestions!
", "messages": [ { "id": "2993939", "postDate": "09/20/2024 11:41:54", "content": "Starter Notebook: Multi-Target Prediction Using CatBoost
\nIn this notebook, we demonstrate how to use CatBoost's MultiRegression to predict multiple targets. The goal is to build a baseline model that can be useful for Kaggle competitions.
\nMulti-Target Prediction Using CatBoost's MultiRegression
\nConverting PCIAT-PCIAT_Total Predictions to Sii Predictions
\nUtilizing Features from the Parquet File
\nWe plan to incorporate features from the parquet files to further improve the model's performance.
Pseudo-Labeling
\nWe aim to implement pseudo-labeling to improve performance when labeled data is scarce.
Feature Engineering
\nWe will explore feature combinations and transformations to enhance the model's accuracy.
This notebook serves as a useful baseline for multi-target prediction, especially for Kaggle participants. We plan to update it with improvements and new ideas in the future. Please feel free to share your feedback or suggestions!
", "rawMarkdown": "[Starter Notebook: Multi-Target Prediction Using CatBoost](https://www.kaggle.com/code/tubotubo/starter-notebook-multi-target-prediction?scriptVersionId=197454913)\n\n## Introduction\nIn this notebook, we demonstrate how to use CatBoost's MultiRegression to predict multiple targets. The goal is to build a baseline model that can be useful for Kaggle competitions.\n\n## What We Did\n\n1. **Multi-Target Prediction Using CatBoost's MultiRegression** \n - **Features**: We only used the features from the CSV file and did not use any from the parquet file. \n - **Data Preprocessing**: Rows with missing target values were excluded from the training data.\n\n2. **Converting PCIAT-PCIAT_Total Predictions to Sii Predictions** \n - Based on the predicted values, we converted the predictions to Sii using a threshold. \n - The threshold was optimized using [Optuna](https://optuna.org/).\n\n## Result\n\n- CV: 0.461\n- LB: 0.441\n\n## Future Directions\n\n- **Utilizing Features from the Parquet File** \n We plan to incorporate features from the parquet files to further improve the model's performance.\n\n- **Pseudo-Labeling** \n We aim to implement pseudo-labeling to improve performance when labeled data is scarce.\n\n- **Feature Engineering** \n We will explore feature combinations and transformations to enhance the model's accuracy.\n\nThis notebook serves as a useful baseline for multi-target prediction, especially for Kaggle participants. We plan to update it with improvements and new ideas in the future. Please feel free to share your feedback or suggestions!", "votes": null }, { "id": "2994301", "postDate": "09/20/2024 18:43:46", "content": "Problem is MultiClass Classification ? or MultiClass Regression.
\nmy CV : 0.3299 | LB : 0.345
", "rawMarkdown": "Problem is MultiClass Classification ? or MultiClass Regression. \n\nmy CV : 0.3299 | LB : 0.345", "votes": null }, { "id": "2994416", "postDate": "09/20/2024 22:38:04", "content": "This is a tricky problem! You can try either approach @abdmental01
", "rawMarkdown": "This is a tricky problem! You can try either approach @abdmental01", "votes": null }, { "id": "2994422", "postDate": "09/20/2024 22:54:42", "content": "Can you Suggest Which one is more Suitable?
", "rawMarkdown": "Can you Suggest Which one is more Suitable?", "votes": null }, { "id": "2996156", "postDate": "09/23/2024 06:20:27", "content": "It would be great if you can suggest upon how to utilize features from the Parquet file. It is very difficult for me.
", "rawMarkdown": "It would be great if you can suggest upon how to utilize features from the Parquet file. It is very difficult for me.", "votes": null }, { "id": "2996510", "postDate": "09/23/2024 15:08:52", "content": "Given there's only four classes, I'd stick with classification from my end at first. However, I think people have had great success reconfiguring these problems as a regression task. Specifically, I'm thinking of the Common Lit prizes.
\nI think this has to do with the fact the labels in this case are ordered, meaning a score of 1 is greater than a score of 0 and score of 2 greater than 1, etc. This means the labels have an inherent order to them and might help framing it as regression.
\nBut as the case always is, do both, experiment, and pick what works best.
", "rawMarkdown": "Given there's only four classes, I'd stick with classification from my end at first. However, I think people have had great success reconfiguring these problems as a regression task. Specifically, I'm thinking of the Common Lit prizes.\n\nI think this has to do with the fact the labels in this case are ordered, meaning a score of 1 is greater than a score of 0 and score of 2 greater than 1, etc. This means the labels have an inherent order to them and might help framing it as regression.\n\nBut as the case always is, do both, experiment, and pick what works best.", "votes": null }, { "id": "2996884", "postDate": "09/24/2024 03:12:22", "content": "Thanks, Mate .
", "rawMarkdown": "Thanks, Mate .", "votes": null }, { "id": "3015681", "postDate": "10/12/2024 18:48:32", "content": "is true that lightgbm, catboost, xgboost has upper uper hand over deep learning such pytorch in tabular data such as in this competition (most public notebboks use lightgbm , catboost in their approch)?
", "rawMarkdown": "is true that lightgbm, catboost, xgboost has upper uper hand over deep learning such pytorch in tabular data such as in this competition (most public notebboks use lightgbm , catboost in their approch)?", "votes": null }, { "id": "3029874", "postDate": "10/27/2024 21:09:32", "content": "This is a great starting point.
", "rawMarkdown": "This is a great starting point.", "votes": null } ], "comments": [ { "id": 2994301, "author_name": "abdmental01", "author_url": "", "post_date": "09/20/2024 18:43:46", "content": "Problem is MultiClass Classification ? or MultiClass Regression.
\nmy CV : 0.3299 | LB : 0.345
", "votes": null, "replies": [ { "id": 2994416, "author_name": "ravi20076", "author_url": "", "post_date": "09/20/2024 22:38:04", "content": "This is a tricky problem! You can try either approach @abdmental01
", "votes": null, "replies": [ { "id": 2994422, "author_name": "abdmental01", "author_url": "", "post_date": "09/20/2024 22:54:42", "content": "Can you Suggest Which one is more Suitable?
", "votes": null, "replies": [ { "id": 2996510, "author_name": "msthil", "author_url": "", "post_date": "09/23/2024 15:08:52", "content": "Given there's only four classes, I'd stick with classification from my end at first. However, I think people have had great success reconfiguring these problems as a regression task. Specifically, I'm thinking of the Common Lit prizes.
\nI think this has to do with the fact the labels in this case are ordered, meaning a score of 1 is greater than a score of 0 and score of 2 greater than 1, etc. This means the labels have an inherent order to them and might help framing it as regression.
\nBut as the case always is, do both, experiment, and pick what works best.
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2996156, "author_name": "akihiroorita", "author_url": "", "post_date": "09/23/2024 06:20:27", "content": "It would be great if you can suggest upon how to utilize features from the Parquet file. It is very difficult for me.
", "votes": null, "replies": [] }, { "id": 2996884, "author_name": "k2001g", "author_url": "", "post_date": "09/24/2024 03:12:22", "content": "Thanks, Mate .
", "votes": null, "replies": [] }, { "id": 3015681, "author_name": "saidkoussi", "author_url": "", "post_date": "10/12/2024 18:48:32", "content": "is true that lightgbm, catboost, xgboost has upper uper hand over deep learning such pytorch in tabular data such as in this competition (most public notebboks use lightgbm , catboost in their approch)?
", "votes": null, "replies": [] }, { "id": 3029874, "author_name": "wekeed", "author_url": "", "post_date": "10/27/2024 21:09:32", "content": "This is a great starting point.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2993939": "[Starter Notebook: Multi-Target Prediction Using CatBoost](https://www.kaggle.com/code/tubotubo/starter-notebook-multi-target-prediction?scriptVersionId=197454913)\n\n## Introduction\nIn this notebook, we demonstrate how to use CatBoost's MultiRegression to predict multiple targets. The goal is to build a baseline model that can be useful for Kaggle competitions.\n\n## What We Did\n\n1. **Multi-Target Prediction Using CatBoost's MultiRegression** \n - **Features**: We only used the features from the CSV file and did not use any from the parquet file. \n - **Data Preprocessing**: Rows with missing target values were excluded from the training data.\n\n2. **Converting PCIAT-PCIAT_Total Predictions to Sii Predictions** \n - Based on the predicted values, we converted the predictions to Sii using a threshold. \n - The threshold was optimized using [Optuna](https://optuna.org/).\n\n## Result\n\n- CV: 0.461\n- LB: 0.441\n\n## Future Directions\n\n- **Utilizing Features from the Parquet File** \n We plan to incorporate features from the parquet files to further improve the model's performance.\n\n- **Pseudo-Labeling** \n We aim to implement pseudo-labeling to improve performance when labeled data is scarce.\n\n- **Feature Engineering** \n We will explore feature combinations and transformations to enhance the model's accuracy.\n\nThis notebook serves as a useful baseline for multi-target prediction, especially for Kaggle participants. We plan to update it with improvements and new ideas in the future. Please feel free to share your feedback or suggestions!", "2994301": "Problem is MultiClass Classification ? or MultiClass Regression. \n\nmy CV : 0.3299 | LB : 0.345", "2994416": "This is a tricky problem! You can try either approach @abdmental01", "2994422": "Can you Suggest Which one is more Suitable?", "2996156": "It would be great if you can suggest upon how to utilize features from the Parquet file. It is very difficult for me.", "2996510": "Given there's only four classes, I'd stick with classification from my end at first. However, I think people have had great success reconfiguring these problems as a regression task. Specifically, I'm thinking of the Common Lit prizes.\n\nI think this has to do with the fact the labels in this case are ordered, meaning a score of 1 is greater than a score of 0 and score of 2 greater than 1, etc. This means the labels have an inherent order to them and might help framing it as regression.\n\nBut as the case always is, do both, experiment, and pick what works best.", "2996884": "Thanks, Mate .", "3015681": "is true that lightgbm, catboost, xgboost has upper uper hand over deep learning such pytorch in tabular data such as in this competition (most public notebboks use lightgbm , catboost in their approch)?", "3029874": "This is a great starting point." }, "source": "meta" }