![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1496178%2Fe1c980078b9013bab750934e40c26052%2FTotal_for_regressification.png?generation=1733257482416179&alt=media\")
Why do most public notebooks and solution approaches for this competition use regression followed by rounding the results? Why not use a classifier directly, optimized with quadratic weighted kappa for evaluation?
", "messages": [ { "id": "3050091", "postDate": "11/19/2024 20:06:05", "content": "Why do most public notebooks and solution approaches for this competition use regression followed by rounding the results? Why not use a classifier directly, optimized with quadratic weighted kappa for evaluation?
", "rawMarkdown": "Why do most public notebooks and solution approaches for this competition use regression followed by rounding the results? Why not use a classifier directly, optimized with quadratic weighted kappa for evaluation?", "votes": null }, { "id": "3050134", "postDate": "11/19/2024 21:10:30", "content": "It's because we are handling a multi-class classification where there is an inherent ordering between the classes.
\nMost classification algorithms can't handle that well. They only learn if they predicted the target right or wrong. They can't judge, that '0' is worse than '2' if the target is '3'.
The choice between treating this problem as classification or regression depends on how the labels are derived and the nature of the evaluation metric. Since the labels are determined based on thresholds applied to the total questionnaire scores, it suggests that the problem has a continuous underlying structure. This makes regression a natural choice because:
\nContinuous Nature of Scores: Regression models are better suited to predict continuous scores. By training a regression model, you leverage this continuity, potentially capturing more information about the relationship between the features and the target variable.
\nFlexibility in Label Mapping: Once you have a continuous output, you can easily map it back to discrete labels using rounding or applying the predefined thresholds. This approach allows you to use the regression model's full potential and adjust thresholds dynamically if needed.
\nSmooth Optimization: Regression models optimize loss functions like Mean Squared Error (MSE) or Mean Absolute Error (MAE), which are smooth and differentiable, making the training process stable and efficient. On the other hand, optimizing directly for metrics like quadratic weighted kappa (QWK) can be more challenging as it is not differentiable.
\nClassification Challenges: If you treat the problem as a classification task, the labels need to be pre-binned, potentially losing the nuance of the underlying continuous structure. Furthermore, training a classifier optimized for QWK directly is complex, and public tools or frameworks supporting this are limited. Regression followed by rounding is a simpler and more practical approach.
\nIn this competition, regression followed by rounding is popular because it balances predictive accuracy and ease of implementation, while still aligning with the quadratic weighted kappa evaluation metric after the labels are converted.
", "rawMarkdown": "The choice between treating this problem as classification or regression depends on how the labels are derived and the nature of the evaluation metric. Since the labels are determined based on thresholds applied to the total questionnaire scores, it suggests that the problem has a continuous underlying structure. This makes regression a natural choice because:\n\nContinuous Nature of Scores: Regression models are better suited to predict continuous scores. By training a regression model, you leverage this continuity, potentially capturing more information about the relationship between the features and the target variable.\n\nFlexibility in Label Mapping: Once you have a continuous output, you can easily map it back to discrete labels using rounding or applying the predefined thresholds. This approach allows you to use the regression model's full potential and adjust thresholds dynamically if needed.\n\nSmooth Optimization: Regression models optimize loss functions like Mean Squared Error (MSE) or Mean Absolute Error (MAE), which are smooth and differentiable, making the training process stable and efficient. On the other hand, optimizing directly for metrics like quadratic weighted kappa (QWK) can be more challenging as it is not differentiable.\n\nClassification Challenges: If you treat the problem as a classification task, the labels need to be pre-binned, potentially losing the nuance of the underlying continuous structure. Furthermore, training a classifier optimized for QWK directly is complex, and public tools or frameworks supporting this are limited. Regression followed by rounding is a simpler and more practical approach.\n\nIn this competition, regression followed by rounding is popular because it balances predictive accuracy and ease of implementation, while still aligning with the quadratic weighted kappa evaluation metric after the labels are converted.", "votes": null }, { "id": "3052212", "postDate": "11/22/2024 07:27:47", "content": "I think it should be used regression
", "rawMarkdown": "I think it should be used regression", "votes": null }, { "id": "3062700", "postDate": "12/03/2024 20:29:43", "content": "As something of a \"Regress-ification\" hybrid, I'm fitting PCIAT Total but modified it to have gaps at the threshold values: the idea is to provide an MSE signal for the model to pay more attention at the sii boundaries. It does help the score a little (0.005 on LB)
\n
I personally perceive this as a regression problem since the target values 0,1,2,3 resulted directly from the continuous test scores, which are also missing from the test data, so I do believe that using regression corresponds to the nature of this problem!
", "rawMarkdown": "I personally perceive this as a regression problem since the target values 0,1,2,3 resulted directly from the **continuous** test scores, which are also missing from the test data, so I do believe that using regression corresponds to the nature of this problem!", "votes": null } ], "comments": [ { "id": 3050134, "author_name": "mariusheuser", "author_url": "", "post_date": "11/19/2024 21:10:30", "content": "It's because we are handling a multi-class classification where there is an inherent ordering between the classes.
\nMost classification algorithms can't handle that well. They only learn if they predicted the target right or wrong. They can't judge, that '0' is worse than '2' if the target is '3'.
The choice between treating this problem as classification or regression depends on how the labels are derived and the nature of the evaluation metric. Since the labels are determined based on thresholds applied to the total questionnaire scores, it suggests that the problem has a continuous underlying structure. This makes regression a natural choice because:
\nContinuous Nature of Scores: Regression models are better suited to predict continuous scores. By training a regression model, you leverage this continuity, potentially capturing more information about the relationship between the features and the target variable.
\nFlexibility in Label Mapping: Once you have a continuous output, you can easily map it back to discrete labels using rounding or applying the predefined thresholds. This approach allows you to use the regression model's full potential and adjust thresholds dynamically if needed.
\nSmooth Optimization: Regression models optimize loss functions like Mean Squared Error (MSE) or Mean Absolute Error (MAE), which are smooth and differentiable, making the training process stable and efficient. On the other hand, optimizing directly for metrics like quadratic weighted kappa (QWK) can be more challenging as it is not differentiable.
\nClassification Challenges: If you treat the problem as a classification task, the labels need to be pre-binned, potentially losing the nuance of the underlying continuous structure. Furthermore, training a classifier optimized for QWK directly is complex, and public tools or frameworks supporting this are limited. Regression followed by rounding is a simpler and more practical approach.
\nIn this competition, regression followed by rounding is popular because it balances predictive accuracy and ease of implementation, while still aligning with the quadratic weighted kappa evaluation metric after the labels are converted.
", "votes": null, "replies": [] }, { "id": 3052212, "author_name": "dtleez", "author_url": "", "post_date": "11/22/2024 07:27:47", "content": "I think it should be used regression
", "votes": null, "replies": [] }, { "id": 3062700, "author_name": "dan3dewey", "author_url": "", "post_date": "12/03/2024 20:29:43", "content": "As something of a \"Regress-ification\" hybrid, I'm fitting PCIAT Total but modified it to have gaps at the threshold values: the idea is to provide an MSE signal for the model to pay more attention at the sii boundaries. It does help the score a little (0.005 on LB)
\n
I personally perceive this as a regression problem since the target values 0,1,2,3 resulted directly from the continuous test scores, which are also missing from the test data, so I do believe that using regression corresponds to the nature of this problem!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3050091": "Why do most public notebooks and solution approaches for this competition use regression followed by rounding the results? Why not use a classifier directly, optimized with quadratic weighted kappa for evaluation?", "3050134": "It's because we are handling a multi-class classification where there is an inherent ordering between the classes.\nMost classification algorithms can't handle that well. They only learn if they predicted the target right or wrong. They can't judge, that '0' is worse than '2' if the target is '3'.", "3052124": "The choice between treating this problem as classification or regression depends on how the labels are derived and the nature of the evaluation metric. Since the labels are determined based on thresholds applied to the total questionnaire scores, it suggests that the problem has a continuous underlying structure. This makes regression a natural choice because:\n\nContinuous Nature of Scores: Regression models are better suited to predict continuous scores. By training a regression model, you leverage this continuity, potentially capturing more information about the relationship between the features and the target variable.\n\nFlexibility in Label Mapping: Once you have a continuous output, you can easily map it back to discrete labels using rounding or applying the predefined thresholds. This approach allows you to use the regression model's full potential and adjust thresholds dynamically if needed.\n\nSmooth Optimization: Regression models optimize loss functions like Mean Squared Error (MSE) or Mean Absolute Error (MAE), which are smooth and differentiable, making the training process stable and efficient. On the other hand, optimizing directly for metrics like quadratic weighted kappa (QWK) can be more challenging as it is not differentiable.\n\nClassification Challenges: If you treat the problem as a classification task, the labels need to be pre-binned, potentially losing the nuance of the underlying continuous structure. Furthermore, training a classifier optimized for QWK directly is complex, and public tools or frameworks supporting this are limited. Regression followed by rounding is a simpler and more practical approach.\n\nIn this competition, regression followed by rounding is popular because it balances predictive accuracy and ease of implementation, while still aligning with the quadratic weighted kappa evaluation metric after the labels are converted.", "3052212": "I think it should be used regression", "3062700": "As something of a \"Regress-ification\" hybrid, I'm fitting PCIAT Total but modified it to have gaps at the threshold values: the idea is to provide an MSE signal for the model to pay more attention at the sii boundaries. It does help the score a little (0.005 on LB)\n\n", "3075379": "I personally perceive this as a regression problem since the target values 0,1,2,3 resulted directly from the **continuous** test scores, which are also missing from the test data, so I do believe that using regression corresponds to the nature of this problem!" }, "source": "meta" }