Hello \nI think I understand the two concepts that were presented here about quadratic Kappa matrix and ordering classification problem.\n(Thanks to this post -https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97643#latest-563327 and also https://www.kaggle.com/aroraaman/quadratic-kappa-metric-explained-in-5-simple-steps)\nThe main difference is probably that each of the five options should have different penalty error if it is different from the truth (e.g. if the truth is class 4 a prediction of 0 is worse than a prediction of 3 )\nI saw some articles and reference here that describe a way to learn the threshold between the classes.\nBut can you give some hints and idea how to take it from here? I assume that this should impact the loss function and the accuracy calculation.\nThanks
", "messages": [ { "id": "568597", "postDate": "07/05/2019 07:21:44", "content": "Hello \nI think I understand the two concepts that were presented here about quadratic Kappa matrix and ordering classification problem.\n(Thanks to this post -https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97643#latest-563327 and also https://www.kaggle.com/aroraaman/quadratic-kappa-metric-explained-in-5-simple-steps)\nThe main difference is probably that each of the five options should have different penalty error if it is different from the truth (e.g. if the truth is class 4 a prediction of 0 is worse than a prediction of 3 )\nI saw some articles and reference here that describe a way to learn the threshold between the classes.\nBut can you give some hints and idea how to take it from here? I assume that this should impact the loss function and the accuracy calculation.\nThanks
", "rawMarkdown": "Hello \nI think I understand the two concepts that were presented here about quadratic Kappa matrix and ordering classification problem.\n(Thanks to this post -https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97643#latest-563327 and also https://www.kaggle.com/aroraaman/quadratic-kappa-metric-explained-in-5-simple-steps)\nThe main difference is probably that each of the five options should have different penalty error if it is different from the truth (e.g. if the truth is class 4 a prediction of 0 is worse than a prediction of 3 )\nI saw some articles and reference here that describe a way to learn the threshold between the classes.\nBut can you give some hints and idea how to take it from here? I assume that this should impact the loss function and the accuracy calculation.\nThanks", "votes": null }, { "id": "570051", "postDate": "07/07/2019 18:01:53", "content": "I'm not sure if this helps, but that's what I understood:
\n\nLearning the threshold between the classes (Presented in Abhishek's kernel) is a discrete optimization problem with the objective of maximizing the quadratic kappa score, with respect to the predicted class (on which you apply the threshold, e.g. rounding at x.5) and the true score; this could be done on either the training set or the validation set.
I have the same understanding of QW Kappa score as you. In my starter, I provided a simple example of (unweighted) kappa scoring; writing out that example really helped me better understand the concept, so I advise you to try it out. I found that the wikipedia page on the subject was particularly good for explaining the math behind the QWK, and this thread goes over the weighting mechanism
Can some please write a code for this assignment.\nhttps://www.kaggle.com/general/98948`
", "rawMarkdown": "Can some please write a code for this assignment.\nhttps://www.kaggle.com/general/98948`", "votes": null }, { "id": "571254", "postDate": "07/09/2019 11:38:28", "content": "Thanks - I did more reading and eventually I think I figured it out
", "rawMarkdown": "Thanks - I did more reading and eventually I think I figured it out", "votes": null } ], "comments": [ { "id": 570051, "author_name": "xhlulu", "author_url": "", "post_date": "07/07/2019 18:01:53", "content": "I'm not sure if this helps, but that's what I understood:
\n\nLearning the threshold between the classes (Presented in Abhishek's kernel) is a discrete optimization problem with the objective of maximizing the quadratic kappa score, with respect to the predicted class (on which you apply the threshold, e.g. rounding at x.5) and the true score; this could be done on either the training set or the validation set.
I have the same understanding of QW Kappa score as you. In my starter, I provided a simple example of (unweighted) kappa scoring; writing out that example really helped me better understand the concept, so I advise you to try it out. I found that the wikipedia page on the subject was particularly good for explaining the math behind the QWK, and this thread goes over the weighting mechanism
Can some please write a code for this assignment.\nhttps://www.kaggle.com/general/98948`
", "votes": null, "replies": [] }, { "id": 571254, "author_name": "omershect", "author_url": "", "post_date": "07/09/2019 11:38:28", "content": "Thanks - I did more reading and eventually I think I figured it out
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "568597": "Hello \nI think I understand the two concepts that were presented here about quadratic Kappa matrix and ordering classification problem.\n(Thanks to this post -https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97643#latest-563327 and also https://www.kaggle.com/aroraaman/quadratic-kappa-metric-explained-in-5-simple-steps)\nThe main difference is probably that each of the five options should have different penalty error if it is different from the truth (e.g. if the truth is class 4 a prediction of 0 is worse than a prediction of 3 )\nI saw some articles and reference here that describe a way to learn the threshold between the classes.\nBut can you give some hints and idea how to take it from here? I assume that this should impact the loss function and the accuracy calculation.\nThanks", "570051": "I'm not sure if this helps, but that's what I understood:\n\n* Learning the threshold between the classes (Presented in Abhishek's kernel) is a discrete optimization problem with the objective of maximizing the quadratic kappa score, with respect to the predicted class (on which you apply the threshold, e.g. rounding at x.5) and the true score; this could be done on either the training set or the validation set.\n\n* I have the same understanding of QW Kappa score as you. In my starter, I provided a simple example of (unweighted) kappa scoring; writing out that example really helped me better understand the concept, so I advise you to try it out. I found that the [wikipedia page](https://en.wikipedia.org/wiki/Cohen%27s_kappa#Example) on the subject was particularly good for explaining the math behind the QWK, and this thread goes over the [weighting mechanism](https://stats.stackexchange.com/questions/248583/quadratic-weighted-kappa/248601)", "570100": "Can some please write a code for this assignment.\nhttps://www.kaggle.com/general/98948`", "571254": "Thanks - I did more reading and eventually I think I figured it out" }, "source": "meta" }