{
  "id": 535052,
  "title": "Results of verifying the effectiveness of the custom objective for LGBM with Quadratic Weighted Kappa (QWK)",
  "url": "/competitions/child-mind-institute-problematic-internet-use/discussion/535052",
  "author_name": "",
  "post_date": "2024-09-20T01:51:57.738236900Z",
  "votes": 89,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hello! </p>\n<p>The custom objective for Quadratic Weighted Kappa (QWK) was also introduced in <a href=\"https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2\" target=\"_blank\">the Learning Agency Lab - Automated Essay Scoring 2.0 competition</a> , and its effectiveness was a topic of discussion.</p>\n<p>The base reference code can be found here. Thank you for <a href=\"https://www.kaggle.com/rsakata\" target=\"_blank\">@rsakata</a> </p>\n<p><a href=\"https://www.kaggle.com/code/rsakata/optimize-qwk-by-lgb\" target=\"_blank\">https://www.kaggle.com/code/rsakata/optimize-qwk-by-lgb</a></p>\n<p>Here are the details of my Light GBM experiment (Stratified Kfold 5) in this competition:</p>\n<ul>\n<li><p>multiclass objective : cv 0.2643 , public lb 0.265</p></li>\n<li><p>custom objective for QWK : cv 0.45187, public lb 0.401</p></li>\n</ul>\n<p>It might be possible to find the setup methods for CatBoost and XGBoost if you search through the discussions or code from  <a href=\"https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2\" target=\"_blank\">the Learning Agency Lab - Automated Essay Scoring 2.0 competition</a> (sorry, I haven't checked it yet, but I feel like I've seen it).</p>\n<p>Enjoy !!</p>",
  "messages": [
    {
      "id": "2993616",
      "postDate": "09/20/2024 01:51:57",
      "content": "<p>Hello! </p>\n<p>The custom objective for Quadratic Weighted Kappa (QWK) was also introduced in <a href=\"https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2\" target=\"_blank\">the Learning Agency Lab - Automated Essay Scoring 2.0 competition</a> , and its effectiveness was a topic of discussion.</p>\n<p>The base reference code can be found here. Thank you for <a href=\"https://www.kaggle.com/rsakata\" target=\"_blank\">@rsakata</a> </p>\n<p><a href=\"https://www.kaggle.com/code/rsakata/optimize-qwk-by-lgb\" target=\"_blank\">https://www.kaggle.com/code/rsakata/optimize-qwk-by-lgb</a></p>\n<p>Here are the details of my Light GBM experiment (Stratified Kfold 5) in this competition:</p>\n<ul>\n<li><p>multiclass objective : cv 0.2643 , public lb 0.265</p></li>\n<li><p>custom objective for QWK : cv 0.45187, public lb 0.401</p></li>\n</ul>\n<p>It might be possible to find the setup methods for CatBoost and XGBoost if you search through the discussions or code from  <a href=\"https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2\" target=\"_blank\">the Learning Agency Lab - Automated Essay Scoring 2.0 competition</a> (sorry, I haven't checked it yet, but I feel like I've seen it).</p>\n<p>Enjoy !!</p>",
      "rawMarkdown": "Hello! \n\nThe custom objective for Quadratic Weighted Kappa (QWK) was also introduced in [the Learning Agency Lab - Automated Essay Scoring 2.0 competition](https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2) , and its effectiveness was a topic of discussion.\n\nThe base reference code can be found here. Thank you for @rsakata \n\nhttps://www.kaggle.com/code/rsakata/optimize-qwk-by-lgb\n\nHere are the details of my Light GBM experiment (Stratified Kfold 5) in this competition:\n\n - multiclass objective : cv 0.2643 , public lb 0.265\n\n - custom objective for QWK : cv 0.45187, public lb 0.401\n\nIt might be possible to find the setup methods for CatBoost and XGBoost if you search through the discussions or code from  [the Learning Agency Lab - Automated Essay Scoring 2.0 competition](https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2) (sorry, I haven't checked it yet, but I feel like I've seen it).\n\nEnjoy !!",
      "votes": null
    },
    {
      "id": "2994765",
      "postDate": "09/21/2024 12:34:15",
      "content": "<p>Hi<br>\nhave you check it now?</p>",
      "rawMarkdown": "Hi\nhave you check it now?",
      "votes": null
    },
    {
      "id": "2999411",
      "postDate": "09/26/2024 16:17:34",
      "content": "<p>Thank you for trying my idea! There was a misleading part in my previous notebook, so I created a more concise version of the notebook using the data of this competition!<br>\n<a href=\"https://www.kaggle.com/code/rsakata/cmi-piu-optimize-qwk-by-lgb/notebook\" target=\"_blank\">https://www.kaggle.com/code/rsakata/cmi-piu-optimize-qwk-by-lgb/notebook</a></p>",
      "rawMarkdown": "Thank you for trying my idea! There was a misleading part in my previous notebook, so I created a more concise version of the notebook using the data of this competition!\nhttps://www.kaggle.com/code/rsakata/cmi-piu-optimize-qwk-by-lgb/notebook",
      "votes": null
    },
    {
      "id": "2999832",
      "postDate": "09/26/2024 22:44:04",
      "content": "<p><a href=\"https://www.kaggle.com/rsakata\" target=\"_blank\">@rsakata</a> Thank you for the clean code! We can learn a lot from it!</p>",
      "rawMarkdown": "rsakata Thank you for the clean code! We can learn a lot from it!",
      "votes": null
    },
    {
      "id": "3004398",
      "postDate": "10/01/2024 19:17:58",
      "content": "<p>why haven't you considered the parquet files?</p>",
      "rawMarkdown": "why haven't you considered the parquet files?",
      "votes": null
    },
    {
      "id": "3008634",
      "postDate": "10/06/2024 21:33:37",
      "content": "<p>That's actually impressive! , It almost yeilds the same results for both cv and LB as an xgboost trained to predict PCIAT-PCIAT_Total (regression) with squared error as a loss .</p>",
      "rawMarkdown": "That's actually impressive! , It almost yeilds the same results for both cv and LB as an xgboost trained to predict PCIAT-PCIAT_Total (regression) with squared error as a loss .",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2994765,
      "author_name": "",
      "author_url": "",
      "post_date": "09/21/2024 12:34:15",
      "content": "<p>Hi<br>\nhave you check it now?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2999411,
      "author_name": "rsakata",
      "author_url": "",
      "post_date": "09/26/2024 16:17:34",
      "content": "<p>Thank you for trying my idea! There was a misleading part in my previous notebook, so I created a more concise version of the notebook using the data of this competition!<br>\n<a href=\"https://www.kaggle.com/code/rsakata/cmi-piu-optimize-qwk-by-lgb/notebook\" target=\"_blank\">https://www.kaggle.com/code/rsakata/cmi-piu-optimize-qwk-by-lgb/notebook</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 2999832,
          "author_name": "chumajin",
          "author_url": "",
          "post_date": "09/26/2024 22:44:04",
          "content": "<p><a href=\"https://www.kaggle.com/rsakata\" target=\"_blank\">@rsakata</a> Thank you for the clean code! We can learn a lot from it!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3004398,
      "author_name": "godanu",
      "author_url": "",
      "post_date": "10/01/2024 19:17:58",
      "content": "<p>why haven't you considered the parquet files?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3008634,
      "author_name": "mohammedahmedxx12",
      "author_url": "",
      "post_date": "10/06/2024 21:33:37",
      "content": "<p>That's actually impressive! , It almost yeilds the same results for both cv and LB as an xgboost trained to predict PCIAT-PCIAT_Total (regression) with squared error as a loss .</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2993616": "Hello! \n\nThe custom objective for Quadratic Weighted Kappa (QWK) was also introduced in [the Learning Agency Lab - Automated Essay Scoring 2.0 competition](https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2) , and its effectiveness was a topic of discussion.\n\nThe base reference code can be found here. Thank you for @rsakata \n\nhttps://www.kaggle.com/code/rsakata/optimize-qwk-by-lgb\n\nHere are the details of my Light GBM experiment (Stratified Kfold 5) in this competition:\n\n - multiclass objective : cv 0.2643 , public lb 0.265\n\n - custom objective for QWK : cv 0.45187, public lb 0.401\n\nIt might be possible to find the setup methods for CatBoost and XGBoost if you search through the discussions or code from  [the Learning Agency Lab - Automated Essay Scoring 2.0 competition](https://www.kaggle.com/competitions/learning-agency-lab-automated-essay-scoring-2) (sorry, I haven't checked it yet, but I feel like I've seen it).\n\nEnjoy !!",
    "2994765": "Hi\nhave you check it now?",
    "2999411": "Thank you for trying my idea! There was a misleading part in my previous notebook, so I created a more concise version of the notebook using the data of this competition!\nhttps://www.kaggle.com/code/rsakata/cmi-piu-optimize-qwk-by-lgb/notebook",
    "2999832": "rsakata Thank you for the clean code! We can learn a lot from it!",
    "3004398": "why haven't you considered the parquet files?",
    "3008634": "That's actually impressive! , It almost yeilds the same results for both cv and LB as an xgboost trained to predict PCIAT-PCIAT_Total (regression) with squared error as a loss ."
  },
  "source": "meta"
}