{
  "id": 505557,
  "title": "So... What evaluation methods were suitable for this competition?",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/505557",
  "author_name": "",
  "post_date": "2024-05-18T05:09:31.498123400Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I read many complaints about this evaluation method. But I don't know of any other appropriate method of evaluation. </p>\n<p>So, if you know of any other evaluation methods that should have been used for this competition, please share them with us.</p>",
  "messages": [
    {
      "id": "2821382",
      "postDate": "05/18/2024 05:09:31",
      "content": "<p>I read many complaints about this evaluation method. But I don't know of any other appropriate method of evaluation. </p>\n<p>So, if you know of any other evaluation methods that should have been used for this competition, please share them with us.</p>",
      "rawMarkdown": "I read many complaints about this evaluation method. But I don't know of any other appropriate method of evaluation. \n\nSo, if you know of any other evaluation methods that should have been used for this competition, please share them with us.",
      "votes": null
    },
    {
      "id": "2821391",
      "postDate": "05/18/2024 05:14:09",
      "content": "<p>Simple GINI score could have been used with a separate award for stability metric. Such processes are followed in other competitions too, sometimes with a slightly different approach like the efficiency leaderboard (usage of CPU and fast running code) <a href=\"https://www.kaggle.com/ariafschneider\" target=\"_blank\">@ariafschneider</a> </p>\n<p>Sometimes, over-engineering a problem is also baneful!</p>\n<p>Else, one could have tried to use the log-loss and Brier score also. This could have provided better classifiers and effective calibration also!</p>",
      "rawMarkdown": "Simple GINI score could have been used with a separate award for stability metric. Such processes are followed in other competitions too, sometimes with a slightly different approach like the efficiency leaderboard (usage of CPU and fast running code) @ariafschneider \n\nSometimes, over-engineering a problem is also baneful!\n\nElse, one could have tried to use the log-loss and Brier score also. This could have provided better classifiers and effective calibration also!",
      "votes": null
    },
    {
      "id": "2821590",
      "postDate": "05/18/2024 06:53:47",
      "content": "<p><a href=\"https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/478699\" target=\"_blank\">This post</a> is very helpful (Personally, I think this is the most valuable information in this competition).<br>\nThis is what was discussed while the competition was suspended for about three weeks. The host should have chosen a metric that is nearly impossible to hack, but it seems they were particular about the current one.</p>",
      "rawMarkdown": "[This post](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/478699) is very helpful (Personally, I think this is the most valuable information in this competition).\nThis is what was discussed while the competition was suspended for about three weeks. The host should have chosen a metric that is nearly impossible to hack, but it seems they were particular about the current one.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2821391,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "05/18/2024 05:14:09",
      "content": "<p>Simple GINI score could have been used with a separate award for stability metric. Such processes are followed in other competitions too, sometimes with a slightly different approach like the efficiency leaderboard (usage of CPU and fast running code) <a href=\"https://www.kaggle.com/ariafschneider\" target=\"_blank\">@ariafschneider</a> </p>\n<p>Sometimes, over-engineering a problem is also baneful!</p>\n<p>Else, one could have tried to use the log-loss and Brier score also. This could have provided better classifiers and effective calibration also!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2821590,
      "author_name": "mitsuyasuhoshino",
      "author_url": "",
      "post_date": "05/18/2024 06:53:47",
      "content": "<p><a href=\"https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/478699\" target=\"_blank\">This post</a> is very helpful (Personally, I think this is the most valuable information in this competition).<br>\nThis is what was discussed while the competition was suspended for about three weeks. The host should have chosen a metric that is nearly impossible to hack, but it seems they were particular about the current one.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2821382": "I read many complaints about this evaluation method. But I don't know of any other appropriate method of evaluation. \n\nSo, if you know of any other evaluation methods that should have been used for this competition, please share them with us.",
    "2821391": "Simple GINI score could have been used with a separate award for stability metric. Such processes are followed in other competitions too, sometimes with a slightly different approach like the efficiency leaderboard (usage of CPU and fast running code) @ariafschneider \n\nSometimes, over-engineering a problem is also baneful!\n\nElse, one could have tried to use the log-loss and Brier score also. This could have provided better classifiers and effective calibration also!",
    "2821590": "[This post](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/478699) is very helpful (Personally, I think this is the most valuable information in this competition).\nThis is what was discussed while the competition was suspended for about three weeks. The host should have chosen a metric that is nearly impossible to hack, but it seems they were particular about the current one."
  },
  "source": "meta"
}