{ "id": 478162, "title": "Best single model", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/478162", "author_name": "", "post_date": "2024-02-19T11:55:57.061411500Z", "votes": 3, "comment_count": 5, "views": 0, "content": "

As I've discussed here, the metric focused on stability promotes ensembling of many diverse models.

\n

In this topic, I wanted to ask about the best single model.

\n

Mine is LGBM, AUC cv 0.855, LB 0.591 (after metric hacking).

", "messages": [ { "id": "2658775", "postDate": "02/19/2024 11:55:57", "content": "

As I've discussed here, the metric focused on stability promotes ensembling of many diverse models.

\n

In this topic, I wanted to ask about the best single model.

\n

Mine is LGBM, AUC cv 0.855, LB 0.591 (after metric hacking).

", "rawMarkdown": "As I've discussed [here](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477228), the metric focused on stability promotes ensembling of many diverse models.\n\nIn this topic, I wanted to ask about the best *single* model.\n\nMine is LGBM, AUC cv 0.855, LB 0.591 (after metric hacking).", "votes": null }, { "id": "2658821", "postDate": "02/19/2024 12:32:36", "content": "

Mine is a NN with AUC cv of 0.822 and LB 0.576 (also with metric hacking)

", "rawMarkdown": "Mine is a NN with AUC cv of 0.822 and LB 0.576 (also with metric hacking)", "votes": null }, { "id": "2659541", "postDate": "02/20/2024 00:21:47", "content": "

May I ask which CV has the best performance in this competition?

", "rawMarkdown": "May I ask which CV has the best performance in this competition?", "votes": null }, { "id": "2659673", "postDate": "02/20/2024 04:42:01", "content": "

@narsil from my initial baseline approaches, I concur that the public leaderboard is very unstable and is highly sensitive to the weights used across the models in the blend. This is likely to lead to a big churn at the end. What do you have to say in this regard?

", "rawMarkdown": "narsil from my initial baseline approaches, I concur that the public leaderboard is very unstable and is highly sensitive to the weights used across the models in the blend. This is likely to lead to a big churn at the end. What do you have to say in this regard?", "votes": null }, { "id": "2659777", "postDate": "02/20/2024 06:04:11", "content": "

I will go with recommendation from #1 place, which is to just use weeks since 60 WEEKNUM for validation

", "rawMarkdown": "I will go with recommendation from #1 place, which is to just use weeks since 60 `WEEKNUM` for validation", "votes": null }, { "id": "2659781", "postDate": "02/20/2024 06:06:17", "content": "

If you mean big shakeup - difficult to say because we have 2 opposing forces at play here:

\n", "rawMarkdown": "If you mean big shakeup - difficult to say because we have 2 opposing forces at play here:\n- public vs private split is a mystery -> increasing the magnitude of potential shakeup\n- competition promotes stability - blending of many models -> decreasing the magnitude of shakeup", "votes": null } ], "comments": [ { "id": 2658821, "author_name": "ralfffhs", "author_url": "", "post_date": "02/19/2024 12:32:36", "content": "

Mine is a NN with AUC cv of 0.822 and LB 0.576 (also with metric hacking)

", "votes": null, "replies": [] }, { "id": 2659541, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "02/20/2024 00:21:47", "content": "

May I ask which CV has the best performance in this competition?

", "votes": null, "replies": [ { "id": 2659777, "author_name": "narsil", "author_url": "", "post_date": "02/20/2024 06:04:11", "content": "

I will go with recommendation from #1 place, which is to just use weeks since 60 WEEKNUM for validation

", "votes": null, "replies": [] } ] }, { "id": 2659673, "author_name": "ravi20076", "author_url": "", "post_date": "02/20/2024 04:42:01", "content": "

@narsil from my initial baseline approaches, I concur that the public leaderboard is very unstable and is highly sensitive to the weights used across the models in the blend. This is likely to lead to a big churn at the end. What do you have to say in this regard?

", "votes": null, "replies": [ { "id": 2659781, "author_name": "narsil", "author_url": "", "post_date": "02/20/2024 06:06:17", "content": "

If you mean big shakeup - difficult to say because we have 2 opposing forces at play here:

\n", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2658775": "As I've discussed [here](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477228), the metric focused on stability promotes ensembling of many diverse models.\n\nIn this topic, I wanted to ask about the best *single* model.\n\nMine is LGBM, AUC cv 0.855, LB 0.591 (after metric hacking).", "2658821": "Mine is a NN with AUC cv of 0.822 and LB 0.576 (also with metric hacking)", "2659541": "May I ask which CV has the best performance in this competition?", "2659673": "narsil from my initial baseline approaches, I concur that the public leaderboard is very unstable and is highly sensitive to the weights used across the models in the blend. This is likely to lead to a big churn at the end. What do you have to say in this regard?", "2659777": "I will go with recommendation from #1 place, which is to just use weeks since 60 `WEEKNUM` for validation", "2659781": "If you mean big shakeup - difficult to say because we have 2 opposing forces at play here:\n- public vs private split is a mystery -> increasing the magnitude of potential shakeup\n- competition promotes stability - blending of many models -> decreasing the magnitude of shakeup" }, "source": "meta" }