{ "id": 476872, "title": "Has this competition turned into metric hacking? ", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/476872", "author_name": "", "post_date": "2024-02-13T19:40:15.105925Z", "votes": 7, "comment_count": 3, "views": 0, "content": "

Looking at the LB scores evolution, it seems to me there are many adversarial attacks on the metric, according to the awesome strategy outlined here

\n

Some attacks seem to be very successful.

\n

Has this competition turned into metric hacking, like \"Who can decrease their gini in the first weeks of evaluation in the most efficient way?\"

\n

UPDATE:

\n

Hosts made an announcement they would fix the metric

", "messages": [ { "id": "2650989", "postDate": "02/13/2024 19:40:15", "content": "

Looking at the LB scores evolution, it seems to me there are many adversarial attacks on the metric, according to the awesome strategy outlined here

\n

Some attacks seem to be very successful.

\n

Has this competition turned into metric hacking, like \"Who can decrease their gini in the first weeks of evaluation in the most efficient way?\"

\n

UPDATE:

\n

Hosts made an announcement they would fix the metric

", "rawMarkdown": "Looking at the LB scores evolution, it seems to me there are many adversarial attacks on the metric, according to the awesome strategy outlined [here](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/476449)\n\nSome attacks seem to be very successful.\n\nHas this competition turned into metric hacking, like \"Who can decrease their gini in the first weeks of evaluation in the most efficient way?\"\n\nUPDATE:\n\nHosts made [an announcement](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/476867#2650940) they would fix the metric", "votes": null }, { "id": "2651016", "postDate": "02/13/2024 20:19:40", "content": "

Host made an announcement here. They plan to address this issue.

", "rawMarkdown": "Host made an announcement [here][1]. They plan to address this issue.\n\n[1]: https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/476867", "votes": null }, { "id": "2651032", "postDate": "02/13/2024 20:39:18", "content": "

I don’t think the current LB state matters much, as the metric will be fixed in a couple of days.

", "rawMarkdown": "I don’t think the current LB state matters much, as the metric will be fixed in a couple of days.", "votes": null }, { "id": "2651041", "postDate": "02/13/2024 20:55:27", "content": "

Thanks Chris, I didn't notice it

", "rawMarkdown": "Thanks Chris, I didn't notice it", "votes": null } ], "comments": [ { "id": 2651016, "author_name": "cdeotte", "author_url": "", "post_date": "02/13/2024 20:19:40", "content": "

Host made an announcement here. They plan to address this issue.

", "votes": null, "replies": [ { "id": 2651041, "author_name": "narsil", "author_url": "", "post_date": "02/13/2024 20:55:27", "content": "

Thanks Chris, I didn't notice it

", "votes": null, "replies": [] } ] }, { "id": 2651032, "author_name": "kononenko", "author_url": "", "post_date": "02/13/2024 20:39:18", "content": "

I don’t think the current LB state matters much, as the metric will be fixed in a couple of days.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2650989": "Looking at the LB scores evolution, it seems to me there are many adversarial attacks on the metric, according to the awesome strategy outlined [here](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/476449)\n\nSome attacks seem to be very successful.\n\nHas this competition turned into metric hacking, like \"Who can decrease their gini in the first weeks of evaluation in the most efficient way?\"\n\nUPDATE:\n\nHosts made [an announcement](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/476867#2650940) they would fix the metric", "2651016": "Host made an announcement [here][1]. They plan to address this issue.\n\n[1]: https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/476867", "2651032": "I don’t think the current LB state matters much, as the metric will be fixed in a couple of days.", "2651041": "Thanks Chris, I didn't notice it" }, "source": "meta" }