Many kagglers share their concerns about the eval metric and offer profound insights, but the host still chooses not to replace the metric and allow metric hacking to exist?
\nIt’s crazy. Or maybe I haven't participated in enough competitions.
", "messages": [ { "id": "2798175", "postDate": "05/07/2024 06:19:58", "content": "Many kagglers share their concerns about the eval metric and offer profound insights, but the host still chooses not to replace the metric and allow metric hacking to exist?
\nIt’s crazy. Or maybe I haven't participated in enough competitions.
", "rawMarkdown": "Many kagglers share their concerns about the eval metric and offer profound insights, but the host still chooses not to replace the metric and allow metric hacking to exist?\n\nIt’s crazy. Or maybe I haven't participated in enough competitions.", "votes": null }, { "id": "2798366", "postDate": "05/07/2024 07:44:25", "content": "…and people say that Kaggle is nothing like the RealWorld™ 🤣
", "rawMarkdown": "...and people say that Kaggle is nothing like the RealWorld™ 🤣", "votes": null }, { "id": "2798883", "postDate": "05/07/2024 13:34:43", "content": "This is the worst competition I have been part of, and trust me, I have been in a lot. Terrible way to host one, this is how you kill a product.
", "rawMarkdown": "This is the worst competition I have been part of, and trust me, I have been in a lot. Terrible way to host one, this is how you kill a product.", "votes": null }, { "id": "2799261", "postDate": "05/07/2024 16:56:52", "content": "it is was it is !! problem is so real not metric 😂
", "rawMarkdown": "it is was it is !! problem is so real not metric 😂", "votes": null }, { "id": "2800468", "postDate": "05/08/2024 07:49:35", "content": "haha fair enough!
", "rawMarkdown": "haha fair enough!", "votes": null }, { "id": "2802986", "postDate": "05/09/2024 09:16:46", "content": "Once I thought it would be LLM Prompt Recovery, until I saw Home Credit. LOL
", "rawMarkdown": "Once I thought it would be LLM Prompt Recovery, until I saw Home Credit. LOL", "votes": null }, { "id": "2803113", "postDate": "05/09/2024 10:45:39", "content": "I don't think the current hacking game matches the host's original purpose for holding this competition. But after that 3 week outage period, I'm convinced the host completely made a mistake.
", "rawMarkdown": "I don't think the current hacking game matches the host's original purpose for holding this competition. But after that 3 week outage period, I'm convinced the host completely made a mistake.", "votes": null }, { "id": "2803900", "postDate": "05/09/2024 17:37:31", "content": "I agree, and it's getting worse. I can't wait to see the stability approaches that these top lb are using.
", "rawMarkdown": "I agree, and it's getting worse. I can't wait to see the stability approaches that these top lb are using.", "votes": null }, { "id": "2804750", "postDate": "05/10/2024 07:27:33", "content": "This is why after the re-launch I decided don't play. Interesting data ruined by a very bad selection of metric.
", "rawMarkdown": "This is why after the re-launch I decided don't play. Interesting data ruined by a very bad selection of metric.", "votes": null }, { "id": "2805069", "postDate": "05/10/2024 10:49:11", "content": "In fact, if the public and private test data were divided by timeseries split, the metric hacking obtained with WEEK_NUM would have no meaning. But I guess it's too late for that.
", "rawMarkdown": "In fact, if the public and private test data were divided by timeseries split, the metric hacking obtained with WEEK_NUM would have no meaning. But I guess it's too late for that.", "votes": null }, { "id": "2806454", "postDate": "05/11/2024 05:25:20", "content": "Can using metric hacking in this competition really achieve the goal? Is using hacking to improve scores just for ranking purposes? I don't understand.
", "rawMarkdown": "Can using metric hacking in this competition really achieve the goal? Is using hacking to improve scores just for ranking purposes? I don't understand.", "votes": null }, { "id": "2806776", "postDate": "05/11/2024 09:33:51", "content": "I guess the most simple way to fix this is to use solely AUC. This is also the only thing we can optimize in the real world.
", "rawMarkdown": "I guess the most simple way to fix this is to use solely AUC. This is also the only thing we can optimize in the real world.", "votes": null } ], "comments": [ { "id": 2798366, "author_name": "carlmcbrideellis", "author_url": "", "post_date": "05/07/2024 07:44:25", "content": "…and people say that Kaggle is nothing like the RealWorld™ 🤣
", "votes": null, "replies": [ { "id": 2800468, "author_name": "octaviograu", "author_url": "", "post_date": "05/08/2024 07:49:35", "content": "haha fair enough!
", "votes": null, "replies": [] } ] }, { "id": 2798883, "author_name": "carloshuertas", "author_url": "", "post_date": "05/07/2024 13:34:43", "content": "This is the worst competition I have been part of, and trust me, I have been in a lot. Terrible way to host one, this is how you kill a product.
", "votes": null, "replies": [ { "id": 2802986, "author_name": "sweetyheehee", "author_url": "", "post_date": "05/09/2024 09:16:46", "content": "Once I thought it would be LLM Prompt Recovery, until I saw Home Credit. LOL
", "votes": null, "replies": [] }, { "id": 2803900, "author_name": "mehdi0694", "author_url": "", "post_date": "05/09/2024 17:37:31", "content": "I agree, and it's getting worse. I can't wait to see the stability approaches that these top lb are using.
", "votes": null, "replies": [] }, { "id": 2804750, "author_name": "blindape", "author_url": "", "post_date": "05/10/2024 07:27:33", "content": "This is why after the re-launch I decided don't play. Interesting data ruined by a very bad selection of metric.
", "votes": null, "replies": [] } ] }, { "id": 2799261, "author_name": "kunduruanil", "author_url": "", "post_date": "05/07/2024 16:56:52", "content": "it is was it is !! problem is so real not metric 😂
", "votes": null, "replies": [] }, { "id": 2803113, "author_name": "mitsuyasuhoshino", "author_url": "", "post_date": "05/09/2024 10:45:39", "content": "I don't think the current hacking game matches the host's original purpose for holding this competition. But after that 3 week outage period, I'm convinced the host completely made a mistake.
", "votes": null, "replies": [] }, { "id": 2805069, "author_name": "greysky", "author_url": "", "post_date": "05/10/2024 10:49:11", "content": "In fact, if the public and private test data were divided by timeseries split, the metric hacking obtained with WEEK_NUM would have no meaning. But I guess it's too late for that.
", "votes": null, "replies": [ { "id": 2806776, "author_name": "makeli", "author_url": "", "post_date": "05/11/2024 09:33:51", "content": "I guess the most simple way to fix this is to use solely AUC. This is also the only thing we can optimize in the real world.
", "votes": null, "replies": [] } ] }, { "id": 2806454, "author_name": "archerpan", "author_url": "", "post_date": "05/11/2024 05:25:20", "content": "Can using metric hacking in this competition really achieve the goal? Is using hacking to improve scores just for ranking purposes? I don't understand.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2798175": "Many kagglers share their concerns about the eval metric and offer profound insights, but the host still chooses not to replace the metric and allow metric hacking to exist?\n\nIt’s crazy. Or maybe I haven't participated in enough competitions.", "2798366": "...and people say that Kaggle is nothing like the RealWorld™ 🤣", "2798883": "This is the worst competition I have been part of, and trust me, I have been in a lot. Terrible way to host one, this is how you kill a product.", "2799261": "it is was it is !! problem is so real not metric 😂", "2800468": "haha fair enough!", "2802986": "Once I thought it would be LLM Prompt Recovery, until I saw Home Credit. LOL", "2803113": "I don't think the current hacking game matches the host's original purpose for holding this competition. But after that 3 week outage period, I'm convinced the host completely made a mistake.", "2803900": "I agree, and it's getting worse. I can't wait to see the stability approaches that these top lb are using.", "2804750": "This is why after the re-launch I decided don't play. Interesting data ruined by a very bad selection of metric.", "2805069": "In fact, if the public and private test data were divided by timeseries split, the metric hacking obtained with WEEK_NUM would have no meaning. But I guess it's too late for that.", "2806454": "Can using metric hacking in this competition really achieve the goal? Is using hacking to improve scores just for ranking purposes? I don't understand.", "2806776": "I guess the most simple way to fix this is to use solely AUC. This is also the only thing we can optimize in the real world." }, "source": "meta" }