Honestly, I was pretty pissed off at first when I saw that I had plummeted on the public leaderboard because of these metric tricks. Before that, I had a model ensemble with a score of 0.593. Not an outstanding achievement, but not bad at all. But, seeing how the result changes in the public leaderboard, I couldn't resist and applied the so-called “dim mak” (Death Touch, a reference to the famous scene from the movie Bloodsport :)).
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12127548%2F03c5c830368f1b6029cfb9ed4cc27658%2Fdt.PNG?generation=1716051832885637&alt=media\")
\nNow I'm already calm about the results. It's even fun to watch how quickly the leaderboard changes. It's akin to a fascinating TV series :). I'm sure that this competition will be remembered for a long time. It's already interesting to see if someone can beat 0,7.
\nHave a positive mood and stock up on popcorn, because the potential shake-up is not long away!
Honestly, I was pretty pissed off at first when I saw that I had plummeted on the public leaderboard because of these metric tricks. Before that, I had a model ensemble with a score of 0.593. Not an outstanding achievement, but not bad at all. But, seeing how the result changes in the public leaderboard, I couldn't resist and applied the so-called “dim mak” (Death Touch, a reference to the famous scene from the movie Bloodsport :)).
\n
\nNow I'm already calm about the results. It's even fun to watch how quickly the leaderboard changes. It's akin to a fascinating TV series :). I'm sure that this competition will be remembered for a long time. It's already interesting to see if someone can beat 0,7.
\nHave a positive mood and stock up on popcorn, because the potential shake-up is not long away!
Most of us are experiencing a torrid time in this competition and the last thing we may expect is a flipped leaderboard and a mighty shakeup.
\nThis easily competes with ICR challenge to be the worst competition on this platform @bratkovskyevgeny
\nProbably the host could release the competition with a smaller dataset as a playground challenge a bit later and have GINI score as the metric. I am sure that we could collectively build better models there and yield better collective learning thereby.
\nI am not sure where this pandemonium is heading. I am certainly sure that this is not ML by any means
", "rawMarkdown": "Most of us are experiencing a torrid time in this competition and the last thing we may expect is a flipped leaderboard and a mighty shakeup. \n\nThis easily competes with ICR challenge to be the worst competition on this platform @bratkovskyevgeny \n\nProbably the host could release the competition with a smaller dataset as a playground challenge a bit later and have GINI score as the metric. I am sure that we could collectively build better models there and yield better collective learning thereby.\n\nI am not sure where this pandemonium is heading. I am certainly sure that this is not ML by any means", "votes": null }, { "id": "2822718", "postDate": "05/18/2024 18:13:36", "content": "Absolutely agree @ravi20076 . I wonder how they will do a scorecard afterwards. Do they have requirements for the interpretability of the model? In my current job at a bank, interpretability is a strict requirement for scoring models, so I have to use logistic regression. This makes it easier for my colleagues to create scorecard and perform secondary validation using PSI and other metrics. And in this competition, the organizer is likely to get \"Frankenstein\" models that are hardly suitable for use in a production environment. And how to interpret them when they are probably full of blending or stacking.
", "rawMarkdown": "Absolutely agree @ravi20076 . I wonder how they will do a scorecard afterwards. Do they have requirements for the interpretability of the model? In my current job at a bank, interpretability is a strict requirement for scoring models, so I have to use logistic regression. This makes it easier for my colleagues to create scorecard and perform secondary validation using PSI and other metrics. And in this competition, the organizer is likely to get \"Frankenstein\" models that are hardly suitable for use in a production environment. And how to interpret them when they are probably full of blending or stacking.", "votes": null }, { "id": "2822848", "postDate": "05/18/2024 19:51:59", "content": "Of course nobody will beat 0.8. Current metric has upper limit of mean gini by week.
", "rawMarkdown": "Of course nobody will beat 0.8. Current metric has upper limit of mean gini by week.", "votes": null }, { "id": "2823069", "postDate": "05/19/2024 02:43:28", "content": "I am pretty new to Kaggle, rather the world of DS/ML, but my impression from hearsay was that real-world = more emphasis on interpretability and Kaggle/other competitions = more emphasis on model performance. Reading your discussion only re-emphasized it. Thank you for sharing.
\nAs far as data drift is concerned, (which will cause a model drift eventually), it should be addressed by feature engineering rather than modeling. Initially the host seemed to be caring more about feature engineering to address this, but, over time this seems to be lost!
", "rawMarkdown": "I am pretty new to Kaggle, rather the world of DS/ML, but my impression from hearsay was that real-world = more emphasis on interpretability and Kaggle/other competitions = more emphasis on model performance. Reading your discussion only re-emphasized it. Thank you for sharing.\n\nAs far as data drift is concerned, (which will cause a model drift eventually), it should be addressed by feature engineering rather than modeling. Initially the host seemed to be caring more about feature engineering to address this, but, over time this seems to be lost!", "votes": null } ], "comments": [ { "id": 2822657, "author_name": "ravi20076", "author_url": "", "post_date": "05/18/2024 17:41:21", "content": "Most of us are experiencing a torrid time in this competition and the last thing we may expect is a flipped leaderboard and a mighty shakeup.
\nThis easily competes with ICR challenge to be the worst competition on this platform @bratkovskyevgeny
\nProbably the host could release the competition with a smaller dataset as a playground challenge a bit later and have GINI score as the metric. I am sure that we could collectively build better models there and yield better collective learning thereby.
\nI am not sure where this pandemonium is heading. I am certainly sure that this is not ML by any means
", "votes": null, "replies": [ { "id": 2822718, "author_name": "bratkovskyevgeny", "author_url": "", "post_date": "05/18/2024 18:13:36", "content": "Absolutely agree @ravi20076 . I wonder how they will do a scorecard afterwards. Do they have requirements for the interpretability of the model? In my current job at a bank, interpretability is a strict requirement for scoring models, so I have to use logistic regression. This makes it easier for my colleagues to create scorecard and perform secondary validation using PSI and other metrics. And in this competition, the organizer is likely to get \"Frankenstein\" models that are hardly suitable for use in a production environment. And how to interpret them when they are probably full of blending or stacking.
", "votes": null, "replies": [ { "id": 2823069, "author_name": "varuniraothumsi", "author_url": "", "post_date": "05/19/2024 02:43:28", "content": "I am pretty new to Kaggle, rather the world of DS/ML, but my impression from hearsay was that real-world = more emphasis on interpretability and Kaggle/other competitions = more emphasis on model performance. Reading your discussion only re-emphasized it. Thank you for sharing.
\nAs far as data drift is concerned, (which will cause a model drift eventually), it should be addressed by feature engineering rather than modeling. Initially the host seemed to be caring more about feature engineering to address this, but, over time this seems to be lost!
", "votes": null, "replies": [] } ] } ] }, { "id": 2822848, "author_name": "skrrydg", "author_url": "", "post_date": "05/18/2024 19:51:59", "content": "Of course nobody will beat 0.8. Current metric has upper limit of mean gini by week.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2822564": "Honestly, I was pretty pissed off at first when I saw that I had plummeted on the public leaderboard because of these metric tricks. Before that, I had a model ensemble with a score of 0.593. Not an outstanding achievement, but not bad at all. But, seeing how the result changes in the public leaderboard, I couldn't resist and applied the so-called “dim mak” (Death Touch, a reference to the famous scene from the movie Bloodsport :)). \n\nNow I'm already calm about the results. It's even fun to watch how quickly the leaderboard changes. It's akin to a fascinating TV series :). I'm sure that this competition will be remembered for a long time. It's already interesting to see if someone can beat 0,7. \n**Have a positive mood and stock up on popcorn, because the potential shake-up is not long away!**", "2822657": "Most of us are experiencing a torrid time in this competition and the last thing we may expect is a flipped leaderboard and a mighty shakeup. \n\nThis easily competes with ICR challenge to be the worst competition on this platform @bratkovskyevgeny \n\nProbably the host could release the competition with a smaller dataset as a playground challenge a bit later and have GINI score as the metric. I am sure that we could collectively build better models there and yield better collective learning thereby.\n\nI am not sure where this pandemonium is heading. I am certainly sure that this is not ML by any means", "2822718": "Absolutely agree @ravi20076 . I wonder how they will do a scorecard afterwards. Do they have requirements for the interpretability of the model? In my current job at a bank, interpretability is a strict requirement for scoring models, so I have to use logistic regression. This makes it easier for my colleagues to create scorecard and perform secondary validation using PSI and other metrics. And in this competition, the organizer is likely to get \"Frankenstein\" models that are hardly suitable for use in a production environment. And how to interpret them when they are probably full of blending or stacking.", "2822848": "Of course nobody will beat 0.8. Current metric has upper limit of mean gini by week.", "2823069": "I am pretty new to Kaggle, rather the world of DS/ML, but my impression from hearsay was that real-world = more emphasis on interpretability and Kaggle/other competitions = more emphasis on model performance. Reading your discussion only re-emphasized it. Thank you for sharing.\n\nAs far as data drift is concerned, (which will cause a model drift eventually), it should be addressed by feature engineering rather than modeling. Initially the host seemed to be caring more about feature engineering to address this, but, over time this seems to be lost!" }, "source": "meta" }