![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8273630%2F24fb26e82c0a07bb8555b2e3b0cdeb39%2FLB.png?generation=1734654183971334&alt=media\")
Hello all,
\nI think this is the last featured competition to complete in the calendar year 2024. As expected, this one has ended up with an unimaginable level of churn and a tumultous ride and finish for many of us! Let's delve into the churn plot and gather a few takeaways!
\n
The whole plot is interesting, but let's focus on 4 regions to start with!
\n| Region Label | \nComments | \n
|---|---|
| R1 | \n* Stable teams across both the leaderboards - congrats! * This is a huge success in my opinion, to win a medal and stay afloat across both the leaderboards! | \n
| R2 | \n* This indicates the teams falling off the perch in the private leaderboard from a top 10% rank on the public leaderboard * This was expected given the nature of the top public notebooks, luck-driven blends and a lot of random-state driven solutions that made their way to the top | \n
| R3 | \n* Congratulations for moving up in the churn! * Most of the gold medalists in the competition belong to this group! * This is a pretty thick pool of participants, indicating the extent of the churn! | \n
| R4 | \n* This overlaps with R2 and is an interesting region * Is this driven by a public kernel? Comments! | \n
| Medal zone | \nLast rank | \nComments | \nStability Ratio | \n
|---|---|---|---|
| Gold medal | \n17 | \n* Stable teams = 0 * Entering from below = 17 | \n0/ 17 = 0% | \n
| Silver medal | \n181 | \n* Stable teams = 4 * Entering from below = 159 * Entering from above = 1 | \n4 / 164 = 2.43% | \n
| Bronze medal | \n362 | \n* Stable teams = 4 * Entering from below = 167 * Entering from above = 10 | \n4 / 181 = 2.20% | \n
Only 8 teams in the entire top 10% finishers were able to stay stable! Incredible!!! I am one of the 8 stable teams here and am quite lucky!
| Event | \nPublic LB rank | \nPrivate LB rank | \n
|---|---|---|
| Highest upward move on the LB | \n2754 | \n58 | \n
| Biggest fall on the LB | \n299 | \n3202 | \n
| Highest upward move on the LB and finishing in the top 10% | \n2754 | \n58 | \n
| Biggest fall on the LB and finishing in the top 10% | \n7 | \n290 | \n
Wishing you the best for your future competitions in 2025! Happy learning and festive period and enjoy the Kaggle ride!
\nBest regards,
\nRavi Ramakrishnan
Hello all,
\nI think this is the last featured competition to complete in the calendar year 2024. As expected, this one has ended up with an unimaginable level of churn and a tumultous ride and finish for many of us! Let's delve into the churn plot and gather a few takeaways!
\n
The whole plot is interesting, but let's focus on 4 regions to start with!
\n| Region Label | \nComments | \n
|---|---|
| R1 | \n* Stable teams across both the leaderboards - congrats! * This is a huge success in my opinion, to win a medal and stay afloat across both the leaderboards! | \n
| R2 | \n* This indicates the teams falling off the perch in the private leaderboard from a top 10% rank on the public leaderboard * This was expected given the nature of the top public notebooks, luck-driven blends and a lot of random-state driven solutions that made their way to the top | \n
| R3 | \n* Congratulations for moving up in the churn! * Most of the gold medalists in the competition belong to this group! * This is a pretty thick pool of participants, indicating the extent of the churn! | \n
| R4 | \n* This overlaps with R2 and is an interesting region * Is this driven by a public kernel? Comments! | \n
| Medal zone | \nLast rank | \nComments | \nStability Ratio | \n
|---|---|---|---|
| Gold medal | \n17 | \n* Stable teams = 0 * Entering from below = 17 | \n0/ 17 = 0% | \n
| Silver medal | \n181 | \n* Stable teams = 4 * Entering from below = 159 * Entering from above = 1 | \n4 / 164 = 2.43% | \n
| Bronze medal | \n362 | \n* Stable teams = 4 * Entering from below = 167 * Entering from above = 10 | \n4 / 181 = 2.20% | \n
Only 8 teams in the entire top 10% finishers were able to stay stable! Incredible!!! I am one of the 8 stable teams here and am quite lucky!
| Event | \nPublic LB rank | \nPrivate LB rank | \n
|---|---|---|
| Highest upward move on the LB | \n2754 | \n58 | \n
| Biggest fall on the LB | \n299 | \n3202 | \n
| Highest upward move on the LB and finishing in the top 10% | \n2754 | \n58 | \n
| Biggest fall on the LB and finishing in the top 10% | \n7 | \n290 | \n
Wishing you the best for your future competitions in 2025! Happy learning and festive period and enjoy the Kaggle ride!
\nBest regards,
\nRavi Ramakrishnan
I'm in total shock. I didn't expect the shake-up to be this much in my favor. 😯 Currently got a cold, so a write-up on my (mostly lucky) solution will take a bit longer.
\nIn the meantime, here is my notebook. Which is based on a notebook I shared early on in the competition.
", "rawMarkdown": "I'm in total shock. I didn't expect the shake-up to be this much in my favor. 😯 Currently got a cold, so a write-up on my (mostly lucky) solution will take a bit longer.\n\nIn the meantime, here is my [notebook](https://www.kaggle.com/code/lennarthaupts/1st-place-cmi-model-v4-1-1-reduced). Which is based on a [notebook](https://www.kaggle.com/code/lennarthaupts/cmi-detecting-problematic-digital-behavior) I shared early on in the competition.", "votes": 14, "replies": [ { "id": 3076474, "postDate": "2024-12-20T01:35:38.583Z", "content": "Wow, your CV score and your private LB score are in sync! Getting a 0.479+ CV is great here @lennarthaupts
", "rawMarkdown": "Wow, your CV score and your private LB score are in sync! Getting a 0.479+ CV is great here @lennarthaupts ", "votes": 2 }, { "id": 3076582, "postDate": "2024-12-20T05:09:04.797Z", "content": "Congrats Lennart! 🥳 Looking forward to the write-up!
", "rawMarkdown": "Congrats Lennart! 🥳 Looking forward to the write-up!", "votes": 2 }, { "id": 3077394, "postDate": "2024-12-20T22:49:06.360Z", "content": "I second this.
", "rawMarkdown": "I second this.", "votes": 2 } ] }, { "id": 3076413, "postDate": "2024-12-20T00:46:21.687Z", "content": "For anyone wondering what happened. The metric forced us to separate balls in a pit by color with two straight lines, but you get your score after someone jumped into it, mixing the balls, while your seperation lines stay the same :)
\n![\"pit\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F20325352%2F2e0061cb531bf4a1faaefd4619675cb7%2Fball_pit.PNG?generation=1734655308807082&alt=media\")
A very fun plot with lots of structure!
\nRe: R4: Maybe this comes from teams that LB-submitted a forked version of a top notebook (lining up in rank) and then left that as their Private submission so that they are still lined up but now at lower ranks?
\n(There a bit of an apples-oranges aspect of the plot: the team's model used for the Public LB rank (highest submitted) is not the same as the team's model selected for the Private rank?)
", "rawMarkdown": "A very fun plot with lots of structure!\n\nRe: R4: Maybe this comes from teams that LB-submitted a forked version of a top notebook (lining up in rank) and then left that as their Private submission so that they are still lined up but now at lower ranks?\n\n(There a bit of an apples-oranges aspect of the plot: the team's model used for the Public LB rank (highest submitted) is not the same as the team's model selected for the Private rank?)", "votes": 1, "replies": [ { "id": 3079797, "postDate": "2024-12-24T06:27:01.203Z", "content": "True @dan3dewey
\nThe same applies to me as well- I simply forked the 0.494 notebook knowing well that I will not even consider it for my final submission.
\nMy final submissions would have placed me below rank 2000 on the public LB.
The world upside down! Thanks @ravi20076 for your analysis
", "rawMarkdown": "The world upside down! Thanks @ravi20076 for your analysis", "votes": 1 }, { "id": 3076819, "postDate": "2024-12-20T09:45:51.823Z", "content": "Thanks for sharing this wrap-up! I have a quick question: in the event of an exact same score (e.g., using a shared public kernel), how does the ranking system work? Is it based on the time of submission?
", "rawMarkdown": "Thanks for sharing this wrap-up! I have a quick question: in the event of an exact same score (e.g., using a shared public kernel), how does the ranking system work? Is it based on the time of submission?", "votes": 1, "replies": [ { "id": 3076878, "postDate": "2024-12-20T11:02:38.853Z", "content": "Yes early submission gets a better rank @hamedabedi
", "rawMarkdown": "Yes early submission gets a better rank @hamedabedi ", "votes": 1 } ] }, { "id": 3076751, "postDate": "2024-12-20T08:27:12.253Z", "content": "What a wonderful randomizer happened in the final. Hope organizers will not put in to production models with several submissions only. Does it make sense to get an insights from the competition results except \"result absence is also a result\".
", "rawMarkdown": "What a wonderful randomizer happened in the final. Hope organizers will not put in to production models with several submissions only. Does it make sense to get an insights from the competition results except \"result absence is also a result\". ", "votes": 1, "replies": [ { "id": 3076762, "postDate": "2024-12-20T08:36:17.360Z", "content": "I wonder what CMI gained from this assignment @samvelkoch?
\nWe can't say that the winning model is great and can't opine that an average model is bad. It is just a game of luck!
That is a question CMI could answer. They have best perspective view to the topic and spend at least $60000 for the prizes. Maybe @gkiar07 will clarify something for the Kaggle community. Personally I'm very interested in the topic and cases how organizations decided to come to the Kaggle, what are the goals they want to achieve, how competitions results answer the questions and if they implement something in future.
", "rawMarkdown": "That is a question CMI could answer. They have best perspective view to the topic and spend at least $60000 for the prizes. Maybe @gkiar07 will clarify something for the Kaggle community. Personally I'm very interested in the topic and cases how organizations decided to come to the Kaggle, what are the goals they want to achieve, how competitions results answer the questions and if they implement something in future. ", "votes": 1 } ] } ] }, { "id": 3076412, "postDate": "2024-12-20T00:46:13.617Z", "content": "Good analysis, but I think successful people probably don't study open top notebooks in last-minute submissions
", "rawMarkdown": "Good analysis, but I think successful people probably don't study open top notebooks in last-minute submissions", "votes": 1, "replies": [ { "id": 3076415, "postDate": "2024-12-20T00:47:32.917Z", "content": "In this type of problem, 99% of the score improvement caused by fine-tuning the threshold or hyperparameter in the final stage of the race is overfitting
", "rawMarkdown": "In this type of problem, 99% of the score improvement caused by fine-tuning the threshold or hyperparameter in the final stage of the race is overfitting", "votes": 1, "replies": [ { "id": 3076785, "postDate": "2024-12-20T09:04:34.977Z", "content": "\"That's a great point! Fine-tuning thresholds or hyperparameters in the final stage does risk overfitting, especially when relying too heavily on the leaderboard feedback. I tried to mitigate this by focusing on robust cross-validation strategies and ensuring my model generalized well across different data splits. Ultimately, it’s a delicate balance between optimizing for the competition and maintaining a realistic, deployable solution. Thanks for bringing this up!\"
", "rawMarkdown": "\"That's a great point! Fine-tuning thresholds or hyperparameters in the final stage does risk overfitting, especially when relying too heavily on the leaderboard feedback. I tried to mitigate this by focusing on robust cross-validation strategies and ensuring my model generalized well across different data splits. Ultimately, it’s a delicate balance between optimizing for the competition and maintaining a realistic, deployable solution. Thanks for bringing this up!\"", "votes": 1 } ] } ] }, { "id": 3076618, "postDate": "2024-12-20T05:46:47.200Z", "content": "Honesty is a key. It's necessary to hold your own original way and take trade-offs into account.
", "rawMarkdown": "Honesty is a key. It's necessary to hold your own original way and take trade-offs into account.", "votes": 2 }, { "id": 3076449, "postDate": "2024-12-20T01:07:43.377Z", "content": "I used a stable model and did not choose an overfitting model, which helped me get a bronze medal. I would like to thank you for your previous guidance.
", "rawMarkdown": "I used a stable model and did not choose an overfitting model, which helped me get a bronze medal. I would like to thank you for your previous guidance.", "votes": 2, "replies": [ { "id": 3076466, "postDate": "2024-12-20T01:29:14.027Z", "content": "Most welcome my friend @jiaoyouzhang
", "rawMarkdown": "Most welcome my friend @jiaoyouzhang ", "votes": 1 } ] }, { "id": 3076614, "postDate": "2024-12-20T05:42:23.970Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 3076615, "postDate": "2024-12-20T05:44:53.040Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 3076468, "author_name": "Lennart Haupts", "author_url": "", "post_date": "2024-12-20T01:31:04.747000", "content": "I'm in total shock. I didn't expect the shake-up to be this much in my favor. 😯 Currently got a cold, so a write-up on my (mostly lucky) solution will take a bit longer.
\nIn the meantime, here is my notebook. Which is based on a notebook I shared early on in the competition.
", "votes": 14, "replies": [ { "id": 3076474, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-12-20T01:35:38.583000", "content": "Wow, your CV score and your private LB score are in sync! Getting a 0.479+ CV is great here @lennarthaupts
", "votes": 2, "replies": [] }, { "id": 3076582, "author_name": "Fabian Henning", "author_url": "", "post_date": "2024-12-20T05:09:04.797000", "content": "Congrats Lennart! 🥳 Looking forward to the write-up!
", "votes": 2, "replies": [] }, { "id": 3077394, "author_name": "Aradhye Agarwal", "author_url": "", "post_date": "2024-12-20T22:49:06.360000", "content": "I second this.
", "votes": 2, "replies": [] } ] }, { "id": 3076413, "author_name": "Marius Heuser", "author_url": "", "post_date": "2024-12-20T00:46:21.687000", "content": "For anyone wondering what happened. The metric forced us to separate balls in a pit by color with two straight lines, but you get your score after someone jumped into it, mixing the balls, while your seperation lines stay the same :)
\n
A very fun plot with lots of structure!
\nRe: R4: Maybe this comes from teams that LB-submitted a forked version of a top notebook (lining up in rank) and then left that as their Private submission so that they are still lined up but now at lower ranks?
\n(There a bit of an apples-oranges aspect of the plot: the team's model used for the Public LB rank (highest submitted) is not the same as the team's model selected for the Private rank?)
", "votes": 1, "replies": [ { "id": 3079797, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-12-24T06:27:01.203000", "content": "True @dan3dewey
\nThe same applies to me as well- I simply forked the 0.494 notebook knowing well that I will not even consider it for my final submission.
\nMy final submissions would have placed me below rank 2000 on the public LB.
The world upside down! Thanks @ravi20076 for your analysis
", "votes": 1, "replies": [] }, { "id": 3076819, "author_name": "Hamed Abedi", "author_url": "", "post_date": "2024-12-20T09:45:51.823000", "content": "Thanks for sharing this wrap-up! I have a quick question: in the event of an exact same score (e.g., using a shared public kernel), how does the ranking system work? Is it based on the time of submission?
", "votes": 1, "replies": [ { "id": 3076878, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-12-20T11:02:38.853000", "content": "Yes early submission gets a better rank @hamedabedi
", "votes": 1, "replies": [] } ] }, { "id": 3076751, "author_name": "Samvel Kocharyan", "author_url": "", "post_date": "2024-12-20T08:27:12.253000", "content": "What a wonderful randomizer happened in the final. Hope organizers will not put in to production models with several submissions only. Does it make sense to get an insights from the competition results except \"result absence is also a result\".
", "votes": 1, "replies": [ { "id": 3076762, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-12-20T08:36:17.360000", "content": "I wonder what CMI gained from this assignment @samvelkoch?
\nWe can't say that the winning model is great and can't opine that an average model is bad. It is just a game of luck!
That is a question CMI could answer. They have best perspective view to the topic and spend at least $60000 for the prizes. Maybe @gkiar07 will clarify something for the Kaggle community. Personally I'm very interested in the topic and cases how organizations decided to come to the Kaggle, what are the goals they want to achieve, how competitions results answer the questions and if they implement something in future.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3076412, "author_name": "Reki", "author_url": "", "post_date": "2024-12-20T00:46:13.617000", "content": "Good analysis, but I think successful people probably don't study open top notebooks in last-minute submissions
", "votes": 1, "replies": [ { "id": 3076415, "author_name": "Reki", "author_url": "", "post_date": "2024-12-20T00:47:32.917000", "content": "In this type of problem, 99% of the score improvement caused by fine-tuning the threshold or hyperparameter in the final stage of the race is overfitting
", "votes": 1, "replies": [ { "id": 3076785, "author_name": "Sudhansu_IISC_Bangalore", "author_url": "", "post_date": "2024-12-20T09:04:34.977000", "content": "\"That's a great point! Fine-tuning thresholds or hyperparameters in the final stage does risk overfitting, especially when relying too heavily on the leaderboard feedback. I tried to mitigate this by focusing on robust cross-validation strategies and ensuring my model generalized well across different data splits. Ultimately, it’s a delicate balance between optimizing for the competition and maintaining a realistic, deployable solution. Thanks for bringing this up!\"
", "votes": 1, "replies": [] } ] } ] }, { "id": 3076618, "author_name": "Vladimir Demidov", "author_url": "", "post_date": "2024-12-20T05:46:47.200000", "content": "Honesty is a key. It's necessary to hold your own original way and take trade-offs into account.
", "votes": 2, "replies": [] }, { "id": 3076449, "author_name": "Less", "author_url": "", "post_date": "2024-12-20T01:07:43.377000", "content": "I used a stable model and did not choose an overfitting model, which helped me get a bronze medal. I would like to thank you for your previous guidance.
", "votes": 2, "replies": [ { "id": 3076466, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-12-20T01:29:14.027000", "content": "Most welcome my friend @jiaoyouzhang
", "votes": 1, "replies": [] } ] }, { "id": 3076614, "author_name": "", "author_url": "", "post_date": "2024-12-20T05:42:23.970000", "content": "", "votes": 1, "replies": [ { "id": 3076615, "author_name": "", "author_url": "", "post_date": "2024-12-20T05:44:53.040000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "3076409": "Hello all,\n\nI think this is the last featured competition to complete in the calendar year 2024. As expected, this one has ended up with an unimaginable level of churn and a tumultous ride and finish for many of us! Let's delve into the churn plot and gather a few takeaways!\n\n\n\n# Interesting regions\nThe whole plot is interesting, but let's focus on 4 regions to start with!\n\n| Region Label | Comments |\n| --- | --- |\n| R1 | * Stable teams across both the leaderboards - congrats!