{ "id": 551458, "title": "Are we heading the Home Credit route or the ICR way?", "url": "/competitions/child-mind-institute-problematic-internet-use/discussion/551458", "author_name": "", "post_date": "2024-12-13T08:33:28.856454900Z", "votes": 18, "comment_count": 14, "views": 0, "content": "

Hello all,

\n

Considering the surge in public scores over the past 12-24 hours on the leaderboard, I wonder if this competition is headed the Home Credit - Credit Risk Model Stability way or the ICR - Identifying Age-Related Conditions way.

\n

Both competitions ended up with almost flipped leaderboard with the high scoring public submissions finding themselves in the bottom half of the private LB.

\n

The churn in the Home Credit competition is visualized here

\n

Wishing you the best for a crucial week ahead and a good choice for the final submissions.

\n

All the best!

", "messages": [ { "id": "3071010", "postDate": "12/13/2024 08:33:28", "content": "

Hello all,

\n

Considering the surge in public scores over the past 12-24 hours on the leaderboard, I wonder if this competition is headed the Home Credit - Credit Risk Model Stability way or the ICR - Identifying Age-Related Conditions way.

\n

Both competitions ended up with almost flipped leaderboard with the high scoring public submissions finding themselves in the bottom half of the private LB.

\n

The churn in the Home Credit competition is visualized here

\n

Wishing you the best for a crucial week ahead and a good choice for the final submissions.

\n

All the best!

", "rawMarkdown": "Hello all,\n\nConsidering the surge in public scores over the past 12-24 hours on the leaderboard, I wonder if this competition is headed the [Home Credit - Credit Risk Model Stability](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability) way or the [ICR - Identifying Age-Related Conditions](https://www.kaggle.com/competitions/icr-identify-age-related-conditions) way. \n\nBoth competitions ended up with almost flipped leaderboard with the high scoring public submissions finding themselves in the bottom half of the private LB. \n\nThe churn in the Home Credit competition is visualized [here](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/507951)\n\nWishing you the best for a crucial week ahead and a good choice for the final submissions. \n\nAll the best!", "votes": null }, { "id": "3071046", "postDate": "12/13/2024 09:24:38", "content": "

There is no way to tell how it ends. I think it is quite certain, that there is alot of noise and variance.
\nAnd public LB and private LB are probably unrelated. But that doesnt mean that the leaderboard will flip. Could be that first place now goes down 1000 places and 5th places goes up to 1. The score doesn't represent how the model will behave on the private LB. It's alot of luck sadly but that also makes it quite exciting for a beginner like myself. I will be disappointed if a random 1 submission model places higher than myself, but that's what i signed up for :)

", "rawMarkdown": "There is no way to tell how it ends. I think it is quite certain, that there is alot of noise and variance.\nAnd public LB and private LB are probably unrelated. But that doesnt mean that the leaderboard will flip. Could be that first place now goes down 1000 places and 5th places goes up to 1. The score doesn't represent how the model will behave on the private LB. It's alot of luck sadly but that also makes it quite exciting for a beginner like myself. I will be disappointed if a random 1 submission model places higher than myself, but that's what i signed up for :)", "votes": null }, { "id": "3071109", "postDate": "12/13/2024 10:50:06", "content": "

The metric seems to have an issue but it is no way exploitable like Home Credit Competition. Also, there seems to some shift in data distribution in the test set. I am guessing this is going to be ICR season 3 :)

", "rawMarkdown": "The metric seems to have an issue but it is no way exploitable like Home Credit Competition. Also, there seems to some shift in data distribution in the test set. I am guessing this is going to be ICR season 3 :)", "votes": null }, { "id": "3071120", "postDate": "12/13/2024 11:16:04", "content": "

What is ICR season 2 @basu1999 ? Home credit?

", "rawMarkdown": "What is ICR season 2 @basu1999 ? Home credit?", "votes": null }, { "id": "3071123", "postDate": "12/13/2024 11:21:50", "content": "

ISIC 2024 @ravi20076

", "rawMarkdown": "ISIC 2024 @ravi20076", "votes": null }, { "id": "3071145", "postDate": "12/13/2024 12:02:28", "content": "

Despite all other factors like overfitting and QWK evaluation etc this is in for a shake up given that all top 1000 teams are differing by the third decimal and private test set will have around 2400 samples only.

", "rawMarkdown": "Despite all other factors like overfitting and QWK evaluation etc this is in for a shake up given that all top 1000 teams are differing by the third decimal and private test set will have around 2400 samples only.", "votes": null }, { "id": "3071154", "postDate": "12/13/2024 12:13:41", "content": "

Though I have only participated in the Home Credit competition (only read about the ICR), I suspect the outcome will be more similar to the ICR one. Also, unlike the Home Credit competition, the dataset is very small.

\n

After the Home Credit outcome, that's the reason I did not give much effort for this competition: custom metric plus a very small dataset will result to a significant shakeup…

", "rawMarkdown": "Though I have only participated in the Home Credit competition (only read about the ICR), I suspect the outcome will be more similar to the ICR one. Also, unlike the Home Credit competition, the dataset is very small. \n\nAfter the Home Credit outcome, that's the reason I did not give much effort for this competition: custom metric plus a very small dataset will result to a significant shakeup...", "votes": null }, { "id": "3071241", "postDate": "12/13/2024 14:20:29", "content": "

Oh yes
\nI was there too @basu1999

", "rawMarkdown": "Oh yes \nI was there too @basu1999", "votes": null }, { "id": "3071284", "postDate": "12/13/2024 15:24:11", "content": "

I think it is a lottery game:)

", "rawMarkdown": "I think it is a lottery game:)", "votes": null }, { "id": "3071560", "postDate": "12/13/2024 23:38:55", "content": "

Keep in mind that we each pick two of our submissions to be evaluated for the private LB -- these two selected need not include our highest scoring LB model. For example, I don't think your ( @ravi20076 ) submission with LB 0.497 will be one of the two you select 🙃

", "rawMarkdown": "Keep in mind that we each pick two of our submissions to be evaluated for the private LB -- these two selected need not include our highest scoring LB model. For example, I don't think your ( @ravi20076 ) submission with LB 0.497 will be one of the two you select 🙃", "votes": null }, { "id": "3071893", "postDate": "12/14/2024 13:20:43", "content": "

Everyone has given up shaking it anyway, but shouldn't we be making a better effort to focus on increasing the robustness of the model?

", "rawMarkdown": "Everyone has given up shaking it anyway, but shouldn't we be making a better effort to focus on increasing the robustness of the model?", "votes": null }, { "id": "3071918", "postDate": "12/14/2024 14:10:29", "content": "

In my personal opinion, several solutions with high PUBLIC SCORE have obvious problems with their feature engineering and QWK calculations, a GM user has already posted about this in the discussion forum, but of course that's just my personal opinion, and I'm probably too limited in my thinking.

", "rawMarkdown": "In my personal opinion, several solutions with high PUBLIC SCORE have obvious problems with their feature engineering and QWK calculations, a GM user has already posted about this in the discussion forum, but of course that's just my personal opinion, and I'm probably too limited in my thinking.", "votes": null }, { "id": "3072911", "postDate": "12/15/2024 18:58:27", "content": "

I believe that it's not a lottery game, the current leaderboard makes our think so. Of course, It's not the same way as ICR, but also not the same way as Home Credit. Trust CV is all we need.

", "rawMarkdown": "I believe that it's not a lottery game, the current leaderboard makes our think so. Of course, It's not the same way as ICR, but also not the same way as Home Credit. Trust CV is all we need.", "votes": null }, { "id": "3073152", "postDate": "12/16/2024 05:36:19", "content": "

@bibanh I am doing this as of now and hoping it gets me the results 3 days later

", "rawMarkdown": "bibanh I am doing this as of now and hoping it gets me the results 3 days later", "votes": null }, { "id": "3074455", "postDate": "12/17/2024 17:31:00", "content": "

Exactly. We don't need to overthink what the final scores will be. Need to focus your preprocessing and the robustness of your model. @sfukushi

", "rawMarkdown": "Exactly. We don't need to overthink what the final scores will be. Need to focus your preprocessing and the robustness of your model. @sfukushi", "votes": null } ], "comments": [ { "id": 3071046, "author_name": "mariusheuser", "author_url": "", "post_date": "12/13/2024 09:24:38", "content": "

There is no way to tell how it ends. I think it is quite certain, that there is alot of noise and variance.
\nAnd public LB and private LB are probably unrelated. But that doesnt mean that the leaderboard will flip. Could be that first place now goes down 1000 places and 5th places goes up to 1. The score doesn't represent how the model will behave on the private LB. It's alot of luck sadly but that also makes it quite exciting for a beginner like myself. I will be disappointed if a random 1 submission model places higher than myself, but that's what i signed up for :)

", "votes": null, "replies": [] }, { "id": 3071109, "author_name": "basu1999", "author_url": "", "post_date": "12/13/2024 10:50:06", "content": "

The metric seems to have an issue but it is no way exploitable like Home Credit Competition. Also, there seems to some shift in data distribution in the test set. I am guessing this is going to be ICR season 3 :)

", "votes": null, "replies": [ { "id": 3071120, "author_name": "ravi20076", "author_url": "", "post_date": "12/13/2024 11:16:04", "content": "

What is ICR season 2 @basu1999 ? Home credit?

", "votes": null, "replies": [ { "id": 3071123, "author_name": "basu1999", "author_url": "", "post_date": "12/13/2024 11:21:50", "content": "

ISIC 2024 @ravi20076

", "votes": null, "replies": [ { "id": 3071241, "author_name": "ravi20076", "author_url": "", "post_date": "12/13/2024 14:20:29", "content": "

Oh yes
\nI was there too @basu1999

", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3071145, "author_name": "bhatiji", "author_url": "", "post_date": "12/13/2024 12:02:28", "content": "

Despite all other factors like overfitting and QWK evaluation etc this is in for a shake up given that all top 1000 teams are differing by the third decimal and private test set will have around 2400 samples only.

", "votes": null, "replies": [] }, { "id": 3071154, "author_name": "andreasbis", "author_url": "", "post_date": "12/13/2024 12:13:41", "content": "

Though I have only participated in the Home Credit competition (only read about the ICR), I suspect the outcome will be more similar to the ICR one. Also, unlike the Home Credit competition, the dataset is very small.

\n

After the Home Credit outcome, that's the reason I did not give much effort for this competition: custom metric plus a very small dataset will result to a significant shakeup…

", "votes": null, "replies": [] }, { "id": 3071284, "author_name": "algerwang", "author_url": "", "post_date": "12/13/2024 15:24:11", "content": "

I think it is a lottery game:)

", "votes": null, "replies": [] }, { "id": 3071560, "author_name": "dan3dewey", "author_url": "", "post_date": "12/13/2024 23:38:55", "content": "

Keep in mind that we each pick two of our submissions to be evaluated for the private LB -- these two selected need not include our highest scoring LB model. For example, I don't think your ( @ravi20076 ) submission with LB 0.497 will be one of the two you select 🙃

", "votes": null, "replies": [] }, { "id": 3071893, "author_name": "sfukushi", "author_url": "", "post_date": "12/14/2024 13:20:43", "content": "

Everyone has given up shaking it anyway, but shouldn't we be making a better effort to focus on increasing the robustness of the model?

", "votes": null, "replies": [ { "id": 3074455, "author_name": "trcnveli", "author_url": "", "post_date": "12/17/2024 17:31:00", "content": "

Exactly. We don't need to overthink what the final scores will be. Need to focus your preprocessing and the robustness of your model. @sfukushi

", "votes": null, "replies": [] } ] }, { "id": 3071918, "author_name": "aristotlechen", "author_url": "", "post_date": "12/14/2024 14:10:29", "content": "

In my personal opinion, several solutions with high PUBLIC SCORE have obvious problems with their feature engineering and QWK calculations, a GM user has already posted about this in the discussion forum, but of course that's just my personal opinion, and I'm probably too limited in my thinking.

", "votes": null, "replies": [] }, { "id": 3072911, "author_name": "bibanh", "author_url": "", "post_date": "12/15/2024 18:58:27", "content": "

I believe that it's not a lottery game, the current leaderboard makes our think so. Of course, It's not the same way as ICR, but also not the same way as Home Credit. Trust CV is all we need.

", "votes": null, "replies": [ { "id": 3073152, "author_name": "ravi20076", "author_url": "", "post_date": "12/16/2024 05:36:19", "content": "

@bibanh I am doing this as of now and hoping it gets me the results 3 days later

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3071010": "Hello all,\n\nConsidering the surge in public scores over the past 12-24 hours on the leaderboard, I wonder if this competition is headed the [Home Credit - Credit Risk Model Stability](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability) way or the [ICR - Identifying Age-Related Conditions](https://www.kaggle.com/competitions/icr-identify-age-related-conditions) way. \n\nBoth competitions ended up with almost flipped leaderboard with the high scoring public submissions finding themselves in the bottom half of the private LB. \n\nThe churn in the Home Credit competition is visualized [here](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/507951)\n\nWishing you the best for a crucial week ahead and a good choice for the final submissions. \n\nAll the best!", "3071046": "There is no way to tell how it ends. I think it is quite certain, that there is alot of noise and variance.\nAnd public LB and private LB are probably unrelated. But that doesnt mean that the leaderboard will flip. Could be that first place now goes down 1000 places and 5th places goes up to 1. The score doesn't represent how the model will behave on the private LB. It's alot of luck sadly but that also makes it quite exciting for a beginner like myself. I will be disappointed if a random 1 submission model places higher than myself, but that's what i signed up for :)", "3071109": "The metric seems to have an issue but it is no way exploitable like Home Credit Competition. Also, there seems to some shift in data distribution in the test set. I am guessing this is going to be ICR season 3 :)", "3071120": "What is ICR season 2 @basu1999 ? Home credit?", "3071123": "ISIC 2024 @ravi20076", "3071145": "Despite all other factors like overfitting and QWK evaluation etc this is in for a shake up given that all top 1000 teams are differing by the third decimal and private test set will have around 2400 samples only.", "3071154": "Though I have only participated in the Home Credit competition (only read about the ICR), I suspect the outcome will be more similar to the ICR one. Also, unlike the Home Credit competition, the dataset is very small. \n\nAfter the Home Credit outcome, that's the reason I did not give much effort for this competition: custom metric plus a very small dataset will result to a significant shakeup...", "3071241": "Oh yes \nI was there too @basu1999", "3071284": "I think it is a lottery game:)", "3071560": "Keep in mind that we each pick two of our submissions to be evaluated for the private LB -- these two selected need not include our highest scoring LB model. For example, I don't think your ( @ravi20076 ) submission with LB 0.497 will be one of the two you select 🙃", "3071893": "Everyone has given up shaking it anyway, but shouldn't we be making a better effort to focus on increasing the robustness of the model?", "3071918": "In my personal opinion, several solutions with high PUBLIC SCORE have obvious problems with their feature engineering and QWK calculations, a GM user has already posted about this in the discussion forum, but of course that's just my personal opinion, and I'm probably too limited in my thinking.", "3072911": "I believe that it's not a lottery game, the current leaderboard makes our think so. Of course, It's not the same way as ICR, but also not the same way as Home Credit. Trust CV is all we need.", "3073152": "bibanh I am doing this as of now and hoping it gets me the results 3 days later", "3074455": "Exactly. We don't need to overthink what the final scores will be. Need to focus your preprocessing and the robustness of your model. @sfukushi" }, "source": "meta" }