{ "id": 492873, "title": "Regarding the 'shake' and prevention measures for this competition", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/492873", "author_name": "yunsuxiaozi", "post_date": "2024-04-11T08:26:16.193000", "votes": 8, "comment_count": 4, "views": 0, "content": "

let me explain the reason why the competition will shake:
\n1.This is a time series prediction competition, and the amount and distribution of test data are unknown,especially the test data is still during the epidemic period.
\n2.The focus of this competition is actually on predicting the stability of the model, so any slight decrease in model stability will have a significant impact on the score.
\n3.Everyone's scores in this competition are very tight.

\n

My suggestions:

\n

1.When constructing features, it is necessary to strike a balance between the effectiveness and stability of the model.If introducing a large amount of noise (such as 50 features) can only bring a slight improvement to the model's performance (CV increase of 0.001), it is better to make the model's performance worse.
\n2.Using fusion models. Using fusion models not only improves CV but also enhances model stability.

\n

good luck!

", "messages": [ { "id": 2746467, "postDate": "2024-04-11T08:26:16.193Z", "content": "

let me explain the reason why the competition will shake:
\n1.This is a time series prediction competition, and the amount and distribution of test data are unknown,especially the test data is still during the epidemic period.
\n2.The focus of this competition is actually on predicting the stability of the model, so any slight decrease in model stability will have a significant impact on the score.
\n3.Everyone's scores in this competition are very tight.

\n

My suggestions:

\n

1.When constructing features, it is necessary to strike a balance between the effectiveness and stability of the model.If introducing a large amount of noise (such as 50 features) can only bring a slight improvement to the model's performance (CV increase of 0.001), it is better to make the model's performance worse.
\n2.Using fusion models. Using fusion models not only improves CV but also enhances model stability.

\n

good luck!

", "rawMarkdown": "let me explain the reason why the competition will shake:\n1.This is a time series prediction competition, and the amount and distribution of test data are unknown,especially the test data is still during the epidemic period.\n2.The focus of this competition is actually on predicting the stability of the model, so any slight decrease in model stability will have a significant impact on the score.\n3.Everyone's scores in this competition are very tight.\n\nMy suggestions:\n\n1.When constructing features, it is necessary to strike a balance between the effectiveness and stability of the model.If introducing a large amount of noise (such as 50 features) can only bring a slight improvement to the model's performance (CV increase of 0.001), it is better to make the model's performance worse.\n2.Using fusion models. Using fusion models not only improves CV but also enhances model stability.\n\ngood luck!\n", "votes": 8 }, { "id": 2746512, "postDate": "2024-04-11T09:21:22.480Z", "content": "

Thank you for sharing your idea!

\n

I'd like to ask two questions.
\n1:What is noise in your suggestion1?
\nDoes \"noise\" mean features we think these are effective to predict?
\n2:I can't understand \"it is better to make the model's performance worse.\"
\nDoes it mean we can enhance our model's stability by making our model performance worth?

", "rawMarkdown": "Thank you for sharing your idea!\n\nI'd like to ask two questions.\n1:What is noise in your suggestion1?\nDoes \"noise\" mean features we think these are effective to predict?\n2:I can't understand \"it is better to make the model's performance worse.\"\nDoes it mean we can enhance our model's stability by making our model performance worth?", "replies": [ { "id": 2746551, "postDate": "2024-04-11T10:06:41.847Z", "content": "

For example, we added 50 features, and the 'gini' increased by 0.01, but this caused a greater loss in the stability of the model, leading to a decrease in the ranking score. It would be better to have a lower 'gini' in our offline cross-validation (CV).

", "rawMarkdown": "For example, we added 50 features, and the 'gini' increased by 0.01, but this caused a greater loss in the stability of the model, leading to a decrease in the ranking score. It would be better to have a lower 'gini' in our offline cross-validation (CV).", "votes": 2, "replies": [ { "id": 2746656, "postDate": "2024-04-11T11:35:54.843Z", "content": "

Thank you!
\nI don't have such idea.
\nI have to rethink how to enhance stability just not AUC.
\nThis is a demanding task!!🤣

", "rawMarkdown": "Thank you!\nI don't have such idea.\nI have to rethink how to enhance stability just not AUC.\nThis is a demanding task!!🤣", "votes": 1 } ] } ] }, { "id": 2747488, "postDate": "2024-04-11T23:28:03.917Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2746512, "author_name": "Kouhei Miyazaki", "author_url": "", "post_date": "2024-04-11T09:21:22.480000", "content": "

Thank you for sharing your idea!

\n

I'd like to ask two questions.
\n1:What is noise in your suggestion1?
\nDoes \"noise\" mean features we think these are effective to predict?
\n2:I can't understand \"it is better to make the model's performance worse.\"
\nDoes it mean we can enhance our model's stability by making our model performance worth?

", "votes": 0, "replies": [ { "id": 2746551, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "2024-04-11T10:06:41.847000", "content": "

For example, we added 50 features, and the 'gini' increased by 0.01, but this caused a greater loss in the stability of the model, leading to a decrease in the ranking score. It would be better to have a lower 'gini' in our offline cross-validation (CV).

", "votes": 2, "replies": [ { "id": 2746656, "author_name": "Kouhei Miyazaki", "author_url": "", "post_date": "2024-04-11T11:35:54.843000", "content": "

Thank you!
\nI don't have such idea.
\nI have to rethink how to enhance stability just not AUC.
\nThis is a demanding task!!🤣

", "votes": 1, "replies": [] } ] } ] }, { "id": 2747488, "author_name": "", "author_url": "", "post_date": "2024-04-11T23:28:03.917000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2746467": "let me explain the reason why the competition will shake:\n1.This is a time series prediction competition, and the amount and distribution of test data are unknown,especially the test data is still during the epidemic period.\n2.The focus of this competition is actually on predicting the stability of the model, so any slight decrease in model stability will have a significant impact on the score.\n3.Everyone's scores in this competition are very tight.\n\nMy suggestions:\n\n1.When constructing features, it is necessary to strike a balance between the effectiveness and stability of the model.If introducing a large amount of noise (such as 50 features) can only bring a slight improvement to the model's performance (CV increase of 0.001), it is better to make the model's performance worse.\n2.Using fusion models. Using fusion models not only improves CV but also enhances model stability.\n\ngood luck!\n", "2746512": "Thank you for sharing your idea!\n\nI'd like to ask two questions.\n1:What is noise in your suggestion1?\nDoes \"noise\" mean features we think these are effective to predict?\n2:I can't understand \"it is better to make the model's performance worse.\"\nDoes it mean we can enhance our model's stability by making our model performance worth?", "2747488": "" } }