{ "id": 507942, "title": "What a shake", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/507942", "author_name": "Andrey Chankin", "post_date": "2024-05-28T00:02:12.482000", "votes": 3, "comment_count": 8, "views": 0, "content": "

heh, didnt expect, ppl get +1200, could not find good CV myself

", "messages": [ { "id": 2840047, "postDate": "2024-05-28T00:02:12.483Z", "content": "

heh, didnt expect, ppl get +1200, could not find good CV myself

", "rawMarkdown": "heh, didnt expect, ppl get +1200, could not find good CV myself", "votes": 3 }, { "id": 2840133, "postDate": "2024-05-28T01:42:31.830Z", "content": "

I used the training set to observe the metric trick, and if the time is extended, the subsequent oscillation will be large.

", "rawMarkdown": "I used the training set to observe the metric trick, and if the time is extended, the subsequent oscillation will be large.![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F20129689%2Fded49e486563198b7f15693e6415847d%2F465ff5ecb6f8f0c9dece04f6e5a9806.png?generation=1716860549259016&alt=media)", "replies": [ { "id": 2840138, "postDate": "2024-05-28T01:44:22.050Z", "content": "

In this case, the most we reduce (0.073) is probably the score of the distant future。

", "rawMarkdown": "In this case, the most we reduce (0.073) is probably the score of the distant future。", "replies": [ { "id": 2840142, "postDate": "2024-05-28T01:45:43.187Z", "content": "

I don't know how big the test set is or how long it will be. I had to gamble, and I lost.

", "rawMarkdown": "I don't know how big the test set is or how long it will be. I had to gamble, and I lost." } ] } ] }, { "id": 2840049, "postDate": "2024-05-28T00:09:00.247Z", "content": "

I'm wondering why private scores are so much lower than public scores. I assumed that as more data are tested, the standard deviation would decrease so the stability score would be higher (?)

", "rawMarkdown": "I'm wondering why private scores are so much lower than public scores. I assumed that as more data are tested, the standard deviation would decrease so the stability score would be higher (?)", "replies": [ { "id": 2840051, "postDate": "2024-05-28T00:11:39.090Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2840052, "postDate": "2024-05-28T00:12:01.660Z", "content": "

Because it is a long-term prediction, the gini of the model itself will decrease

", "rawMarkdown": "Because it is a long-term prediction, the gini of the model itself will decrease", "replies": [ { "id": 2840053, "postDate": "2024-05-28T00:15:14.690Z", "content": "

Well, I guess if 30% public test data and 70% private test data approximately have the same distribution, the gini should not decrease.

", "rawMarkdown": "Well, I guess if 30% public test data and 70% private test data approximately have the same distribution, the gini should not decrease." } ] }, { "id": 2840121, "postDate": "2024-05-28T01:38:02.307Z", "content": "

cause the private dataset contains 3-4 years of data while the public one only contains last 2 years

", "rawMarkdown": "cause the private dataset contains 3-4 years of data while the public one only contains last 2 years" } ] } ], "comments": [ { "id": 2840133, "author_name": "", "author_url": "", "post_date": "2024-05-28T01:42:31.830000", "content": "

I used the training set to observe the metric trick, and if the time is extended, the subsequent oscillation will be large.

", "votes": 0, "replies": [ { "id": 2840138, "author_name": "", "author_url": "", "post_date": "2024-05-28T01:44:22.050000", "content": "

In this case, the most we reduce (0.073) is probably the score of the distant future。

", "votes": 0, "replies": [ { "id": 2840142, "author_name": "", "author_url": "", "post_date": "2024-05-28T01:45:43.187000", "content": "

I don't know how big the test set is or how long it will be. I had to gamble, and I lost.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2840049, "author_name": "Lorry Zou", "author_url": "", "post_date": "2024-05-28T00:09:00.247000", "content": "

I'm wondering why private scores are so much lower than public scores. I assumed that as more data are tested, the standard deviation would decrease so the stability score would be higher (?)

", "votes": 0, "replies": [ { "id": 2840051, "author_name": "", "author_url": "", "post_date": "2024-05-28T00:11:39.090000", "content": "", "votes": 0, "replies": [] }, { "id": 2840052, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "2024-05-28T00:12:01.660000", "content": "

Because it is a long-term prediction, the gini of the model itself will decrease

", "votes": 0, "replies": [ { "id": 2840053, "author_name": "Lorry Zou", "author_url": "", "post_date": "2024-05-28T00:15:14.690000", "content": "

Well, I guess if 30% public test data and 70% private test data approximately have the same distribution, the gini should not decrease.

", "votes": 0, "replies": [] } ] }, { "id": 2840121, "author_name": "hongyi shui", "author_url": "", "post_date": "2024-05-28T01:38:02.307000", "content": "

cause the private dataset contains 3-4 years of data while the public one only contains last 2 years

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2840047": "heh, didnt expect, ppl get +1200, could not find good CV myself", "2840133": "I used the training set to observe the metric trick, and if the time is extended, the subsequent oscillation will be large.![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F20129689%2Fded49e486563198b7f15693e6415847d%2F465ff5ecb6f8f0c9dece04f6e5a9806.png?generation=1716860549259016&alt=media)", "2840049": "I'm wondering why private scores are so much lower than public scores. I assumed that as more data are tested, the standard deviation would decrease so the stability score would be higher (?)" } }