Do you see correlation between your CV and LB?
\nMy LB and CV are absolutely not correlated:
\nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nMaybe while i'm increasing CV, i'm reducing stability?
\nDid you face with this problem?
Do you see correlation between your CV and LB?
\nMy LB and CV are absolutely not correlated:
\nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nLB: , CV: \nMaybe while i'm increasing CV, i'm reducing stability?
\nDid you face with this problem?
The CV that you have calculated is the which metrics? auc or roc_auc or gini stability score or something else
", "rawMarkdown": "The CV that you have calculated is the which metrics? auc or roc_auc or gini stability score or something else", "votes": -1, "replies": [ { "id": 2706283, "postDate": "2024-03-19T19:31:16.527Z", "content": "It's roc_auc metric
", "rawMarkdown": "It's roc_auc metric", "replies": [ { "id": 2706311, "postDate": "2024-03-19T19:56:13.313Z", "content": "then it the other terms in the calculation of gini stability like falling rate and std of residuals must be responsible for this non correlation
", "rawMarkdown": "then it the other terms in the calculation of gini stability like falling rate and std of residuals must be responsible for this non correlation" } ] } ] }, { "id": 2705720, "postDate": "2024-03-19T13:39:31.517Z", "content": "May I ask if you are using all the data or partial data for training? My current score is still 0.52, and my CV is calculated as \"mean_gini\". I feel that there is still a correlation overall.
", "rawMarkdown": "May I ask if you are using all the data or partial data for training? My current score is still 0.52, and my CV is calculated as \"mean_gini\". I feel that there is still a correlation overall.", "replies": [ { "id": 2705777, "postDate": "2024-03-19T14:19:28.903Z", "content": "Yes, I split train dataset in 5 fold, train on 4 of them and predict on the last.(Do it 5 times).
\nStandart cross validation, you can find it in any training notebook.
Based on this 5 folds scoring - accumulate auc scores in a dataframe and then apply the gini stability formula (found in many notebooks) over your scores grouped by week so you'll have the same scoring metric between CV and LB
", "rawMarkdown": "Based on this 5 folds scoring - accumulate auc scores in a dataframe and then apply the gini stability formula (found in many notebooks) over your scores grouped by week so you'll have the same scoring metric between CV and LB" }, { "id": 2719342, "postDate": "2024-03-27T16:37:06.813Z", "content": "Of course i did it. It's not correlated with LB as well
", "rawMarkdown": "Of course i did it. It's not correlated with LB as well" } ] } ] }, { "id": 2706172, "postDate": "2024-03-19T17:46:00.100Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2706066, "postDate": "2024-03-19T16:22:56.030Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 2705725, "postDate": "2024-03-19T13:44:02.930Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2706268, "author_name": "Shreyas Bhatt", "author_url": "", "post_date": "2024-03-19T19:19:41.313000", "content": "The CV that you have calculated is the which metrics? auc or roc_auc or gini stability score or something else
", "votes": -1, "replies": [ { "id": 2706283, "author_name": "skrrydg", "author_url": "", "post_date": "2024-03-19T19:31:16.527000", "content": "It's roc_auc metric
", "votes": 0, "replies": [ { "id": 2706311, "author_name": "Shreyas Bhatt", "author_url": "", "post_date": "2024-03-19T19:56:13.313000", "content": "then it the other terms in the calculation of gini stability like falling rate and std of residuals must be responsible for this non correlation
", "votes": 0, "replies": [] } ] } ] }, { "id": 2705720, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "2024-03-19T13:39:31.517000", "content": "May I ask if you are using all the data or partial data for training? My current score is still 0.52, and my CV is calculated as \"mean_gini\". I feel that there is still a correlation overall.
", "votes": 0, "replies": [ { "id": 2705777, "author_name": "skrrydg", "author_url": "", "post_date": "2024-03-19T14:19:28.903000", "content": "Yes, I split train dataset in 5 fold, train on 4 of them and predict on the last.(Do it 5 times).
\nStandart cross validation, you can find it in any training notebook.
Based on this 5 folds scoring - accumulate auc scores in a dataframe and then apply the gini stability formula (found in many notebooks) over your scores grouped by week so you'll have the same scoring metric between CV and LB
", "votes": 0, "replies": [] }, { "id": 2719342, "author_name": "skrrydg", "author_url": "", "post_date": "2024-03-27T16:37:06.813000", "content": "Of course i did it. It's not correlated with LB as well
", "votes": 0, "replies": [] } ] } ] }, { "id": 2706172, "author_name": "", "author_url": "", "post_date": "2024-03-19T17:46:00.100000", "content": "", "votes": 0, "replies": [] }, { "id": 2706066, "author_name": "", "author_url": "", "post_date": "2024-03-19T16:22:56.030000", "content": "", "votes": 1, "replies": [] }, { "id": 2705725, "author_name": "", "author_url": "", "post_date": "2024-03-19T13:44:02.930000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2705684": "Do you see correlation between your CV and LB?\n\nMy LB and CV are absolutely not correlated:\n```python\nLB: 0.555, CV: 0.8345\nLB: 0.549, CV: 0.8357\nLB: 0.561, CV: 0.8356\nLB: 0.568, CV: 0.8356\nLB: 0.564, CV: 0.83559\nLB: 0.545, CV: 0.8371\nLB: 0.557, CV: 0.83566\nLB: 0.558, CV: 0.8358\n```\n\nMaybe while i'm increasing CV, i'm reducing stability?\nDid you face with this problem?", "2706268": "The CV that you have calculated is the which metrics? auc or roc_auc or gini stability score or something else", "2705720": "May I ask if you are using all the data or partial data for training? My current score is still 0.52, and my CV is calculated as \"mean_gini\". I feel that there is still a correlation overall.", "2706172": "", "2706066": "", "2705725": "" } }