![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F16303%2F87541803b8f07744e9b17d4883b2a3e9%2Fshakeup.PNG?generation=1567961333467148&alt=media\")
I see lots being said about shake up, and upon looking at my private leaderboard, also had this feeling. However when actually plotting data, I see very little shake up in solutions I explored. If anything in CV I trust ever more.\n
It was worth doing this post analysis to check if my workflow was not compromised. As many I was worried about choosing a correct model for the private test. But if your workflow is decent, you should just trust your CV, instead of blindly choosing best LB sumission.
\n\nIf anybody else kept their records of CV scores, I would like to see similar plots from you guys/girls.
", "messages": [ { "id": "621561", "postDate": "09/08/2019 16:56:52", "content": "I see lots being said about shake up, and upon looking at my private leaderboard, also had this feeling. However when actually plotting data, I see very little shake up in solutions I explored. If anything in CV I trust ever more.\n
It was worth doing this post analysis to check if my workflow was not compromised. As many I was worried about choosing a correct model for the private test. But if your workflow is decent, you should just trust your CV, instead of blindly choosing best LB sumission.
\n\nIf anybody else kept their records of CV scores, I would like to see similar plots from you guys/girls.
", "rawMarkdown": "I see lots being said about shake up, and upon looking at my private leaderboard, also had this feeling. However when actually plotting data, I see very little shake up in solutions I explored. If anything in CV I trust ever more.\n\n\nIt was worth doing this post analysis to check if my workflow was not compromised. As many I was worried about choosing a correct model for the private test. But if your workflow is decent, you should just trust your CV, instead of blindly choosing best LB sumission.\n\nIf anybody else kept their records of CV scores, I would like to see similar plots from you guys/girls.", "votes": null }, { "id": "621676", "postDate": "09/08/2019 20:30:44", "content": "My experiments showed a strong correlation between my CV and private LB (although I can't confirm since I don't have much data to support it). You can see my results here
", "rawMarkdown": "My experiments showed a strong correlation between my CV and private LB (although I can't confirm since I don't have much data to support it). You can see my results [here](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/108017#latest-621570)", "votes": null }, { "id": "621709", "postDate": "09/08/2019 21:03:40", "content": "Nice, indeed very CV and private LB very much in line with each other
", "rawMarkdown": "Nice, indeed very CV and private LB very much in line with each other", "votes": null }, { "id": "622059", "postDate": "09/09/2019 08:18:11", "content": "Sure there is correlation, but the amount of variance that you can see, of about 0.005 kappa, it is enough to change the rankings by 250. It is a highly competitive environment. Correct me if I am wrong.
", "rawMarkdown": "Sure there is correlation, but the amount of variance that you can see, of about 0.005 kappa, it is enough to change the rankings by 250. It is a highly competitive environment. Correct me if I am wrong.", "votes": null } ], "comments": [ { "id": 621676, "author_name": "tahsin", "author_url": "", "post_date": "09/08/2019 20:30:44", "content": "My experiments showed a strong correlation between my CV and private LB (although I can't confirm since I don't have much data to support it). You can see my results here
", "votes": null, "replies": [] }, { "id": 621709, "author_name": "wantowin", "author_url": "", "post_date": "09/08/2019 21:03:40", "content": "Nice, indeed very CV and private LB very much in line with each other
", "votes": null, "replies": [] }, { "id": 622059, "author_name": "arusia", "author_url": "", "post_date": "09/09/2019 08:18:11", "content": "Sure there is correlation, but the amount of variance that you can see, of about 0.005 kappa, it is enough to change the rankings by 250. It is a highly competitive environment. Correct me if I am wrong.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "621561": "I see lots being said about shake up, and upon looking at my private leaderboard, also had this feeling. However when actually plotting data, I see very little shake up in solutions I explored. If anything in CV I trust ever more.\n\n\nIt was worth doing this post analysis to check if my workflow was not compromised. As many I was worried about choosing a correct model for the private test. But if your workflow is decent, you should just trust your CV, instead of blindly choosing best LB sumission.\n\nIf anybody else kept their records of CV scores, I would like to see similar plots from you guys/girls.", "621676": "My experiments showed a strong correlation between my CV and private LB (although I can't confirm since I don't have much data to support it). You can see my results [here](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/108017#latest-621570)", "621709": "Nice, indeed very CV and private LB very much in line with each other", "622059": "Sure there is correlation, but the amount of variance that you can see, of about 0.005 kappa, it is enough to change the rankings by 250. It is a highly competitive environment. Correct me if I am wrong." }, "source": "meta" }