![\"\"](\"https://i.ibb.co/wsZx3N5/Unbenannt.jpg\")
I follow the approach of Darek Kłeczek on https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/221550. And I split the cropped cells into training and val dataset with a ratio of 4:1 just like the approach does. First, when I train a resnet50 on training dataset and eval on val dataset with average_precision_score, I get average_precision_score=0.5+, and the result on leader board is 0.32. When I train the resnet50 with more epochs with a decay learning rate scheduler, I get a higher average_precision_score=0.8+ on val dataset. Howerver, the result on leader board becomes worse. (0.29). I dont know whether the metrics like average_precision_score is unreliable or the splitting method causes some leakage. Do you have any idea? Thanks a lot.
I still cannot find reliable val metric that can correlate with LB. Will be glad to know if someone found it.
", "rawMarkdown": "I still cannot find reliable val metric that can correlate with LB. Will be glad to know if someone found it.", "votes": 5 }, { "id": 1247994, "postDate": "2021-03-22T08:46:44.260Z", "content": "I follow the approach of Darek Kłeczek on https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/221550. And I split the cropped cells into training and val dataset with a ratio of 4:1 just like the approach does. First, when I train a resnet50 on training dataset and eval on val dataset with average_precision_score, I get average_precision_score=0.5+, and the result on leader board is 0.32. When I train the resnet50 with more epochs with a decay learning rate scheduler, I get a higher average_precision_score=0.8+ on val dataset. Howerver, the result on leader board becomes worse. (0.29). I dont know whether the metrics like average_precision_score is unreliable or the splitting method causes some leakage. Do you have any idea? Thanks a lot.
Isn't this awesome, my number of trained epochs correlate negatively with the Public LB -> easy metrics :)
\n
Where is 0.428 on the chart? ;)
", "rawMarkdown": "Where is 0.428 on the chart? ;) " }, { "id": 1258595, "postDate": "2021-03-31T18:31:07.593Z", "content": "Thats ensembling :)
", "rawMarkdown": "Thats ensembling :)", "votes": 2 }, { "id": 1258621, "postDate": "2021-03-31T18:53:52.647Z", "content": "Ensemble works best for me as well
", "rawMarkdown": "Ensemble works best for me as well" }, { "id": 1258980, "postDate": "2021-04-01T04:21:40.230Z", "content": "My experiments also show negative correlation…..
", "rawMarkdown": "My experiments also show negative correlation....." }, { "id": 1259160, "postDate": "2021-04-01T07:24:51.793Z", "content": "how do you explain the negative correlation?
", "rawMarkdown": "how do you explain the negative correlation?" } ] }, { "id": 1257947, "postDate": "2021-03-31T08:21:17.380Z", "content": "Its like try and error. I guess because the training set is weakly labeled.
", "rawMarkdown": "Its like try and error. I guess because the training set is weakly labeled. ", "votes": 1 } ], "comments": [ { "id": 1248149, "author_name": "Vladislav Ostankovich", "author_url": "", "post_date": "2021-03-22T11:40:31.867000", "content": "I still cannot find reliable val metric that can correlate with LB. Will be glad to know if someone found it.
", "votes": 5, "replies": [] }, { "id": 1258562, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-03-31T17:56:37.977000", "content": "Isn't this awesome, my number of trained epochs correlate negatively with the Public LB -> easy metrics :)
\n
Where is 0.428 on the chart? ;)
", "votes": 0, "replies": [] }, { "id": 1258595, "author_name": "Alexander Riedel", "author_url": "", "post_date": "2021-03-31T18:31:07.593000", "content": "Thats ensembling :)
", "votes": 2, "replies": [] }, { "id": 1258621, "author_name": "LucaMTB", "author_url": "", "post_date": "2021-03-31T18:53:52.647000", "content": "Ensemble works best for me as well
", "votes": 0, "replies": [] }, { "id": 1258980, "author_name": "Nin7a1", "author_url": "", "post_date": "2021-04-01T04:21:40.230000", "content": "My experiments also show negative correlation…..
", "votes": 0, "replies": [] }, { "id": 1259160, "author_name": "LucaMTB", "author_url": "", "post_date": "2021-04-01T07:24:51.793000", "content": "how do you explain the negative correlation?
", "votes": 0, "replies": [] } ] }, { "id": 1257947, "author_name": "LucaMTB", "author_url": "", "post_date": "2021-03-31T08:21:17.380000", "content": "Its like try and error. I guess because the training set is weakly labeled.
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "1248149": "I still cannot find reliable val metric that can correlate with LB. Will be glad to know if someone found it.", "1247994": "I follow the approach of Darek Kłeczek on https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/221550. And I split the cropped cells into training and val dataset with a ratio of 4:1 just like the approach does. First, when I train a resnet50 on training dataset and eval on val dataset with `average_precision_score`, I get average_precision_score=0.5+, and the result on leader board is 0.32. When I train the resnet50 with more epochs with a decay learning rate scheduler, I get a higher average_precision_score=0.8+ on val dataset. Howerver, the result on leader board becomes worse. (0.29). I dont know whether the metrics like average_precision_score is unreliable or the splitting method causes some leakage. Do you have any idea? Thanks a lot. ", "1258562": "Isn't this awesome, my number of trained epochs correlate negatively with the Public LB -> easy metrics :)\n\n", "1257947": "Its like try and error. I guess because the training set is weakly labeled. " } }