{ "id": 238561, "title": "tricks for high public LB score?", "url": "/competitions/hpa-single-cell-image-classification/discussion/238561", "author_name": "Guanshuo Xu", "post_date": "2021-05-12T15:38:00.136000", "votes": 6, "comment_count": 4, "views": 0, "content": "

Looking at my own submissions, my public and private LB scores are always close. For example, public 0.540 -> private 0.538. I was using public LB for validation, yet still could not \"overfit\" to public LB. What's the magic/tricks for the top teams with such high scores on the public LB?

", "messages": [ { "id": 1304383, "postDate": "2021-05-12T15:38:00.137Z", "content": "

Looking at my own submissions, my public and private LB scores are always close. For example, public 0.540 -> private 0.538. I was using public LB for validation, yet still could not \"overfit\" to public LB. What's the magic/tricks for the top teams with such high scores on the public LB?

", "rawMarkdown": "Looking at my own submissions, my public and private LB scores are always close. For example, public 0.540 -> private 0.538. I was using public LB for validation, yet still could not \"overfit\" to public LB. What's the magic/tricks for the top teams with such high scores on the public LB?", "votes": 6 }, { "id": 1304505, "postDate": "2021-05-12T16:51:08.457Z", "content": "

There are ~260 duplicates between train and public test, and ~400 duplicated between train+ext and public test. Thus serve leakage exists in public test but most of team use public LB as a metric.
\nMy teammate find these leakage and when we train without duplicated sample our score drop about 0.01~0.03. We also use a arcface model to take use the label of duplicated but no boost in private LB. Our score without this model is 0.582 and I think this score minus ~0.025 is the real score, about ~0.557, close to our private LB score.

", "rawMarkdown": "There are ~260 duplicates between train and public test, and ~400 duplicated between train+ext and public test. Thus serve leakage exists in public test but most of team use public LB as a metric. \nMy teammate find these leakage and when we train without duplicated sample our score drop about 0.01~0.03. We also use a arcface model to take use the label of duplicated but no boost in private LB. Our score without this model is 0.582 and I think this score minus ~0.025 is the real score, about ~0.557, close to our private LB score.", "votes": 5, "replies": [ { "id": 1304614, "postDate": "2021-05-12T18:27:35.453Z", "content": "

Yes. But I am still curious about how 0.611 was reached; looks like there's some additional sauce.

", "rawMarkdown": "Yes. But I am still curious about how 0.611 was reached; looks like there's some additional sauce.", "votes": 1 } ] }, { "id": 1304994, "postDate": "2021-05-13T03:54:41.533Z", "content": "

well, you made the fewest submissions among top teams (except fake accounts), so naturally the overfitting is less

", "rawMarkdown": "well, you made the fewest submissions among top teams (except fake accounts), so naturally the overfitting is less ", "votes": 1 }, { "id": 1304387, "postDate": "2021-05-12T15:41:18.597Z", "content": "

I was wondering how you were the only one who didn't overfitt 😅

", "rawMarkdown": "I was wondering how you were the only one who didn't overfitt 😅", "votes": 1 } ], "comments": [ { "id": 1304505, "author_name": "sheep", "author_url": "", "post_date": "2021-05-12T16:51:08.457000", "content": "

There are ~260 duplicates between train and public test, and ~400 duplicated between train+ext and public test. Thus serve leakage exists in public test but most of team use public LB as a metric.
\nMy teammate find these leakage and when we train without duplicated sample our score drop about 0.01~0.03. We also use a arcface model to take use the label of duplicated but no boost in private LB. Our score without this model is 0.582 and I think this score minus ~0.025 is the real score, about ~0.557, close to our private LB score.

", "votes": 5, "replies": [ { "id": 1304614, "author_name": "sin", "author_url": "", "post_date": "2021-05-12T18:27:35.453000", "content": "

Yes. But I am still curious about how 0.611 was reached; looks like there's some additional sauce.

", "votes": 1, "replies": [] } ] }, { "id": 1304994, "author_name": "Bo", "author_url": "", "post_date": "2021-05-13T03:54:41.533000", "content": "

well, you made the fewest submissions among top teams (except fake accounts), so naturally the overfitting is less

", "votes": 1, "replies": [] }, { "id": 1304387, "author_name": "CroDoc", "author_url": "", "post_date": "2021-05-12T15:41:18.597000", "content": "

I was wondering how you were the only one who didn't overfitt 😅

", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "1304383": "Looking at my own submissions, my public and private LB scores are always close. For example, public 0.540 -> private 0.538. I was using public LB for validation, yet still could not \"overfit\" to public LB. What's the magic/tricks for the top teams with such high scores on the public LB?", "1304505": "There are ~260 duplicates between train and public test, and ~400 duplicated between train+ext and public test. Thus serve leakage exists in public test but most of team use public LB as a metric. \nMy teammate find these leakage and when we train without duplicated sample our score drop about 0.01~0.03. We also use a arcface model to take use the label of duplicated but no boost in private LB. Our score without this model is 0.582 and I think this score minus ~0.025 is the real score, about ~0.557, close to our private LB score.", "1304994": "well, you made the fewest submissions among top teams (except fake accounts), so naturally the overfitting is less ", "1304387": "I was wondering how you were the only one who didn't overfitt 😅" } }