{ "id": 207517, "title": "Free LB boost [0.015-0.020] - global mask shift ", "url": "/competitions/hubmap-kidney-segmentation/discussion/207517", "author_name": "", "post_date": "2020-12-30T03:07:03.444833500Z", "votes": 51, "comment_count": 25, "views": 0, "content": "

TL/DR: shift mask of afa5e9098 by [-42, -24] and your LB score should go up by 0.015-0.020.

\n

Who doesn’t love a free LB boost?

\n

I was hoping the label shifts would have been fixed by now, but since not and we’re just over a month from the competition ending I think the time is right to share mask shift info. Applying the proposed shift should boost your LB score by ~0.015-0.020.

\n

Note: This should not help your private LB score as there’s no way to know if a GT shift will exist on that data. Once the labels are corrected by the sponsors this should hopefully become a moot point. In the mean time what this may do is give you better signal to noise on real improvements you make to your models.

\n

As @hengck23 pointed out here, you can apply a global offset to a mask and determine whether it improves or degrades dice. The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.

\n

Applying an offset of y=-42, x=-24 to afa5e8098 is all you need. I’ve posted this notebook which takes the current best public notebook from here of 0.848 and improves the score to 0.865.

", "messages": [ { "id": "1131846", "postDate": "12/30/2020 03:07:03", "content": "

TL/DR: shift mask of afa5e9098 by [-42, -24] and your LB score should go up by 0.015-0.020.

\n

Who doesn’t love a free LB boost?

\n

I was hoping the label shifts would have been fixed by now, but since not and we’re just over a month from the competition ending I think the time is right to share mask shift info. Applying the proposed shift should boost your LB score by ~0.015-0.020.

\n

Note: This should not help your private LB score as there’s no way to know if a GT shift will exist on that data. Once the labels are corrected by the sponsors this should hopefully become a moot point. In the mean time what this may do is give you better signal to noise on real improvements you make to your models.

\n

As @hengck23 pointed out here, you can apply a global offset to a mask and determine whether it improves or degrades dice. The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.

\n

Applying an offset of y=-42, x=-24 to afa5e8098 is all you need. I’ve posted this notebook which takes the current best public notebook from here of 0.848 and improves the score to 0.865.

", "rawMarkdown": "TL/DR: shift mask of afa5e9098 by [-42, -24] and your LB score should go up by 0.015-0.020.\n\nWho doesn’t love a free LB boost?\n\nI was hoping the label shifts would have been fixed by now, but since not and we’re just over a month from the competition ending I think the time is right to share mask shift info. Applying the proposed shift should boost your LB score by ~0.015-0.020.\n\nNote: This should not help your private LB score as there’s no way to know if a GT shift will exist on that data. Once the labels are corrected by the sponsors this should hopefully become a moot point. In the mean time what this may do is give you better signal to noise on real improvements you make to your models.\n\nAs @hengck23 pointed out [here](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201816#1116263), you can apply a global offset to a mask and determine whether it improves or degrades dice. The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.\n\nApplying an offset of y=-42, x=-24 to afa5e8098 is all you need. I’ve posted [this](https://www.kaggle.com/tivfrvqhs5/global-mask-shift) notebook which takes the current best public notebook from [here](https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-subm) of 0.848 and improves the score to 0.865.", "votes": null }, { "id": "1131857", "postDate": "12/30/2020 03:19:43", "content": "

😂 It give me a amazing boost

", "rawMarkdown": "😂 It give me a amazing boost", "votes": null }, { "id": "1131880", "postDate": "12/30/2020 03:55:58", "content": "

This is a good way to improve LB, but I don't know if other image masks have such problems?

", "rawMarkdown": "This is a good way to improve LB, but I don't know if other image masks have such problems?", "votes": null }, { "id": "1131884", "postDate": "12/30/2020 03:59:07", "content": "

The other public test masks don't seem to have this problem at least not close to the same magnitude

", "rawMarkdown": "The other public test masks don't seem to have this problem at least not close to the same magnitude", "votes": null }, { "id": "1131899", "postDate": "12/30/2020 04:12:10", "content": "

Thank you for your reply. Imagine if the image ID name of the private data set can be detected (of course, this is very difficult or almost impossible), but I guess there may be private test data on the professional website. If all the image ID names suitable for this project on the professional website are detected, then the test image ID name is known, and the corresponding data of the test image ID is known, Then you can follow your method to see if y compares with the image data on the professional website according to the image ID name, whether you can see the difference?

", "rawMarkdown": "Thank you for your reply. Imagine if the image ID name of the private data set can be detected (of course, this is very difficult or almost impossible), but I guess there may be private test data on the professional website. If all the image ID names suitable for this project on the professional website are detected, then the test image ID name is known, and the corresponding data of the test image ID is known, Then you can follow your method to see if y compares with the image data on the professional website according to the image ID name, whether you can see the difference?", "votes": null }, { "id": "1131914", "postDate": "12/30/2020 04:29:48", "content": "

Thanks for this info, although as you rightly mentioned this won't help with the actual private test data. But did you use this trick for your current LB score as well?

", "rawMarkdown": "Thanks for this info, although as you rightly mentioned this won't help with the actual private test data. But did you use this trick for your current LB score as well?", "votes": null }, { "id": "1132034", "postDate": "12/30/2020 06:11:34", "content": "

LB got wrecked 😂

", "rawMarkdown": "LB got wrecked 😂", "votes": null }, { "id": "1132041", "postDate": "12/30/2020 06:14:53", "content": "

Yes, but it was going to be wrecked anyway as soon as the sponsors updated the labels. I would argue that \"wrecked\" in this case just means more accurate

", "rawMarkdown": "Yes, but it was going to be wrecked anyway as soon as the sponsors updated the labels. I would argue that \"wrecked\" in this case just means more accurate", "votes": null }, { "id": "1132047", "postDate": "12/30/2020 06:17:57", "content": "

amazing boost! thanks

", "rawMarkdown": "amazing boost! thanks", "votes": null }, { "id": "1132053", "postDate": "12/30/2020 06:21:54", "content": "

I hope the host can solve this asap along with the test set leak issue. And besides these, Kaggle is dealing with something really huge as well https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357

", "rawMarkdown": "I hope the host can solve this asap along with the test set leak issue. And besides these, Kaggle is dealing with something really huge as well https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357", "votes": null }, { "id": "1132074", "postDate": "12/30/2020 06:41:20", "content": "
\n

as there’s no way to know if a GT shift will exist on that data.

\n
\n

See here how to probe which images are in private test

\n

https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248

\n

and here how you could get the private leaderboard score of applying the shift

\n

https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357

", "rawMarkdown": "> as there’s no way to know if a GT shift will exist on that data.\n\nSee here how to probe which images are in private test\n\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248\n\nand here how you could get the private leaderboard score of applying the shift \n\nhttps://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357", "votes": null }, { "id": "1132088", "postDate": "12/30/2020 06:56:39", "content": "

Hi,is there a hidden test set in Private set?

", "rawMarkdown": "Hi,is there a hidden test set in Private set?", "votes": null }, { "id": "1132127", "postDate": "12/30/2020 07:16:05", "content": "

I don’t know if anyone saw the scores of this competition‘s private score yesterday. If someone did, it would be unfair.

", "rawMarkdown": "I don’t know if anyone saw the scores of this competition‘s private score yesterday. If someone did, it would be unfair.", "votes": null }, { "id": "1132213", "postDate": "12/30/2020 08:36:52", "content": "

I guess it doesn't matter that much because in this competition, the images of the private set is publicly available anyway😂 (without gt masks of course) I asked the host for an update https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248 but haven't heard from them yet

", "rawMarkdown": "I guess it doesn't matter that much because in this competition, the images of the private set is publicly available anyway😂 (without gt masks of course) I asked the host for an update https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248 but haven't heard from them yet", "votes": null }, { "id": "1132397", "postDate": "12/30/2020 11:34:49", "content": "

Thanks for sharing!

\n
\n

The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.

\n
\n

Are you using shifted training data as well?

", "rawMarkdown": "Thanks for sharing!\n\n> The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.\n\nAre you using shifted training data as well?", "votes": null }, { "id": "1132475", "postDate": "12/30/2020 12:36:20", "content": "

Thank you @tivfrvqhs5 , this trick improves my dice from 0.750 to 0.835. Is the c68fe75ea not shifted as well?
\nI know that this do not gives improvements for the private LB, but at it gives a better signal to noise, and I could know that pseudo labels are more accurate.

", "rawMarkdown": "Thank you @tivfrvqhs5 , this trick improves my dice from `0.750` to `0.835`. Is the `c68fe75ea` not shifted as well?\nI know that this do not gives improvements for the private LB, but at it gives a better signal to noise, and I could know that pseudo labels are more accurate.", "votes": null }, { "id": "1132767", "postDate": "12/30/2020 17:04:43", "content": "

Yes there are three training masks that still have significant shifts in them. Still waiting for the sponsors to update them.

", "rawMarkdown": "Yes there are three training masks that still have significant shifts in them. Still waiting for the sponsors to update them.", "votes": null }, { "id": "1134298", "postDate": "01/01/2021 05:39:55", "content": "

@tivfrvqhs5 How was this shift in ground truth for afa5e8098 discovered?

", "rawMarkdown": "tivfrvqhs5 How was this shift in ground truth for afa5e8098 discovered?", "votes": null }, { "id": "1134802", "postDate": "01/01/2021 15:34:24", "content": "

This is one way to probe the gt shift https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201816#1116263

", "rawMarkdown": "This is one way to probe the gt shift https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201816#1116263", "votes": null }, { "id": "1135023", "postDate": "01/01/2021 19:52:59", "content": "

I did not detect much of a shift on c68fe75ea but you can try some shift values yourself and see

", "rawMarkdown": "I did not detect much of a shift on c68fe75ea but you can try some shift values yourself and see", "votes": null }, { "id": "1140042", "postDate": "01/05/2021 19:06:20", "content": "

Did you apply this shift to all test images or only to afa5e8098

", "rawMarkdown": "Did you apply this shift to all test images or only to afa5e8098", "votes": null }, { "id": "1140235", "postDate": "01/05/2021 21:47:56", "content": "

Only afa5e8098. c68fe75ea doesn't appear to have a significant shift and the other three get well above 0.9 so I didn't bother checking. Nice avatar 😊

", "rawMarkdown": "Only afa5e8098. c68fe75ea doesn't appear to have a significant shift and the other three get well above 0.9 so I didn't bother checking. Nice avatar 😊", "votes": null }, { "id": "1140750", "postDate": "01/06/2021 08:32:29", "content": "

Oh ok got it, makes sense to use it for the one with a lesser LB score than the others.
\nHaha yeah Gilfoyle is the real deal, people just don't get it :P

", "rawMarkdown": "Oh ok got it, makes sense to use it for the one with a lesser LB score than the others. \nHaha yeah Gilfoyle is the real deal, people just don't get it :P", "votes": null }, { "id": "1242470", "postDate": "03/17/2021 16:25:35", "content": "

@tivfrvqhs5 will it work with new data ?

", "rawMarkdown": "tivfrvqhs5 will it work with new data ?", "votes": null }, { "id": "1244233", "postDate": "03/18/2021 19:50:11", "content": "

nope, problem with data was fixed.

", "rawMarkdown": "nope, problem with data was fixed.", "votes": null }, { "id": "1301934", "postDate": "05/11/2021 10:17:11", "content": "

yes, the data have shift

", "rawMarkdown": "yes, the data have shift", "votes": null } ], "comments": [ { "id": 1131857, "author_name": "hesene", "author_url": "", "post_date": "12/30/2020 03:19:43", "content": "

😂 It give me a amazing boost

", "votes": null, "replies": [ { "id": 1140042, "author_name": "ckanth090", "author_url": "", "post_date": "01/05/2021 19:06:20", "content": "

Did you apply this shift to all test images or only to afa5e8098

", "votes": null, "replies": [] }, { "id": 1140235, "author_name": "tivfrvqhs5", "author_url": "", "post_date": "01/05/2021 21:47:56", "content": "

Only afa5e8098. c68fe75ea doesn't appear to have a significant shift and the other three get well above 0.9 so I didn't bother checking. Nice avatar 😊

", "votes": null, "replies": [] }, { "id": 1140750, "author_name": "ckanth090", "author_url": "", "post_date": "01/06/2021 08:32:29", "content": "

Oh ok got it, makes sense to use it for the one with a lesser LB score than the others.
\nHaha yeah Gilfoyle is the real deal, people just don't get it :P

", "votes": null, "replies": [] } ] }, { "id": 1131880, "author_name": "zhangeng", "author_url": "", "post_date": "12/30/2020 03:55:58", "content": "

This is a good way to improve LB, but I don't know if other image masks have such problems?

", "votes": null, "replies": [ { "id": 1131884, "author_name": "tivfrvqhs5", "author_url": "", "post_date": "12/30/2020 03:59:07", "content": "

The other public test masks don't seem to have this problem at least not close to the same magnitude

", "votes": null, "replies": [] }, { "id": 1131899, "author_name": "zhangeng", "author_url": "", "post_date": "12/30/2020 04:12:10", "content": "

Thank you for your reply. Imagine if the image ID name of the private data set can be detected (of course, this is very difficult or almost impossible), but I guess there may be private test data on the professional website. If all the image ID names suitable for this project on the professional website are detected, then the test image ID name is known, and the corresponding data of the test image ID is known, Then you can follow your method to see if y compares with the image data on the professional website according to the image ID name, whether you can see the difference?

", "votes": null, "replies": [] } ] }, { "id": 1131914, "author_name": "aramisvesal", "author_url": "", "post_date": "12/30/2020 04:29:48", "content": "

Thanks for this info, although as you rightly mentioned this won't help with the actual private test data. But did you use this trick for your current LB score as well?

", "votes": null, "replies": [] }, { "id": 1132034, "author_name": "jihangz", "author_url": "", "post_date": "12/30/2020 06:11:34", "content": "

LB got wrecked 😂

", "votes": null, "replies": [ { "id": 1132041, "author_name": "tivfrvqhs5", "author_url": "", "post_date": "12/30/2020 06:14:53", "content": "

Yes, but it was going to be wrecked anyway as soon as the sponsors updated the labels. I would argue that \"wrecked\" in this case just means more accurate

", "votes": null, "replies": [] }, { "id": 1132053, "author_name": "jihangz", "author_url": "", "post_date": "12/30/2020 06:21:54", "content": "

I hope the host can solve this asap along with the test set leak issue. And besides these, Kaggle is dealing with something really huge as well https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357

", "votes": null, "replies": [] }, { "id": 1132127, "author_name": "hesene", "author_url": "", "post_date": "12/30/2020 07:16:05", "content": "

I don’t know if anyone saw the scores of this competition‘s private score yesterday. If someone did, it would be unfair.

", "votes": null, "replies": [] }, { "id": 1132213, "author_name": "jihangz", "author_url": "", "post_date": "12/30/2020 08:36:52", "content": "

I guess it doesn't matter that much because in this competition, the images of the private set is publicly available anyway😂 (without gt masks of course) I asked the host for an update https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248 but haven't heard from them yet

", "votes": null, "replies": [] } ] }, { "id": 1132047, "author_name": "tiandaye", "author_url": "", "post_date": "12/30/2020 06:17:57", "content": "

amazing boost! thanks

", "votes": null, "replies": [] }, { "id": 1132074, "author_name": "christofhenkel", "author_url": "", "post_date": "12/30/2020 06:41:20", "content": "
\n

as there’s no way to know if a GT shift will exist on that data.

\n
\n

See here how to probe which images are in private test

\n

https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248

\n

and here how you could get the private leaderboard score of applying the shift

\n

https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357

", "votes": null, "replies": [] }, { "id": 1132088, "author_name": "zekun98", "author_url": "", "post_date": "12/30/2020 06:56:39", "content": "

Hi,is there a hidden test set in Private set?

", "votes": null, "replies": [] }, { "id": 1132397, "author_name": "wrrosa", "author_url": "", "post_date": "12/30/2020 11:34:49", "content": "

Thanks for sharing!

\n
\n

The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.

\n
\n

Are you using shifted training data as well?

", "votes": null, "replies": [ { "id": 1132767, "author_name": "tivfrvqhs5", "author_url": "", "post_date": "12/30/2020 17:04:43", "content": "

Yes there are three training masks that still have significant shifts in them. Still waiting for the sponsors to update them.

", "votes": null, "replies": [] } ] }, { "id": 1132475, "author_name": "bessenyeiszilrd", "author_url": "", "post_date": "12/30/2020 12:36:20", "content": "

Thank you @tivfrvqhs5 , this trick improves my dice from 0.750 to 0.835. Is the c68fe75ea not shifted as well?
\nI know that this do not gives improvements for the private LB, but at it gives a better signal to noise, and I could know that pseudo labels are more accurate.

", "votes": null, "replies": [ { "id": 1135023, "author_name": "tivfrvqhs5", "author_url": "", "post_date": "01/01/2021 19:52:59", "content": "

I did not detect much of a shift on c68fe75ea but you can try some shift values yourself and see

", "votes": null, "replies": [] } ] }, { "id": 1134298, "author_name": "pratikkumar", "author_url": "", "post_date": "01/01/2021 05:39:55", "content": "

@tivfrvqhs5 How was this shift in ground truth for afa5e8098 discovered?

", "votes": null, "replies": [ { "id": 1134802, "author_name": "bessenyeiszilrd", "author_url": "", "post_date": "01/01/2021 15:34:24", "content": "

This is one way to probe the gt shift https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201816#1116263

", "votes": null, "replies": [] } ] }, { "id": 1242470, "author_name": "jaideepvalani", "author_url": "", "post_date": "03/17/2021 16:25:35", "content": "

@tivfrvqhs5 will it work with new data ?

", "votes": null, "replies": [ { "id": 1244233, "author_name": "andrasferenczi", "author_url": "", "post_date": "03/18/2021 19:50:11", "content": "

nope, problem with data was fixed.

", "votes": null, "replies": [] } ] }, { "id": 1301934, "author_name": "shajiayu", "author_url": "", "post_date": "05/11/2021 10:17:11", "content": "

yes, the data have shift

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1131846": "TL/DR: shift mask of afa5e9098 by [-42, -24] and your LB score should go up by 0.015-0.020.\n\nWho doesn’t love a free LB boost?\n\nI was hoping the label shifts would have been fixed by now, but since not and we’re just over a month from the competition ending I think the time is right to share mask shift info. Applying the proposed shift should boost your LB score by ~0.015-0.020.\n\nNote: This should not help your private LB score as there’s no way to know if a GT shift will exist on that data. Once the labels are corrected by the sponsors this should hopefully become a moot point. In the mean time what this may do is give you better signal to noise on real improvements you make to your models.\n\nAs @hengck23 pointed out [here](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201816#1116263), you can apply a global offset to a mask and determine whether it improves or degrades dice. The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.\n\nApplying an offset of y=-42, x=-24 to afa5e8098 is all you need. I’ve posted [this](https://www.kaggle.com/tivfrvqhs5/global-mask-shift) notebook which takes the current best public notebook from [here](https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-subm) of 0.848 and improves the score to 0.865.", "1131857": "😂 It give me a amazing boost", "1131880": "This is a good way to improve LB, but I don't know if other image masks have such problems?", "1131884": "The other public test masks don't seem to have this problem at least not close to the same magnitude", "1131899": "Thank you for your reply. Imagine if the image ID name of the private data set can be detected (of course, this is very difficult or almost impossible), but I guess there may be private test data on the professional website. If all the image ID names suitable for this project on the professional website are detected, then the test image ID name is known, and the corresponding data of the test image ID is known, Then you can follow your method to see if y compares with the image data on the professional website according to the image ID name, whether you can see the difference?", "1131914": "Thanks for this info, although as you rightly mentioned this won't help with the actual private test data. But did you use this trick for your current LB score as well?", "1132034": "LB got wrecked 😂", "1132041": "Yes, but it was going to be wrecked anyway as soon as the sponsors updated the labels. I would argue that \"wrecked\" in this case just means more accurate", "1132047": "amazing boost! thanks", "1132053": "I hope the host can solve this asap along with the test set leak issue. And besides these, Kaggle is dealing with something really huge as well https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357", "1132074": "> as there’s no way to know if a GT shift will exist on that data.\n\nSee here how to probe which images are in private test\n\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248\n\nand here how you could get the private leaderboard score of applying the shift \n\nhttps://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/207357", "1132088": "Hi,is there a hidden test set in Private set?", "1132127": "I don’t know if anyone saw the scores of this competition‘s private score yesterday. If someone did, it would be unfair.", "1132213": "I guess it doesn't matter that much because in this competition, the images of the private set is publicly available anyway😂 (without gt masks of course) I asked the host for an update https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201248 but haven't heard from them yet", "1132397": "Thanks for sharing!\n\n> The same logic can be applied to your train masks, and apply shift to GT mask to determine the optimal offset.\n\nAre you using shifted training data as well?", "1132475": "Thank you @tivfrvqhs5 , this trick improves my dice from `0.750` to `0.835`. Is the `c68fe75ea` not shifted as well?\nI know that this do not gives improvements for the private LB, but at it gives a better signal to noise, and I could know that pseudo labels are more accurate.", "1132767": "Yes there are three training masks that still have significant shifts in them. Still waiting for the sponsors to update them.", "1134298": "tivfrvqhs5 How was this shift in ground truth for afa5e8098 discovered?", "1134802": "This is one way to probe the gt shift https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/201816#1116263", "1135023": "I did not detect much of a shift on c68fe75ea but you can try some shift values yourself and see", "1140042": "Did you apply this shift to all test images or only to afa5e8098", "1140235": "Only afa5e8098. c68fe75ea doesn't appear to have a significant shift and the other three get well above 0.9 so I didn't bother checking. Nice avatar 😊", "1140750": "Oh ok got it, makes sense to use it for the one with a lesser LB score than the others. \nHaha yeah Gilfoyle is the real deal, people just don't get it :P", "1242470": "tivfrvqhs5 will it work with new data ?", "1244233": "nope, problem with data was fixed.", "1301934": "yes, the data have shift" }, "source": "meta" }