I tried both the tiled (256x256) and also simple resize to 1024 and the model I used is the segmentation-models-pytorch unet with different backbones (e.g. efficientnet, segformer e.t.c) also with albumentations but i'm only getting 0.6 at local cv.
\nI get similar results like this notebook. https://www.kaggle.com/code/bibhabasumohapatra/train-hubmap-resized-images
\nIn the end I just gave up…. Can anyone help me and tell me what I was missing please?
", "messages": [ { "id": "1956258", "postDate": "09/26/2022 11:07:42", "content": "I tried both the tiled (256x256) and also simple resize to 1024 and the model I used is the segmentation-models-pytorch unet with different backbones (e.g. efficientnet, segformer e.t.c) also with albumentations but i'm only getting 0.6 at local cv.
\nI get similar results like this notebook. https://www.kaggle.com/code/bibhabasumohapatra/train-hubmap-resized-images
\nIn the end I just gave up…. Can anyone help me and tell me what I was missing please?
", "rawMarkdown": "I tried both the tiled (256x256) and also simple resize to 1024 and the model I used is the segmentation-models-pytorch unet with different backbones (e.g. efficientnet, segformer e.t.c) also with albumentations but i'm only getting 0.6 at local cv. \n\nI get similar results like this notebook. https://www.kaggle.com/code/bibhabasumohapatra/train-hubmap-resized-images\n\nIn the end I just gave up.... Can anyone help me and tell me what I was missing please?", "votes": null }, { "id": "1956353", "postDate": "09/26/2022 12:04:48", "content": "Did you submit your code and check what score you've got? If your LB score is higher than 0.6, then your dice score computation function could be wrong. What was your LB score?
", "rawMarkdown": "Did you submit your code and check what score you've got? If your LB score is higher than 0.6, then your dice score computation function could be wrong. What was your LB score?", "votes": null }, { "id": "1956373", "postDate": "09/26/2022 12:19:34", "content": "I did not submit my code to lb, I was too sad because I was only getting 0.6 while I saw others was getting 0.75+.
\nI dont think there is anything wrong with the dice score computation as I tested it.
\nI used the below to compute the dice score, which returns 0.25 correctly, the average of dice scores of the two example (0.5 and 0).
def DICE_COEFF(mask1, mask2):\n intersect = torch.sum(torch.sum(mask1*mask2,axis=2),axis=2)\n sum1 = torch.sum(torch.sum(mask1,axis=2),axis=2)\n sum2 = torch.sum(torch.sum(mask2,axis=2),axis=2)\n dice = 2*intersect/(sum1+sum2)\n return torch.mean(dice).item()\n\nm1=np.array([[[0,0,1],[0,0,1],[1,0,0]],[[0,0,1],[0,0,1],[1,0,0]]]).reshape((2,1,3,3))\nm2=np.array([[[0,0,0],[0,0,0],[1,0,0]],[[0,0,0],[0,0,0],[0,0,0]]]).reshape((2,1,3,3))\n\nDICE_COEFF(torch.tensor(m1),torch.tensor(m2))\nWhat steps did u take to get local CV score of 0.75+? Did you do anything special during the preprocessing of the images?
\nAlso to add I was using an auxiliary head to predict the class of organ as well.
", "rawMarkdown": "What steps did u take to get local CV score of 0.75+? Did you do anything special during the preprocessing of the images? \n\nAlso to add I was using an auxiliary head to predict the class of organ as well.", "votes": null }, { "id": "1956536", "postDate": "09/26/2022 13:52:13", "content": "IMHO, you have to submit your code first in order to compare your lb score to validation score, regardless if your model converges at 0.6. It'd be a lot easier to debug your code with lb score than without it.
\nYou said you've tried Segformer so I think your \"basic Segformer\" model should have hit higher than 0.65 local cv, with resized images (3000->512). If it didn't, then you have to debug your code to find out what's wrong.
Thank you! I will try again and I may be wrong with the Segformer. I used the SMP library with mit_b2 as the encoder. Which library did you use for the segformer? Also, if I may ask what was your local CV value, and if u have any tips on how to boost it up to 0.75+? That seems still like a big jump
", "rawMarkdown": "Thank you! I will try again and I may be wrong with the Segformer. I used the SMP library with mit_b2 as the encoder. Which library did you use for the segformer? Also, if I may ask what was your local CV value, and if u have any tips on how to boost it up to 0.75+? That seems still like a big jump", "votes": null }, { "id": "1956656", "postDate": "09/26/2022 14:46:11", "content": "Maybe you can try load the pretrained model of segformer. Loading pretrained model is very helpful for me.
", "rawMarkdown": "Maybe you can try load the pretrained model of segformer. Loading pretrained model is very helpful for me.", "votes": null }, { "id": "1956717", "postDate": "09/26/2022 15:16:32", "content": "Thank you, what is your local dice cv score with segformer with a simple dataset?
", "rawMarkdown": "Thank you, what is your local dice cv score with segformer with a simple dataset?", "votes": null }, { "id": "1956768", "postDate": "09/26/2022 15:34:40", "content": "cv 0.80+
\nI forget some details. I will check them tomorrow
I used mmsegmentation to implement Segformer but it won't be different from segformer in the SMP library. My local cv was \"initially\" 0.77~.79 so my main focus was to reduce the gap between cv and lb, not to increase cv. By the way, have you submitted your model? What score you got?
", "rawMarkdown": "I used mmsegmentation to implement Segformer but it won't be different from segformer in the SMP library. My local cv was \"initially\" 0.77~.79 so my main focus was to reduce the gap between cv and lb, not to increase cv. By the way, have you submitted your model? What score you got?", "votes": null }, { "id": "1957395", "postDate": "09/27/2022 00:09:19", "content": "Thank you! One other reason I did not submit to LB is also because of the pixel issue, I did not understand why the different pixel sizes will cause an issue. Did you do any pixel adjustments when submitting to LB?
\nI will create a simple pipeline later and submit to leaderboard
\nTrain 3000->512 with SMP Segformer encoder and albumentations.
\nTest -> Resize to 512 and predict.
\nI will be expecting 0.6+ here probably
", "rawMarkdown": "Thank you! One other reason I did not submit to LB is also because of the pixel issue, I did not understand why the different pixel sizes will cause an issue. Did you do any pixel adjustments when submitting to LB?\n\nI will create a simple pipeline later and submit to leaderboard\n\nTrain 3000->512 with SMP Segformer encoder and albumentations.\n\nTest -> Resize to 512 and predict.\n\nI will be expecting 0.6+ here probably", "votes": null }, { "id": "1957403", "postDate": "09/27/2022 00:34:09", "content": "I think handling the pixel issue is not the priority. Without it, you can hit +.65. In my case, I didn't do anything for the pixel adjustments when submitting. Let me know when you submit your program and its score (including cv too).
", "rawMarkdown": "I think handling the pixel issue is not the priority. Without it, you can hit +.65. In my case, I didn't do anything for the pixel adjustments when submitting. Let me know when you submit your program and its score (including cv too).", "votes": null }, { "id": "1957409", "postDate": "09/27/2022 00:43:57", "content": "Ok! I will post it in 24+hours as I have work now, thanks for helping BTW!!
", "rawMarkdown": "Ok! I will post it in 24+hours as I have work now, thanks for helping BTW!!", "votes": null } ], "comments": [ { "id": 1956353, "author_name": "cheulkay", "author_url": "", "post_date": "09/26/2022 12:04:48", "content": "Did you submit your code and check what score you've got? If your LB score is higher than 0.6, then your dice score computation function could be wrong. What was your LB score?
", "votes": null, "replies": [ { "id": 1956373, "author_name": "kagglerz", "author_url": "", "post_date": "09/26/2022 12:19:34", "content": "I did not submit my code to lb, I was too sad because I was only getting 0.6 while I saw others was getting 0.75+.
\nI dont think there is anything wrong with the dice score computation as I tested it.
\nI used the below to compute the dice score, which returns 0.25 correctly, the average of dice scores of the two example (0.5 and 0).
def DICE_COEFF(mask1, mask2):\n intersect = torch.sum(torch.sum(mask1*mask2,axis=2),axis=2)\n sum1 = torch.sum(torch.sum(mask1,axis=2),axis=2)\n sum2 = torch.sum(torch.sum(mask2,axis=2),axis=2)\n dice = 2*intersect/(sum1+sum2)\n return torch.mean(dice).item()\n\nm1=np.array([[[0,0,1],[0,0,1],[1,0,0]],[[0,0,1],[0,0,1],[1,0,0]]]).reshape((2,1,3,3))\nm2=np.array([[[0,0,0],[0,0,0],[1,0,0]],[[0,0,0],[0,0,0],[0,0,0]]]).reshape((2,1,3,3))\n\nDICE_COEFF(torch.tensor(m1),torch.tensor(m2))\nWhat steps did u take to get local CV score of 0.75+? Did you do anything special during the preprocessing of the images?
\nAlso to add I was using an auxiliary head to predict the class of organ as well.
", "votes": null, "replies": [] }, { "id": 1956536, "author_name": "cheulkay", "author_url": "", "post_date": "09/26/2022 13:52:13", "content": "IMHO, you have to submit your code first in order to compare your lb score to validation score, regardless if your model converges at 0.6. It'd be a lot easier to debug your code with lb score than without it.
\nYou said you've tried Segformer so I think your \"basic Segformer\" model should have hit higher than 0.65 local cv, with resized images (3000->512). If it didn't, then you have to debug your code to find out what's wrong.
Thank you! I will try again and I may be wrong with the Segformer. I used the SMP library with mit_b2 as the encoder. Which library did you use for the segformer? Also, if I may ask what was your local CV value, and if u have any tips on how to boost it up to 0.75+? That seems still like a big jump
", "votes": null, "replies": [] }, { "id": 1957384, "author_name": "cheulkay", "author_url": "", "post_date": "09/26/2022 23:48:18", "content": "I used mmsegmentation to implement Segformer but it won't be different from segformer in the SMP library. My local cv was \"initially\" 0.77~.79 so my main focus was to reduce the gap between cv and lb, not to increase cv. By the way, have you submitted your model? What score you got?
", "votes": null, "replies": [] }, { "id": 1957395, "author_name": "kagglerz", "author_url": "", "post_date": "09/27/2022 00:09:19", "content": "Thank you! One other reason I did not submit to LB is also because of the pixel issue, I did not understand why the different pixel sizes will cause an issue. Did you do any pixel adjustments when submitting to LB?
\nI will create a simple pipeline later and submit to leaderboard
\nTrain 3000->512 with SMP Segformer encoder and albumentations.
\nTest -> Resize to 512 and predict.
\nI will be expecting 0.6+ here probably
", "votes": null, "replies": [] }, { "id": 1957403, "author_name": "cheulkay", "author_url": "", "post_date": "09/27/2022 00:34:09", "content": "I think handling the pixel issue is not the priority. Without it, you can hit +.65. In my case, I didn't do anything for the pixel adjustments when submitting. Let me know when you submit your program and its score (including cv too).
", "votes": null, "replies": [] }, { "id": 1957409, "author_name": "kagglerz", "author_url": "", "post_date": "09/27/2022 00:43:57", "content": "Ok! I will post it in 24+hours as I have work now, thanks for helping BTW!!
", "votes": null, "replies": [] } ] }, { "id": 1956656, "author_name": "gray98", "author_url": "", "post_date": "09/26/2022 14:46:11", "content": "Maybe you can try load the pretrained model of segformer. Loading pretrained model is very helpful for me.
", "votes": null, "replies": [ { "id": 1956717, "author_name": "kagglerz", "author_url": "", "post_date": "09/26/2022 15:16:32", "content": "Thank you, what is your local dice cv score with segformer with a simple dataset?
", "votes": null, "replies": [] }, { "id": 1956768, "author_name": "gray98", "author_url": "", "post_date": "09/26/2022 15:34:40", "content": "cv 0.80+
\nI forget some details. I will check them tomorrow