Hi everyone, the competition has been going on for a while, the first place is far ahead of us, I created a topic, Would you mind share current best single model's CV and LB?
\nmy:
\n`
\nmodel: Unet-se_resnext50
\nimg_sz: 256
\nsplit: random (75/25) by images
\noptim: Adam
\nepoch: ~30
\nloss: symmetric lovasz loss(same as kernel)
CV : 0.895
\nLB: 0.853
\n`
Hi everyone, the competition has been going on for a while, the first place is far ahead of us, I created a topic, Would you mind share current best single model's CV and LB?
\nmy:
\n`
\nmodel: Unet-se_resnext50
\nimg_sz: 256
\nsplit: random (75/25) by images
\noptim: Adam
\nepoch: ~30
\nloss: symmetric lovasz loss(same as kernel)
CV : 0.895
\nLB: 0.853
\n`
model: unet with se_resnet34
\nimg_sz: 512
\nsplit: random choose a .tiff file from all .tiff files
\noptim: Adam
\nepoch: ~60
\nloss: bce
single fold
\ncv: 0.932
\nlb: 0.862
Wow, Good job
", "rawMarkdown": "Wow, Good job", "votes": 1 } ] }, { "id": 1119385, "postDate": "2020-12-20T02:09:27.463Z", "content": "Model: Unet with efficientnetb4
\nsize: 512
\noptimizer: AdamW
\nepoch:30
\nLR: 0.0005
\nLOSS:DICE
\nLB:0.844
Single Fold:
\nTraining 4 folds: gives me cv 0.892, lb 0.855. Need a better ensemble approach
\nUpdate: stronger augmentation, worse local dice, better LB 😲
", "rawMarkdown": "Single Fold:\n- Model: se-resnext50 fpn\n- Training tile size: 256\n- Split: 6 img train, 2 valid\n- Epoch: 60 then pick the checkpoint with best soft dice\n- Loss: symmetric lovasz\n- Local Dice: 0.908 (easy split - other splits are worse)\n- LB: 0.855\n\nTraining 4 folds: gives me cv 0.892, lb 0.855. Need a better ensemble approach\n\nUpdate: stronger augmentation, worse local dice, better LB 😲", "votes": 4, "replies": [ { "id": 1119400, "postDate": "2020-12-20T02:41:24.867Z", "content": "By the way, I think epoch here is pretty trivial. It depends on how you generate training tiles. My pipeline generates 13k 256 tiles but only 4.5k 512 tiles.
", "rawMarkdown": "By the way, I think epoch here is pretty trivial. It depends on how you generate training tiles. My pipeline generates 13k 256 tiles but only 4.5k 512 tiles." }, { "id": 1119478, "postDate": "2020-12-20T05:24:48.637Z", "content": "Thanks for your sharing, Training 4 folds gives me from 0.853 to 0.857, what is your single folds score?
", "rawMarkdown": "Thanks for your sharing, Training 4 folds gives me from 0.853 to 0.857, what is your single folds score?" }, { "id": 1120441, "postDate": "2020-12-20T20:12:20.283Z", "content": "The list of details there is the result of a single fold. \"Training 4 folds gives me cv 0.892, lb 0.855. Need a better ensemble approach\" is the result of training 4 folds. Both training single fold and training 4 folds give me 0.855 LB. I edited the original post to make it clearer
\nQuick Update:
\nAnother single fold trained, even stronger augmentation, worse local dice, more better LB
unet+b4，512，lb0.932
", "rawMarkdown": "unet+b4，512，lb0.932", "votes": 1 }, { "id": 1286796, "postDate": "2021-04-28T12:16:09.813Z", "content": "please don't sink, effb4 + bce, 0.922lb
", "rawMarkdown": "please don't sink, effb4 + bce, 0.922lb", "votes": 1, "replies": [ { "id": 1286863, "postDate": "2021-04-28T13:25:24.870Z", "content": "good job ! what is your scale?
", "rawMarkdown": "good job ! what is your scale?" }, { "id": 1286898, "postDate": "2021-04-28T14:10:08.573Z", "content": "img_size 512, scale 2
", "rawMarkdown": "img_size 512, scale 2" } ] }, { "id": 1292877, "postDate": "2021-05-04T11:19:03.097Z", "content": "UPDATE: EfficientNetb5+BCE, 4TTA 0.938
\n+++++++++++++++++++++++++++
\nEfficientNetb5+BCE, 4TTA ,0.936 LB
did you use unet or fpn?
", "rawMarkdown": "did you use unet or fpn?\n" }, { "id": 1292914, "postDate": "2021-05-04T12:00:39.857Z", "content": "UNet, but I think the network is not the key point, data is the key to improve the score
", "rawMarkdown": "UNet, but I think the network is not the key point, data is the key to improve the score", "votes": 1 }, { "id": 1292998, "postDate": "2021-05-04T13:21:44.200Z", "content": "What scale and size are you use?
\nAlso what about LB score for it without TTA?
scale 2 and size 512.
\nwithout TTA 0.932
Hello,how to use TTA in Deepflash?
", "rawMarkdown": "Hello,how to use TTA in Deepflash?" }, { "id": 1297481, "postDate": "2021-05-08T04:32:53.257Z", "content": "Which TTAs have you used?
", "rawMarkdown": "Which TTAs have you used?" } ] }, { "id": 1118893, "postDate": "2020-12-19T13:55:30.287Z", "content": "In addition, I increased the image scale to 512 (batch size from 128 to 32), but I didn’t get any gain on LB. Do you have this situation?
", "rawMarkdown": "In addition, I increased the image scale to 512 (batch size from 128 to 32), but I didn’t get any gain on LB. Do you have this situation?", "votes": 1, "replies": [ { "id": 1123781, "postDate": "2020-12-23T13:50:44.720Z", "content": "What is the img_sz? Does that mean you reduced the images by 4 times to 256x256?
", "rawMarkdown": "What is the img_sz? Does that mean you reduced the images by 4 times to 256x256?" }, { "id": 1123823, "postDate": "2020-12-23T14:25:24.533Z", "content": "Yes, I try reduced 4 times crop 256x256, reduced 2 times crop 512x512 and reduced 4 times crop 512x512, reduced 4 times crop 256x256 give me a better score.
", "rawMarkdown": "Yes, I try reduced 4 times crop 256x256, reduced 2 times crop 512x512 and reduced 4 times crop 512x512, reduced 4 times crop 256x256 give me a better score." }, { "id": 1123869, "postDate": "2020-12-23T14:49:33.403Z", "content": "Thank you!
", "rawMarkdown": "Thank you!" } ] }, { "id": 1119131, "postDate": "2020-12-19T18:30:29.710Z", "content": "My CV is 0.895 but lb is 0.824 , I do not understand the the gap between the LB and CV
\nModel: Unet with EfficientNet B4 encoder, Segmentation_models (light augmentations)
\nsize: 512
\noptimizer: Adam
\nepoch:~30
\nloss: Jaccard
\ndata: https://www.kaggle.com/wrrosa/hubmap-tfrecords-1024-512
Your LB and CV's gap is really large, maybe you can reference these topic, I get a lot from them:
\nNotebooks:
\nhttps://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter
\nhttps://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train
\nDiscussion:
\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200955
\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626
Any chance that you have leakage between your training and test data?
", "rawMarkdown": "Any chance that you have leakage between your training and test data?" } ] }, { "id": 1119577, "postDate": "2020-12-20T07:36:02.053Z", "content": "Do you really need symmetric_xxx_loss? In other words: Does the image need to be symmetric? Others remain unchanged, after changing to symmetric_bce_loss, cv changes from 87.8=>88.8, but lb changes from 84.5=>83.8.
", "rawMarkdown": "Do you really need symmetric_xxx_loss? In other words: Does the image need to be symmetric? Others remain unchanged, after changing to symmetric_bce_loss, cv changes from 87.8=>88.8, but lb changes from 84.5=>83.8.", "replies": [ { "id": 1119586, "postDate": "2020-12-20T07:53:20.510Z", "content": "Hi, at present, I have not done too many experiments in the loss function. In my experiment, symmetric lovasz loss get a little improve than BCE loss.
", "rawMarkdown": "Hi, at present, I have not done too many experiments in the loss function. In my experiment, symmetric lovasz loss get a little improve than BCE loss." }, { "id": 1119593, "postDate": "2020-12-20T07:59:35.517Z", "content": "thanks,understood
", "rawMarkdown": "thanks,understood" } ] }, { "id": 1119481, "postDate": "2020-12-20T05:25:36.453Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1119480, "postDate": "2020-12-20T05:25:15.490Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1119521, "author_name": "gakki", "author_url": "", "post_date": "2020-12-20T06:13:57.507000", "content": "model: unet with se_resnet34
\nimg_sz: 512
\nsplit: random choose a .tiff file from all .tiff files
\noptim: Adam
\nepoch: ~60
\nloss: bce
single fold
\ncv: 0.932
\nlb: 0.862
Wow, Good job
", "votes": 1, "replies": [] } ] }, { "id": 1119385, "author_name": "HarryWang", "author_url": "", "post_date": "2020-12-20T02:09:27.463000", "content": "Model: Unet with efficientnetb4
\nsize: 512
\noptimizer: AdamW
\nepoch:30
\nLR: 0.0005
\nLOSS:DICE
\nLB:0.844
Single Fold:
\nTraining 4 folds: gives me cv 0.892, lb 0.855. Need a better ensemble approach
\nUpdate: stronger augmentation, worse local dice, better LB 😲
", "votes": 4, "replies": [ { "id": 1119400, "author_name": "Darkate", "author_url": "", "post_date": "2020-12-20T02:41:24.867000", "content": "By the way, I think epoch here is pretty trivial. It depends on how you generate training tiles. My pipeline generates 13k 256 tiles but only 4.5k 512 tiles.
", "votes": 0, "replies": [] }, { "id": 1119478, "author_name": "He", "author_url": "", "post_date": "2020-12-20T05:24:48.637000", "content": "Thanks for your sharing, Training 4 folds gives me from 0.853 to 0.857, what is your single folds score?
", "votes": 0, "replies": [] }, { "id": 1120441, "author_name": "Darkate", "author_url": "", "post_date": "2020-12-20T20:12:20.283000", "content": "The list of details there is the result of a single fold. \"Training 4 folds gives me cv 0.892, lb 0.855. Need a better ensemble approach\" is the result of training 4 folds. Both training single fold and training 4 folds give me 0.855 LB. I edited the original post to make it clearer
\nQuick Update:
\nAnother single fold trained, even stronger augmentation, worse local dice, more better LB
unet+b4，512，lb0.932
", "votes": 1, "replies": [] }, { "id": 1286796, "author_name": "Yi Wu", "author_url": "", "post_date": "2021-04-28T12:16:09.813000", "content": "please don't sink, effb4 + bce, 0.922lb
", "votes": 1, "replies": [ { "id": 1286863, "author_name": "He", "author_url": "", "post_date": "2021-04-28T13:25:24.870000", "content": "good job ! what is your scale?
", "votes": 0, "replies": [] }, { "id": 1286898, "author_name": "Yi Wu", "author_url": "", "post_date": "2021-04-28T14:10:08.573000", "content": "img_size 512, scale 2
", "votes": 0, "replies": [] } ] }, { "id": 1292877, "author_name": "zxyu", "author_url": "", "post_date": "2021-05-04T11:19:03.097000", "content": "UPDATE: EfficientNetb5+BCE, 4TTA 0.938
\n+++++++++++++++++++++++++++
\nEfficientNetb5+BCE, 4TTA ,0.936 LB
did you use unet or fpn?
", "votes": 0, "replies": [] }, { "id": 1292914, "author_name": "zxyu", "author_url": "", "post_date": "2021-05-04T12:00:39.857000", "content": "UNet, but I think the network is not the key point, data is the key to improve the score
", "votes": 1, "replies": [] }, { "id": 1292998, "author_name": "anthony", "author_url": "", "post_date": "2021-05-04T13:21:44.200000", "content": "What scale and size are you use?
\nAlso what about LB score for it without TTA?
scale 2 and size 512.
\nwithout TTA 0.932
Hello,how to use TTA in Deepflash?
", "votes": 0, "replies": [] }, { "id": 1297481, "author_name": "Aman Deep Gupta", "author_url": "", "post_date": "2021-05-08T04:32:53.257000", "content": "Which TTAs have you used?
", "votes": 0, "replies": [] } ] }, { "id": 1118893, "author_name": "He", "author_url": "", "post_date": "2020-12-19T13:55:30.287000", "content": "In addition, I increased the image scale to 512 (batch size from 128 to 32), but I didn’t get any gain on LB. Do you have this situation?
", "votes": 1, "replies": [ { "id": 1123781, "author_name": "pursue", "author_url": "", "post_date": "2020-12-23T13:50:44.720000", "content": "What is the img_sz? Does that mean you reduced the images by 4 times to 256x256?
", "votes": 0, "replies": [] }, { "id": 1123823, "author_name": "He", "author_url": "", "post_date": "2020-12-23T14:25:24.533000", "content": "Yes, I try reduced 4 times crop 256x256, reduced 2 times crop 512x512 and reduced 4 times crop 512x512, reduced 4 times crop 256x256 give me a better score.
", "votes": 0, "replies": [] }, { "id": 1123869, "author_name": "pursue", "author_url": "", "post_date": "2020-12-23T14:49:33.403000", "content": "Thank you!
", "votes": 0, "replies": [] } ] }, { "id": 1119131, "author_name": "Sagar", "author_url": "", "post_date": "2020-12-19T18:30:29.710000", "content": "My CV is 0.895 but lb is 0.824 , I do not understand the the gap between the LB and CV
\nModel: Unet with EfficientNet B4 encoder, Segmentation_models (light augmentations)
\nsize: 512
\noptimizer: Adam
\nepoch:~30
\nloss: Jaccard
\ndata: https://www.kaggle.com/wrrosa/hubmap-tfrecords-1024-512
Your LB and CV's gap is really large, maybe you can reference these topic, I get a lot from them:
\nNotebooks:
\nhttps://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter
\nhttps://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train
\nDiscussion:
\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200955
\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626
Any chance that you have leakage between your training and test data?
", "votes": 0, "replies": [] } ] }, { "id": 1119577, "author_name": "zhangeng", "author_url": "", "post_date": "2020-12-20T07:36:02.053000", "content": "Do you really need symmetric_xxx_loss? In other words: Does the image need to be symmetric? Others remain unchanged, after changing to symmetric_bce_loss, cv changes from 87.8=>88.8, but lb changes from 84.5=>83.8.
", "votes": 0, "replies": [ { "id": 1119586, "author_name": "He", "author_url": "", "post_date": "2020-12-20T07:53:20.510000", "content": "Hi, at present, I have not done too many experiments in the loss function. In my experiment, symmetric lovasz loss get a little improve than BCE loss.
", "votes": 0, "replies": [] }, { "id": 1119593, "author_name": "zhangeng", "author_url": "", "post_date": "2020-12-20T07:59:35.517000", "content": "thanks,understood
", "votes": 0, "replies": [] } ] }, { "id": 1119481, "author_name": "", "author_url": "", "post_date": "2020-12-20T05:25:36.453000", "content": "", "votes": 0, "replies": [] }, { "id": 1119480, "author_name": "", "author_url": "", "post_date": "2020-12-20T05:25:15.490000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1118881": "Hi everyone, the competition has been going on for a while, the first place is far ahead of us, I created a topic, Would you mind share current best single model's CV and LB?\nmy:\n`\nmodel: Unet-se_resnext50\nimg_sz: 256\nsplit: random (75/25) by images\noptim: Adam\nepoch: ~30\nloss: symmetric lovasz loss(same as kernel)\n\nCV : 0.895\nLB: 0.853\n`", "1119521": "model: unet with se_resnet34\nimg_sz: 512\nsplit: random choose a .tiff file from all .tiff files\noptim: Adam\nepoch: ~60\nloss: bce\n\nsingle fold\ncv: 0.932\nlb: 0.862", "1119385": "\nModel: Unet with efficientnetb4 \nsize: 512\noptimizer: AdamW\nepoch:30\nLR: 0.0005\nLOSS:DICE\nLB:0.844 ", "1124246": "* unet resnet34\n* bce\n* ~130 epochs\n* augmentation: hsv, rotate, contrast, noise, flip\n* RAdam + lookahead\n* 1 fold -> 1 image for validation, other training\nprobability treshold at generation csv 0.5 -> LB 0.853, 0.35 ->LB 0.857", "1119197": "Single Fold:\n- Model: se-resnext50 fpn\n- Training tile size: 256\n- Split: 6 img train, 2 valid\n- Epoch: 60 then pick the checkpoint with best soft dice\n- Loss: symmetric lovasz\n- Local Dice: 0.908 (easy split - other splits are worse)\n- LB: 0.855\n\nTraining 4 folds: gives me cv 0.892, lb 0.855. Need a better ensemble approach\n\nUpdate: stronger augmentation, worse local dice, better LB 😲", "1300480": "unet+b4，512，lb0.932", "1286796": "please don't sink, effb4 + bce, 0.922lb", "1292877": "UPDATE: EfficientNetb5+BCE, 4TTA 0.938\n\n+++++++++++++++++++++++++++\nEfficientNetb5+BCE, 4TTA ,0.936 LB", "1118893": "In addition, I increased the image scale to 512 (batch size from 128 to 32), but I didn’t get any gain on LB. Do you have this situation?", "1119131": "My CV is 0.895 but lb is 0.824 , I do not understand the the gap between the LB and CV\nModel: Unet with EfficientNet B4 encoder, Segmentation_models (light augmentations)\nsize: 512\noptimizer: Adam\nepoch:~30\nloss: Jaccard\ndata: https://www.kaggle.com/wrrosa/hubmap-tfrecords-1024-512\n", "1119577": "Do you really need symmetric_xxx_loss? In other words: Does the image need to be symmetric? Others remain unchanged, after changing to symmetric_bce_loss, cv changes from 87.8=>88.8, but lb changes from 84.5=>83.8.", "1119481": "", "1119480": "" } }