![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8569374%2F8f2f99a9af1303c1e0ee3a8958018ef4%2Foutput.png?generation=1663907208170410&alt=media\")
I'm very inexperienced because It's my first time to compete as a kaggler, and I have learnt so much from this competition! I will share some tricks which really work in my experiments. Following sores mentioned are reached by single model(5 folds).
\nbackbone: resnext-50
\nimage size: (256,256)
\ntile: True
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.225,
\n lung=0.225
\ncutmix: True
\nWe just used unext without any other tricks. And we tiled the original size into 1024 and then resized 1024 to 256.
Thanks for the sharing from https://www.kaggle.com/hengck23 who really helped us.
\nbackbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.225,
\n lung=0.225
backbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.1,
\n lung=0.1
\ncutout: True
\ncutout can help accelerate Training and improve your scores! A better threshold can bring very much improvement.
backbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.1,
\n lung=0.1
\ncutout: True
\nmultiple initialization: True
\nstain tools: True
\nMultiple initialization is helpful to model ensembling!
init = [xavier_uniform_init,xavier_normal_init,he_init,kiming_init,orthogonal_init]\nmodel.aux.apply(init[fold%len(init)])\nmodel.head.apply(init[fold%len(init)])\nmodel.logit.apply(init[fold%len(init)])\nbackbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.1,
\n lung=0.1
\ncutout: True
\nmultiple initialization: True
\npseudo label: True
\nstain tools: True
\nWe only use the pseudo label for lung while using the pseudo label for other organs will be harmful to model.
backbone: coat
\nimage size: (1024,1024)
\ntile False
\nthreshold: kidney=0.45,
\n prostate=0.45,
\n largeintestine=0.5,
\n spleen=0.2,
\n lung=0.1
\ncutout: True
\nmultiple initialization: True
\npseudo label: True
\nstain tools: True
\nThanks for the sharing from https://www.kaggle.com/hengck23 again!
We have spent too much time experimenting on cutmix which means we paste the label copied from another sample so that we don't have enough time to do more helpful experiments just like I have mentioned. We are anxious that cutmix will work well but the truth is that it is harmful to training. However, we used self-cutmix for model ensemble and the probability is 100% which means some models are trained on very different data. In this way, the harm can be controlled and it's helpful for ensembling.
\nUpdating……………
", "messages": [ { "id": 1951305, "postDate": "2022-09-23T02:11:02.710Z", "content": "I'm very inexperienced because It's my first time to compete as a kaggler, and I have learnt so much from this competition! I will share some tricks which really work in my experiments. Following sores mentioned are reached by single model(5 folds).
\nbackbone: resnext-50
\nimage size: (256,256)
\ntile: True
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.225,
\n lung=0.225
\ncutmix: True
\nWe just used unext without any other tricks. And we tiled the original size into 1024 and then resized 1024 to 256.
Thanks for the sharing from https://www.kaggle.com/hengck23 who really helped us.
\nbackbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.225,
\n lung=0.225
backbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.1,
\n lung=0.1
\ncutout: True
\ncutout can help accelerate Training and improve your scores! A better threshold can bring very much improvement.
backbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.1,
\n lung=0.1
\ncutout: True
\nmultiple initialization: True
\nstain tools: True
\nMultiple initialization is helpful to model ensembling!
init = [xavier_uniform_init,xavier_normal_init,he_init,kiming_init,orthogonal_init]\nmodel.aux.apply(init[fold%len(init)])\nmodel.head.apply(init[fold%len(init)])\nmodel.logit.apply(init[fold%len(init)])\nbackbone: mit-b2
\nimage size: (768,768)
\ntile False
\nthreshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.1,
\n lung=0.1
\ncutout: True
\nmultiple initialization: True
\npseudo label: True
\nstain tools: True
\nWe only use the pseudo label for lung while using the pseudo label for other organs will be harmful to model.
backbone: coat
\nimage size: (1024,1024)
\ntile False
\nthreshold: kidney=0.45,
\n prostate=0.45,
\n largeintestine=0.5,
\n spleen=0.2,
\n lung=0.1
\ncutout: True
\nmultiple initialization: True
\npseudo label: True
\nstain tools: True
\nThanks for the sharing from https://www.kaggle.com/hengck23 again!
We have spent too much time experimenting on cutmix which means we paste the label copied from another sample so that we don't have enough time to do more helpful experiments just like I have mentioned. We are anxious that cutmix will work well but the truth is that it is harmful to training. However, we used self-cutmix for model ensemble and the probability is 100% which means some models are trained on very different data. In this way, the harm can be controlled and it's helpful for ensembling.
\nUpdating……………
", "rawMarkdown": "I'm very inexperienced because It's my first time to compete as a kaggler, and I have learnt so much from this competition! I will share some tricks which really work in my experiments. Following sores mentioned are reached by single model(5 folds).\n## Expriments process\n### 0.68\n**backbone**: resnext-50\n**image size**: (256,256)\n**tile**: True\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.225,\n lung=0.225\n**cutmix**: True\nWe just used unext without any other tricks. And we tiled the original size into 1024 and then resized 1024 to 256.\n### 0.76\nThanks for the sharing from [https://www.kaggle.com/hengck23](url) who really helped us.\n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.225,\n lung=0.225\n### 0.79 \n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.1,\n **lung**=0.1\n**cutout**: True\ncutout can help accelerate Training and improve your scores! A better threshold can bring very much improvement.\n### 0.80 \n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.1,\n lung=0.1\n**cutout**: True\n**multiple initialization**: True\n**stain tools**: True\nMultiple initialization is helpful to model ensembling!\n\n init = [xavier_uniform_init,xavier_normal_init,he_init,kiming_init,orthogonal_init]\n model.aux.apply(init[fold%len(init)])\n model.head.apply(init[fold%len(init)])\n model.logit.apply(init[fold%len(init)])\n### 0.81\n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.1,\n lung=0.1\n**cutout**: True\n**multiple initialization**: True\n**pseudo label**: True\n**stain tools**: True\nWe only use the pseudo label for lung while using the pseudo label for other organs will be harmful to model.\n### 0.82\n**backbone**: coat\n**image size**: (1024,1024)\n**tile** False\n**threshold**: kidney=0.45,\n prostate=0.45,\n largeintestine=0.5,\n spleen=0.2,\n lung=0.1\n**cutout**: True\n**multiple initialization**: True\n**pseudo label**: True\n**stain tools**: True\nThanks for the sharing from [https://www.kaggle.com/hengck23](url) again!\n### Other experiments\nWe have spent too much time experimenting on cutmix which means we paste the label copied from another sample so that we don't have enough time to do more helpful experiments just like I have mentioned. We are anxious that cutmix will work well but the truth is that it is harmful to training. However, we used self-cutmix for model ensemble and the probability is 100% which means some models are trained on very different data. In this way, the harm can be controlled and it's helpful for ensembling.\n#### I really want to know how to reach 0.83 or 0.84? Select the best models from dozens models you have trained? Find better thresholds for different organs? Larger image size like 1536?\nUpdating...............", "votes": 25 }, { "id": 1951449, "postDate": "2022-09-23T04:26:04.477Z", "content": "For me, get 0.83 in pb just use 768 and 1024 ensemble. But in lb is 0.81, but some submiision with 0.82 can reach 0.82 in lb, so i chose the wront.
", "rawMarkdown": "For me, get 0.83 in pb just use 768 and 1024 ensemble. But in lb is 0.81, but some submiision with 0.82 can reach 0.82 in lb, so i chose the wront.", "votes": 1, "replies": [ { "id": 1951457, "postDate": "2022-09-23T04:32:29.473Z", "content": "Thanks for sharing! I have thought about use 1536 and 1024 ensemble but we don't have enough time to do more experiments😭. What a pity!
", "rawMarkdown": "Thanks for sharing! I have thought about use 1536 and 1024 ensemble but we don't have enough time to do more experiments😭. What a pity!" } ] }, { "id": 1951322, "postDate": "2022-09-23T02:20:54.747Z", "content": "I used the stain tools for tranfer learning, and 50% stained data will be used while training and validing.https://www.kaggle.com/code/chinartist/resize-to-1024-stain/data
", "rawMarkdown": "I used the stain tools for tranfer learning, and 50% stained data will be used while training and validing.[https://www.kaggle.com/code/chinartist/resize-to-1024-stain/data](url)", "votes": 1 }, { "id": 1972593, "postDate": "2022-10-05T08:27:03.127Z", "content": "\n\n0.68
\n
\n backbone: resnext-50
\n image size: (256,256)
\n tile: True
\n threshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.225,
\n lung=0.225
\n We just used unext without any other tricks. And we tiled the original size into 1024 and then resized 1024 to 256.
Did you get this score without any color augmentation and pixel size modification?
\nI have tried heavy augmentation and pixel size modification and etc… but I can't score LB 0.65+.
You can try cutmix
", "rawMarkdown": "You can try cutmix" } ] }, { "id": 1954240, "postDate": "2022-09-25T05:33:43.337Z", "content": "Hello, I used the same training method as you in the first experiment (tile 256 and 5-folds), but I can only achieve a score of 0.58. Did you randomly divide the data into 5 folds and then do the training? Can you share your specific training method? I'm new to image segmentation, so I don't know what went wrong.
", "rawMarkdown": "Hello, I used the same training method as you in the first experiment (tile 256 and 5-folds), but I can only achieve a score of 0.58. Did you randomly divide the data into 5 folds and then do the training? Can you share your specific training method? I'm new to image segmentation, so I don't know what went wrong.", "replies": [ { "id": 1954251, "postDate": "2022-09-25T06:05:20.933Z", "content": "In fact, I used my cutmix in the first experiment which would improve a lot when the score was not high. But when I begin other experiments, my cutmix performered so pooly that I didn't want to use it anymore. If you want to reach 0.68 in the first experiment, you may need cutmix. Some contestants has found cutmix may over fit their train set so you should be careful.
", "rawMarkdown": "In fact, I used my cutmix in the first experiment which would improve a lot when the score was not high. But when I begin other experiments, my cutmix performered so pooly that I didn't want to use it anymore. If you want to reach 0.68 in the first experiment, you may need cutmix. Some contestants has found cutmix may over fit their train set so you should be careful." } ] }, { "id": 1952231, "postDate": "2022-09-23T14:12:36.403Z", "content": "Accordding to the third place resolution, I just have found why my cutmix performs very poorly. Instead of only selecting the masked part of another example, we should randomly select a part. I guess the resolution of us will over fit the hpa dataset.
", "rawMarkdown": "Accordding to the third place resolution, I just have found why my cutmix performs very poorly. Instead of only selecting the masked part of another example, we should randomly select a part. I guess the resolution of us will over fit the hpa dataset.", "replies": [ { "id": 1952271, "postDate": "2022-09-23T14:36:50.860Z", "content": "I missed the problem too, cutmix does overfit hpa for me.
", "rawMarkdown": "I missed the problem too, cutmix does overfit hpa for me." }, { "id": 1972591, "postDate": "2022-10-05T08:24:34.340Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1951897, "postDate": "2022-09-23T10:08:39.310Z", "content": "Thanks for sharing the experiment process 😃.
\nI'm curious about the multiple initialization approach for model ensembling.
\nIs there any reference for this kind of method?
I'm not sure if anyone has advanced reserch on this. It just works very well.
", "rawMarkdown": "I'm not sure if anyone has advanced reserch on this. It just works very well.", "votes": 1 } ] }, { "id": 1951446, "postDate": "2022-09-23T04:23:09.333Z", "content": "Maybe cutmix will destroy the texture which may be important for training?
", "rawMarkdown": "Maybe cutmix will destroy the texture which may be important for training? ", "replies": [ { "id": 1951448, "postDate": "2022-09-23T04:24:56.347Z", "content": "
Congratulation on the silver! one quick question
\nIs that transformed image output of cutmix?
Yes, just copy and paste on the same image.
", "rawMarkdown": "Yes, just copy and paste on the same image." } ] }, { "id": 1951354, "postDate": "2022-09-23T02:50:48.127Z", "content": "Very well-explained experiments. Thanks for sharing your knowledge.
", "rawMarkdown": "Very well-explained experiments. Thanks for sharing your knowledge." } ], "comments": [ { "id": 1951449, "author_name": "Ksana", "author_url": "", "post_date": "2022-09-23T04:26:04.477000", "content": "For me, get 0.83 in pb just use 768 and 1024 ensemble. But in lb is 0.81, but some submiision with 0.82 can reach 0.82 in lb, so i chose the wront.
", "votes": 1, "replies": [ { "id": 1951457, "author_name": "Chinartist", "author_url": "", "post_date": "2022-09-23T04:32:29.473000", "content": "Thanks for sharing! I have thought about use 1536 and 1024 ensemble but we don't have enough time to do more experiments😭. What a pity!
", "votes": 0, "replies": [] } ] }, { "id": 1951322, "author_name": "Chinartist", "author_url": "", "post_date": "2022-09-23T02:20:54.747000", "content": "I used the stain tools for tranfer learning, and 50% stained data will be used while training and validing.https://www.kaggle.com/code/chinartist/resize-to-1024-stain/data
", "votes": 1, "replies": [] }, { "id": 1972593, "author_name": "yoshoo", "author_url": "", "post_date": "2022-10-05T08:27:03.127000", "content": "\n\n0.68
\n
\n backbone: resnext-50
\n image size: (256,256)
\n tile: True
\n threshold: kidney=0.225,
\n prostate=0.225,
\n largeintestine=0.225,
\n spleen=0.225,
\n lung=0.225
\n We just used unext without any other tricks. And we tiled the original size into 1024 and then resized 1024 to 256.
Did you get this score without any color augmentation and pixel size modification?
\nI have tried heavy augmentation and pixel size modification and etc… but I can't score LB 0.65+.
You can try cutmix
", "votes": 0, "replies": [] } ] }, { "id": 1954240, "author_name": "yuans hug", "author_url": "", "post_date": "2022-09-25T05:33:43.337000", "content": "Hello, I used the same training method as you in the first experiment (tile 256 and 5-folds), but I can only achieve a score of 0.58. Did you randomly divide the data into 5 folds and then do the training? Can you share your specific training method? I'm new to image segmentation, so I don't know what went wrong.
", "votes": 0, "replies": [ { "id": 1954251, "author_name": "Chinartist", "author_url": "", "post_date": "2022-09-25T06:05:20.933000", "content": "In fact, I used my cutmix in the first experiment which would improve a lot when the score was not high. But when I begin other experiments, my cutmix performered so pooly that I didn't want to use it anymore. If you want to reach 0.68 in the first experiment, you may need cutmix. Some contestants has found cutmix may over fit their train set so you should be careful.
", "votes": 0, "replies": [] } ] }, { "id": 1952231, "author_name": "Chinartist", "author_url": "", "post_date": "2022-09-23T14:12:36.403000", "content": "Accordding to the third place resolution, I just have found why my cutmix performs very poorly. Instead of only selecting the masked part of another example, we should randomly select a part. I guess the resolution of us will over fit the hpa dataset.
", "votes": 0, "replies": [ { "id": 1952271, "author_name": "Ksana", "author_url": "", "post_date": "2022-09-23T14:36:50.860000", "content": "I missed the problem too, cutmix does overfit hpa for me.
", "votes": 0, "replies": [] }, { "id": 1972591, "author_name": "", "author_url": "", "post_date": "2022-10-05T08:24:34.340000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1951897, "author_name": "Kuo-Hsin Tu", "author_url": "", "post_date": "2022-09-23T10:08:39.310000", "content": "Thanks for sharing the experiment process 😃.
\nI'm curious about the multiple initialization approach for model ensembling.
\nIs there any reference for this kind of method?
I'm not sure if anyone has advanced reserch on this. It just works very well.
", "votes": 1, "replies": [] } ] }, { "id": 1951446, "author_name": "Chinartist", "author_url": "", "post_date": "2022-09-23T04:23:09.333000", "content": "Maybe cutmix will destroy the texture which may be important for training?
", "votes": 0, "replies": [ { "id": 1951448, "author_name": "Chinartist", "author_url": "", "post_date": "2022-09-23T04:24:56.347000", "content": "
Congratulation on the silver! one quick question
\nIs that transformed image output of cutmix?
Yes, just copy and paste on the same image.
", "votes": 0, "replies": [] } ] }, { "id": 1951354, "author_name": "Mugdha Hardikar", "author_url": "", "post_date": "2022-09-23T02:50:48.127000", "content": "Very well-explained experiments. Thanks for sharing your knowledge.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1951305": "I'm very inexperienced because It's my first time to compete as a kaggler, and I have learnt so much from this competition! I will share some tricks which really work in my experiments. Following sores mentioned are reached by single model(5 folds).\n## Expriments process\n### 0.68\n**backbone**: resnext-50\n**image size**: (256,256)\n**tile**: True\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.225,\n lung=0.225\n**cutmix**: True\nWe just used unext without any other tricks. And we tiled the original size into 1024 and then resized 1024 to 256.\n### 0.76\nThanks for the sharing from [https://www.kaggle.com/hengck23](url) who really helped us.\n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.225,\n lung=0.225\n### 0.79 \n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.1,\n **lung**=0.1\n**cutout**: True\ncutout can help accelerate Training and improve your scores! A better threshold can bring very much improvement.\n### 0.80 \n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.1,\n lung=0.1\n**cutout**: True\n**multiple initialization**: True\n**stain tools**: True\nMultiple initialization is helpful to model ensembling!\n\n init = [xavier_uniform_init,xavier_normal_init,he_init,kiming_init,orthogonal_init]\n model.aux.apply(init[fold%len(init)])\n model.head.apply(init[fold%len(init)])\n model.logit.apply(init[fold%len(init)])\n### 0.81\n**backbone**: mit-b2\n**image size**: (768,768)\n**tile** False\n**threshold**: kidney=0.225,\n prostate=0.225,\n largeintestine=0.225,\n spleen=0.1,\n lung=0.1\n**cutout**: True\n**multiple initialization**: True\n**pseudo label**: True\n**stain tools**: True\nWe only use the pseudo label for lung while using the pseudo label for other organs will be harmful to model.\n### 0.82\n**backbone**: coat\n**image size**: (1024,1024)\n**tile** False\n**threshold**: kidney=0.45,\n prostate=0.45,\n largeintestine=0.5,\n spleen=0.2,\n lung=0.1\n**cutout**: True\n**multiple initialization**: True\n**pseudo label**: True\n**stain tools**: True\nThanks for the sharing from [https://www.kaggle.com/hengck23](url) again!\n### Other experiments\nWe have spent too much time experimenting on cutmix which means we paste the label copied from another sample so that we don't have enough time to do more helpful experiments just like I have mentioned. We are anxious that cutmix will work well but the truth is that it is harmful to training. However, we used self-cutmix for model ensemble and the probability is 100% which means some models are trained on very different data. In this way, the harm can be controlled and it's helpful for ensembling.\n#### I really want to know how to reach 0.83 or 0.84? Select the best models from dozens models you have trained? Find better thresholds for different organs? Larger image size like 1536?\nUpdating...............", "1951449": "For me, get 0.83 in pb just use 768 and 1024 ensemble. But in lb is 0.81, but some submiision with 0.82 can reach 0.82 in lb, so i chose the wront.", "1951322": "I used the stain tools for tranfer learning, and 50% stained data will be used while training and validing.[https://www.kaggle.com/code/chinartist/resize-to-1024-stain/data](url)", "1972593": "> 0.68\nbackbone: resnext-50\nimage size: (256,256)\ntile: True\nthreshold: kidney=0.225,\nprostate=0.225,\nlargeintestine=0.225,\nspleen=0.225,\nlung=0.225\nWe just used unext without any other tricks. And we tiled the original size into 1024 and then resized 1024 to 256.\n\nDid you get this score without any color augmentation and pixel size modification?\nI have tried heavy augmentation and pixel size modification and etc... but I can't score LB 0.65+.", "1954240": "Hello, I used the same training method as you in the first experiment (tile 256 and 5-folds), but I can only achieve a score of 0.58. Did you randomly divide the data into 5 folds and then do the training? Can you share your specific training method? I'm new to image segmentation, so I don't know what went wrong.", "1952231": "Accordding to the third place resolution, I just have found why my cutmix performs very poorly. Instead of only selecting the masked part of another example, we should randomly select a part. I guess the resolution of us will over fit the hpa dataset.", "1951897": "Thanks for sharing the experiment process 😃.\nI'm curious about the multiple initialization approach for model ensembling.\nIs there any reference for this kind of method?", "1951446": "Maybe cutmix will destroy the texture which may be important for training? ", "1951354": "Very well-explained experiments. Thanks for sharing your knowledge." } }