![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9334354%2F74ad73b605892f8de8d3bc401dae6d0a%2Fmake%20pseudo%20labels.png?generation=1690855159125845&alt=media\")
I am so happy to have won my first gold medal and even a solo medal in this competition. Also thanks to the OpenMMLab, SenseTime and The Chinese University of Hong Kong for making a really great library mmdetection.
\nSummary
\nI used the pseudo labels of Dataset3 to train the Cascade Mask RCNN + Convnext v2 large.
Model
\nAugmentation
\n (\n =,\n transforms=[\n (=, p=),\n (=, p=),\n (=, p=),\n ], p=),\nTraining process
\nFirst, weighted segment fusion was performed on the output values from the 4 models made of 4 types of data sets to create pseudo labels for Dataset3.
\nI did WSF by referring to the guide below -> https://www.kaggle.com/code/mistag/sartorius-tta-with-weighted-segments-fusion
\n
second. The model was pre-trained using the pseudo label of Dataset 3 and the human label of Dataset 2.
\nFinally, the model was fine-tuned using only Dataset1 data and submitted.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9334354%2F49ef424a4d828f38c7e44d374a2021c2%2Ffinetune.png?generation=1690855729462161&alt=media\")
I am so happy to have won my first gold medal and even a solo medal in this competition. Also thanks to the OpenMMLab, SenseTime and The Chinese University of Hong Kong for making a really great library mmdetection.
\nSummary
\nI used the pseudo labels of Dataset3 to train the Cascade Mask RCNN + Convnext v2 large.
Model
\nAugmentation
\n (\n =,\n transforms=[\n (=, p=),\n (=, p=),\n (=, p=),\n ], p=),\nTraining process
\nFirst, weighted segment fusion was performed on the output values from the 4 models made of 4 types of data sets to create pseudo labels for Dataset3.
\nI did WSF by referring to the guide below -> https://www.kaggle.com/code/mistag/sartorius-tta-with-weighted-segments-fusion
\n
second. The model was pre-trained using the pseudo label of Dataset 3 and the human label of Dataset 2.
\nFinally, the model was fine-tuned using only Dataset1 data and submitted.
\n
Hi, what is the concept of pseudo label here?
\nThanks in advance.
", "rawMarkdown": "Hi, what is the concept of pseudo label here?\n\nThanks in advance.", "votes": null }, { "id": "2368145", "postDate": "08/01/2023 03:08:12", "content": "Hi What is the meaning of the concept here?
\nI thought that there would be differences for each WSI, so I created 4 folds for each WSI. The predicted values of the four models thus created were WSFed and used as pseudo labels. It is.
", "rawMarkdown": "Hi What is the meaning of the concept here?\n\nI thought that there would be differences for each WSI, so I created 4 folds for each WSI. The predicted values of the four models thus created were WSFed and used as pseudo labels. It is.", "votes": null }, { "id": "2368149", "postDate": "08/01/2023 03:11:43", "content": "Congrats on solo gold, but I want to correct you that the mmdetection is made by OpenMMLab which is created by SenseTime and The Chinese University of Hong Kong😌
", "rawMarkdown": "Congrats on solo gold, but I want to correct you that the mmdetection is made by OpenMMLab which is created by SenseTime and The Chinese University of Hong Kong😌", "votes": null }, { "id": "2368151", "postDate": "08/01/2023 03:14:04", "content": "Congrats for the solo gold medal!
\nI have 2 questions:
Sorry My mistake, Thanks for the advice.
", "rawMarkdown": "Sorry My mistake, Thanks for the advice.", "votes": null }, { "id": "2368155", "postDate": "08/01/2023 03:21:34", "content": "Sorry for not being able to explain clearly.
", "rawMarkdown": "1. yes, \"best epoch model pth\" with Train (dataset 2 of wsi 1,2 and dataset 3) valid (only dataset 1)\n2. no i didn't freeze layer\n\nSorry for not being able to explain clearly.", "votes": null }, { "id": "2368189", "postDate": "08/01/2023 03:59:52", "content": "Thank you so much for your response!
\nThis is very similar with my first approach for this competition. But i don't use this because i think this is data leakage. What do you think about this ?
Yes, I thought it was difficult to trust the validation score because Dataset1 was used together when pseudo labeling was performed.
\nTherefore, this idea was not applied right away, but was applied after finding the best hyperparameters and epochs through CV until just before the end of the competition.
", "rawMarkdown": "Yes, I thought it was difficult to trust the validation score because Dataset1 was used together when pseudo labeling was performed.\n\nTherefore, this idea was not applied right away, but was applied after finding the best hyperparameters and epochs through CV until just before the end of the competition.", "votes": null }, { "id": "2368250", "postDate": "08/01/2023 04:49:11", "content": "Congrats on the medal @devchopin !
\nEven I am a university student (just completed my first year). I have not taken any formal courses yet but I try to read up on articles and am currently reading hands on ML with sci-kit learn and TensorFlow. How exactly do you go about preparing for competitions and how much time do you spend?
", "rawMarkdown": "Congrats on the medal @devchopin !\n\nEven I am a university student (just completed my first year). I have not taken any formal courses yet but I try to read up on articles and am currently reading hands on ML with sci-kit learn and TensorFlow. How exactly do you go about preparing for competitions and how much time do you spend?", "votes": null }, { "id": "2368272", "postDate": "08/01/2023 05:04:48", "content": "Since I am also a beginner, I am cautious about giving advice, but in my case, participating in various competitions was helpful. I also read a lot of META research papers.
", "rawMarkdown": "Since I am also a beginner, I am cautious about giving advice, but in my case, participating in various competitions was helpful. I also read a lot of META research papers.", "votes": null }, { "id": "2368398", "postDate": "08/01/2023 06:37:56", "content": "Thank you
", "rawMarkdown": "Thank you", "votes": null }, { "id": "2368764", "postDate": "08/01/2023 11:15:19", "content": "Congrads on your solo gold! I think this pseudo labeling method is insightful. I want to know a little detail about it. When you fine tuning the pretrained model, what validation data you use for it?(you say \"under the same conditions\", you mean you use 20% dataset1 to validate for fine tuning stage too?)
", "rawMarkdown": "Congrads on your solo gold! I think this pseudo labeling method is insightful. I want to know a little detail about it. When you fine tuning the pretrained model, what validation data you use for it?(you say \"under the same conditions\", you mean you use 20% dataset1 to validate for fine tuning stage too?)", "votes": null }, { "id": "2369631", "postDate": "08/01/2023 22:12:22", "content": "The pseudo labels were blood vessel, glomerulus and unsure?.
\nThanks in advance.
", "rawMarkdown": "The pseudo labels were blood vessel, glomerulus and unsure?.\n\nThanks in advance.", "votes": null }, { "id": "2369673", "postDate": "08/02/2023 00:02:08", "content": "Only blood vessel, Thank you
", "rawMarkdown": "Only blood vessel, Thank you", "votes": null }, { "id": "2369676", "postDate": "08/02/2023 00:11:19", "content": "Hello, thank you for your kind words.
\nThere are a total of 4 steps.
\nCreate a pre-trained model using Dataset2 + Dataset3 data. (24 epoch was the best result for me here.)
Train Dataset1 data from 24 epoch(from pre-train model). Here, it was verified using train (80%) and validation (20%). (This step is to find out which epoch is best.)
I found that training up to 28 epochs in step 2 is the best mAP60.
Finally, submit the model trained on 28 epochs of 100% of Dataset1 data.
If there is something I don't understand, I will answer again. thank you
", "rawMarkdown": "Hello, thank you for your kind words.\n\nThere are a total of 4 steps.\n\n1. Create a pre-trained model using Dataset2 + Dataset3 data. (24 epoch was the best result for me here.)\n\n2. Train Dataset1 data from 24 epoch(from pre-train model). Here, it was verified using train (80%) and validation (20%). (This step is to find out which epoch is best.)\n\n3. I found that training up to 28 epochs in step 2 is the best mAP60.\n\n4. Finally, submit the model trained on 28 epochs of 100% of Dataset1 data.\n\nIf there is something I don't understand, I will answer again. thank you", "votes": null }, { "id": "2369760", "postDate": "08/02/2023 02:30:15", "content": "I understand, thank you
", "rawMarkdown": "I understand, thank you", "votes": null }, { "id": "2369765", "postDate": "08/02/2023 02:40:18", "content": "Congratulations on your solo gold!
\nI have a question for you. I assume that the model used for pseudo-labeling contains a fair amount of low-score false positives. After applying the WSF, did you pseudo-label all the predicted masks directly, or did you apply any post-processing to the masks before pseudo-labeling them? (e.g., excluding small instances, filtering by probability values)
", "rawMarkdown": "Congratulations on your solo gold! \n\nI have a question for you. I assume that the model used for pseudo-labeling contains a fair amount of low-score false positives. After applying the WSF, did you pseudo-label all the predicted masks directly, or did you apply any post-processing to the masks before pseudo-labeling them? (e.g., excluding small instances, filtering by probability values)", "votes": null }, { "id": "2369769", "postDate": "08/02/2023 02:46:14", "content": "there is two step
\nThank you for your response!
\n I understand your explanation.
Hi, what is the concept of pseudo label here?
\nThanks in advance.
", "votes": null, "replies": [ { "id": 2368145, "author_name": "devchopin", "author_url": "", "post_date": "08/01/2023 03:08:12", "content": "Hi What is the meaning of the concept here?
\nI thought that there would be differences for each WSI, so I created 4 folds for each WSI. The predicted values of the four models thus created were WSFed and used as pseudo labels. It is.
", "votes": null, "replies": [ { "id": 2369631, "author_name": "pablolarrosa", "author_url": "", "post_date": "08/01/2023 22:12:22", "content": "The pseudo labels were blood vessel, glomerulus and unsure?.
\nThanks in advance.
", "votes": null, "replies": [ { "id": 2369673, "author_name": "devchopin", "author_url": "", "post_date": "08/02/2023 00:02:08", "content": "Only blood vessel, Thank you
", "votes": null, "replies": [] } ] }, { "id": 2369765, "author_name": "yuyagi", "author_url": "", "post_date": "08/02/2023 02:40:18", "content": "Congratulations on your solo gold!
\nI have a question for you. I assume that the model used for pseudo-labeling contains a fair amount of low-score false positives. After applying the WSF, did you pseudo-label all the predicted masks directly, or did you apply any post-processing to the masks before pseudo-labeling them? (e.g., excluding small instances, filtering by probability values)
", "votes": null, "replies": [ { "id": 2369769, "author_name": "devchopin", "author_url": "", "post_date": "08/02/2023 02:46:14", "content": "there is two step
\nThank you for your response!
\n I understand your explanation.
Congrats on solo gold, but I want to correct you that the mmdetection is made by OpenMMLab which is created by SenseTime and The Chinese University of Hong Kong😌
", "votes": null, "replies": [ { "id": 2368153, "author_name": "devchopin", "author_url": "", "post_date": "08/01/2023 03:15:37", "content": "Sorry My mistake, Thanks for the advice.
", "votes": null, "replies": [] } ] }, { "id": 2368151, "author_name": "researchbntz", "author_url": "", "post_date": "08/01/2023 03:14:04", "content": "Congrats for the solo gold medal!
\nI have 2 questions:
Sorry for not being able to explain clearly.
", "votes": null, "replies": [ { "id": 2368189, "author_name": "researchbntz", "author_url": "", "post_date": "08/01/2023 03:59:52", "content": "Thank you so much for your response!
\nThis is very similar with my first approach for this competition. But i don't use this because i think this is data leakage. What do you think about this ?
Yes, I thought it was difficult to trust the validation score because Dataset1 was used together when pseudo labeling was performed.
\nTherefore, this idea was not applied right away, but was applied after finding the best hyperparameters and epochs through CV until just before the end of the competition.
", "votes": null, "replies": [ { "id": 2368398, "author_name": "researchbntz", "author_url": "", "post_date": "08/01/2023 06:37:56", "content": "Thank you
", "votes": null, "replies": [] } ] } ] } ] } ] }, { "id": 2368764, "author_name": "m1dsolo", "author_url": "", "post_date": "08/01/2023 11:15:19", "content": "Congrads on your solo gold! I think this pseudo labeling method is insightful. I want to know a little detail about it. When you fine tuning the pretrained model, what validation data you use for it?(you say \"under the same conditions\", you mean you use 20% dataset1 to validate for fine tuning stage too?)
", "votes": null, "replies": [ { "id": 2369676, "author_name": "devchopin", "author_url": "", "post_date": "08/02/2023 00:11:19", "content": "Hello, thank you for your kind words.
\nThere are a total of 4 steps.
\nCreate a pre-trained model using Dataset2 + Dataset3 data. (24 epoch was the best result for me here.)
Train Dataset1 data from 24 epoch(from pre-train model). Here, it was verified using train (80%) and validation (20%). (This step is to find out which epoch is best.)
I found that training up to 28 epochs in step 2 is the best mAP60.
Finally, submit the model trained on 28 epochs of 100% of Dataset1 data.
If there is something I don't understand, I will answer again. thank you
", "votes": null, "replies": [ { "id": 2369760, "author_name": "m1dsolo", "author_url": "", "post_date": "08/02/2023 02:30:15", "content": "I understand, thank you
", "votes": null, "replies": [] } ] } ] } ] } ] }, { "id": 2368250, "author_name": "arnavmodi", "author_url": "", "post_date": "08/01/2023 04:49:11", "content": "Congrats on the medal @devchopin !
\nEven I am a university student (just completed my first year). I have not taken any formal courses yet but I try to read up on articles and am currently reading hands on ML with sci-kit learn and TensorFlow. How exactly do you go about preparing for competitions and how much time do you spend?
", "votes": null, "replies": [ { "id": 2368272, "author_name": "devchopin", "author_url": "", "post_date": "08/01/2023 05:04:48", "content": "Since I am also a beginner, I am cautious about giving advice, but in my case, participating in various competitions was helpful. I also read a lot of META research papers.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2368093": "I am so happy to have won my first gold medal and even a solo medal in this competition. Also thanks to the OpenMMLab, SenseTime and The Chinese University of Hong Kong for making a really great library mmdetection.\n\n**Summary**\nI used the pseudo labels of Dataset3 to train the Cascade Mask RCNN + Convnext v2 large.\n\n**Model**\n- Detector : Cascade Mask RCNN \n- Backbone : Convnext v2 Large\n- Loss : FocalLoss\n\n**Augmentation**\n- Albumentations\n - Distortion worked very well for me.\n```python\n dict(\n type='OneOf',\n transforms=[\n dict(type='OpticalDistortion', p=0.3),\n dict(type='GridDistortion', p=0.3),\n dict(type='ElasticTransform', p=0.1),\n ], p=0.5),\n```\n\n- mmdet augmentation\n - AutoAugment, MixUp, Mosaic, RandomErasing\n\n**Training process**\n\nFirst, weighted segment fusion was performed on the output values from the 4 models made of 4 types of data sets to create pseudo labels for Dataset3.\n\nI did WSF by referring to the guide below -> https://www.kaggle.com/code/mistag/sartorius-tta-with-weighted-segments-fusion\n\n\n\nsecond. The model was pre-trained using the pseudo label of Dataset 3 and the human label of Dataset 2.\n\nFinally, the model was fine-tuned using only Dataset1 data and submitted.\n\n", "2368137": "Hi, what is the concept of pseudo label here?\n\nThanks in advance.", "2368145": "Hi What is the meaning of the concept here?\n\nI thought that there would be differences for each WSI, so I created 4 folds for each WSI. The predicted values of the four models thus created were WSFed and used as pseudo labels. It is.", "2368149": "Congrats on solo gold, but I want to correct you that the mmdetection is made by OpenMMLab which is created by SenseTime and The Chinese University of Hong Kong😌", "2368151": "Congrats for the solo gold medal!\nI have 2 questions:\n1. What is your configs in the final fine tuning step ? Do you still keep the same config from the previous step or use something else ?\n2. In the final training pre-train model and fine tuning step, what set do you use for valid ?", "2368153": "Sorry My mistake, Thanks for the advice.", "2368155": "1. Yes, In the pre-training step, I select the best epoch model pth and resume fine-tuning in the same configs\n2. First, find the epoch with the best valid loss using 20% of Dataset1 as validatoion. After that, fine-tuning was performed on all Dataset1 data under the same conditions.", "2368177": "1. By \"best epoch model pth\" u mean best model of training pre-train model (dataset 2 of wsi 1,2 and dataset 3) right ?\n2. Do you freeze any layer in the fine tuning step ?", "2368183": "1. yes, \"best epoch model pth\" with Train (dataset 2 of wsi 1,2 and dataset 3) valid (only dataset 1)\n2. no i didn't freeze layer\n\nSorry for not being able to explain clearly.", "2368189": "Thank you so much for your response!\nThis is very similar with my first approach for this competition. But i don't use this because i think this is data leakage. What do you think about this ?", "2368220": "Yes, I thought it was difficult to trust the validation score because Dataset1 was used together when pseudo labeling was performed.\n\nTherefore, this idea was not applied right away, but was applied after finding the best hyperparameters and epochs through CV until just before the end of the competition.", "2368250": "Congrats on the medal @devchopin !\n\nEven I am a university student (just completed my first year). I have not taken any formal courses yet but I try to read up on articles and am currently reading hands on ML with sci-kit learn and TensorFlow. How exactly do you go about preparing for competitions and how much time do you spend?", "2368272": "Since I am also a beginner, I am cautious about giving advice, but in my case, participating in various competitions was helpful. I also read a lot of META research papers.", "2368398": "Thank you", "2368764": "Congrads on your solo gold! I think this pseudo labeling method is insightful. I want to know a little detail about it. When you fine tuning the pretrained model, what validation data you use for it?(you say \"under the same conditions\", you mean you use 20% dataset1 to validate for fine tuning stage too?)", "2369631": "The pseudo labels were blood vessel, glomerulus and unsure?.\n\nThanks in advance.", "2369673": "Only blood vessel, Thank you", "2369676": "Hello, thank you for your kind words.\n\nThere are a total of 4 steps.\n\n1. Create a pre-trained model using Dataset2 + Dataset3 data. (24 epoch was the best result for me here.)\n\n2. Train Dataset1 data from 24 epoch(from pre-train model). Here, it was verified using train (80%) and validation (20%). (This step is to find out which epoch is best.)\n\n3. I found that training up to 28 epochs in step 2 is the best mAP60.\n\n4. Finally, submit the model trained on 28 epochs of 100% of Dataset1 data.\n\nIf there is something I don't understand, I will answer again. thank you", "2369760": "I understand, thank you", "2369765": "Congratulations on your solo gold! \n\nI have a question for you. I assume that the model used for pseudo-labeling contains a fair amount of low-score false positives. After applying the WSF, did you pseudo-label all the predicted masks directly, or did you apply any post-processing to the masks before pseudo-labeling them? (e.g., excluding small instances, filtering by probability values)", "2369769": "there is two step\n\n1. After WSF, only masks with a confidence score of 0.5 or higher were used.\n2. And I filtered the mask of area smaller than 100.", "2369805": "Thank you for your response!\n I understand your explanation." }, "source": "meta" }