{
  "id": 428295,
  "title": "7th Place Solution",
  "url": "/competitions/hubmap-hacking-the-human-vasculature/writeups/yu4u-7th-place-solution",
  "author_name": "",
  "post_date": "2023-08-01T01:16:50.247Z",
  "votes": 63,
  "comment_count": 22,
  "views": 0,
  "content": "<p>Thanks to HuBMAP for hosting the exciting competition, and congrats to all prize and medal winners!</p>\n<ul>\n<li>Summary<ul>\n<li>Mask R-CNN model trained with dataset1, 2, 3 (pseudo labels)</li></ul></li>\n<li>Training pipeline<ul>\n<li>Train models with dataset 1 (5 folds)</li>\n<li>Create pseudo labels for dataset 2, 3 using the above models (for each fold)</li>\n<li>Train models with dataset 1, 2, 3 (5 folds)</li>\n<li>For dataset 2, both original (dilated) annotations and pseudo labels were used</li></ul></li>\n<li>Model<ul>\n<li>Mask R-CNN (Swin Transformer backbone, HTC RoI head)</li></ul></li>\n<li>Augmentation<ul>\n<li>Random resize (768-1536), flip, Rot90, RandomBrightnessContrast, HueSaturationValue</li></ul></li>\n<li>TTA<ul>\n<li>Resize (1024, 1536), hvflip</li></ul></li>\n<li>Ensemble<ul>\n<li>Ensemble on both region proposal and RoI head</li>\n<li>See \"ensemble detection model\" part of this solution</li>\n<li><a href=\"https://www.kaggle.com/competitions/sartorius-cell-instance-segmentation/discussion/298146\" target=\"_blank\">https://www.kaggle.com/competitions/sartorius-cell-instance-segmentation/discussion/298146</a></li></ul></li>\n<li>Post-processing<ul>\n<li>Dilation</li>\n<li>Remove small masks</li>\n<li>Remove masks that contain glomerulus regions</li></ul></li>\n<li>Does not work for me<ul>\n<li>Train with test images' pseudo labels (train in submission)</li>\n<li>External dataset <a href=\"https://data.mendeley.com/datasets/m2t49zf6xr/1\" target=\"_blank\">https://data.mendeley.com/datasets/m2t49zf6xr/1</a></li>\n<li>YOLOv8</li>\n<li>Puzzle in submission</li>\n<li><a href=\"https://www.kaggle.com/competitions/hubmap-hacking-the-human-vasculature/discussion/417314\" target=\"_blank\">https://www.kaggle.com/competitions/hubmap-hacking-the-human-vasculature/discussion/417314</a></li></ul></li>\n<li>Dilate or not dilate<ul>\n<li>I have found experimentally that when using only dataset1 for training, the score is higher without dilation than with dilation. Therefore, I suspected that the success of dilation was brought from noisy dataset2 and was an overfitting method to LB. Thus I have tried to minimize the difference in score with and without dilation by using pseudo labels and dilated annotation masks for dataset2.</li>\n<li>In the first submission, the dilation score was 0.1 better than without dilation, but in the final submission, the difference was reduced to 0.02. However, the submission with dilation was still better for both public and private LBs.</li></ul></li>\n</ul>",
  "messages": [
    {
      "id": "2367952",
      "postDate": "08/01/2023 00:25:40",
      "content": "<p>Thanks to HuBMAP for hosting the exciting competition, and congrats to all prize and medal winners!</p>\n<ul>\n<li>Summary<ul>\n<li>Mask R-CNN model trained with dataset1, 2, 3 (pseudo labels)</li></ul></li>\n<li>Training pipeline<ul>\n<li>Train models with dataset 1 (5 folds)</li>\n<li>Create pseudo labels for dataset 2, 3 using the above models (for each fold)</li>\n<li>Train models with dataset 1, 2, 3 (5 folds)</li>\n<li>For dataset 2, both original (dilated) annotations and pseudo labels were used</li></ul></li>\n<li>Model<ul>\n<li>Mask R-CNN (Swin Transformer backbone, HTC RoI head)</li></ul></li>\n<li>Augmentation<ul>\n<li>Random resize (768-1536), flip, Rot90, RandomBrightnessContrast, HueSaturationValue</li></ul></li>\n<li>TTA<ul>\n<li>Resize (1024, 1536), hvflip</li></ul></li>\n<li>Ensemble<ul>\n<li>Ensemble on both region proposal and RoI head</li>\n<li>See \"ensemble detection model\" part of this solution</li>\n<li><a href=\"https://www.kaggle.com/competitions/sartorius-cell-instance-segmentation/discussion/298146\" target=\"_blank\">https://www.kaggle.com/competitions/sartorius-cell-instance-segmentation/discussion/298146</a></li></ul></li>\n<li>Post-processing<ul>\n<li>Dilation</li>\n<li>Remove small masks</li>\n<li>Remove masks that contain glomerulus regions</li></ul></li>\n<li>Does not work for me<ul>\n<li>Train with test images' pseudo labels (train in submission)</li>\n<li>External dataset <a href=\"https://data.mendeley.com/datasets/m2t49zf6xr/1\" target=\"_blank\">https://data.mendeley.com/datasets/m2t49zf6xr/1</a></li>\n<li>YOLOv8</li>\n<li>Puzzle in submission</li>\n<li><a href=\"https://www.kaggle.com/competitions/hubmap-hacking-the-human-vasculature/discussion/417314\" target=\"_blank\">https://www.kaggle.com/competitions/hubmap-hacking-the-human-vasculature/discussion/417314</a></li></ul></li>\n<li>Dilate or not dilate<ul>\n<li>I have found experimentally that when using only dataset1 for training, the score is higher without dilation than with dilation. Therefore, I suspected that the success of dilation was brought from noisy dataset2 and was an overfitting method to LB. Thus I have tried to minimize the difference in score with and without dilation by using pseudo labels and dilated annotation masks for dataset2.</li>\n<li>In the first submission, the dilation score was 0.1 better than without dilation, but in the final submission, the difference was reduced to 0.02. However, the submission with dilation was still better for both public and private LBs.</li></ul></li>\n</ul>",
      "rawMarkdown": "Thanks to HuBMAP for hosting the exciting competition, and congrats to all prize and medal winners!\n\n- Summary\n  - Mask R-CNN model trained with dataset1, 2, 3 (pseudo labels)\n- Training pipeline\n  - Train models with dataset 1 (5 folds)\n  - Create pseudo labels for dataset 2, 3 using the above models (for each fold)\n  - Train models with dataset 1, 2, 3 (5 folds)\n    - For dataset 2, both original (dilated) annotations and pseudo labels were used\n- Model\n  - Mask R-CNN (Swin Transformer backbone, HTC RoI head)\n- Augmentation\n  - Random resize (768-1536), flip, Rot90, RandomBrightnessContrast, HueSaturationValue\n- TTA\n  - Resize (1024, 1536), hvflip\n- Ensemble\n  - Ensemble on both region proposal and RoI head\n  - See \"ensemble detection model\" part of this solution\n    - https://www.kaggle.com/competitions/sartorius-cell-instance-segmentation/discussion/298146\n- Post-processing\n  - Dilation\n  - Remove small masks\n  - Remove masks that contain glomerulus regions\n- Does not work for me\n  - Train with test images' pseudo labels (train in submission)\n  - External dataset https://data.mendeley.com/datasets/m2t49zf6xr/1\n  - YOLOv8\n  - Puzzle in submission\n    - https://www.kaggle.com/competitions/hubmap-hacking-the-human-vasculature/discussion/417314\n- Dilate or not dilate\n  - I have found experimentally that when using only dataset1 for training, the score is higher without dilation than with dilation. Therefore, I suspected that the success of dilation was brought from noisy dataset2 and was an overfitting method to LB. Thus I have tried to minimize the difference in score with and without dilation by using pseudo labels and dilated annotation masks for dataset2.\n  - In the first submission, the dilation score was 0.1 better than without dilation, but in the final submission, the difference was reduced to 0.02. However, the submission with dilation was still better for both public and private LBs.",
      "votes": null
    },
    {
      "id": "2367999",
      "postDate": "08/01/2023 01:09:21",
      "content": "<p>Congratulations to keep your gold place.</p>\n<p>I have some questions about your solution.<br>\nI'm glad to recieve your answer, thank you.</p>\n<ul>\n<li>Which version of mmdet did you use, 2x or 3x ?</li>\n<li>What's kfold did you choice when you train with 5fold, random kfold, group kfold, stratified kfold, …</li>\n<li>Have you tried other models such as mask2former ?</li>\n</ul>",
      "rawMarkdown": "Congratulations to keep your gold place.\n\nI have some questions about your solution.\nI'm glad to recieve your answer, thank you.\n\n- Which version of mmdet did you use, 2x or 3x ?\n- What's kfold did you choice when you train with 5fold, random kfold, group kfold, stratified kfold, ...\n- Have you tried other models such as mask2former ?",
      "votes": null
    },
    {
      "id": "2368006",
      "postDate": "08/01/2023 01:20:32",
      "content": "<p>Thx!</p>\n<ul>\n<li>2.x. I have not yet migrated configs from 2.x to 3.x.</li>\n<li>random kfold</li>\n<li>I tried mask2former, but failed to make them learn well.</li>\n</ul>",
      "rawMarkdown": "Thx!\n\n- 2.x. I have not yet migrated configs from 2.x to 3.x.\n- random kfold\n- I tried mask2former, but failed to make them learn well.",
      "votes": null
    },
    {
      "id": "2368010",
      "postDate": "08/01/2023 01:27:25",
      "content": "<p>thank you  for quick answer ! <br>\nI got it.</p>",
      "rawMarkdown": "thank you  for quick answer ! \nI got it.",
      "votes": null
    },
    {
      "id": "2368096",
      "postDate": "08/01/2023 02:24:42",
      "content": "<p>Congratulations! I have questions about your submissions. <br>\n1) Does 0.562 private correspond to 0.575 public?<br>\n2) \"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?</p>",
      "rawMarkdown": "Congratulations! I have questions about your submissions. \n1) Does 0.562 private correspond to 0.575 public?\n2) \"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?",
      "votes": null
    },
    {
      "id": "2368102",
      "postDate": "08/01/2023 02:32:00",
      "content": "<p>Thank you for sharing your solution. I have a question about training with pseudo labels. </p>\n<blockquote>\n  <ul>\n  <li>Train models with dataset 1 (5 folds)</li>\n  <li>Create pseudo labels for dataset 2, 3 using the above models (for each fold)</li>\n  <li>Train models with dataset 1, 2, 3 (5 folds)</li>\n  </ul>\n</blockquote>\n<p>Q1 :What is the CV of the model trained on dataset 1?</p>\n<p>Q2 :Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.</p>\n<p>Q3 : Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.</p>\n<p>Congratulations on your gold medal!</p>",
      "rawMarkdown": "Thank you for sharing your solution. I have a question about training with pseudo labels. \n> - Train models with dataset 1 (5 folds)\n> - Create pseudo labels for dataset 2, 3 using the above models (for each fold)\n> - Train models with dataset 1, 2, 3 (5 folds)\n\nQ1 :What is the CV of the model trained on dataset 1?\n\nQ2 :Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.\n\nQ3 : Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.\n\nCongratulations on your gold medal!",
      "votes": null
    },
    {
      "id": "2368129",
      "postDate": "08/01/2023 02:54:06",
      "content": "<p>I'm sorry. I forgot to ask you another question. </p>\n<p>Q4 : Did you use all of the instances output from each model to create pseudo label? </p>\n<p>As far as I have experimented, many instances from mmdet have overlapping masks, whereas training data has basically no overlapping masks. </p>\n<p>I think possible post processing is like using only those above a certain level of confidence, removing duplicate boxes using methods such as nms, etc. Did you perform such post-processing in this solution?</p>",
      "rawMarkdown": "I'm sorry. I forgot to ask you another question. \n\nQ4 : Did you use all of the instances output from each model to create pseudo label? \n\nAs far as I have experimented, many instances from mmdet have overlapping masks, whereas training data has basically no overlapping masks. \n\nI think possible post processing is like using only those above a certain level of confidence, removing duplicate boxes using methods such as nms, etc. Did you perform such post-processing in this solution?",
      "votes": null
    },
    {
      "id": "2368140",
      "postDate": "08/01/2023 03:03:47",
      "content": "<p>Congrats on solo gold！May I ask a little bit more about the pseudo labeling procedure?</p>\n<ol>\n<li>How many rounds pseudo labeling have you done?</li>\n<li>Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?</li>\n<li>After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?</li>\n<li>How you mixed the labels for dataset 2?<br>\nMany thanks!</li>\n</ol>",
      "rawMarkdown": "Congrats on solo gold！May I ask a little bit more about the pseudo labeling procedure?\n1. How many rounds pseudo labeling have you done?\n2. Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?\n3. After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?\n4. How you mixed the labels for dataset 2?\nMany thanks!",
      "votes": null
    },
    {
      "id": "2368566",
      "postDate": "08/01/2023 08:36:17",
      "content": "<p>Thx!</p>\n<ol>\n<li>How many rounds pseudo labeling have you done?<ul>\n<li>Only once. Repeating the procedure does not work for me.</li></ul></li>\n<li>Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?<ul>\n<li>I used detection results with a score of 0.7 or higher as pseudo labels.</li></ul></li>\n<li>After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?<ul>\n<li>Pseudo labels were created for each pretrained model individually, and used for each training of corresponding fold to avoid leakage.</li></ul></li>\n<li>How you mixed the labels for dataset 2?<ul>\n<li>Simply add pseudo labels to ground truth annotations. Thus there would be many similar masks but I did not think this brings a negative impact.</li></ul></li>\n</ol>",
      "rawMarkdown": "Thx!\n\n1. How many rounds pseudo labeling have you done?\n  - Only once. Repeating the procedure does not work for me.\n2. Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?\n  - I used detection results with a score of 0.7 or higher as pseudo labels.\n3. After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?\n  - Pseudo labels were created for each pretrained model individually, and used for each training of corresponding fold to avoid leakage.\n4. How you mixed the labels for dataset 2?\n  - Simply add pseudo labels to ground truth annotations. Thus there would be many similar masks but I did not think this brings a negative impact.",
      "votes": null
    },
    {
      "id": "2368594",
      "postDate": "08/01/2023 08:53:04",
      "content": "<p>Thx!</p>\n<ol>\n<li>What is the CV of the model trained on dataset 1?<ul>\n<li>Average of best val scores (segm_mAP_60) is 0.7004. This score is with resize TTA (1024, 1536) and without hvflip TTA.</li></ul></li>\n<li>Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.<ul>\n<li>I thought that the poor quality (in terms of missing masks and inaccurate mask regions) of dataset 2 would result in poor quality pseudo labels.</li></ul></li>\n<li>Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.<ul>\n<li>I used dataset 1 with gt labels, dataset 2 with pseudo labels + gt labels, and dataset 3 with pseudo labels. In training, I oversampled dataset 1 and 2 for four times for balance.</li></ul></li>\n<li>Did you use all of the instances output from each model to create pseudo label?<ul>\n<li>Partially Yes. I used only detection results with higher scores but I did not care about overlapping. In model inference, NMS is already performed to some extent.</li></ul></li>\n</ol>",
      "rawMarkdown": "Thx!\n\n1. What is the CV of the model trained on dataset 1?\n  - Average of best val scores (segm_mAP_60) is 0.7004. This score is with resize TTA (1024, 1536) and without hvflip TTA.\n2. Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.\n  - I thought that the poor quality (in terms of missing masks and inaccurate mask regions) of dataset 2 would result in poor quality pseudo labels.\n3. Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.\n  - I used dataset 1 with gt labels, dataset 2 with pseudo labels + gt labels, and dataset 3 with pseudo labels. In training, I oversampled dataset 1 and 2 for four times for balance.\n4. Did you use all of the instances output from each model to create pseudo label?\n  - Partially Yes. I used only detection results with higher scores but I did not care about overlapping. In model inference, NMS is already performed to some extent.",
      "votes": null
    },
    {
      "id": "2368600",
      "postDate": "08/01/2023 09:00:34",
      "content": "<ol>\n<li>Does 0.562 private correspond to 0.575 public?<ul>\n<li>No. I selected a submissioin with public LB score of 0.574 considering local CV scores and my intuition.</li></ul></li>\n<li>\"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?<ul>\n<li>No. The selected two submissions are:</li></ul><ol>\n<li>with dilation: public 0.574, private 0.562</li>\n<li>without dilation: public 0.559, private 0.562</li></ol><ul>\n<li>I intended the difference in scores for public LB (because we could not see private scores during competition)</li></ul></li>\n</ol>",
      "rawMarkdown": "1. Does 0.562 private correspond to 0.575 public?\n  - No. I selected a submissioin with public LB score of 0.574 considering local CV scores and my intuition.\n2. \"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?\n  - No. The selected two submissions are:\n    1. with dilation: public 0.574, private 0.562\n    2. without dilation: public 0.559, private 0.562\n  - I intended the difference in scores for public LB (because we could not see private scores during competition)",
      "votes": null
    },
    {
      "id": "2368614",
      "postDate": "08/01/2023 09:10:05",
      "content": "<p>Appreciate for your reply! Please allow me ask more about your reply on question 3.<br>\nSo the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?</p>",
      "rawMarkdown": "Appreciate for your reply! Please allow me ask more about your reply on question 3.\nSo the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?",
      "votes": null
    },
    {
      "id": "2368877",
      "postDate": "08/01/2023 12:34:06",
      "content": "<blockquote>\n  <p>So the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?</p>\n</blockquote>\n<p>Exactly.</p>",
      "rawMarkdown": "> So the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?\n\nExactly.",
      "votes": null
    },
    {
      "id": "2368982",
      "postDate": "08/01/2023 13:35:52",
      "content": "<p>Congrats on solo gold! I have a small question. You mentioned \"Ensemble on both region proposal and RoI head\", I wonder if this method needs significant code modification of the source code of mmdetection?</p>",
      "rawMarkdown": "Congrats on solo gold! I have a small question. You mentioned \"Ensemble on both region proposal and RoI head\", I wonder if this method needs significant code modification of the source code of mmdetection?",
      "votes": null
    },
    {
      "id": "2369139",
      "postDate": "08/01/2023 14:47:06",
      "content": "<p>The implementation is somewhat complex but you can simply write ensemble detector class in notebook, and there is no need to modify mmdet itself.</p>\n<p>The implentation is something like this: <a href=\"https://github.com/amirassov/kaggle-imaterialist/blob/f1ae37100801203500d20119b9de7e19b0d89a1c/mmdetection/mmdet/models/detectors/ensemble_htc.py#L7\" target=\"_blank\">https://github.com/amirassov/kaggle-imaterialist/blob/f1ae37100801203500d20119b9de7e19b0d89a1c/mmdetection/mmdet/models/detectors/ensemble_htc.py#L7</a></p>",
      "rawMarkdown": "The implementation is somewhat complex but you can simply write ensemble detector class in notebook, and there is no need to modify mmdet itself.\n\nThe implentation is something like this: https://github.com/amirassov/kaggle-imaterialist/blob/f1ae37100801203500d20119b9de7e19b0d89a1c/mmdetection/mmdet/models/detectors/ensemble_htc.py#L7",
      "votes": null
    },
    {
      "id": "2369178",
      "postDate": "08/01/2023 15:10:15",
      "content": "<p>Thanks for the resource! It is complex and needs a very in-depth understanding of mmdetection. Good to know that.</p>",
      "rawMarkdown": "Thanks for the resource! It is complex and needs a very in-depth understanding of mmdetection. Good to know that.",
      "votes": null
    },
    {
      "id": "2369213",
      "postDate": "08/01/2023 15:47:13",
      "content": "<p>Congratulations! Thanks for giving the solution.</p>",
      "rawMarkdown": "Congratulations! Thanks for giving the solution.",
      "votes": null
    },
    {
      "id": "2369684",
      "postDate": "08/02/2023 00:26:45",
      "content": "<p>Hi, it seems like a great idea to use both pseudo labels and human labels in Dataset2. Congratulations.</p>\n<p>By the way,</p>\n<ol>\n<li><p>Is there a  reference or reason why only HTC Roi Head was used in Mask R-CNN?</p></li>\n<li><p>and how did you set the threshold for the small mask? (Area or Width * height?)</p></li>\n</ol>",
      "rawMarkdown": "Hi, it seems like a great idea to use both pseudo labels and human labels in Dataset2. Congratulations.\n\nBy the way,\n\n1. Is there a  reference or reason why only HTC Roi Head was used in Mask R-CNN?\n\n2. and how did you set the threshold for the small mask? (Area or Width * height?)",
      "votes": null
    },
    {
      "id": "2369705",
      "postDate": "08/02/2023 01:13:46",
      "content": "<ol>\n<li>Is there a reference or reason why only HTC Roi Head was used in Mask R-CNN?<ul>\n<li>Although not compared in this competition, I empirically believe that HTC is more accurate than Cascade Mask R-CNN.<br>\nI did not utilize multiple architectures for ensemble as I prefer a simple solution.</li></ul></li>\n<li>how did you set the threshold for the small mask? (Area or Width * height?)<ul>\n<li>Area (mask.sum() &lt; 100). From EDA, there were very few masks with area &lt; 100. I did not optimize this, but when I changed the threshold to 1 in late submission, both private/public LBs were decreased. </li></ul></li>\n</ol>",
      "rawMarkdown": "1. Is there a reference or reason why only HTC Roi Head was used in Mask R-CNN?\n  - Although not compared in this competition, I empirically believe that HTC is more accurate than Cascade Mask R-CNN.\nI did not utilize multiple architectures for ensemble as I prefer a simple solution.\n2. how did you set the threshold for the small mask? (Area or Width * height?)\n  - Area (mask.sum() < 100). From EDA, there were very few masks with area < 100. I did not optimize this, but when I changed the threshold to 1 in late submission, both private/public LBs were decreased.",
      "votes": null
    },
    {
      "id": "2369706",
      "postDate": "08/02/2023 01:17:08",
      "content": "<p>Thanks a lot!</p>",
      "rawMarkdown": "Thanks a lot!",
      "votes": null
    },
    {
      "id": "2369813",
      "postDate": "08/02/2023 03:33:24",
      "content": "<p>Congratulations on solo gold. May I ask you more about your validation strategy?<br>\nI thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold?<br>\nWhich score did you trust?, Your CV, LB or something else?</p>",
      "rawMarkdown": "Congratulations on solo gold. May I ask you more about your validation strategy?\nI thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold?\nWhich score did you trust?, Your CV, LB or something else?",
      "votes": null
    },
    {
      "id": "2369844",
      "postDate": "08/02/2023 04:00:00",
      "content": "<p>Good question! That's exactly the question I'd like to ask the other participants, especially to <a href=\"https://www.kaggle.com/tascj0\" target=\"_blank\">@tascj0</a> !</p>\n<blockquote>\n  <p>I thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold? Which score did you trust?, Your CV, LB or something else?</p>\n</blockquote>\n<ul>\n<li>For me, I splitted folds according to WSI first. However, training was not stable and models were quickly overfitted. Instead, I evaluated my models trained with only dataset 1 (= without WSI 3, 4) on public LB. This evaluation will allow me to see if my model is generalized to a WSI I have not seen before.</li>\n<li>I got a public LB score of 0.549 using models <strong>without dilation and dataset 2</strong>. This gave me the confidence to not shake down.</li>\n</ul>",
      "rawMarkdown": "Good question! That's exactly the question I'd like to ask the other participants, especially to @tascj0 !\n\n> I thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold? Which score did you trust?, Your CV, LB or something else?\n\n  - For me, I splitted folds according to WSI first. However, training was not stable and models were quickly overfitted. Instead, I evaluated my models trained with only dataset 1 (= without WSI 3, 4) on public LB. This evaluation will allow me to see if my model is generalized to a WSI I have not seen before.\n  - I got a public LB score of 0.549 using models **without dilation and dataset 2**. This gave me the confidence to not shake down.",
      "votes": null
    },
    {
      "id": "2370147",
      "postDate": "08/02/2023 08:20:05",
      "content": "<p>Thank you for sharing the process of experiments. I didn't realize that I could validate the models using public LB.</p>",
      "rawMarkdown": "Thank you for sharing the process of experiments. I didn't realize that I could validate the models using public LB.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2367999,
      "author_name": "kaerunantoka",
      "author_url": "",
      "post_date": "08/01/2023 01:09:21",
      "content": "<p>Congratulations to keep your gold place.</p>\n<p>I have some questions about your solution.<br>\nI'm glad to recieve your answer, thank you.</p>\n<ul>\n<li>Which version of mmdet did you use, 2x or 3x ?</li>\n<li>What's kfold did you choice when you train with 5fold, random kfold, group kfold, stratified kfold, …</li>\n<li>Have you tried other models such as mask2former ?</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 2368006,
          "author_name": "ren4yu",
          "author_url": "",
          "post_date": "08/01/2023 01:20:32",
          "content": "<p>Thx!</p>\n<ul>\n<li>2.x. I have not yet migrated configs from 2.x to 3.x.</li>\n<li>random kfold</li>\n<li>I tried mask2former, but failed to make them learn well.</li>\n</ul>",
          "votes": null,
          "replies": [
            {
              "id": 2368010,
              "author_name": "kaerunantoka",
              "author_url": "",
              "post_date": "08/01/2023 01:27:25",
              "content": "<p>thank you  for quick answer ! <br>\nI got it.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2368096,
      "author_name": "huyduong7101",
      "author_url": "",
      "post_date": "08/01/2023 02:24:42",
      "content": "<p>Congratulations! I have questions about your submissions. <br>\n1) Does 0.562 private correspond to 0.575 public?<br>\n2) \"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2368600,
          "author_name": "ren4yu",
          "author_url": "",
          "post_date": "08/01/2023 09:00:34",
          "content": "<ol>\n<li>Does 0.562 private correspond to 0.575 public?<ul>\n<li>No. I selected a submissioin with public LB score of 0.574 considering local CV scores and my intuition.</li></ul></li>\n<li>\"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?<ul>\n<li>No. The selected two submissions are:</li></ul><ol>\n<li>with dilation: public 0.574, private 0.562</li>\n<li>without dilation: public 0.559, private 0.562</li></ol><ul>\n<li>I intended the difference in scores for public LB (because we could not see private scores during competition)</li></ul></li>\n</ol>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2368102,
      "author_name": "qhapaq49",
      "author_url": "",
      "post_date": "08/01/2023 02:32:00",
      "content": "<p>Thank you for sharing your solution. I have a question about training with pseudo labels. </p>\n<blockquote>\n  <ul>\n  <li>Train models with dataset 1 (5 folds)</li>\n  <li>Create pseudo labels for dataset 2, 3 using the above models (for each fold)</li>\n  <li>Train models with dataset 1, 2, 3 (5 folds)</li>\n  </ul>\n</blockquote>\n<p>Q1 :What is the CV of the model trained on dataset 1?</p>\n<p>Q2 :Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.</p>\n<p>Q3 : Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.</p>\n<p>Congratulations on your gold medal!</p>",
      "votes": null,
      "replies": [
        {
          "id": 2368129,
          "author_name": "qhapaq49",
          "author_url": "",
          "post_date": "08/01/2023 02:54:06",
          "content": "<p>I'm sorry. I forgot to ask you another question. </p>\n<p>Q4 : Did you use all of the instances output from each model to create pseudo label? </p>\n<p>As far as I have experimented, many instances from mmdet have overlapping masks, whereas training data has basically no overlapping masks. </p>\n<p>I think possible post processing is like using only those above a certain level of confidence, removing duplicate boxes using methods such as nms, etc. Did you perform such post-processing in this solution?</p>",
          "votes": null,
          "replies": [
            {
              "id": 2368594,
              "author_name": "ren4yu",
              "author_url": "",
              "post_date": "08/01/2023 08:53:04",
              "content": "<p>Thx!</p>\n<ol>\n<li>What is the CV of the model trained on dataset 1?<ul>\n<li>Average of best val scores (segm_mAP_60) is 0.7004. This score is with resize TTA (1024, 1536) and without hvflip TTA.</li></ul></li>\n<li>Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.<ul>\n<li>I thought that the poor quality (in terms of missing masks and inaccurate mask regions) of dataset 2 would result in poor quality pseudo labels.</li></ul></li>\n<li>Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.<ul>\n<li>I used dataset 1 with gt labels, dataset 2 with pseudo labels + gt labels, and dataset 3 with pseudo labels. In training, I oversampled dataset 1 and 2 for four times for balance.</li></ul></li>\n<li>Did you use all of the instances output from each model to create pseudo label?<ul>\n<li>Partially Yes. I used only detection results with higher scores but I did not care about overlapping. In model inference, NMS is already performed to some extent.</li></ul></li>\n</ol>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2368140,
      "author_name": "traptinblur",
      "author_url": "",
      "post_date": "08/01/2023 03:03:47",
      "content": "<p>Congrats on solo gold！May I ask a little bit more about the pseudo labeling procedure?</p>\n<ol>\n<li>How many rounds pseudo labeling have you done?</li>\n<li>Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?</li>\n<li>After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?</li>\n<li>How you mixed the labels for dataset 2?<br>\nMany thanks!</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 2368566,
          "author_name": "ren4yu",
          "author_url": "",
          "post_date": "08/01/2023 08:36:17",
          "content": "<p>Thx!</p>\n<ol>\n<li>How many rounds pseudo labeling have you done?<ul>\n<li>Only once. Repeating the procedure does not work for me.</li></ul></li>\n<li>Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?<ul>\n<li>I used detection results with a score of 0.7 or higher as pseudo labels.</li></ul></li>\n<li>After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?<ul>\n<li>Pseudo labels were created for each pretrained model individually, and used for each training of corresponding fold to avoid leakage.</li></ul></li>\n<li>How you mixed the labels for dataset 2?<ul>\n<li>Simply add pseudo labels to ground truth annotations. Thus there would be many similar masks but I did not think this brings a negative impact.</li></ul></li>\n</ol>",
          "votes": null,
          "replies": [
            {
              "id": 2368614,
              "author_name": "traptinblur",
              "author_url": "",
              "post_date": "08/01/2023 09:10:05",
              "content": "<p>Appreciate for your reply! Please allow me ask more about your reply on question 3.<br>\nSo the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2368877,
                  "author_name": "ren4yu",
                  "author_url": "",
                  "post_date": "08/01/2023 12:34:06",
                  "content": "<blockquote>\n  <p>So the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?</p>\n</blockquote>\n<p>Exactly.</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2368982,
      "author_name": "snaker",
      "author_url": "",
      "post_date": "08/01/2023 13:35:52",
      "content": "<p>Congrats on solo gold! I have a small question. You mentioned \"Ensemble on both region proposal and RoI head\", I wonder if this method needs significant code modification of the source code of mmdetection?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2369139,
          "author_name": "ren4yu",
          "author_url": "",
          "post_date": "08/01/2023 14:47:06",
          "content": "<p>The implementation is somewhat complex but you can simply write ensemble detector class in notebook, and there is no need to modify mmdet itself.</p>\n<p>The implentation is something like this: <a href=\"https://github.com/amirassov/kaggle-imaterialist/blob/f1ae37100801203500d20119b9de7e19b0d89a1c/mmdetection/mmdet/models/detectors/ensemble_htc.py#L7\" target=\"_blank\">https://github.com/amirassov/kaggle-imaterialist/blob/f1ae37100801203500d20119b9de7e19b0d89a1c/mmdetection/mmdet/models/detectors/ensemble_htc.py#L7</a></p>",
          "votes": null,
          "replies": [
            {
              "id": 2369178,
              "author_name": "snaker",
              "author_url": "",
              "post_date": "08/01/2023 15:10:15",
              "content": "<p>Thanks for the resource! It is complex and needs a very in-depth understanding of mmdetection. Good to know that.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2369213,
      "author_name": "osmaniii",
      "author_url": "",
      "post_date": "08/01/2023 15:47:13",
      "content": "<p>Congratulations! Thanks for giving the solution.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2369684,
      "author_name": "devchopin",
      "author_url": "",
      "post_date": "08/02/2023 00:26:45",
      "content": "<p>Hi, it seems like a great idea to use both pseudo labels and human labels in Dataset2. Congratulations.</p>\n<p>By the way,</p>\n<ol>\n<li><p>Is there a  reference or reason why only HTC Roi Head was used in Mask R-CNN?</p></li>\n<li><p>and how did you set the threshold for the small mask? (Area or Width * height?)</p></li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 2369705,
          "author_name": "ren4yu",
          "author_url": "",
          "post_date": "08/02/2023 01:13:46",
          "content": "<ol>\n<li>Is there a reference or reason why only HTC Roi Head was used in Mask R-CNN?<ul>\n<li>Although not compared in this competition, I empirically believe that HTC is more accurate than Cascade Mask R-CNN.<br>\nI did not utilize multiple architectures for ensemble as I prefer a simple solution.</li></ul></li>\n<li>how did you set the threshold for the small mask? (Area or Width * height?)<ul>\n<li>Area (mask.sum() &lt; 100). From EDA, there were very few masks with area &lt; 100. I did not optimize this, but when I changed the threshold to 1 in late submission, both private/public LBs were decreased. </li></ul></li>\n</ol>",
          "votes": null,
          "replies": [
            {
              "id": 2369706,
              "author_name": "devchopin",
              "author_url": "",
              "post_date": "08/02/2023 01:17:08",
              "content": "<p>Thanks a lot!</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2369813,
      "author_name": "tamotamo",
      "author_url": "",
      "post_date": "08/02/2023 03:33:24",
      "content": "<p>Congratulations on solo gold. May I ask you more about your validation strategy?<br>\nI thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold?<br>\nWhich score did you trust?, Your CV, LB or something else?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2369844,
          "author_name": "ren4yu",
          "author_url": "",
          "post_date": "08/02/2023 04:00:00",
          "content": "<p>Good question! That's exactly the question I'd like to ask the other participants, especially to <a href=\"https://www.kaggle.com/tascj0\" target=\"_blank\">@tascj0</a> !</p>\n<blockquote>\n  <p>I thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold? Which score did you trust?, Your CV, LB or something else?</p>\n</blockquote>\n<ul>\n<li>For me, I splitted folds according to WSI first. However, training was not stable and models were quickly overfitted. Instead, I evaluated my models trained with only dataset 1 (= without WSI 3, 4) on public LB. This evaluation will allow me to see if my model is generalized to a WSI I have not seen before.</li>\n<li>I got a public LB score of 0.549 using models <strong>without dilation and dataset 2</strong>. This gave me the confidence to not shake down.</li>\n</ul>",
          "votes": null,
          "replies": [
            {
              "id": 2370147,
              "author_name": "tamotamo",
              "author_url": "",
              "post_date": "08/02/2023 08:20:05",
              "content": "<p>Thank you for sharing the process of experiments. I didn't realize that I could validate the models using public LB.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2367952": "Thanks to HuBMAP for hosting the exciting competition, and congrats to all prize and medal winners!\n\n- Summary\n  - Mask R-CNN model trained with dataset1, 2, 3 (pseudo labels)\n- Training pipeline\n  - Train models with dataset 1 (5 folds)\n  - Create pseudo labels for dataset 2, 3 using the above models (for each fold)\n  - Train models with dataset 1, 2, 3 (5 folds)\n    - For dataset 2, both original (dilated) annotations and pseudo labels were used\n- Model\n  - Mask R-CNN (Swin Transformer backbone, HTC RoI head)\n- Augmentation\n  - Random resize (768-1536), flip, Rot90, RandomBrightnessContrast, HueSaturationValue\n- TTA\n  - Resize (1024, 1536), hvflip\n- Ensemble\n  - Ensemble on both region proposal and RoI head\n  - See \"ensemble detection model\" part of this solution\n    - https://www.kaggle.com/competitions/sartorius-cell-instance-segmentation/discussion/298146\n- Post-processing\n  - Dilation\n  - Remove small masks\n  - Remove masks that contain glomerulus regions\n- Does not work for me\n  - Train with test images' pseudo labels (train in submission)\n  - External dataset https://data.mendeley.com/datasets/m2t49zf6xr/1\n  - YOLOv8\n  - Puzzle in submission\n    - https://www.kaggle.com/competitions/hubmap-hacking-the-human-vasculature/discussion/417314\n- Dilate or not dilate\n  - I have found experimentally that when using only dataset1 for training, the score is higher without dilation than with dilation. Therefore, I suspected that the success of dilation was brought from noisy dataset2 and was an overfitting method to LB. Thus I have tried to minimize the difference in score with and without dilation by using pseudo labels and dilated annotation masks for dataset2.\n  - In the first submission, the dilation score was 0.1 better than without dilation, but in the final submission, the difference was reduced to 0.02. However, the submission with dilation was still better for both public and private LBs.",
    "2367999": "Congratulations to keep your gold place.\n\nI have some questions about your solution.\nI'm glad to recieve your answer, thank you.\n\n- Which version of mmdet did you use, 2x or 3x ?\n- What's kfold did you choice when you train with 5fold, random kfold, group kfold, stratified kfold, ...\n- Have you tried other models such as mask2former ?",
    "2368006": "Thx!\n\n- 2.x. I have not yet migrated configs from 2.x to 3.x.\n- random kfold\n- I tried mask2former, but failed to make them learn well.",
    "2368010": "thank you  for quick answer ! \nI got it.",
    "2368096": "Congratulations! I have questions about your submissions. \n1) Does 0.562 private correspond to 0.575 public?\n2) \"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?",
    "2368102": "Thank you for sharing your solution. I have a question about training with pseudo labels. \n> - Train models with dataset 1 (5 folds)\n> - Create pseudo labels for dataset 2, 3 using the above models (for each fold)\n> - Train models with dataset 1, 2, 3 (5 folds)\n\nQ1 :What is the CV of the model trained on dataset 1?\n\nQ2 :Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.\n\nQ3 : Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.\n\nCongratulations on your gold medal!",
    "2368129": "I'm sorry. I forgot to ask you another question. \n\nQ4 : Did you use all of the instances output from each model to create pseudo label? \n\nAs far as I have experimented, many instances from mmdet have overlapping masks, whereas training data has basically no overlapping masks. \n\nI think possible post processing is like using only those above a certain level of confidence, removing duplicate boxes using methods such as nms, etc. Did you perform such post-processing in this solution?",
    "2368140": "Congrats on solo gold！May I ask a little bit more about the pseudo labeling procedure?\n1. How many rounds pseudo labeling have you done?\n2. Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?\n3. After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?\n4. How you mixed the labels for dataset 2?\nMany thanks!",
    "2368566": "Thx!\n\n1. How many rounds pseudo labeling have you done?\n  - Only once. Repeating the procedure does not work for me.\n2. Have you used all instances in your pseudo labels? If not, how you filtered your pseudo labels?\n  - I used detection results with a score of 0.7 or higher as pseudo labels.\n3. After pseudo labeling, did the 5 folds split only on dataset 3 or the hole mixed dataset?\n  - Pseudo labels were created for each pretrained model individually, and used for each training of corresponding fold to avoid leakage.\n4. How you mixed the labels for dataset 2?\n  - Simply add pseudo labels to ground truth annotations. Thus there would be many similar masks but I did not think this brings a negative impact.",
    "2368594": "Thx!\n\n1. What is the CV of the model trained on dataset 1?\n  - Average of best val scores (segm_mAP_60) is 0.7004. This score is with resize TTA (1024, 1536) and without hvflip TTA.\n2. Why did you train the model only datasets 1 to create pseudo labels for dataset 3? The number of dataset 1 is very small, so I think adding dataset 2 will improve the accuracy of the pseudo-labels.\n  - I thought that the poor quality (in terms of missing masks and inaccurate mask regions) of dataset 2 would result in poor quality pseudo labels.\n3. Did you use all of dataset 2, dataset 2 with pseudo-labels, and dataset 3 with pseudo-labels? The number of data is about 400 (dataset 1), 1200 (dataset 2), and 5500 (dataset 3). If all data are used, the ratio of training data would be 320 : 1200+1200 : 5500.\n  - I used dataset 1 with gt labels, dataset 2 with pseudo labels + gt labels, and dataset 3 with pseudo labels. In training, I oversampled dataset 1 and 2 for four times for balance.\n4. Did you use all of the instances output from each model to create pseudo label?\n  - Partially Yes. I used only detection results with higher scores but I did not care about overlapping. In model inference, NMS is already performed to some extent.",
    "2368600": "1. Does 0.562 private correspond to 0.575 public?\n  - No. I selected a submissioin with public LB score of 0.574 considering local CV scores and my intuition.\n2. \"the difference was reduced to 0.02\" means 0.560 and 0.562 private for non-dilatation and dilation, doesn't it?\n  - No. The selected two submissions are:\n    1. with dilation: public 0.574, private 0.562\n    2. without dilation: public 0.559, private 0.562\n  - I intended the difference in scores for public LB (because we could not see private scores during competition)",
    "2368614": "Appreciate for your reply! Please allow me ask more about your reply on question 3.\nSo the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?",
    "2368877": "> So the folds are unchanged once you splitting the dataset 1, and use a single fold-model to create fold-specific pseudo labels?\n\nExactly.",
    "2368982": "Congrats on solo gold! I have a small question. You mentioned \"Ensemble on both region proposal and RoI head\", I wonder if this method needs significant code modification of the source code of mmdetection?",
    "2369139": "The implementation is somewhat complex but you can simply write ensemble detector class in notebook, and there is no need to modify mmdet itself.\n\nThe implentation is something like this: https://github.com/amirassov/kaggle-imaterialist/blob/f1ae37100801203500d20119b9de7e19b0d89a1c/mmdetection/mmdet/models/detectors/ensemble_htc.py#L7",
    "2369178": "Thanks for the resource! It is complex and needs a very in-depth understanding of mmdetection. Good to know that.",
    "2369213": "Congratulations! Thanks for giving the solution.",
    "2369684": "Hi, it seems like a great idea to use both pseudo labels and human labels in Dataset2. Congratulations.\n\nBy the way,\n\n1. Is there a  reference or reason why only HTC Roi Head was used in Mask R-CNN?\n\n2. and how did you set the threshold for the small mask? (Area or Width * height?)",
    "2369705": "1. Is there a reference or reason why only HTC Roi Head was used in Mask R-CNN?\n  - Although not compared in this competition, I empirically believe that HTC is more accurate than Cascade Mask R-CNN.\nI did not utilize multiple architectures for ensemble as I prefer a simple solution.\n2. how did you set the threshold for the small mask? (Area or Width * height?)\n  - Area (mask.sum() < 100). From EDA, there were very few masks with area < 100. I did not optimize this, but when I changed the threshold to 1 in late submission, both private/public LBs were decreased.",
    "2369706": "Thanks a lot!",
    "2369813": "Congratulations on solo gold. May I ask you more about your validation strategy?\nI thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold?\nWhich score did you trust?, Your CV, LB or something else?",
    "2369844": "Good question! That's exactly the question I'd like to ask the other participants, especially to @tascj0 !\n\n> I thought it would be better to divide the folds by WSI in this competition, why did you use random K-fold? Which score did you trust?, Your CV, LB or something else?\n\n  - For me, I splitted folds according to WSI first. However, training was not stable and models were quickly overfitted. Instead, I evaluated my models trained with only dataset 1 (= without WSI 3, 4) on public LB. This evaluation will allow me to see if my model is generalized to a WSI I have not seen before.\n  - I got a public LB score of 0.549 using models **without dilation and dataset 2**. This gave me the confidence to not shake down.",
    "2370147": "Thank you for sharing the process of experiments. I didn't realize that I could validate the models using public LB."
  },
  "source": "meta"
}