{
  "id": 428372,
  "title": "30th place solution",
  "url": "/competitions/hubmap-hacking-the-human-vasculature/discussion/428372",
  "author_name": "Gunes Evitan",
  "post_date": "2023-08-01T07:09:54.461000",
  "votes": 14,
  "comment_count": 3,
  "views": 0,
  "content": "<h2>Overview</h2>\n<p><a href=\"https://www.kaggle.com/ayberkmir\" target=\"_blank\">@ayberkmir</a> and I joined the competition very late. I wasted my first week because I didn't read the metric properly and submitting not accordingly. We didn't have time to try so many diverse things so it will be quite simple.</p>\n<h2>Validation</h2>\n<p>Ayberk was using stratified single train/test split (stratified on blood vessel count) and I was using 4 fold leave-one-WSI-out cross-validation. Surprisingly both of them worked on private leaderboard.</p>\n<h2>Models</h2>\n<p>I mainly worked on Mask R-CNN and Cascade Mask R-CNN with ResNeXt101 backbone and Ayberk worked on YOLOv7 and v8 models.</p>\n<h2>Training</h2>\n<p>I dropped duplicate annotations by simply suppressing overlapping boxes with 0.99 IoU when I was creating datasets. I also included glomeruli annotations and used unsure annotations as blood vessels. I thought if I treat unsure as blood vessels, training would be more stable. Ayberk didn't use glomeruli or unsure annotations and YOLO was already handling duplicate annotations.</p>\n<p>Since the WSIs were kinda similar, I decided not to use stain augmentations. I used multi scale training and scales are arranged between 1.5 and 2x raw scale i.e.</p>\n<pre><code>train_scales = [\n    (, ), (, ), (, ),\n    (, ), (, ), (, ),\n    (, ), (, ), (, )\n]\n</code></pre>\n<p>Other training augmentatios were random horizontal, vertical and diagonal flip, random brightness, contrast, hue and saturation.</p>\n<p>Ayberk was using fixed size of 1024x1024 because we thought built-in augmentations in YOLO are already strong enough.</p>\n<p>We used almost default values for optimizer and schedulers. I only added another learning rate multiplier step.</p>\n<h2>Post-processing</h2>\n<p>Both of us thought post-processing was the most important in this competition because of the weird metric. We dedicated most of our time on this part rather than modelling.</p>\n<p>First of all, we used 3x TTA (horizontal, vertical and diagonal flip) when we were predicting with our models. TTA predictions were merged using weighted boxes fusion. Masks that belong to fused boxes are averaged. Unfortunately, I couldn't find a way to retrieve soft mask predictions in mmdetection and I had to average binary masks. We didn't use multi scale TTA. YOLO model were taking 1024x1024 and Mask R-CNN models were taking 1280x1280 inputs.</p>\n<p>Since I had 4 folds, I also had to merge their predictions and I used weighted boxes fusion with same configurations. We also tried to merge YOLO and Mask R-CNN predictions at this stage. It was working as good as other predictions on public leaderboard but always failed on private leaderboard for some reason.</p>\n<p>We also tried the same submissions with and without dilation and we found that TTA was reducing the effect of dilation A LOT on public leaderboard. It was a really good sign because TTA is safe and dilation isn't.</p>\n<p>We had different submissions with combinations of models, folds, TTA, dilation, score multiplication, WBF tuning and etc.</p>\n<p>When we sort our submissions by public score, we get submission with dilation or ensemble.<br>\n<img src=\"https://i.ibb.co/8PpVPxX/Screenshot-from-2023-08-01-09-38-13.png\" alt=\"public\"></p>\n<p>When we sort our submission by private score, we get submissions with TTA, without dilation and single models but I also have one submission with dilation that scores 0.50x on private which is weird.<br>\n<img src=\"https://i.ibb.co/Ss8WSFy/Screenshot-from-2023-08-01-09-41-36.png\" alt=\"private\"></p>\n<p>Our best submission was 14th place on private leaderboard but at the end Ayberk was able select a good enough submission that made us finish 30th.</p>",
  "messages": [
    {
      "id": 2368446,
      "postDate": "2023-08-01T07:09:54.463Z",
      "content": "<h2>Overview</h2>\n<p><a href=\"https://www.kaggle.com/ayberkmir\" target=\"_blank\">@ayberkmir</a> and I joined the competition very late. I wasted my first week because I didn't read the metric properly and submitting not accordingly. We didn't have time to try so many diverse things so it will be quite simple.</p>\n<h2>Validation</h2>\n<p>Ayberk was using stratified single train/test split (stratified on blood vessel count) and I was using 4 fold leave-one-WSI-out cross-validation. Surprisingly both of them worked on private leaderboard.</p>\n<h2>Models</h2>\n<p>I mainly worked on Mask R-CNN and Cascade Mask R-CNN with ResNeXt101 backbone and Ayberk worked on YOLOv7 and v8 models.</p>\n<h2>Training</h2>\n<p>I dropped duplicate annotations by simply suppressing overlapping boxes with 0.99 IoU when I was creating datasets. I also included glomeruli annotations and used unsure annotations as blood vessels. I thought if I treat unsure as blood vessels, training would be more stable. Ayberk didn't use glomeruli or unsure annotations and YOLO was already handling duplicate annotations.</p>\n<p>Since the WSIs were kinda similar, I decided not to use stain augmentations. I used multi scale training and scales are arranged between 1.5 and 2x raw scale i.e.</p>\n<pre><code>train_scales = [\n    (, ), (, ), (, ),\n    (, ), (, ), (, ),\n    (, ), (, ), (, )\n]\n</code></pre>\n<p>Other training augmentatios were random horizontal, vertical and diagonal flip, random brightness, contrast, hue and saturation.</p>\n<p>Ayberk was using fixed size of 1024x1024 because we thought built-in augmentations in YOLO are already strong enough.</p>\n<p>We used almost default values for optimizer and schedulers. I only added another learning rate multiplier step.</p>\n<h2>Post-processing</h2>\n<p>Both of us thought post-processing was the most important in this competition because of the weird metric. We dedicated most of our time on this part rather than modelling.</p>\n<p>First of all, we used 3x TTA (horizontal, vertical and diagonal flip) when we were predicting with our models. TTA predictions were merged using weighted boxes fusion. Masks that belong to fused boxes are averaged. Unfortunately, I couldn't find a way to retrieve soft mask predictions in mmdetection and I had to average binary masks. We didn't use multi scale TTA. YOLO model were taking 1024x1024 and Mask R-CNN models were taking 1280x1280 inputs.</p>\n<p>Since I had 4 folds, I also had to merge their predictions and I used weighted boxes fusion with same configurations. We also tried to merge YOLO and Mask R-CNN predictions at this stage. It was working as good as other predictions on public leaderboard but always failed on private leaderboard for some reason.</p>\n<p>We also tried the same submissions with and without dilation and we found that TTA was reducing the effect of dilation A LOT on public leaderboard. It was a really good sign because TTA is safe and dilation isn't.</p>\n<p>We had different submissions with combinations of models, folds, TTA, dilation, score multiplication, WBF tuning and etc.</p>\n<p>When we sort our submissions by public score, we get submission with dilation or ensemble.<br>\n<img src=\"https://i.ibb.co/8PpVPxX/Screenshot-from-2023-08-01-09-38-13.png\" alt=\"public\"></p>\n<p>When we sort our submission by private score, we get submissions with TTA, without dilation and single models but I also have one submission with dilation that scores 0.50x on private which is weird.<br>\n<img src=\"https://i.ibb.co/Ss8WSFy/Screenshot-from-2023-08-01-09-41-36.png\" alt=\"private\"></p>\n<p>Our best submission was 14th place on private leaderboard but at the end Ayberk was able select a good enough submission that made us finish 30th.</p>",
      "rawMarkdown": "## Overview\n\n@ayberkmir and I joined the competition very late. I wasted my first week because I didn't read the metric properly and submitting not accordingly. We didn't have time to try so many diverse things so it will be quite simple.\n\n## Validation\n\nAyberk was using stratified single train/test split (stratified on blood vessel count) and I was using 4 fold leave-one-WSI-out cross-validation. Surprisingly both of them worked on private leaderboard.\n\n## Models\n\nI mainly worked on Mask R-CNN and Cascade Mask R-CNN with ResNeXt101 backbone and Ayberk worked on YOLOv7 and v8 models.\n\n## Training\n\nI dropped duplicate annotations by simply suppressing overlapping boxes with 0.99 IoU when I was creating datasets. I also included glomeruli annotations and used unsure annotations as blood vessels. I thought if I treat unsure as blood vessels, training would be more stable. Ayberk didn't use glomeruli or unsure annotations and YOLO was already handling duplicate annotations.\n\nSince the WSIs were kinda similar, I decided not to use stain augmentations. I used multi scale training and scales are arranged between 1.5 and 2x raw scale i.e.\n\n```python\ntrain_scales = [\n    (768, 768), (832, 832), (896, 896),\n    (960, 960), (1024, 1024), (1088, 1088),\n    (1152, 1152), (1216, 1216), (1280, 1280)\n]\n```\nOther training augmentatios were random horizontal, vertical and diagonal flip, random brightness, contrast, hue and saturation.\n\nAyberk was using fixed size of 1024x1024 because we thought built-in augmentations in YOLO are already strong enough.\n\nWe used almost default values for optimizer and schedulers. I only added another learning rate multiplier step.\n\n## Post-processing\n\nBoth of us thought post-processing was the most important in this competition because of the weird metric. We dedicated most of our time on this part rather than modelling.\n\nFirst of all, we used 3x TTA (horizontal, vertical and diagonal flip) when we were predicting with our models. TTA predictions were merged using weighted boxes fusion. Masks that belong to fused boxes are averaged. Unfortunately, I couldn't find a way to retrieve soft mask predictions in mmdetection and I had to average binary masks. We didn't use multi scale TTA. YOLO model were taking 1024x1024 and Mask R-CNN models were taking 1280x1280 inputs.\n\nSince I had 4 folds, I also had to merge their predictions and I used weighted boxes fusion with same configurations. We also tried to merge YOLO and Mask R-CNN predictions at this stage. It was working as good as other predictions on public leaderboard but always failed on private leaderboard for some reason.\n\nWe also tried the same submissions with and without dilation and we found that TTA was reducing the effect of dilation A LOT on public leaderboard. It was a really good sign because TTA is safe and dilation isn't.\n\nWe had different submissions with combinations of models, folds, TTA, dilation, score multiplication, WBF tuning and etc.\n\nWhen we sort our submissions by public score, we get submission with dilation or ensemble.\n![public](https://i.ibb.co/8PpVPxX/Screenshot-from-2023-08-01-09-38-13.png)\n\nWhen we sort our submission by private score, we get submissions with TTA, without dilation and single models but I also have one submission with dilation that scores 0.50x on private which is weird.\n![private](https://i.ibb.co/Ss8WSFy/Screenshot-from-2023-08-01-09-41-36.png)\n\nOur best submission was 14th place on private leaderboard but at the end Ayberk was able select a good enough submission that made us finish 30th.",
      "votes": 12
    },
    {
      "id": 2368760,
      "postDate": "2023-08-01T11:11:48.310Z",
      "content": "<p>First of all, thank you for your explanation and solution. Very well explained!</p>",
      "rawMarkdown": "First of all, thank you for your explanation and solution. Very well explained!",
      "votes": 1
    },
    {
      "id": 2368537,
      "postDate": "2023-08-01T08:20:31.590Z",
      "content": "<p>I think you're doing the right thing. Reprocessing is a big part of this game.</p>",
      "rawMarkdown": "I think you're doing the right thing. Reprocessing is a big part of this game.",
      "votes": 1
    },
    {
      "id": 2372400,
      "postDate": "2023-08-03T16:39:24.057Z",
      "content": "<p>Thanks for sharing！Well done!</p>",
      "rawMarkdown": "Thanks for sharing！Well done!",
      "votes": 1,
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2368760,
      "author_name": "Berk Karabillioglu",
      "author_url": "",
      "post_date": "2023-08-01T11:11:48.310000",
      "content": "<p>First of all, thank you for your explanation and solution. Very well explained!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2368537,
      "author_name": "bent1e",
      "author_url": "",
      "post_date": "2023-08-01T08:20:31.590000",
      "content": "<p>I think you're doing the right thing. Reprocessing is a big part of this game.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2372400,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-08-03T16:39:24.057000",
      "content": "<p>Thanks for sharing！Well done!</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2368446": "## Overview\n\n@ayberkmir and I joined the competition very late. I wasted my first week because I didn't read the metric properly and submitting not accordingly. We didn't have time to try so many diverse things so it will be quite simple.\n\n## Validation\n\nAyberk was using stratified single train/test split (stratified on blood vessel count) and I was using 4 fold leave-one-WSI-out cross-validation. Surprisingly both of them worked on private leaderboard.\n\n## Models\n\nI mainly worked on Mask R-CNN and Cascade Mask R-CNN with ResNeXt101 backbone and Ayberk worked on YOLOv7 and v8 models.\n\n## Training\n\nI dropped duplicate annotations by simply suppressing overlapping boxes with 0.99 IoU when I was creating datasets. I also included glomeruli annotations and used unsure annotations as blood vessels. I thought if I treat unsure as blood vessels, training would be more stable. Ayberk didn't use glomeruli or unsure annotations and YOLO was already handling duplicate annotations.\n\nSince the WSIs were kinda similar, I decided not to use stain augmentations. I used multi scale training and scales are arranged between 1.5 and 2x raw scale i.e.\n\n```python\ntrain_scales = [\n    (768, 768), (832, 832), (896, 896),\n    (960, 960), (1024, 1024), (1088, 1088),\n    (1152, 1152), (1216, 1216), (1280, 1280)\n]\n```\nOther training augmentatios were random horizontal, vertical and diagonal flip, random brightness, contrast, hue and saturation.\n\nAyberk was using fixed size of 1024x1024 because we thought built-in augmentations in YOLO are already strong enough.\n\nWe used almost default values for optimizer and schedulers. I only added another learning rate multiplier step.\n\n## Post-processing\n\nBoth of us thought post-processing was the most important in this competition because of the weird metric. We dedicated most of our time on this part rather than modelling.\n\nFirst of all, we used 3x TTA (horizontal, vertical and diagonal flip) when we were predicting with our models. TTA predictions were merged using weighted boxes fusion. Masks that belong to fused boxes are averaged. Unfortunately, I couldn't find a way to retrieve soft mask predictions in mmdetection and I had to average binary masks. We didn't use multi scale TTA. YOLO model were taking 1024x1024 and Mask R-CNN models were taking 1280x1280 inputs.\n\nSince I had 4 folds, I also had to merge their predictions and I used weighted boxes fusion with same configurations. We also tried to merge YOLO and Mask R-CNN predictions at this stage. It was working as good as other predictions on public leaderboard but always failed on private leaderboard for some reason.\n\nWe also tried the same submissions with and without dilation and we found that TTA was reducing the effect of dilation A LOT on public leaderboard. It was a really good sign because TTA is safe and dilation isn't.\n\nWe had different submissions with combinations of models, folds, TTA, dilation, score multiplication, WBF tuning and etc.\n\nWhen we sort our submissions by public score, we get submission with dilation or ensemble.\n![public](https://i.ibb.co/8PpVPxX/Screenshot-from-2023-08-01-09-38-13.png)\n\nWhen we sort our submission by private score, we get submissions with TTA, without dilation and single models but I also have one submission with dilation that scores 0.50x on private which is weird.\n![private](https://i.ibb.co/Ss8WSFy/Screenshot-from-2023-08-01-09-41-36.png)\n\nOur best submission was 14th place on private leaderboard but at the end Ayberk was able select a good enough submission that made us finish 30th.",
    "2368760": "First of all, thank you for your explanation and solution. Very well explained!",
    "2368537": "I think you're doing the right thing. Reprocessing is a big part of this game.",
    "2372400": "Thanks for sharing！Well done!"
  }
}