{
  "id": 238141,
  "title": "[0.948 private LB] Uptrain boost from 0.943; 384px to 512px",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/238141",
  "author_name": "rosuluc",
  "post_date": "2021-05-11T10:52:19.532000",
  "votes": 6,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I had a hard time coming up with a meaningful CV and also trying to corelate it to the public LB so I missed my best performing models. To get a better picture of the LB I first ran the whole data set and then just d488 and extracted the score score for the other 4 files. This was frustrating to do for every experiment and also decreased submission limit by 2.</p>\n<p>My final submission scored only 0.943 but I have one 0.948 and five 0.947 models.<br>\nWhile most most of my models scored around 0.940-0.945, what provided the boost up to 0.948 was uptraining the previous best scoring model from 384x384 to 512x512.</p>\n<p>Config:</p>\n<ol>\n<li>Unet/effnet-b4</li>\n<li>1536x1536 downsampled to 384 then 512; </li>\n<li>loss: BCE+DICE then uptrain loss: BCE+ modified focal DICE</li>\n<li>single model, 10% StratifiedShuffleSplit wrt to tiff_id</li>\n<li>lr: 1e-3 w/ MultiStepLR(optimizer, milestones=[7, 14], gamma=0.3); uptrain 1e-4</li>\n<li>dice &amp; focal dice are weighted with a centered square window(1024) with windows valued at 1 and sides as 0.2 (i wanted to reduce the amount contributed to the loss by the img edges since they have less context to make accurate predictions but also because the sides are discarded at inference time)</li>\n<li>25eps/5eps uptrain</li>\n<li>some basic augs</li>\n</ol>\n<p>Didn't help:</p>\n<ol>\n<li>d488 hand labels (private LB stays pretty much the same)</li>\n<li>external data (from zenodo) &amp; pseudo labels (couple of tiffs from hubmap portal) (private LB decreased)</li>\n<li>CosineAnnealingWarmRestarts (different configs) (private LB decreased)</li>\n<li>larger encoders, tried effnet-b5 (private LB decreased to 0.938)</li>\n<li>other networks than Unet (private LB stayed the same)</li>\n</ol>\n<p>I'm not sure yet tho how much of this boost is due to the model and how much to the upscaling back to 1536.</p>\n<p>Focal dice:</p>\n<pre><code>    `TP = (probability * targets * weights).sum()\n\n    FP = ((1-targets) * probability**self.gamma * weights).sum()\n    FN = (targets * (1-probability)**self.gamma * weights).sum()\n\n    dice_score = (TP + self.eps) / (TP + 0.5 * FP + 0.5 * FN + self.eps)\n\n    return 1 - dice_score`\n</code></pre>\n<p><img src=\"https://i.ibb.co/rbBCL4c/Capture.png\" alt=\"\"></p>",
  "messages": [
    {
      "id": 1301982,
      "postDate": "2021-05-11T10:52:19.533Z",
      "content": "<p>I had a hard time coming up with a meaningful CV and also trying to corelate it to the public LB so I missed my best performing models. To get a better picture of the LB I first ran the whole data set and then just d488 and extracted the score score for the other 4 files. This was frustrating to do for every experiment and also decreased submission limit by 2.</p>\n<p>My final submission scored only 0.943 but I have one 0.948 and five 0.947 models.<br>\nWhile most most of my models scored around 0.940-0.945, what provided the boost up to 0.948 was uptraining the previous best scoring model from 384x384 to 512x512.</p>\n<p>Config:</p>\n<ol>\n<li>Unet/effnet-b4</li>\n<li>1536x1536 downsampled to 384 then 512; </li>\n<li>loss: BCE+DICE then uptrain loss: BCE+ modified focal DICE</li>\n<li>single model, 10% StratifiedShuffleSplit wrt to tiff_id</li>\n<li>lr: 1e-3 w/ MultiStepLR(optimizer, milestones=[7, 14], gamma=0.3); uptrain 1e-4</li>\n<li>dice &amp; focal dice are weighted with a centered square window(1024) with windows valued at 1 and sides as 0.2 (i wanted to reduce the amount contributed to the loss by the img edges since they have less context to make accurate predictions but also because the sides are discarded at inference time)</li>\n<li>25eps/5eps uptrain</li>\n<li>some basic augs</li>\n</ol>\n<p>Didn't help:</p>\n<ol>\n<li>d488 hand labels (private LB stays pretty much the same)</li>\n<li>external data (from zenodo) &amp; pseudo labels (couple of tiffs from hubmap portal) (private LB decreased)</li>\n<li>CosineAnnealingWarmRestarts (different configs) (private LB decreased)</li>\n<li>larger encoders, tried effnet-b5 (private LB decreased to 0.938)</li>\n<li>other networks than Unet (private LB stayed the same)</li>\n</ol>\n<p>I'm not sure yet tho how much of this boost is due to the model and how much to the upscaling back to 1536.</p>\n<p>Focal dice:</p>\n<pre><code>    `TP = (probability * targets * weights).sum()\n\n    FP = ((1-targets) * probability**self.gamma * weights).sum()\n    FN = (targets * (1-probability)**self.gamma * weights).sum()\n\n    dice_score = (TP + self.eps) / (TP + 0.5 * FP + 0.5 * FN + self.eps)\n\n    return 1 - dice_score`\n</code></pre>\n<p><img src=\"https://i.ibb.co/rbBCL4c/Capture.png\" alt=\"\"></p>",
      "rawMarkdown": "I had a hard time coming up with a meaningful CV and also trying to corelate it to the public LB so I missed my best performing models. To get a better picture of the LB I first ran the whole data set and then just d488 and extracted the score score for the other 4 files. This was frustrating to do for every experiment and also decreased submission limit by 2.\n\nMy final submission scored only 0.943 but I have one 0.948 and five 0.947 models.\nWhile most most of my models scored around 0.940-0.945, what provided the boost up to 0.948 was uptraining the previous best scoring model from 384x384 to 512x512.\n\nConfig:\n1.  Unet/effnet-b4\n2. 1536x1536 downsampled to 384 then 512; \n3. loss: BCE+DICE then uptrain loss: BCE+ modified focal DICE\n4.  single model, 10% StratifiedShuffleSplit wrt to tiff_id\n5. lr: 1e-3 w/ MultiStepLR(optimizer, milestones=[7, 14], gamma=0.3); uptrain 1e-4\n6. dice & focal dice are weighted with a centered square window(1024) with windows valued at 1 and sides as 0.2 (i wanted to reduce the amount contributed to the loss by the img edges since they have less context to make accurate predictions but also because the sides are discarded at inference time)\n7. 25eps/5eps uptrain\n8. some basic augs\n\nDidn't help:\n1. d488 hand labels (private LB stays pretty much the same)\n2. external data (from zenodo) & pseudo labels (couple of tiffs from hubmap portal) (private LB decreased)\n3. CosineAnnealingWarmRestarts (different configs) (private LB decreased)\n4. larger encoders, tried effnet-b5 (private LB decreased to 0.938)\n5. other networks than Unet (private LB stayed the same)\n\n\nI'm not sure yet tho how much of this boost is due to the model and how much to the upscaling back to 1536.\n\nFocal dice:\n\n        `TP = (probability * targets * weights).sum()\n        \n        FP = ((1-targets) * probability**self.gamma * weights).sum()\n        FN = (targets * (1-probability)**self.gamma * weights).sum()\n        \n        dice_score = (TP + self.eps) / (TP + 0.5 * FP + 0.5 * FN + self.eps)\n                       \n        return 1 - dice_score`\n\n![](https://i.ibb.co/rbBCL4c/Capture.png)",
      "votes": 6
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1301982": "I had a hard time coming up with a meaningful CV and also trying to corelate it to the public LB so I missed my best performing models. To get a better picture of the LB I first ran the whole data set and then just d488 and extracted the score score for the other 4 files. This was frustrating to do for every experiment and also decreased submission limit by 2.\n\nMy final submission scored only 0.943 but I have one 0.948 and five 0.947 models.\nWhile most most of my models scored around 0.940-0.945, what provided the boost up to 0.948 was uptraining the previous best scoring model from 384x384 to 512x512.\n\nConfig:\n1.  Unet/effnet-b4\n2. 1536x1536 downsampled to 384 then 512; \n3. loss: BCE+DICE then uptrain loss: BCE+ modified focal DICE\n4.  single model, 10% StratifiedShuffleSplit wrt to tiff_id\n5. lr: 1e-3 w/ MultiStepLR(optimizer, milestones=[7, 14], gamma=0.3); uptrain 1e-4\n6. dice & focal dice are weighted with a centered square window(1024) with windows valued at 1 and sides as 0.2 (i wanted to reduce the amount contributed to the loss by the img edges since they have less context to make accurate predictions but also because the sides are discarded at inference time)\n7. 25eps/5eps uptrain\n8. some basic augs\n\nDidn't help:\n1. d488 hand labels (private LB stays pretty much the same)\n2. external data (from zenodo) & pseudo labels (couple of tiffs from hubmap portal) (private LB decreased)\n3. CosineAnnealingWarmRestarts (different configs) (private LB decreased)\n4. larger encoders, tried effnet-b5 (private LB decreased to 0.938)\n5. other networks than Unet (private LB stayed the same)\n\n\nI'm not sure yet tho how much of this boost is due to the model and how much to the upscaling back to 1536.\n\nFocal dice:\n\n        `TP = (probability * targets * weights).sum()\n        \n        FP = ((1-targets) * probability**self.gamma * weights).sum()\n        FN = (targets * (1-probability)**self.gamma * weights).sum()\n        \n        dice_score = (TP + self.eps) / (TP + 0.5 * FP + 0.5 * FN + self.eps)\n                       \n        return 1 - dice_score`\n\n![](https://i.ibb.co/rbBCL4c/Capture.png)"
  }
}