{
  "id": 280003,
  "title": "662nd place solution",
  "url": "/competitions/rsna-miccai-brain-tumor-radiogenomic-classification/writeups/luminide-trial-662nd-place-solution",
  "author_name": "",
  "post_date": "2021-10-19T22:20:35.155595100Z",
  "votes": 24,
  "comment_count": 8,
  "views": 0,
  "content": "<p>Our submission is a 50/50 average between a model based on Pyradiomics, and a model based on 3d neural networks.</p>\n<p>Knowing that the train sample size was very small and also we have a very low signal-to-noise ratio, we sought to make a robust approach.</p>\n<ul>\n<li>Only focus on the tumor instead of the whole brain's image</li>\n<li>Use information from all 4 modalities</li>\n<li>Use Log Loss in local Cross-Validation</li>\n<li>Use pretrained models where possible, which have seen many more brains than our dataset size</li>\n<li>Remove features where train and test set distributions differ</li>\n<li>LightGBM parameters are very shallow / \"dumb\". Also it uses only 6 features</li>\n<li>Neural networks blended across multiple different architectures</li>\n<li>Models are bagged / models trained with augmentations to reduce volatility</li>\n<li>Combine Pyradiomics submission with Neural Network submission to get the best of both worlds, try to be as robust as possible</li>\n</ul>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>Local CV</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Pyradiomics LGBM</td>\n<td>0.65531</td>\n<td>0.62420</td>\n<td>0.52228</td>\n</tr>\n<tr>\n<td>3d Neural Networks</td>\n<td>0.6474</td>\n<td>0.62315</td>\n<td>???</td>\n</tr>\n<tr>\n<td>50/50 Blend</td>\n<td>???</td>\n<td>0.65380</td>\n<td>0.53221</td>\n</tr>\n</tbody>\n</table>\n<p>See image:<br>\n<img src=\"https://raw.githubusercontent.com/Quetzalcohuatl/rsnamiccaikaggle2021/main/rsna_miccai_brain_tumor_mgmt_kaggle_solution_2021.png\" alt=\"662nd solution\"></p>\n<p>Sincere thanks to Anil for his hard work and Luminide for access to data storage and servers! I thought working on their platform, tracking experiments, and having the instance detach after training was complete was quite useful.</p>",
  "messages": [
    {
      "id": "1550668",
      "postDate": "10/19/2021 22:20:35",
      "content": "<p>Our submission is a 50/50 average between a model based on Pyradiomics, and a model based on 3d neural networks.</p>\n<p>Knowing that the train sample size was very small and also we have a very low signal-to-noise ratio, we sought to make a robust approach.</p>\n<ul>\n<li>Only focus on the tumor instead of the whole brain's image</li>\n<li>Use information from all 4 modalities</li>\n<li>Use Log Loss in local Cross-Validation</li>\n<li>Use pretrained models where possible, which have seen many more brains than our dataset size</li>\n<li>Remove features where train and test set distributions differ</li>\n<li>LightGBM parameters are very shallow / \"dumb\". Also it uses only 6 features</li>\n<li>Neural networks blended across multiple different architectures</li>\n<li>Models are bagged / models trained with augmentations to reduce volatility</li>\n<li>Combine Pyradiomics submission with Neural Network submission to get the best of both worlds, try to be as robust as possible</li>\n</ul>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>Local CV</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Pyradiomics LGBM</td>\n<td>0.65531</td>\n<td>0.62420</td>\n<td>0.52228</td>\n</tr>\n<tr>\n<td>3d Neural Networks</td>\n<td>0.6474</td>\n<td>0.62315</td>\n<td>???</td>\n</tr>\n<tr>\n<td>50/50 Blend</td>\n<td>???</td>\n<td>0.65380</td>\n<td>0.53221</td>\n</tr>\n</tbody>\n</table>\n<p>See image:<br>\n<img src=\"https://raw.githubusercontent.com/Quetzalcohuatl/rsnamiccaikaggle2021/main/rsna_miccai_brain_tumor_mgmt_kaggle_solution_2021.png\" alt=\"662nd solution\"></p>\n<p>Sincere thanks to Anil for his hard work and Luminide for access to data storage and servers! I thought working on their platform, tracking experiments, and having the instance detach after training was complete was quite useful.</p>",
      "rawMarkdown": "Our submission is a 50/50 average between a model based on Pyradiomics, and a model based on 3d neural networks.\n\nKnowing that the train sample size was very small and also we have a very low signal-to-noise ratio, we sought to make a robust approach.\n\n- Only focus on the tumor instead of the whole brain's image\n- Use information from all 4 modalities\n- Use Log Loss in local Cross-Validation\n- Use pretrained models where possible, which have seen many more brains than our dataset size\n- Remove features where train and test set distributions differ\n- LightGBM parameters are very shallow / \"dumb\". Also it uses only 6 features\n- Neural networks blended across multiple different architectures\n- Models are bagged / models trained with augmentations to reduce volatility\n- Combine Pyradiomics submission with Neural Network submission to get the best of both worlds, try to be as robust as possible\n\n| Model | Local CV | Public LB | Private LB |\n| --- | --- | --- | --- |\n| Pyradiomics LGBM | 0.65531 | 0.62420 | 0.52228 |\n| 3d Neural Networks | 0.6474 | 0.62315 | ??? |\n| 50/50 Blend | ??? | 0.65380 | 0.53221 |\n\nSee image:\n![662nd solution](https://raw.githubusercontent.com/Quetzalcohuatl/rsnamiccaikaggle2021/main/rsna_miccai_brain_tumor_mgmt_kaggle_solution_2021.png)\n\nSincere thanks to Anil for his hard work and Luminide for access to data storage and servers! I thought working on their platform, tracking experiments, and having the instance detach after training was complete was quite useful.",
      "votes": null
    },
    {
      "id": "1551003",
      "postDate": "10/20/2021 07:56:16",
      "content": "<p>quality of your approach != your rank, well a quality much higher<br>\nOur solution also involved working with images with tumours and using OD to find them.</p>",
      "rawMarkdown": "quality of your approach != your rank, well a quality much higher\nOur solution also involved working with images with tumours and using OD to find them.",
      "votes": null
    },
    {
      "id": "1551127",
      "postDate": "10/20/2021 10:28:18",
      "content": "<p>Your approach is very similar to ours but radiomics features didn't work for us at all. What was your segmentation method?</p>",
      "rawMarkdown": "Your approach is very similar to ours but radiomics features didn't work for us at all. What was your segmentation method?",
      "votes": null
    },
    {
      "id": "1551275",
      "postDate": "10/20/2021 13:23:32",
      "content": "<p>We also tried Radiomics features from 3D tumor segmentation, and some of the linear models seemed promisingly stable on these features (5-fold CV: 0.64, std &lt; 0.01). However, no luck on private, so we must have overfitted to something :/</p>\n<p>Our steps were:</p>\n<ul>\n<li>Registered all modalities together along with Task-1 segmentation maps.</li>\n<li>Trained a 3D tumor segmentation UNet - transfer learning from MRI tumor segmentation 2D UNet weights</li>\n<li>Calculated Radiomic features from OOF tumor predictions</li>\n<li>Fitted a LinReg model to predict MGMT from Radiomics features</li>\n<li>Trained 2D CNNs from axial slices selected from tumor heights</li>\n<li>Ensembled everything together</li>\n</ul>\n<p>Even though MGMT status was difficult to predict, 3D tumor segmentation learned pretty well. The Figure is from our 3D seg. Unet predictions (green=GT tumor, red=Pred).</p>\n<p><img src=\"https://raw.githubusercontent.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification/main/media/segmentation_model_predictions.png\" alt=\"3d tumor seg\"></p>\n<p><a href=\"https://github.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification\" target=\"_blank\">github link</a></p>",
      "rawMarkdown": "We also tried Radiomics features from 3D tumor segmentation, and some of the linear models seemed promisingly stable on these features (5-fold CV: 0.64, std < 0.01). However, no luck on private, so we must have overfitted to something :/\n\nOur steps were:\n- Registered all modalities together along with Task-1 segmentation maps.\n- Trained a 3D tumor segmentation UNet - transfer learning from MRI tumor segmentation 2D UNet weights\n- Calculated Radiomic features from OOF tumor predictions\n- Fitted a LinReg model to predict MGMT from Radiomics features\n- Trained 2D CNNs from axial slices selected from tumor heights\n- Ensembled everything together\n\nEven though MGMT status was difficult to predict, 3D tumor segmentation learned pretty well. The Figure is from our 3D seg. Unet predictions (green=GT tumor, red=Pred).\n\n![3d tumor seg](https://raw.githubusercontent.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification/main/media/segmentation_model_predictions.png)\n\n[github link](https://github.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification)",
      "votes": null
    },
    {
      "id": "1554393",
      "postDate": "10/23/2021 03:50:01",
      "content": "<p>Thank you Gunes, and big congrats on your and authman's finish. I was actually relying a lot on your TRENDS Neuroimaging solution for some of the work we did in this competition.</p>\n<p>For segmentation, we used the pretrained BraTS 2020 segmentation model provided by MONAI. <a href=\"https://github.com/Project-MONAI/MONAI\" target=\"_blank\">https://github.com/Project-MONAI/MONAI</a></p>",
      "rawMarkdown": "Thank you Gunes, and big congrats on your and authman's finish. I was actually relying a lot on your TRENDS Neuroimaging solution for some of the work we did in this competition.\n\nFor segmentation, we used the pretrained BraTS 2020 segmentation model provided by MONAI. https://github.com/Project-MONAI/MONAI",
      "votes": null
    },
    {
      "id": "1554394",
      "postDate": "10/23/2021 03:52:41",
      "content": "<p>Thank you! We are sad that we did not generalize to the private LB. I was rooting for you and your team because of how much you contributed to the Discussions forum. Congrats on an awesome finish!</p>",
      "rawMarkdown": "Thank you! We are sad that we did not generalize to the private LB. I was rooting for you and your team because of how much you contributed to the Discussions forum. Congrats on an awesome finish!",
      "votes": null
    },
    {
      "id": "1554395",
      "postDate": "10/23/2021 03:59:51",
      "content": "<p>Thank you for sharing your approach.</p>",
      "rawMarkdown": "Thank you for sharing your approach.",
      "votes": null
    },
    {
      "id": "1556133",
      "postDate": "10/24/2021 15:34:49",
      "content": "<p>Wow I didn't expect someone to use our solution from that competition. I almost forgot about that. I'm really glad that it inspired you for this problem. </p>",
      "rawMarkdown": "Wow I didn't expect someone to use our solution from that competition. I almost forgot about that. I'm really glad that it inspired you for this problem.",
      "votes": null
    },
    {
      "id": "1556647",
      "postDate": "10/25/2021 04:11:27",
      "content": "<p>Sadly we didn't choose the submission using the techniques inspired by you. Nor did we choose the \"PCA\" submission inspired by 1st place from Trends. Maybe a lesson for next time!</p>",
      "rawMarkdown": "Sadly we didn't choose the submission using the techniques inspired by you. Nor did we choose the \"PCA\" submission inspired by 1st place from Trends. Maybe a lesson for next time!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1551003,
      "author_name": "mrinath",
      "author_url": "",
      "post_date": "10/20/2021 07:56:16",
      "content": "<p>quality of your approach != your rank, well a quality much higher<br>\nOur solution also involved working with images with tumours and using OD to find them.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1554394,
          "author_name": "returnofsputnik",
          "author_url": "",
          "post_date": "10/23/2021 03:52:41",
          "content": "<p>Thank you! We are sad that we did not generalize to the private LB. I was rooting for you and your team because of how much you contributed to the Discussions forum. Congrats on an awesome finish!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1551127,
      "author_name": "gunesevitan",
      "author_url": "",
      "post_date": "10/20/2021 10:28:18",
      "content": "<p>Your approach is very similar to ours but radiomics features didn't work for us at all. What was your segmentation method?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1554393,
          "author_name": "returnofsputnik",
          "author_url": "",
          "post_date": "10/23/2021 03:50:01",
          "content": "<p>Thank you Gunes, and big congrats on your and authman's finish. I was actually relying a lot on your TRENDS Neuroimaging solution for some of the work we did in this competition.</p>\n<p>For segmentation, we used the pretrained BraTS 2020 segmentation model provided by MONAI. <a href=\"https://github.com/Project-MONAI/MONAI\" target=\"_blank\">https://github.com/Project-MONAI/MONAI</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1556133,
          "author_name": "gunesevitan",
          "author_url": "",
          "post_date": "10/24/2021 15:34:49",
          "content": "<p>Wow I didn't expect someone to use our solution from that competition. I almost forgot about that. I'm really glad that it inspired you for this problem. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1556647,
          "author_name": "returnofsputnik",
          "author_url": "",
          "post_date": "10/25/2021 04:11:27",
          "content": "<p>Sadly we didn't choose the submission using the techniques inspired by you. Nor did we choose the \"PCA\" submission inspired by 1st place from Trends. Maybe a lesson for next time!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1551275,
      "author_name": "qitvision",
      "author_url": "",
      "post_date": "10/20/2021 13:23:32",
      "content": "<p>We also tried Radiomics features from 3D tumor segmentation, and some of the linear models seemed promisingly stable on these features (5-fold CV: 0.64, std &lt; 0.01). However, no luck on private, so we must have overfitted to something :/</p>\n<p>Our steps were:</p>\n<ul>\n<li>Registered all modalities together along with Task-1 segmentation maps.</li>\n<li>Trained a 3D tumor segmentation UNet - transfer learning from MRI tumor segmentation 2D UNet weights</li>\n<li>Calculated Radiomic features from OOF tumor predictions</li>\n<li>Fitted a LinReg model to predict MGMT from Radiomics features</li>\n<li>Trained 2D CNNs from axial slices selected from tumor heights</li>\n<li>Ensembled everything together</li>\n</ul>\n<p>Even though MGMT status was difficult to predict, 3D tumor segmentation learned pretty well. The Figure is from our 3D seg. Unet predictions (green=GT tumor, red=Pred).</p>\n<p><img src=\"https://raw.githubusercontent.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification/main/media/segmentation_model_predictions.png\" alt=\"3d tumor seg\"></p>\n<p><a href=\"https://github.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification\" target=\"_blank\">github link</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1554395,
          "author_name": "returnofsputnik",
          "author_url": "",
          "post_date": "10/23/2021 03:59:51",
          "content": "<p>Thank you for sharing your approach.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1550668": "Our submission is a 50/50 average between a model based on Pyradiomics, and a model based on 3d neural networks.\n\nKnowing that the train sample size was very small and also we have a very low signal-to-noise ratio, we sought to make a robust approach.\n\n- Only focus on the tumor instead of the whole brain's image\n- Use information from all 4 modalities\n- Use Log Loss in local Cross-Validation\n- Use pretrained models where possible, which have seen many more brains than our dataset size\n- Remove features where train and test set distributions differ\n- LightGBM parameters are very shallow / \"dumb\". Also it uses only 6 features\n- Neural networks blended across multiple different architectures\n- Models are bagged / models trained with augmentations to reduce volatility\n- Combine Pyradiomics submission with Neural Network submission to get the best of both worlds, try to be as robust as possible\n\n| Model | Local CV | Public LB | Private LB |\n| --- | --- | --- | --- |\n| Pyradiomics LGBM | 0.65531 | 0.62420 | 0.52228 |\n| 3d Neural Networks | 0.6474 | 0.62315 | ??? |\n| 50/50 Blend | ??? | 0.65380 | 0.53221 |\n\nSee image:\n![662nd solution](https://raw.githubusercontent.com/Quetzalcohuatl/rsnamiccaikaggle2021/main/rsna_miccai_brain_tumor_mgmt_kaggle_solution_2021.png)\n\nSincere thanks to Anil for his hard work and Luminide for access to data storage and servers! I thought working on their platform, tracking experiments, and having the instance detach after training was complete was quite useful.",
    "1551003": "quality of your approach != your rank, well a quality much higher\nOur solution also involved working with images with tumours and using OD to find them.",
    "1551127": "Your approach is very similar to ours but radiomics features didn't work for us at all. What was your segmentation method?",
    "1551275": "We also tried Radiomics features from 3D tumor segmentation, and some of the linear models seemed promisingly stable on these features (5-fold CV: 0.64, std < 0.01). However, no luck on private, so we must have overfitted to something :/\n\nOur steps were:\n- Registered all modalities together along with Task-1 segmentation maps.\n- Trained a 3D tumor segmentation UNet - transfer learning from MRI tumor segmentation 2D UNet weights\n- Calculated Radiomic features from OOF tumor predictions\n- Fitted a LinReg model to predict MGMT from Radiomics features\n- Trained 2D CNNs from axial slices selected from tumor heights\n- Ensembled everything together\n\nEven though MGMT status was difficult to predict, 3D tumor segmentation learned pretty well. The Figure is from our 3D seg. Unet predictions (green=GT tumor, red=Pred).\n\n![3d tumor seg](https://raw.githubusercontent.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification/main/media/segmentation_model_predictions.png)\n\n[github link](https://github.com/jpjuvo/RSNA-MICCAI-Brain-Tumor-Classification)",
    "1554393": "Thank you Gunes, and big congrats on your and authman's finish. I was actually relying a lot on your TRENDS Neuroimaging solution for some of the work we did in this competition.\n\nFor segmentation, we used the pretrained BraTS 2020 segmentation model provided by MONAI. https://github.com/Project-MONAI/MONAI",
    "1554394": "Thank you! We are sad that we did not generalize to the private LB. I was rooting for you and your team because of how much you contributed to the Discussions forum. Congrats on an awesome finish!",
    "1554395": "Thank you for sharing your approach.",
    "1556133": "Wow I didn't expect someone to use our solution from that competition. I almost forgot about that. I'm really glad that it inspired you for this problem.",
    "1556647": "Sadly we didn't choose the submission using the techniques inspired by you. Nor did we choose the \"PCA\" submission inspired by 1st place from Trends. Maybe a lesson for next time!"
  },
  "source": "meta"
}