{
  "id": 280033,
  "title": "2nd place solution",
  "url": "/competitions/rsna-miccai-brain-tumor-radiogenomic-classification/writeups/minh-phan-2nd-place-solution",
  "author_name": "",
  "post_date": "2021-10-22T02:26:00.660Z",
  "votes": 37,
  "comment_count": 15,
  "views": 0,
  "content": "<p>First of all, I would like to thank Kaggle and the competition host for hosting this challenge, and also to all competitors who participated in this challenge. My main reason for joining this competition was to learn from the community, and I sure did learned a lot from you. <br>\nThe second place came as a surprise to me and I was not expecting it at all, since my public LB score was not high (0.65-0.67)<br>\nAnyway, I will share my approach to this competition. My approach is simple, using a CNN-LSTM architecture to do Classification task. For the CNN part I used Efficientnet and I trained LSTM part from scratch.<br>\nAll 4 types of MRI image sequences (FLAIR, T1w, T1wCE, T2w) are used as inputs. </p>\n<h1>Generate fused MRI sequences</h1>\n<ul>\n<li>I obtained images from <a href=\"https://www.kaggle.com/jonathanbesomi\" target=\"_blank\">@jonathanbesomi</a> 's dataset <a href=\"https://www.kaggle.com/jonathanbesomi/rsna-miccai-png\" target=\"_blank\">https://www.kaggle.com/jonathanbesomi/rsna-miccai-png</a>, who converted all images into PNG format and removed empty DICOM images</li>\n<li>For each MRI image sequence, a number of T=10 frames is selected using uniform temporal subsampling. For example, with a video containing 91 frames, the frames 1, 11, 21, …, 91 are selected with this sampling strategy. </li>\n</ul>\n<pre><code>def uniform_temporal_subsample(x, num_samples):\n    '''\n        Moddified from https://github.com/facebookresearch/pytorchvideo/blob/d7874f788bc00a\n7badfb4310a912f6e531ffd6d3/pytorchvideo/transforms/functional.py#L19\n    '''\n    t = len(x)\n    indices = torch.linspace(0, t - 1, num_samples)\n    indices = torch.clamp(indices, 0, t - 1).long()\n    return [x[i] for i in indices]\n</code></pre>\n<ul>\n<li>In each selected time frame, 4 single-channel MRI images are concatenated to one 4-channels feature image before proceeding to the training/inference phase. If one (or more) MRI image type is missing for a patient, that image channel(s) is filled with 0.<br>\n<img src=\"https://drive.google.com/file/d/1ZhzxfxWtIsSoiEXkDj61HdQIEBufJnQS/view?usp=sharing\" alt=\"\"></li>\n</ul>\n<h1>Model</h1>\n<p><img src=\"https://drive.google.com/file/d/1ovsY6s9TlhlORjYl332B-34hEZy7gVY5/view?usp=sharing\" alt=\"\"><br>\nThe chosen CNN model for image features extraction task is a pre-trained EfficientNet B0 model. Since the input image has 4 channels, each corresponding to an MRI image type, a 2D convolution is applied to map the 4-channel image into a 3-channel feature map to fit the input shape of the pre-trained EfficientNet model. The classification head of the pre-trained model is also replaced by a fully-connected layer of size 256.<br>\nAfter obtaining the embeddings from all 10 selected frames, the embeddings are passed to 2 LSTM layers with hidden size=32 for both layers. Following that is a prediction layer with 1 node outputs the score.<br>\nThe model is trained for 15 epochs using binary cross-entropy loss, Adam optimizer with learning rate = 1e-4.</p>\n<h1>Preprocessing and augmentation</h1>\n<p>All input images are normalized and resized to the size of (256, 256). Images from one MRI type from one patient are randomly augmented with the same parameters.<br>\nThe list of augmentations are as follow:</p>\n<ul>\n<li>Horizontal Flip</li>\n<li>ShiftScaleRotate</li>\n<li>RandomBrightnessContrast</li>\n</ul>\n<p>I found that using too many and/or heavy augmentation methods did not help with the training process, therefore I only used simple augmentations when prepared the training data.</p>\n<h1>Cross validation.</h1>\n<p>Stratified K-fold cross validation is used with K=5 on MGMT value. During inference time, the mean prediction value from all 5 models is used as the ensemble’s prediction value.</p>\n<h1>Notebooks</h1>\n<p>All training and inference steps were done with Kaggle notebooks with GPUs:</p>\n<ul>\n<li>Training notebook: <a href=\"https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-train/notebook\" target=\"_blank\">https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-train/notebook</a></li>\n<li>Inference notbook: <a href=\"https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-inference\" target=\"_blank\">https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-inference</a></li>\n<li>Refactored Github <a href=\"https://github.com/minhnhatphan/rnsa21-cnn-lstm\" target=\"_blank\">link</a> and notebook <a href=\"https://www.kaggle.com/minhnhatphan/rnsa21-cnn-lstm-refactored/notebook\" target=\"_blank\">link</a></li>\n</ul>\n<p>I agree with others competitors that the results are not clinically useful. I found myself lucky to be on the top positions in the challenge. However, I think everyone can get something out of this, and the knowledge from this competition can benefit future competition as well as our understandings of similar problems. Again, I'd like to thank all participants and the Kaggle community for your insights and knowledge.</p>",
  "messages": [
    {
      "id": "1550845",
      "postDate": "10/20/2021 04:59:27",
      "content": "<p>First of all, I would like to thank Kaggle and the competition host for hosting this challenge, and also to all competitors who participated in this challenge. My main reason for joining this competition was to learn from the community, and I sure did learned a lot from you. <br>\nThe second place came as a surprise to me and I was not expecting it at all, since my public LB score was not high (0.65-0.67)<br>\nAnyway, I will share my approach to this competition. My approach is simple, using a CNN-LSTM architecture to do Classification task. For the CNN part I used Efficientnet and I trained LSTM part from scratch.<br>\nAll 4 types of MRI image sequences (FLAIR, T1w, T1wCE, T2w) are used as inputs. </p>\n<h1>Generate fused MRI sequences</h1>\n<ul>\n<li>I obtained images from <a href=\"https://www.kaggle.com/jonathanbesomi\" target=\"_blank\">@jonathanbesomi</a> 's dataset <a href=\"https://www.kaggle.com/jonathanbesomi/rsna-miccai-png\" target=\"_blank\">https://www.kaggle.com/jonathanbesomi/rsna-miccai-png</a>, who converted all images into PNG format and removed empty DICOM images</li>\n<li>For each MRI image sequence, a number of T=10 frames is selected using uniform temporal subsampling. For example, with a video containing 91 frames, the frames 1, 11, 21, …, 91 are selected with this sampling strategy. </li>\n</ul>\n<pre><code>def uniform_temporal_subsample(x, num_samples):\n    '''\n        Moddified from https://github.com/facebookresearch/pytorchvideo/blob/d7874f788bc00a\n7badfb4310a912f6e531ffd6d3/pytorchvideo/transforms/functional.py#L19\n    '''\n    t = len(x)\n    indices = torch.linspace(0, t - 1, num_samples)\n    indices = torch.clamp(indices, 0, t - 1).long()\n    return [x[i] for i in indices]\n</code></pre>\n<ul>\n<li>In each selected time frame, 4 single-channel MRI images are concatenated to one 4-channels feature image before proceeding to the training/inference phase. If one (or more) MRI image type is missing for a patient, that image channel(s) is filled with 0.<br>\n<img src=\"https://drive.google.com/file/d/1ZhzxfxWtIsSoiEXkDj61HdQIEBufJnQS/view?usp=sharing\" alt=\"\"></li>\n</ul>\n<h1>Model</h1>\n<p><img src=\"https://drive.google.com/file/d/1ovsY6s9TlhlORjYl332B-34hEZy7gVY5/view?usp=sharing\" alt=\"\"><br>\nThe chosen CNN model for image features extraction task is a pre-trained EfficientNet B0 model. Since the input image has 4 channels, each corresponding to an MRI image type, a 2D convolution is applied to map the 4-channel image into a 3-channel feature map to fit the input shape of the pre-trained EfficientNet model. The classification head of the pre-trained model is also replaced by a fully-connected layer of size 256.<br>\nAfter obtaining the embeddings from all 10 selected frames, the embeddings are passed to 2 LSTM layers with hidden size=32 for both layers. Following that is a prediction layer with 1 node outputs the score.<br>\nThe model is trained for 15 epochs using binary cross-entropy loss, Adam optimizer with learning rate = 1e-4.</p>\n<h1>Preprocessing and augmentation</h1>\n<p>All input images are normalized and resized to the size of (256, 256). Images from one MRI type from one patient are randomly augmented with the same parameters.<br>\nThe list of augmentations are as follow:</p>\n<ul>\n<li>Horizontal Flip</li>\n<li>ShiftScaleRotate</li>\n<li>RandomBrightnessContrast</li>\n</ul>\n<p>I found that using too many and/or heavy augmentation methods did not help with the training process, therefore I only used simple augmentations when prepared the training data.</p>\n<h1>Cross validation.</h1>\n<p>Stratified K-fold cross validation is used with K=5 on MGMT value. During inference time, the mean prediction value from all 5 models is used as the ensemble’s prediction value.</p>\n<h1>Notebooks</h1>\n<p>All training and inference steps were done with Kaggle notebooks with GPUs:</p>\n<ul>\n<li>Training notebook: <a href=\"https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-train/notebook\" target=\"_blank\">https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-train/notebook</a></li>\n<li>Inference notbook: <a href=\"https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-inference\" target=\"_blank\">https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-inference</a></li>\n<li>Refactored Github <a href=\"https://github.com/minhnhatphan/rnsa21-cnn-lstm\" target=\"_blank\">link</a> and notebook <a href=\"https://www.kaggle.com/minhnhatphan/rnsa21-cnn-lstm-refactored/notebook\" target=\"_blank\">link</a></li>\n</ul>\n<p>I agree with others competitors that the results are not clinically useful. I found myself lucky to be on the top positions in the challenge. However, I think everyone can get something out of this, and the knowledge from this competition can benefit future competition as well as our understandings of similar problems. Again, I'd like to thank all participants and the Kaggle community for your insights and knowledge.</p>",
      "rawMarkdown": "First of all, I would like to thank Kaggle and the competition host for hosting this challenge, and also to all competitors who participated in this challenge. My main reason for joining this competition was to learn from the community, and I sure did learned a lot from you. \nThe second place came as a surprise to me and I was not expecting it at all, since my public LB score was not high (0.65-0.67)\nAnyway, I will share my approach to this competition. My approach is simple, using a CNN-LSTM architecture to do Classification task. For the CNN part I used Efficientnet and I trained LSTM part from scratch.\nAll 4 types of MRI image sequences (FLAIR, T1w, T1wCE, T2w) are used as inputs. \n# Generate fused MRI sequences\n- I obtained images from @jonathanbesomi 's dataset https://www.kaggle.com/jonathanbesomi/rsna-miccai-png, who converted all images into PNG format and removed empty DICOM images\n- For each MRI image sequence, a number of T=10 frames is selected using uniform temporal subsampling. For example, with a video containing 91 frames, the frames 1, 11, 21, …, 91 are selected with this sampling strategy. \n```\ndef uniform_temporal_subsample(x, num_samples):\n    '''\n        Moddified from https://github.com/facebookresearch/pytorchvideo/blob/d7874f788bc00a\n7badfb4310a912f6e531ffd6d3/pytorchvideo/transforms/functional.py#L19\n    '''\n    t = len(x)\n    indices = torch.linspace(0, t - 1, num_samples)\n    indices = torch.clamp(indices, 0, t - 1).long()\n    return [x[i] for i in indices]\n```\n- In each selected time frame, 4 single-channel MRI images are concatenated to one 4-channels feature image before proceeding to the training/inference phase. If one (or more) MRI image type is missing for a patient, that image channel(s) is filled with 0.\n![](https://drive.google.com/file/d/1ZhzxfxWtIsSoiEXkDj61HdQIEBufJnQS/view?usp=sharing)\n# Model\n![](https://drive.google.com/file/d/1ovsY6s9TlhlORjYl332B-34hEZy7gVY5/view?usp=sharing)\nThe chosen CNN model for image features extraction task is a pre-trained EfficientNet B0 model. Since the input image has 4 channels, each corresponding to an MRI image type, a 2D convolution is applied to map the 4-channel image into a 3-channel feature map to fit the input shape of the pre-trained EfficientNet model. The classification head of the pre-trained model is also replaced by a fully-connected layer of size 256.\nAfter obtaining the embeddings from all 10 selected frames, the embeddings are passed to 2 LSTM layers with hidden size=32 for both layers. Following that is a prediction layer with 1 node outputs the score.\nThe model is trained for 15 epochs using binary cross-entropy loss, Adam optimizer with learning rate = 1e-4.\n# Preprocessing and augmentation\nAll input images are normalized and resized to the size of (256, 256). Images from one MRI type from one patient are randomly augmented with the same parameters.\nThe list of augmentations are as follow:\n- Horizontal Flip\n- ShiftScaleRotate\n- RandomBrightnessContrast\n\nI found that using too many and/or heavy augmentation methods did not help with the training process, therefore I only used simple augmentations when prepared the training data.\n# Cross validation.\nStratified K-fold cross validation is used with K=5 on MGMT value. During inference time, the mean prediction value from all 5 models is used as the ensemble’s prediction value.\n# Notebooks\nAll training and inference steps were done with Kaggle notebooks with GPUs:\n- Training notebook: https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-train/notebook\n- Inference notbook: https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-inference\n- Refactored Github [link](https://github.com/minhnhatphan/rnsa21-cnn-lstm) and notebook [link](https://www.kaggle.com/minhnhatphan/rnsa21-cnn-lstm-refactored/notebook)\n\nI agree with others competitors that the results are not clinically useful. I found myself lucky to be on the top positions in the challenge. However, I think everyone can get something out of this, and the knowledge from this competition can benefit future competition as well as our understandings of similar problems. Again, I'd like to thank all participants and the Kaggle community for your insights and knowledge.",
      "votes": null
    },
    {
      "id": "1550923",
      "postDate": "10/20/2021 06:43:23",
      "content": "<p>Congratulations!</p>",
      "rawMarkdown": "Congratulations!",
      "votes": null
    },
    {
      "id": "1550946",
      "postDate": "10/20/2021 07:05:41",
      "content": "<p>Thank you!</p>",
      "rawMarkdown": "Thank you!",
      "votes": null
    },
    {
      "id": "1550959",
      "postDate": "10/20/2021 07:21:15",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/minhnhatphan\" target=\"_blank\">@minhnhatphan</a>, congratulations for the second place; well done! 🎉 I'm glad the dataset \"RSNA to PNG\" has been useful. All the best for your future achievements!</p>",
      "rawMarkdown": "Hey @minhnhatphan, congratulations for the second place; well done! 🎉 I'm glad the dataset \"RSNA to PNG\" has been useful. All the best for your future achievements!",
      "votes": null
    },
    {
      "id": "1550978",
      "postDate": "10/20/2021 07:34:26",
      "content": "<p>Congratulations!</p>",
      "rawMarkdown": "Congratulations!",
      "votes": null
    },
    {
      "id": "1551005",
      "postDate": "10/20/2021 07:57:33",
      "content": "<p>Thank you so much! Your dataset helped me out a lot 🔥 Best wishes for you!</p>",
      "rawMarkdown": "Thank you so much! Your dataset helped me out a lot 🔥 Best wishes for you!",
      "votes": null
    },
    {
      "id": "1551006",
      "postDate": "10/20/2021 07:57:41",
      "content": "<p>Thank you!</p>",
      "rawMarkdown": "Thank you!",
      "votes": null
    },
    {
      "id": "1551342",
      "postDate": "10/20/2021 14:21:32",
      "content": "<p>Congratulations! Thanks for the detailed explanation of the training pipeline. In the process of training my models for this competition, I constantly came across the fact that my validation loss either became greater than 0.69, or kept within these values, I see the same trend in your notebook. Don't you think that the model has not learned to properly determine MGMT and the results of all of us on the private liderboard is largely random?</p>",
      "rawMarkdown": "Congratulations! Thanks for the detailed explanation of the training pipeline. In the process of training my models for this competition, I constantly came across the fact that my validation loss either became greater than 0.69, or kept within these values, I see the same trend in your notebook. Don't you think that the model has not learned to properly determine MGMT and the results of all of us on the private liderboard is largely random?",
      "votes": null
    },
    {
      "id": "1551404",
      "postDate": "10/20/2021 15:32:54",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/ivansps\" target=\"_blank\">@ivansps</a>! At least on my part, I got lucky when training the model. I recently ran each of my 5-fold models individually, only 2/5 models has 'good' scores, and one is really close to my private LB, the others are around 0.5 private AUC. <br>\nPersonally I think our models did learn something, but it did not outshine the noise of the given data, which made us not convinced of all of our results. I am truly sorry to all the teams who spent months in this competition but did not their expected results.</p>",
      "rawMarkdown": "Thank you @ivansps! At least on my part, I got lucky when training the model. I recently ran each of my 5-fold models individually, only 2/5 models has 'good' scores, and one is really close to my private LB, the others are around 0.5 private AUC. \nPersonally I think our models did learn something, but it did not outshine the noise of the given data, which made us not convinced of all of our results. I am truly sorry to all the teams who spent months in this competition but did not their expected results.",
      "votes": null
    },
    {
      "id": "1551408",
      "postDate": "10/20/2021 15:36:34",
      "content": "<p>Congratulations. </p>\n<p>My team used the CNN+RNN scheme too.</p>\n<p>I have a question. The sequences are oriented differently in each study. Did your preprocessing pipeline try to orient all the images from the sequences in the same study before you concatenate them into a four-channels feature images? And if you had done that, was there also an attempt to co-register the images from each sequence so the Tumor lines up in roughly the same location?</p>",
      "rawMarkdown": "Congratulations. \n\nMy team used the CNN+RNN scheme too.\n\nI have a question. The sequences are oriented differently in each study. Did your preprocessing pipeline try to orient all the images from the sequences in the same study before you concatenate them into a four-channels feature images? And if you had done that, was there also an attempt to co-register the images from each sequence so the Tumor lines up in roughly the same location?",
      "votes": null
    },
    {
      "id": "1551456",
      "postDate": "10/20/2021 16:24:01",
      "content": "<p>Congratulations!</p>",
      "rawMarkdown": "Congratulations!",
      "votes": null
    },
    {
      "id": "1551923",
      "postDate": "10/21/2021 02:09:54",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/yeeseng\" target=\"_blank\">@yeeseng</a>. I have seen ideas that people orient sequences from all studies to be on the same axis before training and inferencing, which makes a lot of sense in my opinion. Shamefully I didn't do it in this pipeline. </p>",
      "rawMarkdown": "Thank you @yeeseng. I have seen ideas that people orient sequences from all studies to be on the same axis before training and inferencing, which makes a lot of sense in my opinion. Shamefully I didn't do it in this pipeline.",
      "votes": null
    },
    {
      "id": "1551924",
      "postDate": "10/21/2021 02:10:36",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/a11t01n3\" target=\"_blank\">@a11t01n3</a>!</p>",
      "rawMarkdown": "Thank you @a11t01n3!",
      "votes": null
    },
    {
      "id": "1553337",
      "postDate": "10/22/2021 05:12:48",
      "content": "<p>I am curious about what the CNNs are actually focussing on.<br>\nCan you please share a few GradCAM activation maps from your CNN?</p>",
      "rawMarkdown": "I am curious about what the CNNs are actually focussing on.\nCan you please share a few GradCAM activation maps from your CNN?",
      "votes": null
    },
    {
      "id": "1553477",
      "postDate": "10/22/2021 07:58:42",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/pranshu15\" target=\"_blank\">@pranshu15</a>! I'm sorry, I don't have code to generate Grad-CAM maps since I don't have experience in generating activation maps yet</p>",
      "rawMarkdown": "Hi @pranshu15! I'm sorry, I don't have code to generate Grad-CAM maps since I don't have experience in generating activation maps yet",
      "votes": null
    },
    {
      "id": "1554190",
      "postDate": "10/22/2021 22:05:15",
      "content": "<p>Thanks!</p>\n<p>We did try to orient and align the different sequences but I have the feeling that too much processing might actually cause data loss and become counterproductive.</p>\n<p>Again, congrats.</p>",
      "rawMarkdown": "Thanks!\n\nWe did try to orient and align the different sequences but I have the feeling that too much processing might actually cause data loss and become counterproductive.\n\nAgain, congrats.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1550923,
      "author_name": "atsunorifujita",
      "author_url": "",
      "post_date": "10/20/2021 06:43:23",
      "content": "<p>Congratulations!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1550946,
          "author_name": "minhnhatphan",
          "author_url": "",
          "post_date": "10/20/2021 07:05:41",
          "content": "<p>Thank you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1550959,
      "author_name": "jonathanbesomi",
      "author_url": "",
      "post_date": "10/20/2021 07:21:15",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/minhnhatphan\" target=\"_blank\">@minhnhatphan</a>, congratulations for the second place; well done! 🎉 I'm glad the dataset \"RSNA to PNG\" has been useful. All the best for your future achievements!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1551005,
          "author_name": "minhnhatphan",
          "author_url": "",
          "post_date": "10/20/2021 07:57:33",
          "content": "<p>Thank you so much! Your dataset helped me out a lot 🔥 Best wishes for you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1550978,
      "author_name": "nvnnghia",
      "author_url": "",
      "post_date": "10/20/2021 07:34:26",
      "content": "<p>Congratulations!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1551006,
          "author_name": "minhnhatphan",
          "author_url": "",
          "post_date": "10/20/2021 07:57:41",
          "content": "<p>Thank you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1551342,
      "author_name": "ivansps",
      "author_url": "",
      "post_date": "10/20/2021 14:21:32",
      "content": "<p>Congratulations! Thanks for the detailed explanation of the training pipeline. In the process of training my models for this competition, I constantly came across the fact that my validation loss either became greater than 0.69, or kept within these values, I see the same trend in your notebook. Don't you think that the model has not learned to properly determine MGMT and the results of all of us on the private liderboard is largely random?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1551404,
          "author_name": "minhnhatphan",
          "author_url": "",
          "post_date": "10/20/2021 15:32:54",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/ivansps\" target=\"_blank\">@ivansps</a>! At least on my part, I got lucky when training the model. I recently ran each of my 5-fold models individually, only 2/5 models has 'good' scores, and one is really close to my private LB, the others are around 0.5 private AUC. <br>\nPersonally I think our models did learn something, but it did not outshine the noise of the given data, which made us not convinced of all of our results. I am truly sorry to all the teams who spent months in this competition but did not their expected results.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1551408,
      "author_name": "yeeseng",
      "author_url": "",
      "post_date": "10/20/2021 15:36:34",
      "content": "<p>Congratulations. </p>\n<p>My team used the CNN+RNN scheme too.</p>\n<p>I have a question. The sequences are oriented differently in each study. Did your preprocessing pipeline try to orient all the images from the sequences in the same study before you concatenate them into a four-channels feature images? And if you had done that, was there also an attempt to co-register the images from each sequence so the Tumor lines up in roughly the same location?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1551923,
          "author_name": "minhnhatphan",
          "author_url": "",
          "post_date": "10/21/2021 02:09:54",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/yeeseng\" target=\"_blank\">@yeeseng</a>. I have seen ideas that people orient sequences from all studies to be on the same axis before training and inferencing, which makes a lot of sense in my opinion. Shamefully I didn't do it in this pipeline. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1554190,
          "author_name": "yeeseng",
          "author_url": "",
          "post_date": "10/22/2021 22:05:15",
          "content": "<p>Thanks!</p>\n<p>We did try to orient and align the different sequences but I have the feeling that too much processing might actually cause data loss and become counterproductive.</p>\n<p>Again, congrats.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1551456,
      "author_name": "a11t01n3",
      "author_url": "",
      "post_date": "10/20/2021 16:24:01",
      "content": "<p>Congratulations!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1551924,
          "author_name": "minhnhatphan",
          "author_url": "",
          "post_date": "10/21/2021 02:10:36",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/a11t01n3\" target=\"_blank\">@a11t01n3</a>!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1553337,
      "author_name": "pranshu15",
      "author_url": "",
      "post_date": "10/22/2021 05:12:48",
      "content": "<p>I am curious about what the CNNs are actually focussing on.<br>\nCan you please share a few GradCAM activation maps from your CNN?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1553477,
          "author_name": "minhnhatphan",
          "author_url": "",
          "post_date": "10/22/2021 07:58:42",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/pranshu15\" target=\"_blank\">@pranshu15</a>! I'm sorry, I don't have code to generate Grad-CAM maps since I don't have experience in generating activation maps yet</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1550845": "First of all, I would like to thank Kaggle and the competition host for hosting this challenge, and also to all competitors who participated in this challenge. My main reason for joining this competition was to learn from the community, and I sure did learned a lot from you. \nThe second place came as a surprise to me and I was not expecting it at all, since my public LB score was not high (0.65-0.67)\nAnyway, I will share my approach to this competition. My approach is simple, using a CNN-LSTM architecture to do Classification task. For the CNN part I used Efficientnet and I trained LSTM part from scratch.\nAll 4 types of MRI image sequences (FLAIR, T1w, T1wCE, T2w) are used as inputs. \n# Generate fused MRI sequences\n- I obtained images from @jonathanbesomi 's dataset https://www.kaggle.com/jonathanbesomi/rsna-miccai-png, who converted all images into PNG format and removed empty DICOM images\n- For each MRI image sequence, a number of T=10 frames is selected using uniform temporal subsampling. For example, with a video containing 91 frames, the frames 1, 11, 21, …, 91 are selected with this sampling strategy. \n```\ndef uniform_temporal_subsample(x, num_samples):\n    '''\n        Moddified from https://github.com/facebookresearch/pytorchvideo/blob/d7874f788bc00a\n7badfb4310a912f6e531ffd6d3/pytorchvideo/transforms/functional.py#L19\n    '''\n    t = len(x)\n    indices = torch.linspace(0, t - 1, num_samples)\n    indices = torch.clamp(indices, 0, t - 1).long()\n    return [x[i] for i in indices]\n```\n- In each selected time frame, 4 single-channel MRI images are concatenated to one 4-channels feature image before proceeding to the training/inference phase. If one (or more) MRI image type is missing for a patient, that image channel(s) is filled with 0.\n![](https://drive.google.com/file/d/1ZhzxfxWtIsSoiEXkDj61HdQIEBufJnQS/view?usp=sharing)\n# Model\n![](https://drive.google.com/file/d/1ovsY6s9TlhlORjYl332B-34hEZy7gVY5/view?usp=sharing)\nThe chosen CNN model for image features extraction task is a pre-trained EfficientNet B0 model. Since the input image has 4 channels, each corresponding to an MRI image type, a 2D convolution is applied to map the 4-channel image into a 3-channel feature map to fit the input shape of the pre-trained EfficientNet model. The classification head of the pre-trained model is also replaced by a fully-connected layer of size 256.\nAfter obtaining the embeddings from all 10 selected frames, the embeddings are passed to 2 LSTM layers with hidden size=32 for both layers. Following that is a prediction layer with 1 node outputs the score.\nThe model is trained for 15 epochs using binary cross-entropy loss, Adam optimizer with learning rate = 1e-4.\n# Preprocessing and augmentation\nAll input images are normalized and resized to the size of (256, 256). Images from one MRI type from one patient are randomly augmented with the same parameters.\nThe list of augmentations are as follow:\n- Horizontal Flip\n- ShiftScaleRotate\n- RandomBrightnessContrast\n\nI found that using too many and/or heavy augmentation methods did not help with the training process, therefore I only used simple augmentations when prepared the training data.\n# Cross validation.\nStratified K-fold cross validation is used with K=5 on MGMT value. During inference time, the mean prediction value from all 5 models is used as the ensemble’s prediction value.\n# Notebooks\nAll training and inference steps were done with Kaggle notebooks with GPUs:\n- Training notebook: https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-train/notebook\n- Inference notbook: https://www.kaggle.com/minhnhatphan/rnsa-21-cnn-lstm-inference\n- Refactored Github [link](https://github.com/minhnhatphan/rnsa21-cnn-lstm) and notebook [link](https://www.kaggle.com/minhnhatphan/rnsa21-cnn-lstm-refactored/notebook)\n\nI agree with others competitors that the results are not clinically useful. I found myself lucky to be on the top positions in the challenge. However, I think everyone can get something out of this, and the knowledge from this competition can benefit future competition as well as our understandings of similar problems. Again, I'd like to thank all participants and the Kaggle community for your insights and knowledge.",
    "1550923": "Congratulations!",
    "1550946": "Thank you!",
    "1550959": "Hey @minhnhatphan, congratulations for the second place; well done! 🎉 I'm glad the dataset \"RSNA to PNG\" has been useful. All the best for your future achievements!",
    "1550978": "Congratulations!",
    "1551005": "Thank you so much! Your dataset helped me out a lot 🔥 Best wishes for you!",
    "1551006": "Thank you!",
    "1551342": "Congratulations! Thanks for the detailed explanation of the training pipeline. In the process of training my models for this competition, I constantly came across the fact that my validation loss either became greater than 0.69, or kept within these values, I see the same trend in your notebook. Don't you think that the model has not learned to properly determine MGMT and the results of all of us on the private liderboard is largely random?",
    "1551404": "Thank you @ivansps! At least on my part, I got lucky when training the model. I recently ran each of my 5-fold models individually, only 2/5 models has 'good' scores, and one is really close to my private LB, the others are around 0.5 private AUC. \nPersonally I think our models did learn something, but it did not outshine the noise of the given data, which made us not convinced of all of our results. I am truly sorry to all the teams who spent months in this competition but did not their expected results.",
    "1551408": "Congratulations. \n\nMy team used the CNN+RNN scheme too.\n\nI have a question. The sequences are oriented differently in each study. Did your preprocessing pipeline try to orient all the images from the sequences in the same study before you concatenate them into a four-channels feature images? And if you had done that, was there also an attempt to co-register the images from each sequence so the Tumor lines up in roughly the same location?",
    "1551456": "Congratulations!",
    "1551923": "Thank you @yeeseng. I have seen ideas that people orient sequences from all studies to be on the same axis before training and inferencing, which makes a lot of sense in my opinion. Shamefully I didn't do it in this pipeline.",
    "1551924": "Thank you @a11t01n3!",
    "1553337": "I am curious about what the CNNs are actually focussing on.\nCan you please share a few GradCAM activation maps from your CNN?",
    "1553477": "Hi @pranshu15! I'm sorry, I don't have code to generate Grad-CAM maps since I don't have experience in generating activation maps yet",
    "1554190": "Thanks!\n\nWe did try to orient and align the different sequences but I have the feeling that too much processing might actually cause data loss and become counterproductive.\n\nAgain, congrats."
  },
  "source": "meta"
}