![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8251891%2Fe5a2fef330678d164bc8ff54d06c3e34%2F2024-02-07%2021-58-54.png?generation=1707310804960891&alt=media\")
The key factors in enhancing the model's robust performance included Tversky loss, increased inference size, and resolution augmentation. These elements ultimately played a significant role in the model's survival during shakeups. The final model is an ensemble composed of 2D U-Net models based on the RegNetY-016 architecture
\nIt was a truly challenging competition to secure a reliable validation set (which, unfortunately, I couldn't achieve). Upon reviewing the results, I discovered a significant difference between the public and private sets. Surprisingly, there were submissions from the past that would have made it into the gold zone. It's astonishing, considering I didn't think much of those submissions and didn't end up submitting them anyway.
In my experiments, I found that giving a higher weight to the positive class in the Tversky loss (with a larger beta) resulted in better performance on the private leaderboard. However, for my final submission, where I trained with a smaller beta value in the Tversky loss, the model performed well on the public leaderboard but surprisingly poorly on the private leaderboard. This discrepancy may be attributed to the higher resolution of the private dataset, which likely contained more detailed input information. Consequently, it suggests that a lower threshold for the model logits might have been more appropriate in this context. Furthermore, scaling the image (1.2, 1.5, …) during inference seemed to dilute input information, reducing the impact of resolution.
\nThree consecutive slices were used as input for the model, and training was performed with the original image size. Despite experimenting with an increased number of slices (5, 7, 9…), the results revealed a decrease in performance.
\nIn the early stages of the competition, I trained the model by resizing the data to a specific size. However, as I mentioned in the discussion, the Resize function in the Albumentations library applies nearest interpolation to masks, causing significant noise in fine labels and resulting in a notable decrease in performance. Therefore, I opted to use the original image size.
\nAwaring of the different resolutions between public and private data, I aimed to create a model robust to resolution variations. Additionally, understanding the existence of resizing during the binning process, I employed blur augmentation.
\ndef blur_augmentation(x):\n h, w, = x.shape\n = ..uniform(, )\n\n x = A.Resize(int(h*), int(w*))(=x)['']\n x = A.Resize(h, w)(=x)['']\n x\nMoreover, since the channels were constructed by stacking depth, I applied the following augmentations.
\ndef channel_augmentation(x, =0.5, =3):\n assert x.shape[2]==n_channel\n np.random.rand()<prob:\n x = np.flip(x, =2)\n return x\nFinally, given the inherent noise in the data annotations, I implemented strong cutout augmentation to prevent overfitting.
\nA.Cutout(=8, =128, =128, =0.8),\nThese approaches effectively contributed to performance improvement in both CV and LB.
\nThe model, like most others, employed a 2D U-Net architecture. For the CNN backbone, I utilized the lightweight model RegNetY-016. Other than that, the settings remained consistent with the default values in SMP (Segmentation Models PyTorch).
\nDespite investing significant time in developing a 3D-based model, it failed to demonstrate notable score improvements in both CV and LB. Due to resource constraints, I shifted my focus to a 2D model.
\nclass CustomModel(nn.Module):\n def __init__(self):\n super(CustomModel, self).__init__()\n\n self.n_classes = 1\n self.in_chans = 3\n\n self.encoder = timm.create_model(\n ,\n =,\n =,\n =self.in_chans,\n )\n encoder_channels = tuple(\n [self.in_chans]\n + [\n self.encoder.feature_info[i][]\n i range(len(self.encoder.feature_info))\n ]\n )\n self.decoder = UnetDecoder(\n =encoder_channels,\n decoder_channels=(256, 128, 64, 32, 16),\n =5,\n =,\n =,\n =None,\n )\n\n self.segmentation_head = SegmentationHead(\n =16,\n =self.n_classes,\n =None,\n =3,\n )\n\n self.train_loss = smp.losses.TverskyLoss(=, =0.1, =0.9)\n self.test_loss = smp.losses.DiceLoss(=)\n\n\n def forward(self, batch, =):\n\n x_in = batch[]\n\n enc_out = self.encoder(x_in)\n\n decoder_out = self.decoder(*[x_in] + enc_out)\n x_seg = self.segmentation_head(decoder_out)\n\n output = {}\n one_hot_mask = batch[][:, None]\n training:\n loss = self.train_loss(x_seg, one_hot_mask.float())\n :\n loss = self.test_loss(x_seg, one_hot_mask.float())\n\n output[] = loss\n output[] = nn.Sigmoid()(x_seg)[:, 0]\n\n return output\nScaling the image size by 1.5x during inference consistently resulted in score improvements in CV, public LB, and private LB. This acted as a form of dilation, significantly reducing false negatives and enhancing the model's performance. Simply increasing the image size by 1.2 for inference resulted in a 0.1 improvement on the private leaderboard.
\nhttps://github.com/siwooyong/SenNet-HOA-Hacking-the-Human-Vasculature-in-3D
", "messages": [ { "id": 2640657, "postDate": "2024-02-07T03:24:30.267Z", "content": "The key factors in enhancing the model's robust performance included Tversky loss, increased inference size, and resolution augmentation. These elements ultimately played a significant role in the model's survival during shakeups. The final model is an ensemble composed of 2D U-Net models based on the RegNetY-016 architecture
\nIt was a truly challenging competition to secure a reliable validation set (which, unfortunately, I couldn't achieve). Upon reviewing the results, I discovered a significant difference between the public and private sets. Surprisingly, there were submissions from the past that would have made it into the gold zone. It's astonishing, considering I didn't think much of those submissions and didn't end up submitting them anyway.
In my experiments, I found that giving a higher weight to the positive class in the Tversky loss (with a larger beta) resulted in better performance on the private leaderboard. However, for my final submission, where I trained with a smaller beta value in the Tversky loss, the model performed well on the public leaderboard but surprisingly poorly on the private leaderboard. This discrepancy may be attributed to the higher resolution of the private dataset, which likely contained more detailed input information. Consequently, it suggests that a lower threshold for the model logits might have been more appropriate in this context. Furthermore, scaling the image (1.2, 1.5, …) during inference seemed to dilute input information, reducing the impact of resolution.
\nThree consecutive slices were used as input for the model, and training was performed with the original image size. Despite experimenting with an increased number of slices (5, 7, 9…), the results revealed a decrease in performance.
\nIn the early stages of the competition, I trained the model by resizing the data to a specific size. However, as I mentioned in the discussion, the Resize function in the Albumentations library applies nearest interpolation to masks, causing significant noise in fine labels and resulting in a notable decrease in performance. Therefore, I opted to use the original image size.
\nAwaring of the different resolutions between public and private data, I aimed to create a model robust to resolution variations. Additionally, understanding the existence of resizing during the binning process, I employed blur augmentation.
\ndef blur_augmentation(x):\n h, w, = x.shape\n = ..uniform(, )\n\n x = A.Resize(int(h*), int(w*))(=x)['']\n x = A.Resize(h, w)(=x)['']\n x\nMoreover, since the channels were constructed by stacking depth, I applied the following augmentations.
\ndef channel_augmentation(x, =0.5, =3):\n assert x.shape[2]==n_channel\n np.random.rand()<prob:\n x = np.flip(x, =2)\n return x\nFinally, given the inherent noise in the data annotations, I implemented strong cutout augmentation to prevent overfitting.
\nA.Cutout(=8, =128, =128, =0.8),\nThese approaches effectively contributed to performance improvement in both CV and LB.
\nThe model, like most others, employed a 2D U-Net architecture. For the CNN backbone, I utilized the lightweight model RegNetY-016. Other than that, the settings remained consistent with the default values in SMP (Segmentation Models PyTorch).
\nDespite investing significant time in developing a 3D-based model, it failed to demonstrate notable score improvements in both CV and LB. Due to resource constraints, I shifted my focus to a 2D model.
\nclass CustomModel(nn.Module):\n def __init__(self):\n super(CustomModel, self).__init__()\n\n self.n_classes = 1\n self.in_chans = 3\n\n self.encoder = timm.create_model(\n ,\n =,\n =,\n =self.in_chans,\n )\n encoder_channels = tuple(\n [self.in_chans]\n + [\n self.encoder.feature_info[i][]\n i range(len(self.encoder.feature_info))\n ]\n )\n self.decoder = UnetDecoder(\n =encoder_channels,\n decoder_channels=(256, 128, 64, 32, 16),\n =5,\n =,\n =,\n =None,\n )\n\n self.segmentation_head = SegmentationHead(\n =16,\n =self.n_classes,\n =None,\n =3,\n )\n\n self.train_loss = smp.losses.TverskyLoss(=, =0.1, =0.9)\n self.test_loss = smp.losses.DiceLoss(=)\n\n\n def forward(self, batch, =):\n\n x_in = batch[]\n\n enc_out = self.encoder(x_in)\n\n decoder_out = self.decoder(*[x_in] + enc_out)\n x_seg = self.segmentation_head(decoder_out)\n\n output = {}\n one_hot_mask = batch[][:, None]\n training:\n loss = self.train_loss(x_seg, one_hot_mask.float())\n :\n loss = self.test_loss(x_seg, one_hot_mask.float())\n\n output[] = loss\n output[] = nn.Sigmoid()(x_seg)[:, 0]\n\n return output\nScaling the image size by 1.5x during inference consistently resulted in score improvements in CV, public LB, and private LB. This acted as a form of dilation, significantly reducing false negatives and enhancing the model's performance. Simply increasing the image size by 1.2 for inference resulted in a 0.1 improvement on the private leaderboard.
\nhttps://github.com/siwooyong/SenNet-HOA-Hacking-the-Human-Vasculature-in-3D
", "rawMarkdown": "# TLDR\nThe key factors in enhancing the model's robust performance included **Tversky loss**, **increased inference size**, and **resolution augmentation**. These elements ultimately played a significant role in the model's survival during shakeups. The final model is an ensemble composed of 2D U-Net models based on the RegNetY-016 architecture\n\n# Interesting Point\nIt was a truly challenging competition to secure a reliable validation set (which, unfortunately, I couldn't achieve). Upon reviewing the results, I discovered a significant difference between the public and private sets. Surprisingly, there were submissions from the past that would have made it into the gold zone. It's astonishing, considering I didn't think much of those submissions and didn't end up submitting them anyway.\n\nIn my experiments, I found that giving a higher weight to the positive class in the Tversky loss (with a larger beta) resulted in better performance on the private leaderboard. However, for my final submission, where I trained with a smaller beta value in the Tversky loss, the model performed well on the public leaderboard but surprisingly poorly on the private leaderboard. **This discrepancy may be attributed to the higher resolution of the private dataset, which likely contained more detailed input information. Consequently, it suggests that a lower threshold for the model logits might have been more appropriate in this context. Furthermore, scaling the image (1.2, 1.5, ...) during inference seemed to dilute input information, reducing the impact of resolution.**\n\n# Data Processing\nThree consecutive slices were used as input for the model, and training was performed with the original image size. Despite experimenting with an increased number of slices (5, 7, 9...), the results revealed a decrease in performance.\n\nIn the early stages of the competition, I trained the model by resizing the data to a specific size. However, as I mentioned in the [discussion](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/463121), the Resize function in the Albumentations library applies nearest interpolation to masks, causing significant noise in fine labels and resulting in a notable decrease in performance. Therefore, I opted to use the original image size.\n\n# Augmentation\nAwaring of the different resolutions between public and private data, I aimed to create a model robust to resolution variations. Additionally, understanding the existence of resizing during the binning process, I employed blur augmentation. \n\n def blur_augmentation(x):\n h, w, _ = x.shape\n scale = np.random.uniform(0.5, 1.5)\n\n x = A.Resize(int(h*scale), int(w*scale))(image=x)['image']\n x = A.Resize(h, w)(image=x)['image']\n return x\n\nMoreover, since the channels were constructed by stacking depth, I applied the following augmentations.\n\n def channel_augmentation(x, prob=0.5, n_channel=3):\n assert x.shape[2]==n_channel\n if np.random.rand()