As we all know, if we train on imagenet weights, we may take quite a while to converge, even if we finetune it. The intuition is simple, imagenet were trained on many common items in life, and none of them resemble closely to the image structures of X-rays, therefore, the model may have a hard time detecting shapes and details from the X-rays. We can of course unfreeze all the layers and retrain them from scratch, using the State of the Art models' backbone, however, due to limited hardware, we decided it is best to use what others have trained. After all, it is much easier to stand on the shoulder of giants like ammarali. Consequently, I conveniently used a set of pretrained weights trained specifically on this dataset as a starting point. The weights and ideas can be found here.
\nWe used a few models and found out that resnet200d has the best results on this set of training images. There is no deep reason why this model outperforms other State of the Art models, but using gradcam we can see how the model sees the images.
Knowing how to choose a robust and leak free cross validation strategy is extremely important. Data leakage can cause you to have blind confidence on your model. We are also guilty of committing one since we trained our models with the NiH pretrained weights, without taking into consideration if the weights overlap with the training and validation folds information.
\nsklearn :We built-upon Tawara’s Multi-head model for our best scoring models. In particular, we experimented with the activation functions and dropout rates. We found models with Swish activation in the multi-head component of the network to perform best in our experiments. Our best scoring single model is a multi-head model with a resnet200d backbone. In particular, one single fold of resnet200d gives a private score of 0.970.
\nWe started experimenting with sin's pipeline which is similar to qishen ha's pipeline back in Melanoma and used Tawara's multihead approach. We did not have time to experiment with the 3-4 stage training as we joined the competition late.
ResNet200D and SeResNet152dRueben Schmidt mentioned in this post here that some images have black borders around them. I removed them during both the training and inference process. There was no significant increase on the LB score, even if there was, it is in the 3-4th decimal places, but I noticed my local cv to increase. Thus I decided to remove for all. After all, if I keep this consistent in both training and inference, I reckon that no surprise factor would pop out.
\nIn particular, we made use of a different Normalization parameter which is more accustomed to the X-ray pretrained images. Thanks Tawara again! Heavy augmentations are used during Train-Time-Augmentation. But during Test-Time-Augmentation, we merely used a HorizontalFlip with 100% probability, and only used tta_steps=1.
augmentations_class: AlbumentationsAugmentation\naugmentations_train:\n AlbumentationsAugmentation:\n - name: RandomResizedCrop\n params:\n height: 640\n width: 640\n scale: [0.9, 1.0]\n p: 1.0\n - name: HorizontalFlip\n params:\n p: 0.5\n - name: ShiftScaleRotate\n params:\n shift_limit: 0.2\n scale_limit: 0.2\n rotate_limit: 20\n border_mode: 0\n value: 0\n mask_value: 0\n p: 0.5\n - name: HueSaturationValue\n params:\n hue_shift_limit: 10\n sat_shift_limit: 10\n val_shift_limit: 10\n p: 0.7\n - name: RandomBrightnessContrast\n params:\n brightness_limit: [-0.2, 0.2]\n contrast_limit: [-0.2, 0.2]\n p: 0.7\n - name: CLAHE\n params:\n clip_limit: [1,4]\n p: 0.5\n - name: JpegCompression\n params:\n p: 0.2\n - name: IAAPiecewiseAffine\n params:\n p: 0.2\n - name: IAASharpen\n params:\n p: 0.2\n - name: Cutout\n params: \n # use int(image_size * 0.1)\n max_h_size: 64\n max_w_size: 64\n num_holes: 5\n p: 0.5\n - name: Resize\n params:\n height: 640\n width: 640\n p: 1.0\n - name: Normalize\n params:\n mean: [0.4887381077884414]\n std: [0.23064819430546407]\n p: 1.0\n - name: ToTensorV2\n params:\n p: 1.0\naugmentations_val:\n AlbumentationsAugmentation:\n - name: Resize\n params:\n height: 640\n width: 640\n p: 1.0\n - name: Normalize\n params:\n mean: [0.4887381077884414]\n std: [0.23064819430546407]\n p: 1.0\n - name: ToTensorV2\n params:\n p: 1.0\nDue to hardware limitation, we can barely fit in anything more than a batch_size of 8. We quote the well known fact here:
\n\nIt has been observed in practice that when using a larger batch there is a degradation in the quality of the model, as measured by its ability to generalize […]
\n
\n large-batch methods tend to converge to sharp minimizers of the training and testing functions—and as is well known, sharp minima lead to poorer generalization. In contrast, small-batch methods consistently converge to flat minimizers, and our experiments support a commonly held view that this is due to the inherent noise in the gradient estimation.
\n The above shows that large batch size mayfitthe model too well, as the model will learn features of the dataset in less iterations, and may memorize this particular dataset's features, leading to overfitting and poor generalization. However, too small a batch size causes our convergence to go too slow, empirically, we take 32 or 64 as the ideal batch size in this competition.We used both
\ntorch.ampandgradient accumulationto be able to fit more batch sizes. We did not freeze thebatch_normlayers, which still yielded great results. What we should have done is to experiment more on how to freeze the batch norm layers properly, as I believe that it may help. In the end, we used a batch size of 8 and fit 4 iterations usinggradient accumulationand trained a total number of 20 epochs to get a local CV score of roughly 0.969.Optimizer, Scheduler and Loss
\nNothing too fancy here, although we really wanted to try out
\nFocal Lossin this setting. The configuration can be seen here. But note that we incorporatedGradualWarmUpScheduleralong withCosineAnnealingLR.
scheduler: CosineAnnealingLR\nscheduler_params: # Note that in params we must put 1.e instead of 1e\n CosineAnnealingLR:\n T_max: 16\n eta_min: 1.e-7\n last_epoch: -1\n verbose: True \ntrain_step_scheduler: False\nval_step_scheduler: False\noptimizer: Adam\noptimizer_params:\n Adam:\n lr: 0.00002\n betas:\n - 0.9\n - 0.999\n eps: 1.e-7\n weight_decay: 0\n amsgrad: False\ncriterion_train: BCEWithLogitsLoss\ncriterion_val: BCEWithLogitsLoss\ncriterion_params:\n CrossEntropyLoss:\n weight: null\n size_average: null\n ignore_index: -100\n reduce: null\n reduction: mean\n LabelSmoothingLoss:\n classes: 2\n smoothing: 0.05\n dim: -1\nimport torch\nsigmoid = torch.nn.Sigmoid()\nclass Swish(torch.autograd.Function):\n @staticmethod\n def forward(ctx, i):\n result = i * sigmoid(i)\n ctx.save_for_backward(i)\n return result\n @staticmethod\n def backward(ctx, grad_output):\n i = ctx.saved_variables[0]\n sigmoid_i = sigmoid(i)\n return grad_output * (sigmoid_i * (1 + i * (1 - sigmoid_i)))\nclass Swish_Module(torch.nn.Module):\n def forward(self, x):\n return Swish.apply(x)\nOur second best performing model is also a multi-head model with swish activation in the heads, but with a SeResNet152d backbone (seresnet152d).
\nDuring training, we use gradient accumulation so that the bath size can scale up eventually to our desired sizes. For the second model, it scales to 16. The training parameters for our second best performing model above are:
image_size = 768\nseed = 42\nwarmup_epo = 1\ninit_lr = 1e-4\nbatch_size = 4\nvalid_batch_size = 4\nn_epochs = 30\nwarmup_factor = 10\nnum_workers = 4\niters_to_accumulate = 4\nuse_amp = True\ndebug = False\nearly_stop = 10\nWe also used the albumentations library to perform augmentations on the datasets. For the second model, the augmentations for the trainingand validation datasets are as follows:
transforms_train = albumentations.Compose(\n [\n albumentations.RandomResizedCrop(image_size, image_size, scale=(0.9, 1), p=1),\n albumentations.HorizontalFlip(p=0.5),\n albumentations.ShiftScaleRotate(p=0.5),\n albumentations.HueSaturationValue(\n hue_shift_limit=10, sat_shift_limit=10, val_shift_limit=10, p=0.7\n ),\n albumentations.RandomBrightnessContrast(\n brightness_limit=(-0.2, 0.2), contrast_limit=(-0.2, 0.2), p=0.7\n ),\n albumentations.CLAHE(clip_limit=(1, 4), p=0.5),\n albumentations.OneOf(\n [\n albumentations.OpticalDistortion(distort_limit=1.0),\n albumentations.GridDistortion(num_steps=5, distort_limit=1.0),\n albumentations.ElasticTransform(alpha=3),\n ],\n p=0.2,\n ),\n albumentations.OneOf(\n [\n albumentations.GaussNoise(var_limit=[10, 50]),\n albumentations.GaussianBlur(),\n albumentations.MotionBlur(),\n albumentations.MedianBlur(),\n ],\n p=0.2,\n ),\n albumentations.Resize(image_size, image_size),\n albumentations.OneOf(\n [\n JpegCompression(),\n Downscale(scale_min=0.1, scale_max=0.15),\n ],\n p=0.2,\n ),\n IAAPiecewiseAffine(p=0.2),\n IAASharpen(p=0.2),\n albumentations.Cutout(\n max_h_size=int(image_size * 0.1),\n max_w_size=int(image_size * 0.1),\n num_holes=5,\n p=0.5,\n ),\n albumentations.Normalize(),\n ]\n)\ntransforms_valid = albumentations.Compose(\n [albumentations.Resize(image_size, image_size), albumentations.Normalize()]\n)\nOur best submission to the competition comprised of a weighted (convex) ensemble of two models
\nMulti-Head ResNet200d andMulti-head SeResNet152dmultihead_resnet200d_fold0_best_loss.pth, multihead_resnet200d_fold1_best_AUC (1).pth, multihead_resnet200d_fold2_best_AUC.pth, multihead_resnet200d_fold3_best_AUC.pth, and multihead_resnet200d_fold4_best_AUC.pth. The weights for model (2) can be found here, and the 5-folds used in inference are grad_accum_multihead_seresnet152d_swish_fold0_best_AUC.pth, grad_accum_multihead_seresnet152d_swish_fold1_best_AUC.pth, grad_accum_multihead_seresnet152d_swish_fold2_best_AUC.pth, grad_accum_multihead_seresnet152d_swish_fold3_best_AUC.pth, and grad_accum_multihead_seresnet152d_swish_fold4_best_AUC.pth.What we could have done better:
\nclassifier head like GeM.backbone and WE JUST CANNOT MAKE efficietnet work. 😐rank_pct etc.As we all know, if we train on imagenet weights, we may take quite a while to converge, even if we finetune it. The intuition is simple, imagenet were trained on many common items in life, and none of them resemble closely to the image structures of X-rays, therefore, the model may have a hard time detecting shapes and details from the X-rays. We can of course unfreeze all the layers and retrain them from scratch, using the State of the Art models' backbone, however, due to limited hardware, we decided it is best to use what others have trained. After all, it is much easier to stand on the shoulder of giants like ammarali. Consequently, I conveniently used a set of pretrained weights trained specifically on this dataset as a starting point. The weights and ideas can be found here.
\nWe used a few models and found out that resnet200d has the best results on this set of training images. There is no deep reason why this model outperforms other State of the Art models, but using gradcam we can see how the model sees the images.
Knowing how to choose a robust and leak free cross validation strategy is extremely important. Data leakage can cause you to have blind confidence on your model. We are also guilty of committing one since we trained our models with the NiH pretrained weights, without taking into consideration if the weights overlap with the training and validation folds information.
\nsklearn :We built-upon Tawara’s Multi-head model for our best scoring models. In particular, we experimented with the activation functions and dropout rates. We found models with Swish activation in the multi-head component of the network to perform best in our experiments. Our best scoring single model is a multi-head model with a resnet200d backbone. In particular, one single fold of resnet200d gives a private score of 0.970.
\nWe started experimenting with sin's pipeline which is similar to qishen ha's pipeline back in Melanoma and used Tawara's multihead approach. We did not have time to experiment with the 3-4 stage training as we joined the competition late.
ResNet200D and SeResNet152dRueben Schmidt mentioned in this post here that some images have black borders around them. I removed them during both the training and inference process. There was no significant increase on the LB score, even if there was, it is in the 3-4th decimal places, but I noticed my local cv to increase. Thus I decided to remove for all. After all, if I keep this consistent in both training and inference, I reckon that no surprise factor would pop out.
\nIn particular, we made use of a different Normalization parameter which is more accustomed to the X-ray pretrained images. Thanks Tawara again! Heavy augmentations are used during Train-Time-Augmentation. But during Test-Time-Augmentation, we merely used a HorizontalFlip with 100% probability, and only used tta_steps=1.
augmentations_class: AlbumentationsAugmentation\naugmentations_train:\n AlbumentationsAugmentation:\n - name: RandomResizedCrop\n params:\n height: 640\n width: 640\n scale: [0.9, 1.0]\n p: 1.0\n - name: HorizontalFlip\n params:\n p: 0.5\n - name: ShiftScaleRotate\n params:\n shift_limit: 0.2\n scale_limit: 0.2\n rotate_limit: 20\n border_mode: 0\n value: 0\n mask_value: 0\n p: 0.5\n - name: HueSaturationValue\n params:\n hue_shift_limit: 10\n sat_shift_limit: 10\n val_shift_limit: 10\n p: 0.7\n - name: RandomBrightnessContrast\n params:\n brightness_limit: [-0.2, 0.2]\n contrast_limit: [-0.2, 0.2]\n p: 0.7\n - name: CLAHE\n params:\n clip_limit: [1,4]\n p: 0.5\n - name: JpegCompression\n params:\n p: 0.2\n - name: IAAPiecewiseAffine\n params:\n p: 0.2\n - name: IAASharpen\n params:\n p: 0.2\n - name: Cutout\n params: \n # use int(image_size * 0.1)\n max_h_size: 64\n max_w_size: 64\n num_holes: 5\n p: 0.5\n - name: Resize\n params:\n height: 640\n width: 640\n p: 1.0\n - name: Normalize\n params:\n mean: [0.4887381077884414]\n std: [0.23064819430546407]\n p: 1.0\n - name: ToTensorV2\n params:\n p: 1.0\naugmentations_val:\n AlbumentationsAugmentation:\n - name: Resize\n params:\n height: 640\n width: 640\n p: 1.0\n - name: Normalize\n params:\n mean: [0.4887381077884414]\n std: [0.23064819430546407]\n p: 1.0\n - name: ToTensorV2\n params:\n p: 1.0\nDue to hardware limitation, we can barely fit in anything more than a batch_size of 8. We quote the well known fact here:
\n\nIt has been observed in practice that when using a larger batch there is a degradation in the quality of the model, as measured by its ability to generalize […]
\n
\n large-batch methods tend to converge to sharp minimizers of the training and testing functions—and as is well known, sharp minima lead to poorer generalization. In contrast, small-batch methods consistently converge to flat minimizers, and our experiments support a commonly held view that this is due to the inherent noise in the gradient estimation.
\n The above shows that large batch size mayfitthe model too well, as the model will learn features of the dataset in less iterations, and may memorize this particular dataset's features, leading to overfitting and poor generalization. However, too small a batch size causes our convergence to go too slow, empirically, we take 32 or 64 as the ideal batch size in this competition.We used both
\ntorch.ampandgradient accumulationto be able to fit more batch sizes. We did not freeze thebatch_normlayers, which still yielded great results. What we should have done is to experiment more on how to freeze the batch norm layers properly, as I believe that it may help. In the end, we used a batch size of 8 and fit 4 iterations usinggradient accumulationand trained a total number of 20 epochs to get a local CV score of roughly 0.969.Optimizer, Scheduler and Loss
\nNothing too fancy here, although we really wanted to try out
\nFocal Lossin this setting. The configuration can be seen here. But note that we incorporatedGradualWarmUpScheduleralong withCosineAnnealingLR.
scheduler: CosineAnnealingLR\nscheduler_params: # Note that in params we must put 1.e instead of 1e\n CosineAnnealingLR:\n T_max: 16\n eta_min: 1.e-7\n last_epoch: -1\n verbose: True \ntrain_step_scheduler: False\nval_step_scheduler: False\noptimizer: Adam\noptimizer_params:\n Adam:\n lr: 0.00002\n betas:\n - 0.9\n - 0.999\n eps: 1.e-7\n weight_decay: 0\n amsgrad: False\ncriterion_train: BCEWithLogitsLoss\ncriterion_val: BCEWithLogitsLoss\ncriterion_params:\n CrossEntropyLoss:\n weight: null\n size_average: null\n ignore_index: -100\n reduce: null\n reduction: mean\n LabelSmoothingLoss:\n classes: 2\n smoothing: 0.05\n dim: -1\nimport torch\nsigmoid = torch.nn.Sigmoid()\nclass Swish(torch.autograd.Function):\n @staticmethod\n def forward(ctx, i):\n result = i * sigmoid(i)\n ctx.save_for_backward(i)\n return result\n @staticmethod\n def backward(ctx, grad_output):\n i = ctx.saved_variables[0]\n sigmoid_i = sigmoid(i)\n return grad_output * (sigmoid_i * (1 + i * (1 - sigmoid_i)))\nclass Swish_Module(torch.nn.Module):\n def forward(self, x):\n return Swish.apply(x)\nOur second best performing model is also a multi-head model with swish activation in the heads, but with a SeResNet152d backbone (seresnet152d).
\nDuring training, we use gradient accumulation so that the bath size can scale up eventually to our desired sizes. For the second model, it scales to 16. The training parameters for our second best performing model above are:
image_size = 768\nseed = 42\nwarmup_epo = 1\ninit_lr = 1e-4\nbatch_size = 4\nvalid_batch_size = 4\nn_epochs = 30\nwarmup_factor = 10\nnum_workers = 4\niters_to_accumulate = 4\nuse_amp = True\ndebug = False\nearly_stop = 10\nWe also used the albumentations library to perform augmentations on the datasets. For the second model, the augmentations for the trainingand validation datasets are as follows:
transforms_train = albumentations.Compose(\n [\n albumentations.RandomResizedCrop(image_size, image_size, scale=(0.9, 1), p=1),\n albumentations.HorizontalFlip(p=0.5),\n albumentations.ShiftScaleRotate(p=0.5),\n albumentations.HueSaturationValue(\n hue_shift_limit=10, sat_shift_limit=10, val_shift_limit=10, p=0.7\n ),\n albumentations.RandomBrightnessContrast(\n brightness_limit=(-0.2, 0.2), contrast_limit=(-0.2, 0.2), p=0.7\n ),\n albumentations.CLAHE(clip_limit=(1, 4), p=0.5),\n albumentations.OneOf(\n [\n albumentations.OpticalDistortion(distort_limit=1.0),\n albumentations.GridDistortion(num_steps=5, distort_limit=1.0),\n albumentations.ElasticTransform(alpha=3),\n ],\n p=0.2,\n ),\n albumentations.OneOf(\n [\n albumentations.GaussNoise(var_limit=[10, 50]),\n albumentations.GaussianBlur(),\n albumentations.MotionBlur(),\n albumentations.MedianBlur(),\n ],\n p=0.2,\n ),\n albumentations.Resize(image_size, image_size),\n albumentations.OneOf(\n [\n JpegCompression(),\n Downscale(scale_min=0.1, scale_max=0.15),\n ],\n p=0.2,\n ),\n IAAPiecewiseAffine(p=0.2),\n IAASharpen(p=0.2),\n albumentations.Cutout(\n max_h_size=int(image_size * 0.1),\n max_w_size=int(image_size * 0.1),\n num_holes=5,\n p=0.5,\n ),\n albumentations.Normalize(),\n ]\n)\ntransforms_valid = albumentations.Compose(\n [albumentations.Resize(image_size, image_size), albumentations.Normalize()]\n)\nOur best submission to the competition comprised of a weighted (convex) ensemble of two models
\nMulti-Head ResNet200d andMulti-head SeResNet152dmultihead_resnet200d_fold0_best_loss.pth, multihead_resnet200d_fold1_best_AUC (1).pth, multihead_resnet200d_fold2_best_AUC.pth, multihead_resnet200d_fold3_best_AUC.pth, and multihead_resnet200d_fold4_best_AUC.pth. The weights for model (2) can be found here, and the 5-folds used in inference are grad_accum_multihead_seresnet152d_swish_fold0_best_AUC.pth, grad_accum_multihead_seresnet152d_swish_fold1_best_AUC.pth, grad_accum_multihead_seresnet152d_swish_fold2_best_AUC.pth, grad_accum_multihead_seresnet152d_swish_fold3_best_AUC.pth, and grad_accum_multihead_seresnet152d_swish_fold4_best_AUC.pth.What we could have done better:
\nclassifier head like GeM.backbone and WE JUST CANNOT MAKE efficietnet work. 😐rank_pct etc.Congratulations. Thanks for great detailed description
", "rawMarkdown": "Congratulations. Thanks for great detailed description", "votes": null }, { "id": "1241964", "postDate": "03/17/2021 09:58:37", "content": "@ammarali32 Thanks, you are one of the giants whose shoulder we stood on :)
", "rawMarkdown": "ammarali32 Thanks, you are one of the giants whose shoulder we stood on :)", "votes": null }, { "id": "1241976", "postDate": "03/17/2021 10:07:43", "content": "Thanks, I am really glad to hear that and you are the giants who scored so well and got on the top 2%
", "rawMarkdown": "Thanks, I am really glad to hear that and you are the giants who scored so well and got on the top 2%", "votes": null }, { "id": "1242035", "postDate": "03/17/2021 11:00:38", "content": "@reighns Congratulations on Silver Finish Hongnan . Great and Detailed Solution Writeup
", "rawMarkdown": "reighns Congratulations on Silver Finish Hongnan . Great and Detailed Solution Writeup", "votes": null }, { "id": "1242058", "postDate": "03/17/2021 11:23:11", "content": "Congrats on your silver and thank you for the details!
", "rawMarkdown": "Congrats on your silver and thank you for the details!", "votes": null }, { "id": "1242115", "postDate": "03/17/2021 12:10:47", "content": "Thanks a lot! Congrats to you too! Been following you since Melanoma. Haha
", "rawMarkdown": "Thanks a lot! Congrats to you too! Been following you since Melanoma. Haha", "votes": null }, { "id": "1242117", "postDate": "03/17/2021 12:11:10", "content": "Thanks @usharengaraju for all the great resources. Saves us a lot of time.
", "rawMarkdown": "Thanks @usharengaraju for all the great resources. Saves us a lot of time.", "votes": null }, { "id": "1242127", "postDate": "03/17/2021 12:22:29", "content": "Thanks for the detailed write up and congrats on silver!
\nWe tried different approaches for rank ensembles and had a gain in our final score. It can be a worth to try.
\nCongrats again!
@reighns congrats on the silver medal and thanks a lot for sharing such a detailed summary!
", "rawMarkdown": "reighns congrats on the silver medal and thanks a lot for sharing such a detailed summary!", "votes": null }, { "id": "1242312", "postDate": "03/17/2021 14:39:28", "content": "Well done my friend, you deserve it!
", "rawMarkdown": "Well done my friend, you deserve it!", "votes": null }, { "id": "1245267", "postDate": "03/19/2021 16:23:55", "content": "did you find out the mean and std of the images as you have written
\nparams:\n mean: [0.4887381077884414]\n std: [0.23064819430546407]
\nand by but using gradcam are you meaning gradient accumulation or \"Gradient-weighted Class Activation Mapping,\"?
Great Solution
\nand Congratulation on Silver!
\nI never used to try any new things into backbone
Congratulations. Thanks for great detailed description
", "votes": null, "replies": [ { "id": 1241964, "author_name": "reighns", "author_url": "", "post_date": "03/17/2021 09:58:37", "content": "@ammarali32 Thanks, you are one of the giants whose shoulder we stood on :)
", "votes": null, "replies": [] }, { "id": 1241976, "author_name": "ammarali32", "author_url": "", "post_date": "03/17/2021 10:07:43", "content": "Thanks, I am really glad to hear that and you are the giants who scored so well and got on the top 2%
", "votes": null, "replies": [] } ] }, { "id": 1242035, "author_name": "usharengaraju", "author_url": "", "post_date": "03/17/2021 11:00:38", "content": "@reighns Congratulations on Silver Finish Hongnan . Great and Detailed Solution Writeup
", "votes": null, "replies": [ { "id": 1242117, "author_name": "reighns", "author_url": "", "post_date": "03/17/2021 12:11:10", "content": "Thanks @usharengaraju for all the great resources. Saves us a lot of time.
", "votes": null, "replies": [] } ] }, { "id": 1242058, "author_name": "datafan07", "author_url": "", "post_date": "03/17/2021 11:23:11", "content": "Congrats on your silver and thank you for the details!
", "votes": null, "replies": [ { "id": 1242115, "author_name": "reighns", "author_url": "", "post_date": "03/17/2021 12:10:47", "content": "Thanks a lot! Congrats to you too! Been following you since Melanoma. Haha
", "votes": null, "replies": [] } ] }, { "id": 1242127, "author_name": "snnclsr", "author_url": "", "post_date": "03/17/2021 12:22:29", "content": "Thanks for the detailed write up and congrats on silver!
\nWe tried different approaches for rank ensembles and had a gain in our final score. It can be a worth to try.
\nCongrats again!
@reighns congrats on the silver medal and thanks a lot for sharing such a detailed summary!
", "votes": null, "replies": [] }, { "id": 1242312, "author_name": "yannmajewski", "author_url": "", "post_date": "03/17/2021 14:39:28", "content": "Well done my friend, you deserve it!
", "votes": null, "replies": [] }, { "id": 1245267, "author_name": "mrinath", "author_url": "", "post_date": "03/19/2021 16:23:55", "content": "did you find out the mean and std of the images as you have written
\nparams:\n mean: [0.4887381077884414]\n std: [0.23064819430546407]
\nand by but using gradcam are you meaning gradient accumulation or \"Gradient-weighted Class Activation Mapping,\"?
Great Solution
\nand Congratulation on Silver!
\nI never used to try any new things into backbone