Ensembling different folds, or different models, First rank all the probabilities of each model/fold, to ensure they are evenly distributed. In pandas, it can be done by df['pred'] = df['pred'].rank(pct=True) ( From 1st place solution )
Ensembling different folds, or different models, First rank all the probabilities of each model/fold, to ensure they are evenly distributed. In pandas, it can be done by df['pred'] = df['pred'].rank(pct=True) ( From 1st place solution )
",
"rawMarkdown": "**Best Practices**\n- OptimizeAUC\n- Crop small size from large image ( 256 from 1024 )\n- Create 100s of models means diverse if OOF not improving and stable\n- Use of noisy-student weights for EfficientNet for diversification\n- Smoothing in case of imbalance\n- AdamW , Adam and RAdam\n- Loss-Label Smoothing\n- Attentionhead & Squeeze & Extinction head\n- Coarse dropout , circular crop (a.k.a microscope augmentation), Cutmix\n- TTA ( x20 )\n- 2020 data for validation and different combinations of 2017/18/19/20 data for training.\n - Image segmentation to detect and crop the lesions and constructed the cropped data set.\n- Power averaging with the power of 2\n- Stacking using ExtraTreesClassifier (https://scikit-l- learn.org/stable/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html) \n- Random label smoothingRankaverage\n- RandAugment strategy, implemented here: https://github.com/ildoonet/pytorch-randaugment.\n- Ensembling different folds, or different models, First rank all the probabilities of each model/fold, to ensure they are evenly distributed. In pandas, it can be done by df['pred'] = df['pred'].rank(pct=True) ( From 1st place solution )\n\n**Good discussion /Link**\n- https://machinelearningmastery.com/weighted-average-ensemble-for-deep-learning-neural-- networks/?fbclid=IwAR3GYgj0Fu4Mp3RhTeyacb99H2QyP5uuWJizR7ei6DOOC-NbERKQIGyBB4o\n- https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175721\n- https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175412\n\n",
"votes": 5
}
],
"comments": [],
"raw_markdown_by_id": {
"989735": "**Best Practices**\n- OptimizeAUC\n- Crop small size from large image ( 256 from 1024 )\n- Create 100s of models means diverse if OOF not improving and stable\n- Use of noisy-student weights for EfficientNet for diversification\n- Smoothing in case of imbalance\n- AdamW , Adam and RAdam\n- Loss-Label Smoothing\n- Attentionhead & Squeeze & Extinction head\n- Coarse dropout , circular crop (a.k.a microscope augmentation), Cutmix\n- TTA ( x20 )\n- 2020 data for validation and different combinations of 2017/18/19/20 data for training.\n - Image segmentation to detect and crop the lesions and constructed the cropped data set.\n- Power averaging with the power of 2\n- Stacking using ExtraTreesClassifier (https://scikit-l- learn.org/stable/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html) \n- Random label smoothingRankaverage\n- RandAugment strategy, implemented here: https://github.com/ildoonet/pytorch-randaugment.\n- Ensembling different folds, or different models, First rank all the probabilities of each model/fold, to ensure they are evenly distributed. In pandas, it can be done by df['pred'] = df['pred'].rank(pct=True) ( From 1st place solution )\n\n**Good discussion /Link**\n- https://machinelearningmastery.com/weighted-average-ensemble-for-deep-learning-neural-- networks/?fbclid=IwAR3GYgj0Fu4Mp3RhTeyacb99H2QyP5uuWJizR7ei6DOOC-NbERKQIGyBB4o\n- https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175721\n- https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175412\n\n"
}
}