We would like to thank the competition host(s) and Kaggle for organizing the competition and congratulate all the winners, and anyone who benefited in some way from the competition. Special thanks to my teammates @zehuigong @thedrcat @dschettler8845 @felipebihaiek
\nI am sorry for late share,because I am busy with two competitions at the same time, I only spent the last 3 weeks preparing for the competition. If I start earlier, I still have a lot of ideas that I haven’t been able to realize.
\nBaseline solution: The settings of our baseline is as follows:
\n• Backbone: EfficientNet-b3;
\n• Data: Only training data, cell-level classification.
\n• Optimizer: Adam, BS = 128, LR=0.001, warm up + constant learning, 16 epoch.
\n• Image size: 320 for training and inference.
\n• Augmentations: The optimized transform policy of the previous third place solution
\n• Focal loss (gamma=2)
\n• Using the offline SWA to generate the inference weights.
\nThe above model achieved 0.351 LB score.
(1) Label smoothing with gaussian random noise, eps=0.05, the smooth label are generated as follows: (0.351 -> 0.412)
\n![\"random\"](\"https://github.com/yangsenwxy/png_githu1/blob/main/2.png?raw=true\")
\n(2) Mixup augmentation (0.412 ->0.428);
\n(3) Attention class head (boost ~0.002);
\n(4) Train + public (except for class 0 and 16) + label smoothing + mixup: 0.507
\n(5) Deep supervision, add supervision on the middle layer of the backbone network, 2 layer supervision (boost ~0.004, 0.507-> 0.511), 3 layer supervision (0.511 -> 0.513).
\n(6) Merge the cell-level prediction with image-level(RGBY+G) prediction.
• Backbone: EfficientNet-b7;
\n• Data: training +public data(RGBY+G), img-level classification.
\n• Image size: 600 for training and inference.
\n• Focal loss (gamma=2).
We use 8-TTA for inference, and the inference image size is the same as training image, e.g., 320.Segmentation Model is original HPA Segmentator
\nWe use a threshold of 0.001 to filter the predictions, and merge the image-level and cell-level classification results, which achieves an improvement of 0.04 mAP, with the weight of 0.7 for cell-level, 0.3 for image level predictions, respectively (0.513 -> 0.553).
\nA detailed breakdown of this step has been posted in a separate discussion
\n(1) Using GAP and GMP(global max pooling), and concatenate two output, before the final classification layer, we add two more fc layers.
\n(2) Currently, our cell-level image are all multiply with the corresponding cell mask, we train a model with cell-level images without multiplying the cell mask and, adding the cell mask as an additional input channel;
\n(3) BCE + Focal
\n(4) Class aware training sampling, we assign the sampling ratio for each of the training images according to the class frequency. This will lower the sampling ratio for the major class images, while improve the ratios for the minor class images.
We would like to thank the competition host(s) and Kaggle for organizing the competition and congratulate all the winners, and anyone who benefited in some way from the competition. Special thanks to my teammates @zehuigong @thedrcat @dschettler8845 @felipebihaiek
\nI am sorry for late share,because I am busy with two competitions at the same time, I only spent the last 3 weeks preparing for the competition. If I start earlier, I still have a lot of ideas that I haven’t been able to realize.
\nBaseline solution: The settings of our baseline is as follows:
\n• Backbone: EfficientNet-b3;
\n• Data: Only training data, cell-level classification.
\n• Optimizer: Adam, BS = 128, LR=0.001, warm up + constant learning, 16 epoch.
\n• Image size: 320 for training and inference.
\n• Augmentations: The optimized transform policy of the previous third place solution
\n• Focal loss (gamma=2)
\n• Using the offline SWA to generate the inference weights.
\nThe above model achieved 0.351 LB score.
(1) Label smoothing with gaussian random noise, eps=0.05, the smooth label are generated as follows: (0.351 -> 0.412)
\n
\n(2) Mixup augmentation (0.412 ->0.428);
\n(3) Attention class head (boost ~0.002);
\n(4) Train + public (except for class 0 and 16) + label smoothing + mixup: 0.507
\n(5) Deep supervision, add supervision on the middle layer of the backbone network, 2 layer supervision (boost ~0.004, 0.507-> 0.511), 3 layer supervision (0.511 -> 0.513).
\n(6) Merge the cell-level prediction with image-level(RGBY+G) prediction.
• Backbone: EfficientNet-b7;
\n• Data: training +public data(RGBY+G), img-level classification.
\n• Image size: 600 for training and inference.
\n• Focal loss (gamma=2).
We use 8-TTA for inference, and the inference image size is the same as training image, e.g., 320.Segmentation Model is original HPA Segmentator
\nWe use a threshold of 0.001 to filter the predictions, and merge the image-level and cell-level classification results, which achieves an improvement of 0.04 mAP, with the weight of 0.7 for cell-level, 0.3 for image level predictions, respectively (0.513 -> 0.553).
\nA detailed breakdown of this step has been posted in a separate discussion
\n(1) Using GAP and GMP(global max pooling), and concatenate two output, before the final classification layer, we add two more fc layers.
\n(2) Currently, our cell-level image are all multiply with the corresponding cell mask, we train a model with cell-level images without multiplying the cell mask and, adding the cell mask as an additional input channel;
\n(3) BCE + Focal
\n(4) Class aware training sampling, we assign the sampling ratio for each of the training images according to the class frequency. This will lower the sampling ratio for the major class images, while improve the ratios for the minor class images.
It's been a pleasure partnering on this competition, thanks for the write up!
", "rawMarkdown": "It's been a pleasure partnering on this competition, thanks for the write up!" } ], "comments": [ { "id": 1317273, "author_name": "Darek Kłeczek", "author_url": "", "post_date": "2021-05-21T09:08:22.733000", "content": "It's been a pleasure partnering on this competition, thanks for the write up!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1308178": "We would like to thank the competition host(s) and Kaggle for organizing the competition and congratulate all the winners, and anyone who benefited in some way from the competition. Special thanks to my teammates @zehuigong @thedrcat @dschettler8845 @felipebihaiek \n\nI am sorry for late share,because I am busy with two competitions at the same time, I only spent the last 3 weeks preparing for the competition. If I start earlier, I still have a lot of ideas that I haven’t been able to realize.\n# Solution Components:\n- Cell level models\n- Image-level Model\n- Gridify and Gapmask Inference\n\n\n# 1. Cell level models\nBaseline solution: The settings of our baseline is as follows:\n• Backbone: EfficientNet-b3;\n• Data: Only training data, cell-level classification.\n• Optimizer: Adam, BS = 128, LR=0.001, warm up + constant learning, 16 epoch.\n• Image size: 320 for training and inference.\n• Augmentations: The optimized transform policy of the previous third place solution\n• Focal loss (gamma=2)\n• Using the offline SWA to generate the inference weights.\nThe above model achieved 0.351 LB score.\n\n#### Tricks on the baseline:\n(1)\tLabel smoothing with gaussian random noise, eps=0.05, the smooth label are generated as follows: (0.351 -> 0.412)\n\n(2)\tMixup augmentation (0.412 ->0.428);\n(3)\tAttention class head (boost ~0.002);\n(4)\tTrain + public (except for class 0 and 16) + label smoothing + mixup: 0.507\n(5)\tDeep supervision, add supervision on the middle layer of the backbone network, 2 layer supervision (boost ~0.004, 0.507-> 0.511), 3 layer supervision (0.511 -> 0.513). \n(6)\tMerge the cell-level prediction with image-level(RGBY+G) prediction.\n\n\n\n# 2. Image-level Model\n\n• Backbone: EfficientNet-b7;\n• Data: training +public data(RGBY+G), img-level classification.\n• Image size: 600 for training and inference.\n• Focal loss (gamma=2).\n\n\n\n\n\n\n\n#### Inference\n\nWe use 8-TTA for inference, and the inference image size is the same as training image, e.g., 320.Segmentation Model is original HPA Segmentator\n#### Post-processing\nWe use a threshold of 0.001 to filter the predictions, and merge the image-level and cell-level classification results, which achieves an improvement of 0.04 mAP, with the weight of 0.7 for cell-level, 0.3 for image level predictions, respectively (0.513 -> 0.553). \n\n\n# 3. Gridify and Gapmask Inference\nA detailed breakdown of this step has been posted in a separate [discussion](https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/238365)\n\n\n## Final Results\n1. Original HPA Segmentatorse+2 layer +3 layer supervision cell level+img-level +Gridify and Gapmask Inference( Public: 0.5771 , Private: 0.5499)\n2. Faster HPA Segmentatorse+2 layer +3 layer supervision cell level+img-level +Gridify and Gapmask Inference( Public: 0.5724 , Private: 0.5410)\n\n\n\n## Ideas in mind\n - OOF models.\n - Use metric learning technique, such as ranking loss. (paper: Improving Pairwise Ranking for Multi-label Image Classification)\n## Things didn’t work for us\n(1)\tUsing GAP and GMP(global max pooling), and concatenate two output, before the final classification layer, we add two more fc layers.\n(2)\tCurrently, our cell-level image are all multiply with the corresponding cell mask, we train a model with cell-level images without multiplying the cell mask and, adding the cell mask as an additional input channel;\n(3)\tBCE + Focal\n(4)\tClass aware training sampling, we assign the sampling ratio for each of the training images according to the class frequency. This will lower the sampling ratio for the major class images, while improve the ratios for the minor class images.\n\n", "1317273": "It's been a pleasure partnering on this competition, thanks for the write up!" } }