![\"img.png\"](\"https://i.postimg.cc/6QJpjkZV/img.png\")
We would like to thank Kaggle and Sartorius for organizing such a great competition.
\nOur solution is as follows. In many respects, it is similar to the 2nd place solution.
\n
At the very beginning of the competition, we decided to build a solution using box-based instance segmentation, and focus more on the bbox detection part. We think the mask prediction performance is mainly limited by annotation quality so we did not pay much attention to it.
\nDuring the competition, we used COCO mAP as our validation metric, we believe that high mAP and proper thresholding would give a high LB score. Following is the validation score we achieved at the end of the competition
Evaluating bbox…
\nLoading and preparing results…
\nDONE (t=0.09s)
\ncreating index…
\nindex created!
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.396
\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.764
\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.364
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.354
\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.305
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.572
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.386
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.491
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.579
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.550
\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.552
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.757
+----------+-------+----------+-------+----------+-------+
\n| category | AP | category | AP | category | AP |
\n+----------+-------+----------+-------+----------+-------+
\n| shsy5y | 0.334 | astro | 0.399 | cort | 0.456 |
\n+----------+-------+----------+-------+----------+-------+
Evaluating segm…
\nLoading and preparing results…
\nDONE (t=0.57s)
\ncreating index…
\nindex created!
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.362
\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.767
\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.294
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.308
\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.419
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.461
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.349
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.441
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.516
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.514
\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.469
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.465
+----------+-------+----------+-------+----------+-------+
\n| category | AP | category | AP | category | AP |
\n+----------+-------+----------+-------+----------+-------+
\n| shsy5y | 0.328 | astro | 0.302 | cort | 0.456 |
\n+----------+-------+----------+-------+----------+-------+
We found YOLOX performed impressively well without hyperparameter tuning. We used train/val split to monitor validation scores, and then trained models using all training data.
\nThings worked:
\nStrong feature extractor (CB DBS-FPN, EffDetD7, CSPDarknet-YOLOXPAFPN)
\nLarge input size (1536)
\nLivecell pretrain
In the livecell dataset, some images have thousands of instances, and this amount could be doubled by mixup. Some operations in SimOTA would cause OOM frequently when we use large input sizes and have many ground-truth instances. @tascj0 optimized SimOTA with some CUDA extensions to save memory and speedup training.
\nWe used two Mask R-CNN models with CB-DBS as the backbone. According to the results of Revisiting Mask-Head Architectures for Novel Class Instance Segmentation, training was done using GT's bbox and mask. In our experiment, the score was comparable to the one using the RPN head proposal as usual.
\nAt the time of inference, we passed the ensembled bbox to the mask head to get predictions.
However, in the end, this part only slightly boosted the score. The main part on the mask side is UPerNet.
\nIn this part, we also used the data cropped by GT's bbox and mask to train. Since the instances are small in pixel size, it’s important to do cropping and pasting accurately. Thus we used ROIAlign to crop&resize input image&mask and used grid_sample to paste a prediction to its bbox location before thresholding. It’s also important to resize training target(mask) using bilinear interpolation then threshold it, instead of using nearest neighbor interpolation. Basically, the point is to follow the setting of Mask R-CNN mask head.
\nWe trained 4 UPerNet models with Swin or ResNet101 as the backbone, pretrain on livecell and finetune on competition data. We used ensembled bboxes for inference. In mask ensemble, we simply averaged the probs of each model's (UPerNets and Mask R-CNNs) prediction.
We re-scored predicted instances by the score of bbox × average of score of mask (prob >= 0.5). This improves validation COCO mAP of astro by 0.01.
\nSimple thresholding, overlap removal and dropping small number_of_pixels instances.
\nWe used mmdetection and mmsegmentation from open-mmlab to build our pipeline. We are very grateful that such an easy to use tool is being developed in open source.
\nHere we release a minimum version of our solution to show to implement the ideas given those awesome tools
\nhttps://github.com/tascj/kaggle-sartorius-cell-instance-segmentation-solution
\ntakuoko is a member of Z by HP Data Science Global Ambassadors. Special Thanks to Z by HP for sponsoring me a Z8G4 Workstation with dual A6000 GPU and a ZBook with RTX5000 GPU.
", "messages": [ { "id": 1638774, "postDate": "2022-01-05T02:41:19.753Z", "content": "We would like to thank Kaggle and Sartorius for organizing such a great competition.
\nOur solution is as follows. In many respects, it is similar to the 2nd place solution.
\n
At the very beginning of the competition, we decided to build a solution using box-based instance segmentation, and focus more on the bbox detection part. We think the mask prediction performance is mainly limited by annotation quality so we did not pay much attention to it.
\nDuring the competition, we used COCO mAP as our validation metric, we believe that high mAP and proper thresholding would give a high LB score. Following is the validation score we achieved at the end of the competition
Evaluating bbox…
\nLoading and preparing results…
\nDONE (t=0.09s)
\ncreating index…
\nindex created!
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.396
\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.764
\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.364
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.354
\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.305
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.572
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.386
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.491
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.579
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.550
\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.552
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.757
+----------+-------+----------+-------+----------+-------+
\n| category | AP | category | AP | category | AP |
\n+----------+-------+----------+-------+----------+-------+
\n| shsy5y | 0.334 | astro | 0.399 | cort | 0.456 |
\n+----------+-------+----------+-------+----------+-------+
Evaluating segm…
\nLoading and preparing results…
\nDONE (t=0.57s)
\ncreating index…
\nindex created!
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.362
\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.767
\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.294
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.308
\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.419
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.461
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.349
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.441
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.516
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.514
\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.469
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.465
+----------+-------+----------+-------+----------+-------+
\n| category | AP | category | AP | category | AP |
\n+----------+-------+----------+-------+----------+-------+
\n| shsy5y | 0.328 | astro | 0.302 | cort | 0.456 |
\n+----------+-------+----------+-------+----------+-------+
We found YOLOX performed impressively well without hyperparameter tuning. We used train/val split to monitor validation scores, and then trained models using all training data.
\nThings worked:
\nStrong feature extractor (CB DBS-FPN, EffDetD7, CSPDarknet-YOLOXPAFPN)
\nLarge input size (1536)
\nLivecell pretrain
In the livecell dataset, some images have thousands of instances, and this amount could be doubled by mixup. Some operations in SimOTA would cause OOM frequently when we use large input sizes and have many ground-truth instances. @tascj0 optimized SimOTA with some CUDA extensions to save memory and speedup training.
\nWe used two Mask R-CNN models with CB-DBS as the backbone. According to the results of Revisiting Mask-Head Architectures for Novel Class Instance Segmentation, training was done using GT's bbox and mask. In our experiment, the score was comparable to the one using the RPN head proposal as usual.
\nAt the time of inference, we passed the ensembled bbox to the mask head to get predictions.
However, in the end, this part only slightly boosted the score. The main part on the mask side is UPerNet.
\nIn this part, we also used the data cropped by GT's bbox and mask to train. Since the instances are small in pixel size, it’s important to do cropping and pasting accurately. Thus we used ROIAlign to crop&resize input image&mask and used grid_sample to paste a prediction to its bbox location before thresholding. It’s also important to resize training target(mask) using bilinear interpolation then threshold it, instead of using nearest neighbor interpolation. Basically, the point is to follow the setting of Mask R-CNN mask head.
\nWe trained 4 UPerNet models with Swin or ResNet101 as the backbone, pretrain on livecell and finetune on competition data. We used ensembled bboxes for inference. In mask ensemble, we simply averaged the probs of each model's (UPerNets and Mask R-CNNs) prediction.
We re-scored predicted instances by the score of bbox × average of score of mask (prob >= 0.5). This improves validation COCO mAP of astro by 0.01.
\nSimple thresholding, overlap removal and dropping small number_of_pixels instances.
\nWe used mmdetection and mmsegmentation from open-mmlab to build our pipeline. We are very grateful that such an easy to use tool is being developed in open source.
\nHere we release a minimum version of our solution to show to implement the ideas given those awesome tools
\nhttps://github.com/tascj/kaggle-sartorius-cell-instance-segmentation-solution
\ntakuoko is a member of Z by HP Data Science Global Ambassadors. Special Thanks to Z by HP for sponsoring me a Z8G4 Workstation with dual A6000 GPU and a ZBook with RTX5000 GPU.
", "rawMarkdown": "We would like to thank Kaggle and Sartorius for organizing such a great competition.\n\n# Overview\nOur solution is as follows. In many respects, it is similar to the 2nd place solution.\n\n[](https://postimg.cc/tskb7m27)\n\nAt the very beginning of the competition, we decided to build a solution using box-based instance segmentation, and focus more on the bbox detection part. We think the mask prediction performance is mainly limited by annotation quality so we did not pay much attention to it.\nDuring the competition, we used COCO mAP as our validation metric, we believe that high mAP and proper thresholding would give a high LB score. Following is the validation score we achieved at the end of the competition\n\nEvaluating bbox... \nLoading and preparing results... \nDONE (t=0.09s) \ncreating index... \nindex created! \n \n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.396 \n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.764 \n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.364 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.354 \n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.305 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.572 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.386 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.491 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.579 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.550 \n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.552 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.757 \n \n \n+----------+-------+----------+-------+----------+-------+ \n| category | AP | category | AP | category | AP | \n+----------+-------+----------+-------+----------+-------+ \n| shsy5y | 0.334 | astro | 0.399 | cort | 0.456 | \n+----------+-------+----------+-------+----------+-------+ \n \nEvaluating segm... \nLoading and preparing results... \nDONE (t=0.57s) \ncreating index... \nindex created! \n \n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.362 \n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.767 \n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.294 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.308 \n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.419 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.461 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.349 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.441 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.516 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.514 \n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.469 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.465 \n \n \n+----------+-------+----------+-------+----------+-------+ \n| category | AP | category | AP | category | AP | \n+----------+-------+----------+-------+----------+-------+ \n| shsy5y | 0.328 | astro | 0.302 | cort | 0.456 | \n+----------+-------+----------+-------+----------+-------+ \n\n\n\n# Bbox part\nWe found YOLOX performed impressively well without hyperparameter tuning. We used train/val split to monitor validation scores, and then trained models using all training data.\nThings worked:\nStrong feature extractor (CB DBS-FPN, EffDetD7, CSPDarknet-YOLOXPAFPN)\nLarge input size (1536)\nLivecell pretrain\n\nIn the livecell dataset, some images have thousands of instances, and this amount could be doubled by mixup. Some operations in SimOTA would cause OOM frequently when we use large input sizes and have many ground-truth instances. @tascj0 optimized SimOTA with some CUDA extensions to save memory and speedup training.\n\n# Segmentation, Mask R-CNN part\nWe used two Mask R-CNN models with CB-DBS as the backbone. According to the results of [Revisiting Mask-Head Architectures for Novel Class Instance Segmentation](https://ai.googleblog.com/2021/09/revisiting-mask-head-architectures-for.html), training was done using GT's bbox and mask. In our experiment, the score was comparable to the one using the RPN head proposal as usual.\nAt the time of inference, we passed the ensembled bbox to the mask head to get predictions.\n\nHowever, in the end, this part only slightly boosted the score. The main part on the mask side is UPerNet.\n\n# Segmentation, UPerNet part\nIn this part, we also used the data cropped by GT's bbox and mask to train. Since the instances are small in pixel size, it’s important to do cropping and pasting accurately. Thus we used ROIAlign to crop&resize input image&mask and used grid_sample to paste a prediction to its bbox location before thresholding. It’s also important to resize training target(mask) using bilinear interpolation then threshold it, instead of using nearest neighbor interpolation. Basically, the point is to follow the setting of Mask R-CNN mask head.\nWe trained 4 UPerNet models with Swin or ResNet101 as the backbone, pretrain on livecell and finetune on competition data. We used ensembled bboxes for inference. In mask ensemble, we simply averaged the probs of each model's (UPerNets and Mask R-CNNs) prediction.\n\n# Reranking\nWe re-scored predicted instances by the score of bbox × average of score of mask (prob >= 0.5). This improves validation COCO mAP of astro by 0.01.\n\n# Post process\nSimple thresholding, overlap removal and dropping small number_of_pixels instances.\n\n# Code\nWe used mmdetection and mmsegmentation from open-mmlab to build our pipeline. We are very grateful that such an easy to use tool is being developed in open source.\n\nHere we release a minimum version of our solution to show to implement the ideas given those awesome tools\n\nhttps://github.com/tascj/kaggle-sartorius-cell-instance-segmentation-solution\n\n\n# Acknowledge\ntakuoko is a member of Z by HP Data Science Global Ambassadors. Special Thanks to Z by HP for sponsoring me a Z8G4 Workstation with dual A6000 GPU and a ZBook with RTX5000 GPU. \n", "votes": 101 }, { "id": 1638822, "postDate": "2022-01-05T04:33:25.340Z", "content": "Congratulations!
\nOne doubt, in the bbox part you said you used those backbone models, are all of them pretrained?
\nAnd can we really use other backbones for yolox?Didnt know this
Thank you for the question.
\nSorry we forgot to describe the details here. We took COCO pretrained CB DBS-FPN R-CNN and EffDetD7, replace their heads with YOLOXHead, and finetuned the models on COCO dataset before livecell pretrain.
", "rawMarkdown": "Thank you for the question.\n\nSorry we forgot to describe the details here. We took COCO pretrained CB DBS-FPN R-CNN and EffDetD7, replace their heads with YOLOXHead, and finetuned the models on COCO dataset before livecell pretrain.", "votes": 6 }, { "id": 1639021, "postDate": "2022-01-05T10:02:06.970Z", "content": "Thanks, got it!
\ncongrats again!
Nice solution! Well deserved, congrats to the winners @takuok and team. Thanks for sharing solution and github code.
", "rawMarkdown": "Nice solution! Well deserved, congrats to the winners @takuok and team. Thanks for sharing solution and github code.", "votes": 2 }, { "id": 2592052, "postDate": "2024-01-08T11:29:35.077Z", "content": "Great work! I greatly appreciate the team effort.
\nCurrently, I am engaged in a real-world project focused on Cancer cell instance segmentation. I'm hopeful that your contributions will provide a significant starting point for my work.
Congratulations!
\nI am wondering where you get the dataset. According to your instruction, the unzipped files are like this:
\n├── LIVECell_dataset_2021
\n│ ├── images
\n│ ├── livecell_coco_train.json
\n│ ├── livecell_coco_val.json
\n│ └── livecell_coco_test.json
\n├── train
\n├── train_semi_supervised
\n└── train.csv
\nBut it seems that the livecell dataset on https://github.com/sartorius-research/LIVECell isn't exactly like this. Thanks.
Did you solve this problem?
", "rawMarkdown": "Did you solve this problem?" } ] }, { "id": 1647268, "postDate": "2022-01-12T12:57:57.387Z", "content": "@takuoko, @tascj Congratulation on your victory!
", "rawMarkdown": "@takuoko, @tascj Congratulation on your victory! " }, { "id": 1643480, "postDate": "2022-01-09T12:54:54.180Z", "content": "Thanks for sharing the proces step by step!
\nHow long does it normally take to make those precise model? Just curious 😀
It's possible to make a gold-medal-precise model using 20~30 RTX 3090 hours (check the released code).
\nOur final ensemble models took weeks and multiple RTX3090/A6000 to train.
Congratulations
", "rawMarkdown": "Congratulations" }, { "id": 1642824, "postDate": "2022-01-08T18:02:52.477Z", "content": "Congratulations. Very nice solution. Thanks for sharing
", "rawMarkdown": "Congratulations. Very nice solution. Thanks for sharing " }, { "id": 1642476, "postDate": "2022-01-08T12:25:56.977Z", "content": "Nice solution! Well deserved, congrats to the winners Thank you
", "rawMarkdown": "Nice solution! Well deserved, congrats to the winners Thank you" }, { "id": 1641777, "postDate": "2022-01-07T17:40:00.690Z", "content": "Nice work!
", "rawMarkdown": "Nice work!" }, { "id": 1639798, "postDate": "2022-01-05T22:58:55.883Z", "content": "Thanks for sharing and congratulations on 1st place 💪
", "rawMarkdown": "Thanks for sharing and congratulations on 1st place 💪" }, { "id": 1639760, "postDate": "2022-01-05T21:56:11.137Z", "content": "Congratulations on the first place! 🥇
", "rawMarkdown": "Congratulations on the first place! 🥇" }, { "id": 1639566, "postDate": "2022-01-05T18:37:17.407Z", "content": "Incredible! Congrats on the first place winning!
", "rawMarkdown": "Incredible! Congrats on the first place winning!" }, { "id": 1638965, "postDate": "2022-01-05T08:08:49.677Z", "content": "Excellent solution! your team is powerful!
", "rawMarkdown": "Excellent solution! your team is powerful!" }, { "id": 1639348, "postDate": "2022-01-05T15:14:21.897Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2674293, "postDate": "2024-02-29T08:26:15.797Z", "content": "Great solution!
\nThank you!
Thanks! Nice Solution!
", "rawMarkdown": "Thanks! Nice Solution!" }, { "id": 1641640, "postDate": "2022-01-07T15:48:33.343Z", "content": "thank you for sharing this!
", "rawMarkdown": "thank you for sharing this!" }, { "id": 1639061, "postDate": "2022-01-05T10:48:03.640Z", "content": "this is really nice. Thanks for sharing
", "rawMarkdown": "this is really nice. Thanks for sharing " } ], "comments": [ { "id": 1638822, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2022-01-05T04:33:25.340000", "content": "Congratulations!
\nOne doubt, in the bbox part you said you used those backbone models, are all of them pretrained?
\nAnd can we really use other backbones for yolox?Didnt know this
Thank you for the question.
\nSorry we forgot to describe the details here. We took COCO pretrained CB DBS-FPN R-CNN and EffDetD7, replace their heads with YOLOXHead, and finetuned the models on COCO dataset before livecell pretrain.
", "votes": 6, "replies": [] }, { "id": 1639021, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2022-01-05T10:02:06.970000", "content": "Thanks, got it!
\ncongrats again!
Nice solution! Well deserved, congrats to the winners @takuok and team. Thanks for sharing solution and github code.
", "votes": 2, "replies": [] }, { "id": 2592052, "author_name": "bibinsee", "author_url": "", "post_date": "2024-01-08T11:29:35.077000", "content": "Great work! I greatly appreciate the team effort.
\nCurrently, I am engaged in a real-world project focused on Cancer cell instance segmentation. I'm hopeful that your contributions will provide a significant starting point for my work.
Congratulations!
\nI am wondering where you get the dataset. According to your instruction, the unzipped files are like this:
\n├── LIVECell_dataset_2021
\n│ ├── images
\n│ ├── livecell_coco_train.json
\n│ ├── livecell_coco_val.json
\n│ └── livecell_coco_test.json
\n├── train
\n├── train_semi_supervised
\n└── train.csv
\nBut it seems that the livecell dataset on https://github.com/sartorius-research/LIVECell isn't exactly like this. Thanks.
Did you solve this problem?
", "votes": 0, "replies": [] } ] }, { "id": 1647268, "author_name": "Firuz Juraev", "author_url": "", "post_date": "2022-01-12T12:57:57.387000", "content": "@takuoko, @tascj Congratulation on your victory!
", "votes": 0, "replies": [] }, { "id": 1643480, "author_name": "SOYOUNG PARK", "author_url": "", "post_date": "2022-01-09T12:54:54.180000", "content": "Thanks for sharing the proces step by step!
\nHow long does it normally take to make those precise model? Just curious 😀
It's possible to make a gold-medal-precise model using 20~30 RTX 3090 hours (check the released code).
\nOur final ensemble models took weeks and multiple RTX3090/A6000 to train.
Congratulations
", "votes": 0, "replies": [] }, { "id": 1642824, "author_name": "Binaya Bajgain", "author_url": "", "post_date": "2022-01-08T18:02:52.477000", "content": "Congratulations. Very nice solution. Thanks for sharing
", "votes": 0, "replies": [] }, { "id": 1642476, "author_name": "DineshManikanta", "author_url": "", "post_date": "2022-01-08T12:25:56.977000", "content": "Nice solution! Well deserved, congrats to the winners Thank you
", "votes": 0, "replies": [] }, { "id": 1641777, "author_name": "ZAID ABDELFATTAH", "author_url": "", "post_date": "2022-01-07T17:40:00.690000", "content": "Nice work!
", "votes": 0, "replies": [] }, { "id": 1639798, "author_name": "Niek van der Zwaag", "author_url": "", "post_date": "2022-01-05T22:58:55.883000", "content": "Thanks for sharing and congratulations on 1st place 💪
", "votes": 0, "replies": [] }, { "id": 1639760, "author_name": "Julián Peller (dataista0)", "author_url": "", "post_date": "2022-01-05T21:56:11.137000", "content": "Congratulations on the first place! 🥇
", "votes": 0, "replies": [] }, { "id": 1639566, "author_name": "Mintwater", "author_url": "", "post_date": "2022-01-05T18:37:17.407000", "content": "Incredible! Congrats on the first place winning!
", "votes": 0, "replies": [] }, { "id": 1638965, "author_name": "Ctrl_CV", "author_url": "", "post_date": "2022-01-05T08:08:49.677000", "content": "Excellent solution! your team is powerful!
", "votes": 0, "replies": [] }, { "id": 1639348, "author_name": "", "author_url": "", "post_date": "2022-01-05T15:14:21.897000", "content": "", "votes": 0, "replies": [] }, { "id": 2674293, "author_name": "Matvey K. Smekalin", "author_url": "", "post_date": "2024-02-29T08:26:15.797000", "content": "Great solution!
\nThank you!
Thanks! Nice Solution!
", "votes": 0, "replies": [] }, { "id": 1641640, "author_name": "Ma Yuan", "author_url": "", "post_date": "2022-01-07T15:48:33.343000", "content": "thank you for sharing this!
", "votes": 0, "replies": [] }, { "id": 1639061, "author_name": "Haw Keat", "author_url": "", "post_date": "2022-01-05T10:48:03.640000", "content": "this is really nice. Thanks for sharing
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1638774": "We would like to thank Kaggle and Sartorius for organizing such a great competition.\n\n# Overview\nOur solution is as follows. In many respects, it is similar to the 2nd place solution.\n\n[](https://postimg.cc/tskb7m27)\n\nAt the very beginning of the competition, we decided to build a solution using box-based instance segmentation, and focus more on the bbox detection part. We think the mask prediction performance is mainly limited by annotation quality so we did not pay much attention to it.\nDuring the competition, we used COCO mAP as our validation metric, we believe that high mAP and proper thresholding would give a high LB score. Following is the validation score we achieved at the end of the competition\n\nEvaluating bbox... \nLoading and preparing results... \nDONE (t=0.09s) \ncreating index... \nindex created! \n \n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.396 \n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.764 \n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.364 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.354 \n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.305 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.572 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.386 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.491 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.579 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.550 \n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.552 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.757 \n \n \n+----------+-------+----------+-------+----------+-------+ \n| category | AP | category | AP | category | AP | \n+----------+-------+----------+-------+----------+-------+ \n| shsy5y | 0.334 | astro | 0.399 | cort | 0.456 | \n+----------+-------+----------+-------+----------+-------+ \n \nEvaluating segm... \nLoading and preparing results... \nDONE (t=0.57s) \ncreating index... \nindex created! \n \n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.362 \n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=2000 ] = 0.767 \n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=2000 ] = 0.294 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.308 \n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.419 \n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.461 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.349 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.441 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=2000 ] = 0.516 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=2000 ] = 0.514 \n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=2000 ] = 0.469 \n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=2000 ] = 0.465 \n \n \n+----------+-------+----------+-------+----------+-------+ \n| category | AP | category | AP | category | AP | \n+----------+-------+----------+-------+----------+-------+ \n| shsy5y | 0.328 | astro | 0.302 | cort | 0.456 | \n+----------+-------+----------+-------+----------+-------+ \n\n\n\n# Bbox part\nWe found YOLOX performed impressively well without hyperparameter tuning. We used train/val split to monitor validation scores, and then trained models using all training data.\nThings worked:\nStrong feature extractor (CB DBS-FPN, EffDetD7, CSPDarknet-YOLOXPAFPN)\nLarge input size (1536)\nLivecell pretrain\n\nIn the livecell dataset, some images have thousands of instances, and this amount could be doubled by mixup. Some operations in SimOTA would cause OOM frequently when we use large input sizes and have many ground-truth instances. @tascj0 optimized SimOTA with some CUDA extensions to save memory and speedup training.\n\n# Segmentation, Mask R-CNN part\nWe used two Mask R-CNN models with CB-DBS as the backbone. According to the results of [Revisiting Mask-Head Architectures for Novel Class Instance Segmentation](https://ai.googleblog.com/2021/09/revisiting-mask-head-architectures-for.html), training was done using GT's bbox and mask. In our experiment, the score was comparable to the one using the RPN head proposal as usual.\nAt the time of inference, we passed the ensembled bbox to the mask head to get predictions.\n\nHowever, in the end, this part only slightly boosted the score. The main part on the mask side is UPerNet.\n\n# Segmentation, UPerNet part\nIn this part, we also used the data cropped by GT's bbox and mask to train. Since the instances are small in pixel size, it’s important to do cropping and pasting accurately. Thus we used ROIAlign to crop&resize input image&mask and used grid_sample to paste a prediction to its bbox location before thresholding. It’s also important to resize training target(mask) using bilinear interpolation then threshold it, instead of using nearest neighbor interpolation. Basically, the point is to follow the setting of Mask R-CNN mask head.\nWe trained 4 UPerNet models with Swin or ResNet101 as the backbone, pretrain on livecell and finetune on competition data. We used ensembled bboxes for inference. In mask ensemble, we simply averaged the probs of each model's (UPerNets and Mask R-CNNs) prediction.\n\n# Reranking\nWe re-scored predicted instances by the score of bbox × average of score of mask (prob >= 0.5). This improves validation COCO mAP of astro by 0.01.\n\n# Post process\nSimple thresholding, overlap removal and dropping small number_of_pixels instances.\n\n# Code\nWe used mmdetection and mmsegmentation from open-mmlab to build our pipeline. We are very grateful that such an easy to use tool is being developed in open source.\n\nHere we release a minimum version of our solution to show to implement the ideas given those awesome tools\n\nhttps://github.com/tascj/kaggle-sartorius-cell-instance-segmentation-solution\n\n\n# Acknowledge\ntakuoko is a member of Z by HP Data Science Global Ambassadors. Special Thanks to Z by HP for sponsoring me a Z8G4 Workstation with dual A6000 GPU and a ZBook with RTX5000 GPU. \n", "1638822": "Congratulations!\nOne doubt, in the bbox part you said you used those backbone models, are all of them pretrained?\nAnd can we really use other backbones for yolox?Didnt know this", "1638827": "Nice solution! Well deserved, congrats to the winners @takuok and team. Thanks for sharing solution and github code.", "2592052": "Great work! I greatly appreciate the team effort.\nCurrently, I am engaged in a real-world project focused on Cancer cell instance segmentation. I'm hopeful that your contributions will provide a significant starting point for my work.\n", "1825261": "Congratulations!\nI am wondering where you get the dataset. According to your instruction, the unzipped files are like this: \n├── LIVECell_dataset_2021\n│ ├── images\n│ ├── livecell_coco_train.json\n│ ├── livecell_coco_val.json\n│ └── livecell_coco_test.json\n├── train\n├── train_semi_supervised\n└── train.csv\nBut it seems that the livecell dataset on https://github.com/sartorius-research/LIVECell isn't exactly like this. Thanks.", "1647268": "@takuoko, @tascj Congratulation on your victory! ", "1643480": "Thanks for sharing the proces step by step! \nHow long does it normally take to make those precise model? Just curious 😀", "1642899": "Congratulations", "1642824": "Congratulations. Very nice solution. Thanks for sharing ", "1642476": "Nice solution! Well deserved, congrats to the winners Thank you", "1641777": "Nice work!", "1639798": "Thanks for sharing and congratulations on 1st place 💪", "1639760": "Congratulations on the first place! 🥇", "1639566": "Incredible! Congrats on the first place winning!", "1638965": "Excellent solution! your team is powerful!", "1639348": "", "2674293": "Great solution!\nThank you!", "1646227": "Thanks! Nice Solution!", "1641640": "thank you for sharing this!", "1639061": "this is really nice. Thanks for sharing " } }