\nStrangely, my CV increased with a learning rate of 0.01 but LB decreased.
\nSee this for instance (left column indicates CV and right LB)
\n
![\"\"](\"https://github.com/Gladiator07/Kaggle-images/blob/main/sartorius_scheduler.png\")
Anyone willing to share their best single model? I will go first:
\n-EDIT- Thanks to the suggestion by @omallo, I am adding more information here
\nlibrary: mmdetection
\nmodel: mask_rcnn_r50_fpn (single fold)
\nepochs: 12
\nimg_size: 1280 x 800
\nlb w/ standard thresholds (0.5 for each class): 0.284
\nlb w/ tuned thresholds: Not tried yet
Anyone willing to share their best single model? I will go first:
\n-EDIT- Thanks to the suggestion by @omallo, I am adding more information here
\nlibrary: mmdetection
\nmodel: mask_rcnn_r50_fpn (single fold)
\nepochs: 12
\nimg_size: 1280 x 800
\nlb w/ standard thresholds (0.5 for each class): 0.284
\nlb w/ tuned thresholds: Not tried yet
I think that the number of trained epochs is also relevant for people so they know how much they might have to invest to get a good model so I'm also including that information (others might want to include that as well):
\nlibrary: detectron2
\nmodel: mask_rcnn_x101_fpn (single fold)
\nepochs: 50
\ntraining time: 8h
\ntrain image sizes (short edge, multi scale): 480, 520, 560, 640, 672, 704, 736, 768, 800
\ntest image size (short edge): 800
\nlb w/ standard thresholds (0.5 for each class): 0.299
\nlb w/ tuned thresholds: 0.306
Here is my model, i think i should try with larger backbone
\nlibrary: detectron2
\nmodel: mask_rcnn_r50_fpn (single fold)
\nepochs: ~40
\ntraining time: 3h
\nlb w/ standard threshold: 0.291
\nlb w/ tuned thresholds: 0.296
where to setup epochs?
\nI just set the iteration number.
", "rawMarkdown": "where to setup epochs?\n\nI just set the iteration number." }, { "id": 1589647, "postDate": "2021-11-20T13:17:39.090Z", "content": "detectron doesn't specify epochs number, you can calculate by:
\nMAX_ITER * IMS_PER_BATCH / TOTAL_NUM_IMAGES
When you say standard threshold ,what does it mean ? No threshold per class ? Or 0.5 for all class … ?
", "rawMarkdown": "When you say standard threshold ,what does it mean ? No threshold per class ? Or 0.5 for all class ... ? " }, { "id": 1589840, "postDate": "2021-11-20T16:26:15.533Z", "content": "By default I meant 0.5 for each class. I updated my comment accordingly.
", "rawMarkdown": "By default I meant 0.5 for each class. I updated my comment accordingly.", "votes": 1 }, { "id": 1589860, "postDate": "2021-11-20T16:45:19.463Z", "content": "Thanks much
", "rawMarkdown": "Thanks much " }, { "id": 1589864, "postDate": "2021-11-20T16:48:21.073Z", "content": "I am curious about the epoch config in detectron2. if It is computed as you said, then I understand it. Thanks
", "rawMarkdown": "I am curious about the epoch config in detectron2. if It is computed as you said, then I understand it. Thanks" }, { "id": 1589908, "postDate": "2021-11-20T17:26:42.177Z", "content": "Thanks for sharing @omallo. Great idea on including epochs as this is important. I think image size is also relevant so I updated the main thread with that information.
", "rawMarkdown": "Thanks for sharing @omallo. Great idea on including epochs as this is important. I think image size is also relevant so I updated the main thread with that information. " }, { "id": 1590033, "postDate": "2021-11-20T20:47:04.247Z", "content": "@trushk , I also added mine :) I'm using multi scale image sizes for training, in the hope that it also prevents overfitting.
", "rawMarkdown": "@trushk , I also added mine :) I'm using multi scale image sizes for training, in the hope that it also prevents overfitting.", "votes": 1 }, { "id": 1590037, "postDate": "2021-11-20T20:50:35.670Z", "content": "@omallo thanks. what is your long edge 1333?
", "rawMarkdown": "@omallo thanks. what is your long edge 1333? " }, { "id": 1590046, "postDate": "2021-11-20T21:13:23.843Z", "content": "@trushk , I've set MAX_SIZE_TRAIN and MAX_SIZE_TEST to 1333 but from my understanding, this just limits the image size when scaling. The long edge is always derived from the short edge and it can never exceed 1333. As an example, if the short edge is set to 800 (my biggest size), then the long edge will be (800 / 520) * 704 = 1083 where 520 and 704 are the dimensions of the original images.
Hi!
\nThanks!
\nWhat learning rate did you set?
I'm currently using a learning rate of 0.01 and a batch size of 2 but, to be honest, I don't think that those are ideal values. Currently, I'm a bit forced to use a small batch size due to the GPU memory consumption. So I think you might want to experiment with other values if you have the possibility.
", "rawMarkdown": "I'm currently using a learning rate of 0.01 and a batch size of 2 but, to be honest, I don't think that those are ideal values. Currently, I'm a bit forced to use a small batch size due to the GPU memory consumption. So I think you might want to experiment with other values if you have the possibility." }, { "id": 1596278, "postDate": "2021-11-26T10:40:22.637Z", "content": "Hi, do you lower the learning rate after some iterations or using any kind of learning schedule ?
\nStrangely, my CV increased with a learning rate of 0.01 but LB decreased.
\nSee this for instance (left column indicates CV and right LB)
\n
@atharvaingle , I'm using the default LR schedule of Detectron2: There is a warm up period at the beginning (~5 epochs) where the LR is continually incremented from 0.0001 to 0.01. Then I train with 0.01 and in the last ~5 epochs, I lower the LR to 0.001. But I haven't made any comparisons with other LR schedules yet so I'm not sure whether this is a particularly good approach or not.
", "rawMarkdown": "@atharvaingle , I'm using the default LR schedule of Detectron2: There is a warm up period at the beginning (~5 epochs) where the LR is continually incremented from 0.0001 to 0.01. Then I train with 0.01 and in the last ~5 epochs, I lower the LR to 0.001. But I haven't made any comparisons with other LR schedules yet so I'm not sure whether this is a particularly good approach or not.", "votes": 4 }, { "id": 1607538, "postDate": "2021-12-06T00:47:45.507Z", "content": "You, my sir, is the real hero. Thank you for the generous information, very helpful indeed.
", "rawMarkdown": "You, my sir, is the real hero. Thank you for the generous information, very helpful indeed." }, { "id": 1618834, "postDate": "2021-12-15T11:51:21.980Z", "content": "hey, why would you want to use a larger batch size doesn't that reduce accuracy?
", "rawMarkdown": "hey, why would you want to use a larger batch size doesn't that reduce accuracy?", "votes": 1 }, { "id": 1629777, "postDate": "2021-12-26T14:34:21.403Z", "content": "train image sizes (short edge, multi scale): 480, 520, 560, 640, 672, 704, 736, 768, 800
\nHi, @omallo What do you mean by this? it would be really helpful, if you can give a hint. Thank you😇.
@soumya9977 , this is an augmentation method where the short edge of the images is rescaled to one of the given sizes. One of the sizes is picked at random for each batch.
\nMore concretely, the original images in this competition have the shape (520, 704). If e.g. the scale 640 is chosen for a given batch, then all images within that batch are scaled to the shape (640, 866), i.e. the short edge is set to 640 and the long edge is calculated so that the original image ratio is preserved. In this case, the long edge is calculated as (640 / 520) * 704 = 866.
The multi scale training is a feature supported by detectron2 and also other deep learning libraries.
\nI hope this helps :-)
", "rawMarkdown": "@soumya9977 , this is an augmentation method where the short edge of the images is rescaled to one of the given sizes. One of the sizes is picked at random for each batch.\n\nMore concretely, the original images in this competition have the shape `(520, 704)`. If e.g. the scale `640` is chosen for a given batch, then all images within that batch are scaled to the shape `(640, 866)`, i.e. the short edge is set to `640` and the long edge is calculated so that the original image ratio is preserved. In this case, the long edge is calculated as `(640 / 520) * 704 = 866`.\n\nThe multi scale training is a feature supported by detectron2 and also other deep learning libraries.\n\nI hope this helps :-)", "votes": 1 }, { "id": 1630883, "postDate": "2021-12-27T20:06:41.963Z", "content": "Thank you for explaining @omallo , I will try to add this in my augmentations.
\nQuick questions:
@somu159776 , you're welcome.
\nI'm only using basic augmentations, i.e. the described multi-scale resizing, horizontal/vertical flipping, and cropping. However, I have not experimented with other augmentations so I cannot really say whether the help or not.
", "rawMarkdown": "@somu159776 , you're welcome.\n\nI'm only using basic augmentations, i.e. the described multi-scale resizing, horizontal/vertical flipping, and cropping. However, I have not experimented with other augmentations so I cannot really say whether the help or not.", "votes": 1 }, { "id": 1630951, "postDate": "2021-12-27T21:28:46.013Z", "content": "Thank you, your explanations really helped😇.
", "rawMarkdown": "Thank you, your explanations really helped😇. ", "votes": 1 } ] }, { "id": 1595076, "postDate": "2021-11-25T11:39:33.910Z", "content": "library: mmdetection
\nmodel: cascade mask rcnn (single fold)
\nepochs: 20
\nCV 0.2746 - LB 0.315
\nCV 0.2827 - LB 0.320
\nCV 0.2880 - LB 0.324
[update-1]
\nCV 0.2983 - LB 0.327
\nCV 0.2995 - LB 0.328
[update-2]
\nCV 0.3002 - LB 0.330
\nCV 0.3013 - LB 0.331
\nCV 0.3038 - LB 0.332
[update-3]
\nCV 0.3047 - LB 0.335
\nCV 0.3049 - LB 0.336
That shows great correlation. Thanks for sharing! Is this with original train or after some mask fixes.
", "rawMarkdown": "That shows great correlation. Thanks for sharing! Is this with original train or after some mask fixes. " }, { "id": 1595868, "postDate": "2021-11-26T04:00:10.657Z", "content": "with original train
", "rawMarkdown": "with original train", "votes": 1 }, { "id": 1596091, "postDate": "2021-11-26T07:32:38.107Z", "content": "Do you apply any post-processing?
", "rawMarkdown": "Do you apply any post-processing?" }, { "id": 1596186, "postDate": "2021-11-26T08:48:14.200Z", "content": "@tyaiga did u use resent50 as backbone?
", "rawMarkdown": "@tyaiga did u use resent50 as backbone?" }, { "id": 1596206, "postDate": "2021-11-26T09:00:52.130Z", "content": "No, bigger one.
", "rawMarkdown": "No, bigger one.", "votes": 1 }, { "id": 1597255, "postDate": "2021-11-27T10:56:26.090Z", "content": "@tyaiga , your performance is really nice since you get a great score with a low number of epochs. Also, the gain on the LB is quite big.
\nMay I ask what you think brought you extra performance, i.e. was it the choice of the right model, LR schedule, augmentations, post-processing, …? Of course, you don't have to disclose any details.
", "rawMarkdown": "@tyaiga , your performance is really nice since you get a great score with a low number of epochs. Also, the gain on the LB is quite big.\n\nMay I ask what you think brought you extra performance, i.e. was it the choice of the right model, LR schedule, augmentations, post-processing, ...? Of course, you don't have to disclose any details." }, { "id": 1597350, "postDate": "2021-11-27T12:42:07.037Z", "content": "pretty cool! which metric do you use in mmdet about CV0.2995?
", "rawMarkdown": "pretty cool! which metric do you use in mmdet about CV0.2995?" }, { "id": 1597365, "postDate": "2021-11-27T12:50:57.823Z", "content": "I use this after mmdet training.
", "rawMarkdown": "I use [this](https://www.kaggle.com/theoviel/competition-metric-map-iou/notebook) after mmdet training.", "votes": 1 }, { "id": 1597370, "postDate": "2021-11-27T12:56:49.703Z", "content": "thanks😊. you mean you evaluate the model one by one by the metric after all the training end?
", "rawMarkdown": "thanks😊. you mean you evaluate the model one by one by the metric after all the training end?" }, { "id": 1597802, "postDate": "2021-11-27T23:19:39.793Z", "content": "Could you please share,your method to convert .csv to COCO for mmdet? I used this to 5-fold split, but always get:
\n**Average Precision (AP) @[ IoU=0.50:0.95 area= large maxDets=1000 ] = -1.000 **
\nduring COCOEval.
", "rawMarkdown": "Could you please share,your method to convert .csv to COCO for mmdet? I used [this](https://www.kaggle.com/mistag/sartorius-create-coco-annotations) to 5-fold split, but always get:\n\n**Average Precision (AP) @[ IoU=0.50:0.95 area= large maxDets=1000 ] = -1.000 **\n\nduring COCOEval. \n\n" }, { "id": 1598594, "postDate": "2021-11-28T16:03:40.030Z", "content": "I haven't tried yet
", "rawMarkdown": "I haven't tried yet" }, { "id": 1601237, "postDate": "2021-12-01T05:40:51.827Z", "content": "CV/LB updated
", "rawMarkdown": "CV/LB updated" }, { "id": 1601258, "postDate": "2021-12-01T06:15:58.967Z", "content": "great scores and correlation! so all of these have been generated using one of a k-fold split of \"cascade mask rcnn 20 epoch\" model? Ofourse your pre/postporcess techniques may have been varied but I just wanted to know whether you use the same base throughout
", "rawMarkdown": "great scores and correlation! so all of these have been generated using one of a k-fold split of \"cascade mask rcnn 20 epoch\" model? Ofourse your pre/postporcess techniques may have been varied but I just wanted to know whether you use the same base throughout\n " }, { "id": 1601261, "postDate": "2021-12-01T06:25:45.907Z", "content": "Yes, the above scores are generated by \"one of a k-fold split of cascade mask rcnn 20 epoch model\".
", "rawMarkdown": "Yes, the above scores are generated by \"one of a k-fold split of cascade mask rcnn 20 epoch model\"." }, { "id": 1601267, "postDate": "2021-12-01T06:30:26.837Z", "content": "okay thanks for the share :)
", "rawMarkdown": "okay thanks for the share :)" }, { "id": 1604838, "postDate": "2021-12-03T18:54:56.167Z", "content": "@tyaiga I wrote this for eval for mmdet. It will be great if you can provide insights: https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/292852
", "rawMarkdown": "@tyaiga I wrote this for eval for mmdet. It will be great if you can provide insights: https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/292852", "votes": 1 } ] }, { "id": 1592871, "postDate": "2021-11-23T12:35:26.913Z", "content": "library: mmdetection
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.301 - LB 0.314
Thanks @ren4yu Do you use a custom eval function for mmdet?
", "rawMarkdown": "Thanks @ren4yu Do you use a custom eval function for mmdet? " }, { "id": 1593062, "postDate": "2021-11-23T15:45:39.383Z", "content": "I evaluated the best model after training with
\nhttps://www.kaggle.com/theoviel/competition-metric-map-iou/notebook
Did you calculate CV score after eliminating pixel overlap process?
", "rawMarkdown": "Did you calculate CV score after eliminating pixel overlap process?" }, { "id": 1593470, "postDate": "2021-11-24T00:41:30.077Z", "content": "\n\nDid you calculate CV score after eliminating pixel overlap process?
\n
Yes. It is evaluated after some post processing including pixel overlap elimination because I want to evaluate whole submission pipeline.
", "rawMarkdown": "> Did you calculate CV score after eliminating pixel overlap process?\n\nYes. It is evaluated after some post processing including pixel overlap elimination because I want to evaluate whole submission pipeline.", "votes": 4 }, { "id": 1593537, "postDate": "2021-11-24T03:34:00.347Z", "content": "Do you use LiveCell dataset or external data in train_semi_supervised directory?
", "rawMarkdown": "Do you use LiveCell dataset or external data in train_semi_supervised directory?" }, { "id": 1593538, "postDate": "2021-11-24T03:36:32.533Z", "content": "No, I used only train.csv and train images.
", "rawMarkdown": "No, I used only train.csv and train images.", "votes": 4 }, { "id": 1595353, "postDate": "2021-11-25T16:02:42.343Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1597354, "postDate": "2021-11-27T12:44:34.183Z", "content": "I'm new to mmdet.😭 Can I know how do you add the evaluate metric https://www.kaggle.com/theoviel/competition-metric-map-iou/notebook to mmdet to evaluate your model
", "rawMarkdown": "I'm new to mmdet.😭 Can I know how do you add the evaluate metric https://www.kaggle.com/theoviel/competition-metric-map-iou/notebook to mmdet to evaluate your model" } ] }, { "id": 1596964, "postDate": "2021-11-27T03:51:20.663Z", "content": "library: mmdetection
\nmodel: mask rcnn (single fold)
\ntrain_size: 484 images
\ntest_size: 122 images
\nCV: 0.2587 LB: 0.299
\nNo model modified. No preprocessing. No postprocessing except for overlap fixed.
that's cool. Can I know which metric do you use in mmdetection for CV0.2587, do you mean map or official metric
", "rawMarkdown": "that's cool. Can I know which metric do you use in mmdetection for CV0.2587, do you mean map or official metric " }, { "id": 1597490, "postDate": "2021-11-27T14:58:54.523Z", "content": "I use metric from the public notebook (here).
", "rawMarkdown": "I use metric from the public notebook ([here](https://www.kaggle.com/theoviel/competition-metric-map-iou/notebook))." } ] }, { "id": 1601463, "postDate": "2021-12-01T10:03:04.273Z", "content": "single model,CV0.3223, LB0.332, detectron2
", "rawMarkdown": "single model,CV0.3223, LB0.332, detectron2", "votes": 8, "replies": [ { "id": 1605605, "postDate": "2021-12-04T11:55:57.607Z", "content": "is this your Test train split cv or N fold cv ? @guohey
", "rawMarkdown": "is this your Test train split cv or N fold cv ? @guohey " }, { "id": 1605736, "postDate": "2021-12-04T13:56:13.320Z", "content": "well, i use Test train split
", "rawMarkdown": "well, i use Test train split" }, { "id": 1608423, "postDate": "2021-12-06T11:38:34.407Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1615641, "postDate": "2021-12-12T13:53:06.603Z", "content": "Detectron Mask R-CNN. Public LB score is 0.32. I'm still trying to figure out an efficient way to ensemble predictions.
\nFold 1 - mAP: 0.313467 ({'astro': 0.20072195936428244, 'cort': 0.4026399670002614, 'shsy5y': 0.22756584415732165})\nFold 2 - mAP: 0.297655 ({'astro': 0.16545075040715596, 'cort': 0.3963139150087539, 'shsy5y': 0.20485488471168073})\nFold 3 - mAP: 0.306650 ({'astro': 0.1683234036456997, 'cort': 0.4000650243514869, 'shsy5y': 0.22981052684883063})\nFold 4 - mAP: 0.301520 ({'astro': 0.1767558512239449, 'cort': 0.3947693877607034, 'shsy5y': 0.21364495836951708})\nFold 5 - mAP: 0.298685 ({'astro': 0.18354330953728468, 'cort': 0.3803528848335181, 'shsy5y': 0.22665148818681236})\n------------------------------\nOOF mAP: 0.303612 ({'astro': 0.17912518387787665, 'cort': 0.3948282357909447, 'shsy5y': 0.2205055404548325})\n👀the mAP is so high，you mean mAP score or official metric socre？
", "rawMarkdown": "👀the mAP is so high，you mean mAP score or official metric socre？" }, { "id": 1615722, "postDate": "2021-12-12T15:44:16.433Z", "content": "indeed good job @gunesevitan - would you mind if you share your evaluator setup (and per class scores)
", "rawMarkdown": "indeed good job @gunesevitan - would you mind if you share your evaluator setup (and per class scores) " }, { "id": 1615723, "postDate": "2021-12-12T15:44:21.190Z", "content": "I think a lot of competitors have shifted to using the mAP metric provided in the livecell github repo
", "rawMarkdown": "I think a lot of competitors have shifted to using the mAP metric provided in the livecell github repo" }, { "id": 1615757, "postDate": "2021-12-12T16:36:14.977Z", "content": "umm mAP is official metric of this competition?
\n@imeintanis Yes, I can share my evaluator, but the scores above is produced by my inference pipeline which I can't share. My evaluator is very similar to one that is shared. I only added couple lines to store labels in order to calculate per class scores.
\nclass InstanceSegmentationEvaluator(DatasetEvaluator):\n\n def __init__(self, dataset_name, segmentation_format='bitmask'):\n\n dataset = DatasetCatalog.get(dataset_name)\n self.annotations_cache = {item['image_id']: item['annotations'] for item in dataset}\n self.segmentation_format = segmentation_format\n\n def reset(self):\n\n self.scores = []\n self.labels = []\n\n def process(self, inputs, outputs):\n\n for input_, output in zip(inputs, outputs):\n if len(output['instances']) == 0:\n # Set 0 mAP when there are no objects predicted by model\n self.scores.append(0)\n annotation = self.annotations_cache[input_['image_id']]\n label = np.unique(list(map(lambda x: x['category_id'], annotation)))[0]\n self.labels.append(label)\n else:\n # Calculate mAP with predicted objects\n annotation = self.annotations_cache[input_['image_id']]\n prediction_masks = output['instances'].pred_masks.cpu().numpy()\n average_precision = metrics.get_average_precision_detectron(\n ground_truth_masks=annotation,\n prediction_masks=prediction_masks,\n ground_truth_mask_format=self.segmentation_format,\n verbose=False\n )\n self.scores.append(average_precision)\n label = np.unique(list(map(lambda x: x['category_id'], annotation)))[0]\n self.labels.append(label)\n\n def evaluate(self):\n\n df_scores = pd.DataFrame(columns=['scores', 'labels'])\n df_scores['scores'] = np.array(self.scores)\n df_scores['labels'] = np.array(self.labels)\n df_scores = df_scores.groupby('labels')['scores'].mean().to_dict()\n\n return {'mAP': np.mean(self.scores), 'mAP cort': df_scores[0], 'mAP shsy5y': df_scores[1], 'mAP astro': df_scores[2]}\n@gunesevitan how many epochs do you train.
", "rawMarkdown": "@gunesevitan how many epochs do you train. " } ] }, { "id": 1589283, "postDate": "2021-11-20T05:41:48.083Z", "content": "detectron: mask_rcnn_r50_fpn(single-fold)
\nlb: 0.299
Update(2021-12-06)
\ndetectron: mask_rcnn_r50_fpn(single-fold)
\ncv: 0.319
\nlb: 0.322
@Roc, what hyper-parameters did you tune in detectron2?
\nI am not getting more than 0.252 in lb
\nWhat is your ensembling strategy for the models from different folds?
Torch Mask-RCNN R50 (single fold)
\nmAP IoU: 0.249 --> LB: 0.266 - 0.268 (depends on thresholds per class) Detectron Mask-RCNN R50 (single fold)
\nmAP IoU: 0.260 --> LB: 0.246 - 0.294 (depends on thresholds per class) Thanks for making this comparison. Performance gap is kinda significant. What is the difference between torchvision and detectron implementations?
", "rawMarkdown": "Thanks for making this comparison. Performance gap is kinda significant. What is the difference between torchvision and detectron implementations? ", "votes": 1 }, { "id": 1588781, "postDate": "2021-11-19T16:16:21.790Z", "content": "Indeed, to be honest I haven't dig too much in the details between the two, I started with Torch for my convenience and then moved quickly to Detectron framework, it's the first time I use it so I'm still in exploration mode :)
\nBackbones and fold data are same however epochs/iter and LR schedules were different.
", "rawMarkdown": "Indeed, to be honest I haven't dig too much in the details between the two, I started with Torch for my convenience and then moved quickly to Detectron framework, it's the first time I use it so I'm still in exploration mode :) \n\nBackbones and fold data are same however epochs/iter and LR schedules were different. ", "votes": 1 } ] }, { "id": 1608131, "postDate": "2021-12-06T08:41:25.697Z", "content": "library: torchvision
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.303 - LB 0.308
[update]
\nlibrary: detectron2
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.309 - LB 0.319
detectron2: mask rcnn r50 fpn(single-fold)
\nstandard thresholds: 0.291
\ntuned thresholds: 0.301
How to tune Thresholds?
", "rawMarkdown": "How to tune Thresholds?" }, { "id": 1589842, "postDate": "2021-11-20T16:29:02.647Z", "content": "you may check this discussion
\nhere
@iwrbeverly Yes. I was inspired by the discussion written by @slawekbiel Threshold by class
", "rawMarkdown": "@iwrbeverly Yes. I was inspired by the discussion written by @slawekbiel [Threshold by class](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/284215)", "votes": 1 }, { "id": 1590297, "postDate": "2021-11-21T04:33:14.937Z", "content": "BTW, has anyone tried Swin-Transformer backbone? The resnet backbone is just dominating this competition + fine-tuning tricks/ensemble. I think maybe the backbone still has the room for improvement?
", "rawMarkdown": "BTW, has anyone tried Swin-Transformer backbone? The resnet backbone is just dominating this competition + fine-tuning tricks/ensemble. I think maybe the backbone still has the room for improvement? " }, { "id": 1590968, "postDate": "2021-11-21T19:48:43.003Z", "content": "The Swin Transformer I have tried under detectron2 was underperforming.
", "rawMarkdown": "The Swin Transformer I have tried under detectron2 was underperforming.", "votes": 1 }, { "id": 1591157, "postDate": "2021-11-22T03:38:39.530Z", "content": "@hsadeghian I am trying it under detectron2, too. May I know how much did u score on LB by Swin Transformer? ( I got .295 with this single model + simple tuned thresholds.)
", "rawMarkdown": "@hsadeghian I am trying it under detectron2, too. May I know how much did u score on LB by Swin Transformer? ( I got .295 with this single model + simple tuned thresholds.)", "votes": 1 }, { "id": 1591169, "postDate": "2021-11-22T03:58:48.413Z", "content": "I got less than you, LB=.285 with swinT with tuned thresholds
", "rawMarkdown": "I got less than you, LB=.285 with swinT with tuned thresholds", "votes": 1 }, { "id": 1596422, "postDate": "2021-11-26T13:35:48.083Z", "content": "What is your learning rate set?
", "rawMarkdown": "What is your learning rate set?" }, { "id": 1620851, "postDate": "2021-12-17T07:10:28.977Z", "content": "I can got cv 303, lb303 using SwinT.
", "rawMarkdown": "I can got cv 303, lb303 using SwinT.", "votes": 1 }, { "id": 1620958, "postDate": "2021-12-17T09:34:46.100Z", "content": "@tiandaye Hi, How do you evaluate model for getting CV 0.303? In training and inference? My CV did not exceed 0.273. Thanks!
", "rawMarkdown": "@tiandaye Hi, How do you evaluate model for getting CV 0.303? In training and inference? My CV did not exceed 0.273. Thanks!" } ] }, { "id": 1620904, "postDate": "2021-12-17T08:15:23.030Z", "content": "library: detectron2
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.273 - LB 0.308
mmdet:cascade_mask_rcnn_r50_fpn(single fold)
\nlb:0.294
model: 5-Layer-U-Net as described by Ronneberger et al. (but no augmentations)
\nLB: 0.177
", "rawMarkdown": "model: 5-Layer-U-Net as described by Ronneberger et al. (but no augmentations)\n\nLB: 0.177" }, { "id": 1597416, "postDate": "2021-11-27T13:47:07.987Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1608286, "postDate": "2021-12-06T10:32:59.343Z", "content": "thanks for sharing @trushk
", "rawMarkdown": "thanks for sharing @trushk ", "votes": -2 } ], "comments": [ { "id": 1589499, "author_name": "omallo", "author_url": "", "post_date": "2021-11-20T10:46:46.747000", "content": "I think that the number of trained epochs is also relevant for people so they know how much they might have to invest to get a good model so I'm also including that information (others might want to include that as well):
\nlibrary: detectron2
\nmodel: mask_rcnn_x101_fpn (single fold)
\nepochs: 50
\ntraining time: 8h
\ntrain image sizes (short edge, multi scale): 480, 520, 560, 640, 672, 704, 736, 768, 800
\ntest image size (short edge): 800
\nlb w/ standard thresholds (0.5 for each class): 0.299
\nlb w/ tuned thresholds: 0.306
Here is my model, i think i should try with larger backbone
\nlibrary: detectron2
\nmodel: mask_rcnn_r50_fpn (single fold)
\nepochs: ~40
\ntraining time: 3h
\nlb w/ standard threshold: 0.291
\nlb w/ tuned thresholds: 0.296
where to setup epochs?
\nI just set the iteration number.
", "votes": 0, "replies": [] }, { "id": 1589647, "author_name": "Vu", "author_url": "", "post_date": "2021-11-20T13:17:39.090000", "content": "detectron doesn't specify epochs number, you can calculate by:
\nMAX_ITER * IMS_PER_BATCH / TOTAL_NUM_IMAGES
When you say standard threshold ,what does it mean ? No threshold per class ? Or 0.5 for all class … ?
", "votes": 0, "replies": [] }, { "id": 1589840, "author_name": "omallo", "author_url": "", "post_date": "2021-11-20T16:26:15.533000", "content": "By default I meant 0.5 for each class. I updated my comment accordingly.
", "votes": 1, "replies": [] }, { "id": 1589860, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2021-11-20T16:45:19.463000", "content": "Thanks much
", "votes": 0, "replies": [] }, { "id": 1589864, "author_name": "dragon zhang", "author_url": "", "post_date": "2021-11-20T16:48:21.073000", "content": "I am curious about the epoch config in detectron2. if It is computed as you said, then I understand it. Thanks
", "votes": 0, "replies": [] }, { "id": 1589908, "author_name": "Trushant Kalyanpur", "author_url": "", "post_date": "2021-11-20T17:26:42.177000", "content": "Thanks for sharing @omallo. Great idea on including epochs as this is important. I think image size is also relevant so I updated the main thread with that information.
", "votes": 0, "replies": [] }, { "id": 1590033, "author_name": "omallo", "author_url": "", "post_date": "2021-11-20T20:47:04.247000", "content": "@trushk , I also added mine :) I'm using multi scale image sizes for training, in the hope that it also prevents overfitting.
", "votes": 1, "replies": [] }, { "id": 1590037, "author_name": "Trushant Kalyanpur", "author_url": "", "post_date": "2021-11-20T20:50:35.670000", "content": "@omallo thanks. what is your long edge 1333?
", "votes": 0, "replies": [] }, { "id": 1590046, "author_name": "omallo", "author_url": "", "post_date": "2021-11-20T21:13:23.843000", "content": "@trushk , I've set MAX_SIZE_TRAIN and MAX_SIZE_TEST to 1333 but from my understanding, this just limits the image size when scaling. The long edge is always derived from the short edge and it can never exceed 1333. As an example, if the short edge is set to 800 (my biggest size), then the long edge will be (800 / 520) * 704 = 1083 where 520 and 704 are the dimensions of the original images.
Hi!
\nThanks!
\nWhat learning rate did you set?
I'm currently using a learning rate of 0.01 and a batch size of 2 but, to be honest, I don't think that those are ideal values. Currently, I'm a bit forced to use a small batch size due to the GPU memory consumption. So I think you might want to experiment with other values if you have the possibility.
", "votes": 0, "replies": [] }, { "id": 1596278, "author_name": "Atharva Ingle", "author_url": "", "post_date": "2021-11-26T10:40:22.637000", "content": "Hi, do you lower the learning rate after some iterations or using any kind of learning schedule ?
\nStrangely, my CV increased with a learning rate of 0.01 but LB decreased.
\nSee this for instance (left column indicates CV and right LB)
\n
@atharvaingle , I'm using the default LR schedule of Detectron2: There is a warm up period at the beginning (~5 epochs) where the LR is continually incremented from 0.0001 to 0.01. Then I train with 0.01 and in the last ~5 epochs, I lower the LR to 0.001. But I haven't made any comparisons with other LR schedules yet so I'm not sure whether this is a particularly good approach or not.
", "votes": 4, "replies": [] }, { "id": 1607538, "author_name": "Woprime", "author_url": "", "post_date": "2021-12-06T00:47:45.507000", "content": "You, my sir, is the real hero. Thank you for the generous information, very helpful indeed.
", "votes": 0, "replies": [] }, { "id": 1618834, "author_name": "Hannah B", "author_url": "", "post_date": "2021-12-15T11:51:21.980000", "content": "hey, why would you want to use a larger batch size doesn't that reduce accuracy?
", "votes": 1, "replies": [] }, { "id": 1629777, "author_name": "somuSan", "author_url": "", "post_date": "2021-12-26T14:34:21.403000", "content": "train image sizes (short edge, multi scale): 480, 520, 560, 640, 672, 704, 736, 768, 800
\nHi, @omallo What do you mean by this? it would be really helpful, if you can give a hint. Thank you😇.
@soumya9977 , this is an augmentation method where the short edge of the images is rescaled to one of the given sizes. One of the sizes is picked at random for each batch.
\nMore concretely, the original images in this competition have the shape (520, 704). If e.g. the scale 640 is chosen for a given batch, then all images within that batch are scaled to the shape (640, 866), i.e. the short edge is set to 640 and the long edge is calculated so that the original image ratio is preserved. In this case, the long edge is calculated as (640 / 520) * 704 = 866.
The multi scale training is a feature supported by detectron2 and also other deep learning libraries.
\nI hope this helps :-)
", "votes": 1, "replies": [] }, { "id": 1630883, "author_name": "somuSan", "author_url": "", "post_date": "2021-12-27T20:06:41.963000", "content": "Thank you for explaining @omallo , I will try to add this in my augmentations.
\nQuick questions:
@somu159776 , you're welcome.
\nI'm only using basic augmentations, i.e. the described multi-scale resizing, horizontal/vertical flipping, and cropping. However, I have not experimented with other augmentations so I cannot really say whether the help or not.
", "votes": 1, "replies": [] }, { "id": 1630951, "author_name": "somuSan", "author_url": "", "post_date": "2021-12-27T21:28:46.013000", "content": "Thank you, your explanations really helped😇.
", "votes": 1, "replies": [] } ] }, { "id": 1595076, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-11-25T11:39:33.910000", "content": "library: mmdetection
\nmodel: cascade mask rcnn (single fold)
\nepochs: 20
\nCV 0.2746 - LB 0.315
\nCV 0.2827 - LB 0.320
\nCV 0.2880 - LB 0.324
[update-1]
\nCV 0.2983 - LB 0.327
\nCV 0.2995 - LB 0.328
[update-2]
\nCV 0.3002 - LB 0.330
\nCV 0.3013 - LB 0.331
\nCV 0.3038 - LB 0.332
[update-3]
\nCV 0.3047 - LB 0.335
\nCV 0.3049 - LB 0.336
That shows great correlation. Thanks for sharing! Is this with original train or after some mask fixes.
", "votes": 0, "replies": [] }, { "id": 1595868, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-11-26T04:00:10.657000", "content": "with original train
", "votes": 1, "replies": [] }, { "id": 1596091, "author_name": "Phat Tran", "author_url": "", "post_date": "2021-11-26T07:32:38.107000", "content": "Do you apply any post-processing?
", "votes": 0, "replies": [] }, { "id": 1596186, "author_name": "Zhikun Xu", "author_url": "", "post_date": "2021-11-26T08:48:14.200000", "content": "@tyaiga did u use resent50 as backbone?
", "votes": 0, "replies": [] }, { "id": 1596206, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-11-26T09:00:52.130000", "content": "No, bigger one.
", "votes": 1, "replies": [] }, { "id": 1597255, "author_name": "omallo", "author_url": "", "post_date": "2021-11-27T10:56:26.090000", "content": "@tyaiga , your performance is really nice since you get a great score with a low number of epochs. Also, the gain on the LB is quite big.
\nMay I ask what you think brought you extra performance, i.e. was it the choice of the right model, LR schedule, augmentations, post-processing, …? Of course, you don't have to disclose any details.
", "votes": 0, "replies": [] }, { "id": 1597350, "author_name": "zw", "author_url": "", "post_date": "2021-11-27T12:42:07.037000", "content": "pretty cool! which metric do you use in mmdet about CV0.2995?
", "votes": 0, "replies": [] }, { "id": 1597365, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-11-27T12:50:57.823000", "content": "I use this after mmdet training.
", "votes": 1, "replies": [] }, { "id": 1597370, "author_name": "zw", "author_url": "", "post_date": "2021-11-27T12:56:49.703000", "content": "thanks😊. you mean you evaluate the model one by one by the metric after all the training end?
", "votes": 0, "replies": [] }, { "id": 1597802, "author_name": "Mark", "author_url": "", "post_date": "2021-11-27T23:19:39.793000", "content": "Could you please share,your method to convert .csv to COCO for mmdet? I used this to 5-fold split, but always get:
\n**Average Precision (AP) @[ IoU=0.50:0.95 area= large maxDets=1000 ] = -1.000 **
\nduring COCOEval.
", "votes": 0, "replies": [] }, { "id": 1598594, "author_name": "zw", "author_url": "", "post_date": "2021-11-28T16:03:40.030000", "content": "I haven't tried yet
", "votes": 0, "replies": [] }, { "id": 1601237, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-12-01T05:40:51.827000", "content": "CV/LB updated
", "votes": 0, "replies": [] }, { "id": 1601258, "author_name": "Hannah B", "author_url": "", "post_date": "2021-12-01T06:15:58.967000", "content": "great scores and correlation! so all of these have been generated using one of a k-fold split of \"cascade mask rcnn 20 epoch\" model? Ofourse your pre/postporcess techniques may have been varied but I just wanted to know whether you use the same base throughout
", "votes": 0, "replies": [] }, { "id": 1601261, "author_name": "Yamame🐟", "author_url": "", "post_date": "2021-12-01T06:25:45.907000", "content": "Yes, the above scores are generated by \"one of a k-fold split of cascade mask rcnn 20 epoch model\".
", "votes": 0, "replies": [] }, { "id": 1601267, "author_name": "Hannah B", "author_url": "", "post_date": "2021-12-01T06:30:26.837000", "content": "okay thanks for the share :)
", "votes": 0, "replies": [] }, { "id": 1604838, "author_name": "Trushant Kalyanpur", "author_url": "", "post_date": "2021-12-03T18:54:56.167000", "content": "@tyaiga I wrote this for eval for mmdet. It will be great if you can provide insights: https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/292852
", "votes": 1, "replies": [] } ] }, { "id": 1592871, "author_name": "yu4u", "author_url": "", "post_date": "2021-11-23T12:35:26.913000", "content": "library: mmdetection
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.301 - LB 0.314
Thanks @ren4yu Do you use a custom eval function for mmdet?
", "votes": 0, "replies": [] }, { "id": 1593062, "author_name": "yu4u", "author_url": "", "post_date": "2021-11-23T15:45:39.383000", "content": "I evaluated the best model after training with
\nhttps://www.kaggle.com/theoviel/competition-metric-map-iou/notebook
Did you calculate CV score after eliminating pixel overlap process?
", "votes": 0, "replies": [] }, { "id": 1593470, "author_name": "yu4u", "author_url": "", "post_date": "2021-11-24T00:41:30.077000", "content": "\n\nDid you calculate CV score after eliminating pixel overlap process?
\n
Yes. It is evaluated after some post processing including pixel overlap elimination because I want to evaluate whole submission pipeline.
", "votes": 4, "replies": [] }, { "id": 1593537, "author_name": "HungNT", "author_url": "", "post_date": "2021-11-24T03:34:00.347000", "content": "Do you use LiveCell dataset or external data in train_semi_supervised directory?
", "votes": 0, "replies": [] }, { "id": 1593538, "author_name": "yu4u", "author_url": "", "post_date": "2021-11-24T03:36:32.533000", "content": "No, I used only train.csv and train images.
", "votes": 4, "replies": [] }, { "id": 1595353, "author_name": "", "author_url": "", "post_date": "2021-11-25T16:02:42.343000", "content": "", "votes": 0, "replies": [] }, { "id": 1597354, "author_name": "zw", "author_url": "", "post_date": "2021-11-27T12:44:34.183000", "content": "I'm new to mmdet.😭 Can I know how do you add the evaluate metric https://www.kaggle.com/theoviel/competition-metric-map-iou/notebook to mmdet to evaluate your model
", "votes": 0, "replies": [] } ] }, { "id": 1596964, "author_name": "blueboy-97", "author_url": "", "post_date": "2021-11-27T03:51:20.663000", "content": "library: mmdetection
\nmodel: mask rcnn (single fold)
\ntrain_size: 484 images
\ntest_size: 122 images
\nCV: 0.2587 LB: 0.299
\nNo model modified. No preprocessing. No postprocessing except for overlap fixed.
that's cool. Can I know which metric do you use in mmdetection for CV0.2587, do you mean map or official metric
", "votes": 0, "replies": [] }, { "id": 1597490, "author_name": "blueboy-97", "author_url": "", "post_date": "2021-11-27T14:58:54.523000", "content": "I use metric from the public notebook (here).
", "votes": 0, "replies": [] } ] }, { "id": 1601463, "author_name": "Ctrl_CV", "author_url": "", "post_date": "2021-12-01T10:03:04.273000", "content": "single model,CV0.3223, LB0.332, detectron2
", "votes": 8, "replies": [ { "id": 1605605, "author_name": "Jaideep", "author_url": "", "post_date": "2021-12-04T11:55:57.607000", "content": "is this your Test train split cv or N fold cv ? @guohey
", "votes": 0, "replies": [] }, { "id": 1605736, "author_name": "Ctrl_CV", "author_url": "", "post_date": "2021-12-04T13:56:13.320000", "content": "well, i use Test train split
", "votes": 0, "replies": [] }, { "id": 1608423, "author_name": "", "author_url": "", "post_date": "2021-12-06T11:38:34.407000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1615641, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2021-12-12T13:53:06.603000", "content": "Detectron Mask R-CNN. Public LB score is 0.32. I'm still trying to figure out an efficient way to ensemble predictions.
\nFold 1 - mAP: 0.313467 ({'astro': 0.20072195936428244, 'cort': 0.4026399670002614, 'shsy5y': 0.22756584415732165})\nFold 2 - mAP: 0.297655 ({'astro': 0.16545075040715596, 'cort': 0.3963139150087539, 'shsy5y': 0.20485488471168073})\nFold 3 - mAP: 0.306650 ({'astro': 0.1683234036456997, 'cort': 0.4000650243514869, 'shsy5y': 0.22981052684883063})\nFold 4 - mAP: 0.301520 ({'astro': 0.1767558512239449, 'cort': 0.3947693877607034, 'shsy5y': 0.21364495836951708})\nFold 5 - mAP: 0.298685 ({'astro': 0.18354330953728468, 'cort': 0.3803528848335181, 'shsy5y': 0.22665148818681236})\n------------------------------\nOOF mAP: 0.303612 ({'astro': 0.17912518387787665, 'cort': 0.3948282357909447, 'shsy5y': 0.2205055404548325})\n👀the mAP is so high，you mean mAP score or official metric socre？
", "votes": 0, "replies": [] }, { "id": 1615722, "author_name": "Ioannis M", "author_url": "", "post_date": "2021-12-12T15:44:16.433000", "content": "indeed good job @gunesevitan - would you mind if you share your evaluator setup (and per class scores)
", "votes": 0, "replies": [] }, { "id": 1615723, "author_name": "Hannah B", "author_url": "", "post_date": "2021-12-12T15:44:21.190000", "content": "I think a lot of competitors have shifted to using the mAP metric provided in the livecell github repo
", "votes": 0, "replies": [] }, { "id": 1615757, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2021-12-12T16:36:14.977000", "content": "umm mAP is official metric of this competition?
\n@imeintanis Yes, I can share my evaluator, but the scores above is produced by my inference pipeline which I can't share. My evaluator is very similar to one that is shared. I only added couple lines to store labels in order to calculate per class scores.
\nclass InstanceSegmentationEvaluator(DatasetEvaluator):\n\n def __init__(self, dataset_name, segmentation_format='bitmask'):\n\n dataset = DatasetCatalog.get(dataset_name)\n self.annotations_cache = {item['image_id']: item['annotations'] for item in dataset}\n self.segmentation_format = segmentation_format\n\n def reset(self):\n\n self.scores = []\n self.labels = []\n\n def process(self, inputs, outputs):\n\n for input_, output in zip(inputs, outputs):\n if len(output['instances']) == 0:\n # Set 0 mAP when there are no objects predicted by model\n self.scores.append(0)\n annotation = self.annotations_cache[input_['image_id']]\n label = np.unique(list(map(lambda x: x['category_id'], annotation)))[0]\n self.labels.append(label)\n else:\n # Calculate mAP with predicted objects\n annotation = self.annotations_cache[input_['image_id']]\n prediction_masks = output['instances'].pred_masks.cpu().numpy()\n average_precision = metrics.get_average_precision_detectron(\n ground_truth_masks=annotation,\n prediction_masks=prediction_masks,\n ground_truth_mask_format=self.segmentation_format,\n verbose=False\n )\n self.scores.append(average_precision)\n label = np.unique(list(map(lambda x: x['category_id'], annotation)))[0]\n self.labels.append(label)\n\n def evaluate(self):\n\n df_scores = pd.DataFrame(columns=['scores', 'labels'])\n df_scores['scores'] = np.array(self.scores)\n df_scores['labels'] = np.array(self.labels)\n df_scores = df_scores.groupby('labels')['scores'].mean().to_dict()\n\n return {'mAP': np.mean(self.scores), 'mAP cort': df_scores[0], 'mAP shsy5y': df_scores[1], 'mAP astro': df_scores[2]}\n@gunesevitan how many epochs do you train.
", "votes": 0, "replies": [] } ] }, { "id": 1589283, "author_name": "Roc", "author_url": "", "post_date": "2021-11-20T05:41:48.083000", "content": "detectron: mask_rcnn_r50_fpn(single-fold)
\nlb: 0.299
Update(2021-12-06)
\ndetectron: mask_rcnn_r50_fpn(single-fold)
\ncv: 0.319
\nlb: 0.322
@Roc, what hyper-parameters did you tune in detectron2?
\nI am not getting more than 0.252 in lb
\nWhat is your ensembling strategy for the models from different folds?
Torch Mask-RCNN R50 (single fold)
\nmAP IoU: 0.249 --> LB: 0.266 - 0.268 (depends on thresholds per class) Detectron Mask-RCNN R50 (single fold)
\nmAP IoU: 0.260 --> LB: 0.246 - 0.294 (depends on thresholds per class) Thanks for making this comparison. Performance gap is kinda significant. What is the difference between torchvision and detectron implementations?
", "votes": 1, "replies": [] }, { "id": 1588781, "author_name": "Ioannis M", "author_url": "", "post_date": "2021-11-19T16:16:21.790000", "content": "Indeed, to be honest I haven't dig too much in the details between the two, I started with Torch for my convenience and then moved quickly to Detectron framework, it's the first time I use it so I'm still in exploration mode :)
\nBackbones and fold data are same however epochs/iter and LR schedules were different.
", "votes": 1, "replies": [] } ] }, { "id": 1608131, "author_name": "Qingyao Shuai", "author_url": "", "post_date": "2021-12-06T08:41:25.697000", "content": "library: torchvision
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.303 - LB 0.308
[update]
\nlibrary: detectron2
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.309 - LB 0.319
detectron2: mask rcnn r50 fpn(single-fold)
\nstandard thresholds: 0.291
\ntuned thresholds: 0.301
How to tune Thresholds?
", "votes": 0, "replies": [] }, { "id": 1589842, "author_name": "Ioannis M", "author_url": "", "post_date": "2021-11-20T16:29:02.647000", "content": "you may check this discussion
\nhere
@iwrbeverly Yes. I was inspired by the discussion written by @slawekbiel Threshold by class
", "votes": 1, "replies": [] }, { "id": 1590297, "author_name": "Zhikun Xu", "author_url": "", "post_date": "2021-11-21T04:33:14.937000", "content": "BTW, has anyone tried Swin-Transformer backbone? The resnet backbone is just dominating this competition + fine-tuning tricks/ensemble. I think maybe the backbone still has the room for improvement?
", "votes": 0, "replies": [] }, { "id": 1590968, "author_name": "Hoda", "author_url": "", "post_date": "2021-11-21T19:48:43.003000", "content": "The Swin Transformer I have tried under detectron2 was underperforming.
", "votes": 1, "replies": [] }, { "id": 1591157, "author_name": "Zhikun Xu", "author_url": "", "post_date": "2021-11-22T03:38:39.530000", "content": "@hsadeghian I am trying it under detectron2, too. May I know how much did u score on LB by Swin Transformer? ( I got .295 with this single model + simple tuned thresholds.)
", "votes": 1, "replies": [] }, { "id": 1591169, "author_name": "Hoda", "author_url": "", "post_date": "2021-11-22T03:58:48.413000", "content": "I got less than you, LB=.285 with swinT with tuned thresholds
", "votes": 1, "replies": [] }, { "id": 1596422, "author_name": "yerongg", "author_url": "", "post_date": "2021-11-26T13:35:48.083000", "content": "What is your learning rate set?
", "votes": 0, "replies": [] }, { "id": 1620851, "author_name": "Tian", "author_url": "", "post_date": "2021-12-17T07:10:28.977000", "content": "I can got cv 303, lb303 using SwinT.
", "votes": 1, "replies": [] }, { "id": 1620958, "author_name": "NoChanged", "author_url": "", "post_date": "2021-12-17T09:34:46.100000", "content": "@tiandaye Hi, How do you evaluate model for getting CV 0.303? In training and inference? My CV did not exceed 0.273. Thanks!
", "votes": 0, "replies": [] } ] }, { "id": 1620904, "author_name": "Zaakcii Ru", "author_url": "", "post_date": "2021-12-17T08:15:23.030000", "content": "library: detectron2
\nmodel: mask_rcnn_r50_fpn (single fold)
\nCV 0.273 - LB 0.308
mmdet:cascade_mask_rcnn_r50_fpn(single fold)
\nlb:0.294
model: 5-Layer-U-Net as described by Ronneberger et al. (but no augmentations)
\nLB: 0.177
", "votes": 0, "replies": [] }, { "id": 1597416, "author_name": "", "author_url": "", "post_date": "2021-11-27T13:47:07.987000", "content": "", "votes": 0, "replies": [] }, { "id": 1608286, "author_name": "Burak", "author_url": "", "post_date": "2021-12-06T10:32:59.343000", "content": "thanks for sharing @trushk
", "votes": -2, "replies": [] } ], "raw_markdown_by_id": { "1587877": "Anyone willing to share their best single model? I will go first:\n\n-EDIT- Thanks to the suggestion by @omallo, I am adding more information here\n\n`library`: mmdetection\n`model`: mask_rcnn_r50_fpn (single fold)\n`epochs`: 12\n`img_size`: 1280 x 800\n`lb w/ standard thresholds (0.5 for each class)`: 0.284\n`lb w/ tuned thresholds`: Not tried yet", "1589499": "I think that the number of trained epochs is also relevant for people so they know how much they might have to invest to get a good model so I'm also including that information (others might want to include that as well):\n\nlibrary: detectron2\nmodel: mask_rcnn_x101_fpn (single fold)\nepochs: 50\ntraining time: 8h\ntrain image sizes (short edge, multi scale): 480, 520, 560, 640, 672, 704, 736, 768, 800\ntest image size (short edge): 800\nlb w/ standard thresholds (0.5 for each class): 0.299\nlb w/ tuned thresholds: 0.306", "1595076": "library: mmdetection\nmodel: cascade mask rcnn (single fold)\nepochs: 20\nCV 0.2746 - LB 0.315\nCV 0.2827 - LB 0.320\nCV 0.2880 - LB 0.324\n\n[update-1]\nCV 0.2983 - LB 0.327\nCV 0.2995 - LB 0.328\n\n[update-2]\nCV 0.3002 - LB 0.330\nCV 0.3013 - LB 0.331\nCV 0.3038 - LB 0.332\n\n[update-3]\nCV 0.3047 - LB 0.335\nCV 0.3049 - LB 0.336", "1592871": "library: mmdetection\nmodel: mask_rcnn_r50_fpn (single fold)\nCV 0.301 - LB 0.314", "1596964": "library: mmdetection\nmodel: mask rcnn (single fold)\ntrain_size: 484 images\ntest_size: 122 images\nCV: 0.2587 LB: 0.299\nNo model modified. No preprocessing. No postprocessing except for overlap fixed. ", "1601463": "single model,CV0.3223, LB0.332, detectron2", "1615641": "Detectron Mask R-CNN. Public LB score is 0.32. I'm still trying to figure out an efficient way to ensemble predictions.\n\n```\nFold 1 - mAP: 0.313467 ({'astro': 0.20072195936428244, 'cort': 0.4026399670002614, 'shsy5y': 0.22756584415732165})\nFold 2 - mAP: 0.297655 ({'astro': 0.16545075040715596, 'cort': 0.3963139150087539, 'shsy5y': 0.20485488471168073})\nFold 3 - mAP: 0.306650 ({'astro': 0.1683234036456997, 'cort': 0.4000650243514869, 'shsy5y': 0.22981052684883063})\nFold 4 - mAP: 0.301520 ({'astro': 0.1767558512239449, 'cort': 0.3947693877607034, 'shsy5y': 0.21364495836951708})\nFold 5 - mAP: 0.298685 ({'astro': 0.18354330953728468, 'cort': 0.3803528848335181, 'shsy5y': 0.22665148818681236})\n------------------------------\nOOF mAP: 0.303612 ({'astro': 0.17912518387787665, 'cort': 0.3948282357909447, 'shsy5y': 0.2205055404548325})\n```", "1589283": "detectron: mask_rcnn_r50_fpn(single-fold)\nlb: 0.299\n\nUpdate(2021-12-06)\ndetectron: mask_rcnn_r50_fpn(single-fold)\ncv: 0.319 \nlb: 0.322", "1588420": "Torch Mask-RCNN R50 (single fold) \n- `mAP IoU: 0.249` --> `LB: 0.266 - 0.268` (depends on thresholds per class) \n\nDetectron Mask-RCNN R50 (single fold) \n- `mAP IoU: 0.260` --> `LB: 0.246 - 0.294` (depends on thresholds per class) ", "1608131": "library: torchvision\nmodel: mask_rcnn_r50_fpn (single fold)\nCV 0.303 - LB 0.308\n\n[update]\nlibrary: detectron2\nmodel: mask_rcnn_r50_fpn (single fold)\nCV 0.309 - LB 0.319", "1589541": "detectron2: mask rcnn r50 fpn(single-fold)\nstandard thresholds: 0.291\ntuned thresholds: 0.301", "1620904": "library: detectron2\nmodel: mask_rcnn_r50_fpn (single fold)\nCV 0.273 - LB 0.308", "1588636": "mmdet:cascade_mask_rcnn_r50_fpn(single fold) \nlb:0.294", "1626893": "model: 5-Layer-U-Net as described by Ronneberger et al. (but no augmentations)\n\nLB: 0.177", "1597416": "", "1608286": "thanks for sharing @trushk " } }