{ "id": 297988, "title": "2nd place solution", "url": "/competitions/sartorius-cell-instance-segmentation/discussion/297988", "author_name": "nvnn", "post_date": "2021-12-31T00:36:56.231000", "votes": 146, "comment_count": 77, "views": 0, "content": "

Thanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.

\n

Our solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble.
\n\"\"

\n

We use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below.
\nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.

\n

\"\"

\n

The output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn.

\n

We use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation.

\n

\"\"

\n

\"\"
\nFrom left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.

", "messages": [ { "id": 1633636, "postDate": "2021-12-31T00:36:56.233Z", "content": "

Thanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.

\n

Our solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble.
\n\"\"

\n

We use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below.
\nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.

\n

\"\"

\n

The output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn.

\n

We use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation.

\n

\"\"

\n

\"\"
\nFrom left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.

", "rawMarkdown": "Thanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.\n\nOur solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble. \n![](https://i.ibb.co/WVPHQ93/cell.png)\n\nWe use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below. \nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.\n\n![](https://i.ibb.co/98vHr2N/cell1.png)\n\nThe output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn. \n\nWe use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation. \n\n![](https://i.ibb.co/LPDPsbK/cell2.png)\n\n\n![](https://i.ibb.co/vxK9M3R/s4.jpg)\nFrom left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.", "votes": 146 }, { "id": 1638069, "postDate": "2022-01-04T12:23:02.667Z", "content": "

congratulation and code is really helpful and learned a lot

", "rawMarkdown": "congratulation and code is really helpful and learned a lot", "votes": 8 }, { "id": 1633693, "postDate": "2021-12-31T01:37:29.657Z", "content": "

Impressive solution @nvnnghia

\n

I have a question regarding general approach. How do you arrive at such a solution?

\n

Did you come up with the plan from the beginning and decided to go for it or started with a simpler plan and added components as you go? If you can share more about how your thought process, that would be great!

", "rawMarkdown": "Impressive solution @nvnnghia \n\nI have a question regarding general approach. How do you arrive at such a solution?\n\nDid you come up with the plan from the beginning and decided to go for it or started with a simpler plan and added components as you go? If you can share more about how your thought process, that would be great!", "votes": 8, "replies": [ { "id": 1633704, "postDate": "2021-12-31T01:56:02.410Z", "content": "

Thanks for the question @yousof9
\nwe started with simpler approach. At the beginning sheep use a maskrcnn and yolov5 while my approach is yolov5 and unet. After teaming up we realized we lose global information if we use only unet on the cropped cell, then we tried to ensemble both unet (for better local feature) and maskrcnn (for better global feature). Later on we notice that the detection model is the key to have a good score, so we focus on improving object detection part by tuning and adding more detection model.

", "rawMarkdown": "Thanks for the question @yousof9 \nwe started with simpler approach. At the beginning sheep use a maskrcnn and yolov5 while my approach is yolov5 and unet. After teaming up we realized we lose global information if we use only unet on the cropped cell, then we tried to ensemble both unet (for better local feature) and maskrcnn (for better global feature). Later on we notice that the detection model is the key to have a good score, so we focus on improving object detection part by tuning and adding more detection model.", "votes": 18 }, { "id": 1633709, "postDate": "2021-12-31T02:01:01.827Z", "content": "

Amazing, exactly what I was looking for.

\n

Thanks for sharing @nvnnghia and what a way to end the year. All the best in 2022!

", "rawMarkdown": "Amazing, exactly what I was looking for.\n\nThanks for sharing @nvnnghia and what a way to end the year. All the best in 2022!", "votes": 1 }, { "id": 1633715, "postDate": "2021-12-31T02:05:29.503Z", "content": "

There is another reason for doing such heavy ensembles with object detection models: There are some broken mask annotion and the ratio is not low, thus the mask-head of maskrcnn and other one-stage instance segmentation model may get higher noise then normal situation. But the bbox is correct, so we put more OD model then usual.

", "rawMarkdown": "There is another reason for doing such heavy ensembles with object detection models: There are some broken mask annotion and the ratio is not low, thus the mask-head of maskrcnn and other one-stage instance segmentation model may get higher noise then normal situation. But the bbox is correct, so we put more OD model then usual.", "votes": 9 } ] }, { "id": 1634634, "postDate": "2021-12-31T23:37:32.940Z", "content": "

Congratulations Nvnn and Sheep! Fantastic pipeline. Great combination of models.

", "rawMarkdown": "Congratulations Nvnn and Sheep! Fantastic pipeline. Great combination of models.", "votes": 5 }, { "id": 1634763, "postDate": "2022-01-01T06:23:14.103Z", "content": "

An enlightening solution with a pretty simple way to understand! Really learned a lot👍 @nvnnghia

", "rawMarkdown": "An enlightening solution with a pretty simple way to understand! Really learned a lot👍 @nvnnghia", "votes": 3 }, { "id": 1633690, "postDate": "2021-12-31T01:32:20.073Z", "content": "

Congrats on 2nd place @nvnnghia and @steamedsheep, nice solution. Thanks for the writeup!

", "rawMarkdown": "Congrats on 2nd place @nvnnghia and @steamedsheep, nice solution. Thanks for the writeup!", "votes": 3, "replies": [ { "id": 1633699, "postDate": "2021-12-31T01:46:38.697Z", "content": "

thank you

", "rawMarkdown": "thank you\n" } ] }, { "id": 1636936, "postDate": "2022-01-03T11:49:32.853Z", "content": "

Thank you so much :) This opened me new doors to think about these types of problems.

", "rawMarkdown": "Thank you so much :) This opened me new doors to think about these types of problems.", "votes": 1 }, { "id": 1636734, "postDate": "2022-01-03T08:18:31.317Z", "content": "

Congratulations, Good solution

", "rawMarkdown": "Congratulations, Good solution", "votes": 1 }, { "id": 1634989, "postDate": "2022-01-01T11:25:04.530Z", "content": "

@nvnnghia - Many Congratulations !!
\nThanks so much not just for posting your solution but for the way you have done so. It has allowed me to gather a lot of insight ! I do have a couple of questions though,

\n", "rawMarkdown": "@nvnnghia - Many Congratulations !!\nThanks so much not just for posting your solution but for the way you have done so. It has allowed me to gather a lot of insight ! I do have a couple of questions though,\n- Did you do all your training on Kaggle Kernels ? If not can you let us know what kind of hardware you are using. \n- How many epochs of training did you require in each stage to get to a decent score ? ", "votes": 1, "replies": [ { "id": 1634996, "postDate": "2022-01-01T11:39:13.563Z", "content": "", "rawMarkdown": "- I use 2X 3090 in this competition. \n- I train 50 epoch in pretraining stage and 100 epoch fine tuning with competition data.", "votes": 1 }, { "id": 1635023, "postDate": "2022-01-01T12:18:32.260Z", "content": "

Thanks so much for the response !!

", "rawMarkdown": "Thanks so much for the response !!", "votes": 1 } ] }, { "id": 1634122, "postDate": "2021-12-31T11:13:02.620Z", "content": "

Great solution. Congrats on 2nd place!!

", "rawMarkdown": "Great solution. Congrats on 2nd place!!", "votes": 1 }, { "id": 1634091, "postDate": "2021-12-31T10:59:30.260Z", "content": "

Congratulations, nice and awesome solution.

", "rawMarkdown": "Congratulations, nice and awesome solution.", "votes": 1 }, { "id": 1634001, "postDate": "2021-12-31T08:46:20.880Z", "content": "

Impressive solution, congrats on 2nd place! 🙌

", "rawMarkdown": "Impressive solution, congrats on 2nd place! 🙌", "votes": 1 }, { "id": 1633888, "postDate": "2021-12-31T06:09:47.880Z", "content": "

Impressive solution. Congratulations author. \"center cell and neighbor cell as 2-class segmentation\".Did you make the dataset yourself? What form is the dataset？

", "rawMarkdown": "Impressive solution. Congratulations author. \"center cell and neighbor cell as 2-class segmentation\".Did you make the dataset yourself? What form is the dataset？", "votes": 1, "replies": [ { "id": 1634024, "postDate": "2021-12-31T09:20:18.890Z", "content": "

basically this

\n
class CellDataset(Dataset):\n    def __init__(self, cfg, df, tfms=None, fold_id = 0, is_train = True):\n        super().__init__()\n        self.df = df.reset_index(drop=True)\n        self.cfg = cfg\n        self.fold_id = fold_id\n        self.transform = tfms\n        self.is_train = is_train\n\n    def __getitem__(self, index):\n        row = self.df.loc[index]\n        img_id = row.id \n\n        img_path = f'{self.cfg.data_dir}/train/{img_id}.png'\n        assert os.path.isfile(img_path), f'{img_path}!!!!'\n        image = cv2.imread(img_path)\n        im_h, im_w = image.shape[:2]\n\n        mask = rle_decode(row.annotation, (row.height, row.width))\n\n        all_mask = cv2.imread(f'{self.cfg.mask_dir}/{img_id}.png',0)\n        all_mask = np.uint8(1*(all_mask>0))\n\n        ys, xs = np.where(mask)\n        x1, x2 = min(xs), max(xs)\n        y1, y2 = min(ys), max(ys)\n\n        cr_w = x2 - x1 \n        cr_h = y2 - y1\n        shift_ratio = 0.1\n        if self.is_train:\n            pad_left = random.randint(-1*int(shift_ratio*cr_w), int(shift_ratio*cr_w))\n            pad_right = random.randint(-1*int(shift_ratio*cr_w), int(shift_ratio*cr_w))\n            pad_top = random.randint(-1*int(shift_ratio*cr_h), int(shift_ratio*cr_h))\n            pad_bot = random.randint(-1*int(shift_ratio*cr_h), int(shift_ratio*cr_h))\n        else:\n            pad_left = 0\n            pad_right = 0\n            pad_top = 0\n            pad_bot = 0\n\n        cr_x1 = round(max(0, x1-pad_left))\n        cr_y1 = round(max(0, y1-pad_top))\n        cr_x2 = round(min(im_w-1, x2+pad_right))\n        cr_y2 = round(min(im_h-1, y2+pad_bot))\n\n        image = image[cr_y1:cr_y2, cr_x1:cr_x2]\n        mask = mask[cr_y1:cr_y2, cr_x1:cr_x2]\n\n\n        all_mask = all_mask[cr_y1:cr_y2, cr_x1:cr_x2]\n        all_mask[mask>0] = 0\n\n        mask = np.stack([mask,all_mask]).transpose(1,2,0)\n\n        if self.transform is not None:\n            res = self.transform(image=image, mask=mask)\n            mask = res[\"mask\"]\n            image = res['image']\n\n        image = image.transpose(2,0,1)\n        image = image/255\n\n        mask = mask.transpose(2,0,1)\n\n        return torch.from_numpy(image), torch.from_numpy(mask)\n\n\n    def __len__(self):\n        return len(self.df)\n
", "rawMarkdown": "basically this\n\n```\nclass CellDataset(Dataset):\n def __init__(self, cfg, df, tfms=None, fold_id = 0, is_train = True):\n super().__init__()\n self.df = df.reset_index(drop=True)\n self.cfg = cfg\n self.fold_id = fold_id\n self.transform = tfms\n self.is_train = is_train\n\n def __getitem__(self, index):\n row = self.df.loc[index]\n img_id = row.id \n\n img_path = f'{self.cfg.data_dir}/train/{img_id}.png'\n assert os.path.isfile(img_path), f'{img_path}!!!!'\n image = cv2.imread(img_path)\n im_h, im_w = image.shape[:2]\n\n mask = rle_decode(row.annotation, (row.height, row.width))\n\n all_mask = cv2.imread(f'{self.cfg.mask_dir}/{img_id}.png',0)\n all_mask = np.uint8(1*(all_mask>0))\n\n ys, xs = np.where(mask)\n x1, x2 = min(xs), max(xs)\n y1, y2 = min(ys), max(ys)\n\n cr_w = x2 - x1 \n cr_h = y2 - y1\n shift_ratio = 0.1\n if self.is_train:\n pad_left = random.randint(-1*int(shift_ratio*cr_w), int(shift_ratio*cr_w))\n pad_right = random.randint(-1*int(shift_ratio*cr_w), int(shift_ratio*cr_w))\n pad_top = random.randint(-1*int(shift_ratio*cr_h), int(shift_ratio*cr_h))\n pad_bot = random.randint(-1*int(shift_ratio*cr_h), int(shift_ratio*cr_h))\n else:\n pad_left = 0\n pad_right = 0\n pad_top = 0\n pad_bot = 0\n\n cr_x1 = round(max(0, x1-pad_left))\n cr_y1 = round(max(0, y1-pad_top))\n cr_x2 = round(min(im_w-1, x2+pad_right))\n cr_y2 = round(min(im_h-1, y2+pad_bot))\n\n image = image[cr_y1:cr_y2, cr_x1:cr_x2]\n mask = mask[cr_y1:cr_y2, cr_x1:cr_x2]\n\n\n all_mask = all_mask[cr_y1:cr_y2, cr_x1:cr_x2]\n all_mask[mask>0] = 0\n\n mask = np.stack([mask,all_mask]).transpose(1,2,0)\n\n if self.transform is not None:\n res = self.transform(image=image, mask=mask)\n mask = res[\"mask\"]\n image = res['image']\n\n image = image.transpose(2,0,1)\n image = image/255\n\n mask = mask.transpose(2,0,1)\n\n return torch.from_numpy(image), torch.from_numpy(mask)\n\n \n def __len__(self):\n return len(self.df)\n```", "votes": 4 }, { "id": 1638045, "postDate": "2022-01-04T11:59:28.210Z", "content": "

Thank you for sharing.

", "rawMarkdown": "Thank you for sharing." } ] }, { "id": 1633762, "postDate": "2021-12-31T03:26:53.517Z", "content": "

Congrats! Are you going to share the code? @nvnnghia

", "rawMarkdown": "Congrats! Are you going to share the code? @nvnnghia ", "votes": 1, "replies": [ { "id": 1633849, "postDate": "2021-12-31T05:26:12.463Z", "content": "

hi, we don't have a plan to share the code yet. if we do, I will update it in the post later.

", "rawMarkdown": "hi, we don't have a plan to share the code yet. if we do, I will update it in the post later.", "votes": 1 } ] }, { "id": 1633754, "postDate": "2021-12-31T03:07:48.180Z", "content": "

Congrats on getting 2nd place with an impressive solution!

", "rawMarkdown": "Congrats on getting 2nd place with an impressive solution! ", "votes": 1 }, { "id": 1633714, "postDate": "2021-12-31T02:04:38.577Z", "content": "

Congratulations on 2nd
\nI took the same approach initially but the results were not promising mostly due to the neighbor cells becoming part of the mask. It never clicked to me to take the neighbor cells as a different class mask in itself.

\n

Impressive solution. Great to see that this approach worked and glad to be proved wrong :)

", "rawMarkdown": "Congratulations on 2nd\nI took the same approach initially but the results were not promising mostly due to the neighbor cells becoming part of the mask. It never clicked to me to take the neighbor cells as a different class mask in itself. \n\nImpressive solution. Great to see that this approach worked and glad to be proved wrong :)\n", "votes": 1 }, { "id": 1633655, "postDate": "2021-12-31T00:59:07.863Z", "content": "

Very impressive and powerful solution! Happy new year to you, sir!

", "rawMarkdown": "Very impressive and powerful solution! Happy new year to you, sir!", "votes": 1 }, { "id": 1634701, "postDate": "2022-01-01T03:38:45.923Z", "content": "

Awesome solution! Congratulations 🎉

", "rawMarkdown": "Awesome solution! Congratulations 🎉", "votes": 2 }, { "id": 1634239, "postDate": "2021-12-31T12:55:44.167Z", "content": "

Great Work ! Congrats on getting 2nd place.

", "rawMarkdown": "Great Work ! Congrats on getting 2nd place.", "votes": 2 }, { "id": 1634032, "postDate": "2021-12-31T09:40:18.407Z", "content": "

Congratz, very nice solution ! Simple to understand yet cleverly designed and far from easy to come up with.

\n

We knew you would perform very well as soon as we saw you merging, and it was only a matter a time before we were overtaken :)

", "rawMarkdown": "Congratz, very nice solution ! Simple to understand yet cleverly designed and far from easy to come up with.\n\nWe knew you would perform very well as soon as we saw you merging, and it was only a matter a time before we were overtaken :)", "votes": 2, "replies": [ { "id": 1634130, "postDate": "2021-12-31T11:16:42.920Z", "content": "

thank you

", "rawMarkdown": "thank you ", "votes": 1 } ] }, { "id": 1633933, "postDate": "2021-12-31T07:17:46.043Z", "content": "

Congratulations, awesome solution!

", "rawMarkdown": "Congratulations, awesome solution!", "votes": 2 }, { "id": 1633684, "postDate": "2021-12-31T01:23:49.983Z", "content": "

@nvnnghia Congrats. btw did you guys do anything to handle the varying sizes of cropped-cell? What image size did you use to train the final Unet ?

", "rawMarkdown": "@nvnnghia Congrats. btw did you guys do anything to handle the varying sizes of **cropped-cell**? What image size did you use to train the final **Unet** ?", "votes": 2, "replies": [ { "id": 1633698, "postDate": "2021-12-31T01:44:56.423Z", "content": "

I just resize all cell to a fixed size 128x128 for unet.

", "rawMarkdown": "I just resize all cell to a fixed size 128x128 for unet." } ] }, { "id": 1633646, "postDate": "2021-12-31T00:49:20.943Z", "content": "

Nicely done. method didn't not work for me: you had a strong first stage. I was missing the predicted neighboor cell on second stage unet.

", "rawMarkdown": "Nicely done. method didn't not work for me: you had a strong first stage. I was missing the predicted neighboor cell on second stage unet.", "votes": 2, "replies": [ { "id": 1633652, "postDate": "2021-12-31T00:54:52.080Z", "content": "

thanks, the detection part is very important. predicted neighboor cell does not help too much.

", "rawMarkdown": "thanks, the detection part is very important. predicted neighboor cell does not help too much. " } ] }, { "id": 1638094, "postDate": "2022-01-04T12:59:49.533Z", "content": "

日本語訳
\nThanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.

\n

Our solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble.

\n

次の図に示すように、私たちのソリューションは、2つのオブジェクト検出モデル、1つのunetモデル、および2つのmaskrcnnモデルのアンサンブルです。 @sheepがmaskrcnnとアンサンブルに焦点を当てている間、私はオブジェクト検出とUnetを担当しています。

\n

We use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below.
\nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.

\n

オブジェクト検出タスクにはyolov5x6とeffdetD3を使用します。 下の図に示すように、トレーニング手順は両方のモデルで同じです。
\nモデルは、競合データで微調整する前に、Livecellおよびtrain-semi-supervisedデータセットで数ラウンドトレーニングされます。 推論フェーズでは、2つのモデルの出力ボックスがWBFを使用してmaskrcnnボックスとアンサンブルされます。

\n

The output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn.

\n

We use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation.

\n

WBF後の出力ボックスは、各ボックスのセグメンテーションマスクを取得するために、UnetおよびMaskrcnn（マスクヘッド）にフィードされます。 加重平均を使用して、UnetとMaskrcnnの生のマスクをアンサンブルします。

\n

効率的なb5エンコーダーを備えたunetを使用して、トリミングされたセルのセグメンテーションを実行します。 トリミングされたセルには隣接セルが含まれることがあるため、中央のセルと隣接セルのマスクを2クラスのセグメンテーションとして予測します。

\n

From left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.

", "rawMarkdown": "日本語訳\nThanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.\n\nOur solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble.\n\n次の図に示すように、私たちのソリューションは、2つのオブジェクト検出モデル、1つのunetモデル、および2つのmaskrcnnモデルのアンサンブルです。 @sheepがmaskrcnnとアンサンブルに焦点を当てている間、私はオブジェクト検出とUnetを担当しています。\n\nWe use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below.\nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.\n\nオブジェクト検出タスクにはyolov5x6とeffdetD3を使用します。 下の図に示すように、トレーニング手順は両方のモデルで同じです。\nモデルは、競合データで微調整する前に、Livecellおよびtrain-semi-supervisedデータセットで数ラウンドトレーニングされます。 推論フェーズでは、2つのモデルの出力ボックスがWBFを使用してmaskrcnnボックスとアンサンブルされます。\n\n\nThe output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn.\n\nWe use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation.\n\nWBF後の出力ボックスは、各ボックスのセグメンテーションマスクを取得するために、UnetおよびMaskrcnn（マスクヘッド）にフィードされます。 加重平均を使用して、UnetとMaskrcnnの生のマスクをアンサンブルします。\n\n効率的なb5エンコーダーを備えたunetを使用して、トリミングされたセルのセグメンテーションを実行します。 トリミングされたセルには隣接セルが含まれることがあるため、中央のセルと隣接セルのマスクを2クラスのセグメンテーションとして予測します。\n\n\nFrom left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.", "votes": -1 }, { "id": 1635384, "postDate": "2022-01-01T17:57:38.633Z", "content": "

Congratulations !great work

", "rawMarkdown": "Congratulations !great work", "votes": -1 }, { "id": 2165081, "postDate": "2023-03-02T01:22:08.633Z", "content": "

congratulation and code is really helpful and learned a lot

", "rawMarkdown": "congratulation and code is really helpful and learned a lot" }, { "id": 1643787, "postDate": "2022-01-09T18:16:44.987Z", "content": "

Nice job, Nvnn and Sheep.
\nAlready, I can find one or two ideas I could use from this.
\nCongratulations!
\nI don't see much difference between first and second place winners, anyway.

", "rawMarkdown": "Nice job, Nvnn and Sheep.\nAlready, I can find one or two ideas I could use from this.\nCongratulations!\nI don't see much difference between first and second place winners, anyway." }, { "id": 1642984, "postDate": "2022-01-08T21:14:15.477Z", "content": "

Congrats again.
\nI have question about the second stage training: what image size are you using for the unet? How do you constitute your train set: are you taking crop of the original train set or are you resizing every cell of the original dataset to the size of unet?

", "rawMarkdown": "Congrats again.\nI have question about the second stage training: what image size are you using for the unet? How do you constitute your train set: are you taking crop of the original train set or are you resizing every cell of the original dataset to the size of unet?" }, { "id": 1642474, "postDate": "2022-01-08T12:24:34.523Z", "content": "

great solution Congratulations buddy
\ngreat team work

", "rawMarkdown": "great solution Congratulations buddy\ngreat team work " }, { "id": 1642306, "postDate": "2022-01-08T08:51:36.500Z", "content": "

Awesome solution! Congratulations 🎉.

", "rawMarkdown": "Awesome solution! Congratulations 🎉." }, { "id": 1638095, "postDate": "2022-01-04T13:00:44.110Z", "content": "

good one.congrats.

", "rawMarkdown": "good one.congrats." }, { "id": 1637671, "postDate": "2022-01-04T05:40:52.190Z", "content": "

Great work.

", "rawMarkdown": "Great work." }, { "id": 1637022, "postDate": "2022-01-03T13:36:26.577Z", "content": "

Well Done! Congratulations

", "rawMarkdown": "Well Done! Congratulations" }, { "id": 1636577, "postDate": "2022-01-03T04:39:50.430Z", "content": "

Great combination of models, thank you and congratulations

", "rawMarkdown": "Great combination of models, thank you and congratulations" }, { "id": 1636430, "postDate": "2022-01-02T21:47:37.280Z", "content": "

Impressive solution, congratulations! Truly inspiring :)

", "rawMarkdown": "Impressive solution, congratulations! Truly inspiring :)" }, { "id": 1636148, "postDate": "2022-01-02T16:44:23.330Z", "content": "

Wow, thank you for this useful information! and congratulations!

", "rawMarkdown": "Wow, thank you for this useful information! and congratulations!" }, { "id": 1635968, "postDate": "2022-01-02T13:02:21.850Z", "content": "

Awesome solution! Congratulations 🎉

", "rawMarkdown": "Awesome solution! Congratulations 🎉\n\n" }, { "id": 1635913, "postDate": "2022-01-02T11:18:16.060Z", "content": "

Great work!

", "rawMarkdown": "Great work!" }, { "id": 1635837, "postDate": "2022-01-02T08:50:21.023Z", "content": "

Awesome solution ! Congrats for the 2nd place !

", "rawMarkdown": "Awesome solution ! Congrats for the 2nd place !" }, { "id": 1635296, "postDate": "2022-01-01T16:50:34.977Z", "content": "

Congrats! Very impressive solution

", "rawMarkdown": "Congrats! Very impressive solution" }, { "id": 1635174, "postDate": "2022-01-01T14:37:08.080Z", "content": "

Congraulations!
\nMay I ask which library you used for the implementation?

", "rawMarkdown": "Congraulations!\nMay I ask which library you used for the implementation?\n\n" }, { "id": 1635020, "postDate": "2022-01-01T12:12:18.127Z", "content": "

very creative!

", "rawMarkdown": "very creative!\n" }, { "id": 1634864, "postDate": "2022-01-01T09:10:00.657Z", "content": "

Congratulations

", "rawMarkdown": "Congratulations" }, { "id": 1634025, "postDate": "2021-12-31T09:21:20.283Z", "content": "

Congrats to u!
\nGot a question here. Just in case of prudence. The output of a unet or mask head is a 0 and 1 value map. In the last picture it appears to be float values between.Or did u post the output before softmax actully?

", "rawMarkdown": "Congrats to u!\nGot a question here. Just in case of prudence. The output of a unet or mask head is a 0 and 1 value map. In the last picture it appears to be float values between.Or did u post the output before softmax actully?", "replies": [ { "id": 1634027, "postDate": "2021-12-31T09:25:45.393Z", "content": "

the output mask of unet or maskrcnn is raw pixel values (the value is in range [0-1]).

", "rawMarkdown": "the output mask of unet or maskrcnn is raw pixel values (the value is in range [0-1]). " }, { "id": 1634031, "postDate": "2021-12-31T09:37:18.997Z", "content": "

Thank you. And could you share about self-supervise trainning? Do u just choose higher scoring unlabel images or is there any tricky thing because the cells are distributed very unbalancedly as well as with large counts, which we don't usually have to cope with in common instance detection task.

", "rawMarkdown": "Thank you. And could you share about self-supervise trainning? Do u just choose higher scoring unlabel images or is there any tricky thing because the cells are distributed very unbalancedly as well as with large counts, which we don't usually have to cope with in common instance detection task." }, { "id": 1634137, "postDate": "2021-12-31T11:22:06.273Z", "content": "

we use all unlabel images. We actually didn't focus on semi-supervised learning, maybe your method can work a little better.

", "rawMarkdown": "we use all unlabel images. We actually didn't focus on semi-supervised learning, maybe your method can work a little better. " } ] }, { "id": 1633929, "postDate": "2021-12-31T07:11:57.953Z", "content": "

Wow @nvnnghia great approach, this is really impressive. btw what is WBF?
\nAre you guys planning to publish the code? would love to check that out. Thank you.

", "rawMarkdown": "Wow @nvnnghia great approach, this is really impressive. btw what is `WBF`?\nAre you guys planning to publish the code? would love to check that out. Thank you.", "replies": [ { "id": 1634020, "postDate": "2021-12-31T09:18:10.187Z", "content": "

WBF stands for Weighted Boxes Fusion. It is a method to ensemble object detection models. We have no plan for the code yet.

", "rawMarkdown": "WBF stands for Weighted Boxes Fusion. It is a method to ensemble object detection models. We have no plan for the code yet.", "votes": 2 }, { "id": 1634220, "postDate": "2021-12-31T12:38:44.667Z", "content": "

Oh I see, thank you for answering.

", "rawMarkdown": "Oh I see, thank you for answering." } ] }, { "id": 1633904, "postDate": "2021-12-31T06:36:39.770Z", "content": "

Congratulations @nvnnghia, the solution is indeed impressive!

", "rawMarkdown": "Congratulations @nvnnghia, the solution is indeed impressive!" }, { "id": 1633783, "postDate": "2021-12-31T04:07:00.440Z", "content": "

Congrats…

", "rawMarkdown": "Congrats..." }, { "id": 1633737, "postDate": "2021-12-31T02:41:42.160Z", "content": "

Great Work

", "rawMarkdown": "Great Work" }, { "id": 1633730, "postDate": "2021-12-31T02:25:24.370Z", "content": "

Congrats on 2nd place! May I ask what the learning rates were used during pretraining and finetuning stages?

", "rawMarkdown": "Congrats on 2nd place! May I ask what the learning rates were used during pretraining and finetuning stages?", "replies": [ { "id": 1633850, "postDate": "2021-12-31T05:29:06.430Z", "content": "

hi, we use the same learning rate in both stages. default yolov5 lr and 3e-5 for effdetD3

", "rawMarkdown": "hi, we use the same learning rate in both stages. default yolov5 lr and 3e-5 for effdetD3", "votes": 1 } ] }, { "id": 1636752, "postDate": "2022-01-03T08:41:06.743Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1636301, "postDate": "2022-01-02T18:32:24.770Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1636124, "postDate": "2022-01-02T16:04:52.173Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1633653, "postDate": "2021-12-31T00:57:34.803Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1633650, "postDate": "2021-12-31T00:53:33.307Z", "content": "

Nice work !!! Thanks for sharing it.

", "rawMarkdown": "Nice work !!! Thanks for sharing it. ", "votes": 1 }, { "id": 1633712, "postDate": "2021-12-31T02:01:54.663Z", "content": "

nice solution. Thank you.

", "rawMarkdown": "nice solution. Thank you." }, { "id": 1639144, "postDate": "2022-01-05T12:25:27.293Z", "content": "

Thank you for sharing.

", "rawMarkdown": "Thank you for sharing." }, { "id": 1638669, "postDate": "2022-01-04T23:40:19.017Z", "content": "

Thank you for sharing

", "rawMarkdown": "Thank you for sharing" }, { "id": 1637461, "postDate": "2022-01-03T22:57:38.627Z", "content": "

Good solutions, thanks

", "rawMarkdown": "Good solutions, thanks" }, { "id": 1636700, "postDate": "2022-01-03T07:14:39.437Z", "content": "

Thank you for your pipeline

", "rawMarkdown": "Thank you for your pipeline" }, { "id": 1636422, "postDate": "2022-01-02T21:27:17.927Z", "content": "

Thanks for posting!

", "rawMarkdown": "Thanks for posting!" } ], "comments": [ { "id": 1638069, "author_name": "Dev Ansodariya", "author_url": "", "post_date": "2022-01-04T12:23:02.667000", "content": "

congratulation and code is really helpful and learned a lot

", "votes": 8, "replies": [] }, { "id": 1633693, "author_name": "Yousef Rabi", "author_url": "", "post_date": "2021-12-31T01:37:29.657000", "content": "

Impressive solution @nvnnghia

\n

I have a question regarding general approach. How do you arrive at such a solution?

\n

Did you come up with the plan from the beginning and decided to go for it or started with a simpler plan and added components as you go? If you can share more about how your thought process, that would be great!

", "votes": 8, "replies": [ { "id": 1633704, "author_name": "nvnn", "author_url": "", "post_date": "2021-12-31T01:56:02.410000", "content": "

Thanks for the question @yousof9
\nwe started with simpler approach. At the beginning sheep use a maskrcnn and yolov5 while my approach is yolov5 and unet. After teaming up we realized we lose global information if we use only unet on the cropped cell, then we tried to ensemble both unet (for better local feature) and maskrcnn (for better global feature). Later on we notice that the detection model is the key to have a good score, so we focus on improving object detection part by tuning and adding more detection model.

", "votes": 18, "replies": [] }, { "id": 1633709, "author_name": "Yousef Rabi", "author_url": "", "post_date": "2021-12-31T02:01:01.827000", "content": "

Amazing, exactly what I was looking for.

\n

Thanks for sharing @nvnnghia and what a way to end the year. All the best in 2022!

", "votes": 1, "replies": [] }, { "id": 1633715, "author_name": "sheep", "author_url": "", "post_date": "2021-12-31T02:05:29.503000", "content": "

There is another reason for doing such heavy ensembles with object detection models: There are some broken mask annotion and the ratio is not low, thus the mask-head of maskrcnn and other one-stage instance segmentation model may get higher noise then normal situation. But the bbox is correct, so we put more OD model then usual.

", "votes": 9, "replies": [] } ] }, { "id": 1634634, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-12-31T23:37:32.940000", "content": "

Congratulations Nvnn and Sheep! Fantastic pipeline. Great combination of models.

", "votes": 5, "replies": [] }, { "id": 1634763, "author_name": "ForcewithMe", "author_url": "", "post_date": "2022-01-01T06:23:14.103000", "content": "

An enlightening solution with a pretty simple way to understand! Really learned a lot👍 @nvnnghia

", "votes": 3, "replies": [] }, { "id": 1633690, "author_name": "KhanhVD", "author_url": "", "post_date": "2021-12-31T01:32:20.073000", "content": "

Congrats on 2nd place @nvnnghia and @steamedsheep, nice solution. Thanks for the writeup!

", "votes": 3, "replies": [ { "id": 1633699, "author_name": "nvnn", "author_url": "", "post_date": "2021-12-31T01:46:38.697000", "content": "

thank you

", "votes": 0, "replies": [] } ] }, { "id": 1636936, "author_name": "B3d1r", "author_url": "", "post_date": "2022-01-03T11:49:32.853000", "content": "

Thank you so much :) This opened me new doors to think about these types of problems.

", "votes": 1, "replies": [] }, { "id": 1636734, "author_name": "hotdog1029", "author_url": "", "post_date": "2022-01-03T08:18:31.317000", "content": "

Congratulations, Good solution

", "votes": 1, "replies": [] }, { "id": 1634989, "author_name": "Arjun", "author_url": "", "post_date": "2022-01-01T11:25:04.530000", "content": "

@nvnnghia - Many Congratulations !!
\nThanks so much not just for posting your solution but for the way you have done so. It has allowed me to gather a lot of insight ! I do have a couple of questions though,

\n", "votes": 1, "replies": [ { "id": 1634996, "author_name": "nvnn", "author_url": "", "post_date": "2022-01-01T11:39:13.563000", "content": "", "votes": 1, "replies": [] }, { "id": 1635023, "author_name": "Arjun", "author_url": "", "post_date": "2022-01-01T12:18:32.260000", "content": "

Thanks so much for the response !!

", "votes": 1, "replies": [] } ] }, { "id": 1634122, "author_name": "Ichimaru Gin", "author_url": "", "post_date": "2021-12-31T11:13:02.620000", "content": "

Great solution. Congrats on 2nd place!!

", "votes": 1, "replies": [] }, { "id": 1634091, "author_name": "Aditya Sharma", "author_url": "", "post_date": "2021-12-31T10:59:30.260000", "content": "

Congratulations, nice and awesome solution.

", "votes": 1, "replies": [] }, { "id": 1634001, "author_name": "Niek van der Zwaag", "author_url": "", "post_date": "2021-12-31T08:46:20.880000", "content": "

Impressive solution, congrats on 2nd place! 🙌

", "votes": 1, "replies": [] }, { "id": 1633888, "author_name": "Jingyq", "author_url": "", "post_date": "2021-12-31T06:09:47.880000", "content": "

Impressive solution. Congratulations author. \"center cell and neighbor cell as 2-class segmentation\".Did you make the dataset yourself? What form is the dataset？

", "votes": 1, "replies": [ { "id": 1634024, "author_name": "nvnn", "author_url": "", "post_date": "2021-12-31T09:20:18.890000", "content": "

basically this

\n
class CellDataset(Dataset):\n    def __init__(self, cfg, df, tfms=None, fold_id = 0, is_train = True):\n        super().__init__()\n        self.df = df.reset_index(drop=True)\n        self.cfg = cfg\n        self.fold_id = fold_id\n        self.transform = tfms\n        self.is_train = is_train\n\n    def __getitem__(self, index):\n        row = self.df.loc[index]\n        img_id = row.id \n\n        img_path = f'{self.cfg.data_dir}/train/{img_id}.png'\n        assert os.path.isfile(img_path), f'{img_path}!!!!'\n        image = cv2.imread(img_path)\n        im_h, im_w = image.shape[:2]\n\n        mask = rle_decode(row.annotation, (row.height, row.width))\n\n        all_mask = cv2.imread(f'{self.cfg.mask_dir}/{img_id}.png',0)\n        all_mask = np.uint8(1*(all_mask>0))\n\n        ys, xs = np.where(mask)\n        x1, x2 = min(xs), max(xs)\n        y1, y2 = min(ys), max(ys)\n\n        cr_w = x2 - x1 \n        cr_h = y2 - y1\n        shift_ratio = 0.1\n        if self.is_train:\n            pad_left = random.randint(-1*int(shift_ratio*cr_w), int(shift_ratio*cr_w))\n            pad_right = random.randint(-1*int(shift_ratio*cr_w), int(shift_ratio*cr_w))\n            pad_top = random.randint(-1*int(shift_ratio*cr_h), int(shift_ratio*cr_h))\n            pad_bot = random.randint(-1*int(shift_ratio*cr_h), int(shift_ratio*cr_h))\n        else:\n            pad_left = 0\n            pad_right = 0\n            pad_top = 0\n            pad_bot = 0\n\n        cr_x1 = round(max(0, x1-pad_left))\n        cr_y1 = round(max(0, y1-pad_top))\n        cr_x2 = round(min(im_w-1, x2+pad_right))\n        cr_y2 = round(min(im_h-1, y2+pad_bot))\n\n        image = image[cr_y1:cr_y2, cr_x1:cr_x2]\n        mask = mask[cr_y1:cr_y2, cr_x1:cr_x2]\n\n\n        all_mask = all_mask[cr_y1:cr_y2, cr_x1:cr_x2]\n        all_mask[mask>0] = 0\n\n        mask = np.stack([mask,all_mask]).transpose(1,2,0)\n\n        if self.transform is not None:\n            res = self.transform(image=image, mask=mask)\n            mask = res[\"mask\"]\n            image = res['image']\n\n        image = image.transpose(2,0,1)\n        image = image/255\n\n        mask = mask.transpose(2,0,1)\n\n        return torch.from_numpy(image), torch.from_numpy(mask)\n\n\n    def __len__(self):\n        return len(self.df)\n
", "votes": 4, "replies": [] }, { "id": 1638045, "author_name": "Jingyq", "author_url": "", "post_date": "2022-01-04T11:59:28.210000", "content": "

Thank you for sharing.

", "votes": 0, "replies": [] } ] }, { "id": 1633762, "author_name": "Johnp", "author_url": "", "post_date": "2021-12-31T03:26:53.517000", "content": "

Congrats! Are you going to share the code? @nvnnghia

", "votes": 1, "replies": [ { "id": 1633849, "author_name": "nvnn", "author_url": "", "post_date": "2021-12-31T05:26:12.463000", "content": "

hi, we don't have a plan to share the code yet. if we do, I will update it in the post later.

", "votes": 1, "replies": [] } ] }, { "id": 1633754, "author_name": "Jerry Zhu", "author_url": "", "post_date": "2021-12-31T03:07:48.180000", "content": "

Congrats on getting 2nd place with an impressive solution!

", "votes": 1, "replies": [] }, { "id": 1633714, "author_name": "bendang", "author_url": "", "post_date": "2021-12-31T02:04:38.577000", "content": "

Congratulations on 2nd
\nI took the same approach initially but the results were not promising mostly due to the neighbor cells becoming part of the mask. It never clicked to me to take the neighbor cells as a different class mask in itself.

\n

Impressive solution. Great to see that this approach worked and glad to be proved wrong :)

", "votes": 1, "replies": [] }, { "id": 1633655, "author_name": "Mintwater", "author_url": "", "post_date": "2021-12-31T00:59:07.863000", "content": "

Very impressive and powerful solution! Happy new year to you, sir!

", "votes": 1, "replies": [] }, { "id": 1634701, "author_name": "Bhavya Dhingra", "author_url": "", "post_date": "2022-01-01T03:38:45.923000", "content": "

Awesome solution! Congratulations 🎉

", "votes": 2, "replies": [] }, { "id": 1634239, "author_name": "Bala Vashan", "author_url": "", "post_date": "2021-12-31T12:55:44.167000", "content": "

Great Work ! Congrats on getting 2nd place.

", "votes": 2, "replies": [] }, { "id": 1634032, "author_name": "Theo Viel", "author_url": "", "post_date": "2021-12-31T09:40:18.407000", "content": "

Congratz, very nice solution ! Simple to understand yet cleverly designed and far from easy to come up with.

\n

We knew you would perform very well as soon as we saw you merging, and it was only a matter a time before we were overtaken :)

", "votes": 2, "replies": [ { "id": 1634130, "author_name": "nvnn", "author_url": "", "post_date": "2021-12-31T11:16:42.920000", "content": "

thank you

", "votes": 1, "replies": [] } ] }, { "id": 1633933, "author_name": "Roc", "author_url": "", "post_date": "2021-12-31T07:17:46.043000", "content": "

Congratulations, awesome solution!

", "votes": 2, "replies": [] }, { "id": 1633684, "author_name": "Awsaf", "author_url": "", "post_date": "2021-12-31T01:23:49.983000", "content": "

@nvnnghia Congrats. btw did you guys do anything to handle the varying sizes of cropped-cell? What image size did you use to train the final Unet ?

", "votes": 2, "replies": [ { "id": 1633698, "author_name": "nvnn", "author_url": "", "post_date": "2021-12-31T01:44:56.423000", "content": "

I just resize all cell to a fixed size 128x128 for unet.

", "votes": 0, "replies": [] } ] }, { "id": 1633646, "author_name": "eagle4", "author_url": "", "post_date": "2021-12-31T00:49:20.943000", "content": "

Nicely done. method didn't not work for me: you had a strong first stage. I was missing the predicted neighboor cell on second stage unet.

", "votes": 2, "replies": [ { "id": 1633652, "author_name": "nvnn", "author_url": "", "post_date": "2021-12-31T00:54:52.080000", "content": "

thanks, the detection part is very important. predicted neighboor cell does not help too much.

", "votes": 0, "replies": [] } ] }, { "id": 1638094, "author_name": "pixyz0130", "author_url": "", "post_date": "2022-01-04T12:59:49.533000", "content": "

日本語訳
\nThanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.

\n

Our solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble.

\n

次の図に示すように、私たちのソリューションは、2つのオブジェクト検出モデル、1つのunetモデル、および2つのmaskrcnnモデルのアンサンブルです。 @sheepがmaskrcnnとアンサンブルに焦点を当てている間、私はオブジェクト検出とUnetを担当しています。

\n

We use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below.
\nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.

\n

オブジェクト検出タスクにはyolov5x6とeffdetD3を使用します。 下の図に示すように、トレーニング手順は両方のモデルで同じです。
\nモデルは、競合データで微調整する前に、Livecellおよびtrain-semi-supervisedデータセットで数ラウンドトレーニングされます。 推論フェーズでは、2つのモデルの出力ボックスがWBFを使用してmaskrcnnボックスとアンサンブルされます。

\n

The output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn.

\n

We use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation.

\n

WBF後の出力ボックスは、各ボックスのセグメンテーションマスクを取得するために、UnetおよびMaskrcnn（マスクヘッド）にフィードされます。 加重平均を使用して、UnetとMaskrcnnの生のマスクをアンサンブルします。

\n

効率的なb5エンコーダーを備えたunetを使用して、トリミングされたセルのセグメンテーションを実行します。 トリミングされたセルには隣接セルが含まれることがあるため、中央のセルと隣接セルのマスクを2クラスのセグメンテーションとして予測します。

\n

From left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.

", "votes": -1, "replies": [] }, { "id": 1635384, "author_name": "Aiswarya Sivakumar", "author_url": "", "post_date": "2022-01-01T17:57:38.633000", "content": "

Congratulations !great work

", "votes": -1, "replies": [] }, { "id": 2165081, "author_name": "lionlamb", "author_url": "", "post_date": "2023-03-02T01:22:08.633000", "content": "

congratulation and code is really helpful and learned a lot

", "votes": 0, "replies": [] }, { "id": 1643787, "author_name": "Adeyemi Adewole", "author_url": "", "post_date": "2022-01-09T18:16:44.987000", "content": "

Nice job, Nvnn and Sheep.
\nAlready, I can find one or two ideas I could use from this.
\nCongratulations!
\nI don't see much difference between first and second place winners, anyway.

", "votes": 0, "replies": [] }, { "id": 1642984, "author_name": "eagle4", "author_url": "", "post_date": "2022-01-08T21:14:15.477000", "content": "

Congrats again.
\nI have question about the second stage training: what image size are you using for the unet? How do you constitute your train set: are you taking crop of the original train set or are you resizing every cell of the original dataset to the size of unet?

", "votes": 0, "replies": [] }, { "id": 1642474, "author_name": "DineshManikanta", "author_url": "", "post_date": "2022-01-08T12:24:34.523000", "content": "

great solution Congratulations buddy
\ngreat team work

", "votes": 0, "replies": [] }, { "id": 1642306, "author_name": "Prashant Pathak", "author_url": "", "post_date": "2022-01-08T08:51:36.500000", "content": "

Awesome solution! Congratulations 🎉.

", "votes": 0, "replies": [] }, { "id": 1638095, "author_name": "", "author_url": "", "post_date": "2022-01-04T13:00:44.110000", "content": "", "votes": 0, "replies": [] }, { "id": 1637671, "author_name": "", "author_url": "", "post_date": "2022-01-04T05:40:52.190000", "content": "", "votes": 0, "replies": [] }, { "id": 1637022, "author_name": "", "author_url": "", "post_date": "2022-01-03T13:36:26.577000", "content": "", "votes": 0, "replies": [] }, { "id": 1636577, "author_name": "", "author_url": "", "post_date": "2022-01-03T04:39:50.430000", "content": "", "votes": 0, "replies": [] }, { "id": 1636430, "author_name": "", "author_url": "", "post_date": "2022-01-02T21:47:37.280000", "content": "", "votes": 0, "replies": [] }, { "id": 1636148, "author_name": "", "author_url": "", "post_date": "2022-01-02T16:44:23.330000", "content": "", "votes": 0, "replies": [] }, { "id": 1635968, "author_name": "", "author_url": "", "post_date": "2022-01-02T13:02:21.850000", "content": "", "votes": 0, "replies": [] }, { "id": 1635913, "author_name": "", "author_url": "", "post_date": "2022-01-02T11:18:16.060000", "content": "", "votes": 0, "replies": [] }, { "id": 1635837, "author_name": "", "author_url": "", "post_date": "2022-01-02T08:50:21.023000", "content": "", "votes": 0, "replies": [] }, { "id": 1635296, "author_name": "", "author_url": "", "post_date": "2022-01-01T16:50:34.977000", "content": "", "votes": 0, "replies": [] }, { "id": 1635174, "author_name": "", "author_url": "", "post_date": "2022-01-01T14:37:08.080000", "content": "", "votes": 0, "replies": [] }, { "id": 1635020, "author_name": "", "author_url": "", "post_date": "2022-01-01T12:12:18.127000", "content": "", "votes": 0, "replies": [] }, { "id": 1634864, "author_name": "", "author_url": "", "post_date": "2022-01-01T09:10:00.657000", "content": "", "votes": 0, "replies": [] }, { "id": 1634025, "author_name": "", "author_url": "", "post_date": "2021-12-31T09:21:20.283000", "content": "", "votes": 0, "replies": [ { "id": 1634027, "author_name": "", "author_url": "", "post_date": "2021-12-31T09:25:45.393000", "content": "", "votes": 0, "replies": [] }, { "id": 1634031, "author_name": "", "author_url": "", "post_date": "2021-12-31T09:37:18.997000", "content": "", "votes": 0, "replies": [] }, { "id": 1634137, "author_name": "", "author_url": "", "post_date": "2021-12-31T11:22:06.273000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1633929, "author_name": "", "author_url": "", "post_date": "2021-12-31T07:11:57.953000", "content": "", "votes": 0, "replies": [ { "id": 1634020, "author_name": "", "author_url": "", "post_date": "2021-12-31T09:18:10.187000", "content": "", "votes": 2, "replies": [] }, { "id": 1634220, "author_name": "", "author_url": "", "post_date": "2021-12-31T12:38:44.667000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1633904, "author_name": "", "author_url": "", "post_date": "2021-12-31T06:36:39.770000", "content": "", "votes": 0, "replies": [] }, { "id": 1633783, "author_name": "", "author_url": "", "post_date": "2021-12-31T04:07:00.440000", "content": "", "votes": 0, "replies": [] }, { "id": 1633737, "author_name": "", "author_url": "", "post_date": "2021-12-31T02:41:42.160000", "content": "", "votes": 0, "replies": [] }, { "id": 1633730, "author_name": "", "author_url": "", "post_date": "2021-12-31T02:25:24.370000", "content": "", "votes": 0, "replies": [ { "id": 1633850, "author_name": "", "author_url": "", "post_date": "2021-12-31T05:29:06.430000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 1636752, "author_name": "", "author_url": "", "post_date": "2022-01-03T08:41:06.743000", "content": "", "votes": 0, "replies": [] }, { "id": 1636301, "author_name": "", "author_url": "", "post_date": "2022-01-02T18:32:24.770000", "content": "", "votes": 0, "replies": [] }, { "id": 1636124, "author_name": "", "author_url": "", "post_date": "2022-01-02T16:04:52.173000", "content": "", "votes": 0, "replies": [] }, { "id": 1633653, "author_name": "", "author_url": "", "post_date": "2021-12-31T00:57:34.803000", "content": "", "votes": 0, "replies": [] }, { "id": 1633650, "author_name": "", "author_url": "", "post_date": "2021-12-31T00:53:33.307000", "content": "", "votes": 1, "replies": [] }, { "id": 1633712, "author_name": "", "author_url": "", "post_date": "2021-12-31T02:01:54.663000", "content": "", "votes": 0, "replies": [] }, { "id": 1639144, "author_name": "", "author_url": "", "post_date": "2022-01-05T12:25:27.293000", "content": "", "votes": 0, "replies": [] }, { "id": 1638669, "author_name": "", "author_url": "", "post_date": "2022-01-04T23:40:19.017000", "content": "", "votes": 0, "replies": [] }, { "id": 1637461, "author_name": "", "author_url": "", "post_date": "2022-01-03T22:57:38.627000", "content": "", "votes": 0, "replies": [] }, { "id": 1636700, "author_name": "", "author_url": "", "post_date": "2022-01-03T07:14:39.437000", "content": "", "votes": 0, "replies": [] }, { "id": 1636422, "author_name": "", "author_url": "", "post_date": "2022-01-02T21:27:17.927000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1633636": "Thanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.\n\nOur solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble. \n![](https://i.ibb.co/WVPHQ93/cell.png)\n\nWe use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below. \nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.\n\n![](https://i.ibb.co/98vHr2N/cell1.png)\n\nThe output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn. \n\nWe use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation. \n\n![](https://i.ibb.co/LPDPsbK/cell2.png)\n\n\n![](https://i.ibb.co/vxK9M3R/s4.jpg)\nFrom left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.", "1638069": "congratulation and code is really helpful and learned a lot", "1633693": "Impressive solution @nvnnghia \n\nI have a question regarding general approach. How do you arrive at such a solution?\n\nDid you come up with the plan from the beginning and decided to go for it or started with a simpler plan and added components as you go? If you can share more about how your thought process, that would be great!", "1634634": "Congratulations Nvnn and Sheep! Fantastic pipeline. Great combination of models.", "1634763": "An enlightening solution with a pretty simple way to understand! Really learned a lot👍 @nvnnghia", "1633690": "Congrats on 2nd place @nvnnghia and @steamedsheep, nice solution. Thanks for the writeup!", "1636936": "Thank you so much :) This opened me new doors to think about these types of problems.", "1636734": "Congratulations, Good solution", "1634989": "@nvnnghia - Many Congratulations !!\nThanks so much not just for posting your solution but for the way you have done so. It has allowed me to gather a lot of insight ! I do have a couple of questions though,\n- Did you do all your training on Kaggle Kernels ? If not can you let us know what kind of hardware you are using. \n- How many epochs of training did you require in each stage to get to a decent score ? ", "1634122": "Great solution. Congrats on 2nd place!!", "1634091": "Congratulations, nice and awesome solution.", "1634001": "Impressive solution, congrats on 2nd place! 🙌", "1633888": "Impressive solution. Congratulations author. \"center cell and neighbor cell as 2-class segmentation\".Did you make the dataset yourself? What form is the dataset？", "1633762": "Congrats! Are you going to share the code? @nvnnghia ", "1633754": "Congrats on getting 2nd place with an impressive solution! ", "1633714": "Congratulations on 2nd\nI took the same approach initially but the results were not promising mostly due to the neighbor cells becoming part of the mask. It never clicked to me to take the neighbor cells as a different class mask in itself. \n\nImpressive solution. Great to see that this approach worked and glad to be proved wrong :)\n", "1633655": "Very impressive and powerful solution! Happy new year to you, sir!", "1634701": "Awesome solution! Congratulations 🎉", "1634239": "Great Work ! Congrats on getting 2nd place.", "1634032": "Congratz, very nice solution ! Simple to understand yet cleverly designed and far from easy to come up with.\n\nWe knew you would perform very well as soon as we saw you merging, and it was only a matter a time before we were overtaken :)", "1633933": "Congratulations, awesome solution!", "1633684": "@nvnnghia Congrats. btw did you guys do anything to handle the varying sizes of **cropped-cell**? What image size did you use to train the final **Unet** ?", "1633646": "Nicely done. method didn't not work for me: you had a strong first stage. I was missing the predicted neighboor cell on second stage unet.", "1638094": "日本語訳\nThanks to Sartorius and Kaggle for hosting this interesting competition. I also would like to thank @steamedsheep for the great collaborative teamwork during the competition, we both work very hard to achieve this result.\n\nOur solution is an ensemble of 2 object detection model, 1 unet and 2 maskrcnn model as shown in figure below. I am in charge of object detection and Unet while @sheep focuses on maskrcnn and ensemble.\n\n次の図に示すように、私たちのソリューションは、2つのオブジェクト検出モデル、1つのunetモデル、および2つのmaskrcnnモデルのアンサンブルです。 @sheepがmaskrcnnとアンサンブルに焦点を当てている間、私はオブジェクト検出とUnetを担当しています。\n\nWe use yolov5x6 and effdetD3 for the object detection task. The training procedure is the same for both model as shown in the figure below.\nThe models are trained several rounds on Livecell and train-semi-supervised dataset before finetuning with the competition data. In the inference phase the output boxes of 2 model are ensemble with maskrcnn boxes using WBF.\n\nオブジェクト検出タスクにはyolov5x6とeffdetD3を使用します。 下の図に示すように、トレーニング手順は両方のモデルで同じです。\nモデルは、競合データで微調整する前に、Livecellおよびtrain-semi-supervisedデータセットで数ラウンドトレーニングされます。 推論フェーズでは、2つのモデルの出力ボックスがWBFを使用してmaskrcnnボックスとアンサンブルされます。\n\n\nThe output boxes after WBF are feeded into a Unet and Maskrcnn (mask head) to get the segmentation mask for each box. We use weighted average to ensemble the raw mask of Unet and Maskrcnn.\n\nWe use an unet with effificientb5 encoder to do segmentation on the cropped cell. Since the cropped cell sometimes include the neighbor cell, we predict the mask of the center cell and neighbor cell as 2-class segmentation.\n\nWBF後の出力ボックスは、各ボックスのセグメンテーションマスクを取得するために、UnetおよびMaskrcnn（マスクヘッド）にフィードされます。 加重平均を使用して、UnetとMaskrcnnの生のマスクをアンサンブルします。\n\n効率的なb5エンコーダーを備えたunetを使用して、トリミングされたセルのセグメンテーションを実行します。 トリミングされたセルには隣接セルが含まれることがあるため、中央のセルと隣接セルのマスクを2クラスのセグメンテーションとして予測します。\n\n\nFrom left to right: cropped cell; ground-truth of center mask; predicted mask of center mask; ground-truth of neighbor cell; predicted mask of neighbor cell.", "1635384": "Congratulations !great work", "2165081": "congratulation and code is really helpful and learned a lot", "1643787": "Nice job, Nvnn and Sheep.\nAlready, I can find one or two ideas I could use from this.\nCongratulations!\nI don't see much difference between first and second place winners, anyway.", "1642984": "Congrats again.\nI have question about the second stage training: what image size are you using for the unet? How do you constitute your train set: are you taking crop of the original train set or are you resizing every cell of the original dataset to the size of unet?", "1642474": "great solution Congratulations buddy\ngreat team work ", "1642306": "Awesome solution! Congratulations 🎉.", "1638095": "good one.congrats.", "1637671": "Great work.", "1637022": "Well Done! Congratulations", "1636577": "Great combination of models, thank you and congratulations", "1636430": "Impressive solution, congratulations! Truly inspiring :)", "1636148": "Wow, thank you for this useful information! and congratulations!", "1635968": "Awesome solution! Congratulations 🎉\n\n", "1635913": "Great work!", "1635837": "Awesome solution ! Congrats for the 2nd place !", "1635296": "Congrats! Very impressive solution", "1635174": "Congraulations!\nMay I ask which library you used for the implementation?\n\n", "1635020": "very creative!\n", "1634864": "Congratulations", "1634025": "Congrats to u!\nGot a question here. Just in case of prudence. The output of a unet or mask head is a 0 and 1 value map. In the last picture it appears to be float values between.Or did u post the output before softmax actully?", "1633929": "Wow @nvnnghia great approach, this is really impressive. btw what is `WBF`?\nAre you guys planning to publish the code? would love to check that out. Thank you.", "1633904": "Congratulations @nvnnghia, the solution is indeed impressive!", "1633783": "Congrats...", "1633737": "Great Work", "1633730": "Congrats on 2nd place! May I ask what the learning rates were used during pretraining and finetuning stages?", "1636752": "", "1636301": "", "1636124": "", "1633653": "", "1633650": "Nice work !!! Thanks for sharing it. ", "1633712": "nice solution. Thank you.", "1639144": "Thank you for sharing.", "1638669": "Thank you for sharing", "1637461": "Good solutions, thanks", "1636700": "Thank you for your pipeline", "1636422": "Thanks for posting!" } }