{ "id": 549750, "title": "CZII CryoET - SAM2.1 ROI Generate High Quality Masks", "url": "/competitions/czii-cryo-et-object-identification/discussion/549750", "author_name": "", "post_date": "2024-12-03T18:49:34.597171500Z", "votes": 16, "comment_count": 8, "views": 0, "content": "

Hi, In this notebook I have demonstrated the use of SAM 2.1 (sam2.1_hiera_large) to extract High Quality masks. I have used boundingboxes generated from this dataset to extract ROI mask for each label frame by frame.

\n

Notebook: CZII - SAM2 ROI Extract High Quality Masks

\n

Code Snippet:

\n
 sam2.sam2_image_predictor  SAM2ImagePredictor\n\n\npredictor = SAM2ImagePredictor.from_pretrained(, device=device)\n\n\npredictor.set_image(img)\n\nmasks, scores, _ = predictor.predict(\n    box=input_boxes,\n    multimask_output=\n)\n
\n

Some visualizations of generated masks:
\n\"results__13_0\"
\n\"results__13_2\"
\n\"results__14_0\"
\n\"results__14_1\"
\n\"results__15_0\"
\n\"results__15_1\"

", "messages": [ { "id": "3062644", "postDate": "12/03/2024 18:49:34", "content": "

Hi, In this notebook I have demonstrated the use of SAM 2.1 (sam2.1_hiera_large) to extract High Quality masks. I have used boundingboxes generated from this dataset to extract ROI mask for each label frame by frame.

\n

Notebook: CZII - SAM2 ROI Extract High Quality Masks

\n

Code Snippet:

\n
 sam2.sam2_image_predictor  SAM2ImagePredictor\n\n\npredictor = SAM2ImagePredictor.from_pretrained(, device=device)\n\n\npredictor.set_image(img)\n\nmasks, scores, _ = predictor.predict(\n    box=input_boxes,\n    multimask_output=\n)\n
\n

Some visualizations of generated masks:
\n\"results__13_0\"
\n\"results__13_2\"
\n\"results__14_0\"
\n\"results__14_1\"
\n\"results__15_0\"
\n\"results__15_1\"

", "rawMarkdown": "Hi, In this notebook I have demonstrated the use of SAM 2.1 ([sam2.1_hiera_large](https://www.kaggle.com/models/metaresearch/segment-anything-2.1/PyTorch/sam2.1-hiera-large/1)) to extract High Quality masks. I have used boundingboxes generated from this [dataset](https://www.kaggle.com/datasets/gowrishankarp/czii-cryoet-630x630-png-dataset-816bit) to extract ROI mask for each label frame by frame.\n\nNotebook: [CZII - SAM2 ROI Extract High Quality Masks](https://www.kaggle.com/code/gowrishankarp/czii-sam2-roi-extract-high-quality-masks)\n\n**Code Snippet:**\n```python\nfrom sam2.sam2_image_predictor import SAM2ImagePredictor\n\n# SAM Load model instance\npredictor = SAM2ImagePredictor.from_pretrained(\"facebook/sam2.1-hiera-large\", device=device)\n\n# SAM Set image and predict\npredictor.set_image(img)\n \nmasks, scores, _ = predictor.predict(\n box=input_boxes,\n multimask_output=False\n)\n\n```\n**Some visualizations of generated masks:**\n![results__13_0](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2Fb6da1b47f912b2d2be6a67d5d6543d41%2F__results___13_0.png?generation=1733251108075901&alt=media)\n![results__13_2](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2Fc359ae197df3f0e3ec7d82fedd8754cd%2F__results___13_2.png?generation=1733251124221969&alt=media)\n![results__14_0](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F37cacd78309fd1836eb6cee12aacca4a%2F__results___14_0.png?generation=1733251200121225&alt=media)\n![results__14_1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F7cfd5087d8b0e28f86afb7b7d0736f62%2F__results___14_1.png?generation=1733251225729228&alt=media)\n![results__15_0](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F0b33dea01e47fa69839f7a1bcec38c2d%2F__results___15_0.png?generation=1733251257486465&alt=media)\n![results__15_1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F8446e63705b4d22c39c3c2a8d9bfa9e5%2F__results___15_1.png?generation=1733251280308506&alt=media)", "votes": null }, { "id": "3062711", "postDate": "12/03/2024 20:41:45", "content": "

Nice idea! Did you try to check if 3d mask centroids are close to the original centroids?

", "rawMarkdown": "Nice idea! Did you try to check if 3d mask centroids are close to the original centroids?", "votes": null }, { "id": "3062831", "postDate": "12/04/2024 00:33:37", "content": "

Nice one, thank you for sharing! Did you try submitting this to the LB?

", "rawMarkdown": "Nice one, thank you for sharing! Did you try submitting this to the LB?", "votes": null }, { "id": "3062980", "postDate": "12/04/2024 04:10:04", "content": "

Not yet, but sure will do some submissions and share the CV & LB.

", "rawMarkdown": "Not yet, but sure will do some submissions and share the CV & LB.", "votes": null }, { "id": "3062983", "postDate": "12/04/2024 04:14:02", "content": "

Haven't @sakvaua, can give a more detail, what we can validate?

", "rawMarkdown": "Haven't @sakvaua, can give a more detail, what we can validate?", "votes": null }, { "id": "3063036", "postDate": "12/04/2024 05:34:48", "content": "

Create masks for all the train images, find centroids, and score them vs true train centroids.

", "rawMarkdown": "Create masks for all the train images, find centroids, and score them vs true train centroids.", "votes": null }, { "id": "3064272", "postDate": "12/05/2024 12:53:30", "content": "

@gowrishankarp Any luck?

", "rawMarkdown": "gowrishankarp Any luck?", "votes": null }, { "id": "3066184", "postDate": "12/07/2024 19:33:26", "content": "

Hi, I have calculate the mae and mse btw the generated and target coordinates for each label, (check the latest iteration of the Notebook)… I have observed the pixels are shifted by very few pixels (1-2 pixels), but would recommend to do some exp with this data.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
labelmaemse
1.00.0046020.001382
2.00.0022050.000004
3.00.0028710.000099
4.00.0035350.000280
5.00.0052360.001272
6.00.0028700.000607
", "rawMarkdown": "Hi, I have calculate the `mae` and `mse` btw the generated and target coordinates for each label, (check the latest iteration of the Notebook)... I have observed the pixels are shifted by very few pixels (1-2 pixels), but would recommend to do some exp with this data.\n\n| label | mae | mse |\n|-------|-----------|-----------|\n| 1.0 | 0.004602 | 0.001382 |\n| 2.0 | 0.002205 | 0.000004 |\n| 3.0 | 0.002871 | 0.000099 |\n| 4.0 | 0.003535 | 0.000280 |\n| 5.0 | 0.005236 | 0.001272 |\n| 6.0 | 0.002870 | 0.000607 |", "votes": null }, { "id": "3066210", "postDate": "12/07/2024 20:22:59", "content": "

Cool! But mae or mse tells us too little here. I would create a pandas dataframe with ground truth and anotehr one with calculated centroids and calculate the competition score to see if it is different from 1.0

", "rawMarkdown": "Cool! But mae or mse tells us too little here. I would create a pandas dataframe with ground truth and anotehr one with calculated centroids and calculate the competition score to see if it is different from 1.0", "votes": null } ], "comments": [ { "id": 3062711, "author_name": "sakvaua", "author_url": "", "post_date": "12/03/2024 20:41:45", "content": "

Nice idea! Did you try to check if 3d mask centroids are close to the original centroids?

", "votes": null, "replies": [ { "id": 3062983, "author_name": "gowrishankarp", "author_url": "", "post_date": "12/04/2024 04:14:02", "content": "

Haven't @sakvaua, can give a more detail, what we can validate?

", "votes": null, "replies": [ { "id": 3063036, "author_name": "sakvaua", "author_url": "", "post_date": "12/04/2024 05:34:48", "content": "

Create masks for all the train images, find centroids, and score them vs true train centroids.

", "votes": null, "replies": [] }, { "id": 3064272, "author_name": "sakvaua", "author_url": "", "post_date": "12/05/2024 12:53:30", "content": "

@gowrishankarp Any luck?

", "votes": null, "replies": [] }, { "id": 3066184, "author_name": "gowrishankarp", "author_url": "", "post_date": "12/07/2024 19:33:26", "content": "

Hi, I have calculate the mae and mse btw the generated and target coordinates for each label, (check the latest iteration of the Notebook)… I have observed the pixels are shifted by very few pixels (1-2 pixels), but would recommend to do some exp with this data.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
labelmaemse
1.00.0046020.001382
2.00.0022050.000004
3.00.0028710.000099
4.00.0035350.000280
5.00.0052360.001272
6.00.0028700.000607
", "votes": null, "replies": [ { "id": 3066210, "author_name": "sakvaua", "author_url": "", "post_date": "12/07/2024 20:22:59", "content": "

Cool! But mae or mse tells us too little here. I would create a pandas dataframe with ground truth and anotehr one with calculated centroids and calculate the competition score to see if it is different from 1.0

", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3062831, "author_name": "snnclsr", "author_url": "", "post_date": "12/04/2024 00:33:37", "content": "

Nice one, thank you for sharing! Did you try submitting this to the LB?

", "votes": null, "replies": [ { "id": 3062980, "author_name": "gowrishankarp", "author_url": "", "post_date": "12/04/2024 04:10:04", "content": "

Not yet, but sure will do some submissions and share the CV & LB.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3062644": "Hi, In this notebook I have demonstrated the use of SAM 2.1 ([sam2.1_hiera_large](https://www.kaggle.com/models/metaresearch/segment-anything-2.1/PyTorch/sam2.1-hiera-large/1)) to extract High Quality masks. I have used boundingboxes generated from this [dataset](https://www.kaggle.com/datasets/gowrishankarp/czii-cryoet-630x630-png-dataset-816bit) to extract ROI mask for each label frame by frame.\n\nNotebook: [CZII - SAM2 ROI Extract High Quality Masks](https://www.kaggle.com/code/gowrishankarp/czii-sam2-roi-extract-high-quality-masks)\n\n**Code Snippet:**\n```python\nfrom sam2.sam2_image_predictor import SAM2ImagePredictor\n\n# SAM Load model instance\npredictor = SAM2ImagePredictor.from_pretrained(\"facebook/sam2.1-hiera-large\", device=device)\n\n# SAM Set image and predict\npredictor.set_image(img)\n \nmasks, scores, _ = predictor.predict(\n box=input_boxes,\n multimask_output=False\n)\n\n```\n**Some visualizations of generated masks:**\n![results__13_0](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2Fb6da1b47f912b2d2be6a67d5d6543d41%2F__results___13_0.png?generation=1733251108075901&alt=media)\n![results__13_2](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2Fc359ae197df3f0e3ec7d82fedd8754cd%2F__results___13_2.png?generation=1733251124221969&alt=media)\n![results__14_0](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F37cacd78309fd1836eb6cee12aacca4a%2F__results___14_0.png?generation=1733251200121225&alt=media)\n![results__14_1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F7cfd5087d8b0e28f86afb7b7d0736f62%2F__results___14_1.png?generation=1733251225729228&alt=media)\n![results__15_0](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F0b33dea01e47fa69839f7a1bcec38c2d%2F__results___15_0.png?generation=1733251257486465&alt=media)\n![results__15_1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8096091%2F8446e63705b4d22c39c3c2a8d9bfa9e5%2F__results___15_1.png?generation=1733251280308506&alt=media)", "3062711": "Nice idea! Did you try to check if 3d mask centroids are close to the original centroids?", "3062831": "Nice one, thank you for sharing! Did you try submitting this to the LB?", "3062980": "Not yet, but sure will do some submissions and share the CV & LB.", "3062983": "Haven't @sakvaua, can give a more detail, what we can validate?", "3063036": "Create masks for all the train images, find centroids, and score them vs true train centroids.", "3064272": "gowrishankarp Any luck?", "3066184": "Hi, I have calculate the `mae` and `mse` btw the generated and target coordinates for each label, (check the latest iteration of the Notebook)... I have observed the pixels are shifted by very few pixels (1-2 pixels), but would recommend to do some exp with this data.\n\n| label | mae | mse |\n|-------|-----------|-----------|\n| 1.0 | 0.004602 | 0.001382 |\n| 2.0 | 0.002205 | 0.000004 |\n| 3.0 | 0.002871 | 0.000099 |\n| 4.0 | 0.003535 | 0.000280 |\n| 5.0 | 0.005236 | 0.001272 |\n| 6.0 | 0.002870 | 0.000607 |", "3066210": "Cool! But mae or mse tells us too little here. I would create a pandas dataframe with ground truth and anotehr one with calculated centroids and calculate the competition score to see if it is different from 1.0" }, "source": "meta" }