Image segmentation is about grouping pixels with different semantics, e.g., category or instance membership, where each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing specialized architectures for each task. We present Masked-attention Mask Transformer (Mask2Former), a new architecture capable of addressing any image segmentation task (panoptic, instance, or semantic). Its key components include masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best-specialized architectures by a significant margin on four popular datasets. Most notably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO), and semantic segmentation (57.7 mIoU on ADE20K).
\nPaper: https://arxiv.org/pdf/2112.01527v1.pdf
\nGithub: https://github.com/facebookresearch/Mask2Former
", "messages": [ { "id": "1610240", "postDate": "12/07/2021 04:21:15", "content": "Image segmentation is about grouping pixels with different semantics, e.g., category or instance membership, where each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing specialized architectures for each task. We present Masked-attention Mask Transformer (Mask2Former), a new architecture capable of addressing any image segmentation task (panoptic, instance, or semantic). Its key components include masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best-specialized architectures by a significant margin on four popular datasets. Most notably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO), and semantic segmentation (57.7 mIoU on ADE20K).
\nPaper: https://arxiv.org/pdf/2112.01527v1.pdf
\nGithub: https://github.com/facebookresearch/Mask2Former
", "rawMarkdown": "Image segmentation is about grouping pixels with different semantics, e.g., category or instance membership, where each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing specialized architectures for each task. We present Masked-attention Mask Transformer (Mask2Former), a new architecture capable of addressing any image segmentation task (panoptic, instance, or semantic). Its key components include masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best-specialized architectures by a significant margin on four popular datasets. Most notably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO), and semantic segmentation (57.7 mIoU on ADE20K).\n\nPaper: https://arxiv.org/pdf/2112.01527v1.pdf\n\nGithub: https://github.com/facebookresearch/Mask2Former", "votes": null }, { "id": "1612781", "postDate": "12/09/2021 10:09:30", "content": "Thank you for sharing. BTW how do you know so early? Do you register some newsletters?
\nI register Deep Learning Weekly. I know mask2transformer today by this newsletter.
Sorry I found it.
\nhttps://www.kaggle.com/general/71954
I visit paperswithcode.com regularly
", "rawMarkdown": "I visit paperswithcode.com regularly", "votes": null }, { "id": "1614063", "postDate": "12/10/2021 14:51:01", "content": "Thank you very much!
", "rawMarkdown": "Thank you very much!", "votes": null } ], "comments": [ { "id": 1612781, "author_name": "osamurai", "author_url": "", "post_date": "12/09/2021 10:09:30", "content": "Thank you for sharing. BTW how do you know so early? Do you register some newsletters?
\nI register Deep Learning Weekly. I know mask2transformer today by this newsletter.
Sorry I found it.
\nhttps://www.kaggle.com/general/71954
I visit paperswithcode.com regularly
", "votes": null, "replies": [] }, { "id": 1614063, "author_name": "osamurai", "author_url": "", "post_date": "12/10/2021 14:51:01", "content": "Thank you very much!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1610240": "Image segmentation is about grouping pixels with different semantics, e.g., category or instance membership, where each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing specialized architectures for each task. We present Masked-attention Mask Transformer (Mask2Former), a new architecture capable of addressing any image segmentation task (panoptic, instance, or semantic). Its key components include masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best-specialized architectures by a significant margin on four popular datasets. Most notably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO), and semantic segmentation (57.7 mIoU on ADE20K).\n\nPaper: https://arxiv.org/pdf/2112.01527v1.pdf\n\nGithub: https://github.com/facebookresearch/Mask2Former", "1612781": "Thank you for sharing. BTW how do you know so early? Do you register some newsletters?\nI register [Deep Learning Weekly](https://www.deeplearningweekly.com/). I know mask2transformer today by this newsletter.", "1612833": "Sorry I found it.\nhttps://www.kaggle.com/general/71954", "1614049": "I visit paperswithcode.com regularly", "1614063": "Thank you very much!" }, "source": "meta" }