{ "id": 507536, "title": "OmniGlue: Generalizable Feature Matching with Foundation Model Guidance", "url": "/competitions/image-matching-challenge-2024/discussion/507536", "author_name": "Anil Ozturk", "post_date": "2024-05-26T08:02:49.599000", "votes": 10, "comment_count": 0, "views": 0, "content": "

There is a new work from Google Research called OmniGlue and I thought it would also be relevant with this competition. The repository is under Apache 2.0 license and the model is available at HF.

\n

The authors say:

\n
\n

In this paper, we introduce OmniGlue, the first learnable image matcher that is designed with generalization as a core principle. OmniGlue leverages broad knowledge from a vision foundation model to guide the feature matching process, boosting generalization to domains not seen at training time. Additionally, we propose a novel keypoint position-guided attention mechanism which disentangles spatial and appearance information, leading to enhanced matching descriptors. We perform comprehensive experiments on a suite of 6 datasets with varied image domains, including scene-level, object-centric and aerial images. OmniGlue’s novel components lead to relative gains on unseen domains of 18.8% with respect to a directly comparable reference model, while also outperforming the recent LightGlue method by 10.1% relatively.

\n
\n

\n

GitHub: https://github.com/google-research/omniglue
\nHF Demo: https://huggingface.co/spaces/qubvel-hf/omniglue

", "messages": [ { "id": 2836985, "postDate": "2024-05-26T08:02:49.600Z", "content": "

There is a new work from Google Research called OmniGlue and I thought it would also be relevant with this competition. The repository is under Apache 2.0 license and the model is available at HF.

\n

The authors say:

\n
\n

In this paper, we introduce OmniGlue, the first learnable image matcher that is designed with generalization as a core principle. OmniGlue leverages broad knowledge from a vision foundation model to guide the feature matching process, boosting generalization to domains not seen at training time. Additionally, we propose a novel keypoint position-guided attention mechanism which disentangles spatial and appearance information, leading to enhanced matching descriptors. We perform comprehensive experiments on a suite of 6 datasets with varied image domains, including scene-level, object-centric and aerial images. OmniGlue’s novel components lead to relative gains on unseen domains of 18.8% with respect to a directly comparable reference model, while also outperforming the recent LightGlue method by 10.1% relatively.

\n
\n

\n

GitHub: https://github.com/google-research/omniglue
\nHF Demo: https://huggingface.co/spaces/qubvel-hf/omniglue

", "rawMarkdown": "There is a new work from Google Research called **OmniGlue** and I thought it would also be relevant with this competition. The repository is under Apache 2.0 license and the model is available at HF.\n\n***The authors say:***\n>In this paper, we introduce OmniGlue, the first learnable image matcher that is designed with generalization as a core principle. OmniGlue leverages broad knowledge from a vision foundation model to guide the feature matching process, boosting generalization to domains not seen at training time. Additionally, we propose a novel keypoint position-guided attention mechanism which disentangles spatial and appearance information, leading to enhanced matching descriptors. We perform comprehensive experiments on a suite of 6 datasets with varied image domains, including scene-level, object-centric and aerial images. OmniGlue’s novel components lead to relative gains on unseen domains of 18.8% with respect to a directly comparable reference model, while also outperforming the recent LightGlue method by 10.1% relatively.\n\n![](https://raw.githubusercontent.com/google-research/omniglue/main/res/og_diagram.png)\n\n**GitHub:** https://github.com/google-research/omniglue\n**HF Demo:** https://huggingface.co/spaces/qubvel-hf/omniglue", "votes": 10 } ], "comments": [], "raw_markdown_by_id": { "2836985": "There is a new work from Google Research called **OmniGlue** and I thought it would also be relevant with this competition. The repository is under Apache 2.0 license and the model is available at HF.\n\n***The authors say:***\n>In this paper, we introduce OmniGlue, the first learnable image matcher that is designed with generalization as a core principle. OmniGlue leverages broad knowledge from a vision foundation model to guide the feature matching process, boosting generalization to domains not seen at training time. Additionally, we propose a novel keypoint position-guided attention mechanism which disentangles spatial and appearance information, leading to enhanced matching descriptors. We perform comprehensive experiments on a suite of 6 datasets with varied image domains, including scene-level, object-centric and aerial images. OmniGlue’s novel components lead to relative gains on unseen domains of 18.8% with respect to a directly comparable reference model, while also outperforming the recent LightGlue method by 10.1% relatively.\n\n![](https://raw.githubusercontent.com/google-research/omniglue/main/res/og_diagram.png)\n\n**GitHub:** https://github.com/google-research/omniglue\n**HF Demo:** https://huggingface.co/spaces/qubvel-hf/omniglue" } }