![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F761268%2F6a614d7ba0790ce115480787e25a2343%2F1.png?generation=1712761543651578&alt=media\")
Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences
\n\n\nGiven two images, we can estimate the relative camera pose between them by establishing image-to-image correspondences. Usually, correspondences are 2D-to-2D and the pose we estimate is defined only up to scale. Some applications, aiming at instant augmented reality anywhere, require scale-metric pose estimates, and hence, they rely on external depth estimators to recover the scale. We present MicKey, a keypoint matching pipeline that is able to predict metric correspondences in 3D camera space. By learning to match 3D coordinates across images, we are able to infer the metric relative pose without depth measurements. Depth measurements are also not required for training, nor are scene reconstructions or image overlap information. MicKey is supervised only by pairs of images and their relative poses. MicKey achieves state-of-the-art performance on the Map-Free Relocalisation benchmark while requiring less supervision than competing approaches.
\n
Code & Model - https://github.com/nianticlabs/mickey - top 2nd model in MapFree Reloc Benchmark
\n\n\nYou may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test). by @oldufo => follow papers from both twitter ducha_aiki + zhenjun_zhao
\n
\n \n\n
\n \n", "messages": [ { "id": "2745351", "postDate": "04/10/2024 15:08:18", "content": "
Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences
\n\n\nGiven two images, we can estimate the relative camera pose between them by establishing image-to-image correspondences. Usually, correspondences are 2D-to-2D and the pose we estimate is defined only up to scale. Some applications, aiming at instant augmented reality anywhere, require scale-metric pose estimates, and hence, they rely on external depth estimators to recover the scale. We present MicKey, a keypoint matching pipeline that is able to predict metric correspondences in 3D camera space. By learning to match 3D coordinates across images, we are able to infer the metric relative pose without depth measurements. Depth measurements are also not required for training, nor are scene reconstructions or image overlap information. MicKey is supervised only by pairs of images and their relative poses. MicKey achieves state-of-the-art performance on the Map-Free Relocalisation benchmark while requiring less supervision than competing approaches.
\n
Code & Model - https://github.com/nianticlabs/mickey - top 2nd model in MapFree Reloc Benchmark
\n\n\nYou may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test). by @oldufo => follow papers from both twitter ducha_aiki + zhenjun_zhao
\n
\n \n\n
\n \n", "rawMarkdown": "[Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences](https://arxiv.org/abs/2404.06337)\n> Given two images, we can estimate the relative camera pose between them by establishing image-to-image correspondences. Usually, correspondences are 2D-to-2D and the pose we estimate is defined only up to scale. Some applications, aiming at instant augmented reality anywhere, require scale-metric pose estimates, and hence, they rely on external depth estimators to recover the scale. We present MicKey, a keypoint matching pipeline that is able to predict metric correspondences in 3D camera space. By learning to match 3D coordinates across images, we are able to infer the metric relative pose without depth measurements. Depth measurements are also not required for training, nor are scene reconstructions or image overlap information. MicKey is supervised only by pairs of images and their relative poses. MicKey achieves state-of-the-art performance on the Map-Free Relocalisation benchmark while requiring less supervision than competing approaches.\n\n\n\n[Code & Model - https://github.com/nianticlabs/mickey ](https://github.com/nianticlabs/mickey) - top 2nd model in [MapFree Reloc Benchmark](https://research.nianticlabs.com/mapfree-reloc-benchmark/leaderboard)\n\n----\n\n> You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test). **by @oldufo => follow papers from both twitter ducha_aiki + zhenjun_zhao**\n\n----\n\n> [https://twitter.com/ducha_aiki](https://twitter.com/ducha_aiki)\n\n\n---\n\n>[https://twitter.com/zhenjun_zhao](https://twitter.com/zhenjun_zhao)\n", "votes": null }, { "id": "2747539", "postDate": "04/12/2024 00:07:21", "content": "
I think, it can be helpful, @seshurajup. As codes are released, we can check.
", "rawMarkdown": "I think, it can be helpful, @seshurajup. As codes are released, we can check.", "votes": null }, { "id": "2748083", "postDate": "04/12/2024 08:35:35", "content": "That might works for this comp, thanks for the sharing! @seshurajup
", "rawMarkdown": "That might works for this comp, thanks for the sharing! @seshurajup", "votes": null }, { "id": "2748097", "postDate": "04/12/2024 08:42:42", "content": "You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test).
", "rawMarkdown": "You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test).", "votes": null }, { "id": "2748122", "postDate": "04/12/2024 08:59:47", "content": "Thank for explanation @oldufo, will explore the papers shared on twitter, thanks for sharing.
\n-- Updated topic too --
\n\nYou may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test).
\n
\n", "rawMarkdown": "> You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (**as very little papers survive a Kaggle test**).\n\n---\n\n> is it means, Kaggle test set is diverse from public datasets in papers ( maybe it is clue for where to focus )? @oldufo", "votes": null }, { "id": "2748335", "postDate": "04/12/2024 11:46:37", "content": "is it means, Kaggle test set is diverse from public datasets in papers ( maybe it is clue for where to focus )? @oldufo
\n
I believe what @oldufo means is that it's quite difficult to port these new techniques to Kaggle notebooks, especially without an internet connection.
", "rawMarkdown": "I believe what @oldufo means is that it's quite difficult to port these new techniques to Kaggle notebooks, especially without an internet connection.", "votes": null } ], "comments": [ { "id": 2747539, "author_name": "azminetoushikwasi", "author_url": "", "post_date": "04/12/2024 00:07:21", "content": "I think, it can be helpful, @seshurajup. As codes are released, we can check.
", "votes": null, "replies": [] }, { "id": 2748083, "author_name": "liushundawang", "author_url": "", "post_date": "04/12/2024 08:35:35", "content": "That might works for this comp, thanks for the sharing! @seshurajup
", "votes": null, "replies": [] }, { "id": 2748097, "author_name": "oldufo", "author_url": "", "post_date": "04/12/2024 08:42:42", "content": "You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test).
", "votes": null, "replies": [ { "id": 2748122, "author_name": "seshurajup", "author_url": "", "post_date": "04/12/2024 08:59:47", "content": "Thank for explanation @oldufo, will explore the papers shared on twitter, thanks for sharing.
\n-- Updated topic too --
\n\nYou may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test).
\n
\n", "votes": null, "replies": [ { "id": 2748335, "author_name": "asarvazyan", "author_url": "", "post_date": "04/12/2024 11:46:37", "content": "is it means, Kaggle test set is diverse from public datasets in papers ( maybe it is clue for where to focus )? @oldufo
\n
I believe what @oldufo means is that it's quite difficult to port these new techniques to Kaggle notebooks, especially without an internet connection.
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2745351": "[Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences](https://arxiv.org/abs/2404.06337)\n> Given two images, we can estimate the relative camera pose between them by establishing image-to-image correspondences. Usually, correspondences are 2D-to-2D and the pose we estimate is defined only up to scale. Some applications, aiming at instant augmented reality anywhere, require scale-metric pose estimates, and hence, they rely on external depth estimators to recover the scale. We present MicKey, a keypoint matching pipeline that is able to predict metric correspondences in 3D camera space. By learning to match 3D coordinates across images, we are able to infer the metric relative pose without depth measurements. Depth measurements are also not required for training, nor are scene reconstructions or image overlap information. MicKey is supervised only by pairs of images and their relative poses. MicKey achieves state-of-the-art performance on the Map-Free Relocalisation benchmark while requiring less supervision than competing approaches.\n\n\n\n[Code & Model - https://github.com/nianticlabs/mickey ](https://github.com/nianticlabs/mickey) - top 2nd model in [MapFree Reloc Benchmark](https://research.nianticlabs.com/mapfree-reloc-benchmark/leaderboard)\n\n----\n\n> You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test). **by @oldufo => follow papers from both twitter ducha_aiki + zhenjun_zhao**\n\n----\n\n> [https://twitter.com/ducha_aiki](https://twitter.com/ducha_aiki)\n\n\n---\n\n>[https://twitter.com/zhenjun_zhao](https://twitter.com/zhenjun_zhao)\n", "2747539": "I think, it can be helpful, @seshurajup. As codes are released, we can check.", "2748083": "That might works for this comp, thanks for the sharing! @seshurajup", "2748097": "You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (as very little papers survive a Kaggle test).", "2748122": "Thank for explanation @oldufo, will explore the papers shared on twitter, thanks for sharing.\n-- Updated topic too --", "2748138": "> You may want to check my twitter https://twitter.com/ducha_aiki as well as Zhenjun Zhao https://twitter.com/zhenjun_zhao twitter for the relevant SfM/Image matching papers. However, no guarantees that they would be working (**as very little papers survive a Kaggle test**).\n\n---\n\n> is it means, Kaggle test set is diverse from public datasets in papers ( maybe it is clue for where to focus )? @oldufo", "2748335": "I believe what @oldufo means is that it's quite difficult to port these new techniques to Kaggle notebooks, especially without an internet connection." }, "source": "meta" }![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F761268%2F63a6649fb9ef8af0bed1f8ad866f3aea%2FScreenshot%202024-04-12%20at%202.31.16PM.png?generation=1712912552855326&alt=media\")
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F761268%2F83de4814ed00e1f328eb1e645ecfae17%2FScreenshot%202024-04-12%20at%202.31.26PM.png?generation=1712912597974585&alt=media\")