{
  "id": 490723,
  "title": "LightGlue vs SuperGlue performance",
  "url": "/competitions/image-matching-challenge-2024/discussion/490723",
  "author_name": "old-ufo",
  "post_date": "2024-04-03T09:29:05.284000",
  "votes": 28,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Given the threads <a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490622\" target=\"_blank\">1</a> and <a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490714\" target=\"_blank\">2</a> about license and SuperGlue in particular, we decided to address also a performance question.</p>\n<p>We understand, that given the dominance of SuperGlue in Image Matching Challenge since 2020 until 2023, it is kind of hard to believe that not using it is a good idea. <br>\nHowever, license issues aside, LightGlue is better than SuperGlue performance-wise as well, so there is no good argument to keep using SuperGlue.</p>\n<p>Specifically, let me point out results on IMC-2021 leaderboard, especially with ALIKED features. It provides better result than SuperGlue in all entries, except those, which use additional features and pre/post-processing.<br>\nYou should check the \"MultiView\" score in particular</p>\n<p><a href=\"https://www.cs.ubc.ca/research/image-matching-challenge/2021/leaderboard/\" target=\"_blank\">https://www.cs.ubc.ca/research/image-matching-challenge/2021/leaderboard/</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fe0f4fbda6256645275eb20b40fe242f9%2FScreenshot%202024-04-03%20at%2011.14.09.png?generation=1712135667888200&amp;alt=media\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2F24a32d444e487a5adabf509b494d7c74%2FScreenshot%202024-04-03%20at%2011.15.21.png?generation=1712135737534378&amp;alt=media\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Ff6c4fa1d2dd61b945b30d91281d1680c%2FScreenshot%202024-04-03%20at%2011.16.07.png?generation=1712135783594914&amp;alt=media\"></p>\n<p>Besides IMC-2021 dataset (and all the comparisons in the LightGlue paper itself), here is a comparison on WxBS and EVD datasets<br>\n<a href=\"https://github.com/cvg/glue-factory/pull/52#issue-2065776066\" target=\"_blank\">https://github.com/cvg/glue-factory/pull/52#issue-2065776066</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fdbb3b871c83cfa7ee654a85aa5a3e966%2FScreenshot%202024-04-03%20at%2011.17.05.png?generation=1712135840436199&amp;alt=media\"></p>\n<p>One particular technical issue with LightGlue is that its <em>default</em> parameters are optimized for speed, not maximum performance, so it might seem to be worse than SuperGlue. <br>\nHowever, you can turn off all the speed optimizations with setting up <code>depth_confidence</code> and <code>width_confidence</code> to <code>-1</code>:</p>\n<pre><code>matcher = LightGlue(=, =-1, =-1)\n</code></pre>\n<p>I hope, that this thread, together with license part of the story, will close the SuperGlue issues.</p>",
  "messages": [
    {
      "id": 2732704,
      "postDate": "2024-04-03T09:29:05.283Z",
      "content": "<p>Given the threads <a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490622\" target=\"_blank\">1</a> and <a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490714\" target=\"_blank\">2</a> about license and SuperGlue in particular, we decided to address also a performance question.</p>\n<p>We understand, that given the dominance of SuperGlue in Image Matching Challenge since 2020 until 2023, it is kind of hard to believe that not using it is a good idea. <br>\nHowever, license issues aside, LightGlue is better than SuperGlue performance-wise as well, so there is no good argument to keep using SuperGlue.</p>\n<p>Specifically, let me point out results on IMC-2021 leaderboard, especially with ALIKED features. It provides better result than SuperGlue in all entries, except those, which use additional features and pre/post-processing.<br>\nYou should check the \"MultiView\" score in particular</p>\n<p><a href=\"https://www.cs.ubc.ca/research/image-matching-challenge/2021/leaderboard/\" target=\"_blank\">https://www.cs.ubc.ca/research/image-matching-challenge/2021/leaderboard/</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fe0f4fbda6256645275eb20b40fe242f9%2FScreenshot%202024-04-03%20at%2011.14.09.png?generation=1712135667888200&amp;alt=media\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2F24a32d444e487a5adabf509b494d7c74%2FScreenshot%202024-04-03%20at%2011.15.21.png?generation=1712135737534378&amp;alt=media\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Ff6c4fa1d2dd61b945b30d91281d1680c%2FScreenshot%202024-04-03%20at%2011.16.07.png?generation=1712135783594914&amp;alt=media\"></p>\n<p>Besides IMC-2021 dataset (and all the comparisons in the LightGlue paper itself), here is a comparison on WxBS and EVD datasets<br>\n<a href=\"https://github.com/cvg/glue-factory/pull/52#issue-2065776066\" target=\"_blank\">https://github.com/cvg/glue-factory/pull/52#issue-2065776066</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fdbb3b871c83cfa7ee654a85aa5a3e966%2FScreenshot%202024-04-03%20at%2011.17.05.png?generation=1712135840436199&amp;alt=media\"></p>\n<p>One particular technical issue with LightGlue is that its <em>default</em> parameters are optimized for speed, not maximum performance, so it might seem to be worse than SuperGlue. <br>\nHowever, you can turn off all the speed optimizations with setting up <code>depth_confidence</code> and <code>width_confidence</code> to <code>-1</code>:</p>\n<pre><code>matcher = LightGlue(=, =-1, =-1)\n</code></pre>\n<p>I hope, that this thread, together with license part of the story, will close the SuperGlue issues.</p>",
      "rawMarkdown": "Given the threads [1](https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490622) and [2](https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490714) about license and SuperGlue in particular, we decided to address also a performance question.\n\nWe understand, that given the dominance of SuperGlue in Image Matching Challenge since 2020 until 2023, it is kind of hard to believe that not using it is a good idea. \nHowever, license issues aside, LightGlue is better than SuperGlue performance-wise as well, so there is no good argument to keep using SuperGlue.\n\nSpecifically, let me point out results on IMC-2021 leaderboard, especially with ALIKED features. It provides better result than SuperGlue in all entries, except those, which use additional features and pre/post-processing.\nYou should check the \"MultiView\" score in particular\n\nhttps://www.cs.ubc.ca/research/image-matching-challenge/2021/leaderboard/\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fe0f4fbda6256645275eb20b40fe242f9%2FScreenshot%202024-04-03%20at%2011.14.09.png?generation=1712135667888200&alt=media)\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2F24a32d444e487a5adabf509b494d7c74%2FScreenshot%202024-04-03%20at%2011.15.21.png?generation=1712135737534378&alt=media)\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Ff6c4fa1d2dd61b945b30d91281d1680c%2FScreenshot%202024-04-03%20at%2011.16.07.png?generation=1712135783594914&alt=media)\n\nBesides IMC-2021 dataset (and all the comparisons in the LightGlue paper itself), here is a comparison on WxBS and EVD datasets\nhttps://github.com/cvg/glue-factory/pull/52#issue-2065776066\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fdbb3b871c83cfa7ee654a85aa5a3e966%2FScreenshot%202024-04-03%20at%2011.17.05.png?generation=1712135840436199&alt=media)\n\nOne particular technical issue with LightGlue is that its _default_ parameters are optimized for speed, not maximum performance, so it might seem to be worse than SuperGlue. \nHowever, you can turn off all the speed optimizations with setting up `depth_confidence` and `width_confidence` to `-1`:\n\n```\nmatcher = LightGlue(features='FEATURENAME', depth_confidence=-1, width_confidence=-1)\n```\n\nI hope, that this thread, together with license part of the story, will close the SuperGlue issues.",
      "votes": 27
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2732704": "Given the threads [1](https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490622) and [2](https://www.kaggle.com/competitions/image-matching-challenge-2024/discussion/490714) about license and SuperGlue in particular, we decided to address also a performance question.\n\nWe understand, that given the dominance of SuperGlue in Image Matching Challenge since 2020 until 2023, it is kind of hard to believe that not using it is a good idea. \nHowever, license issues aside, LightGlue is better than SuperGlue performance-wise as well, so there is no good argument to keep using SuperGlue.\n\nSpecifically, let me point out results on IMC-2021 leaderboard, especially with ALIKED features. It provides better result than SuperGlue in all entries, except those, which use additional features and pre/post-processing.\nYou should check the \"MultiView\" score in particular\n\nhttps://www.cs.ubc.ca/research/image-matching-challenge/2021/leaderboard/\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fe0f4fbda6256645275eb20b40fe242f9%2FScreenshot%202024-04-03%20at%2011.14.09.png?generation=1712135667888200&alt=media)\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2F24a32d444e487a5adabf509b494d7c74%2FScreenshot%202024-04-03%20at%2011.15.21.png?generation=1712135737534378&alt=media)\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Ff6c4fa1d2dd61b945b30d91281d1680c%2FScreenshot%202024-04-03%20at%2011.16.07.png?generation=1712135783594914&alt=media)\n\nBesides IMC-2021 dataset (and all the comparisons in the LightGlue paper itself), here is a comparison on WxBS and EVD datasets\nhttps://github.com/cvg/glue-factory/pull/52#issue-2065776066\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F210302%2Fdbb3b871c83cfa7ee654a85aa5a3e966%2FScreenshot%202024-04-03%20at%2011.17.05.png?generation=1712135840436199&alt=media)\n\nOne particular technical issue with LightGlue is that its _default_ parameters are optimized for speed, not maximum performance, so it might seem to be worse than SuperGlue. \nHowever, you can turn off all the speed optimizations with setting up `depth_confidence` and `width_confidence` to `-1`:\n\n```\nmatcher = LightGlue(features='FEATURENAME', depth_confidence=-1, width_confidence=-1)\n```\n\nI hope, that this thread, together with license part of the story, will close the SuperGlue issues."
  }
}