{
  "id": 416816,
  "title": "5th place solution: kNN shortlist and rotation correction",
  "url": "/competitions/image-matching-challenge-2023/discussion/416816",
  "author_name": "Kohei",
  "post_date": "2023-06-13T04:41:50.096000",
  "votes": 44,
  "comment_count": 24,
  "views": 0,
  "content": "<p>I will be in Vancouver for the CVPR IMW. I am very much looking forward to learning from you all!</p>\n<h2>Overview</h2>\n<p>Basically, the stereo matching part is the same as that of the winning solution in 2022, <a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2022/discussion/329131\" target=\"_blank\">the crop and multi-scale ensemble composition by DBSCAN</a>. In addition to this, the improvement of the shortlist, the improvement of computational efficiency, and rotation correction are the main features of my solution.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F522450db593ec4545cbb3ab6d3f59cd6%2Fmatcher.png?generation=1686648814730059&amp;alt=media\" alt=\"\"></p>\n<h2>kNN shortlist + completion</h2>\n<p>In the baseline, the euclidean distance of the global descriptor is used to filter stereo matching candidates by a threshold value. However, since the scale of the distance differs among datasets, I thought it would not be appropriate to use a common threshold value. It’s not robust to unknown data sets.</p>\n<p>Since the number of images per scene is only about 200 at most, stereo matching and RANSAC verification can be performed <strong>using a very lightweight model</strong> for all image combinations. The shortlist was generated by extracting the k nearest neighbors for each image based on the number of inliers.</p>\n<p>The very lightweight model may not match well enough. Some scenes produce images that do not match any of the images. This is fatal because it prevents camera pose estimation. Therefore, image pairs were complementarily added to the shortlist so that there are at least k neighbors for all images by the number of matching keypoints.</p>\n<p><strong>The very lightweight model</strong>: I used SPSG as the very lightweight model. The number of keypoints was set to 512 for efficiency as shown in Fig. 11, Appendix B of superglue's paper.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F6f82729162d57442693ff3fa2b408209%2Finference_time.png?generation=1686630524341903&amp;alt=media\" alt=\"\"></p>\n<h2>Rotation correction</h2>\n<p>A notable weakness of superglue's pretrained model is its lack of robustness to rotation. This is especially noticeable in the heritage dataset of the IMC2023 training set, cyprus scene. I addressed this issue without training by just rotating images.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F212bfdaf476e1ec79efa5692162d99d2%2Fcyprus1.png?generation=1686630454984938&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F75ce985499c56086b721df6a3e63e5d1%2Fcyprus2.png?generation=1686630470817962&amp;alt=media\" alt=\"\"></p>\n<p>Specifically, 4 different rotated images were prepared and matched using the very lightweight model. All images were resized to the same size (840x840) and processed in one batch.</p>\n<h2>Parallel Execution</h2>\n<p>GPU-intensive and CPU-intensive tasks can be executed in parallel for efficiency. Specifically, COLMAP BA is a CPU-intensive task, but stereo matching is a GPU-intensive task. Therefore, I implemented COLMAP processing in a separate thread and run it in parallel with stereo matching.</p>\n<h2>Local Validation</h2>\n<p>Only some difficult scenes were verified in the local environment. In particular, dioscuri, which has 174 images, was used as a reference to adjust the algorithm so that it would not time out.</p>\n<table>\n<thead>\n<tr>\n<th>scene</th>\n<th># images</th>\n<th>score</th>\n<th>time (local env)</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>bike</td>\n<td>15</td>\n<td>0.9342</td>\n<td>134 s</td>\n</tr>\n<tr>\n<td>kyiv-puppet-theater</td>\n<td>27</td>\n<td>0.7781</td>\n<td>291 s</td>\n</tr>\n<tr>\n<td>cyprus</td>\n<td>30</td>\n<td>0.6239</td>\n<td>312 s</td>\n</tr>\n<tr>\n<td>wall</td>\n<td>43</td>\n<td>0.4753</td>\n<td>851 s</td>\n</tr>\n<tr>\n<td>dioscuri</td>\n<td>174</td>\n<td>0.8775</td>\n<td>2045 s</td>\n</tr>\n</tbody>\n</table>\n<h2>Processing time</h2>\n<p>My best submisison run time was <strong>7 hours and 40 minutes</strong>. This is well under the time limit of 9 hours. Therefore, I added LoFTR to ensemble on the last day, but the Kaggle server is broken and the notebook is still running. If the notebook had been successfully processed, My best submission could have been a little better.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F538925ab8b7ece79f2630f2a9f18708e%2Ferrors.png?generation=1686630579463016&amp;alt=media\" alt=\"\"></p>\n<p>The issue of not being able to submit on the last day is very stressful. Something similar occurred with Google Landmark Recognition in 2019. I pray that this kind of trouble will not occur again🙏</p>\n<h2>Thing that didn't work</h2>\n<p>Too many things to try to do alone :-)</p>\n<ul>\n<li>TTA … no significant improvement</li>\n<li>Pixel-Perfect SfM .. I got a slight improvement in my local environment but gave up because I couldn't set up ceres in my kaggle notebook</li>\n<li>TensorRT .. I had to downgrade from PyTorch 2.0, but the setup was successful. I can't say with much certainty, but at least my experiments did not improve execution speed.</li>\n<li>Half precision … It was about 10%(?) faster, but mAA was worse, so it was not adopted in the end.</li>\n<li>other matchers … Tried DKMv3, LoFTR, Silk, but finally used only SPSG and MatchFormer due to high mAA.</li>\n<li>OpenGLUE … Trained DISK+OpenGLUE and SuperPoint+OpenGLUE on the MegaDepth dataset but did not reach the scores of the SPSG pretrained model. Converted phototourist dataset to OpenGLUE format and built MegaDepth + phototourist dataset, but did not validate it because of lack of time.</li>\n<li>Incremental Mapper parameter tuning … contributed to the stable high score in the local environment. Specifically, I reduced the max refinement change of BA and increased the max num iteration. However, all the submissions to kaggle were out of memory, so I did not check the improvement on the leaderboard.</li>\n</ul>",
  "messages": [
    {
      "id": 2300191,
      "postDate": "2023-06-13T04:41:50.097Z",
      "content": "<p>I will be in Vancouver for the CVPR IMW. I am very much looking forward to learning from you all!</p>\n<h2>Overview</h2>\n<p>Basically, the stereo matching part is the same as that of the winning solution in 2022, <a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2022/discussion/329131\" target=\"_blank\">the crop and multi-scale ensemble composition by DBSCAN</a>. In addition to this, the improvement of the shortlist, the improvement of computational efficiency, and rotation correction are the main features of my solution.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F522450db593ec4545cbb3ab6d3f59cd6%2Fmatcher.png?generation=1686648814730059&amp;alt=media\" alt=\"\"></p>\n<h2>kNN shortlist + completion</h2>\n<p>In the baseline, the euclidean distance of the global descriptor is used to filter stereo matching candidates by a threshold value. However, since the scale of the distance differs among datasets, I thought it would not be appropriate to use a common threshold value. It’s not robust to unknown data sets.</p>\n<p>Since the number of images per scene is only about 200 at most, stereo matching and RANSAC verification can be performed <strong>using a very lightweight model</strong> for all image combinations. The shortlist was generated by extracting the k nearest neighbors for each image based on the number of inliers.</p>\n<p>The very lightweight model may not match well enough. Some scenes produce images that do not match any of the images. This is fatal because it prevents camera pose estimation. Therefore, image pairs were complementarily added to the shortlist so that there are at least k neighbors for all images by the number of matching keypoints.</p>\n<p><strong>The very lightweight model</strong>: I used SPSG as the very lightweight model. The number of keypoints was set to 512 for efficiency as shown in Fig. 11, Appendix B of superglue's paper.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F6f82729162d57442693ff3fa2b408209%2Finference_time.png?generation=1686630524341903&amp;alt=media\" alt=\"\"></p>\n<h2>Rotation correction</h2>\n<p>A notable weakness of superglue's pretrained model is its lack of robustness to rotation. This is especially noticeable in the heritage dataset of the IMC2023 training set, cyprus scene. I addressed this issue without training by just rotating images.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F212bfdaf476e1ec79efa5692162d99d2%2Fcyprus1.png?generation=1686630454984938&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F75ce985499c56086b721df6a3e63e5d1%2Fcyprus2.png?generation=1686630470817962&amp;alt=media\" alt=\"\"></p>\n<p>Specifically, 4 different rotated images were prepared and matched using the very lightweight model. All images were resized to the same size (840x840) and processed in one batch.</p>\n<h2>Parallel Execution</h2>\n<p>GPU-intensive and CPU-intensive tasks can be executed in parallel for efficiency. Specifically, COLMAP BA is a CPU-intensive task, but stereo matching is a GPU-intensive task. Therefore, I implemented COLMAP processing in a separate thread and run it in parallel with stereo matching.</p>\n<h2>Local Validation</h2>\n<p>Only some difficult scenes were verified in the local environment. In particular, dioscuri, which has 174 images, was used as a reference to adjust the algorithm so that it would not time out.</p>\n<table>\n<thead>\n<tr>\n<th>scene</th>\n<th># images</th>\n<th>score</th>\n<th>time (local env)</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>bike</td>\n<td>15</td>\n<td>0.9342</td>\n<td>134 s</td>\n</tr>\n<tr>\n<td>kyiv-puppet-theater</td>\n<td>27</td>\n<td>0.7781</td>\n<td>291 s</td>\n</tr>\n<tr>\n<td>cyprus</td>\n<td>30</td>\n<td>0.6239</td>\n<td>312 s</td>\n</tr>\n<tr>\n<td>wall</td>\n<td>43</td>\n<td>0.4753</td>\n<td>851 s</td>\n</tr>\n<tr>\n<td>dioscuri</td>\n<td>174</td>\n<td>0.8775</td>\n<td>2045 s</td>\n</tr>\n</tbody>\n</table>\n<h2>Processing time</h2>\n<p>My best submisison run time was <strong>7 hours and 40 minutes</strong>. This is well under the time limit of 9 hours. Therefore, I added LoFTR to ensemble on the last day, but the Kaggle server is broken and the notebook is still running. If the notebook had been successfully processed, My best submission could have been a little better.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F538925ab8b7ece79f2630f2a9f18708e%2Ferrors.png?generation=1686630579463016&amp;alt=media\" alt=\"\"></p>\n<p>The issue of not being able to submit on the last day is very stressful. Something similar occurred with Google Landmark Recognition in 2019. I pray that this kind of trouble will not occur again🙏</p>\n<h2>Thing that didn't work</h2>\n<p>Too many things to try to do alone :-)</p>\n<ul>\n<li>TTA … no significant improvement</li>\n<li>Pixel-Perfect SfM .. I got a slight improvement in my local environment but gave up because I couldn't set up ceres in my kaggle notebook</li>\n<li>TensorRT .. I had to downgrade from PyTorch 2.0, but the setup was successful. I can't say with much certainty, but at least my experiments did not improve execution speed.</li>\n<li>Half precision … It was about 10%(?) faster, but mAA was worse, so it was not adopted in the end.</li>\n<li>other matchers … Tried DKMv3, LoFTR, Silk, but finally used only SPSG and MatchFormer due to high mAA.</li>\n<li>OpenGLUE … Trained DISK+OpenGLUE and SuperPoint+OpenGLUE on the MegaDepth dataset but did not reach the scores of the SPSG pretrained model. Converted phototourist dataset to OpenGLUE format and built MegaDepth + phototourist dataset, but did not validate it because of lack of time.</li>\n<li>Incremental Mapper parameter tuning … contributed to the stable high score in the local environment. Specifically, I reduced the max refinement change of BA and increased the max num iteration. However, all the submissions to kaggle were out of memory, so I did not check the improvement on the leaderboard.</li>\n</ul>",
      "rawMarkdown": "I will be in Vancouver for the CVPR IMW. I am very much looking forward to learning from you all!\n\n## Overview\n\nBasically, the stereo matching part is the same as that of the winning solution in 2022, [the crop and multi-scale ensemble composition by DBSCAN](https://www.kaggle.com/competitions/image-matching-challenge-2022/discussion/329131). In addition to this, the improvement of the shortlist, the improvement of computational efficiency, and rotation correction are the main features of my solution.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F522450db593ec4545cbb3ab6d3f59cd6%2Fmatcher.png?generation=1686648814730059&alt=media)\n\n## kNN shortlist + completion\n\nIn the baseline, the euclidean distance of the global descriptor is used to filter stereo matching candidates by a threshold value. However, since the scale of the distance differs among datasets, I thought it would not be appropriate to use a common threshold value. It’s not robust to unknown data sets.\n\nSince the number of images per scene is only about 200 at most, stereo matching and RANSAC verification can be performed **using a very lightweight model** for all image combinations. The shortlist was generated by extracting the k nearest neighbors for each image based on the number of inliers.\n\nThe very lightweight model may not match well enough. Some scenes produce images that do not match any of the images. This is fatal because it prevents camera pose estimation. Therefore, image pairs were complementarily added to the shortlist so that there are at least k neighbors for all images by the number of matching keypoints.\n\n**The very lightweight model**: I used SPSG as the very lightweight model. The number of keypoints was set to 512 for efficiency as shown in Fig. 11, Appendix B of superglue's paper.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F6f82729162d57442693ff3fa2b408209%2Finference_time.png?generation=1686630524341903&alt=media)\n\n## Rotation correction\n\nA notable weakness of superglue's pretrained model is its lack of robustness to rotation. This is especially noticeable in the heritage dataset of the IMC2023 training set, cyprus scene. I addressed this issue without training by just rotating images.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F212bfdaf476e1ec79efa5692162d99d2%2Fcyprus1.png?generation=1686630454984938&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F75ce985499c56086b721df6a3e63e5d1%2Fcyprus2.png?generation=1686630470817962&alt=media)\n\nSpecifically, 4 different rotated images were prepared and matched using the very lightweight model. All images were resized to the same size (840x840) and processed in one batch.\n\n## Parallel Execution\n\nGPU-intensive and CPU-intensive tasks can be executed in parallel for efficiency. Specifically, COLMAP BA is a CPU-intensive task, but stereo matching is a GPU-intensive task. Therefore, I implemented COLMAP processing in a separate thread and run it in parallel with stereo matching.\n\n## Local Validation\n\nOnly some difficult scenes were verified in the local environment. In particular, dioscuri, which has 174 images, was used as a reference to adjust the algorithm so that it would not time out.\n\n| scene | # images |  score | time (local env) |\n|---|---|--|---|\n| bike  | 15 | 0.9342 | 134 s |\n| kyiv-puppet-theater | 27 | 0.7781 | 291 s |\n| cyprus | 30 | 0.6239 | 312 s |\n| wall | 43 | 0.4753 | 851 s |\n| dioscuri | 174 | 0.8775 | 2045 s |\n\n## Processing time\n\nMy best submisison run time was **7 hours and 40 minutes**. This is well under the time limit of 9 hours. Therefore, I added LoFTR to ensemble on the last day, but the Kaggle server is broken and the notebook is still running. If the notebook had been successfully processed, My best submission could have been a little better.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F538925ab8b7ece79f2630f2a9f18708e%2Ferrors.png?generation=1686630579463016&alt=media)\n\nThe issue of not being able to submit on the last day is very stressful. Something similar occurred with Google Landmark Recognition in 2019. I pray that this kind of trouble will not occur again:pray:\n\n## Thing that didn't work\n\nToo many things to try to do alone :-)\n\n- TTA … no significant improvement\n- Pixel-Perfect SfM .. I got a slight improvement in my local environment but gave up because I couldn't set up ceres in my kaggle notebook\n- TensorRT .. I had to downgrade from PyTorch 2.0, but the setup was successful. I can't say with much certainty, but at least my experiments did not improve execution speed.\n- Half precision … It was about 10%(?) faster, but mAA was worse, so it was not adopted in the end.\n- other matchers … Tried DKMv3, LoFTR, Silk, but finally used only SPSG and MatchFormer due to high mAA.\n- OpenGLUE ... Trained DISK+OpenGLUE and SuperPoint+OpenGLUE on the MegaDepth dataset but did not reach the scores of the SPSG pretrained model. Converted phototourist dataset to OpenGLUE format and built MegaDepth + phototourist dataset, but did not validate it because of lack of time.\n- Incremental Mapper parameter tuning ... contributed to the stable high score in the local environment. Specifically, I reduced the max refinement change of BA and increased the max num iteration. However, all the submissions to kaggle were out of memory, so I did not check the improvement on the leaderboard.",
      "votes": 44
    },
    {
      "id": 2300362,
      "postDate": "2023-06-13T06:54:54.177Z",
      "content": "<p><a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a> Congrats for the solo gold, interesting approach.<br>\nFor Pixel-Perfect we have ran it successfully on kaggle but </p>\n<ul>\n<li>For reconstruction<br>\n1- It will not pass for large scenes because of memory issues<br>\n2- For scenes less than 40 images it caused a huge drop</li>\n<li>For refinement only<br>\n1- It caused a drop as well <br>\nIf you would like to try it again here are the datasets<br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/sfm-pre-pkgs\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/sfm-pre-pkgs</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/pyceres\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/pyceres</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/pixsfm\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/pixsfm</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/pycolmap-040\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/pycolmap-040</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/nvidia-libs\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/nvidia-libs</a></li>\n</ul>",
      "rawMarkdown": "@confirm Congrats for the solo gold, interesting approach.\nFor Pixel-Perfect we have ran it successfully on kaggle but \n- For reconstruction\n1- It will not pass for large scenes because of memory issues\n2- For scenes less than 40 images it caused a huge drop\n- For refinement only\n1- It caused a drop as well \nIf you would like to try it again here are the datasets\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/sfm-pre-pkgs\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/pyceres\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/pixsfm\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/pycolmap-040\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/nvidia-libs",
      "votes": 5,
      "replies": [
        {
          "id": 2300400,
          "postDate": "2023-06-13T07:24:05.267Z",
          "content": "<p>Great findings! Thanks for sharing the dataset! Congratulations on your Gold Medal!<br>\nI enjoyed your team name, TLE forever. I saw endless TLEs and OOMs too.</p>",
          "rawMarkdown": "Great findings! Thanks for sharing the dataset! Congratulations on your Gold Medal!\nI enjoyed your team name, TLE forever. I saw endless TLEs and OOMs too.",
          "votes": 2,
          "replies": [
            {
              "id": 2301076,
              "postDate": "2023-06-13T15:50:21.307Z",
              "content": "<p>😂 Yeah TLEs more than passed submissions 😂😂</p>",
              "rawMarkdown": "😂 Yeah TLEs more than passed submissions 😂😂",
              "votes": 2
            }
          ]
        }
      ]
    },
    {
      "id": 2300333,
      "postDate": "2023-06-13T06:23:56.233Z",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a>! You have nerves of steel. As far as I remember yesterday you were still in the silver zone. It is important to withstand the stress and choose the right solution. Congrats on both your nerve control, your believe in the solution and your great submission choice. 👋👋👋💪👍</p>",
      "rawMarkdown": "Congratulations @confirm! You have nerves of steel. As far as I remember yesterday you were still in the silver zone. It is important to withstand the stress and choose the right solution. Congrats on both your nerve control, your believe in the solution and your great submission choice. 👋👋👋💪👍",
      "votes": 5,
      "replies": [
        {
          "id": 2300381,
          "postDate": "2023-06-13T07:14:11.707Z",
          "content": "<p>Thanks for your comment! Congratulations <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> and your teammates!! I was confident that I could move up, but I didn't think it would be this big jump. I still can't stop my excitement :-)</p>",
          "rawMarkdown": "Thanks for your comment! Congratulations @remekkinas and your teammates!! I was confident that I could move up, but I didn't think it would be this big jump. I still can't stop my excitement :-)",
          "votes": 1
        }
      ]
    },
    {
      "id": 2300288,
      "postDate": "2023-06-13T05:50:54.467Z",
      "content": "<p><a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a> Addressing the rotation issue was an important step, nice approach! Great performance tricks! Thanks for clear write-up.<br>\nCongratulations with a solo gold and green zone!</p>",
      "rawMarkdown": "@confirm Addressing the rotation issue was an important step, nice approach! Great performance tricks! Thanks for clear write-up.\nCongratulations with a solo gold and green zone!",
      "votes": 3,
      "replies": [
        {
          "id": 2300366,
          "postDate": "2023-06-13T06:56:01.143Z",
          "content": "<p>Thank you!! Congratulations to you too! I look forward to learning from your team's solutions.</p>",
          "rawMarkdown": "Thank you!! Congratulations to you too! I look forward to learning from your team's solutions.",
          "votes": 1
        }
      ]
    },
    {
      "id": 2300285,
      "postDate": "2023-06-13T05:47:59.020Z",
      "content": "<p>Thank you for this write-up &amp; congrats on the result!</p>\n<p>How much of an improvement did the rotations bring? We experimented with rotations as well, using a pretrained angle prediction model that we found online: <a href=\"https://github.com/pidahbus/deep-image-orientation-angle-detection\" target=\"_blank\">https://github.com/pidahbus/deep-image-orientation-angle-detection</a> We rotated the image and rotated the matched keypoints afterwards, but didn't see much of an improvement?</p>",
      "rawMarkdown": "Thank you for this write-up & congrats on the result!\n\nHow much of an improvement did the rotations bring? We experimented with rotations as well, using a pretrained angle prediction model that we found online: https://github.com/pidahbus/deep-image-orientation-angle-detection We rotated the image and rotated the matched keypoints afterwards, but didn't see much of an improvement?",
      "votes": 3,
      "replies": [
        {
          "id": 2300327,
          "postDate": "2023-06-13T06:20:02.143Z",
          "content": "<p>I think it depends on what feature detectors/matchers you were using, what we found is that, if they are already rotation invariant (like the KeyNetAffNetHardNet + Adalam combo in the baseline), then rotating the images back and forth won't bring an improvement, but if it's not really rotation invariant (like SuperPoint + SG or LoFTR), then adding this step will bring significant improvement.</p>",
          "rawMarkdown": "I think it depends on what feature detectors/matchers you were using, what we found is that, if they are already rotation invariant (like the KeyNetAffNetHardNet + Adalam combo in the baseline), then rotating the images back and forth won't bring an improvement, but if it's not really rotation invariant (like SuperPoint + SG or LoFTR), then adding this step will bring significant improvement.",
          "votes": 3
        }
      ]
    },
    {
      "id": 2300275,
      "postDate": "2023-06-13T05:37:53.023Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a>.  It's a clear and brilliant solution. I love the rotation part. It is what I have been striving to implement throughout the last month, trying dozens of methods but ultimately failed to find a way to achieve. </p>\n<p>Congratulations on obtaining yet another solo gold medal. 🎉</p>\n<hr>\n<p>May you make a clearer description for the rotation part? Did you rotate each image for 4 times? Since it's an pair, it make a batch size of 16. Did you first use the lightweight spsg to match on the batch16 for stage 1, then use a heavy spsg or spsg+matchformer for stage 2?</p>",
      "rawMarkdown": "Hi @confirm.  It's a clear and brilliant solution. I love the rotation part. It is what I have been striving to implement throughout the last month, trying dozens of methods but ultimately failed to find a way to achieve. \n\nCongratulations on obtaining yet another solo gold medal. 🎉\n\n**************************************\nMay you make a clearer description for the rotation part? Did you rotate each image for 4 times? Since it's an pair, it make a batch size of 16. Did you first use the lightweight spsg to match on the batch16 for stage 1, then use a heavy spsg or spsg+matchformer for stage 2?\n",
      "votes": 4,
      "replies": [
        {
          "id": 2300349,
          "postDate": "2023-06-13T06:45:39.310Z",
          "content": "<p>Thanks for your comment! It was very encouraging to see you at the top of the leaderboard early in the contest.</p>\n<p>Also, thanks a lot for the question! I realize that my explanation is not very accurate. I will fix it later. Only one of the pair rotates. One of the four is not rotated. This very lightweight model is only used to compensate for rotation and build a shortlist. The keypoints calculated here are not used in the two-stage matcher.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F68b3c0c1944e1a4f430db4feec0e6b42%2Fkpts.png?generation=1686639018524622&amp;alt=media\" alt=\"\"></p>",
          "rawMarkdown": "Thanks for your comment! It was very encouraging to see you at the top of the leaderboard early in the contest.\n\nAlso, thanks a lot for the question! I realize that my explanation is not very accurate. I will fix it later. Only one of the pair rotates. One of the four is not rotated. This very lightweight model is only used to compensate for rotation and build a shortlist. The keypoints calculated here are not used in the two-stage matcher.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F68b3c0c1944e1a4f430db4feec0e6b42%2Fkpts.png?generation=1686639018524622&alt=media)",
          "votes": 6,
          "replies": [
            {
              "id": 2300509,
              "postDate": "2023-06-13T08:24:59.180Z",
              "content": "<p>Wow, I would't expect that just rotating the keypoints without change in the descriptors would work. Myself I would probably rotate image, and then extract features from there, so having 4 SP passes.</p>",
              "rawMarkdown": "Wow, I would't expect that just rotating the keypoints without change in the descriptors would work. Myself I would probably rotate image, and then extract features from there, so having 4 SP passes.",
              "votes": 4
            },
            {
              "id": 2300531,
              "postDate": "2023-06-13T08:40:07.953Z",
              "content": "<p>Great! You are right. This may be my mistake. I focused too much on saving GPU runtime. In fact the computational cost of keypoint extraction is relatively small.</p>",
              "rawMarkdown": "Great! You are right. This may be my mistake. I focused too much on saving GPU runtime. In fact the computational cost of keypoint extraction is relatively small.",
              "votes": 2
            }
          ]
        }
      ]
    },
    {
      "id": 2300528,
      "postDate": "2023-06-13T08:38:10.990Z",
      "content": "<p>Do you have numbers for the other matchers and features?</p>",
      "rawMarkdown": "Do you have numbers for the other matchers and features?",
      "votes": 1,
      "replies": [
        {
          "id": 2300595,
          "postDate": "2023-06-13T09:29:08.403Z",
          "content": "<p>Thanks for the question! Just updated the description. I used 5 matchers in a large scene. 9 matchers for small scenes.</p>",
          "rawMarkdown": "Thanks for the question! Just updated the description. I used 5 matchers in a large scene. 9 matchers for small scenes.",
          "votes": 1,
          "replies": [
            {
              "id": 2300609,
              "postDate": "2023-06-13T09:42:33.003Z",
              "content": "<p>Thank you. I actually meant the mAA results on val/public/private datasets to show the effect of them, as well things didn’t work </p>",
              "rawMarkdown": "Thank you. I actually meant the mAA results on val/public/private datasets to show the effect of them, as well things didn’t work ",
              "votes": 1
            },
            {
              "id": 2301661,
              "postDate": "2023-06-14T04:49:50.437Z",
              "content": "<p>Understood. I am curious about those numbers too. I will have it ready by the workshop!<br>\nIn my memory, SPSG, crop and rotation correction were particularly important. I think the other changes were not that significant.</p>",
              "rawMarkdown": "Understood. I am curious about those numbers too. I will have it ready by the workshop!\nIn my memory, SPSG, crop and rotation correction were particularly important. I think the other changes were not that significant.",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 2300512,
      "postDate": "2023-06-13T08:26:39.260Z",
      "content": "<p>Great write-up!</p>",
      "rawMarkdown": "Great write-up!",
      "votes": 2
    },
    {
      "id": 2312701,
      "postDate": "2023-06-22T06:15:33.703Z",
      "content": "<p>Congrats, the legend. </p>",
      "rawMarkdown": "Congrats, the legend. "
    },
    {
      "id": 2303389,
      "postDate": "2023-06-15T09:17:06.830Z",
      "content": "<p>Hi, congrats on solo gold and thanks you for write-up.<br>\nCould you please explain more the knn part. Are you willing to share your code?<br>\nthanks </p>",
      "rawMarkdown": "Hi, congrats on solo gold and thanks you for write-up.\nCould you please explain more the knn part. Are you willing to share your code?\nthanks \n "
    },
    {
      "id": 2302322,
      "postDate": "2023-06-14T13:27:55.983Z",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a>!  Complete and great solution.</p>\n<p>My question is, we were also working on colmap parameter tuning, but it didn't work because of the randomness of the CV scores. Please let me know if you have any ideas on how to optimize in such an unstable situation.</p>",
      "rawMarkdown": "Congratulations @confirm!  Complete and great solution.\n\nMy question is, we were also working on colmap parameter tuning, but it didn't work because of the randomness of the CV scores. Please let me know if you have any ideas on how to optimize in such an unstable situation."
    },
    {
      "id": 2301341,
      "postDate": "2023-06-13T19:58:08.903Z",
      "content": "<p>Brilliant idea for rotation correction! excellent job!</p>",
      "rawMarkdown": "Brilliant idea for rotation correction! excellent job!"
    },
    {
      "id": 2300507,
      "postDate": "2023-06-13T08:23:05.880Z",
      "content": "<p>I am curious. what did you used matchformer for? Was it working better than SG?</p>",
      "rawMarkdown": "I am curious. what did you used matchformer for? Was it working better than SG?"
    },
    {
      "id": 2300270,
      "postDate": "2023-06-13T05:33:48.757Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2300362,
      "author_name": "ammarali32",
      "author_url": "",
      "post_date": "2023-06-13T06:54:54.177000",
      "content": "<p><a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a> Congrats for the solo gold, interesting approach.<br>\nFor Pixel-Perfect we have ran it successfully on kaggle but </p>\n<ul>\n<li>For reconstruction<br>\n1- It will not pass for large scenes because of memory issues<br>\n2- For scenes less than 40 images it caused a huge drop</li>\n<li>For refinement only<br>\n1- It caused a drop as well <br>\nIf you would like to try it again here are the datasets<br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/sfm-pre-pkgs\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/sfm-pre-pkgs</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/pyceres\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/pyceres</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/pixsfm\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/pixsfm</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/pycolmap-040\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/pycolmap-040</a><br>\n<a href=\"https://www.kaggle.com/datasets/jaafarmahmoud1/nvidia-libs\" target=\"_blank\">https://www.kaggle.com/datasets/jaafarmahmoud1/nvidia-libs</a></li>\n</ul>",
      "votes": 5,
      "replies": [
        {
          "id": 2300400,
          "author_name": "Kohei",
          "author_url": "",
          "post_date": "2023-06-13T07:24:05.267000",
          "content": "<p>Great findings! Thanks for sharing the dataset! Congratulations on your Gold Medal!<br>\nI enjoyed your team name, TLE forever. I saw endless TLEs and OOMs too.</p>",
          "votes": 2,
          "replies": [
            {
              "id": 2301076,
              "author_name": "ammarali32",
              "author_url": "",
              "post_date": "2023-06-13T15:50:21.307000",
              "content": "<p>😂 Yeah TLEs more than passed submissions 😂😂</p>",
              "votes": 2,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2300333,
      "author_name": "Remek Kinas",
      "author_url": "",
      "post_date": "2023-06-13T06:23:56.233000",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a>! You have nerves of steel. As far as I remember yesterday you were still in the silver zone. It is important to withstand the stress and choose the right solution. Congrats on both your nerve control, your believe in the solution and your great submission choice. 👋👋👋💪👍</p>",
      "votes": 5,
      "replies": [
        {
          "id": 2300381,
          "author_name": "Kohei",
          "author_url": "",
          "post_date": "2023-06-13T07:14:11.707000",
          "content": "<p>Thanks for your comment! Congratulations <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> and your teammates!! I was confident that I could move up, but I didn't think it would be this big jump. I still can't stop my excitement :-)</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2300288,
      "author_name": "Igor Lashkov",
      "author_url": "",
      "post_date": "2023-06-13T05:50:54.467000",
      "content": "<p><a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a> Addressing the rotation issue was an important step, nice approach! Great performance tricks! Thanks for clear write-up.<br>\nCongratulations with a solo gold and green zone!</p>",
      "votes": 3,
      "replies": [
        {
          "id": 2300366,
          "author_name": "Kohei",
          "author_url": "",
          "post_date": "2023-06-13T06:56:01.143000",
          "content": "<p>Thank you!! Congratulations to you too! I look forward to learning from your team's solutions.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2300285,
      "author_name": "Gilles Vandewiele",
      "author_url": "",
      "post_date": "2023-06-13T05:47:59.020000",
      "content": "<p>Thank you for this write-up &amp; congrats on the result!</p>\n<p>How much of an improvement did the rotations bring? We experimented with rotations as well, using a pretrained angle prediction model that we found online: <a href=\"https://github.com/pidahbus/deep-image-orientation-angle-detection\" target=\"_blank\">https://github.com/pidahbus/deep-image-orientation-angle-detection</a> We rotated the image and rotated the matched keypoints afterwards, but didn't see much of an improvement?</p>",
      "votes": 3,
      "replies": [
        {
          "id": 2300327,
          "author_name": "Yuxiang Huang",
          "author_url": "",
          "post_date": "2023-06-13T06:20:02.143000",
          "content": "<p>I think it depends on what feature detectors/matchers you were using, what we found is that, if they are already rotation invariant (like the KeyNetAffNetHardNet + Adalam combo in the baseline), then rotating the images back and forth won't bring an improvement, but if it's not really rotation invariant (like SuperPoint + SG or LoFTR), then adding this step will bring significant improvement.</p>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 2300275,
      "author_name": "ForcewithMe",
      "author_url": "",
      "post_date": "2023-06-13T05:37:53.023000",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a>.  It's a clear and brilliant solution. I love the rotation part. It is what I have been striving to implement throughout the last month, trying dozens of methods but ultimately failed to find a way to achieve. </p>\n<p>Congratulations on obtaining yet another solo gold medal. 🎉</p>\n<hr>\n<p>May you make a clearer description for the rotation part? Did you rotate each image for 4 times? Since it's an pair, it make a batch size of 16. Did you first use the lightweight spsg to match on the batch16 for stage 1, then use a heavy spsg or spsg+matchformer for stage 2?</p>",
      "votes": 4,
      "replies": [
        {
          "id": 2300349,
          "author_name": "Kohei",
          "author_url": "",
          "post_date": "2023-06-13T06:45:39.310000",
          "content": "<p>Thanks for your comment! It was very encouraging to see you at the top of the leaderboard early in the contest.</p>\n<p>Also, thanks a lot for the question! I realize that my explanation is not very accurate. I will fix it later. Only one of the pair rotates. One of the four is not rotated. This very lightweight model is only used to compensate for rotation and build a shortlist. The keypoints calculated here are not used in the two-stage matcher.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F68b3c0c1944e1a4f430db4feec0e6b42%2Fkpts.png?generation=1686639018524622&amp;alt=media\" alt=\"\"></p>",
          "votes": 6,
          "replies": [
            {
              "id": 2300509,
              "author_name": "old-ufo",
              "author_url": "",
              "post_date": "2023-06-13T08:24:59.180000",
              "content": "<p>Wow, I would't expect that just rotating the keypoints without change in the descriptors would work. Myself I would probably rotate image, and then extract features from there, so having 4 SP passes.</p>",
              "votes": 4,
              "replies": []
            },
            {
              "id": 2300531,
              "author_name": "Kohei",
              "author_url": "",
              "post_date": "2023-06-13T08:40:07.953000",
              "content": "<p>Great! You are right. This may be my mistake. I focused too much on saving GPU runtime. In fact the computational cost of keypoint extraction is relatively small.</p>",
              "votes": 2,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2300528,
      "author_name": "old-ufo",
      "author_url": "",
      "post_date": "2023-06-13T08:38:10.990000",
      "content": "<p>Do you have numbers for the other matchers and features?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2300595,
          "author_name": "Kohei",
          "author_url": "",
          "post_date": "2023-06-13T09:29:08.403000",
          "content": "<p>Thanks for the question! Just updated the description. I used 5 matchers in a large scene. 9 matchers for small scenes.</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2300609,
              "author_name": "old-ufo",
              "author_url": "",
              "post_date": "2023-06-13T09:42:33.003000",
              "content": "<p>Thank you. I actually meant the mAA results on val/public/private datasets to show the effect of them, as well things didn’t work </p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2301661,
              "author_name": "Kohei",
              "author_url": "",
              "post_date": "2023-06-14T04:49:50.437000",
              "content": "<p>Understood. I am curious about those numbers too. I will have it ready by the workshop!<br>\nIn my memory, SPSG, crop and rotation correction were particularly important. I think the other changes were not that significant.</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2300512,
      "author_name": "old-ufo",
      "author_url": "",
      "post_date": "2023-06-13T08:26:39.260000",
      "content": "<p>Great write-up!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2312701,
      "author_name": "Yiemon773",
      "author_url": "",
      "post_date": "2023-06-22T06:15:33.703000",
      "content": "<p>Congrats, the legend. </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2303389,
      "author_name": "TanjiroLL",
      "author_url": "",
      "post_date": "2023-06-15T09:17:06.830000",
      "content": "<p>Hi, congrats on solo gold and thanks you for write-up.<br>\nCould you please explain more the knn part. Are you willing to share your code?<br>\nthanks </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2302322,
      "author_name": "Isamu",
      "author_url": "",
      "post_date": "2023-06-14T13:27:55.983000",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/confirm\" target=\"_blank\">@confirm</a>!  Complete and great solution.</p>\n<p>My question is, we were also working on colmap parameter tuning, but it didn't work because of the randomness of the CV scores. Please let me know if you have any ideas on how to optimize in such an unstable situation.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2301341,
      "author_name": "Fang",
      "author_url": "",
      "post_date": "2023-06-13T19:58:08.903000",
      "content": "<p>Brilliant idea for rotation correction! excellent job!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2300507,
      "author_name": "old-ufo",
      "author_url": "",
      "post_date": "2023-06-13T08:23:05.880000",
      "content": "<p>I am curious. what did you used matchformer for? Was it working better than SG?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2300270,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-06-13T05:33:48.757000",
      "content": "",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2300191": "I will be in Vancouver for the CVPR IMW. I am very much looking forward to learning from you all!\n\n## Overview\n\nBasically, the stereo matching part is the same as that of the winning solution in 2022, [the crop and multi-scale ensemble composition by DBSCAN](https://www.kaggle.com/competitions/image-matching-challenge-2022/discussion/329131). In addition to this, the improvement of the shortlist, the improvement of computational efficiency, and rotation correction are the main features of my solution.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F522450db593ec4545cbb3ab6d3f59cd6%2Fmatcher.png?generation=1686648814730059&alt=media)\n\n## kNN shortlist + completion\n\nIn the baseline, the euclidean distance of the global descriptor is used to filter stereo matching candidates by a threshold value. However, since the scale of the distance differs among datasets, I thought it would not be appropriate to use a common threshold value. It’s not robust to unknown data sets.\n\nSince the number of images per scene is only about 200 at most, stereo matching and RANSAC verification can be performed **using a very lightweight model** for all image combinations. The shortlist was generated by extracting the k nearest neighbors for each image based on the number of inliers.\n\nThe very lightweight model may not match well enough. Some scenes produce images that do not match any of the images. This is fatal because it prevents camera pose estimation. Therefore, image pairs were complementarily added to the shortlist so that there are at least k neighbors for all images by the number of matching keypoints.\n\n**The very lightweight model**: I used SPSG as the very lightweight model. The number of keypoints was set to 512 for efficiency as shown in Fig. 11, Appendix B of superglue's paper.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F6f82729162d57442693ff3fa2b408209%2Finference_time.png?generation=1686630524341903&alt=media)\n\n## Rotation correction\n\nA notable weakness of superglue's pretrained model is its lack of robustness to rotation. This is especially noticeable in the heritage dataset of the IMC2023 training set, cyprus scene. I addressed this issue without training by just rotating images.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F212bfdaf476e1ec79efa5692162d99d2%2Fcyprus1.png?generation=1686630454984938&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F75ce985499c56086b721df6a3e63e5d1%2Fcyprus2.png?generation=1686630470817962&alt=media)\n\nSpecifically, 4 different rotated images were prepared and matched using the very lightweight model. All images were resized to the same size (840x840) and processed in one batch.\n\n## Parallel Execution\n\nGPU-intensive and CPU-intensive tasks can be executed in parallel for efficiency. Specifically, COLMAP BA is a CPU-intensive task, but stereo matching is a GPU-intensive task. Therefore, I implemented COLMAP processing in a separate thread and run it in parallel with stereo matching.\n\n## Local Validation\n\nOnly some difficult scenes were verified in the local environment. In particular, dioscuri, which has 174 images, was used as a reference to adjust the algorithm so that it would not time out.\n\n| scene | # images |  score | time (local env) |\n|---|---|--|---|\n| bike  | 15 | 0.9342 | 134 s |\n| kyiv-puppet-theater | 27 | 0.7781 | 291 s |\n| cyprus | 30 | 0.6239 | 312 s |\n| wall | 43 | 0.4753 | 851 s |\n| dioscuri | 174 | 0.8775 | 2045 s |\n\n## Processing time\n\nMy best submisison run time was **7 hours and 40 minutes**. This is well under the time limit of 9 hours. Therefore, I added LoFTR to ensemble on the last day, but the Kaggle server is broken and the notebook is still running. If the notebook had been successfully processed, My best submission could have been a little better.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6388%2F538925ab8b7ece79f2630f2a9f18708e%2Ferrors.png?generation=1686630579463016&alt=media)\n\nThe issue of not being able to submit on the last day is very stressful. Something similar occurred with Google Landmark Recognition in 2019. I pray that this kind of trouble will not occur again:pray:\n\n## Thing that didn't work\n\nToo many things to try to do alone :-)\n\n- TTA … no significant improvement\n- Pixel-Perfect SfM .. I got a slight improvement in my local environment but gave up because I couldn't set up ceres in my kaggle notebook\n- TensorRT .. I had to downgrade from PyTorch 2.0, but the setup was successful. I can't say with much certainty, but at least my experiments did not improve execution speed.\n- Half precision … It was about 10%(?) faster, but mAA was worse, so it was not adopted in the end.\n- other matchers … Tried DKMv3, LoFTR, Silk, but finally used only SPSG and MatchFormer due to high mAA.\n- OpenGLUE ... Trained DISK+OpenGLUE and SuperPoint+OpenGLUE on the MegaDepth dataset but did not reach the scores of the SPSG pretrained model. Converted phototourist dataset to OpenGLUE format and built MegaDepth + phototourist dataset, but did not validate it because of lack of time.\n- Incremental Mapper parameter tuning ... contributed to the stable high score in the local environment. Specifically, I reduced the max refinement change of BA and increased the max num iteration. However, all the submissions to kaggle were out of memory, so I did not check the improvement on the leaderboard.",
    "2300362": "@confirm Congrats for the solo gold, interesting approach.\nFor Pixel-Perfect we have ran it successfully on kaggle but \n- For reconstruction\n1- It will not pass for large scenes because of memory issues\n2- For scenes less than 40 images it caused a huge drop\n- For refinement only\n1- It caused a drop as well \nIf you would like to try it again here are the datasets\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/sfm-pre-pkgs\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/pyceres\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/pixsfm\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/pycolmap-040\nhttps://www.kaggle.com/datasets/jaafarmahmoud1/nvidia-libs",
    "2300333": "Congratulations @confirm! You have nerves of steel. As far as I remember yesterday you were still in the silver zone. It is important to withstand the stress and choose the right solution. Congrats on both your nerve control, your believe in the solution and your great submission choice. 👋👋👋💪👍",
    "2300288": "@confirm Addressing the rotation issue was an important step, nice approach! Great performance tricks! Thanks for clear write-up.\nCongratulations with a solo gold and green zone!",
    "2300285": "Thank you for this write-up & congrats on the result!\n\nHow much of an improvement did the rotations bring? We experimented with rotations as well, using a pretrained angle prediction model that we found online: https://github.com/pidahbus/deep-image-orientation-angle-detection We rotated the image and rotated the matched keypoints afterwards, but didn't see much of an improvement?",
    "2300275": "Hi @confirm.  It's a clear and brilliant solution. I love the rotation part. It is what I have been striving to implement throughout the last month, trying dozens of methods but ultimately failed to find a way to achieve. \n\nCongratulations on obtaining yet another solo gold medal. 🎉\n\n**************************************\nMay you make a clearer description for the rotation part? Did you rotate each image for 4 times? Since it's an pair, it make a batch size of 16. Did you first use the lightweight spsg to match on the batch16 for stage 1, then use a heavy spsg or spsg+matchformer for stage 2?\n",
    "2300528": "Do you have numbers for the other matchers and features?",
    "2300512": "Great write-up!",
    "2312701": "Congrats, the legend. ",
    "2303389": "Hi, congrats on solo gold and thanks you for write-up.\nCould you please explain more the knn part. Are you willing to share your code?\nthanks \n ",
    "2302322": "Congratulations @confirm!  Complete and great solution.\n\nMy question is, we were also working on colmap parameter tuning, but it didn't work because of the randomness of the CV scores. Please let me know if you have any ideas on how to optimize in such an unstable situation.",
    "2301341": "Brilliant idea for rotation correction! excellent job!",
    "2300507": "I am curious. what did you used matchformer for? Was it working better than SG?",
    "2300270": ""
  }
}