I'm new to the field of image matching and especially SFM. Seems like COLMAP is the tool that helps to solve the task. I'm trying to run LoFTR + Colmap pipeline as in example notebook from hosts, but i'm getting different results every rerun. The difference in local score can reach 10-20%. I suppose there is some kind of randomness when running colmap functions (match_exhaustive or incremental_mapping or maybe some another). Moreover I get different number of cameras registered in reconstruction for same scene. It will be useful to understand how exactly colmap works of course, but at this point I'm wondering why is there such a difference every rerun?
", "messages": [ { "id": "2231932", "postDate": "04/23/2023 20:42:01", "content": "I'm new to the field of image matching and especially SFM. Seems like COLMAP is the tool that helps to solve the task. I'm trying to run LoFTR + Colmap pipeline as in example notebook from hosts, but i'm getting different results every rerun. The difference in local score can reach 10-20%. I suppose there is some kind of randomness when running colmap functions (match_exhaustive or incremental_mapping or maybe some another). Moreover I get different number of cameras registered in reconstruction for same scene. It will be useful to understand how exactly colmap works of course, but at this point I'm wondering why is there such a difference every rerun?
", "rawMarkdown": "I'm new to the field of image matching and especially SFM. Seems like COLMAP is the tool that helps to solve the task. I'm trying to run LoFTR + Colmap pipeline as in example notebook from hosts, but i'm getting different results every rerun. The difference in local score can reach 10-20%. I suppose there is some kind of randomness when running colmap functions (match_exhaustive or incremental_mapping or maybe some another). Moreover I get different number of cameras registered in reconstruction for same scene. It will be useful to understand how exactly colmap works of course, but at this point I'm wondering why is there such a difference every rerun?", "votes": null }, { "id": "2232012", "postDate": "04/23/2023 22:35:00", "content": "The matcher uses RANSAC, which is a non-deterministic algorithm. In any case, the solution is \"stable\" if found matches are robust and consistent.
\nSee also here: https://github.com/colmap/colmap/issues/757
", "rawMarkdown": "The matcher uses RANSAC, which is a non-deterministic algorithm. In any case, the solution is \"stable\" if found matches are robust and consistent.\n\nSee also here: https://github.com/colmap/colmap/issues/757", "votes": null }, { "id": "2232352", "postDate": "04/24/2023 08:35:35", "content": "Incremental_mapping also uses RANSAC for PnP algorithm, and the optimizer (colmap uses ceres solver internally) also has randomness . I think some degree of randomness is inevitable as long as pycolmap is used.
", "rawMarkdown": "Incremental_mapping also uses RANSAC for PnP algorithm, and the optimizer (colmap uses ceres solver internally) also has randomness . I think some degree of randomness is inevitable as long as pycolmap is used.", "votes": null }, { "id": "2232356", "postDate": "04/24/2023 08:37:55", "content": "Got it. I'm just being sad that my local score can vary between 0.35-0.55 for the same scene) That kind of randomness scares me.
", "rawMarkdown": "Got it. I'm just being sad that my local score can vary between 0.35-0.55 for the same scene) That kind of randomness scares me.", "votes": null }, { "id": "2232359", "postDate": "04/24/2023 08:42:53", "content": "Yeah, but if I remember well (actually I've never tested it), you can set the random seed for incremental mapper. But as I've said, if your matches are robust, the found solution will be stable even if not the same.
", "rawMarkdown": "Yeah, but if I remember well (actually I've never tested it), you can set the random seed for incremental mapper. But as I've said, if your matches are robust, the found solution will be stable even if not the same.", "votes": null }, { "id": "2232361", "postDate": "04/24/2023 08:46:35", "content": "Thank you!
", "rawMarkdown": "Thank you!", "votes": null }, { "id": "2232363", "postDate": "04/24/2023 08:47:25", "content": "A local score in this range means that the matches are far to be correctly estimated. Moreover, nobody is forced to use COLMAP for providing the solution :)
", "rawMarkdown": "A local score in this range means that the matches are far to be correctly estimated. Moreover, nobody is forced to use COLMAP for providing the solution :)", "votes": null }, { "id": "2251956", "postDate": "05/09/2023 19:10:29", "content": "I'm not sure whether the metric is too unstable or the matching part is too random. I can get mAA between 0 and 0.5 for the bike scene using SIFT and incremental mapping.
", "rawMarkdown": "I'm not sure whether the metric is too unstable or the matching part is too random. I can get mAA between 0 and 0.5 for the bike scene using SIFT and incremental mapping.", "votes": null }, { "id": "2253418", "postDate": "05/10/2023 07:00:00", "content": "It's a small scene, so the metrics will vary more than usual if you fail to register images. You can play with some colmap settings to make it more stable.
", "rawMarkdown": "It's a small scene, so the metrics will vary more than usual if you fail to register images. You can play with some colmap settings to make it more stable.", "votes": null }, { "id": "2261278", "postDate": "05/16/2023 08:11:51", "content": "Hi @vostankovich, did you figure out where randomness comes from?
", "rawMarkdown": "Hi @vostankovich, did you figure out where randomness comes from?", "votes": null }, { "id": "2261476", "postDate": "05/16/2023 11:00:54", "content": "As hosts said above its due to RANSAC =)
", "rawMarkdown": "As hosts said above its due to RANSAC =)", "votes": null }, { "id": "2265104", "postDate": "05/19/2023 01:30:48", "content": "Hi, did you find the way to solve this problem?
", "rawMarkdown": "Hi, did you find the way to solve this problem?", "votes": null }, { "id": "2272176", "postDate": "05/24/2023 10:38:45", "content": "Did you find some other global SFM for this competition？
", "rawMarkdown": "Did you find some other global SFM for this competition？", "votes": null } ], "comments": [ { "id": 2232012, "author_name": "fabiobellavia", "author_url": "", "post_date": "04/23/2023 22:35:00", "content": "The matcher uses RANSAC, which is a non-deterministic algorithm. In any case, the solution is \"stable\" if found matches are robust and consistent.
\nSee also here: https://github.com/colmap/colmap/issues/757
", "votes": null, "replies": [] }, { "id": 2232352, "author_name": "jinhwanlazy", "author_url": "", "post_date": "04/24/2023 08:35:35", "content": "Incremental_mapping also uses RANSAC for PnP algorithm, and the optimizer (colmap uses ceres solver internally) also has randomness . I think some degree of randomness is inevitable as long as pycolmap is used.
", "votes": null, "replies": [ { "id": 2232356, "author_name": "vostankovich", "author_url": "", "post_date": "04/24/2023 08:37:55", "content": "Got it. I'm just being sad that my local score can vary between 0.35-0.55 for the same scene) That kind of randomness scares me.
", "votes": null, "replies": [ { "id": 2232363, "author_name": "fabiobellavia", "author_url": "", "post_date": "04/24/2023 08:47:25", "content": "A local score in this range means that the matches are far to be correctly estimated. Moreover, nobody is forced to use COLMAP for providing the solution :)
", "votes": null, "replies": [] } ] }, { "id": 2232359, "author_name": "fabiobellavia", "author_url": "", "post_date": "04/24/2023 08:42:53", "content": "Yeah, but if I remember well (actually I've never tested it), you can set the random seed for incremental mapper. But as I've said, if your matches are robust, the found solution will be stable even if not the same.
", "votes": null, "replies": [ { "id": 2232361, "author_name": "vostankovich", "author_url": "", "post_date": "04/24/2023 08:46:35", "content": "Thank you!
", "votes": null, "replies": [] } ] } ] }, { "id": 2251956, "author_name": "gunesevitan", "author_url": "", "post_date": "05/09/2023 19:10:29", "content": "I'm not sure whether the metric is too unstable or the matching part is too random. I can get mAA between 0 and 0.5 for the bike scene using SIFT and incremental mapping.
", "votes": null, "replies": [ { "id": 2253418, "author_name": "eduardtrulls", "author_url": "", "post_date": "05/10/2023 07:00:00", "content": "It's a small scene, so the metrics will vary more than usual if you fail to register images. You can play with some colmap settings to make it more stable.
", "votes": null, "replies": [] } ] }, { "id": 2261278, "author_name": "huyduong7101", "author_url": "", "post_date": "05/16/2023 08:11:51", "content": "Hi @vostankovich, did you figure out where randomness comes from?
", "votes": null, "replies": [ { "id": 2261476, "author_name": "vostankovich", "author_url": "", "post_date": "05/16/2023 11:00:54", "content": "As hosts said above its due to RANSAC =)
", "votes": null, "replies": [ { "id": 2265104, "author_name": "leon567", "author_url": "", "post_date": "05/19/2023 01:30:48", "content": "Hi, did you find the way to solve this problem?
", "votes": null, "replies": [] } ] } ] }, { "id": 2272176, "author_name": "pamilovedl", "author_url": "", "post_date": "05/24/2023 10:38:45", "content": "Did you find some other global SFM for this competition？
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2231932": "I'm new to the field of image matching and especially SFM. Seems like COLMAP is the tool that helps to solve the task. I'm trying to run LoFTR + Colmap pipeline as in example notebook from hosts, but i'm getting different results every rerun. The difference in local score can reach 10-20%. I suppose there is some kind of randomness when running colmap functions (match_exhaustive or incremental_mapping or maybe some another). Moreover I get different number of cameras registered in reconstruction for same scene. It will be useful to understand how exactly colmap works of course, but at this point I'm wondering why is there such a difference every rerun?", "2232012": "The matcher uses RANSAC, which is a non-deterministic algorithm. In any case, the solution is \"stable\" if found matches are robust and consistent.\n\nSee also here: https://github.com/colmap/colmap/issues/757", "2232352": "Incremental_mapping also uses RANSAC for PnP algorithm, and the optimizer (colmap uses ceres solver internally) also has randomness . I think some degree of randomness is inevitable as long as pycolmap is used.", "2232356": "Got it. I'm just being sad that my local score can vary between 0.35-0.55 for the same scene) That kind of randomness scares me.", "2232359": "Yeah, but if I remember well (actually I've never tested it), you can set the random seed for incremental mapper. But as I've said, if your matches are robust, the found solution will be stable even if not the same.", "2232361": "Thank you!", "2232363": "A local score in this range means that the matches are far to be correctly estimated. Moreover, nobody is forced to use COLMAP for providing the solution :)", "2251956": "I'm not sure whether the metric is too unstable or the matching part is too random. I can get mAA between 0 and 0.5 for the bike scene using SIFT and incremental mapping.", "2253418": "It's a small scene, so the metrics will vary more than usual if you fail to register images. You can play with some colmap settings to make it more stable.", "2261278": "Hi @vostankovich, did you figure out where randomness comes from?", "2261476": "As hosts said above its due to RANSAC =)", "2265104": "Hi, did you find the way to solve this problem?", "2272176": "Did you find some other global SFM for this competition？" }, "source": "meta" }