{
  "id": 494921,
  "title": "Reproducibility of CV and LB scores",
  "url": "/competitions/image-matching-challenge-2024/discussion/494921",
  "author_name": "",
  "post_date": "2024-04-18T23:14:03.660783300Z",
  "votes": 11,
  "comment_count": 4,
  "views": 0,
  "content": "<p>It seems the randomness seed is fixed within COLMAP. By adjusting the number of threads to 1, we effectively eliminate this randomness. Consequently, submission consistently delivers identical results across multiple runs, allowing a more accurate evaluation of the framework's performance.</p>\n<p>Check out the <a href=\"https://www.kaggle.com/code/nartaa/imc2024-starter\" target=\"_blank\">IMC24 Starter</a> to find out how to set thread numbers to 1 in COLMAP.</p>\n<p>Reference:<br>\n<a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2023/discussion/417407\" target=\"_blank\">https://www.kaggle.com/competitions/image-matching-challenge-2023/discussion/417407</a></p>",
  "messages": [
    {
      "id": "2759827",
      "postDate": "04/18/2024 23:14:03",
      "content": "<p>It seems the randomness seed is fixed within COLMAP. By adjusting the number of threads to 1, we effectively eliminate this randomness. Consequently, submission consistently delivers identical results across multiple runs, allowing a more accurate evaluation of the framework's performance.</p>\n<p>Check out the <a href=\"https://www.kaggle.com/code/nartaa/imc2024-starter\" target=\"_blank\">IMC24 Starter</a> to find out how to set thread numbers to 1 in COLMAP.</p>\n<p>Reference:<br>\n<a href=\"https://www.kaggle.com/competitions/image-matching-challenge-2023/discussion/417407\" target=\"_blank\">https://www.kaggle.com/competitions/image-matching-challenge-2023/discussion/417407</a></p>",
      "rawMarkdown": "It seems the randomness seed is fixed within COLMAP. By adjusting the number of threads to 1, we effectively eliminate this randomness. Consequently, submission consistently delivers identical results across multiple runs, allowing a more accurate evaluation of the framework's performance.\n\nCheck out the [IMC24 Starter](https://www.kaggle.com/code/nartaa/imc2024-starter) to find out how to set thread numbers to 1 in COLMAP.\n\nReference:\nhttps://www.kaggle.com/competitions/image-matching-challenge-2023/discussion/417407",
      "votes": null
    },
    {
      "id": "2759904",
      "postDate": "04/19/2024 02:03:15",
      "content": "<p>Thanks for sharing.</p>\n<blockquote>\n  <p>Check out the IMC24 Starter to find out how to set thread numbers to 1 in COLMAP.</p>\n</blockquote>\n<p>Do you mean <code>sift_options</code>in this code ? <br>\n<code>pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})</code></p>",
      "rawMarkdown": "Thanks for sharing.\n>Check out the IMC24 Starter to find out how to set thread numbers to 1 in COLMAP.\n\nDo you mean `sift_options `in this code ? \n` pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})`",
      "votes": null
    },
    {
      "id": "2760231",
      "postDate": "04/19/2024 07:00:15",
      "content": "<p>You are welcome!</p>\n<p>Yes, number of threads should be set in two places. </p>\n<ul>\n<li>pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})</li>\n<li>pycolmap.IncrementalPipelineOptions({'min_model_size':MIN_MODEL_SIZE, 'max_num_models':MAX_NUM_MODELS, 'num_threads':1})</li>\n</ul>",
      "rawMarkdown": "You are welcome!\n\nYes, number of threads should be set in two places. \n\n- pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})\n- pycolmap.IncrementalPipelineOptions({'min_model_size':MIN_MODEL_SIZE, 'max_num_models':MAX_NUM_MODELS, 'num_threads':1})",
      "votes": null
    },
    {
      "id": "2767797",
      "postDate": "04/22/2024 14:24:38",
      "content": "<p>Thank you for replying, but did you completely reproduce the same result ?<br>\nHonestly, I couldn't…</p>",
      "rawMarkdown": "Thank you for replying, but did you completely reproduce the same result ?\nHonestly, I couldn't...",
      "votes": null
    },
    {
      "id": "2767936",
      "postDate": "04/22/2024 15:26:06",
      "content": "<p>Yes, I did!</p>",
      "rawMarkdown": "Yes, I did!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2759904,
      "author_name": "clearwaterkzk",
      "author_url": "",
      "post_date": "04/19/2024 02:03:15",
      "content": "<p>Thanks for sharing.</p>\n<blockquote>\n  <p>Check out the IMC24 Starter to find out how to set thread numbers to 1 in COLMAP.</p>\n</blockquote>\n<p>Do you mean <code>sift_options</code>in this code ? <br>\n<code>pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})</code></p>",
      "votes": null,
      "replies": [
        {
          "id": 2760231,
          "author_name": "nartaa",
          "author_url": "",
          "post_date": "04/19/2024 07:00:15",
          "content": "<p>You are welcome!</p>\n<p>Yes, number of threads should be set in two places. </p>\n<ul>\n<li>pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})</li>\n<li>pycolmap.IncrementalPipelineOptions({'min_model_size':MIN_MODEL_SIZE, 'max_num_models':MAX_NUM_MODELS, 'num_threads':1})</li>\n</ul>",
          "votes": null,
          "replies": [
            {
              "id": 2767797,
              "author_name": "clearwaterkzk",
              "author_url": "",
              "post_date": "04/22/2024 14:24:38",
              "content": "<p>Thank you for replying, but did you completely reproduce the same result ?<br>\nHonestly, I couldn't…</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2767936,
                  "author_name": "nartaa",
                  "author_url": "",
                  "post_date": "04/22/2024 15:26:06",
                  "content": "<p>Yes, I did!</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2759827": "It seems the randomness seed is fixed within COLMAP. By adjusting the number of threads to 1, we effectively eliminate this randomness. Consequently, submission consistently delivers identical results across multiple runs, allowing a more accurate evaluation of the framework's performance.\n\nCheck out the [IMC24 Starter](https://www.kaggle.com/code/nartaa/imc2024-starter) to find out how to set thread numbers to 1 in COLMAP.\n\nReference:\nhttps://www.kaggle.com/competitions/image-matching-challenge-2023/discussion/417407",
    "2759904": "Thanks for sharing.\n>Check out the IMC24 Starter to find out how to set thread numbers to 1 in COLMAP.\n\nDo you mean `sift_options `in this code ? \n` pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})`",
    "2760231": "You are welcome!\n\nYes, number of threads should be set in two places. \n\n- pycolmap.match_exhaustive(db_name, sift_options={'num_threads':1})\n- pycolmap.IncrementalPipelineOptions({'min_model_size':MIN_MODEL_SIZE, 'max_num_models':MAX_NUM_MODELS, 'num_threads':1})",
    "2767797": "Thank you for replying, but did you completely reproduce the same result ?\nHonestly, I couldn't...",
    "2767936": "Yes, I did!"
  },
  "source": "meta"
}