{
  "id": 497090,
  "title": "fail to reproduce the result",
  "url": "/competitions/image-matching-challenge-2024/discussion/497090",
  "author_name": "",
  "post_date": "2024-04-23T15:10:52.749231600Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I added following code to the published baseline, but I could not reproduce the result in my local env.<br>\nCould someone give me an advice ? </p>\n<pre><code>\n\ncolmap_mapper_options = {\n        : , \n        : ,\n        :\n    }\n\npycolmap.match_exhaustive(database_path, sift_options={:})\nmapper_options = pycolmap.IncrementalPipelineOptions(**config.colmap_mapper_options)\n</code></pre>",
  "messages": [
    {
      "id": "2769825",
      "postDate": "04/23/2024 15:10:52",
      "content": "<p>I added following code to the published baseline, but I could not reproduce the result in my local env.<br>\nCould someone give me an advice ? </p>\n<pre><code>\n\ncolmap_mapper_options = {\n        : , \n        : ,\n        :\n    }\n\npycolmap.match_exhaustive(database_path, sift_options={:})\nmapper_options = pycolmap.IncrementalPipelineOptions(**config.colmap_mapper_options)\n</code></pre>",
      "rawMarkdown": "I added following code to the published baseline, but I could not reproduce the result in my local env.\nCould someone give me an advice ? \n\n```python\n# pycolmap.__version__ == '0.6.1'\n\ncolmap_mapper_options = {\n        \"min_model_size\": 3, \n        \"max_num_models\": 2,\n        \"num_threads\":1\n    }\n\npycolmap.match_exhaustive(database_path, sift_options={'num_threads':1})\nmapper_options = pycolmap.IncrementalPipelineOptions(**config.colmap_mapper_options)\n\n```",
      "votes": null
    },
    {
      "id": "2770514",
      "postDate": "04/23/2024 21:53:59",
      "content": "<p>Have you tried to reproduce the result in Kaggle's notebooks? It works for me, I get consistent results.</p>\n<p>I'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.</p>",
      "rawMarkdown": "Have you tried to reproduce the result in Kaggle's notebooks? It works for me, I get consistent results.\n\nI'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.",
      "votes": null
    },
    {
      "id": "2773916",
      "postDate": "04/25/2024 00:56:04",
      "content": "<p>Thanks for comment.</p>\n<blockquote>\n  <p>Have you tried to reproduce the result in Kaggle's notebooks?</p>\n</blockquote>\n<p>No, I have not. I use only local kaggle-docker env.<br>\nDo you always reproduce the validation result in kaggle notebook ? </p>\n<blockquote>\n  <p>I'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.</p>\n</blockquote>\n<p>I don't use fix_seed function. Should I use ?</p>",
      "rawMarkdown": "Thanks for comment.\n\n>Have you tried to reproduce the result in Kaggle's notebooks?\n\nNo, I have not. I use only local kaggle-docker env.\nDo you always reproduce the validation result in kaggle notebook ? \n\n>I'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.\n\nI don't use fix_seed function. Should I use ?",
      "votes": null
    },
    {
      "id": "2774254",
      "postDate": "04/25/2024 05:30:00",
      "content": "<ul>\n<li><p>Do you always reproduce the validation result in kaggle notebook ?<br>\nYes, to put my mind at ease, I just did! If you are using the <a href=\"https://www.kaggle.com/code/nartaa/imc2024-starter\" target=\"_blank\">IMC2024 Starter</a>, and set the <code>N_SAMPLES = 15</code>, you should get exactly <strong>0.0556</strong> CV score for the church dataset. </p></li>\n<li><p>I don't use fix_seed function. Should I use ?<br>\nit should be explored, I think the issue is probably with randomly picking different images each time you run the cross validation, but if you are using the IMC2024 Starter, the random seed for sampling images is already set:<br>\n    g = g[0],g[1].sample(n,<strong>random_state=42</strong>).reset_index(drop=True)</p>\n<p>OR somehow multithreading is enforced in your machine.</p></li>\n</ul>",
      "rawMarkdown": "Do you always reproduce the validation result in kaggle notebook ?\nYes, to put my mind at ease, I just did! If you are using the [IMC2024 Starter](https://www.kaggle.com/code/nartaa/imc2024-starter), and set the `N_SAMPLES = 15`, you should get exactly **0.0556** CV score for the church dataset. \n\n\n- I don't use fix_seed function. Should I use ?\nit should be explored, I think the issue is probably with randomly picking different images each time you run the cross validation, but if you are using the IMC2024 Starter, the random seed for sampling images is already set:\n        g = g[0],g[1].sample(n,**random_state=42**).reset_index(drop=True)\n\n OR somehow multithreading is enforced in your machine.",
      "votes": null
    },
    {
      "id": "2785205",
      "postDate": "04/30/2024 17:02:29",
      "content": "<p>I've reproduced result with kaggle notebook. <br>\nI ran same code in my local and kaggle notebook, but I could not reproduce the result in my local.<br>\nThanks.</p>",
      "rawMarkdown": "I've reproduced result with kaggle notebook. \nI ran same code in my local and kaggle notebook, but I could not reproduce the result in my local.\nThanks.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2770514,
      "author_name": "nartaa",
      "author_url": "",
      "post_date": "04/23/2024 21:53:59",
      "content": "<p>Have you tried to reproduce the result in Kaggle's notebooks? It works for me, I get consistent results.</p>\n<p>I'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2773916,
          "author_name": "clearwaterkzk",
          "author_url": "",
          "post_date": "04/25/2024 00:56:04",
          "content": "<p>Thanks for comment.</p>\n<blockquote>\n  <p>Have you tried to reproduce the result in Kaggle's notebooks?</p>\n</blockquote>\n<p>No, I have not. I use only local kaggle-docker env.<br>\nDo you always reproduce the validation result in kaggle notebook ? </p>\n<blockquote>\n  <p>I'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.</p>\n</blockquote>\n<p>I don't use fix_seed function. Should I use ?</p>",
          "votes": null,
          "replies": [
            {
              "id": 2774254,
              "author_name": "nartaa",
              "author_url": "",
              "post_date": "04/25/2024 05:30:00",
              "content": "<ul>\n<li><p>Do you always reproduce the validation result in kaggle notebook ?<br>\nYes, to put my mind at ease, I just did! If you are using the <a href=\"https://www.kaggle.com/code/nartaa/imc2024-starter\" target=\"_blank\">IMC2024 Starter</a>, and set the <code>N_SAMPLES = 15</code>, you should get exactly <strong>0.0556</strong> CV score for the church dataset. </p></li>\n<li><p>I don't use fix_seed function. Should I use ?<br>\nit should be explored, I think the issue is probably with randomly picking different images each time you run the cross validation, but if you are using the IMC2024 Starter, the random seed for sampling images is already set:<br>\n    g = g[0],g[1].sample(n,<strong>random_state=42</strong>).reset_index(drop=True)</p>\n<p>OR somehow multithreading is enforced in your machine.</p></li>\n</ul>",
              "votes": null,
              "replies": [
                {
                  "id": 2785205,
                  "author_name": "clearwaterkzk",
                  "author_url": "",
                  "post_date": "04/30/2024 17:02:29",
                  "content": "<p>I've reproduced result with kaggle notebook. <br>\nI ran same code in my local and kaggle notebook, but I could not reproduce the result in my local.<br>\nThanks.</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2769825": "I added following code to the published baseline, but I could not reproduce the result in my local env.\nCould someone give me an advice ? \n\n```python\n# pycolmap.__version__ == '0.6.1'\n\ncolmap_mapper_options = {\n        \"min_model_size\": 3, \n        \"max_num_models\": 2,\n        \"num_threads\":1\n    }\n\npycolmap.match_exhaustive(database_path, sift_options={'num_threads':1})\nmapper_options = pycolmap.IncrementalPipelineOptions(**config.colmap_mapper_options)\n\n```",
    "2770514": "Have you tried to reproduce the result in Kaggle's notebooks? It works for me, I get consistent results.\n\nI'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.",
    "2773916": "Thanks for comment.\n\n>Have you tried to reproduce the result in Kaggle's notebooks?\n\nNo, I have not. I use only local kaggle-docker env.\nDo you always reproduce the validation result in kaggle notebook ? \n\n>I'm assuming you've tried setting the random seeds for Python, Numpy,Torch and it didn't work.\n\nI don't use fix_seed function. Should I use ?",
    "2774254": "Do you always reproduce the validation result in kaggle notebook ?\nYes, to put my mind at ease, I just did! If you are using the [IMC2024 Starter](https://www.kaggle.com/code/nartaa/imc2024-starter), and set the `N_SAMPLES = 15`, you should get exactly **0.0556** CV score for the church dataset. \n\n\n- I don't use fix_seed function. Should I use ?\nit should be explored, I think the issue is probably with randomly picking different images each time you run the cross validation, but if you are using the IMC2024 Starter, the random seed for sampling images is already set:\n        g = g[0],g[1].sample(n,**random_state=42**).reset_index(drop=True)\n\n OR somehow multithreading is enforced in your machine.",
    "2785205": "I've reproduced result with kaggle notebook. \nI ran same code in my local and kaggle notebook, but I could not reproduce the result in my local.\nThanks."
  },
  "source": "meta"
}