{
  "id": 617806,
  "title": "The Impact of Randomness in Deep Learning",
  "url": "/competitions/recodai-luc-scientific-image-forgery-detection/discussion/617806",
  "author_name": "",
  "post_date": "2025-11-12T06:42:50.625598800Z",
  "votes": 3,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Two identical notebooks yield different training results with improved metrics, which may be due to the impact of random numbers.</p>\n<ul>\n<li><a href=\"url\" target=\"_blank\">https://www.kaggle.com/code/djamilabenchikh/cnn-dinov2-hybrid</a>(3.10)&nbsp;</li>\n<li><a href=\"url\" target=\"_blank\">https://www.kaggle.com/code/maheenriaz1122/cnn-dinov2-hybrid-c0bb37</a>(3.15)</li>\n</ul>\n<p>The core reason lies in random factors—even with identical code, multiple random links during training will lead to discrepancies in results.\nKey random points include:</p>\n<ul>\n<li><strong>Data level</strong>: Training/validation set splitting, random transformations in data augmentation (e.g., random cropping, flipping).</li>\n<li><strong>Model level</strong>: Weight initialization (default random assignment), random neuron deactivation in Dropout, batch statistics fluctuations in BatchNorm.</li>\n<li><strong>Training level</strong>: Random gradient noise of the optimizer, precision differences in GPU parallel computing.</li>\n</ul>\n<p><strong>Some possible solutions</strong></p>\n<pre><code> random\nrandom.seed()  \n\n\n numpy  np\nnp.random.seed()\n\n\n torch\ntorch.manual_seed()\ntorch.cuda.manual_seed()\ntorch.cuda.manual_seed_all()  \ntorch.backends.cudnn.deterministic =   \ntorch.backends.cudnn.benchmark =   \n\n\n tensorflow  tf\ntf.random.set_seed()\n</code></pre>\n<p>Or directly load the pre-trained weight files saved by others—for example, the aforementioned DINOV2—by loading the .pt files.</p>\n<pre><code>model_weights_path =  \nmodel_seg.load_state_dict(torch.load(model_weights_path, map_location=device))\nmodel_seg.()  \n</code></pre>",
  "messages": [
    {
      "id": "3319987",
      "postDate": "11/12/2025 06:42:50",
      "content": "<p>Two identical notebooks yield different training results with improved metrics, which may be due to the impact of random numbers.</p>\n<ul>\n<li><a href=\"url\" target=\"_blank\">https://www.kaggle.com/code/djamilabenchikh/cnn-dinov2-hybrid</a>(3.10)&nbsp;</li>\n<li><a href=\"url\" target=\"_blank\">https://www.kaggle.com/code/maheenriaz1122/cnn-dinov2-hybrid-c0bb37</a>(3.15)</li>\n</ul>\n<p>The core reason lies in random factors—even with identical code, multiple random links during training will lead to discrepancies in results.\nKey random points include:</p>\n<ul>\n<li><strong>Data level</strong>: Training/validation set splitting, random transformations in data augmentation (e.g., random cropping, flipping).</li>\n<li><strong>Model level</strong>: Weight initialization (default random assignment), random neuron deactivation in Dropout, batch statistics fluctuations in BatchNorm.</li>\n<li><strong>Training level</strong>: Random gradient noise of the optimizer, precision differences in GPU parallel computing.</li>\n</ul>\n<p><strong>Some possible solutions</strong></p>\n<pre><code> random\nrandom.seed()  \n\n\n numpy  np\nnp.random.seed()\n\n\n torch\ntorch.manual_seed()\ntorch.cuda.manual_seed()\ntorch.cuda.manual_seed_all()  \ntorch.backends.cudnn.deterministic =   \ntorch.backends.cudnn.benchmark =   \n\n\n tensorflow  tf\ntf.random.set_seed()\n</code></pre>\n<p>Or directly load the pre-trained weight files saved by others—for example, the aforementioned DINOV2—by loading the .pt files.</p>\n<pre><code>model_weights_path =  \nmodel_seg.load_state_dict(torch.load(model_weights_path, map_location=device))\nmodel_seg.()  \n</code></pre>",
      "rawMarkdown": "Two identical notebooks yield different training results with improved metrics, which may be due to the impact of random numbers.\n- [https://www.kaggle.com/code/djamilabenchikh/cnn-dinov2-hybrid](url)(3.10) \n- [https://www.kaggle.com/code/maheenriaz1122/cnn-dinov2-hybrid-c0bb37](url)(3.15)\n\nThe core reason lies in random factors—even with identical code, multiple random links during training will lead to discrepancies in results.\nKey random points include:\n- **Data level**: Training/validation set splitting, random transformations in data augmentation (e.g., random cropping, flipping).\n- **Model level**: Weight initialization (default random assignment), random neuron deactivation in Dropout, batch statistics fluctuations in BatchNorm.\n- **Training level**: Random gradient noise of the optimizer, precision differences in GPU parallel computing.\n\n**Some possible solutions**\n```python\nimport random\nrandom.seed(42)  \n\n# NumPy\nimport numpy as np\nnp.random.seed(42)\n\n# PyTorch（GPU+CPU）\nimport torch\ntorch.manual_seed(42)\ntorch.cuda.manual_seed(42)\ntorch.cuda.manual_seed_all(42)  \ntorch.backends.cudnn.deterministic = True  \ntorch.backends.cudnn.benchmark = False  \n\n# TensorFlow\nimport tensorflow as tf\ntf.random.set_seed(42)\n```\n\nOr directly load the pre-trained weight files saved by others—for example, the aforementioned DINOV2—by loading the .pt files.\n```python\nmodel_weights_path = \"/kaggle/input/cnn-dinov2-hybrid/model_seg_final.pt\" \nmodel_seg.load_state_dict(torch.load(model_weights_path, map_location=device))\nmodel_seg.eval()  # evaluate mode\n```",
      "votes": null
    },
    {
      "id": "3328757",
      "postDate": "11/16/2025 01:59:11",
      "content": "<p>hey please tell me how your score has increase </p>",
      "rawMarkdown": "hey please tell me how your score has increase",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3328757,
      "author_name": "maheenriaz1122",
      "author_url": "",
      "post_date": "11/16/2025 01:59:11",
      "content": "<p>hey please tell me how your score has increase </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3319987": "Two identical notebooks yield different training results with improved metrics, which may be due to the impact of random numbers.\n- [https://www.kaggle.com/code/djamilabenchikh/cnn-dinov2-hybrid](url)(3.10) \n- [https://www.kaggle.com/code/maheenriaz1122/cnn-dinov2-hybrid-c0bb37](url)(3.15)\n\nThe core reason lies in random factors—even with identical code, multiple random links during training will lead to discrepancies in results.\nKey random points include:\n- **Data level**: Training/validation set splitting, random transformations in data augmentation (e.g., random cropping, flipping).\n- **Model level**: Weight initialization (default random assignment), random neuron deactivation in Dropout, batch statistics fluctuations in BatchNorm.\n- **Training level**: Random gradient noise of the optimizer, precision differences in GPU parallel computing.\n\n**Some possible solutions**\n```python\nimport random\nrandom.seed(42)  \n\n# NumPy\nimport numpy as np\nnp.random.seed(42)\n\n# PyTorch（GPU+CPU）\nimport torch\ntorch.manual_seed(42)\ntorch.cuda.manual_seed(42)\ntorch.cuda.manual_seed_all(42)  \ntorch.backends.cudnn.deterministic = True  \ntorch.backends.cudnn.benchmark = False  \n\n# TensorFlow\nimport tensorflow as tf\ntf.random.set_seed(42)\n```\n\nOr directly load the pre-trained weight files saved by others—for example, the aforementioned DINOV2—by loading the .pt files.\n```python\nmodel_weights_path = \"/kaggle/input/cnn-dinov2-hybrid/model_seg_final.pt\" \nmodel_seg.load_state_dict(torch.load(model_weights_path, map_location=device))\nmodel_seg.eval()  # evaluate mode\n```",
    "3328757": "hey please tell me how your score has increase"
  },
  "source": "meta"
}