{
  "id": 103407,
  "title": "Score fluctuations ",
  "url": "/competitions/aptos2019-blindness-detection/discussion/103407",
  "author_name": "",
  "post_date": "2019-08-09T04:42:37.669093200Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I submitted the same inference kernel twice and got two very different scores. Any idea why this is happening and how to avoid it?</p>\n\n<p>Thanks.</p>",
  "messages": [
    {
      "id": "595298",
      "postDate": "08/09/2019 04:42:37",
      "content": "<p>I submitted the same inference kernel twice and got two very different scores. Any idea why this is happening and how to avoid it?</p>\n\n<p>Thanks.</p>",
      "rawMarkdown": "I submitted the same inference kernel twice and got two very different scores. Any idea why this is happening and how to avoid it?\n\nThanks.",
      "votes": null
    },
    {
      "id": "595321",
      "postDate": "08/09/2019 05:23:36",
      "content": "<p>If you don't fix random seeds and do TTA, the results may differ.</p>",
      "rawMarkdown": "If you don't fix random seeds and do TTA, the results may differ.",
      "votes": null
    },
    {
      "id": "595342",
      "postDate": "08/09/2019 05:49:17",
      "content": "<p>I did fix a random seed, and didn't use TTA. Still have a variance around 0.2.</p>",
      "rawMarkdown": "I did fix a random seed, and didn't use TTA. Still have a variance around 0.2.",
      "votes": null
    },
    {
      "id": "596781",
      "postDate": "08/11/2019 09:34:57",
      "content": "<p>Try this function (I found it in one of kernels)</p>\n\n<blockquote>\n  <p>def seed_everything(seed):\n      random.seed(seed)\n      os.environ['PYTHONHASHSEED'] = str(seed)\n      np.random.seed(seed)\n      torch.manual_seed(seed)\n      torch.cuda.manual_seed(seed)\n      torch.backends.cudnn.deterministic = True</p>\n</blockquote>",
      "rawMarkdown": "Try this function (I found it in one of kernels)\n\n&gt; def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 595321,
      "author_name": "harangdev",
      "author_url": "",
      "post_date": "08/09/2019 05:23:36",
      "content": "<p>If you don't fix random seeds and do TTA, the results may differ.</p>",
      "votes": null,
      "replies": [
        {
          "id": 595342,
          "author_name": "abyaadrafid",
          "author_url": "",
          "post_date": "08/09/2019 05:49:17",
          "content": "<p>I did fix a random seed, and didn't use TTA. Still have a variance around 0.2.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 596781,
      "author_name": "talalaev",
      "author_url": "",
      "post_date": "08/11/2019 09:34:57",
      "content": "<p>Try this function (I found it in one of kernels)</p>\n\n<blockquote>\n  <p>def seed_everything(seed):\n      random.seed(seed)\n      os.environ['PYTHONHASHSEED'] = str(seed)\n      np.random.seed(seed)\n      torch.manual_seed(seed)\n      torch.cuda.manual_seed(seed)\n      torch.backends.cudnn.deterministic = True</p>\n</blockquote>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "595298": "I submitted the same inference kernel twice and got two very different scores. Any idea why this is happening and how to avoid it?\n\nThanks.",
    "595321": "If you don't fix random seeds and do TTA, the results may differ.",
    "595342": "I did fix a random seed, and didn't use TTA. Still have a variance around 0.2.",
    "596781": "Try this function (I found it in one of kernels)\n\n&gt; def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True"
  },
  "source": "meta"
}