{
  "id": 344636,
  "title": "upsample_bilinear2d_backward_cuda does not have a deterministic implementation",
  "url": "/competitions/hubmap-organ-segmentation/discussion/344636",
  "author_name": "",
  "post_date": "2022-08-16T04:10:46.966407100Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>anyone have a fix for this ? <br>\nmodel is non reproducible without this …</p>",
  "messages": [
    {
      "id": "1900469",
      "postDate": "08/16/2022 04:10:46",
      "content": "<p>anyone have a fix for this ? <br>\nmodel is non reproducible without this …</p>",
      "rawMarkdown": "anyone have a fix for this ? \nmodel is non reproducible without this ...",
      "votes": null
    },
    {
      "id": "1908026",
      "postDate": "08/21/2022 09:44:46",
      "content": "<p>It might be related to deterministic cuDNN.</p>\n<p>Are you making those calls while setting seed?</p>\n<pre><code>torch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n</code></pre>",
      "rawMarkdown": "It might be related to deterministic cuDNN.\n\nAre you making those calls while setting seed?\n\n```\ntorch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n```",
      "votes": null
    },
    {
      "id": "1908596",
      "postDate": "08/21/2022 20:19:42",
      "content": "<p>Yes I have those . <br>\nIt is still undeterministic</p>",
      "rawMarkdown": "Yes I have those . \nIt is still undeterministic",
      "votes": null
    },
    {
      "id": "1908819",
      "postDate": "08/22/2022 04:18:23",
      "content": "<p>Yes, but torch might search for a deterministic upsample implementation in the backend and fail. Try commenting out those lines.</p>",
      "rawMarkdown": "Yes, but torch might search for a deterministic upsample implementation in the backend and fail. Try commenting out those lines.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1908026,
      "author_name": "gunesevitan",
      "author_url": "",
      "post_date": "08/21/2022 09:44:46",
      "content": "<p>It might be related to deterministic cuDNN.</p>\n<p>Are you making those calls while setting seed?</p>\n<pre><code>torch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n</code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 1908596,
          "author_name": "rarun2596",
          "author_url": "",
          "post_date": "08/21/2022 20:19:42",
          "content": "<p>Yes I have those . <br>\nIt is still undeterministic</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1908819,
          "author_name": "gunesevitan",
          "author_url": "",
          "post_date": "08/22/2022 04:18:23",
          "content": "<p>Yes, but torch might search for a deterministic upsample implementation in the backend and fail. Try commenting out those lines.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1900469": "anyone have a fix for this ? \nmodel is non reproducible without this ...",
    "1908026": "It might be related to deterministic cuDNN.\n\nAre you making those calls while setting seed?\n\n```\ntorch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n```",
    "1908596": "Yes I have those . \nIt is still undeterministic",
    "1908819": "Yes, but torch might search for a deterministic upsample implementation in the backend and fail. Try commenting out those lines."
  },
  "source": "meta"
}