{
  "id": 233561,
  "title": "Trouble training with pytorch",
  "url": "/competitions/bms-molecular-translation/discussion/233561",
  "author_name": "",
  "post_date": "2021-04-19T23:10:13.489037100Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>When I try to train my model on my local gpu it gets deadlocked when enumerating through the trainloader. Anyone know what the problem is as torch does recognize my gpu and i have both cuda and cudnn installed</p>",
  "messages": [
    {
      "id": "1278462",
      "postDate": "04/19/2021 23:10:13",
      "content": "<p>When I try to train my model on my local gpu it gets deadlocked when enumerating through the trainloader. Anyone know what the problem is as torch does recognize my gpu and i have both cuda and cudnn installed</p>",
      "rawMarkdown": "When I try to train my model on my local gpu it gets deadlocked when enumerating through the trainloader. Anyone know what the problem is as torch does recognize my gpu and i have both cuda and cudnn installed",
      "votes": null
    },
    {
      "id": "1278481",
      "postDate": "04/20/2021 00:12:40",
      "content": "<p>the thread is dead.<br>\nyou can set timeout variable in dataloader, it won't solve the problem but it will print the error and exit.</p>",
      "rawMarkdown": "the thread is dead.\nyou can set timeout variable in dataloader, it won't solve the problem but it will print the error and exit.",
      "votes": null
    },
    {
      "id": "1278606",
      "postDate": "04/20/2021 05:35:13",
      "content": "<p>If you generate synthetic data on the fly, there are inchis that kill a thread.<br>\nEliminate these inchis beforehand and only train on those that are not threadrippers.</p>",
      "rawMarkdown": "If you generate synthetic data on the fly, there are inchis that kill a thread.\nEliminate these inchis beforehand and only train on those that are not threadrippers.",
      "votes": null
    },
    {
      "id": "1278916",
      "postDate": "04/20/2021 12:12:03",
      "content": "<p>I had a problem with RAM overflow when using DataLoader (it starts from 7-8gb and goes to 14+ by the end of the data; iterating through torch.Dataset has no such problem)<br>\nfixed it by reducing the size of the epoch</p>",
      "rawMarkdown": "I had a problem with RAM overflow when using DataLoader (it starts from 7-8gb and goes to 14+ by the end of the data; iterating through torch.Dataset has no such problem)\nfixed it by reducing the size of the epoch",
      "votes": null
    },
    {
      "id": "1280661",
      "postDate": "04/22/2021 07:59:05",
      "content": "<p>I don‘t know are you like me，maybe you can try num_workers=0</p>",
      "rawMarkdown": "I don‘t know are you like me，maybe you can try num_workers=0",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1278481,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "04/20/2021 00:12:40",
      "content": "<p>the thread is dead.<br>\nyou can set timeout variable in dataloader, it won't solve the problem but it will print the error and exit.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1278606,
      "author_name": "nofreewill",
      "author_url": "",
      "post_date": "04/20/2021 05:35:13",
      "content": "<p>If you generate synthetic data on the fly, there are inchis that kill a thread.<br>\nEliminate these inchis beforehand and only train on those that are not threadrippers.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1278916,
      "author_name": "kalinin",
      "author_url": "",
      "post_date": "04/20/2021 12:12:03",
      "content": "<p>I had a problem with RAM overflow when using DataLoader (it starts from 7-8gb and goes to 14+ by the end of the data; iterating through torch.Dataset has no such problem)<br>\nfixed it by reducing the size of the epoch</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1280661,
      "author_name": "xlwan11",
      "author_url": "",
      "post_date": "04/22/2021 07:59:05",
      "content": "<p>I don‘t know are you like me，maybe you can try num_workers=0</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1278462": "When I try to train my model on my local gpu it gets deadlocked when enumerating through the trainloader. Anyone know what the problem is as torch does recognize my gpu and i have both cuda and cudnn installed",
    "1278481": "the thread is dead.\nyou can set timeout variable in dataloader, it won't solve the problem but it will print the error and exit.",
    "1278606": "If you generate synthetic data on the fly, there are inchis that kill a thread.\nEliminate these inchis beforehand and only train on those that are not threadrippers.",
    "1278916": "I had a problem with RAM overflow when using DataLoader (it starts from 7-8gb and goes to 14+ by the end of the data; iterating through torch.Dataset has no such problem)\nfixed it by reducing the size of the epoch",
    "1280661": "I don‘t know are you like me，maybe you can try num_workers=0"
  },
  "source": "meta"
}