{
  "id": 133671,
  "title": "load weights ",
  "url": "/competitions/bengaliai-cv19/discussion/133671",
  "author_name": "",
  "post_date": "2020-03-03T19:24:10.505842300Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi,how to load weights trained on multi gpu model to the single gpu model?</p>",
  "messages": [
    {
      "id": "762748",
      "postDate": "03/03/2020 19:24:10",
      "content": "<p>Hi,how to load weights trained on multi gpu model to the single gpu model?</p>",
      "rawMarkdown": "Hi,how to load weights trained on multi gpu model to the single gpu model?",
      "votes": null
    },
    {
      "id": "762808",
      "postDate": "03/03/2020 20:26:31",
      "content": "<p>You have to load weights with tf.distribute.MirroredStrategy()\n`\nstrategy = tf.distribute.MirroredStrategy()</p>\n\n<p>Here is a issue about it.\nwith strategy.scope():\n              model.load_wieghts(...)\n`\n<a href=\"https://github.com/tensorflow/tensorflow/issues/32561\">https://github.com/tensorflow/tensorflow/issues/32561</a></p>",
      "rawMarkdown": "You have to load weights with tf.distribute.MirroredStrategy()\n`\nstrategy = tf.distribute.MirroredStrategy()\n\nHere is a issue about it.\nwith strategy.scope():\n              model.load_wieghts(...)\n`\nhttps://github.com/tensorflow/tensorflow/issues/32561",
      "votes": null
    },
    {
      "id": "762848",
      "postDate": "03/03/2020 21:19:31",
      "content": "<p>In TensorFlow, nothing special, works like normal. Train on multi GPU, then use <code>model.save_weights()</code> like normal. Then upload those weights to Kaggle and use <code>model.load_weights()</code>.</p>",
      "rawMarkdown": "In TensorFlow, nothing special, works like normal. Train on multi GPU, then use `model.save_weights()` like normal. Then upload those weights to Kaggle and use `model.load_weights()`.",
      "votes": null
    },
    {
      "id": "762887",
      "postDate": "03/03/2020 22:51:45",
      "content": "<p>If you are using pytorch, you probably ran into the problem, that your keys include an extra <code>model.</code> You will need to strip that part of the key to load the weights.</p>\n\n<p>try this code (with some tweaks to fit your model)</p>\n\n<p><code>\nfrom collections import OrderedDict\nnew_state_dict = OrderedDict()\nfor k, v in checkpoint['model_state_dict'].items():  # checkpoint is your saved checkpoint including your model state\n    name = k[16:]  # remove `predictor.module` , change to your needs\n    name = \"predictor\" + name  # add back `predictor` , change to your needs\n    new_state_dict[name] = v\nclassifier.load_state_dict(new_state_dict)\n</code></p>",
      "rawMarkdown": "If you are using pytorch, you probably ran into the problem, that your keys include an extra `model.` You will need to strip that part of the key to load the weights.\n\ntry this code (with some tweaks to fit your model)\n\n```\nfrom collections import OrderedDict\nnew_state_dict = OrderedDict()\nfor k, v in checkpoint['model_state_dict'].items():  # checkpoint is your saved checkpoint including your model state\n    name = k[16:]  # remove `predictor.module` , change to your needs\n    name = \"predictor\" + name  # add back `predictor` , change to your needs\n    new_state_dict[name] = v\nclassifier.load_state_dict(new_state_dict)\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 762808,
      "author_name": "xarshila",
      "author_url": "",
      "post_date": "03/03/2020 20:26:31",
      "content": "<p>You have to load weights with tf.distribute.MirroredStrategy()\n`\nstrategy = tf.distribute.MirroredStrategy()</p>\n\n<p>Here is a issue about it.\nwith strategy.scope():\n              model.load_wieghts(...)\n`\n<a href=\"https://github.com/tensorflow/tensorflow/issues/32561\">https://github.com/tensorflow/tensorflow/issues/32561</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 762848,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "03/03/2020 21:19:31",
      "content": "<p>In TensorFlow, nothing special, works like normal. Train on multi GPU, then use <code>model.save_weights()</code> like normal. Then upload those weights to Kaggle and use <code>model.load_weights()</code>.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 762887,
      "author_name": "ilu000",
      "author_url": "",
      "post_date": "03/03/2020 22:51:45",
      "content": "<p>If you are using pytorch, you probably ran into the problem, that your keys include an extra <code>model.</code> You will need to strip that part of the key to load the weights.</p>\n\n<p>try this code (with some tweaks to fit your model)</p>\n\n<p><code>\nfrom collections import OrderedDict\nnew_state_dict = OrderedDict()\nfor k, v in checkpoint['model_state_dict'].items():  # checkpoint is your saved checkpoint including your model state\n    name = k[16:]  # remove `predictor.module` , change to your needs\n    name = \"predictor\" + name  # add back `predictor` , change to your needs\n    new_state_dict[name] = v\nclassifier.load_state_dict(new_state_dict)\n</code></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "762748": "Hi,how to load weights trained on multi gpu model to the single gpu model?",
    "762808": "You have to load weights with tf.distribute.MirroredStrategy()\n`\nstrategy = tf.distribute.MirroredStrategy()\n\nHere is a issue about it.\nwith strategy.scope():\n              model.load_wieghts(...)\n`\nhttps://github.com/tensorflow/tensorflow/issues/32561",
    "762848": "In TensorFlow, nothing special, works like normal. Train on multi GPU, then use `model.save_weights()` like normal. Then upload those weights to Kaggle and use `model.load_weights()`.",
    "762887": "If you are using pytorch, you probably ran into the problem, that your keys include an extra `model.` You will need to strip that part of the key to load the weights.\n\ntry this code (with some tweaks to fit your model)\n\n```\nfrom collections import OrderedDict\nnew_state_dict = OrderedDict()\nfor k, v in checkpoint['model_state_dict'].items():  # checkpoint is your saved checkpoint including your model state\n    name = k[16:]  # remove `predictor.module` , change to your needs\n    name = \"predictor\" + name  # add back `predictor` , change to your needs\n    new_state_dict[name] = v\nclassifier.load_state_dict(new_state_dict)\n```"
  },
  "source": "meta"
}