{
  "id": 122798,
  "title": "About Pytorch‘s backward()",
  "url": "/competitions/pku-autonomous-driving/discussion/122798",
  "author_name": "",
  "post_date": "2019-12-23T00:35:59.030547900Z",
  "votes": -1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I am now training a network with Pytorch. In the training, I use 64 bins to predict the dimensions.\n I find I can't make it. My loss.backward() can't work!\n```python\n            output_coor_x_ = output_coor_x_.squeeze()\n            output_coor_y_ = output_coor_y_.squeeze()\n            output_coor_z_ = output_coor_z_.squeeze()</p>\n\n<pre><code>       ####\n\n        output_coor_ = torch.stack([torch.argmax(output_coor_x_, axis=0),\n                                 torch.argmax(output_coor_y_, axis=0),\n                                 torch.argmax(output_coor_z_, axis=0)], axis=2)\n        output_coor_[output_coor_ == cfg.network.coor_bin] = 0\n        output_coor_ = 2.0 * output_coor_.float() / (63.0-1.0) - 1.0      # [-1,1]\n</code></pre>\n\n<p>```\nI try to use variable with before '####', loss.backward() work; but when I use variable after '####',  loss.backward() not work and output as follow:</p>\n\n<p><code>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</code></p>\n\n<p>I am sure that parameters's requires_grad=True in the model.</p>\n\n<p>Maybe I shouldn't use torch.argmax() ?</p>",
  "messages": [
    {
      "id": "701001",
      "postDate": "12/23/2019 00:35:59",
      "content": "<p>I am now training a network with Pytorch. In the training, I use 64 bins to predict the dimensions.\n I find I can't make it. My loss.backward() can't work!\n```python\n            output_coor_x_ = output_coor_x_.squeeze()\n            output_coor_y_ = output_coor_y_.squeeze()\n            output_coor_z_ = output_coor_z_.squeeze()</p>\n\n<pre><code>       ####\n\n        output_coor_ = torch.stack([torch.argmax(output_coor_x_, axis=0),\n                                 torch.argmax(output_coor_y_, axis=0),\n                                 torch.argmax(output_coor_z_, axis=0)], axis=2)\n        output_coor_[output_coor_ == cfg.network.coor_bin] = 0\n        output_coor_ = 2.0 * output_coor_.float() / (63.0-1.0) - 1.0      # [-1,1]\n</code></pre>\n\n<p>```\nI try to use variable with before '####', loss.backward() work; but when I use variable after '####',  loss.backward() not work and output as follow:</p>\n\n<p><code>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</code></p>\n\n<p>I am sure that parameters's requires_grad=True in the model.</p>\n\n<p>Maybe I shouldn't use torch.argmax() ?</p>",
      "rawMarkdown": "I am now training a network with Pytorch. In the training, I use 64 bins to predict the dimensions.\n I find I can't make it. My loss.backward() can't work!\n```python\n            output_coor_x_ = output_coor_x_.squeeze()\n            output_coor_y_ = output_coor_y_.squeeze()\n            output_coor_z_ = output_coor_z_.squeeze()\n\n           ####\n\n            output_coor_ = torch.stack([torch.argmax(output_coor_x_, axis=0),\n                                     torch.argmax(output_coor_y_, axis=0),\n                                     torch.argmax(output_coor_z_, axis=0)], axis=2)\n            output_coor_[output_coor_ == cfg.network.coor_bin] = 0\n            output_coor_ = 2.0 * output_coor_.float() / (63.0-1.0) - 1.0      # [-1,1]\n```\nI try to use variable with before '####', loss.backward() work; but when I use variable after '####',  loss.backward() not work and output as follow:\n\n`RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn`\n\nI am sure that parameters's requires_grad=True in the model.\n\nMaybe I shouldn't use torch.argmax() ?",
      "votes": null
    },
    {
      "id": "701147",
      "postDate": "12/23/2019 06:41:59",
      "content": "<p>argmax is not differenciable, so you should not use it.\nAnd if you think it as differenciable, its derivative is always 0.\nSo you Need to formulate it in differentiable form.</p>",
      "rawMarkdown": "argmax is not differenciable, so you should not use it.\nAnd if you think it as differenciable, its derivative is always 0.\nSo you Need to formulate it in differentiable form.",
      "votes": null
    },
    {
      "id": "701276",
      "postDate": "12/23/2019 09:58:14",
      "content": "<p>Thank you !\nNow I change some code to make it , and loss.backward() work:\n<code>\n            test_x = torch.argmax(output_coor_x_, axis=0).float()\n            test_y = torch.argmax(output_coor_y_, axis=0).float()\n            test_z = torch.argmax(output_coor_z_, axis=0).float()\n            test_x.requires_grad = True\n            test_y.requires_grad = True\n            test_z.requires_grad = True\n            output_coor_ = torch.stack([test_x,\n                                     test_y,\n                                     test_z], axis=2)\n</code>\nBut , will this get me what I want ?\nIf not , do you have any Suggestions to formulate it in differentiable form?\nI am a beginner, my skill is very average, I hope you can give me some advice😄 </p>",
      "rawMarkdown": "Thank you !\nNow I change some code to make it , and loss.backward() work:\n```\n            test_x = torch.argmax(output_coor_x_, axis=0).float()\n            test_y = torch.argmax(output_coor_y_, axis=0).float()\n            test_z = torch.argmax(output_coor_z_, axis=0).float()\n            test_x.requires_grad = True\n            test_y.requires_grad = True\n            test_z.requires_grad = True\n            output_coor_ = torch.stack([test_x,\n                                     test_y,\n                                     test_z], axis=2)\n```\nBut , will this get me what I want ?\nIf not , do you have any Suggestions to formulate it in differentiable form?\nI am a beginner, my skill is very average, I hope you can give me some advice😄",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 701147,
      "author_name": "phalanx",
      "author_url": "",
      "post_date": "12/23/2019 06:41:59",
      "content": "<p>argmax is not differenciable, so you should not use it.\nAnd if you think it as differenciable, its derivative is always 0.\nSo you Need to formulate it in differentiable form.</p>",
      "votes": null,
      "replies": [
        {
          "id": 701276,
          "author_name": "kiruto",
          "author_url": "",
          "post_date": "12/23/2019 09:58:14",
          "content": "<p>Thank you !\nNow I change some code to make it , and loss.backward() work:\n<code>\n            test_x = torch.argmax(output_coor_x_, axis=0).float()\n            test_y = torch.argmax(output_coor_y_, axis=0).float()\n            test_z = torch.argmax(output_coor_z_, axis=0).float()\n            test_x.requires_grad = True\n            test_y.requires_grad = True\n            test_z.requires_grad = True\n            output_coor_ = torch.stack([test_x,\n                                     test_y,\n                                     test_z], axis=2)\n</code>\nBut , will this get me what I want ?\nIf not , do you have any Suggestions to formulate it in differentiable form?\nI am a beginner, my skill is very average, I hope you can give me some advice😄 </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "701001": "I am now training a network with Pytorch. In the training, I use 64 bins to predict the dimensions.\n I find I can't make it. My loss.backward() can't work!\n```python\n            output_coor_x_ = output_coor_x_.squeeze()\n            output_coor_y_ = output_coor_y_.squeeze()\n            output_coor_z_ = output_coor_z_.squeeze()\n\n           ####\n\n            output_coor_ = torch.stack([torch.argmax(output_coor_x_, axis=0),\n                                     torch.argmax(output_coor_y_, axis=0),\n                                     torch.argmax(output_coor_z_, axis=0)], axis=2)\n            output_coor_[output_coor_ == cfg.network.coor_bin] = 0\n            output_coor_ = 2.0 * output_coor_.float() / (63.0-1.0) - 1.0      # [-1,1]\n```\nI try to use variable with before '####', loss.backward() work; but when I use variable after '####',  loss.backward() not work and output as follow:\n\n`RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn`\n\nI am sure that parameters's requires_grad=True in the model.\n\nMaybe I shouldn't use torch.argmax() ?",
    "701147": "argmax is not differenciable, so you should not use it.\nAnd if you think it as differenciable, its derivative is always 0.\nSo you Need to formulate it in differentiable form.",
    "701276": "Thank you !\nNow I change some code to make it , and loss.backward() work:\n```\n            test_x = torch.argmax(output_coor_x_, axis=0).float()\n            test_y = torch.argmax(output_coor_y_, axis=0).float()\n            test_z = torch.argmax(output_coor_z_, axis=0).float()\n            test_x.requires_grad = True\n            test_y.requires_grad = True\n            test_z.requires_grad = True\n            output_coor_ = torch.stack([test_x,\n                                     test_y,\n                                     test_z], axis=2)\n```\nBut , will this get me what I want ?\nIf not , do you have any Suggestions to formulate it in differentiable form?\nI am a beginner, my skill is very average, I hope you can give me some advice😄"
  },
  "source": "meta"
}