{
  "id": 401155,
  "title": "Very simple tensorflow question (ChatGPT fails ... help please) ",
  "url": "/competitions/asl-signs/discussion/401155",
  "author_name": "",
  "post_date": "2023-04-12T02:48:07.737058300Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I am struggling with something rather simple (or at least I think it should be because it is simple with PyTorch and numpy) <br>\nI want to convert this one line to tensorflow <code>xyz[:,RLIST]=ref</code></p>\n<pre><code>ref = np.random.randn(15,90,3)\nxyz = np.random.randn(15,543,3)\n\nRLIST = [468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488] + \\\n[522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542] + \\\n[504, 502, 500, 501, 503, 505, 512, 513] + [\n61, 185, 40, 39, 37, 0, 267, 269, 270, 409,\n291, 146, 91, 181, 84, 17, 314, 405, 321, 375,\n78, 191, 80, 81, 82, 13, 312, 311, 310, 415,\n95, 88, 178, 87, 14, 317, 402, 318, 324, 308] ## len(RLIST) = 90\n\nxyz[:,RLIST]=ref\n</code></pre>\n<p>Thank you so much !</p>",
  "messages": [
    {
      "id": "2218773",
      "postDate": "04/12/2023 02:48:07",
      "content": "<p>I am struggling with something rather simple (or at least I think it should be because it is simple with PyTorch and numpy) <br>\nI want to convert this one line to tensorflow <code>xyz[:,RLIST]=ref</code></p>\n<pre><code>ref = np.random.randn(15,90,3)\nxyz = np.random.randn(15,543,3)\n\nRLIST = [468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488] + \\\n[522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542] + \\\n[504, 502, 500, 501, 503, 505, 512, 513] + [\n61, 185, 40, 39, 37, 0, 267, 269, 270, 409,\n291, 146, 91, 181, 84, 17, 314, 405, 321, 375,\n78, 191, 80, 81, 82, 13, 312, 311, 310, 415,\n95, 88, 178, 87, 14, 317, 402, 318, 324, 308] ## len(RLIST) = 90\n\nxyz[:,RLIST]=ref\n</code></pre>\n<p>Thank you so much !</p>",
      "rawMarkdown": "I am struggling with something rather simple (or at least I think it should be because it is simple with PyTorch and numpy) \nI want to convert this one line to tensorflow `xyz[:,RLIST]=ref`\n\n```\nref = np.random.randn(15,90,3)\nxyz = np.random.randn(15,543,3)\n\nRLIST = [468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488] + \\\n[522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542] + \\\n[504, 502, 500, 501, 503, 505, 512, 513] + [\n61, 185, 40, 39, 37, 0, 267, 269, 270, 409,\n291, 146, 91, 181, 84, 17, 314, 405, 321, 375,\n78, 191, 80, 81, 82, 13, 312, 311, 310, 415,\n95, 88, 178, 87, 14, 317, 402, 318, 324, 308] ## len(RLIST) = 90\n\nxyz[:,RLIST]=ref\n```\n\nThank you so much !",
      "votes": null
    },
    {
      "id": "2219057",
      "postDate": "04/12/2023 09:07:00",
      "content": "<p>I have faced the similar problem and my solution was like:</p>\n<ol>\n<li>Sort indices in each group. And split groups with step 1. For example [25, 26, 28, 29] to [25, 26] and [28, 29]</li>\n<li>Split your xyz array to few arrays. Example: <code>xyz[:, np.arange(0, 25)], xyz[:, [25, 26]] , xyz[:, 27], xyz[:, [28, 29]]</code></li>\n<li>Then you can concatenate this to new tensor using needed parts like: <code>torch.cat([xyz[:, np.arange(0, 25)], ref[:, [0, 1, 2, 3]], ... ], dim=1)</code></li>\n</ol>\n<p>Or just use tensorflow preprocessing with function <code>tf.tensor_scatter_nd_update</code> or another from this type which is capable to your case</p>",
      "rawMarkdown": "I have faced the similar problem and my solution was like:\n1. Sort indices in each group. And split groups with step 1. For example [25, 26, 28, 29] to [25, 26] and [28, 29]\n2. Split your xyz array to few arrays. Example: `xyz[:, np.arange(0, 25)], xyz[:, [25, 26]] , xyz[:, 27], xyz[:, [28, 29]]`\n3. Then you can concatenate this to new tensor using needed parts like: `torch.cat([xyz[:, np.arange(0, 25)], ref[:, [0, 1, 2, 3]], ... ], dim=1)`\n\nOr just use tensorflow preprocessing with function `tf.tensor_scatter_nd_update` or another from this type which is capable to your case",
      "votes": null
    },
    {
      "id": "2219096",
      "postDate": "04/12/2023 09:42:26",
      "content": "<p>Thank you so much for your reply.  I have tried <code>tf.tensor_scatter_nd_update</code> and couldn't make it work. I'll try the sorting method that you have suggested. <br>\nI spent so much time with this <code>tf.tensor_scatter_nd_update</code> (chat gpt suggestion) and didn't get anywhere … after that all I could think of is cursing tensorflow 😅.  But, thank you so much for the sorting suggestion I'll do that. </p>",
      "rawMarkdown": "Thank you so much for your reply.  I have tried `tf.tensor_scatter_nd_update` and couldn't make it work. I'll try the sorting method that you have suggested. \nI spent so much time with this `tf.tensor_scatter_nd_update` (chat gpt suggestion) and didn't get anywhere ... after that all I could think of is cursing tensorflow 😅.  But, thank you so much for the sorting suggestion I'll do that.",
      "votes": null
    },
    {
      "id": "2219147",
      "postDate": "04/12/2023 10:30:17",
      "content": "<pre><code>ref = np.random.randn(,,)\nxyz = np.random.randn(,,)\n\nRLIST = [,,,,,,,,,,,,,,,,,,,,] + \\\n[,,,,,,,,,,,,,,,,,,,,] + \\\n[, , , , , , , ] + [\n, , , , , , , , , ,\n, , , , , , , , , ,\n, , , , , , , , , ,\n, , , , , , , , , ] \n\n\nxyz = tf.transpose(xyz, [,,])\nref = tf.transpose(ref, [,,])\nxyz = tf.tensor_scatter_nd_update(xyz, tf.expand_dims(RLIST, -), ref)\nxyz = tf.transpose(xyz, [,,])\nref = tf.transpose(ref, [,,])\n\nnp.allclose(xyz.numpy()[:,RLIST], ref)\n\n</code></pre>\n<p>tensor assignment with slicing is not supported in tensorflow(I don't know why though.. maybe they need to change the whole design to support this). you can use tf.tensor_scatter_nd_update instead. But when you need to assign value to any other axis than the first one, it becomes harder to define indices param. so one easy workaround is to transpose the tensor(or use tf.numpy.experimental.moveaxis) to change the order of the axis so that the axis you want the assignment to be the first one. hope this helps :)</p>",
      "rawMarkdown": "```python\nref = np.random.randn(15,90,3)\nxyz = np.random.randn(15,543,3)\n\nRLIST = [468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488] + \\\n[522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542] + \\\n[504, 502, 500, 501, 503, 505, 512, 513] + [\n61, 185, 40, 39, 37, 0, 267, 269, 270, 409,\n291, 146, 91, 181, 84, 17, 314, 405, 321, 375,\n78, 191, 80, 81, 82, 13, 312, 311, 310, 415,\n95, 88, 178, 87, 14, 317, 402, 318, 324, 308] ## len(RLIST) = 90\n\n# xyz[:,RLIST]=ref\nxyz = tf.transpose(xyz, [1,0,2])\nref = tf.transpose(ref, [1,0,2])\nxyz = tf.tensor_scatter_nd_update(xyz, tf.expand_dims(RLIST, -1), ref)\nxyz = tf.transpose(xyz, [1,0,2])\nref = tf.transpose(ref, [1,0,2])\n\nnp.allclose(xyz.numpy()[:,RLIST], ref)\nTrue\n```\n\n\ntensor assignment with slicing is not supported in tensorflow(I don't know why though.. maybe they need to change the whole design to support this). you can use tf.tensor_scatter_nd_update instead. But when you need to assign value to any other axis than the first one, it becomes harder to define indices param. so one easy workaround is to transpose the tensor(or use tf.numpy.experimental.moveaxis) to change the order of the axis so that the axis you want the assignment to be the first one. hope this helps :)",
      "votes": null
    },
    {
      "id": "2219286",
      "postDate": "04/12/2023 13:11:16",
      "content": "<p>Thank you so much for code and explanation ! </p>",
      "rawMarkdown": "Thank you so much for code and explanation !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2219057,
      "author_name": "kolyaforrat",
      "author_url": "",
      "post_date": "04/12/2023 09:07:00",
      "content": "<p>I have faced the similar problem and my solution was like:</p>\n<ol>\n<li>Sort indices in each group. And split groups with step 1. For example [25, 26, 28, 29] to [25, 26] and [28, 29]</li>\n<li>Split your xyz array to few arrays. Example: <code>xyz[:, np.arange(0, 25)], xyz[:, [25, 26]] , xyz[:, 27], xyz[:, [28, 29]]</code></li>\n<li>Then you can concatenate this to new tensor using needed parts like: <code>torch.cat([xyz[:, np.arange(0, 25)], ref[:, [0, 1, 2, 3]], ... ], dim=1)</code></li>\n</ol>\n<p>Or just use tensorflow preprocessing with function <code>tf.tensor_scatter_nd_update</code> or another from this type which is capable to your case</p>",
      "votes": null,
      "replies": [
        {
          "id": 2219096,
          "author_name": "rashmibanthia",
          "author_url": "",
          "post_date": "04/12/2023 09:42:26",
          "content": "<p>Thank you so much for your reply.  I have tried <code>tf.tensor_scatter_nd_update</code> and couldn't make it work. I'll try the sorting method that you have suggested. <br>\nI spent so much time with this <code>tf.tensor_scatter_nd_update</code> (chat gpt suggestion) and didn't get anywhere … after that all I could think of is cursing tensorflow 😅.  But, thank you so much for the sorting suggestion I'll do that. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2219147,
      "author_name": "hoyso48",
      "author_url": "",
      "post_date": "04/12/2023 10:30:17",
      "content": "<pre><code>ref = np.random.randn(,,)\nxyz = np.random.randn(,,)\n\nRLIST = [,,,,,,,,,,,,,,,,,,,,] + \\\n[,,,,,,,,,,,,,,,,,,,,] + \\\n[, , , , , , , ] + [\n, , , , , , , , , ,\n, , , , , , , , , ,\n, , , , , , , , , ,\n, , , , , , , , , ] \n\n\nxyz = tf.transpose(xyz, [,,])\nref = tf.transpose(ref, [,,])\nxyz = tf.tensor_scatter_nd_update(xyz, tf.expand_dims(RLIST, -), ref)\nxyz = tf.transpose(xyz, [,,])\nref = tf.transpose(ref, [,,])\n\nnp.allclose(xyz.numpy()[:,RLIST], ref)\n\n</code></pre>\n<p>tensor assignment with slicing is not supported in tensorflow(I don't know why though.. maybe they need to change the whole design to support this). you can use tf.tensor_scatter_nd_update instead. But when you need to assign value to any other axis than the first one, it becomes harder to define indices param. so one easy workaround is to transpose the tensor(or use tf.numpy.experimental.moveaxis) to change the order of the axis so that the axis you want the assignment to be the first one. hope this helps :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 2219286,
          "author_name": "rashmibanthia",
          "author_url": "",
          "post_date": "04/12/2023 13:11:16",
          "content": "<p>Thank you so much for code and explanation ! </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2218773": "I am struggling with something rather simple (or at least I think it should be because it is simple with PyTorch and numpy) \nI want to convert this one line to tensorflow `xyz[:,RLIST]=ref`\n\n```\nref = np.random.randn(15,90,3)\nxyz = np.random.randn(15,543,3)\n\nRLIST = [468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488] + \\\n[522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542] + \\\n[504, 502, 500, 501, 503, 505, 512, 513] + [\n61, 185, 40, 39, 37, 0, 267, 269, 270, 409,\n291, 146, 91, 181, 84, 17, 314, 405, 321, 375,\n78, 191, 80, 81, 82, 13, 312, 311, 310, 415,\n95, 88, 178, 87, 14, 317, 402, 318, 324, 308] ## len(RLIST) = 90\n\nxyz[:,RLIST]=ref\n```\n\nThank you so much !",
    "2219057": "I have faced the similar problem and my solution was like:\n1. Sort indices in each group. And split groups with step 1. For example [25, 26, 28, 29] to [25, 26] and [28, 29]\n2. Split your xyz array to few arrays. Example: `xyz[:, np.arange(0, 25)], xyz[:, [25, 26]] , xyz[:, 27], xyz[:, [28, 29]]`\n3. Then you can concatenate this to new tensor using needed parts like: `torch.cat([xyz[:, np.arange(0, 25)], ref[:, [0, 1, 2, 3]], ... ], dim=1)`\n\nOr just use tensorflow preprocessing with function `tf.tensor_scatter_nd_update` or another from this type which is capable to your case",
    "2219096": "Thank you so much for your reply.  I have tried `tf.tensor_scatter_nd_update` and couldn't make it work. I'll try the sorting method that you have suggested. \nI spent so much time with this `tf.tensor_scatter_nd_update` (chat gpt suggestion) and didn't get anywhere ... after that all I could think of is cursing tensorflow 😅.  But, thank you so much for the sorting suggestion I'll do that.",
    "2219147": "```python\nref = np.random.randn(15,90,3)\nxyz = np.random.randn(15,543,3)\n\nRLIST = [468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488] + \\\n[522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542] + \\\n[504, 502, 500, 501, 503, 505, 512, 513] + [\n61, 185, 40, 39, 37, 0, 267, 269, 270, 409,\n291, 146, 91, 181, 84, 17, 314, 405, 321, 375,\n78, 191, 80, 81, 82, 13, 312, 311, 310, 415,\n95, 88, 178, 87, 14, 317, 402, 318, 324, 308] ## len(RLIST) = 90\n\n# xyz[:,RLIST]=ref\nxyz = tf.transpose(xyz, [1,0,2])\nref = tf.transpose(ref, [1,0,2])\nxyz = tf.tensor_scatter_nd_update(xyz, tf.expand_dims(RLIST, -1), ref)\nxyz = tf.transpose(xyz, [1,0,2])\nref = tf.transpose(ref, [1,0,2])\n\nnp.allclose(xyz.numpy()[:,RLIST], ref)\nTrue\n```\n\n\ntensor assignment with slicing is not supported in tensorflow(I don't know why though.. maybe they need to change the whole design to support this). you can use tf.tensor_scatter_nd_update instead. But when you need to assign value to any other axis than the first one, it becomes harder to define indices param. so one easy workaround is to transpose the tensor(or use tf.numpy.experimental.moveaxis) to change the order of the axis so that the axis you want the assignment to be the first one. hope this helps :)",
    "2219286": "Thank you so much for code and explanation !"
  },
  "source": "meta"
}