![\"layout/xla/default/train/alexnet_train_batch_32.npz\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F253181%2Fa59bde825c0d42686be5d769be8dd8d1%2Falexnet_train_batch_32.png?generation=1695908074632543&alt=media\")
On the github page of the dataset it is written that:
\n: , \nBut shouldn't the output be the last node since they are ordered topologically? Because I have at least one tile that has multiple \"is_root\" nodes, while I think there should be only one output node per tile?
", "messages": [ { "id": "2452770", "postDate": "09/23/2023 15:12:35", "content": "On the github page of the dataset it is written that:
\n: , \nBut shouldn't the output be the last node since they are ordered topologically? Because I have at least one tile that has multiple \"is_root\" nodes, while I think there should be only one output node per tile?
", "rawMarkdown": "On the github page of the dataset it is written that:\n```python\n0: \"is_root\", # - whether this node is the output\",\n```\n\nBut shouldn't the output be the last node since they are ordered topologically? Because I have at least one tile that has multiple \"is_root\" nodes, while I think there should be only one output node per tile?", "votes": null }, { "id": "2453211", "postDate": "09/23/2023 21:47:09", "content": "It might be that a model produces multiple outputs.
\nFor example let's say you have a model like SAM (Segment Anything Now), it can give you mask segmentation and bounding boxes, so each of those is likely to be a different head and those would be root.
Ok that makes sense, thank you!
", "rawMarkdown": "Ok that makes sense, thank you!", "votes": null }, { "id": "2456025", "postDate": "09/25/2023 22:13:26", "content": "There could be a nested fusion, meaning that a fusion op calls another computation subgraph. In such case, the parent calling node will points to the root (output) node (is_root = true) of the nested fusion subgraph, and there will be multiple nodes with is_root = true.
\nFor example, this is one fused subgraph in HLO text format, and there are 3 nodes with is_root = true:
\nHloModule fusion.665, is_scheduled=true\n\n%model_combiner_hidden_1_attention_layer_norm_moments_mean-reduction.v3 (x.24: f32, y.24: f32) -> f32 {\n %x.24 = f32 parameter(0)\n %y.24 = f32 parameter(1)\n ROOT %add.23 = f32 add(f32 %x.24, f32 %y.24)\n}\n\n%fused_computation.647 (constant.1.param_1.327: f32, bitcast.59.param_1: f32, bitcast.60.param_2: f32) -> (f32, f32) {\n %bitcast.59.param_1 = f32{2,1,0} parameter(1)\n %bitcast.60.param_2 = f32{2,1,0} parameter(2)\n %convolution-base-dilated.45 = f32{2,1,0} convolution(f32{2,1,0} %bitcast.59.param_1, f32{2,1,0} %bitcast.60.param_2), window={size=32 stride=31 lhs_dilate=32}, dim_labels=0bf_0io->0bf\n %constant.1.param_1.327 = f32 parameter(0)\n %reduce.395 = f32{1,0} reduce(f32{2,1,0} %convolution-base-dilated.45, f32 %constant.1.param_1.327), dimensions={2}, to_apply=%model_combiner_hidden_1_attention_layer_norm_moments_mean-reduction.v3\n ROOT %tuple.783 = (f32{1,0}, f32{2,1,0}) tuple(f32{1,0} %reduce.395, f32{2,1,0} %convolution-base-dilated.45)\n}\n\nENTRY %fusion.665 (parameter.0: f32, parameter.1: f32, parameter.2: f32) -> (f32, f32) {\n %parameter.0 = f32 parameter(0)\n %parameter.1 = f32{2,1,0} parameter(1)\n %parameter.2 = f32{2,1,0} parameter(2)\n ROOT %fusion.665 = (f32{1,0}, f32{2,1,0}) fusion(f32 %parameter.0, f32{2,1,0} %parameter.1, f32{2,1,0} %parameter.2), =kOutput, calls=%fused_computation.647\n}\nHi @mangpophothilimthana, thanks for the explanation. But something is still not clear.
\nLets take for example layout/xla/default/train/alexnet_train_batch_32.npz. This one has 23 nodes which are is_root=1.
Nodes at indices 2, 5, 14, 20, ... etc do not appear to be outputs, because they have egress edges.
\nThe node at index 371 (tuple) appears to be the last node in the graph, so could be an output.
What is the difference between node 371 (tuple) and one of the other ones like 323 (compare)?
For reference here is a visualisation of the graph:
\n
is_root indicates that the node is an output of a local computation (you can think of a computation as a basic block). An entire program graph often contains multiple computations. Therefore, an output of a computation can be an input to another node.
", "rawMarkdown": "is_root indicates that the node is an output of a local computation (you can think of a computation as a basic block). An entire program graph often contains multiple computations. Therefore, an output of a computation can be an input to another node.", "votes": null } ], "comments": [ { "id": 2453211, "author_name": "ksmcg90", "author_url": "", "post_date": "09/23/2023 21:47:09", "content": "It might be that a model produces multiple outputs.
\nFor example let's say you have a model like SAM (Segment Anything Now), it can give you mask segmentation and bounding boxes, so each of those is likely to be a different head and those would be root.
Ok that makes sense, thank you!
", "votes": null, "replies": [] } ] }, { "id": 2456025, "author_name": "mangpophothilimthana", "author_url": "", "post_date": "09/25/2023 22:13:26", "content": "There could be a nested fusion, meaning that a fusion op calls another computation subgraph. In such case, the parent calling node will points to the root (output) node (is_root = true) of the nested fusion subgraph, and there will be multiple nodes with is_root = true.
\nFor example, this is one fused subgraph in HLO text format, and there are 3 nodes with is_root = true:
\nHloModule fusion.665, is_scheduled=true\n\n%model_combiner_hidden_1_attention_layer_norm_moments_mean-reduction.v3 (x.24: f32, y.24: f32) -> f32 {\n %x.24 = f32 parameter(0)\n %y.24 = f32 parameter(1)\n ROOT %add.23 = f32 add(f32 %x.24, f32 %y.24)\n}\n\n%fused_computation.647 (constant.1.param_1.327: f32, bitcast.59.param_1: f32, bitcast.60.param_2: f32) -> (f32, f32) {\n %bitcast.59.param_1 = f32{2,1,0} parameter(1)\n %bitcast.60.param_2 = f32{2,1,0} parameter(2)\n %convolution-base-dilated.45 = f32{2,1,0} convolution(f32{2,1,0} %bitcast.59.param_1, f32{2,1,0} %bitcast.60.param_2), window={size=32 stride=31 lhs_dilate=32}, dim_labels=0bf_0io->0bf\n %constant.1.param_1.327 = f32 parameter(0)\n %reduce.395 = f32{1,0} reduce(f32{2,1,0} %convolution-base-dilated.45, f32 %constant.1.param_1.327), dimensions={2}, to_apply=%model_combiner_hidden_1_attention_layer_norm_moments_mean-reduction.v3\n ROOT %tuple.783 = (f32{1,0}, f32{2,1,0}) tuple(f32{1,0} %reduce.395, f32{2,1,0} %convolution-base-dilated.45)\n}\n\nENTRY %fusion.665 (parameter.0: f32, parameter.1: f32, parameter.2: f32) -> (f32, f32) {\n %parameter.0 = f32 parameter(0)\n %parameter.1 = f32{2,1,0} parameter(1)\n %parameter.2 = f32{2,1,0} parameter(2)\n ROOT %fusion.665 = (f32{1,0}, f32{2,1,0}) fusion(f32 %parameter.0, f32{2,1,0} %parameter.1, f32{2,1,0} %parameter.2), =kOutput, calls=%fused_computation.647\n}\nHi @mangpophothilimthana, thanks for the explanation. But something is still not clear.
\nLets take for example layout/xla/default/train/alexnet_train_batch_32.npz. This one has 23 nodes which are is_root=1.
Nodes at indices 2, 5, 14, 20, ... etc do not appear to be outputs, because they have egress edges.
\nThe node at index 371 (tuple) appears to be the last node in the graph, so could be an output.
What is the difference between node 371 (tuple) and one of the other ones like 323 (compare)?
For reference here is a visualisation of the graph:
\n
is_root indicates that the node is an output of a local computation (you can think of a computation as a basic block). An entire program graph often contains multiple computations. Therefore, an output of a computation can be an input to another node.
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2452770": "On the github page of the dataset it is written that:\n```python\n0: \"is_root\", # - whether this node is the output\",\n```\n\nBut shouldn't the output be the last node since they are ordered topologically? Because I have at least one tile that has multiple \"is_root\" nodes, while I think there should be only one output node per tile?", "2453211": "It might be that a model produces multiple outputs.\nFor example let's say you have a model like SAM (Segment Anything Now), it can give you mask segmentation and bounding boxes, so each of those is likely to be a different head and those would be root.", "2453641": "Ok that makes sense, thank you!", "2456025": "There could be a nested fusion, meaning that a fusion op calls another computation subgraph. In such case, the parent calling node will points to the root (output) node (is_root = true) of the nested fusion subgraph, and there will be multiple nodes with is_root = true.\n\nFor example, this is one fused subgraph in HLO text format, and there are 3 nodes with is_root = true:\n```\nHloModule fusion.665, is_scheduled=true\n\n%model_combiner_hidden_1_attention_layer_norm_moments_mean-reduction.v3 (x.24: f32[], y.24: f32[]) -> f32[] {\n %x.24 = f32[] parameter(0)\n %y.24 = f32[] parameter(1)\n ROOT %add.23 = f32[] add(f32[] %x.24, f32[] %y.24)\n}\n\n%fused_computation.647 (constant.1.param_1.327: f32[], bitcast.59.param_1: f32[32,128,128], bitcast.60.param_2: f32[32,128,512]) -> (f32[32,128], f32[32,128,512]) {\n %bitcast.59.param_1 = f32[32,128,128]{2,1,0} parameter(1)\n %bitcast.60.param_2 = f32[32,128,512]{2,1,0} parameter(2)\n %convolution-base-dilated.45 = f32[32,128,512]{2,1,0} convolution(f32[32,128,128]{2,1,0} %bitcast.59.param_1, f32[32,128,512]{2,1,0} %bitcast.60.param_2), window={size=32 stride=31 lhs_dilate=32}, dim_labels=0bf_0io->0bf\n %constant.1.param_1.327 = f32[] parameter(0)\n %reduce.395 = f32[32,128]{1,0} reduce(f32[32,128,512]{2,1,0} %convolution-base-dilated.45, f32[] %constant.1.param_1.327), dimensions={2}, to_apply=%model_combiner_hidden_1_attention_layer_norm_moments_mean-reduction.v3\n ROOT %tuple.783 = (f32[32,128]{1,0}, f32[32,128,512]{2,1,0}) tuple(f32[32,128]{1,0} %reduce.395, f32[32,128,512]{2,1,0} %convolution-base-dilated.45)\n}\n\nENTRY %fusion.665 (parameter.0: f32[], parameter.1: f32[32,128,128], parameter.2: f32[32,128,512]) -> (f32[32,128], f32[32,128,512]) {\n %parameter.0 = f32[] parameter(0)\n %parameter.1 = f32[32,128,128]{2,1,0} parameter(1)\n %parameter.2 = f32[32,128,512]{2,1,0} parameter(2)\n ROOT %fusion.665 = (f32[32,128]{1,0}, f32[32,128,512]{2,1,0}) fusion(f32[] %parameter.0, f32[32,128,128]{2,1,0} %parameter.1, f32[32,128,512]{2,1,0} %parameter.2), kind=kOutput, calls=%fused_computation.647\n}\n```", "2459948": "Hi @mangpophothilimthana, thanks for the explanation. But something is still not clear.\n\nLets take for example `layout/xla/default/train/alexnet_train_batch_32.npz`. This one has 23 nodes which are `is_root=1`.\n\nNodes at indices `2, 5, 14, 20, ...` etc do not appear to be outputs, because they have egress edges. \nThe node at index `371 (tuple)` appears to be the last node in the graph, so could be an output.\n\nWhat is the difference between node `371 (tuple)` and one of the other ones like `323 (compare)`?\n\nFor reference here is a visualisation of the graph:\n\n", "2460773": "is_root indicates that the node is an output of a local computation (you can think of a computation as a basic block). An entire program graph often contains multiple computations. Therefore, an output of a computation can be an input to another node." }, "source": "meta" }