\n
![\"Screenshot-2023-09-04-at-13-54-11.png\"](\"https://i.postimg.cc/W4RKCprD/Screenshot-2023-09-04-at-13-54-11.png\")
Hi everyone,
\nWhen exploring the data, I'm confused about the following statement in the data tab,
\n\"edge_index\" contains int32 matrix with shape (m, 2). If entry i is = [u, v] (where 0 <= u, v < n), then there is a directed edge from node u to node v, where u consumes the output of v.Following is an illustration of the statement,
\n
If I'm not mistaken, the last sentence should be modified as,
\n\n\nwhere
\nvconsumes the output ofu.
Could you help confirm it, thanks a lot. @mangpophothilimthana
", "messages": [ { "id": "2422576", "postDate": "09/04/2023 05:59:56", "content": "Hi everyone,
\nWhen exploring the data, I'm confused about the following statement in the data tab,
\n\"edge_index\" contains int32 matrix with shape (m, 2). If entry i is = [u, v] (where 0 <= u, v < n), then there is a directed edge from node u to node v, where u consumes the output of v.Following is an illustration of the statement,
\n
If I'm not mistaken, the last sentence should be modified as,
\n\n\nwhere
\nvconsumes the output ofu.
Could you help confirm it, thanks a lot. @mangpophothilimthana
", "rawMarkdown": "Hi everyone,\n\nWhen exploring the data, I'm confused about the following statement in the [data tab](https://www.kaggle.com/competitions/predict-ai-model-runtime/data),\n\n* Key `\"edge_index\"` contains `int32` matrix with shape `(m, 2)`. If entry `i` is = `[u, v]` (where `0 <= u, v < n`), then there is a directed edge from node `u` to node `v`, where `u` consumes the output of `v`.\n\nFollowing is an illustration of the statement,\n[](https://postimg.cc/dhB4TY5v)\n\n\nIf I'm not mistaken, the last sentence should be modified as,\n\n> where `v` consumes the output of `u`.\n\nCould you help confirm it, thanks a lot. @mangpophothilimthana", "votes": null }, { "id": "2422863", "postDate": "09/04/2023 09:47:36", "content": "Was wondering about that too. Great question…
", "rawMarkdown": "Was wondering about that too. Great question...", "votes": null }, { "id": "2423075", "postDate": "09/04/2023 12:36:58", "content": "Perhaps you're right. Don't understand much of C++, but it seems that the edge matrix is built here: tpu_graphs/tpu_graphs/process_data/xla/hlo_encoder.cc
{\n (rows_advanced_, )\n << ;\n (indices_out_->(), )\n << ;\n (edges_added_ + , indices_out_->())\n << ;\n (from, to) << ;\n\n i = edges_added_;\n dim_idx = rows_advanced_ - ;\n from_idx = identifiers_maps_.(dim_idx).(from);\n to_idx = identifiers_maps_.(dim_idx).(to);\n\n indices_eigen = indices_out_-><>();\n (i, ) = dim_idx;\n (i, ) = from_idx;\n (i, ) = to_idx;\n edges_added_ += ;\n\n max_index_ = std::(std::(max_index_, from_idx), to_idx);\n}\nalthough this is a (n, 3) matrix.
", "rawMarkdown": "Perhaps you're right. Don't understand much of C++, but it seems that the edge matrix is built here: `tpu_graphs/tpu_graphs/process_data/xla/hlo_encoder.cc`\n\n```\nvoid EdgeListAdjMatrixBuilder::AddEdge(const int64_t from, const int64_t to) {\n CHECK_GE(rows_advanced_, 1)\n << \"AdvanceRow must be called at least once before adding an edge\";\n CHECK_GT(indices_out_->dim_size(0), 0)\n << \"Cannot add edge to adj. matrix with no capacity\";\n CHECK_LE(edges_added_ + 1, indices_out_->dim_size(0))\n << \"Output tensors are full\";\n CHECK_NE(from, to) << \"Cannot add self-edges\";\n\n const auto i = edges_added_;\n const auto dim_idx = rows_advanced_ - 1;\n const auto from_idx = identifiers_maps_.at(dim_idx).at(from);\n const auto to_idx = identifiers_maps_.at(dim_idx).at(to);\n\n auto indices_eigen = indices_out_->matrixYou are right. I will go ahead and update the documentation right now. Thank you!
", "rawMarkdown": "You are right. I will go ahead and update the documentation right now. Thank you!", "votes": null }, { "id": "2424109", "postDate": "09/05/2023 03:58:23", "content": "Hi @samihaija,
\nThanks for your quick reply!
", "rawMarkdown": "Hi @samihaija,\n\nThanks for your quick reply!", "votes": null }, { "id": "2424111", "postDate": "09/05/2023 03:59:55", "content": "Hi @intheflesh,
\nThanks for sharing this. I haven't had time to go through the source code, but I'll check it out!
", "rawMarkdown": "Hi @intheflesh,\n\nThanks for sharing this. I haven't had time to go through the source code, but I'll check it out!", "votes": null }, { "id": "2425289", "postDate": "09/05/2023 18:59:58", "content": "Correction: the original text is indeed accurate.
\nThe figure above is correct, but the definition of an edge in this problem is not intuitive. The source of an edge is a consumer of a tensor in XLA, and the target is a producer of a tensor in XLA.
\nHere is where an edge gets added: https://github.com/google-research-datasets/tpu_graphs/blob/main/tpu_graphs/process_data/xla/hlo_encoder.cc#L426C41-L426C41
\nIt points from an instruction (consuming operation) to an operand (producer).
", "rawMarkdown": "Correction: the original text is indeed accurate.\n\nThe figure above is correct, but the definition of an edge in this problem is not intuitive. The source of an edge is a consumer of a tensor in XLA, and the target is a producer of a tensor in XLA.\n\nHere is where an edge gets added: https://github.com/google-research-datasets/tpu_graphs/blob/main/tpu_graphs/process_data/xla/hlo_encoder.cc#L426C41-L426C41\n\nIt points from an instruction (consuming operation) to an operand (producer).", "votes": null }, { "id": "2429405", "postDate": "09/08/2023 15:27:22", "content": "\nIt‘s counterintuitive at first glance! Thanks again for your clarification and the modification on the data tab.
", "rawMarkdown": "Hi @mangpophothilimthana,\n\nIt‘s counterintuitive at first glance! Thanks again for your clarification and the modification on the data tab.", "votes": null } ], "comments": [ { "id": 2422863, "author_name": "roysegalz", "author_url": "", "post_date": "09/04/2023 09:47:36", "content": "Was wondering about that too. Great question…
", "votes": null, "replies": [] }, { "id": 2423075, "author_name": "intheflesh", "author_url": "", "post_date": "09/04/2023 12:36:58", "content": "Perhaps you're right. Don't understand much of C++, but it seems that the edge matrix is built here: tpu_graphs/tpu_graphs/process_data/xla/hlo_encoder.cc
{\n (rows_advanced_, )\n << ;\n (indices_out_->(), )\n << ;\n (edges_added_ + , indices_out_->())\n << ;\n (from, to) << ;\n\n i = edges_added_;\n dim_idx = rows_advanced_ - ;\n from_idx = identifiers_maps_.(dim_idx).(from);\n to_idx = identifiers_maps_.(dim_idx).(to);\n\n indices_eigen = indices_out_-><>();\n (i, ) = dim_idx;\n (i, ) = from_idx;\n (i, ) = to_idx;\n edges_added_ += ;\n\n max_index_ = std::(std::(max_index_, from_idx), to_idx);\n}\nalthough this is a (n, 3) matrix.
", "votes": null, "replies": [ { "id": 2424111, "author_name": "abaojiang", "author_url": "", "post_date": "09/05/2023 03:59:55", "content": "Hi @intheflesh,
\nThanks for sharing this. I haven't had time to go through the source code, but I'll check it out!
", "votes": null, "replies": [] } ] }, { "id": 2424016, "author_name": "samihaija", "author_url": "", "post_date": "09/05/2023 01:58:32", "content": "You are right. I will go ahead and update the documentation right now. Thank you!
", "votes": null, "replies": [ { "id": 2424109, "author_name": "abaojiang", "author_url": "", "post_date": "09/05/2023 03:58:23", "content": "Hi @samihaija,
\nThanks for your quick reply!
", "votes": null, "replies": [] } ] }, { "id": 2425289, "author_name": "mangpophothilimthana", "author_url": "", "post_date": "09/05/2023 18:59:58", "content": "Correction: the original text is indeed accurate.
\nThe figure above is correct, but the definition of an edge in this problem is not intuitive. The source of an edge is a consumer of a tensor in XLA, and the target is a producer of a tensor in XLA.
\nHere is where an edge gets added: https://github.com/google-research-datasets/tpu_graphs/blob/main/tpu_graphs/process_data/xla/hlo_encoder.cc#L426C41-L426C41
\nIt points from an instruction (consuming operation) to an operand (producer).
", "votes": null, "replies": [ { "id": 2429405, "author_name": "abaojiang", "author_url": "", "post_date": "09/08/2023 15:27:22", "content": "\nIt‘s counterintuitive at first glance! Thanks again for your clarification and the modification on the data tab.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2422576": "Hi everyone,\n\nWhen exploring the data, I'm confused about the following statement in the [data tab](https://www.kaggle.com/competitions/predict-ai-model-runtime/data),\n\n* Key `\"edge_index\"` contains `int32` matrix with shape `(m, 2)`. If entry `i` is = `[u, v]` (where `0 <= u, v < n`), then there is a directed edge from node `u` to node `v`, where `u` consumes the output of `v`.\n\nFollowing is an illustration of the statement,\n[](https://postimg.cc/dhB4TY5v)\n\n\nIf I'm not mistaken, the last sentence should be modified as,\n\n> where `v` consumes the output of `u`.\n\nCould you help confirm it, thanks a lot. @mangpophothilimthana", "2422863": "Was wondering about that too. Great question...", "2423075": "Perhaps you're right. Don't understand much of C++, but it seems that the edge matrix is built here: `tpu_graphs/tpu_graphs/process_data/xla/hlo_encoder.cc`\n\n```\nvoid EdgeListAdjMatrixBuilder::AddEdge(const int64_t from, const int64_t to) {\n CHECK_GE(rows_advanced_, 1)\n << \"AdvanceRow must be called at least once before adding an edge\";\n CHECK_GT(indices_out_->dim_size(0), 0)\n << \"Cannot add edge to adj. matrix with no capacity\";\n CHECK_LE(edges_added_ + 1, indices_out_->dim_size(0))\n << \"Output tensors are full\";\n CHECK_NE(from, to) << \"Cannot add self-edges\";\n\n const auto i = edges_added_;\n const auto dim_idx = rows_advanced_ - 1;\n const auto from_idx = identifiers_maps_.at(dim_idx).at(from);\n const auto to_idx = identifiers_maps_.at(dim_idx).at(to);\n\n auto indices_eigen = indices_out_->matrix