{
  "id": 453715,
  "title": "Do operations run in parallel?",
  "url": "/competitions/predict-ai-model-runtime/discussion/453715",
  "author_name": "",
  "post_date": "2023-11-07T12:44:07.308767300Z",
  "votes": 5,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi! I was wondering if when running a model, any two or more operations can run at the same time. E.g. One part of the TPU is doing a convolution while another is computing a subtraction. <br>\nFollowing the same idea, Is each model running on a single TPU or the graph might be split between TPUs?<br>\nCheers,<br>\nIgnacio</p>",
  "messages": [
    {
      "id": "2516056",
      "postDate": "11/07/2023 12:44:07",
      "content": "<p>Hi! I was wondering if when running a model, any two or more operations can run at the same time. E.g. One part of the TPU is doing a convolution while another is computing a subtraction. <br>\nFollowing the same idea, Is each model running on a single TPU or the graph might be split between TPUs?<br>\nCheers,<br>\nIgnacio</p>",
      "rawMarkdown": "Hi! I was wondering if when running a model, any two or more operations can run at the same time. E.g. One part of the TPU is doing a convolution while another is computing a subtraction. \nFollowing the same idea, Is each model running on a single TPU or the graph might be split between TPUs?\nCheers,\nIgnacio",
      "votes": null
    },
    {
      "id": "2517701",
      "postDate": "11/08/2023 17:28:03",
      "content": "<p>On TPU v3, a single operation (can be fusion of multiple ops) runs at a time.</p>\n<p>In practice, a graph may run on multiple TPUs (via model and data parallelism). However, for this dataset, we use a single TPU to measure the runtime of each graph. See section A.2 in the appendix of <a href=\"https://arxiv.org/pdf/2308.13490.pdf\" target=\"_blank\">our paper</a> for more info on how we measure the runtime of on a single TPU.</p>",
      "rawMarkdown": "On TPU v3, a single operation (can be fusion of multiple ops) runs at a time.\n\nIn practice, a graph may run on multiple TPUs (via model and data parallelism). However, for this dataset, we use a single TPU to measure the runtime of each graph. See section A.2 in the appendix of [our paper](https://arxiv.org/pdf/2308.13490.pdf) for more info on how we measure the runtime of on a single TPU.",
      "votes": null
    },
    {
      "id": "2517729",
      "postDate": "11/08/2023 17:42:45",
      "content": "<p>Perfect! Thanks for the info 👍</p>",
      "rawMarkdown": "Perfect! Thanks for the info 👍",
      "votes": null
    },
    {
      "id": "2518583",
      "postDate": "11/09/2023 12:36:34",
      "content": "<p>My notebook used GPUs to run the GNN model provided in the starter notebook, but the speedup was minimal. Running on GPU or CPU made no noticeable difference. I wonder if TF-GNN can benefit from GPUs at all? Is there any example about GPU TF-GNN? Thanks. </p>",
      "rawMarkdown": "My notebook used GPUs to run the GNN model provided in the starter notebook, but the speedup was minimal. Running on GPU or CPU made no noticeable difference. I wonder if TF-GNN can benefit from GPUs at all? Is there any example about GPU TF-GNN? Thanks.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2517701,
      "author_name": "mangpophothilimthana",
      "author_url": "",
      "post_date": "11/08/2023 17:28:03",
      "content": "<p>On TPU v3, a single operation (can be fusion of multiple ops) runs at a time.</p>\n<p>In practice, a graph may run on multiple TPUs (via model and data parallelism). However, for this dataset, we use a single TPU to measure the runtime of each graph. See section A.2 in the appendix of <a href=\"https://arxiv.org/pdf/2308.13490.pdf\" target=\"_blank\">our paper</a> for more info on how we measure the runtime of on a single TPU.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2517729,
          "author_name": "ireyes",
          "author_url": "",
          "post_date": "11/08/2023 17:42:45",
          "content": "<p>Perfect! Thanks for the info 👍</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2518583,
          "author_name": "minhsienweng",
          "author_url": "",
          "post_date": "11/09/2023 12:36:34",
          "content": "<p>My notebook used GPUs to run the GNN model provided in the starter notebook, but the speedup was minimal. Running on GPU or CPU made no noticeable difference. I wonder if TF-GNN can benefit from GPUs at all? Is there any example about GPU TF-GNN? Thanks. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2516056": "Hi! I was wondering if when running a model, any two or more operations can run at the same time. E.g. One part of the TPU is doing a convolution while another is computing a subtraction. \nFollowing the same idea, Is each model running on a single TPU or the graph might be split between TPUs?\nCheers,\nIgnacio",
    "2517701": "On TPU v3, a single operation (can be fusion of multiple ops) runs at a time.\n\nIn practice, a graph may run on multiple TPUs (via model and data parallelism). However, for this dataset, we use a single TPU to measure the runtime of each graph. See section A.2 in the appendix of [our paper](https://arxiv.org/pdf/2308.13490.pdf) for more info on how we measure the runtime of on a single TPU.",
    "2517729": "Perfect! Thanks for the info 👍",
    "2518583": "My notebook used GPUs to run the GNN model provided in the starter notebook, but the speedup was minimal. Running on GPU or CPU made no noticeable difference. I wonder if TF-GNN can benefit from GPUs at all? Is there any example about GPU TF-GNN? Thanks."
  },
  "source": "meta"
}