{
  "id": 137856,
  "title": "Model Deployment with TPUs?",
  "url": "/competitions/flower-classification-with-tpus/discussion/137856",
  "author_name": "",
  "post_date": "2020-03-22T19:22:44.502680900Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I was thinking about how good is TPUs for model deployment, in the real world. Is TPUs being supported with Kubernetes clusters, Kubeflow, or TensorFlow serving?</p>\n\n<p>cc: <a href=\"/mgornergoogle\">@mgornergoogle</a></p>",
  "messages": [
    {
      "id": "782898",
      "postDate": "03/22/2020 19:22:44",
      "content": "<p>I was thinking about how good is TPUs for model deployment, in the real world. Is TPUs being supported with Kubernetes clusters, Kubeflow, or TensorFlow serving?</p>\n\n<p>cc: <a href=\"/mgornergoogle\">@mgornergoogle</a></p>",
      "rawMarkdown": "I was thinking about how good is TPUs for model deployment, in the real world. Is TPUs being supported with Kubernetes clusters, Kubeflow, or TensorFlow serving?\n\ncc: @mgornergoogle",
      "votes": null
    },
    {
      "id": "784052",
      "postDate": "03/23/2020 22:57:41",
      "content": "<p>Yes TPUs are supported with Kubernetes. Support for TF serving is in the works.</p>\n\n<p>TPUs are great for inference if you need inference on large batches. For example batch inference jobs. Or online inference jobs with very high traffic. For inference on a single image where low latency is the key metric, TPUs will not be ideal.</p>",
      "rawMarkdown": "Yes TPUs are supported with Kubernetes. Support for TF serving is in the works.\n\nTPUs are great for inference if you need inference on large batches. For example batch inference jobs. Or online inference jobs with very high traffic. For inference on a single image where low latency is the key metric, TPUs will not be ideal.",
      "votes": null
    },
    {
      "id": "784299",
      "postDate": "03/24/2020 05:05:31",
      "content": "<p>Thanks for the detailed reply <a href=\"/mgornergoogle\">@mgornergoogle</a> </p>",
      "rawMarkdown": "Thanks for the detailed reply @mgornergoogle",
      "votes": null
    },
    {
      "id": "821977",
      "postDate": "04/26/2020 14:50:43",
      "content": "<p>I succeeded doing this using cnvrg.io platform.\nOne of their expertise is deploying on kube clusters.</p>\n\n<p>They just launched a free version last month. You can it right here -&gt; <a href=\"https://cnvrg.io/platform/core/\">https://cnvrg.io/platform/core/</a></p>",
      "rawMarkdown": "I succeeded doing this using cnvrg.io platform.\nOne of their expertise is deploying on kube clusters.\n\nThey just launched a free version last month. You can it right here -&gt; https://cnvrg.io/platform/core/",
      "votes": null
    },
    {
      "id": "823563",
      "postDate": "04/27/2020 18:20:16",
      "content": "<p>Thanks for letting us know that. Did you use Kubeflow? or made notebook in a kubernetees instance and host it?</p>",
      "rawMarkdown": "Thanks for letting us know that. Did you use Kubeflow? or made notebook in a kubernetees instance and host it?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 784052,
      "author_name": "mgorner",
      "author_url": "",
      "post_date": "03/23/2020 22:57:41",
      "content": "<p>Yes TPUs are supported with Kubernetes. Support for TF serving is in the works.</p>\n\n<p>TPUs are great for inference if you need inference on large batches. For example batch inference jobs. Or online inference jobs with very high traffic. For inference on a single image where low latency is the key metric, TPUs will not be ideal.</p>",
      "votes": null,
      "replies": [
        {
          "id": 784299,
          "author_name": "kurianbenoy",
          "author_url": "",
          "post_date": "03/24/2020 05:05:31",
          "content": "<p>Thanks for the detailed reply <a href=\"/mgornergoogle\">@mgornergoogle</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 821977,
      "author_name": "omerliberman",
      "author_url": "",
      "post_date": "04/26/2020 14:50:43",
      "content": "<p>I succeeded doing this using cnvrg.io platform.\nOne of their expertise is deploying on kube clusters.</p>\n\n<p>They just launched a free version last month. You can it right here -&gt; <a href=\"https://cnvrg.io/platform/core/\">https://cnvrg.io/platform/core/</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 823563,
          "author_name": "kurianbenoy",
          "author_url": "",
          "post_date": "04/27/2020 18:20:16",
          "content": "<p>Thanks for letting us know that. Did you use Kubeflow? or made notebook in a kubernetees instance and host it?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "782898": "I was thinking about how good is TPUs for model deployment, in the real world. Is TPUs being supported with Kubernetes clusters, Kubeflow, or TensorFlow serving?\n\ncc: @mgornergoogle",
    "784052": "Yes TPUs are supported with Kubernetes. Support for TF serving is in the works.\n\nTPUs are great for inference if you need inference on large batches. For example batch inference jobs. Or online inference jobs with very high traffic. For inference on a single image where low latency is the key metric, TPUs will not be ideal.",
    "784299": "Thanks for the detailed reply @mgornergoogle",
    "821977": "I succeeded doing this using cnvrg.io platform.\nOne of their expertise is deploying on kube clusters.\n\nThey just launched a free version last month. You can it right here -&gt; https://cnvrg.io/platform/core/",
    "823563": "Thanks for letting us know that. Did you use Kubeflow? or made notebook in a kubernetees instance and host it?"
  },
  "source": "meta"
}