![\"distributed](\"https://storage.googleapis.com/image-uploader/kaggle_posts/guide1.png\" "\\"\\"")
Running Cloud Machine Learning Engine (CMLE) with GPU resources is as simple as using one of the GPU-enabled machine types or scale tiers in your yaml configuration file that you pass into your training job. These configuration files usually use either predefined \"scale tiers\", or define a custom scale tier with a custom set of machines, chosen by you. Let's look at each of these in turn, and why you might want to use them.
\n\nWhen first starting out and experimenting with GPUs, you can use the scale tier BASIC_GPU, which is a single worker machine with a GPU (basically the same as a single machine with a GPU attached). You will also want to only use a small subset of your data, since the purpose of this configuration is to set up your workflow. Your config.yaml might look something like this:
trainingInput:\n region: us-central1\n scaleTier: BASIC_GPU\nOnce you have your workflow down, you can graduate to more powerful machine types using a custom scale tier:
\n\nstandard_gpu: single GPU complex_model_m_gpu: four GPUscomplex_model_l_gpu: eight GPUsKeep in mind that when you are configuring your infrastructure, each of these is one machine, so the complex_model_m_gpu configuration is four GPUs attached to the same machine. In a distributed system, you might have several of these machines acting as worker machines, along with some parameter servers. \nSuch a config.yaml file might look something like this:
trainingInput:\n region: us-central1\n scaleTier: CUSTOM\n masterType: complex_model_m\n workerType: complex_model_m_gpu\n parameterServerType: large_model\n workerCount: 4\n parameterServerCount: 2\nNotice that only the workers have GPUs attached -- the other machine types do not need GPUs attached.
\n\n
Different machines consume differing amounts of \"ML Units\". You should go to this page detailing each type of machine and how many ML Units it consumes. The current limit on concurrent training units is 25 ML Units.
\n\nTo compute how many ML Units we used in the example above, we first look up how many units each of the machine types we used account for. The master, complex_model_m, is 3 ML Units. The worker, a complex_model_m_gpu, is 12 ML Units. Finally, the parameter server, a large_model, is 3 ML Units. \nSo our total count is 1x complex_model_m (3 ML Units) + 4x complex_model_m_gpu (12 ML Units) + 2x large_model (3 ML Units) = 3 + 48 + 6 = 57 ML Units!
![\"table](\"https://storage.googleapis.com/image-uploader/kaggle_posts/mlunittable.png\" "\\"\\"")
If we had used standard_gpu instead of complex_model_m_gpu, we would have dramatically reduced our ML Unit usage (it would also cut our GPU count by a factor of 4), since the standard_gpu is only 3 ML Units (which when compared to the ML Unit values of some of the other machine types really makes it quite a good value!). It would have reduced our consumption to only 21 ML Units.
Let's say you want to run that big job, with 57 ML Units consumed. You'd need to increase your quota. However, the quota for ML Units is not in the cloud console's IAM page, so you should head on over to the CMLE Quota Request Form and detail your needs there.
\n\nCurrent quota limits can be found on our Quotas page. Another limit to be aware of is the number of concurrent GPUs, which is 10 by default, so if you need more than 10 GPUs, be sure to request it using the Quota Request Form. In the example above, we had 4x complex_model_m_gpu machines, which each contain 4 GPUs, totaling 16 GPUs, so it would definitely need a quota increase for GPUs as well.
\n\nThe specifics of how you go about accessing GPUs in TensorFlow are a bit out of scope of this guide, but broadly speaking, you have two options: using the high-level APIs (Estimator) or using the low-level APIs. The \"Assigning ops to GPUs\" section of this article goes into more detail.
\n\nTensorFlow ClusterSpec: This will be useful for setting up your training to access your GPU resources
\n\nUsing GPUs On Compute Engine\nhttps://cloud.google.com/compute/docs/gpus/
\n\nDocumentation for using GPUs On Cloud Machine Learning Engine\nhttps://cloud.google.com/ml-engine/docs/how-tos/using-gpus
\n\nReading in the TSA data:\nWill has some start code for reading in the images here: https://www.kaggle.com/wcukierski/reading-images
", "messages": [ { "id": "207580", "postDate": "07/26/2017 23:15:02", "content": "Running Cloud Machine Learning Engine (CMLE) with GPU resources is as simple as using one of the GPU-enabled machine types or scale tiers in your yaml configuration file that you pass into your training job. These configuration files usually use either predefined \"scale tiers\", or define a custom scale tier with a custom set of machines, chosen by you. Let's look at each of these in turn, and why you might want to use them.
\n\nWhen first starting out and experimenting with GPUs, you can use the scale tier BASIC_GPU, which is a single worker machine with a GPU (basically the same as a single machine with a GPU attached). You will also want to only use a small subset of your data, since the purpose of this configuration is to set up your workflow. Your config.yaml might look something like this:
trainingInput:\n region: us-central1\n scaleTier: BASIC_GPU\nOnce you have your workflow down, you can graduate to more powerful machine types using a custom scale tier:
\n\nstandard_gpu: single GPU complex_model_m_gpu: four GPUscomplex_model_l_gpu: eight GPUsKeep in mind that when you are configuring your infrastructure, each of these is one machine, so the complex_model_m_gpu configuration is four GPUs attached to the same machine. In a distributed system, you might have several of these machines acting as worker machines, along with some parameter servers. \nSuch a config.yaml file might look something like this:
trainingInput:\n region: us-central1\n scaleTier: CUSTOM\n masterType: complex_model_m\n workerType: complex_model_m_gpu\n parameterServerType: large_model\n workerCount: 4\n parameterServerCount: 2\nNotice that only the workers have GPUs attached -- the other machine types do not need GPUs attached.
\n\n
Different machines consume differing amounts of \"ML Units\". You should go to this page detailing each type of machine and how many ML Units it consumes. The current limit on concurrent training units is 25 ML Units.
\n\nTo compute how many ML Units we used in the example above, we first look up how many units each of the machine types we used account for. The master, complex_model_m, is 3 ML Units. The worker, a complex_model_m_gpu, is 12 ML Units. Finally, the parameter server, a large_model, is 3 ML Units. \nSo our total count is 1x complex_model_m (3 ML Units) + 4x complex_model_m_gpu (12 ML Units) + 2x large_model (3 ML Units) = 3 + 48 + 6 = 57 ML Units!
If we had used standard_gpu instead of complex_model_m_gpu, we would have dramatically reduced our ML Unit usage (it would also cut our GPU count by a factor of 4), since the standard_gpu is only 3 ML Units (which when compared to the ML Unit values of some of the other machine types really makes it quite a good value!). It would have reduced our consumption to only 21 ML Units.
Let's say you want to run that big job, with 57 ML Units consumed. You'd need to increase your quota. However, the quota for ML Units is not in the cloud console's IAM page, so you should head on over to the CMLE Quota Request Form and detail your needs there.
\n\nCurrent quota limits can be found on our Quotas page. Another limit to be aware of is the number of concurrent GPUs, which is 10 by default, so if you need more than 10 GPUs, be sure to request it using the Quota Request Form. In the example above, we had 4x complex_model_m_gpu machines, which each contain 4 GPUs, totaling 16 GPUs, so it would definitely need a quota increase for GPUs as well.
\n\nThe specifics of how you go about accessing GPUs in TensorFlow are a bit out of scope of this guide, but broadly speaking, you have two options: using the high-level APIs (Estimator) or using the low-level APIs. The \"Assigning ops to GPUs\" section of this article goes into more detail.
\n\nTensorFlow ClusterSpec: This will be useful for setting up your training to access your GPU resources
\n\nUsing GPUs On Compute Engine\nhttps://cloud.google.com/compute/docs/gpus/
\n\nDocumentation for using GPUs On Cloud Machine Learning Engine\nhttps://cloud.google.com/ml-engine/docs/how-tos/using-gpus
\n\nReading in the TSA data:\nWill has some start code for reading in the images here: https://www.kaggle.com/wcukierski/reading-images
", "rawMarkdown": "Running [Cloud Machine Learning Engine (CMLE) with GPU][1] resources is as simple as using one of the GPU-enabled machine types or scale tiers in your yaml configuration file that you pass into your training job. These configuration files usually use either predefined \"scale tiers\", or define a custom scale tier with a custom set of machines, chosen by you. Let's look at each of these in turn, and why you might want to use them. \n\nGPUs by configuration\n================================\n\nWhen first starting out and experimenting with GPUs, you can use the scale tier `BASIC_GPU`, which is a single worker machine with a GPU (basically the same as a single machine with a GPU attached). You will also want to only use a small subset of your data, since the purpose of this configuration is to set up your workflow. Your config.yaml might look something like this:\n\n trainingInput:\n region: us-central1\n scaleTier: BASIC_GPU\n\nOnce you have your workflow down, you can graduate to [more powerful machine types][2] using a custom scale tier: \n\n - `standard_gpu`: single GPU \n - `complex_model_m_gpu`: four GPUs\n - `complex_model_l_gpu`: eight GPUs\n\nKeep in mind that when you are configuring your infrastructure, each of these is *one machine*, so the complex_model_m_gpu configuration is four GPUs attached to the same machine. In a distributed system, you might have several of these machines acting as worker machines, along with some parameter servers. \nSuch a `config.yaml` file might look something like this:\n\n trainingInput:\n region: us-central1\n scaleTier: CUSTOM\n masterType: complex_model_m\n workerType: complex_model_m_gpu\n parameterServerType: large_model\n workerCount: 4\n parameterServerCount: 2\n\nNotice that only the workers have GPUs attached -- the other machine types do not need GPUs attached. \n\n![distributed architecture diagram][3]\n\nUnderstanding ML Units and Quota\n================================\n\nDifferent machines consume differing amounts of \"ML Units\". You should go to [this page][4] detailing each type of machine and how many ML Units it consumes. The current limit on concurrent training units is 25 ML Units. \n\nTo compute how many ML Units we used in the example above, we first look up how many units each of the machine types we used account for. The master, `complex_model_m`, is 3 ML Units. The worker, a `complex_model_m_gpu`, is 12 ML Units. Finally, the parameter server, a `large_model`, is 3 ML Units. \nSo our total count is 1x `complex_model_m` (3 ML Units) + 4x `complex_model_m_gpu` (12 ML Units) + 2x `large_model` (3 ML Units) = 3 + 48 + 6 = 57 ML Units! \n\n![table of ML Unit examples][5]\n\nIf we had used `standard_gpu` instead of `complex_model_m_gpu`, we would have dramatically reduced our ML Unit usage (it would also cut our GPU count by a factor of 4), since the `standard_gpu` is only 3 ML Units (which when compared to the ML Unit values of some of the other machine types really makes it quite a good value!). It would have reduced our consumption to only 21 ML Units.\n\nLet's say you want to run that big job, with 57 ML Units consumed. You'd need to increase your quota. However, the quota for ML Units is not in the cloud console's IAM page, so you should head on over to the [CMLE Quota Request Form][6] and detail your needs there.\n\nCurrent quota limits can be found on our [Quotas page][7]. Another limit to be aware of is the number of concurrent GPUs, which is 10 by default, so if you need more than 10 GPUs, be sure to request it using the [Quota Request Form][8]. In the example above, we had 4x complex_model_m_gpu machines, which each contain 4 GPUs, totaling 16 GPUs, so it would definitely need a quota increase for GPUs as well.\n\nAccessing GPUs in TensorFlow code\n===\n\nThe specifics of how you go about accessing GPUs in TensorFlow are a bit out of scope of this guide, but broadly speaking, you have two options: using the high-level APIs (Estimator) or using the low-level APIs. The \"Assigning ops to GPUs\" section of [this article][9] goes into more detail.\n\nAdditional Resources and Reading\n===\n\n[TensorFlow ClusterSpec][10]: This will be useful for setting up your training to access your GPU resources\n\nUsing GPUs On Compute Engine\nhttps://cloud.google.com/compute/docs/gpus/ \n\nDocumentation for using GPUs On Cloud Machine Learning Engine\nhttps://cloud.google.com/ml-engine/docs/how-tos/using-gpus\n\nReading in the TSA data:\nWill has some start code for reading in the images here: https://www.kaggle.com/wcukierski/reading-images\n\n\n [1]: https://cloud.google.com/ml-engine/docs/how-tos/using-gpus\n [2]: https://cloud.google.com/ml-engine/docs/how-tos/using-gpus#requesting_gpu-enabled_machines\n [3]: https://storage.googleapis.com/image-uploader/kaggle_posts/guide1.png\n [4]: https://cloud.google.com/ml-engine/docs/concepts/training-overview#scale_tier\n [5]: https://storage.googleapis.com/image-uploader/kaggle_posts/mlunittable.png\n [6]: https://docs.google.com/a/google.com/forms/d/e/1FAIpQLScjm2W208wdf0iyQxBIESZ2_UvTS9xVPKleTQg9r_Uc1uPACQ/viewform\n [7]: https://cloud.google.com/ml-engine/quotas#limits_on_concurrent_ml_training_units_and_prediction_nodeshttp://\n [8]: https://docs.google.com/a/google.com/forms/d/e/1FAIpQLScjm2W208wdf0iyQxBIESZ2_UvTS9xVPKleTQg9r_Uc1uPACQ/viewform\n [9]: https://cloud.google.com/ml-engine/docs/how-tos/using-gpus#assigning_ops_to_gpus\n [10]: https://www.tensorflow.org/api_docs/python/tf/train/ClusterSpec", "votes": null }, { "id": "233695", "postDate": "10/20/2017 21:24:23", "content": "Could you please mention how much does it cost to use various GPU configuration?
", "rawMarkdown": "Could you please mention how much does it cost to use various GPU configuration?", "votes": null }, { "id": "234848", "postDate": "10/24/2017 07:33:13", "content": "See this table with all the machine types, and use the number of \"ML units\" to compute pricing.\nThe quick version is that as of this writing, it costs $0.49 per hour, per ML training unit, for training in the US, and $0.54 in Europe/Asia.
", "rawMarkdown": "See [this table](https://cloud.google.com/ml-engine/docs/training-overview#comparing_machine_types) with all the machine types, and use the number of \"ML units\" to compute [pricing](https://cloud.google.com/ml-engine/pricing).\nThe quick version is that as of this writing, it costs $0.49 per hour, per ML training unit, for training in the US, and $0.54 in Europe/Asia.", "votes": null }, { "id": "253648", "postDate": "12/05/2017 11:37:18", "content": "How do I run this on google cloud ?
\n\nimport tensorflow as tf;\nhello = tf.constant('Hello, TensorFlow!');\nsess = tf.Session();\nprint(sess.run(hello));\na = tf.constant([1.0, 2.0, 3.0, 4.0], shape=[2, 2], name='a');\nc = tf.matmul(a,a)\nwith tf.device('/device:GPU:0'):\n print(sess.run(c))\nI put the above code in a hi.py file, add the app.yaml below, run gcloud app deploy but get error: hi.py not found.
runtime: python\nenv: flex\nentrypoint: hi.py\nruntime_config:\n python_version: 3.5\nmanual_scaling:\n instances: 1\nresources:\n cpu: 8\n memory_gb: 51.6\n disk_size_gb: 100\nreadiness_check:\napp_start_timeout_sec: 3600\nI found this example, that works: https://cloud.google.com/ml-engine/docs/command-line
", "rawMarkdown": "I found this example, that works: https://cloud.google.com/ml-engine/docs/command-line", "votes": null } ], "comments": [ { "id": 233695, "author_name": "nikhilper", "author_url": "", "post_date": "10/20/2017 21:24:23", "content": "Could you please mention how much does it cost to use various GPU configuration?
", "votes": null, "replies": [ { "id": 234848, "author_name": "yufengg", "author_url": "", "post_date": "10/24/2017 07:33:13", "content": "See this table with all the machine types, and use the number of \"ML units\" to compute pricing.\nThe quick version is that as of this writing, it costs $0.49 per hour, per ML training unit, for training in the US, and $0.54 in Europe/Asia.
", "votes": null, "replies": [] } ] }, { "id": 253648, "author_name": "jduffy", "author_url": "", "post_date": "12/05/2017 11:37:18", "content": "How do I run this on google cloud ?
\n\nimport tensorflow as tf;\nhello = tf.constant('Hello, TensorFlow!');\nsess = tf.Session();\nprint(sess.run(hello));\na = tf.constant([1.0, 2.0, 3.0, 4.0], shape=[2, 2], name='a');\nc = tf.matmul(a,a)\nwith tf.device('/device:GPU:0'):\n print(sess.run(c))\nI put the above code in a hi.py file, add the app.yaml below, run gcloud app deploy but get error: hi.py not found.
runtime: python\nenv: flex\nentrypoint: hi.py\nruntime_config:\n python_version: 3.5\nmanual_scaling:\n instances: 1\nresources:\n cpu: 8\n memory_gb: 51.6\n disk_size_gb: 100\nreadiness_check:\napp_start_timeout_sec: 3600\nI found this example, that works: https://cloud.google.com/ml-engine/docs/command-line
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "207580": "Running [Cloud Machine Learning Engine (CMLE) with GPU][1] resources is as simple as using one of the GPU-enabled machine types or scale tiers in your yaml configuration file that you pass into your training job. These configuration files usually use either predefined \"scale tiers\", or define a custom scale tier with a custom set of machines, chosen by you. Let's look at each of these in turn, and why you might want to use them. \n\nGPUs by configuration\n================================\n\nWhen first starting out and experimenting with GPUs, you can use the scale tier `BASIC_GPU`, which is a single worker machine with a GPU (basically the same as a single machine with a GPU attached). You will also want to only use a small subset of your data, since the purpose of this configuration is to set up your workflow. Your config.yaml might look something like this:\n\n trainingInput:\n region: us-central1\n scaleTier: BASIC_GPU\n\nOnce you have your workflow down, you can graduate to [more powerful machine types][2] using a custom scale tier: \n\n - `standard_gpu`: single GPU \n - `complex_model_m_gpu`: four GPUs\n - `complex_model_l_gpu`: eight GPUs\n\nKeep in mind that when you are configuring your infrastructure, each of these is *one machine*, so the complex_model_m_gpu configuration is four GPUs attached to the same machine. In a distributed system, you might have several of these machines acting as worker machines, along with some parameter servers. \nSuch a `config.yaml` file might look something like this:\n\n trainingInput:\n region: us-central1\n scaleTier: CUSTOM\n masterType: complex_model_m\n workerType: complex_model_m_gpu\n parameterServerType: large_model\n workerCount: 4\n parameterServerCount: 2\n\nNotice that only the workers have GPUs attached -- the other machine types do not need GPUs attached. \n\n![distributed architecture diagram][3]\n\nUnderstanding ML Units and Quota\n================================\n\nDifferent machines consume differing amounts of \"ML Units\". You should go to [this page][4] detailing each type of machine and how many ML Units it consumes. The current limit on concurrent training units is 25 ML Units. \n\nTo compute how many ML Units we used in the example above, we first look up how many units each of the machine types we used account for. The master, `complex_model_m`, is 3 ML Units. The worker, a `complex_model_m_gpu`, is 12 ML Units. Finally, the parameter server, a `large_model`, is 3 ML Units. \nSo our total count is 1x `complex_model_m` (3 ML Units) + 4x `complex_model_m_gpu` (12 ML Units) + 2x `large_model` (3 ML Units) = 3 + 48 + 6 = 57 ML Units! \n\n![table of ML Unit examples][5]\n\nIf we had used `standard_gpu` instead of `complex_model_m_gpu`, we would have dramatically reduced our ML Unit usage (it would also cut our GPU count by a factor of 4), since the `standard_gpu` is only 3 ML Units (which when compared to the ML Unit values of some of the other machine types really makes it quite a good value!). It would have reduced our consumption to only 21 ML Units.\n\nLet's say you want to run that big job, with 57 ML Units consumed. You'd need to increase your quota. However, the quota for ML Units is not in the cloud console's IAM page, so you should head on over to the [CMLE Quota Request Form][6] and detail your needs there.\n\nCurrent quota limits can be found on our [Quotas page][7]. Another limit to be aware of is the number of concurrent GPUs, which is 10 by default, so if you need more than 10 GPUs, be sure to request it using the [Quota Request Form][8]. In the example above, we had 4x complex_model_m_gpu machines, which each contain 4 GPUs, totaling 16 GPUs, so it would definitely need a quota increase for GPUs as well.\n\nAccessing GPUs in TensorFlow code\n===\n\nThe specifics of how you go about accessing GPUs in TensorFlow are a bit out of scope of this guide, but broadly speaking, you have two options: using the high-level APIs (Estimator) or using the low-level APIs. The \"Assigning ops to GPUs\" section of [this article][9] goes into more detail.\n\nAdditional Resources and Reading\n===\n\n[TensorFlow ClusterSpec][10]: This will be useful for setting up your training to access your GPU resources\n\nUsing GPUs On Compute Engine\nhttps://cloud.google.com/compute/docs/gpus/ \n\nDocumentation for using GPUs On Cloud Machine Learning Engine\nhttps://cloud.google.com/ml-engine/docs/how-tos/using-gpus\n\nReading in the TSA data:\nWill has some start code for reading in the images here: https://www.kaggle.com/wcukierski/reading-images\n\n\n [1]: https://cloud.google.com/ml-engine/docs/how-tos/using-gpus\n [2]: https://cloud.google.com/ml-engine/docs/how-tos/using-gpus#requesting_gpu-enabled_machines\n [3]: https://storage.googleapis.com/image-uploader/kaggle_posts/guide1.png\n [4]: https://cloud.google.com/ml-engine/docs/concepts/training-overview#scale_tier\n [5]: https://storage.googleapis.com/image-uploader/kaggle_posts/mlunittable.png\n [6]: https://docs.google.com/a/google.com/forms/d/e/1FAIpQLScjm2W208wdf0iyQxBIESZ2_UvTS9xVPKleTQg9r_Uc1uPACQ/viewform\n [7]: https://cloud.google.com/ml-engine/quotas#limits_on_concurrent_ml_training_units_and_prediction_nodeshttp://\n [8]: https://docs.google.com/a/google.com/forms/d/e/1FAIpQLScjm2W208wdf0iyQxBIESZ2_UvTS9xVPKleTQg9r_Uc1uPACQ/viewform\n [9]: https://cloud.google.com/ml-engine/docs/how-tos/using-gpus#assigning_ops_to_gpus\n [10]: https://www.tensorflow.org/api_docs/python/tf/train/ClusterSpec", "233695": "Could you please mention how much does it cost to use various GPU configuration?", "234848": "See [this table](https://cloud.google.com/ml-engine/docs/training-overview#comparing_machine_types) with all the machine types, and use the number of \"ML units\" to compute [pricing](https://cloud.google.com/ml-engine/pricing).\nThe quick version is that as of this writing, it costs $0.49 per hour, per ML training unit, for training in the US, and $0.54 in Europe/Asia.", "253648": "How do I run this on google cloud ?\n\n import tensorflow as tf;\n hello = tf.constant('Hello, TensorFlow!');\n sess = tf.Session();\n print(sess.run(hello));\n a = tf.constant([1.0, 2.0, 3.0, 4.0], shape=[2, 2], name='a');\n c = tf.matmul(a,a)\n with tf.device('/device:GPU:0'):\n print(sess.run(c))", "253783": "I put the above code in a hi.py file, add the app.yaml below, run `gcloud app deploy` but get error: hi.py not found.\n\n runtime: python\n env: flex\n entrypoint: hi.py\n runtime_config:\n python_version: 3.5\n manual_scaling:\n instances: 1\n resources:\n cpu: 8\n memory_gb: 51.6\n disk_size_gb: 100\n readiness_check:\n app_start_timeout_sec: 3600", "254149": "I found this example, that works: https://cloud.google.com/ml-engine/docs/command-line" }, "source": "meta" }