{
  "id": 132020,
  "title": "optimizer.apply_gradient not work on tpu",
  "url": "/competitions/flower-classification-with-tpus/discussion/132020",
  "author_name": "",
  "post_date": "2020-02-23T13:09:01.214617700Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1714295%2F133be32fe95a18420174968fbdb63a00%2F2020-02-2321.02.16.png?generation=1582463215455710&amp;alt=media\" alt=\"\"></p>\n\n<p>I want to implement custom training loop. At first I follow <a href=\"https://www.tensorflow.org/guide/keras/train_and_evaluate#part_ii_writing_your_own_training_evaluation_loops_from_scratch\">this official guide</a>，but report a scope error. Then I use  <code>with strategy.scope()</code> wrapper, but report function not support error.</p>",
  "messages": [
    {
      "id": "754359",
      "postDate": "02/23/2020 13:09:01",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1714295%2F133be32fe95a18420174968fbdb63a00%2F2020-02-2321.02.16.png?generation=1582463215455710&amp;alt=media\" alt=\"\"></p>\n\n<p>I want to implement custom training loop. At first I follow <a href=\"https://www.tensorflow.org/guide/keras/train_and_evaluate#part_ii_writing_your_own_training_evaluation_loops_from_scratch\">this official guide</a>，but report a scope error. Then I use  <code>with strategy.scope()</code> wrapper, but report function not support error.</p>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1714295%2F133be32fe95a18420174968fbdb63a00%2F2020-02-2321.02.16.png?generation=1582463215455710&amp;alt=media)\n\nI want to implement custom training loop. At first I follow [this official guide](https://www.tensorflow.org/guide/keras/train_and_evaluate#part_ii_writing_your_own_training_evaluation_loops_from_scratch)，but report a scope error. Then I use  `with strategy.scope()` wrapper, but report function not support error.",
      "votes": null
    },
    {
      "id": "754484",
      "postDate": "02/23/2020 15:56:48",
      "content": "<p>You can have a look on my kernel to get an idea.</p>\n\n<p><a href=\"https://www.kaggle.com/yihdarshieh/custom-training-with-tpu\">https://www.kaggle.com/yihdarshieh/custom-training-with-tpu</a></p>",
      "rawMarkdown": "You can have a look on my kernel to get an idea.\n\n[https://www.kaggle.com/yihdarshieh/custom-training-with-tpu](https://www.kaggle.com/yihdarshieh/custom-training-with-tpu)",
      "votes": null
    },
    {
      "id": "754882",
      "postDate": "02/24/2020 07:06:16",
      "content": "<p>Thanks for your share!</p>",
      "rawMarkdown": "Thanks for your share!",
      "votes": null
    },
    {
      "id": "755499",
      "postDate": "02/24/2020 20:30:35",
      "content": "<p>Thank you <a href=\"/yihdarshieh\">@yihdarshieh</a> for posting an example. I confirm that custom training loops work fine on TPUs and even TPU pods (on GCP). Quick code snippet in <a href=\"https://docs.google.com/presentation/d/e/2PACX-1vRqvlSpX5CVRC2oQ_e_nRNahOSPoDVL6I36kdjuPR_4y_tCPb-_k98Du1QXBwx4sBvVrzsCPulmuPn8/pub?slide=id.g69e4cfce51bcd049_4623\">this presentation</a>.</p>",
      "rawMarkdown": "Thank you @yihdarshieh for posting an example. I confirm that custom training loops work fine on TPUs and even TPU pods (on GCP). Quick code snippet in [this presentation](https://docs.google.com/presentation/d/e/2PACX-1vRqvlSpX5CVRC2oQ_e_nRNahOSPoDVL6I36kdjuPR_4y_tCPb-_k98Du1QXBwx4sBvVrzsCPulmuPn8/pub?slide=id.g69e4cfce51bcd049_4623).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 754484,
      "author_name": "yihdarshieh",
      "author_url": "",
      "post_date": "02/23/2020 15:56:48",
      "content": "<p>You can have a look on my kernel to get an idea.</p>\n\n<p><a href=\"https://www.kaggle.com/yihdarshieh/custom-training-with-tpu\">https://www.kaggle.com/yihdarshieh/custom-training-with-tpu</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 754882,
          "author_name": "swordfaith",
          "author_url": "",
          "post_date": "02/24/2020 07:06:16",
          "content": "<p>Thanks for your share!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 755499,
      "author_name": "mgorner",
      "author_url": "",
      "post_date": "02/24/2020 20:30:35",
      "content": "<p>Thank you <a href=\"/yihdarshieh\">@yihdarshieh</a> for posting an example. I confirm that custom training loops work fine on TPUs and even TPU pods (on GCP). Quick code snippet in <a href=\"https://docs.google.com/presentation/d/e/2PACX-1vRqvlSpX5CVRC2oQ_e_nRNahOSPoDVL6I36kdjuPR_4y_tCPb-_k98Du1QXBwx4sBvVrzsCPulmuPn8/pub?slide=id.g69e4cfce51bcd049_4623\">this presentation</a>.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "754359": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1714295%2F133be32fe95a18420174968fbdb63a00%2F2020-02-2321.02.16.png?generation=1582463215455710&amp;alt=media)\n\nI want to implement custom training loop. At first I follow [this official guide](https://www.tensorflow.org/guide/keras/train_and_evaluate#part_ii_writing_your_own_training_evaluation_loops_from_scratch)，but report a scope error. Then I use  `with strategy.scope()` wrapper, but report function not support error.",
    "754484": "You can have a look on my kernel to get an idea.\n\n[https://www.kaggle.com/yihdarshieh/custom-training-with-tpu](https://www.kaggle.com/yihdarshieh/custom-training-with-tpu)",
    "754882": "Thanks for your share!",
    "755499": "Thank you @yihdarshieh for posting an example. I confirm that custom training loops work fine on TPUs and even TPU pods (on GCP). Quick code snippet in [this presentation](https://docs.google.com/presentation/d/e/2PACX-1vRqvlSpX5CVRC2oQ_e_nRNahOSPoDVL6I36kdjuPR_4y_tCPb-_k98Du1QXBwx4sBvVrzsCPulmuPn8/pub?slide=id.g69e4cfce51bcd049_4623)."
  },
  "source": "meta"
}