{
  "id": 157612,
  "title": "TPU: compile within or outside scope",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/157612",
  "author_name": "",
  "post_date": "2020-06-11T10:58:13.706658700Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I noticed that there are some discrepancies in using <code>with strategy.scope():</code></p>\n\n<p>My original code was </p>\n\n<p>```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n    model.compile(\n        optimizer='adam',\n        loss = 'binary_crossentropy',\n        metrics=[tf.keras.metrics.AUC()])</p>\n\n<pre><code>return model\n</code></pre>\n\n<p>```</p>\n\n<p>which the model was compiled outside of scope; it ran just fine and I was lured into a false sense of security that this is ok. However, there may be some google/tpu update these few days and my model does not run; where the error says that I must compile within scope, in the following manner, notice the indentation:</p>\n\n<p>```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n        model.compile(\n            optimizer='adam',\n            loss = 'binary_crossentropy',\n            metrics=[tf.keras.metrics.AUC()])</p>\n\n<pre><code>return model\n</code></pre>\n\n<p>```</p>\n\n<p>Can any senpai confirm this that compiling the model within scope should be the \"correct way\" , prior to this I do see people using either ways.  :)</p>",
  "messages": [
    {
      "id": "881768",
      "postDate": "06/11/2020 10:58:13",
      "content": "<p>I noticed that there are some discrepancies in using <code>with strategy.scope():</code></p>\n\n<p>My original code was </p>\n\n<p>```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n    model.compile(\n        optimizer='adam',\n        loss = 'binary_crossentropy',\n        metrics=[tf.keras.metrics.AUC()])</p>\n\n<pre><code>return model\n</code></pre>\n\n<p>```</p>\n\n<p>which the model was compiled outside of scope; it ran just fine and I was lured into a false sense of security that this is ok. However, there may be some google/tpu update these few days and my model does not run; where the error says that I must compile within scope, in the following manner, notice the indentation:</p>\n\n<p>```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n        model.compile(\n            optimizer='adam',\n            loss = 'binary_crossentropy',\n            metrics=[tf.keras.metrics.AUC()])</p>\n\n<pre><code>return model\n</code></pre>\n\n<p>```</p>\n\n<p>Can any senpai confirm this that compiling the model within scope should be the \"correct way\" , prior to this I do see people using either ways.  :)</p>",
      "rawMarkdown": "I noticed that there are some discrepancies in using `with strategy.scope():`\n\nMy original code was \n\n```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n    model.compile(\n        optimizer='adam',\n        loss = 'binary_crossentropy',\n        metrics=[tf.keras.metrics.AUC()])\n    \n    return model\n```\n\nwhich the model was compiled outside of scope; it ran just fine and I was lured into a false sense of security that this is ok. However, there may be some google/tpu update these few days and my model does not run; where the error says that I must compile within scope, in the following manner, notice the indentation:\n\n```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n        model.compile(\n            optimizer='adam',\n            loss = 'binary_crossentropy',\n            metrics=[tf.keras.metrics.AUC()])\n    \n    return model\n```\n\nCan any senpai confirm this that compiling the model within scope should be the \"correct way\" , prior to this I do see people using either ways.  :)",
      "votes": null
    },
    {
      "id": "882124",
      "postDate": "06/11/2020 15:47:18",
      "content": "<p>It is best to both create the model, any custom metrics, as well as compile, all inside the scope,\notherwise you may accidentally use say a custom metric that was created in the wrong scope.</p>",
      "rawMarkdown": "It is best to both create the model, any custom metrics, as well as compile, all inside the scope,\notherwise you may accidentally use say a custom metric that was created in the wrong scope.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 882124,
      "author_name": "sirishks",
      "author_url": "",
      "post_date": "06/11/2020 15:47:18",
      "content": "<p>It is best to both create the model, any custom metrics, as well as compile, all inside the scope,\notherwise you may accidentally use say a custom metric that was created in the wrong scope.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "881768": "I noticed that there are some discrepancies in using `with strategy.scope():`\n\nMy original code was \n\n```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n    model.compile(\n        optimizer='adam',\n        loss = 'binary_crossentropy',\n        metrics=[tf.keras.metrics.AUC()])\n    \n    return model\n```\n\nwhich the model was compiled outside of scope; it ran just fine and I was lured into a false sense of security that this is ok. However, there may be some google/tpu update these few days and my model does not run; where the error says that I must compile within scope, in the following manner, notice the indentation:\n\n```\ndef get_model():\n    with strategy.scope():\n        model = tf.keras.Sequential([\n            efn.EfficientNetB3(\n                input_shape=(256,256, 3),\n                weights=\"imagenet\",\n                include_top=False\n            ),\n            L.GlobalAveragePooling2D(),\n            L.Dense(1, activation='sigmoid')\n        ])\n        model.compile(\n            optimizer='adam',\n            loss = 'binary_crossentropy',\n            metrics=[tf.keras.metrics.AUC()])\n    \n    return model\n```\n\nCan any senpai confirm this that compiling the model within scope should be the \"correct way\" , prior to this I do see people using either ways.  :)",
    "882124": "It is best to both create the model, any custom metrics, as well as compile, all inside the scope,\notherwise you may accidentally use say a custom metric that was created in the wrong scope."
  },
  "source": "meta"
}