I'm having a hard time figuring out how to wire an aux loss head into efficientnetv2 in TensorFlow. I'm loading effnetv2 with tensorflow_hub and I cannot access the layers of the model. Similarly, I've loaded the model from the automl library and I cannot access the layers. Could somebody help me and those with the same issue? Maybe with a public notebook ? Thanks in advance and good luck.
", "messages": [ { "id": "1377991", "postDate": "07/06/2021 08:37:35", "content": "I'm having a hard time figuring out how to wire an aux loss head into efficientnetv2 in TensorFlow. I'm loading effnetv2 with tensorflow_hub and I cannot access the layers of the model. Similarly, I've loaded the model from the automl library and I cannot access the layers. Could somebody help me and those with the same issue? Maybe with a public notebook ? Thanks in advance and good luck.
", "rawMarkdown": "I'm having a hard time figuring out how to wire an aux loss head into efficientnetv2 in TensorFlow. I'm loading effnetv2 with tensorflow_hub and I cannot access the layers of the model. Similarly, I've loaded the model from the automl library and I cannot access the layers. Could somebody help me and those with the same issue? Maybe with a public notebook ? Thanks in advance and good luck.", "votes": null }, { "id": "1380301", "postDate": "07/08/2021 02:23:34", "content": "https://stackoverflow.com/questions/61996588/is-there-any-way-to-access-layers-in-tensorflow-hub-keraslayer-object this might help.
", "rawMarkdown": "https://stackoverflow.com/questions/61996588/is-there-any-way-to-access-layers-in-tensorflow-hub-keraslayer-object this might help.", "votes": null }, { "id": "1385036", "postDate": "07/12/2021 12:11:37", "content": "@josephamigo
\nI've been banging my head for the last two days solving the same problem as well. Hope this notebook can help you a bit if it's not too late. I'm not using hub.KerasLayer since it loads the checkpoint way too slow.
\nhttps://www.kaggle.com/gdoong/tf-keras-efficientnetv2
\nThere're two problem though. First, as you can see, the prediction result is different from the colab notebook provided in the github repo, which means that something is changed after calling build/get_layer method. Second, If you plot the model on tensorboard, you'll see that there's another EfficientNet model inside the model block, and the aux branch (from blocks_59 to the new head) are connected to the EfficientNet using control_dependencies_edge. Which raises two problems:
This is the most I can do for now.
", "rawMarkdown": "josephamigo\nI've been banging my head for the last two days solving the same problem as well. Hope this notebook can help you a bit if it's not too late. I'm not using hub.KerasLayer since it loads the checkpoint way too slow.\n[https://www.kaggle.com/gdoong/tf-keras-efficientnetv2](https://www.kaggle.com/gdoong/tf-keras-efficientnetv2)\nThere're two problem though. First, as you can see, the prediction result is different from the colab notebook provided in the github repo, which means that something is changed after calling build/get_layer method. Second, If you plot the model on tensorboard, you'll see that there's another EfficientNet model inside the model block, and the aux branch (from blocks_59 to the new head) are connected to the EfficientNet using **control_dependencies_edge**. Which raises two problems:\n1. Does this affect the performance? or this aux branch is actually doing nothing to the training?\n2. Does this use up more memory?\n\nThis is the most I can do for now.", "votes": null }, { "id": "1386082", "postDate": "07/13/2021 08:22:54", "content": "I have solved problem by using this
\nhub_url = 'gs://cloud-tpu-checkpoints/efficientnet/v2/hub/efficientnetv2-l/feature-vector'
\nmodel = tf.keras.Sequential([
\n tf.keras.layers.InputLayer(input_shape=[IMG_SIZE,IMG_SIZE, 3]),
\n hub.KerasLayer(hub_url, trainable=True),
\n tf.keras.layers.Dropout(rate=0.2),
\n tf.keras.layers.Dense(4, activation='softmax'),
\n ])
That's the detection head, not aux loss head 👀
", "rawMarkdown": "That's the detection head, not aux loss head 👀", "votes": null }, { "id": "1454165", "postDate": "08/06/2021 05:46:30", "content": "from tensorflow.keras import Model\nfrom tensorflow.keras import Sequential\nfrom tensorflow.keras import Input\nfrom tensorflow.keras import layers\nfrom tensorflow.keras import applications\n\n\nclass CovidNet(Model):\n def __init__(self):\n super(CovidNet, self).__init__()\n self.base = efficientnet_v2.EfficientNetV2XL(survivals=None, dropout=1e-6, classes=0, pretrained=None)\n\n # desired model\n self.base = Model(\n [self.base.inputs],\n [self.base.get_layer('post_swish').output, self.base.output]\n )\n\n # tail / head for the classifier\n self.tail = Sequential(\n [\n layers.GlobalAveragePooling2D(),\n layers.Dropout(0.2),\n layers.BatchNormalization(),\n layers.Dense(4),\n layers.Softmax()\n ]\n )\n\n # tail / head for the mask\n self.msk = Sequential(\n [\n layers.Conv2D(filters=512, kernel_size=(1, 1),\n strides=(1, 1), padding=\"same\"),\n layers.ReLU(),\n layers.BatchNormalization(),\n layers.Conv2D(filters=1, kernel_size=(1, 1), padding=\"same\")\n ]\n )\n\n # feed-forwarding\n def call(self, inputs, training=None, **kwargs):\n segg, clss = self.base(inputs['input'])\n\n return {\n 'clss': self.tail(clss),\n 'segg': self.msk(segg)\n }\n\n\ntf.keras.backend.clear_session()\nmodel = CovidNet()\nmodel.build(input_shape={'input': (None, 512, 512, 3)})\nmodel.summary()\nHow about this?
", "rawMarkdown": "```python\nfrom tensorflow.keras import Model\nfrom tensorflow.keras import Sequential\nfrom tensorflow.keras import Input\nfrom tensorflow.keras import layers\nfrom tensorflow.keras import applications\n\n\nclass CovidNet(Model):\n def __init__(self):\n super(CovidNet, self).__init__()\n self.base = efficientnet_v2.EfficientNetV2XL(survivals=None, dropout=1e-6, classes=0, pretrained=None)\n\n # desired model\n self.base = Model(\n [self.base.inputs],\n [self.base.get_layer('post_swish').output, self.base.output]\n )\n\n # tail / head for the classifier\n self.tail = Sequential(\n [\n layers.GlobalAveragePooling2D(),\n layers.Dropout(0.2),\n layers.BatchNormalization(),\n layers.Dense(4),\n layers.Softmax()\n ]\n )\n\n # tail / head for the mask\n self.msk = Sequential(\n [\n layers.Conv2D(filters=512, kernel_size=(1, 1),\n strides=(1, 1), padding=\"same\"),\n layers.ReLU(),\n layers.BatchNormalization(),\n layers.Conv2D(filters=1, kernel_size=(1, 1), padding=\"same\")\n ]\n )\n\n # feed-forwarding\n def call(self, inputs, training=None, **kwargs):\n segg, clss = self.base(inputs['input'])\n\n return {\n 'clss': self.tail(clss),\n 'segg': self.msk(segg)\n }\n\n\ntf.keras.backend.clear_session()\nmodel = CovidNet()\nmodel.build(input_shape={'input': (None, 512, 512, 3)})\nmodel.summary()\n```\n\nHow about this?", "votes": null }, { "id": "1461375", "postDate": "08/09/2021 11:17:26", "content": "@logicng
\nI don't think the code will work?. Based on my experiments, authors of automl's efficientnetv2 didn't really implement keras model's build method to let keras figure out the topo of the model and output shape of the individual layers. That's why in my notebook, I call the call / build method before model.get_layer('blocks_59').output, otherwise it will raise AttributeError complaining layer has no inbound nodes.
https://stackoverflow.com/questions/61996588/is-there-any-way-to-access-layers-in-tensorflow-hub-keraslayer-object this might help.
", "votes": null, "replies": [] }, { "id": 1385036, "author_name": "gdoong", "author_url": "", "post_date": "07/12/2021 12:11:37", "content": "@josephamigo
\nI've been banging my head for the last two days solving the same problem as well. Hope this notebook can help you a bit if it's not too late. I'm not using hub.KerasLayer since it loads the checkpoint way too slow.
\nhttps://www.kaggle.com/gdoong/tf-keras-efficientnetv2
\nThere're two problem though. First, as you can see, the prediction result is different from the colab notebook provided in the github repo, which means that something is changed after calling build/get_layer method. Second, If you plot the model on tensorboard, you'll see that there's another EfficientNet model inside the model block, and the aux branch (from blocks_59 to the new head) are connected to the EfficientNet using control_dependencies_edge. Which raises two problems:
This is the most I can do for now.
", "votes": null, "replies": [] }, { "id": 1386082, "author_name": "darkravager", "author_url": "", "post_date": "07/13/2021 08:22:54", "content": "I have solved problem by using this
\nhub_url = 'gs://cloud-tpu-checkpoints/efficientnet/v2/hub/efficientnetv2-l/feature-vector'
\nmodel = tf.keras.Sequential([
\n tf.keras.layers.InputLayer(input_shape=[IMG_SIZE,IMG_SIZE, 3]),
\n hub.KerasLayer(hub_url, trainable=True),
\n tf.keras.layers.Dropout(rate=0.2),
\n tf.keras.layers.Dense(4, activation='softmax'),
\n ])
That's the detection head, not aux loss head 👀
", "votes": null, "replies": [] } ] }, { "id": 1454165, "author_name": "logicng", "author_url": "", "post_date": "08/06/2021 05:46:30", "content": "from tensorflow.keras import Model\nfrom tensorflow.keras import Sequential\nfrom tensorflow.keras import Input\nfrom tensorflow.keras import layers\nfrom tensorflow.keras import applications\n\n\nclass CovidNet(Model):\n def __init__(self):\n super(CovidNet, self).__init__()\n self.base = efficientnet_v2.EfficientNetV2XL(survivals=None, dropout=1e-6, classes=0, pretrained=None)\n\n # desired model\n self.base = Model(\n [self.base.inputs],\n [self.base.get_layer('post_swish').output, self.base.output]\n )\n\n # tail / head for the classifier\n self.tail = Sequential(\n [\n layers.GlobalAveragePooling2D(),\n layers.Dropout(0.2),\n layers.BatchNormalization(),\n layers.Dense(4),\n layers.Softmax()\n ]\n )\n\n # tail / head for the mask\n self.msk = Sequential(\n [\n layers.Conv2D(filters=512, kernel_size=(1, 1),\n strides=(1, 1), padding=\"same\"),\n layers.ReLU(),\n layers.BatchNormalization(),\n layers.Conv2D(filters=1, kernel_size=(1, 1), padding=\"same\")\n ]\n )\n\n # feed-forwarding\n def call(self, inputs, training=None, **kwargs):\n segg, clss = self.base(inputs['input'])\n\n return {\n 'clss': self.tail(clss),\n 'segg': self.msk(segg)\n }\n\n\ntf.keras.backend.clear_session()\nmodel = CovidNet()\nmodel.build(input_shape={'input': (None, 512, 512, 3)})\nmodel.summary()\nHow about this?
", "votes": null, "replies": [ { "id": 1461375, "author_name": "gdoong", "author_url": "", "post_date": "08/09/2021 11:17:26", "content": "@logicng
\nI don't think the code will work?. Based on my experiments, authors of automl's efficientnetv2 didn't really implement keras model's build method to let keras figure out the topo of the model and output shape of the individual layers. That's why in my notebook, I call the call / build method before model.get_layer('blocks_59').output, otherwise it will raise AttributeError complaining layer has no inbound nodes.