{ "id": 133565, "title": "why the tpu valacc is low but chang to gpu seems ok ", "url": "/competitions/flower-classification-with-tpus/discussion/133565", "author_name": "", "post_date": "2020-03-03T10:18:26.132052Z", "votes": 1, "comment_count": 1, "views": 0, "content": "

I changed the model to resnet50 based on \nhttps://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu\nwhen train the val acc is only 0.03 at every epochs ,but changed the kernel to gpu,the val acc seems ok,why?

\n\n

```\nwith strategy.scope():\n pretrained_model = tf.keras.applications.resnet50.ResNet50(weights='imagenet', include_top=False ,input_shape=[*IMAGE_SIZE, 3])\n pretrained_model.trainable = True # tramsfer learning\n #pretrained_model.trainable = True\n model = tf.keras.Sequential([\n pretrained_model,\n #tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])

\n\n

model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n)\nmodel.summary()\n```\nthis gpu\n\"\"\nthis tpu ,only 0.03 after 5 epoches\"\"

", "messages": [ { "id": "762203", "postDate": "03/03/2020 10:18:26", "content": "

I changed the model to resnet50 based on \nhttps://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu\nwhen train the val acc is only 0.03 at every epochs ,but changed the kernel to gpu,the val acc seems ok,why?

\n\n

```\nwith strategy.scope():\n pretrained_model = tf.keras.applications.resnet50.ResNet50(weights='imagenet', include_top=False ,input_shape=[*IMAGE_SIZE, 3])\n pretrained_model.trainable = True # tramsfer learning\n #pretrained_model.trainable = True\n model = tf.keras.Sequential([\n pretrained_model,\n #tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])

\n\n

model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n)\nmodel.summary()\n```\nthis gpu\n\"\"\nthis tpu ,only 0.03 after 5 epoches\"\"

", "rawMarkdown": "I changed the model to resnet50 based on \nhttps://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu\nwhen train the val acc is only 0.03 at every epochs ,but changed the kernel to gpu,the val acc seems ok,why?\n\n```\nwith strategy.scope():\n pretrained_model = tf.keras.applications.resnet50.ResNet50(weights='imagenet', include_top=False ,input_shape=[*IMAGE_SIZE, 3])\n pretrained_model.trainable = True # tramsfer learning\n #pretrained_model.trainable = True\n model = tf.keras.Sequential([\n pretrained_model,\n #tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])\n\n\nmodel.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n)\nmodel.summary()\n```\nthis gpu\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252031%2F66c54c1c4bba08c9b98069f82c657b30%2FQQ20200303183319.jpg?generation=1583231676656167&alt=media)\nthis tpu ,only 0.03 after 5 epoches![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252031%2F31e9cf5ad3cee153464534826d3c2a98%2FQQ.jpg?generation=1583231923943040&alt=media)", "votes": null }, { "id": "762632", "postDate": "03/03/2020 17:01:45", "content": "

In this Notebook, switching between TPU and GPU also changes the batch size and the learning rate. You are probably using a learning rate / batch size combination that does not work well for fine-tuning ResNet.

", "rawMarkdown": "In this Notebook, switching between TPU and GPU also changes the batch size and the learning rate. You are probably using a learning rate / batch size combination that does not work well for fine-tuning ResNet.", "votes": null } ], "comments": [ { "id": 762632, "author_name": "mgorner", "author_url": "", "post_date": "03/03/2020 17:01:45", "content": "

In this Notebook, switching between TPU and GPU also changes the batch size and the learning rate. You are probably using a learning rate / batch size combination that does not work well for fine-tuning ResNet.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "762203": "I changed the model to resnet50 based on \nhttps://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu\nwhen train the val acc is only 0.03 at every epochs ,but changed the kernel to gpu,the val acc seems ok,why?\n\n```\nwith strategy.scope():\n pretrained_model = tf.keras.applications.resnet50.ResNet50(weights='imagenet', include_top=False ,input_shape=[*IMAGE_SIZE, 3])\n pretrained_model.trainable = True # tramsfer learning\n #pretrained_model.trainable = True\n model = tf.keras.Sequential([\n pretrained_model,\n #tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])\n\n\nmodel.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n)\nmodel.summary()\n```\nthis gpu\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252031%2F66c54c1c4bba08c9b98069f82c657b30%2FQQ20200303183319.jpg?generation=1583231676656167&alt=media)\nthis tpu ,only 0.03 after 5 epoches![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252031%2F31e9cf5ad3cee153464534826d3c2a98%2FQQ.jpg?generation=1583231923943040&alt=media)", "762632": "In this Notebook, switching between TPU and GPU also changes the batch size and the learning rate. You are probably using a learning rate / batch size combination that does not work well for fine-tuning ResNet." }, "source": "meta" }