Hello guys. I'm not sure this has been discussed or not, but I want to share this knowledge that I found out.
\nI built a model using pretrained EfficientNetB4(trainable = True) and trained it for 100 epochs and it got me 90% in validation and 88% in test accuracy. I added more data augmentations and more regularizations, but validation and accuracy did not increase even I increased epochs.
\nI wanted to find out how to improve validation and test result, so I read some codes uploaded by others.
\nAnd from this one, https://www.kaggle.com/nileshsuthar/petal-to-the-metals-all-cnn-models, I found only one difference between this code and mine,
!pip install efficientnet\nimport efficientnet.tfkeras as efficientnet\n...\npretrained_model = efficientnet.EfficientNetB0(...)\nthat he used efficientnet from efficientnet.tfkeras instead from tf.keras.applications
# this is what I used in my code\npretrained_model = tf.keras.applications.EfficientNetB0(\nBut this difference made the test result about 95% accuracy.
\nI googled to find out why and, of course, stackoverflow has the answer
\nhttps://stackoverflow.com/questions/64030221/efficientnet-tfkeras-vs-tf-keras-applications-efficientnet
The main reason is that tf.keras.applications.EfficientNet contains rescaling layer at the very start of the model, which does the normalization of image by dividing 255.
\nBut we already did normalization when loading images. So, using tf.keras.applications.EfficientNet causes normalizing images twice, which makes the value very very small.
\nThat's why accuracy won't increase!!!
Maybe you guys already know this, but I'm very happy that I solved this problem, so I wanted to share this for someone who might have the same struggle with me
\nIn short,
\ntf.keras.EfficientNet has rescaling layer at the front, making values of images very very small if you already normalized images by dividing 255.efficientnet.tfkeras.EfficientNet or erase normalization when loading images and use tf.keras.applications.EfficientNet. I think it’s very insightful and helpful for anyone who wants to use EfficientNet models in this competition. It’s a good example of how small details can make a big difference in performance.
", "rawMarkdown": "I think it’s very insightful and helpful for anyone who wants to use EfficientNet models in this competition. It’s a good example of how small details can make a big difference in performance. ", "votes": 2 }, { "id": 1714650, "postDate": "2022-03-07T07:48:54.760Z", "content": "Hello guys. I'm not sure this has been discussed or not, but I want to share this knowledge that I found out.
\nI built a model using pretrained EfficientNetB4(trainable = True) and trained it for 100 epochs and it got me 90% in validation and 88% in test accuracy. I added more data augmentations and more regularizations, but validation and accuracy did not increase even I increased epochs.
\nI wanted to find out how to improve validation and test result, so I read some codes uploaded by others.
\nAnd from this one, https://www.kaggle.com/nileshsuthar/petal-to-the-metals-all-cnn-models, I found only one difference between this code and mine,
!pip install efficientnet\nimport efficientnet.tfkeras as efficientnet\n...\npretrained_model = efficientnet.EfficientNetB0(...)\nthat he used efficientnet from efficientnet.tfkeras instead from tf.keras.applications
# this is what I used in my code\npretrained_model = tf.keras.applications.EfficientNetB0(\nBut this difference made the test result about 95% accuracy.
\nI googled to find out why and, of course, stackoverflow has the answer
\nhttps://stackoverflow.com/questions/64030221/efficientnet-tfkeras-vs-tf-keras-applications-efficientnet
The main reason is that tf.keras.applications.EfficientNet contains rescaling layer at the very start of the model, which does the normalization of image by dividing 255.
\nBut we already did normalization when loading images. So, using tf.keras.applications.EfficientNet causes normalizing images twice, which makes the value very very small.
\nThat's why accuracy won't increase!!!
Maybe you guys already know this, but I'm very happy that I solved this problem, so I wanted to share this for someone who might have the same struggle with me
\nIn short,
\ntf.keras.EfficientNet has rescaling layer at the front, making values of images very very small if you already normalized images by dividing 255.efficientnet.tfkeras.EfficientNet or erase normalization when loading images and use tf.keras.applications.EfficientNet. thanks.A valuable experience!
", "rawMarkdown": "thanks.A valuable experience!" }, { "id": 1720455, "postDate": "2022-03-12T20:05:31.543Z", "content": "Thanks for sharing this. Very helpful.
", "rawMarkdown": "Thanks for sharing this. Very helpful." } ], "comments": [ { "id": 2221317, "author_name": "Yue Sun", "author_url": "", "post_date": "2023-04-14T06:23:26.300000", "content": "I think it’s very insightful and helpful for anyone who wants to use EfficientNet models in this competition. It’s a good example of how small details can make a big difference in performance.
", "votes": 2, "replies": [] }, { "id": 2724726, "author_name": "water dingdong", "author_url": "", "post_date": "2024-03-31T04:29:19.640000", "content": "thanks.A valuable experience!
", "votes": 0, "replies": [] }, { "id": 1720455, "author_name": "fikret", "author_url": "", "post_date": "2022-03-12T20:05:31.543000", "content": "Thanks for sharing this. Very helpful.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2221317": "I think it’s very insightful and helpful for anyone who wants to use EfficientNet models in this competition. It’s a good example of how small details can make a big difference in performance. ", "1714650": "Hello guys. I'm not sure this has been discussed or not, but I want to share this knowledge that I found out. \n\nI built a model using pretrained EfficientNetB4(trainable = True) and trained it for 100 epochs and it got me 90% in validation and 88% in test accuracy. I added more data augmentations and more regularizations, but validation and accuracy did not increase even I increased epochs. \n\nI wanted to find out how to improve validation and test result, so I read some codes uploaded by others. \nAnd from this one, [https://www.kaggle.com/nileshsuthar/petal-to-the-metals-all-cnn-models](url), I found only one difference between this code and mine, \n\n```\n!pip install efficientnet\nimport efficientnet.tfkeras as efficientnet\n...\npretrained_model = efficientnet.EfficientNetB0(...)\n```\n\nthat he used efficientnet from `efficientnet.tfkeras` instead from `tf.keras.applications`\n\n```\n# this is what I used in my code\npretrained_model = tf.keras.applications.EfficientNetB0(\n```\n\nBut this difference made the test result about 95% accuracy. \n\nI googled to find out why and, of course, stackoverflow has the answer\n[https://stackoverflow.com/questions/64030221/efficientnet-tfkeras-vs-tf-keras-applications-efficientnet](url)\n\nThe main reason is that `tf.keras.applications.EfficientNet` contains `rescaling layer` at the very start of the model, which does the normalization of image by dividing 255. \nBut we already did normalization when loading images. So, using `tf.keras.applications.EfficientNet` causes normalizing images twice, which makes the value very very small. \nThat's why accuracy won't increase!!!\n\nMaybe you guys already know this, but I'm very happy that I solved this problem, so I wanted to share this for someone who might have the same struggle with me\n\nIn short,\n- `tf.keras.EfficientNet` has `rescaling layer` at the front, making values of images very very small if you already normalized images by dividing 255.\n- To avoid this problem and increase the result accuracy, you should either use `efficientnet.tfkeras.EfficientNet` or erase normalization when loading images and use `tf.keras.applications.EfficientNet`. ", "2724726": "thanks.A valuable experience!", "1720455": "Thanks for sharing this. Very helpful." } }