![\"EfficientNet](\"https://raw.githubusercontent.com/tensorflow/tpu/master/models/official/efficientnet/g3doc/params.png\")
All credits are due to https://github.com/qubvel/efficientnet (kudos @pavel92)\nGoogle blog post : https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html
\n\nEfficientNet is the very promising approach to become default architecture in Computer Vision. Here, if you would like to use it in Keras, you can use my public utility script and weight dataset (B0-B5 notop).\nPlease refer to R.Tatman below about how to import kernel script.
\n\nIn short, just click at \"File >> Add utility script\" and then you can use \n```\nfrom efficientnet import *
\n\nbackbone = EfficientNetB4(weights=None,\n include_top=False,\n input_shape=input_shape)
\n\ninput_tensor = backbone.input\nbackbone.load_weigts(...) # see dataset below\n```\nThings to note : in APTOS, I can use only BATCH_SIZE=16 wtih B4 .
\n\nUtility Script\nhttps://www.kaggle.com/ratthachat/efficientnet
\n\nHow to import Scripts by Rachael Tatman\nhttps://www.kaggle.com/product-feedback/91185
\n\nPublic Weights Dataset\nhttps://www.kaggle.com/ratthachat/efficientnet-keras-weights-b0b5/
\n\n
Special thanks to this kernel for introducing me this wonderful repository
\n\nEDIT: The following are from the main code:
\n\n> Re-Implementation of EfficientNet for Keras\n Reference:\n https://arxiv.org/abs/1807.11626\n Args:\n input_shape: optional, if None default_input_shape is used\n EfficientNetB0 - (224, 224, 3)\n EfficientNetB1 - (240, 240, 3)\n EfficientNetB2 - (260, 260, 3)\n EfficientNetB3 - (300, 300, 3)\n EfficientNetB4 - (380, 380, 3)\n EfficientNetB5 - (456, 456, 3)\n EfficientNetB6 - (528, 528, 3)\n EfficientNetB7 - (600, 600, 3)\n input_tensor : optional, if None default_input_tensor is used\n include_top : whether to include the fully-connected\n layer at the top of the network.\n weights: one of None (random initialization),\n imagenet (pre-training on ImageNet).\n classes: optional number of classes to classify images\n into, only to be specified if include_top is True, and\n if no weights argument is specified.\n pooling: optional [None, 'avg', 'max'], if include_top=False\n add global pooling on top of the network\n - avg: GlobalAveragePooling2D \n - max: GlobalMaxPooling2D \n Returns:\n A Keras model instance.
All credits are due to https://github.com/qubvel/efficientnet (kudos @pavel92)\nGoogle blog post : https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html
\n\nEfficientNet is the very promising approach to become default architecture in Computer Vision. Here, if you would like to use it in Keras, you can use my public utility script and weight dataset (B0-B5 notop).\nPlease refer to R.Tatman below about how to import kernel script.
\n\nIn short, just click at \"File >> Add utility script\" and then you can use \n```\nfrom efficientnet import *
\n\nbackbone = EfficientNetB4(weights=None,\n include_top=False,\n input_shape=input_shape)
\n\ninput_tensor = backbone.input\nbackbone.load_weigts(...) # see dataset below\n```\nThings to note : in APTOS, I can use only BATCH_SIZE=16 wtih B4 .
\n\nUtility Script\nhttps://www.kaggle.com/ratthachat/efficientnet
\n\nHow to import Scripts by Rachael Tatman\nhttps://www.kaggle.com/product-feedback/91185
\n\nPublic Weights Dataset\nhttps://www.kaggle.com/ratthachat/efficientnet-keras-weights-b0b5/
\n\n
Special thanks to this kernel for introducing me this wonderful repository
\n\nEDIT: The following are from the main code:
\n\n> Re-Implementation of EfficientNet for Keras\n Reference:\n https://arxiv.org/abs/1807.11626\n Args:\n input_shape: optional, if None default_input_shape is used\n EfficientNetB0 - (224, 224, 3)\n EfficientNetB1 - (240, 240, 3)\n EfficientNetB2 - (260, 260, 3)\n EfficientNetB3 - (300, 300, 3)\n EfficientNetB4 - (380, 380, 3)\n EfficientNetB5 - (456, 456, 3)\n EfficientNetB6 - (528, 528, 3)\n EfficientNetB7 - (600, 600, 3)\n input_tensor : optional, if None default_input_tensor is used\n include_top : whether to include the fully-connected\n layer at the top of the network.\n weights: one of None (random initialization),\n imagenet (pre-training on ImageNet).\n classes: optional number of classes to classify images\n into, only to be specified if include_top is True, and\n if no weights argument is specified.\n pooling: optional [None, 'avg', 'max'], if include_top=False\n add global pooling on top of the network\n - avg: GlobalAveragePooling2D \n - max: GlobalMaxPooling2D \n Returns:\n A Keras model instance.
Thanks @ratthachat
\n\nHere's the link to the original paper EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
\n\nThe Abstract:
\n\n\n", "rawMarkdown": "Thanks @ratthachat \n\nHere's the link to the original paper [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)\n\nThe Abstract:\n>Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. Based on this observation, we propose a new scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient. We demonstrate the effectiveness of this method on scaling up MobileNets and ResNet. To go even further, we use neural architecture search to design a new baseline network and scale it up to obtain a family of models, called EfficientNets, which achieve much better accuracy and efficiency than previous ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.4% top-1 / 97.1% top-5 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet. Our EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters.", "votes": null }, { "id": "579033", "postDate": "07/18/2019 12:15:56", "content": "Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. Based on this observation, we propose a new scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient. We demonstrate the effectiveness of this method on scaling up MobileNets and ResNet. To go even further, we use neural architecture search to design a new baseline network and scale it up to obtain a family of models, called EfficientNets, which achieve much better accuracy and efficiency than previous ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.4% top-1 / 97.1% top-5 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet. Our EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters.
\n
I have created a kernel which you can use EfficientNet B5 now\nEfficientNet
", "rawMarkdown": "I have created a kernel which you can use EfficientNet B5 now\n[EfficientNet](https://www.kaggle.com/runninglion/aptos-2019-efficientnet-keras-model-weights)", "votes": null }, { "id": "579795", "postDate": "07/19/2019 08:13:36", "content": "Great work! Thank you!
", "rawMarkdown": "Great work! Thank you!", "votes": null }, { "id": "580379", "postDate": "07/20/2019 04:44:37", "content": "Thanks for sharing! This is really helpful!
", "rawMarkdown": "Thanks for sharing! This is really helpful!", "votes": null }, { "id": "581040", "postDate": "07/21/2019 09:47:51", "content": "HI,Have you tried using MIXUP, before I used cv on B3 to perform better than not, but it didn't perform well on PB.
", "rawMarkdown": "HI,Have you tried using MIXUP, before I used cv on B3 to perform better than not, but it didn't perform well on PB.", "votes": null }, { "id": "581875", "postDate": "07/22/2019 13:39:05", "content": "@icebergi I faces the same problem as you. I couldn’t make MixUp to work with the model yet.
", "rawMarkdown": "icebergi I faces the same problem as you. I couldn’t make MixUp to work with the model yet.", "votes": null }, { "id": "584277", "postDate": "07/25/2019 16:43:44", "content": "Awesome! Thank for sharing this!
", "rawMarkdown": "Awesome! Thank for sharing this!", "votes": null }, { "id": "608466", "postDate": "08/26/2019 19:28:30", "content": "I used efficeintnet , but it is not completing all epoch = 200 .\nIt stopped very early sometime epoch 14 , 54 . \nAny idea ?\nI am not using EarlyStop callback. Removed it and still the same issue.\nThanks,
", "rawMarkdown": "I used efficeintnet , but it is not completing all epoch = 200 .\nIt stopped very early sometime epoch 14 , 54 . \nAny idea ?\nI am not using EarlyStop callback. Removed it and still the same issue.\nThanks,", "votes": null }, { "id": "608626", "postDate": "08/27/2019 03:14:56", "content": "I read Rachael Tatman post and did the following:
\n\n\n\n\nI \"Copy and edit\" the utility script in this post.\n ran it, made a commit, then clicked \"Make utility script\"\n Navigated to another notebook kernel then clicked \"Add utility script\".\n No utility scripts available.
\n
Why are no utility scripts available? Thank you for all the help!
", "rawMarkdown": "I read Rachael Tatman post and did the following:\n\n>I \"Copy and edit\" the utility script in this post.\n>ran it, made a commit, then clicked \"Make utility script\"\n>Navigated to another notebook kernel then clicked \"Add utility script\".\n>No utility scripts available.\n\nWhy are no utility scripts available? Thank you for all the help!", "votes": null }, { "id": "608848", "postDate": "08/27/2019 07:56:17", "content": "@shu244 One possibility is that your script has \"committing error\" .. Either there is really error in the script, or because the \"kernel error issue\", that everybody has today (27/8/2019)
", "rawMarkdown": "shu244 One possibility is that your script has \"committing error\" .. Either there is really error in the script, or because the \"kernel error issue\", that everybody has today (27/8/2019)", "votes": null }, { "id": "609736", "postDate": "08/28/2019 03:56:24", "content": "Is there a simple way for EfficientNetB4(weights=None, include_top=False, input_shape=input_shape) to return a tf.keras model instead of a keras model? Thanks in advance for the help!
", "rawMarkdown": "Is there a simple way for EfficientNetB4(weights=None, include_top=False, input_shape=input_shape) to return a tf.keras model instead of a keras model? Thanks in advance for the help!", "votes": null }, { "id": "609957", "postDate": "08/28/2019 09:04:06", "content": "@shu244 please follow
\n\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/104084#latest-607455
", "rawMarkdown": "shu244 please follow\n\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/104084#latest-607455", "votes": null }, { "id": "610230", "postDate": "08/28/2019 15:05:13", "content": "Thank you! I read the article and the comments and they helped tremendously!
", "rawMarkdown": "Thank you! I read the article and the comments and they helped tremendously!", "votes": null } ], "comments": [ { "id": 577829, "author_name": "rishabhiitbhu", "author_url": "", "post_date": "07/17/2019 06:01:44", "content": "Thanks @ratthachat
\n\nHere's the link to the original paper EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
\n\nThe Abstract:
\n\n\n", "votes": null, "replies": [] }, { "id": 579033, "author_name": "runninglion", "author_url": "", "post_date": "07/18/2019 12:15:56", "content": "Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. Based on this observation, we propose a new scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient. We demonstrate the effectiveness of this method on scaling up MobileNets and ResNet. To go even further, we use neural architecture search to design a new baseline network and scale it up to obtain a family of models, called EfficientNets, which achieve much better accuracy and efficiency than previous ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.4% top-1 / 97.1% top-5 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet. Our EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters.
\n
I have created a kernel which you can use EfficientNet B5 now\nEfficientNet
", "votes": null, "replies": [] }, { "id": 579795, "author_name": "jiadakong", "author_url": "", "post_date": "07/19/2019 08:13:36", "content": "Great work! Thank you!
", "votes": null, "replies": [] }, { "id": 580379, "author_name": "higepon", "author_url": "", "post_date": "07/20/2019 04:44:37", "content": "Thanks for sharing! This is really helpful!
", "votes": null, "replies": [] }, { "id": 581040, "author_name": "icebergi", "author_url": "", "post_date": "07/21/2019 09:47:51", "content": "HI,Have you tried using MIXUP, before I used cv on B3 to perform better than not, but it didn't perform well on PB.
", "votes": null, "replies": [ { "id": 581875, "author_name": "ratthachat", "author_url": "", "post_date": "07/22/2019 13:39:05", "content": "@icebergi I faces the same problem as you. I couldn’t make MixUp to work with the model yet.
", "votes": null, "replies": [] } ] }, { "id": 584277, "author_name": "carlolepelaars", "author_url": "", "post_date": "07/25/2019 16:43:44", "content": "Awesome! Thank for sharing this!
", "votes": null, "replies": [] }, { "id": 608466, "author_name": "rajnishe", "author_url": "", "post_date": "08/26/2019 19:28:30", "content": "I used efficeintnet , but it is not completing all epoch = 200 .\nIt stopped very early sometime epoch 14 , 54 . \nAny idea ?\nI am not using EarlyStop callback. Removed it and still the same issue.\nThanks,
", "votes": null, "replies": [] }, { "id": 608626, "author_name": "shu244", "author_url": "", "post_date": "08/27/2019 03:14:56", "content": "I read Rachael Tatman post and did the following:
\n\n\n\n\nI \"Copy and edit\" the utility script in this post.\n ran it, made a commit, then clicked \"Make utility script\"\n Navigated to another notebook kernel then clicked \"Add utility script\".\n No utility scripts available.
\n
Why are no utility scripts available? Thank you for all the help!
", "votes": null, "replies": [ { "id": 608848, "author_name": "ratthachat", "author_url": "", "post_date": "08/27/2019 07:56:17", "content": "@shu244 One possibility is that your script has \"committing error\" .. Either there is really error in the script, or because the \"kernel error issue\", that everybody has today (27/8/2019)
", "votes": null, "replies": [] } ] }, { "id": 609736, "author_name": "shu244", "author_url": "", "post_date": "08/28/2019 03:56:24", "content": "Is there a simple way for EfficientNetB4(weights=None, include_top=False, input_shape=input_shape) to return a tf.keras model instead of a keras model? Thanks in advance for the help!
", "votes": null, "replies": [ { "id": 609957, "author_name": "ratthachat", "author_url": "", "post_date": "08/28/2019 09:04:06", "content": "@shu244 please follow
\n\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/104084#latest-607455
", "votes": null, "replies": [] }, { "id": 610230, "author_name": "shu244", "author_url": "", "post_date": "08/28/2019 15:05:13", "content": "Thank you! I read the article and the comments and they helped tremendously!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "577775": "**All credits** are due to https://github.com/qubvel/efficientnet (kudos @pavel92)\nGoogle blog post : https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html\n\nEfficientNet is the very promising approach to become default architecture in Computer Vision. Here, if you would like to use it in Keras, you can use my public utility script and weight dataset (B0-B5 notop).\nPlease refer to R.Tatman below about how to import kernel script. \n\nIn short, just click at \"File >> Add utility script\" and then you can use \n```\nfrom efficientnet import *\n\n\nbackbone = EfficientNetB4(weights=None,\n include_top=False,\n input_shape=input_shape)\n\ninput_tensor = backbone.input\nbackbone.load_weigts(...) # see dataset below\n```\nThings to note : in APTOS, I can use only BATCH_SIZE=16 wtih B4 .\n\n**Utility Script**\nhttps://www.kaggle.com/ratthachat/efficientnet\n\n**How to import Scripts by Rachael Tatman**\nhttps://www.kaggle.com/product-feedback/91185\n\n**Public Weights Dataset**\nhttps://www.kaggle.com/ratthachat/efficientnet-keras-weights-b0b5/\n\n\n\n**Special thanks to [this kernel](https://www.kaggle.com/meaninglesslives/unet-with-efficientnet-encoder-in-keras)** for introducing me this wonderful repository\n\n**EDIT:** The following are from the main code:\n\n> Re-Implementation of EfficientNet for Keras\n Reference:\n https://arxiv.org/abs/1807.11626\n Args:\n ` input_shape`: optional, if `None` ` default_input_shape` is used\n EfficientNetB0 - (224, 224, 3)\n EfficientNetB1 - (240, 240, 3)\n EfficientNetB2 - (260, 260, 3)\n EfficientNetB3 - (300, 300, 3)\n EfficientNetB4 - (380, 380, 3)\n EfficientNetB5 - (456, 456, 3)\n EfficientNetB6 - (528, 528, 3)\n EfficientNetB7 - (600, 600, 3)\n ` input_tensor` : optional, if ``None`` default_input_tensor is used\n ` include_top` : whether to include the fully-connected\n layer at the top of the network.\n weights: one of `None` (random initialization),\n ` imagenet` (pre-training on ImageNet).\n classes: optional number of classes to classify images\n into, only to be specified if `include_top` is True, and\n if no `weights` argument is specified.\n pooling: optional [None, 'avg', 'max'], if ``include_top=False``\n add global pooling on top of the network\n - avg: ` GlobalAveragePooling2D` \n - max: ` GlobalMaxPooling2D` \n Returns:\n A Keras model instance.", "577829": "Thanks @ratthachat \n\nHere's the link to the original paper [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)\n\nThe Abstract:\n>Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. Based on this observation, we propose a new scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient. We demonstrate the effectiveness of this method on scaling up MobileNets and ResNet. To go even further, we use neural architecture search to design a new baseline network and scale it up to obtain a family of models, called EfficientNets, which achieve much better accuracy and efficiency than previous ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.4% top-1 / 97.1% top-5 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet. Our EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters.", "579033": "I have created a kernel which you can use EfficientNet B5 now\n[EfficientNet](https://www.kaggle.com/runninglion/aptos-2019-efficientnet-keras-model-weights)", "579795": "Great work! Thank you!", "580379": "Thanks for sharing! This is really helpful!", "581040": "HI,Have you tried using MIXUP, before I used cv on B3 to perform better than not, but it didn't perform well on PB.", "581875": "icebergi I faces the same problem as you. I couldn’t make MixUp to work with the model yet.", "584277": "Awesome! Thank for sharing this!", "608466": "I used efficeintnet , but it is not completing all epoch = 200 .\nIt stopped very early sometime epoch 14 , 54 . \nAny idea ?\nI am not using EarlyStop callback. Removed it and still the same issue.\nThanks,", "608626": "I read Rachael Tatman post and did the following:\n\n>I \"Copy and edit\" the utility script in this post.\n>ran it, made a commit, then clicked \"Make utility script\"\n>Navigated to another notebook kernel then clicked \"Add utility script\".\n>No utility scripts available.\n\nWhy are no utility scripts available? Thank you for all the help!", "608848": "shu244 One possibility is that your script has \"committing error\" .. Either there is really error in the script, or because the \"kernel error issue\", that everybody has today (27/8/2019)", "609736": "Is there a simple way for EfficientNetB4(weights=None, include_top=False, input_shape=input_shape) to return a tf.keras model instead of a keras model? Thanks in advance for the help!", "609957": "shu244 please follow\n\nhttps://www.kaggle.com/c/aptos2019-blindness-detection/discussion/104084#latest-607455", "610230": "Thank you! I read the article and the comments and they helped tremendously!" }, "source": "meta" }