Hello,
\nDeepmind releases a new State-Of-The-Art Image Classification model — NFNets. NFNets are:
\nFor
\n\nAnd for more detail how to apply in PytorchI recommend this discussion
", "messages": [ { "id": 1200645, "postDate": "2021-02-14T20:18:27.080Z", "content": "Hello,
\nDeepmind releases a new State-Of-The-Art Image Classification model — NFNets. NFNets are:
\nFor
\n\nAnd for more detail how to apply in PytorchI recommend this discussion
", "rawMarkdown": "Hello,\n\nDeepmind releases a new State-Of-The-Art Image Classification model — NFNets. NFNets are:\n- SOTA on ImageNet (86.5% top-1 w/o extra data)\n- Up to 8.7x faster to train than EfficientNets to a given accuracy\n- Normalizer-free (no BatchNorm!)\n\nFor \n- [Paper](http://dpmd.ai/06171)\n- [Code](http://dpmd.ai/nfnets)\n\nAnd for more detail how to apply in **Pytorch**I recommend this [discussion](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/218907)", "votes": 13 }, { "id": 1201277, "postDate": "2021-02-15T09:16:35.703Z", "content": "Although you can find NFNets in timm, the trick in this paper is the Adaptive Gradient Clipping. AFAIK, the example code for AGC hasn't been released yet
Edit: maybe it's there now :) https://github.com/deepmind/deepmind-research/blob/68a1754d293e04d278dfb649b2d59512f32351a7/nfnets/optim.py#L476-L491
", "rawMarkdown": "Although you can find NFNets in `timm`, the trick in this paper is the Adaptive Gradient Clipping. AFAIK, the example code for AGC hasn't been released yet\n\nEdit: maybe it's there now :) https://github.com/deepmind/deepmind-research/blob/68a1754d293e04d278dfb649b2d59512f32351a7/nfnets/optim.py#L476-L491 ", "votes": 3, "replies": [ { "id": 1204518, "postDate": "2021-02-16T07:13:56.097Z", "content": "This is why my model wasn't performing good on NFNet-F1.
\nThanks for clarifying!
One of the things I have been wondering for a while is lots of recent papers have shown that the performance on ImageNet doesn't necessarily convert to additional score on transfer tasks. There is some claim to better transfer in these models, but will take a bit of time to see exactly how it works.
\nThese are also completely massive models. Much bigger than effnet and resnet models.
", "rawMarkdown": "One of the things I have been wondering for a while is lots of recent papers have shown that the performance on ImageNet doesn't necessarily convert to additional score on transfer tasks. There is some claim to better transfer in these models, but will take a bit of time to see exactly how it works.\n\nThese are also completely massive models. Much bigger than effnet and resnet models. ", "votes": 2, "replies": [ { "id": 1204144, "postDate": "2021-02-15T22:04:35.190Z", "content": "Thanks for your feedback, and I totally agree with you.
\nHere is a discussion to apply on Pytorch NFNet-F* models released in pytorch-image-models\n
That feeling when you are a Keras user and EfficientNets have been added not so long ago…
", "rawMarkdown": "That feeling when you are a Keras user and EfficientNets have been added not so long ago...", "votes": 2, "replies": [ { "id": 1204147, "postDate": "2021-02-15T22:06:18.307Z", "content": "It's Deepmind 😑😑
", "rawMarkdown": "It's Deepmind 😑😑", "votes": 1 } ] }, { "id": 1209962, "postDate": "2021-02-19T06:06:26.567Z", "content": "It's a very interesting paper. Talks a lot about Batch normalization.
\nhttps://www.linkedin.com/posts/amritpal-singh-001_paperabrsummary-computervision-google-activity-6767776421348741120-SFos/
\nWould love to hear your opinions.
Thanks for the post! A lot of people are still very less familiar with this new development.
\nBy the way, I have made a Notebook in this competition training NFNet-F1, it can be helpful to get started!
\nHere's the Link.
Although you can find NFNets in timm, the trick in this paper is the Adaptive Gradient Clipping. AFAIK, the example code for AGC hasn't been released yet
Edit: maybe it's there now :) https://github.com/deepmind/deepmind-research/blob/68a1754d293e04d278dfb649b2d59512f32351a7/nfnets/optim.py#L476-L491
", "votes": 3, "replies": [ { "id": 1204518, "author_name": "Tanay Mehta", "author_url": "", "post_date": "2021-02-16T07:13:56.097000", "content": "This is why my model wasn't performing good on NFNet-F1.
\nThanks for clarifying!
One of the things I have been wondering for a while is lots of recent papers have shown that the performance on ImageNet doesn't necessarily convert to additional score on transfer tasks. There is some claim to better transfer in these models, but will take a bit of time to see exactly how it works.
\nThese are also completely massive models. Much bigger than effnet and resnet models.
", "votes": 2, "replies": [ { "id": 1204144, "author_name": "Miloud Belarebia", "author_url": "", "post_date": "2021-02-15T22:04:35.190000", "content": "Thanks for your feedback, and I totally agree with you.
\nHere is a discussion to apply on Pytorch NFNet-F* models released in pytorch-image-models\n
That feeling when you are a Keras user and EfficientNets have been added not so long ago…
", "votes": 2, "replies": [ { "id": 1204147, "author_name": "Miloud Belarebia", "author_url": "", "post_date": "2021-02-15T22:06:18.307000", "content": "It's Deepmind 😑😑
", "votes": 1, "replies": [] } ] }, { "id": 1209962, "author_name": "Dr. Amritpal Singh", "author_url": "", "post_date": "2021-02-19T06:06:26.567000", "content": "It's a very interesting paper. Talks a lot about Batch normalization.
\nhttps://www.linkedin.com/posts/amritpal-singh-001_paperabrsummary-computervision-google-activity-6767776421348741120-SFos/
\nWould love to hear your opinions.
Thanks for the post! A lot of people are still very less familiar with this new development.
\nBy the way, I have made a Notebook in this competition training NFNet-F1, it can be helpful to get started!
\nHere's the Link.