Google has released their new EfficientNetV2, with code and pretrained weights available.
\nRoss Wightman has also made available the model and weights to the timm library.
As the competition only started last week - one thing I've noticed thus far in my experiments, larger and more recent sota models like NFNet seems to perform rather well on CV and LB without augmentations.
", "messages": [ { "id": 1309499, "postDate": "2021-05-16T03:32:55.433Z", "content": "Google has released their new EfficientNetV2, with code and pretrained weights available.
\nRoss Wightman has also made available the model and weights to the timm library.
As the competition only started last week - one thing I've noticed thus far in my experiments, larger and more recent sota models like NFNet seems to perform rather well on CV and LB without augmentations.
", "rawMarkdown": "Google has released their new [EfficientNetV2](https://arxiv.org/abs/2104.00298), with code and pretrained weights [available](https://github.com/google/automl/tree/master/efficientnetv2).\n\nRoss Wightman has also made available the model and weights to the `timm` [library](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/efficientnet.py).\n\nAs the competition only started last week - one thing I've noticed thus far in my experiments, larger and more recent sota models like NFNet seems to perform rather well on CV and LB without augmentations.", "votes": 17 }, { "id": 1309507, "postDate": "2021-05-16T03:50:26.493Z", "content": "For me, smaller models are working better than larger models and are much easier to train. I have tried efficientnet v2 b0 but still can not beat efficientnet_b0.
", "rawMarkdown": "For me, smaller models are working better than larger models and are much easier to train. I have tried efficientnet v2 b0 but still can not beat efficientnet_b0.", "votes": 5, "replies": [ { "id": 1309530, "postDate": "2021-05-16T04:23:39.787Z", "content": "@snaker Interesting, thanks for the note. I'm facing worse performance on smaller models of the same type/class.
", "rawMarkdown": "@snaker Interesting, thanks for the note. I'm facing worse performance on smaller models of the same type/class." }, { "id": 1309656, "postDate": "2021-05-16T07:23:25.530Z", "content": "Yes. I guess its not bigger models, but ratio between image size and depth of model. Like for image size 256*256 b0, b1 and b2 should work.
", "rawMarkdown": "Yes. I guess its not bigger models, but ratio between image size and depth of model. Like for image size 256*256 b0, b1 and b2 should work. ", "votes": 8 } ] }, { "id": 1313724, "postDate": "2021-05-18T17:17:38.450Z", "content": "@khoongweihao not only Nfnets perform better than Efficientnet , but they don't face memory overflow issues in contrast to efficientnets with almost same number of parameters and they are easily adaptable to larger batch and image sizes .
", "rawMarkdown": "@khoongweihao not only Nfnets perform better than Efficientnet , but they don't face memory overflow issues in contrast to efficientnets with almost same number of parameters and they are easily adaptable to larger batch and image sizes .", "votes": 1, "replies": [ { "id": 1314654, "postDate": "2021-05-19T09:50:02.313Z", "content": "@sayedathar11Interesting, thanks for pointing that out
", "rawMarkdown": "@sayedathar11Interesting, thanks for pointing that out" }, { "id": 1314683, "postDate": "2021-05-19T10:12:00.790Z", "content": "EfficientNetV1 was made primarily for TPU and in particular TF + TPU.
\nBut it seems they have fix it a bit on GPU for V2 with TensorRT.
", "rawMarkdown": "EfficientNetV1 was made primarily for TPU and in particular TF + TPU.\n\nBut it seems they have [fix it](https://twitter.com/tanmingxing/status/1393009264541634566) a bit on GPU for V2 with TensorRT. ", "votes": 2 } ] } ], "comments": [ { "id": 1309507, "author_name": "Chenglu", "author_url": "", "post_date": "2021-05-16T03:50:26.493000", "content": "For me, smaller models are working better than larger models and are much easier to train. I have tried efficientnet v2 b0 but still can not beat efficientnet_b0.
", "votes": 5, "replies": [ { "id": 1309530, "author_name": "Wei Hao Khoong", "author_url": "", "post_date": "2021-05-16T04:23:39.787000", "content": "@snaker Interesting, thanks for the note. I'm facing worse performance on smaller models of the same type/class.
", "votes": 0, "replies": [] }, { "id": 1309656, "author_name": "Salman", "author_url": "", "post_date": "2021-05-16T07:23:25.530000", "content": "Yes. I guess its not bigger models, but ratio between image size and depth of model. Like for image size 256*256 b0, b1 and b2 should work.
", "votes": 8, "replies": [] } ] }, { "id": 1313724, "author_name": "Athar Sayed", "author_url": "", "post_date": "2021-05-18T17:17:38.450000", "content": "@khoongweihao not only Nfnets perform better than Efficientnet , but they don't face memory overflow issues in contrast to efficientnets with almost same number of parameters and they are easily adaptable to larger batch and image sizes .
", "votes": 1, "replies": [ { "id": 1314654, "author_name": "Wei Hao Khoong", "author_url": "", "post_date": "2021-05-19T09:50:02.313000", "content": "@sayedathar11Interesting, thanks for pointing that out
", "votes": 0, "replies": [] }, { "id": 1314683, "author_name": "Serigne ", "author_url": "", "post_date": "2021-05-19T10:12:00.790000", "content": "EfficientNetV1 was made primarily for TPU and in particular TF + TPU.
\nBut it seems they have fix it a bit on GPU for V2 with TensorRT.
", "votes": 2, "replies": [] } ] } ], "raw_markdown_by_id": { "1309499": "Google has released their new [EfficientNetV2](https://arxiv.org/abs/2104.00298), with code and pretrained weights [available](https://github.com/google/automl/tree/master/efficientnetv2).\n\nRoss Wightman has also made available the model and weights to the `timm` [library](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/efficientnet.py).\n\nAs the competition only started last week - one thing I've noticed thus far in my experiments, larger and more recent sota models like NFNet seems to perform rather well on CV and LB without augmentations.", "1309507": "For me, smaller models are working better than larger models and are much easier to train. I have tried efficientnet v2 b0 but still can not beat efficientnet_b0.", "1313724": "@khoongweihao not only Nfnets perform better than Efficientnet , but they don't face memory overflow issues in contrast to efficientnets with almost same number of parameters and they are easily adaptable to larger batch and image sizes ." } }