![\"\"](\"https://1.bp.blogspot.com/-NkAlO48dW_k/XsWz-8CLuLI/AAAAAAAAF-8/xbPH2061a8IT4ipiRqwP_6Mutmu3M1IjACLcBGAsYHQ/s1600/image1.gif\")
\nTransfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale up pre-training, and propose a simple recipe that we call Big Transfer (BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over 20 datasets. BiT performs well across a surprisingly wide range of data regimes -- from 1 example per class to 1M total examples. BiT achieves 87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3% on the 19 task Visual Task Adaptation Benchmark (VTAB). On small datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class, and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed analysis of the main components that lead to high transfer performance.\n
Blog : https://ai.googleblog.com/\nCode w/ models: https://github.com/google-research/big_transfer
", "messages": [ { "id": "858820", "postDate": "05/23/2020 20:51:14", "content": "\nTransfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale up pre-training, and propose a simple recipe that we call Big Transfer (BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over 20 datasets. BiT performs well across a surprisingly wide range of data regimes -- from 1 example per class to 1M total examples. BiT achieves 87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3% on the 19 task Visual Task Adaptation Benchmark (VTAB). On small datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class, and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed analysis of the main components that lead to high transfer performance.\n
Blog : https://ai.googleblog.com/\nCode w/ models: https://github.com/google-research/big_transfer
", "rawMarkdown": "# [Big Transfer (BiT): General Visual Representation Learning](https://arxiv.org/pdf/1912.11370.pdf)\n```\nTransfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale up pre-training, and propose a simple recipe that we call Big Transfer (BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over 20 datasets. BiT performs well across a surprisingly wide range of data regimes -- from 1 example per class to 1M total examples. BiT achieves 87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3% on the 19 task Visual Task Adaptation Benchmark (VTAB). On small datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class, and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed analysis of the main components that lead to high transfer performance.\n```\n\n\n\nBlog : https://ai.googleblog.com/\nCode w/ models: https://github.com/google-research/big_transfer", "votes": null }, { "id": "883099", "postDate": "06/12/2020 11:17:26", "content": "Have you used big_transfer in this competition? If so, can you give the kernel link?
", "rawMarkdown": "Have you used big_transfer in this competition? If so, can you give the kernel link?", "votes": null } ], "comments": [ { "id": 883099, "author_name": "tasnimnishatislam", "author_url": "", "post_date": "06/12/2020 11:17:26", "content": "Have you used big_transfer in this competition? If so, can you give the kernel link?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "858820": "# [Big Transfer (BiT): General Visual Representation Learning](https://arxiv.org/pdf/1912.11370.pdf)\n```\nTransfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale up pre-training, and propose a simple recipe that we call Big Transfer (BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over 20 datasets. BiT performs well across a surprisingly wide range of data regimes -- from 1 example per class to 1M total examples. BiT achieves 87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3% on the 19 task Visual Task Adaptation Benchmark (VTAB). On small datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class, and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed analysis of the main components that lead to high transfer performance.\n```\n\n\n\nBlog : https://ai.googleblog.com/\nCode w/ models: https://github.com/google-research/big_transfer", "883099": "Have you used big_transfer in this competition? If so, can you give the kernel link?" }, "source": "meta" }