{
  "id": 312238,
  "title": "strong meta vision-transformer architechture pretrain support",
  "url": "/competitions/herbarium-2022-fgvc9/discussion/312238",
  "author_name": "Joy",
  "post_date": "2022-03-11T02:37:22.125000",
  "votes": 5,
  "comment_count": 1,
  "views": 0,
  "content": "<h3>MetaFormer</h3>\n<p>We are very happy to meet you here and to tell you about our work(MetaFormer). We use a unified framework with meta information to do fine-grained recognition task. We have achieved SOTA performance on INAT18/19/21 and CUB datasets.</p>\n<h3>Fine-grained Datasets</h3>\n<p>Result on fine-grained datasets with different pre-trained model.</p>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>Pretrain</th>\n<th>CUB</th>\n<th>NABirds</th>\n<th>iNat2017</th>\n<th>iNat2018</th>\n<th>Cars</th>\n<th>Aircraft</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-1k</td>\n<td>89.6</td>\n<td>89.1</td>\n<td>75.7</td>\n<td>79.5</td>\n<td>95.0</td>\n<td>91.2</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-21k</td>\n<td>89.7</td>\n<td>89.5</td>\n<td>75.8</td>\n<td>79.9</td>\n<td>94.6</td>\n<td>91.2</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>iNaturalist 2021</td>\n<td>91.8</td>\n<td>91.5</td>\n<td>78.3</td>\n<td>82.9</td>\n<td>95.1</td>\n<td>87.4</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-1k</td>\n<td>89.7</td>\n<td>89.4</td>\n<td>78.2</td>\n<td>81.9</td>\n<td>94.9</td>\n<td>90.8</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-21k</td>\n<td>91.3</td>\n<td>91.6</td>\n<td>79.4</td>\n<td>83.2</td>\n<td>95.0</td>\n<td>92.6</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>iNaturalist 2021</td>\n<td>92.3</td>\n<td>92.7</td>\n<td>82.0</td>\n<td>87.5</td>\n<td>95.0</td>\n<td>92.5</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-1k</td>\n<td>89.7</td>\n<td>89.7</td>\n<td>79.0</td>\n<td>82.6</td>\n<td>95.0</td>\n<td>92.4</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-21k</td>\n<td>91.8</td>\n<td>92.2</td>\n<td>80.4</td>\n<td>84.3</td>\n<td>95.1</td>\n<td>92.9</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>iNaturalist 2021</td>\n<td>92.9</td>\n<td>93.0</td>\n<td>82.8</td>\n<td>87.7</td>\n<td>95.4</td>\n<td>92.8</td>\n</tr>\n</tbody>\n</table>\n<p>Results in iNaturalist 2019, iNaturalist 2018, and iNaturalist 2021 with meta-information.</p>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>Pretrain</th>\n<th>Meta added</th>\n<th>iNat2017</th>\n<th>iNat2018</th>\n<th>iNat2021</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-1k</td>\n<td>N</td>\n<td>75.7</td>\n<td>79.5</td>\n<td>88.4</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-1k</td>\n<td>Y</td>\n<td>79.8(+4.1)</td>\n<td>85.4(+5.9)</td>\n<td>92.6(+4.2)</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-1k</td>\n<td>N</td>\n<td>78.2</td>\n<td>81.9</td>\n<td>90.2</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-1k</td>\n<td>Y</td>\n<td>81.3(+3.1)</td>\n<td>86.5(+4.6)</td>\n<td>93.4(+3.2)</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-1k</td>\n<td>N</td>\n<td>79.0</td>\n<td>82.6</td>\n<td>89.8</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-1k</td>\n<td>Y</td>\n<td>82.0(+3.0)</td>\n<td>86.8(+4.2)</td>\n<td>93.2(+3.4)</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-21k</td>\n<td>N</td>\n<td>80.4</td>\n<td>84.3</td>\n<td>90.3</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-21k</td>\n<td>Y</td>\n<td>83.4(+3.0)</td>\n<td>88.7(+4.4)</td>\n<td>93.6(+3.3)</td>\n</tr>\n</tbody>\n</table>\n<p>We also provide Imagenet 1k/22k and Inaturalist Pretrain models with various resolution, Welcome to use our pretrain models and codebase.</p>\n<h3>Model zoo</h3>\n<table>\n<thead>\n<tr>\n<th>name</th>\n<th>resolution</th>\n<th>1k model</th>\n<th>21k model</th>\n<th>iNat21 model</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MetaFormer-0</td>\n<td>224x224</td>\n<td><a href=\"https://drive.google.com/file/d/1BYbe3mrKioN-Ara6hhJiaiEgJLl_thSH/view?usp=sharing\" target=\"_blank\">metafg_0_1k_224</a></td>\n<td><a href=\"https://drive.google.com/file/d/1834jQ9OPHOBZDgv7jD6Qu5mNLsD9aeZv/view?usp=sharing\" target=\"_blank\">metafg_0_21k_224</a></td>\n<td>-</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>224x224</td>\n<td><a href=\"https://drive.google.com/file/d/1p-nIZgnrDatqmSzzDknTFYw-yEEUD_Rz/view?usp=sharing\" target=\"_blank\">metafg_1_1k_224</a></td>\n<td><a href=\"https://drive.google.com/file/d/1AcybDVEY-kXFT0D79w1G7I0h4r1IxLlG/view?usp=sharing\" target=\"_blank\">metafg_1_21k_224</a></td>\n<td>-</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>224x224</td>\n<td><a href=\"https://drive.google.com/file/d/1K6EEyFKbMUBpPqaEJMvo93YHTXCsgH2V/view?usp=sharing\" target=\"_blank\">metafg_2_1k_224</a></td>\n<td><a href=\"https://drive.google.com/file/d/1VygaD_IwYq25KwoupWfttKRZUm2_SPeK/view?usp=sharing\" target=\"_blank\">metafg_2_21k_224</a></td>\n<td>-</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>384x384</td>\n<td><a href=\"https://drive.google.com/file/d/1r62S3CJFRWV_qA5udC9MOFOJYwRf8mE2/view?usp=sharing\" target=\"_blank\">metafg_0_1k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/1wVmlPjNTA6JKHcF3ROGorEVPxKVO83Ss/view?usp=sharing\" target=\"_blank\">metafg_0_21k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/11gCk_IuSN7krdkOUSWSM4xlf8GGknmxc/view?usp=sharing\" target=\"_blank\">metafg_0_inat21_384</a></td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>384x384</td>\n<td><a href=\"https://drive.google.com/file/d/12OTmZg4J6fMGvs-colOTDfmhdA5EMMvo/view?usp=sharing\" target=\"_blank\">metafg_1_1k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/13dsarbtsNrkhpG5XpCRlN5ogXDGXO3Z_/view?usp=sharing\" target=\"_blank\">metafg_1_21k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/1ATUIrDxaQaGqx4lJ8HE2IwX_evMhblPu/view?usp=sharing\" target=\"_blank\">metafg_1_inat21_384</a></td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>384x384</td>\n<td><a href=\"https://drive.google.com/file/d/167oBaseORq32aFA3Ex6lpHuasvu2PMb8/view?usp=sharing\" target=\"_blank\">metafg_2_1k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/1PnpntloQaYduEokFGQ6y79G7DdyjD_u3/view?usp=sharing\" target=\"_blank\">metafg_2_21k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/17sUNST7ivQhonBAfZEiTOLAgtaHa4F3e/view?usp=sharing\" target=\"_blank\">metafg_2_inat21_384</a></td>\n</tr>\n</tbody>\n</table>\n<p>Repo : <a href=\"https://github.com/dqshuai/MetaFormer\" target=\"_blank\">https://github.com/dqshuai/MetaFormer</a><br>\npaper link: <a href=\"https://arxiv.org/abs/2203.02751\" target=\"_blank\">https://arxiv.org/abs/2203.02751</a></p>",
  "messages": [
    {
      "id": 1718617,
      "postDate": "2022-03-11T02:37:22.127Z",
      "content": "<h3>MetaFormer</h3>\n<p>We are very happy to meet you here and to tell you about our work(MetaFormer). We use a unified framework with meta information to do fine-grained recognition task. We have achieved SOTA performance on INAT18/19/21 and CUB datasets.</p>\n<h3>Fine-grained Datasets</h3>\n<p>Result on fine-grained datasets with different pre-trained model.</p>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>Pretrain</th>\n<th>CUB</th>\n<th>NABirds</th>\n<th>iNat2017</th>\n<th>iNat2018</th>\n<th>Cars</th>\n<th>Aircraft</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-1k</td>\n<td>89.6</td>\n<td>89.1</td>\n<td>75.7</td>\n<td>79.5</td>\n<td>95.0</td>\n<td>91.2</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-21k</td>\n<td>89.7</td>\n<td>89.5</td>\n<td>75.8</td>\n<td>79.9</td>\n<td>94.6</td>\n<td>91.2</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>iNaturalist 2021</td>\n<td>91.8</td>\n<td>91.5</td>\n<td>78.3</td>\n<td>82.9</td>\n<td>95.1</td>\n<td>87.4</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-1k</td>\n<td>89.7</td>\n<td>89.4</td>\n<td>78.2</td>\n<td>81.9</td>\n<td>94.9</td>\n<td>90.8</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-21k</td>\n<td>91.3</td>\n<td>91.6</td>\n<td>79.4</td>\n<td>83.2</td>\n<td>95.0</td>\n<td>92.6</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>iNaturalist 2021</td>\n<td>92.3</td>\n<td>92.7</td>\n<td>82.0</td>\n<td>87.5</td>\n<td>95.0</td>\n<td>92.5</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-1k</td>\n<td>89.7</td>\n<td>89.7</td>\n<td>79.0</td>\n<td>82.6</td>\n<td>95.0</td>\n<td>92.4</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-21k</td>\n<td>91.8</td>\n<td>92.2</td>\n<td>80.4</td>\n<td>84.3</td>\n<td>95.1</td>\n<td>92.9</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>iNaturalist 2021</td>\n<td>92.9</td>\n<td>93.0</td>\n<td>82.8</td>\n<td>87.7</td>\n<td>95.4</td>\n<td>92.8</td>\n</tr>\n</tbody>\n</table>\n<p>Results in iNaturalist 2019, iNaturalist 2018, and iNaturalist 2021 with meta-information.</p>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>Pretrain</th>\n<th>Meta added</th>\n<th>iNat2017</th>\n<th>iNat2018</th>\n<th>iNat2021</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-1k</td>\n<td>N</td>\n<td>75.7</td>\n<td>79.5</td>\n<td>88.4</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>ImageNet-1k</td>\n<td>Y</td>\n<td>79.8(+4.1)</td>\n<td>85.4(+5.9)</td>\n<td>92.6(+4.2)</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-1k</td>\n<td>N</td>\n<td>78.2</td>\n<td>81.9</td>\n<td>90.2</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>ImageNet-1k</td>\n<td>Y</td>\n<td>81.3(+3.1)</td>\n<td>86.5(+4.6)</td>\n<td>93.4(+3.2)</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-1k</td>\n<td>N</td>\n<td>79.0</td>\n<td>82.6</td>\n<td>89.8</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-1k</td>\n<td>Y</td>\n<td>82.0(+3.0)</td>\n<td>86.8(+4.2)</td>\n<td>93.2(+3.4)</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-21k</td>\n<td>N</td>\n<td>80.4</td>\n<td>84.3</td>\n<td>90.3</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>ImageNet-21k</td>\n<td>Y</td>\n<td>83.4(+3.0)</td>\n<td>88.7(+4.4)</td>\n<td>93.6(+3.3)</td>\n</tr>\n</tbody>\n</table>\n<p>We also provide Imagenet 1k/22k and Inaturalist Pretrain models with various resolution, Welcome to use our pretrain models and codebase.</p>\n<h3>Model zoo</h3>\n<table>\n<thead>\n<tr>\n<th>name</th>\n<th>resolution</th>\n<th>1k model</th>\n<th>21k model</th>\n<th>iNat21 model</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MetaFormer-0</td>\n<td>224x224</td>\n<td><a href=\"https://drive.google.com/file/d/1BYbe3mrKioN-Ara6hhJiaiEgJLl_thSH/view?usp=sharing\" target=\"_blank\">metafg_0_1k_224</a></td>\n<td><a href=\"https://drive.google.com/file/d/1834jQ9OPHOBZDgv7jD6Qu5mNLsD9aeZv/view?usp=sharing\" target=\"_blank\">metafg_0_21k_224</a></td>\n<td>-</td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>224x224</td>\n<td><a href=\"https://drive.google.com/file/d/1p-nIZgnrDatqmSzzDknTFYw-yEEUD_Rz/view?usp=sharing\" target=\"_blank\">metafg_1_1k_224</a></td>\n<td><a href=\"https://drive.google.com/file/d/1AcybDVEY-kXFT0D79w1G7I0h4r1IxLlG/view?usp=sharing\" target=\"_blank\">metafg_1_21k_224</a></td>\n<td>-</td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>224x224</td>\n<td><a href=\"https://drive.google.com/file/d/1K6EEyFKbMUBpPqaEJMvo93YHTXCsgH2V/view?usp=sharing\" target=\"_blank\">metafg_2_1k_224</a></td>\n<td><a href=\"https://drive.google.com/file/d/1VygaD_IwYq25KwoupWfttKRZUm2_SPeK/view?usp=sharing\" target=\"_blank\">metafg_2_21k_224</a></td>\n<td>-</td>\n</tr>\n<tr>\n<td>MetaFormer-0</td>\n<td>384x384</td>\n<td><a href=\"https://drive.google.com/file/d/1r62S3CJFRWV_qA5udC9MOFOJYwRf8mE2/view?usp=sharing\" target=\"_blank\">metafg_0_1k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/1wVmlPjNTA6JKHcF3ROGorEVPxKVO83Ss/view?usp=sharing\" target=\"_blank\">metafg_0_21k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/11gCk_IuSN7krdkOUSWSM4xlf8GGknmxc/view?usp=sharing\" target=\"_blank\">metafg_0_inat21_384</a></td>\n</tr>\n<tr>\n<td>MetaFormer-1</td>\n<td>384x384</td>\n<td><a href=\"https://drive.google.com/file/d/12OTmZg4J6fMGvs-colOTDfmhdA5EMMvo/view?usp=sharing\" target=\"_blank\">metafg_1_1k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/13dsarbtsNrkhpG5XpCRlN5ogXDGXO3Z_/view?usp=sharing\" target=\"_blank\">metafg_1_21k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/1ATUIrDxaQaGqx4lJ8HE2IwX_evMhblPu/view?usp=sharing\" target=\"_blank\">metafg_1_inat21_384</a></td>\n</tr>\n<tr>\n<td>MetaFormer-2</td>\n<td>384x384</td>\n<td><a href=\"https://drive.google.com/file/d/167oBaseORq32aFA3Ex6lpHuasvu2PMb8/view?usp=sharing\" target=\"_blank\">metafg_2_1k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/1PnpntloQaYduEokFGQ6y79G7DdyjD_u3/view?usp=sharing\" target=\"_blank\">metafg_2_21k_384</a></td>\n<td><a href=\"https://drive.google.com/file/d/17sUNST7ivQhonBAfZEiTOLAgtaHa4F3e/view?usp=sharing\" target=\"_blank\">metafg_2_inat21_384</a></td>\n</tr>\n</tbody>\n</table>\n<p>Repo : <a href=\"https://github.com/dqshuai/MetaFormer\" target=\"_blank\">https://github.com/dqshuai/MetaFormer</a><br>\npaper link: <a href=\"https://arxiv.org/abs/2203.02751\" target=\"_blank\">https://arxiv.org/abs/2203.02751</a></p>",
      "rawMarkdown": "### MetaFormer\nWe are very happy to meet you here and to tell you about our work(MetaFormer). We use a unified framework with meta information to do fine-grained recognition task. We have achieved SOTA performance on INAT18/19/21 and CUB datasets.\n\n###Fine-grained Datasets\nResult on fine-grained datasets with different pre-trained model.\n| Name       | Pretrain   | CUB | NABirds |  iNat2017   | iNat2018  | Cars | Aircraft |\n| :--------: | :----------: | :--------: | :----------: | :------------: | :------------: | :--------: |:--------: |\n| MetaFormer-0|ImageNet-1k|89.6|89.1|75.7|79.5|95.0|91.2|\n| MetaFormer-0|ImageNet-21k|89.7|89.5|75.8|79.9|94.6|91.2|\n| MetaFormer-0|iNaturalist 2021|91.8|91.5|78.3|82.9|95.1|87.4|\n| MetaFormer-1|ImageNet-1k|89.7|89.4|78.2|81.9|94.9|90.8|\n| MetaFormer-1|ImageNet-21k|91.3|91.6|79.4|83.2|95.0|92.6|\n| MetaFormer-1|iNaturalist 2021|92.3|92.7|82.0|87.5|95.0|92.5|\n| MetaFormer-2|ImageNet-1k|89.7|89.7|79.0|82.6|95.0|92.4|\n| MetaFormer-2|ImageNet-21k|91.8|92.2|80.4|84.3|95.1|92.9|\n| MetaFormer-2|iNaturalist 2021|92.9|93.0|82.8|87.7|95.4|92.8|\n\n\nResults in iNaturalist 2019, iNaturalist 2018, and iNaturalist 2021 with meta-information.\n| Name       | Pretrain   | Meta added| iNat2017   |  iNat2018   | iNat2021   |\n| :--------: | :----------: | :--------: | :---------- | :------------ |:------------ |\n|MetaFormer-0|ImageNet-1k|N|75.7|79.5|88.4|\n|MetaFormer-0|ImageNet-1k|Y|79.8(+4.1)|85.4(+5.9)|92.6(+4.2)|\n|MetaFormer-1|ImageNet-1k|N|78.2|81.9|90.2|\n|MetaFormer-1|ImageNet-1k|Y|81.3(+3.1)|86.5(+4.6)|93.4(+3.2)|\n|MetaFormer-2|ImageNet-1k|N|79.0|82.6|89.8|\n|MetaFormer-2|ImageNet-1k|Y|82.0(+3.0)|86.8(+4.2)|93.2(+3.4)|\n|MetaFormer-2|ImageNet-21k|N|80.4|84.3|90.3|\n|MetaFormer-2|ImageNet-21k|Y|83.4(+3.0)|88.7(+4.4)|93.6(+3.3)|\n\nWe also provide Imagenet 1k/22k and Inaturalist Pretrain models with various resolution, Welcome to use our pretrain models and codebase.\n\n### Model zoo\n| name       | resolution   | 1k model   |  21k model   | iNat21 model   |\n| :--------: | :----------: | :--------: | :----------: | :------------: |\n| MetaFormer-0   | 224x224 | [metafg_0_1k_224](https://drive.google.com/file/d/1BYbe3mrKioN-Ara6hhJiaiEgJLl_thSH/view?usp=sharing)|[metafg_0_21k_224](https://drive.google.com/file/d/1834jQ9OPHOBZDgv7jD6Qu5mNLsD9aeZv/view?usp=sharing)|-|\n| MetaFormer-1   | 224x224 | [metafg_1_1k_224](https://drive.google.com/file/d/1p-nIZgnrDatqmSzzDknTFYw-yEEUD_Rz/view?usp=sharing)|[metafg_1_21k_224](https://drive.google.com/file/d/1AcybDVEY-kXFT0D79w1G7I0h4r1IxLlG/view?usp=sharing)|-|\n| MetaFormer-2   | 224x224 | [metafg_2_1k_224](https://drive.google.com/file/d/1K6EEyFKbMUBpPqaEJMvo93YHTXCsgH2V/view?usp=sharing)|[metafg_2_21k_224](https://drive.google.com/file/d/1VygaD_IwYq25KwoupWfttKRZUm2_SPeK/view?usp=sharing)|-|\n| MetaFormer-0   |     384x384      |  [metafg_0_1k_384](https://drive.google.com/file/d/1r62S3CJFRWV_qA5udC9MOFOJYwRf8mE2/view?usp=sharing)  |  [metafg_0_21k_384](https://drive.google.com/file/d/1wVmlPjNTA6JKHcF3ROGorEVPxKVO83Ss/view?usp=sharing)  |  [metafg_0_inat21_384](https://drive.google.com/file/d/11gCk_IuSN7krdkOUSWSM4xlf8GGknmxc/view?usp=sharing)  |\n| MetaFormer-1   |     384x384      |  [metafg_1_1k_384](https://drive.google.com/file/d/12OTmZg4J6fMGvs-colOTDfmhdA5EMMvo/view?usp=sharing)  |  [metafg_1_21k_384](https://drive.google.com/file/d/13dsarbtsNrkhpG5XpCRlN5ogXDGXO3Z_/view?usp=sharing)  |  [metafg_1_inat21_384](https://drive.google.com/file/d/1ATUIrDxaQaGqx4lJ8HE2IwX_evMhblPu/view?usp=sharing)  |\n| MetaFormer-2   |     384x384      |  [metafg_2_1k_384](https://drive.google.com/file/d/167oBaseORq32aFA3Ex6lpHuasvu2PMb8/view?usp=sharing)  |  [metafg_2_21k_384](https://drive.google.com/file/d/1PnpntloQaYduEokFGQ6y79G7DdyjD_u3/view?usp=sharing)  |  [metafg_2_inat21_384](https://drive.google.com/file/d/17sUNST7ivQhonBAfZEiTOLAgtaHa4F3e/view?usp=sharing)  |\n\nRepo : https://github.com/dqshuai/MetaFormer\npaper link: https://arxiv.org/abs/2203.02751",
      "votes": 5
    },
    {
      "id": 1721167,
      "postDate": "2022-03-13T13:21:47.353Z",
      "content": "<p>This is so cool!!! I knew that one day someone would merge image and metadata into one unified model - doing it through transformers makes it even cooler 😃</p>\n<p>I don't know how iNaturalist's species identification AI works (in their app or web platform) but I'm sure they'd be thrilled to know this exists considering it seems to take at least location data into consideration. Thanks for sharing!</p>",
      "rawMarkdown": "This is so cool!!! I knew that one day someone would merge image and metadata into one unified model - doing it through transformers makes it even cooler 😃\n\nI don't know how iNaturalist's species identification AI works (in their app or web platform) but I'm sure they'd be thrilled to know this exists considering it seems to take at least location data into consideration. Thanks for sharing!"
    }
  ],
  "comments": [
    {
      "id": 1721167,
      "author_name": "Dax Ledesma",
      "author_url": "",
      "post_date": "2022-03-13T13:21:47.353000",
      "content": "<p>This is so cool!!! I knew that one day someone would merge image and metadata into one unified model - doing it through transformers makes it even cooler 😃</p>\n<p>I don't know how iNaturalist's species identification AI works (in their app or web platform) but I'm sure they'd be thrilled to know this exists considering it seems to take at least location data into consideration. Thanks for sharing!</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1718617": "### MetaFormer\nWe are very happy to meet you here and to tell you about our work(MetaFormer). We use a unified framework with meta information to do fine-grained recognition task. We have achieved SOTA performance on INAT18/19/21 and CUB datasets.\n\n###Fine-grained Datasets\nResult on fine-grained datasets with different pre-trained model.\n| Name       | Pretrain   | CUB | NABirds |  iNat2017   | iNat2018  | Cars | Aircraft |\n| :--------: | :----------: | :--------: | :----------: | :------------: | :------------: | :--------: |:--------: |\n| MetaFormer-0|ImageNet-1k|89.6|89.1|75.7|79.5|95.0|91.2|\n| MetaFormer-0|ImageNet-21k|89.7|89.5|75.8|79.9|94.6|91.2|\n| MetaFormer-0|iNaturalist 2021|91.8|91.5|78.3|82.9|95.1|87.4|\n| MetaFormer-1|ImageNet-1k|89.7|89.4|78.2|81.9|94.9|90.8|\n| MetaFormer-1|ImageNet-21k|91.3|91.6|79.4|83.2|95.0|92.6|\n| MetaFormer-1|iNaturalist 2021|92.3|92.7|82.0|87.5|95.0|92.5|\n| MetaFormer-2|ImageNet-1k|89.7|89.7|79.0|82.6|95.0|92.4|\n| MetaFormer-2|ImageNet-21k|91.8|92.2|80.4|84.3|95.1|92.9|\n| MetaFormer-2|iNaturalist 2021|92.9|93.0|82.8|87.7|95.4|92.8|\n\n\nResults in iNaturalist 2019, iNaturalist 2018, and iNaturalist 2021 with meta-information.\n| Name       | Pretrain   | Meta added| iNat2017   |  iNat2018   | iNat2021   |\n| :--------: | :----------: | :--------: | :---------- | :------------ |:------------ |\n|MetaFormer-0|ImageNet-1k|N|75.7|79.5|88.4|\n|MetaFormer-0|ImageNet-1k|Y|79.8(+4.1)|85.4(+5.9)|92.6(+4.2)|\n|MetaFormer-1|ImageNet-1k|N|78.2|81.9|90.2|\n|MetaFormer-1|ImageNet-1k|Y|81.3(+3.1)|86.5(+4.6)|93.4(+3.2)|\n|MetaFormer-2|ImageNet-1k|N|79.0|82.6|89.8|\n|MetaFormer-2|ImageNet-1k|Y|82.0(+3.0)|86.8(+4.2)|93.2(+3.4)|\n|MetaFormer-2|ImageNet-21k|N|80.4|84.3|90.3|\n|MetaFormer-2|ImageNet-21k|Y|83.4(+3.0)|88.7(+4.4)|93.6(+3.3)|\n\nWe also provide Imagenet 1k/22k and Inaturalist Pretrain models with various resolution, Welcome to use our pretrain models and codebase.\n\n### Model zoo\n| name       | resolution   | 1k model   |  21k model   | iNat21 model   |\n| :--------: | :----------: | :--------: | :----------: | :------------: |\n| MetaFormer-0   | 224x224 | [metafg_0_1k_224](https://drive.google.com/file/d/1BYbe3mrKioN-Ara6hhJiaiEgJLl_thSH/view?usp=sharing)|[metafg_0_21k_224](https://drive.google.com/file/d/1834jQ9OPHOBZDgv7jD6Qu5mNLsD9aeZv/view?usp=sharing)|-|\n| MetaFormer-1   | 224x224 | [metafg_1_1k_224](https://drive.google.com/file/d/1p-nIZgnrDatqmSzzDknTFYw-yEEUD_Rz/view?usp=sharing)|[metafg_1_21k_224](https://drive.google.com/file/d/1AcybDVEY-kXFT0D79w1G7I0h4r1IxLlG/view?usp=sharing)|-|\n| MetaFormer-2   | 224x224 | [metafg_2_1k_224](https://drive.google.com/file/d/1K6EEyFKbMUBpPqaEJMvo93YHTXCsgH2V/view?usp=sharing)|[metafg_2_21k_224](https://drive.google.com/file/d/1VygaD_IwYq25KwoupWfttKRZUm2_SPeK/view?usp=sharing)|-|\n| MetaFormer-0   |     384x384      |  [metafg_0_1k_384](https://drive.google.com/file/d/1r62S3CJFRWV_qA5udC9MOFOJYwRf8mE2/view?usp=sharing)  |  [metafg_0_21k_384](https://drive.google.com/file/d/1wVmlPjNTA6JKHcF3ROGorEVPxKVO83Ss/view?usp=sharing)  |  [metafg_0_inat21_384](https://drive.google.com/file/d/11gCk_IuSN7krdkOUSWSM4xlf8GGknmxc/view?usp=sharing)  |\n| MetaFormer-1   |     384x384      |  [metafg_1_1k_384](https://drive.google.com/file/d/12OTmZg4J6fMGvs-colOTDfmhdA5EMMvo/view?usp=sharing)  |  [metafg_1_21k_384](https://drive.google.com/file/d/13dsarbtsNrkhpG5XpCRlN5ogXDGXO3Z_/view?usp=sharing)  |  [metafg_1_inat21_384](https://drive.google.com/file/d/1ATUIrDxaQaGqx4lJ8HE2IwX_evMhblPu/view?usp=sharing)  |\n| MetaFormer-2   |     384x384      |  [metafg_2_1k_384](https://drive.google.com/file/d/167oBaseORq32aFA3Ex6lpHuasvu2PMb8/view?usp=sharing)  |  [metafg_2_21k_384](https://drive.google.com/file/d/1PnpntloQaYduEokFGQ6y79G7DdyjD_u3/view?usp=sharing)  |  [metafg_2_inat21_384](https://drive.google.com/file/d/17sUNST7ivQhonBAfZEiTOLAgtaHa4F3e/view?usp=sharing)  |\n\nRepo : https://github.com/dqshuai/MetaFormer\npaper link: https://arxiv.org/abs/2203.02751",
    "1721167": "This is so cool!!! I knew that one day someone would merge image and metadata into one unified model - doing it through transformers makes it even cooler 😃\n\nI don't know how iNaturalist's species identification AI works (in their app or web platform) but I'm sure they'd be thrilled to know this exists considering it seems to take at least location data into consideration. Thanks for sharing!"
  }
}