{
  "id": 212743,
  "title": "How to train VIT well?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/212743",
  "author_name": "",
  "post_date": "2021-01-20T03:40:14.946567700Z",
  "votes": 4,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I tried to train VIT (using vit_base_patch16_384 from timm package ) in this task. <br>\nBut for the test image (only one), I always got a label 2. It is obviously a bad prediction.<br>\nCan anyone share some skills of training VIT? <br>\nWhat score can the vit get in LB? (k fold+ tta)<br>\nThanks~~~  </p>",
  "messages": [
    {
      "id": "1160616",
      "postDate": "01/20/2021 03:40:14",
      "content": "<p>I tried to train VIT (using vit_base_patch16_384 from timm package ) in this task. <br>\nBut for the test image (only one), I always got a label 2. It is obviously a bad prediction.<br>\nCan anyone share some skills of training VIT? <br>\nWhat score can the vit get in LB? (k fold+ tta)<br>\nThanks~~~  </p>",
      "rawMarkdown": "I tried to train VIT (using vit_base_patch16_384 from timm package ) in this task. \nBut for the test image (only one), I always got a label 2. It is obviously a bad prediction.\nCan anyone share some skills of training VIT? \nWhat score can the vit get in LB? (k fold+ tta)\nThanks~~~",
      "votes": null
    },
    {
      "id": "1160915",
      "postDate": "01/20/2021 08:02:39",
      "content": "<p>Visual transformers require lots of data, more augumentations such as cutmix, mixup, randaug, fmix will give boost. Pair them together by some probability, fine tune those networks, apply label smoothing, ensemble them.</p>",
      "rawMarkdown": "Visual transformers require lots of data, more augumentations such as cutmix, mixup, randaug, fmix will give boost. Pair them together by some probability, fine tune those networks, apply label smoothing, ensemble them.",
      "votes": null
    },
    {
      "id": "1184558",
      "postDate": "02/03/2021 15:23:58",
      "content": "<p>Do you have some ideas these days ? I see you at 12th place</p>",
      "rawMarkdown": "Do you have some ideas these days ? I see you at 12th place",
      "votes": null
    },
    {
      "id": "1184606",
      "postDate": "02/03/2021 15:58:19",
      "content": "<p>That is not necessarily a bad predicition: <a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/216241\" target=\"_blank\">https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/216241</a></p>",
      "rawMarkdown": "That is not necessarily a bad predicition: https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/216241",
      "votes": null
    },
    {
      "id": "1185416",
      "postDate": "02/04/2021 06:44:35",
      "content": "<p>yeah, maybe.  </p>",
      "rawMarkdown": "yeah, maybe.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1160915,
      "author_name": "anku5hk",
      "author_url": "",
      "post_date": "01/20/2021 08:02:39",
      "content": "<p>Visual transformers require lots of data, more augumentations such as cutmix, mixup, randaug, fmix will give boost. Pair them together by some probability, fine tune those networks, apply label smoothing, ensemble them.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1184558,
      "author_name": "clw5180",
      "author_url": "",
      "post_date": "02/03/2021 15:23:58",
      "content": "<p>Do you have some ideas these days ? I see you at 12th place</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1184606,
      "author_name": "alexanderriedel",
      "author_url": "",
      "post_date": "02/03/2021 15:58:19",
      "content": "<p>That is not necessarily a bad predicition: <a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/216241\" target=\"_blank\">https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/216241</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1185416,
          "author_name": "qixinyan",
          "author_url": "",
          "post_date": "02/04/2021 06:44:35",
          "content": "<p>yeah, maybe.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1160616": "I tried to train VIT (using vit_base_patch16_384 from timm package ) in this task. \nBut for the test image (only one), I always got a label 2. It is obviously a bad prediction.\nCan anyone share some skills of training VIT? \nWhat score can the vit get in LB? (k fold+ tta)\nThanks~~~",
    "1160915": "Visual transformers require lots of data, more augumentations such as cutmix, mixup, randaug, fmix will give boost. Pair them together by some probability, fine tune those networks, apply label smoothing, ensemble them.",
    "1184558": "Do you have some ideas these days ? I see you at 12th place",
    "1184606": "That is not necessarily a bad predicition: https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/216241",
    "1185416": "yeah, maybe."
  },
  "source": "meta"
}