{ "id": 222163, "title": "Vit Training Notebook for RANZCR in Tensorflow", "url": "/competitions/ranzcr-clip-catheter-line-classification/discussion/222163", "author_name": "", "post_date": "2021-02-25T16:29:27.857831600Z", "votes": 7, "comment_count": 3, "views": 0, "content": "

I have created a basic vit Training Notebook for RANZCR CLip in Tensorflow. There are currently 4 models implemented by keras for vit.

\n
    \n
  1. For Basic 16 use model vit_b16
    2. \n
  2. For Basic 32 use model vit_b32
    3. \n
  3. For Large 16 use model vit_l16
    4. \n
  4. For Large 32 use model vit_l32
    \nWhere 16 and 32 are basically the patch size.
    5. \n
\n

The model can be trained in TPU/GPU. I have used pretrained weights of 'imagenet21k+imagenet2012' for the vit in the notebook.

\n

The notebook link is here and details of keras implementation here

", "messages": [ { "id": "1218192", "postDate": "02/25/2021 16:29:27", "content": "

I have created a basic vit Training Notebook for RANZCR CLip in Tensorflow. There are currently 4 models implemented by keras for vit.

\n
    \n
  1. For Basic 16 use model vit_b16
    2. \n
  2. For Basic 32 use model vit_b32
    3. \n
  3. For Large 16 use model vit_l16
    4. \n
  4. For Large 32 use model vit_l32
    \nWhere 16 and 32 are basically the patch size.
    5. \n
\n

The model can be trained in TPU/GPU. I have used pretrained weights of 'imagenet21k+imagenet2012' for the vit in the notebook.

\n

The notebook link is here and details of keras implementation here

", "rawMarkdown": "I have created a basic vit Training Notebook for RANZCR CLip in Tensorflow. There are currently 4 models implemented by keras for vit. \n1. For Basic 16 use model vit_b16\n2. For Basic 32 use model vit_b32\n3. For Large 16 use model vit_l16\n4. For Large 32 use model vit_l32\nWhere 16 and 32 are basically the patch size. \n \nThe model can be trained in TPU/GPU. I have used pretrained weights of 'imagenet21k+imagenet2012' for the vit in the notebook.\n\nThe notebook link is [here](https://www.kaggle.com/vickygoyal/ranzcrvittrain) and details of keras implementation [here](https://pypi.org/project/vit-keras/)", "votes": null }, { "id": "1218202", "postDate": "02/25/2021 16:36:58", "content": "

Hello!

\n

Great work! What's your best CV/PB with ViT so far?

", "rawMarkdown": "Hello! \n\nGreat work! What's your best CV/PB with ViT so far?", "votes": null }, { "id": "1218222", "postDate": "02/25/2021 16:49:24", "content": "

CV = 91.7. Have not done submission corresponding to it. Will do if get close to 93/94. Still doing fine tuning. Just thought of sharing the training and inference notebooks for ViT. The only submission made is to check whether the shared inference notebook is working properly or not

", "rawMarkdown": "CV = 91.7. Have not done submission corresponding to it. Will do if get close to 93/94. Still doing fine tuning. Just thought of sharing the training and inference notebooks for ViT. The only submission made is to check whether the shared inference notebook is working properly or not", "votes": null }, { "id": "1218648", "postDate": "02/26/2021 04:52:24", "content": "

Created Inference notebook here

", "rawMarkdown": "Created Inference notebook [here](https://www.kaggle.com/vickygoyal/rancrvitinference)", "votes": null } ], "comments": [ { "id": 1218202, "author_name": "nickuzmenkov", "author_url": "", "post_date": "02/25/2021 16:36:58", "content": "

Hello!

\n

Great work! What's your best CV/PB with ViT so far?

", "votes": null, "replies": [ { "id": 1218222, "author_name": "vickygoyal", "author_url": "", "post_date": "02/25/2021 16:49:24", "content": "

CV = 91.7. Have not done submission corresponding to it. Will do if get close to 93/94. Still doing fine tuning. Just thought of sharing the training and inference notebooks for ViT. The only submission made is to check whether the shared inference notebook is working properly or not

", "votes": null, "replies": [] } ] }, { "id": 1218648, "author_name": "vickygoyal", "author_url": "", "post_date": "02/26/2021 04:52:24", "content": "

Created Inference notebook here

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1218192": "I have created a basic vit Training Notebook for RANZCR CLip in Tensorflow. There are currently 4 models implemented by keras for vit. \n1. For Basic 16 use model vit_b16\n2. For Basic 32 use model vit_b32\n3. For Large 16 use model vit_l16\n4. For Large 32 use model vit_l32\nWhere 16 and 32 are basically the patch size. \n \nThe model can be trained in TPU/GPU. I have used pretrained weights of 'imagenet21k+imagenet2012' for the vit in the notebook.\n\nThe notebook link is [here](https://www.kaggle.com/vickygoyal/ranzcrvittrain) and details of keras implementation [here](https://pypi.org/project/vit-keras/)", "1218202": "Hello! \n\nGreat work! What's your best CV/PB with ViT so far?", "1218222": "CV = 91.7. Have not done submission corresponding to it. Will do if get close to 93/94. Still doing fine tuning. Just thought of sharing the training and inference notebooks for ViT. The only submission made is to check whether the shared inference notebook is working properly or not", "1218648": "Created Inference notebook [here](https://www.kaggle.com/vickygoyal/rancrvitinference)" }, "source": "meta" }