\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2034058%2F55b579924502626e429f70535aa65ee9%2Foverfit.jpg?generation=1610482797925162&alt=media\")
i understand,not everyone has fancy hardware/GPU and i am one of them :)
\nfor people like us,TPU is our last hope and if you are a pytorch lover then pytorch xla is all you need :)
\nlast time i used pytorch xla almost 1 year ago in alaska 2 competition,it had a lot of issue related to memory and performance back then but now in 2021 pytorch xla has changed a lot,it is much stronger than it was before(even though TF TPU is better at this moment)
I have started my first #kaggle notebook of this year Using Pytorch XLA as i am huge fan of pytorch and that too uses this competitions dataset for experiments
\nin this notebook ViT - Pytorch xla (TPU) for leaf disease i tried Visual Transformers/ ViT base model from popular pytorch image models
\nin version 10 of that notebook i was able to achieve 0.88+ validation accuracy on first fold within 10 epoch that took around 1 hour.
then in this notebook Faster torch tpu baseline for leaf disease i tried to convert @haqishen san's magnificent kernel titled Baseline - Modified From Previous Competition which is a very good boilerplate to use in pretty much all future projects. We all know that this competition is very similar to last pandas challenge where we had label noise and in that competition my team used this notebook of qishen ha san called Train EfficientNet-B0 w/ 36 tiles_256 [LB0.87] and missed silver by only 1 place in private lb. we tried a lot of different things and lot of different models using that baseline of qishen ha san which helped us get into 51st private lb position.
\ni believe we can do very fast experiments with this notebook Faster torch tpu baseline for leaf disease, because
\nin draft mode i was able to get ~0.89 validation accuracy after 1st epoch on first fold(which took me only 10 minutes to get that result using ViT Large model)
In pandas competition our cv lb has good correlation and in this competition we can see cv and public lb has some correlation but how much trust we can put on cv for private lb that's a big question i guess!
\n
i spend endless hour debugging a few things on those 2 tpu kernels that i mentioned above,i am still having trouble saving the best weight file of each fold, if i put xm.save() or xser.save() inside epoch loop for saving each epochs weight file or best weight file of each fold then the kernel commit doesn't finish but in interactive mode everything works fine. i have posted about this problem here Tpu kernel commit not finishing but worked well during interactive session
\nanyone faced similar problem? thank you in advance
i understand,not everyone has fancy hardware/GPU and i am one of them :)
\nfor people like us,TPU is our last hope and if you are a pytorch lover then pytorch xla is all you need :)
\nlast time i used pytorch xla almost 1 year ago in alaska 2 competition,it had a lot of issue related to memory and performance back then but now in 2021 pytorch xla has changed a lot,it is much stronger than it was before(even though TF TPU is better at this moment)
I have started my first #kaggle notebook of this year Using Pytorch XLA as i am huge fan of pytorch and that too uses this competitions dataset for experiments
\nin this notebook ViT - Pytorch xla (TPU) for leaf disease i tried Visual Transformers/ ViT base model from popular pytorch image models
\nin version 10 of that notebook i was able to achieve 0.88+ validation accuracy on first fold within 10 epoch that took around 1 hour.
then in this notebook Faster torch tpu baseline for leaf disease i tried to convert @haqishen san's magnificent kernel titled Baseline - Modified From Previous Competition which is a very good boilerplate to use in pretty much all future projects. We all know that this competition is very similar to last pandas challenge where we had label noise and in that competition my team used this notebook of qishen ha san called Train EfficientNet-B0 w/ 36 tiles_256 [LB0.87] and missed silver by only 1 place in private lb. we tried a lot of different things and lot of different models using that baseline of qishen ha san which helped us get into 51st private lb position.
\ni believe we can do very fast experiments with this notebook Faster torch tpu baseline for leaf disease, because
\nin draft mode i was able to get ~0.89 validation accuracy after 1st epoch on first fold(which took me only 10 minutes to get that result using ViT Large model)
In pandas competition our cv lb has good correlation and in this competition we can see cv and public lb has some correlation but how much trust we can put on cv for private lb that's a big question i guess!
\n
i spend endless hour debugging a few things on those 2 tpu kernels that i mentioned above,i am still having trouble saving the best weight file of each fold, if i put xm.save() or xser.save() inside epoch loop for saving each epochs weight file or best weight file of each fold then the kernel commit doesn't finish but in interactive mode everything works fine. i have posted about this problem here Tpu kernel commit not finishing but worked well during interactive session
\nanyone faced similar problem? thank you in advance
Thank you for the notebook!!
", "rawMarkdown": "Thank you for the notebook!!", "votes": null } ], "comments": [ { "id": 1154395, "author_name": "anku5hk", "author_url": "", "post_date": "01/15/2021 16:06:30", "content": "Thank you for the notebook!!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1150777": "i understand,not everyone has fancy hardware/GPU and i am one of them :)\nfor people like us,TPU is our last hope and if you are a pytorch lover then pytorch xla is all you need :)\nlast time i used pytorch xla almost 1 year ago in alaska 2 competition,it had a lot of issue related to memory and performance back then but now in 2021 pytorch xla has changed a lot,it is much stronger than it was before(even though TF TPU is better at this moment) \n\nI have started my first #kaggle notebook of this year Using Pytorch XLA as i am huge fan of pytorch and that too uses this competitions dataset for experiments\n\nin this notebook [ViT - Pytorch xla (TPU) for leaf disease](https://www.kaggle.com/mobassir/vit-pytorch-xla-tpu-for-leaf-disease/) i tried Visual Transformers/ ViT base model from popular [pytorch image models](https://github.com/rwightman/pytorch-image-models)\nin version 10 of that notebook i was able to achieve 0.88+ validation accuracy on first fold within 10 epoch that took around 1 hour.\n\nthen in this notebook [Faster torch tpu baseline for leaf disease](https://www.kaggle.com/mobassir/faster-torch-tpu-baseline-for-leaf-disease) i tried to convert @haqishen san's magnificent kernel titled **Baseline - Modified From Previous Competition** which is a very good boilerplate to use in pretty much all future projects. We all know that this competition is very similar to last pandas challenge where we had label noise and in that competition my team used this notebook of qishen ha san called **Train EfficientNet-B0 w/ 36 tiles_256 [LB0.87]** and missed silver by only 1 place in private lb. we tried a lot of different things and lot of different models using that baseline of qishen ha san which helped us get into 51st private lb position. \n\ni believe we can do very fast experiments with this notebook [Faster torch tpu baseline for leaf disease](https://www.kaggle.com/mobassir/faster-torch-tpu-baseline-for-leaf-disease), because\nin draft mode i was able to get ~0.89 validation accuracy after 1st epoch on first fold(which took me only 10 minutes to get that result using ViT Large model)\n\nIn pandas competition our cv lb has good correlation and in this competition we can see cv and public lb has some correlation but how much trust we can put on cv for private lb that's a big question i guess! \n\n\n# Problem Faced : \ni spend endless hour debugging a few things on those 2 tpu kernels that i mentioned above,i am still having trouble saving the best weight file of each fold, if i put xm.save() or xser.save() inside epoch loop for saving each epochs weight file or best weight file of each fold then the kernel commit doesn't finish but in interactive mode everything works fine. i have posted about this problem here [Tpu kernel commit not finishing but worked well during interactive session](https://www.kaggle.com/discussion/210032)\nanyone faced similar problem? thank you in advance", "1154395": "Thank you for the notebook!!" }, "source": "meta" }