![\"stats\"](\"https://pbs.twimg.com/media/EAd62RHU0AAGXI3?format=jpg&\")
Lately, i started to recognize that more and more people are using Pytorch. Before(previous competitions), i saw more kernels with Tensorflow or Keras but now Pytorch appears to be more popular. Is it worth to switch from Keras to Pytorch in the future or pytorch is just popular for this contest?
", "messages": [ { "id": 591265, "postDate": "2019-08-03T12:24:24.687Z", "content": "Hmm so this reply is longer than I originally intended, but it was nice to write because I'm one of these people who has moved from keras to pytorch over the past year and it's nice to think about why that actually happened :)
\n\nIt turns out (for me) to be a lot of reasons:
\n\nFirstly, and maybe this is just me, but the approach I've taken to work (and kaggle when I have time) has changed. Until maybe a year ago I was quite inexperienced with machine learning and spent a lot of time trying different model architectures. That made keras perfect, I could try lots of things quickly and just do .fit on it to train it
Now I think I've been around a bit longer, I spend less time playing with model architectures and more time thinking about how to train a model differently (and I also spend much much longer doing cleaning, but that's less relevant here), for that I needed a tool that was a bit more flexible and low level than keras' out the box functions but still let me define a model easily. PyTorch has kind of perfectly hit that sweet spot and it became usable right when I needed it
\n\nAnother big factor in all of this is I find pytorch much easier to debug than tf&keras. When trying out an idea or trying to understand something someone's put in a kernel I tend to print out lots of different things, from full tensors to their shape, etc
\n\nSecondly, although I'd been \"using\" pytorch for a while, the Quora Insincere Questions challenge and the solutions people shared afterwards was a big turning point. With the same model architecture, those using pytorch managed to get much better scores by doing the training differently (eg bucketing similar length sentences into batches - this would be possible but fiddly in keras, but is easy in pytorch)
\n\nDuring the Jigsaw Toxic Comments challenge, the pytorch pretrained models were a big help - you can fine tune BERT for example with very little ceremony
\n\nThe pytorch forums are amazing, so if you're ever stuck then it feels like there's always someone there to help
\n\nfast.ai is another big factor, lots of people are now learning deep learning with pytorch
\n\nFinally, I think all these reasons mean it's easier for people to try new ideas and share them. Because of that we get lots of shared pytorch kernels in competitions, which means more people fork pytorch kernels, improve upon them and share...
", "rawMarkdown": "Hmm so this reply is longer than I originally intended, but it was nice to write because I'm one of these people who has moved from keras to pytorch over the past year and it's nice to think about why that actually happened :)\n\nIt turns out (for me) to be a lot of reasons:\n\nFirstly, and maybe this is just me, but the approach I've taken to work (and kaggle when I have time) has changed. Until maybe a year ago I was quite inexperienced with machine learning and spent a lot of time trying different model architectures. That made keras perfect, I could try lots of things quickly and just do `.fit` on it to train it\n\nNow I think I've been around a bit longer, I spend less time playing with model architectures and more time thinking about how to train a model differently (and I also spend much much longer doing cleaning, but that's less relevant here), for that I needed a tool that was a bit more flexible and low level than keras' out the box functions but still let me define a model easily. PyTorch has kind of perfectly hit that sweet spot and it became usable right when I needed it\n\nAnother big factor in all of this is I find pytorch much easier to debug than tf&keras. When trying out an idea or trying to understand something someone's put in a kernel I tend to print out lots of different things, from full tensors to their shape, etc\n\nSecondly, although I'd been \"using\" pytorch for a while, the Quora Insincere Questions challenge and the solutions people shared afterwards was a big turning point. With the same model architecture, those using pytorch managed to get much better scores by doing the training differently (eg bucketing similar length sentences into batches - this would be possible but fiddly in keras, but is easy in pytorch)\n\nDuring the Jigsaw Toxic Comments challenge, the pytorch pretrained models were a big help - you can fine tune BERT for example with very little ceremony\n\nThe [pytorch forums](https://discuss.pytorch.org/) are amazing, so if you're ever stuck then it feels like there's always someone there to help\n\n[fast.ai](https://www.fast.ai/) is another big factor, lots of people are now learning deep learning with pytorch\n\nFinally, I think all these reasons mean it's easier for people to try new ideas and share them. Because of that we get lots of shared pytorch kernels in competitions, which means more people fork pytorch kernels, improve upon them and share...", "votes": 18, "replies": [ { "id": 591302, "postDate": "2019-08-03T13:16:21.923Z", "content": "Thanks for your detailed answer!
", "rawMarkdown": "Thanks for your detailed answer! ", "votes": 1 } ] }, { "id": 590900, "postDate": "2019-08-02T19:17:17.200Z", "content": "Lately, i started to recognize that more and more people are using Pytorch. Before(previous competitions), i saw more kernels with Tensorflow or Keras but now Pytorch appears to be more popular. Is it worth to switch from Keras to Pytorch in the future or pytorch is just popular for this contest?
", "rawMarkdown": "Lately, i started to recognize that more and more people are using Pytorch. Before(previous competitions), i saw more kernels with Tensorflow or Keras but now Pytorch appears to be more popular. Is it worth to switch from Keras to Pytorch in the future or pytorch is just popular for this contest?", "votes": 10 }, { "id": 591257, "postDate": "2019-08-03T12:11:49.713Z", "content": "I switched from Keras to PyTorch. From my recent experience, most of the top teams here on Kaggle are using PyTorch.
", "rawMarkdown": "I switched from Keras to PyTorch. From my recent experience, most of the top teams here on Kaggle are using PyTorch.", "votes": 3 }, { "id": 590927, "postDate": "2019-08-02T20:51:02.423Z", "content": "I switched from Keras to Catalyst and now life is easier for me (especially, when it comes to custom layers and debugging)
", "rawMarkdown": "I switched from Keras to Catalyst and now life is easier for me (especially, when it comes to custom layers and debugging)", "votes": 3 }, { "id": 592215, "postDate": "2019-08-05T02:55:51.357Z", "content": "If you use Tensorflow 2.0, I think it's not a matter which you use ,but if you use Tensorflow 1.X, oh my god , please just give up it , it's the world's disaster .
", "rawMarkdown": "If you use Tensorflow 2.0, I think it's not a matter which you use ,but if you use Tensorflow 1.X, oh my god , please just give up it , it's the world's disaster . ", "votes": 4, "replies": [ { "id": 593451, "postDate": "2019-08-06T16:09:42.643Z", "content": "Exactly! When i tried to learned tf 1.X i gave up and decided to master Keras
", "rawMarkdown": "Exactly! When i tried to learned tf 1.X i gave up and decided to master Keras ", "votes": 1 } ] }, { "id": 591972, "postDate": "2019-08-04T15:21:07.167Z", "content": "Most people say that and I quote \"Experiment with Pytorch and use TensorFlow for deployment\".
", "rawMarkdown": "Most people say that and I quote \"Experiment with Pytorch and use TensorFlow for deployment\".", "votes": 1, "replies": [ { "id": 592268, "postDate": "2019-08-05T05:17:44.213Z", "content": "Yet now. Thats not a problem, pytorch models can be easily converted to ONNX format which can be used with Tensorflow Serving.
", "rawMarkdown": "Yet now. Thats not a problem, pytorch models can be easily converted to ONNX format which can be used with Tensorflow Serving.", "votes": 2 } ] }, { "id": 592631, "postDate": "2019-08-05T15:35:49.833Z", "content": "I started with keras because it close to sklearn implementation.\nI use pytorch now because of fast.ai
", "rawMarkdown": "I started with keras because it close to sklearn implementation.\nI use pytorch now because of fast.ai ", "votes": 2 }, { "id": 592473, "postDate": "2019-08-05T10:34:39.003Z", "content": "It's worth trying at least to see how PyTorch ideology fits your needs and style of coding. It's better to be diverse and play with TF/Keras/PyTorch/etc. Each lib has it's pros & cons.
", "rawMarkdown": "It's worth trying at least to see how PyTorch ideology fits your needs and style of coding. It's better to be diverse and play with TF/Keras/PyTorch/etc. Each lib has it's pros & cons.", "votes": 2 }, { "id": 592432, "postDate": "2019-08-05T09:17:01.957Z", "content": "Keras is really good for straightforward simple model training but you will have to face a nightmare in terms of struggle if the computation graph or training pipeline is a bit complicated. I think for beginners, Keras is a good choice and Pytorch is better for intermediate or advance usage.
", "rawMarkdown": "Keras is really good for straightforward simple model training but you will have to face a nightmare in terms of struggle if the computation graph or training pipeline is a bit complicated. I think for beginners, Keras is a good choice and Pytorch is better for intermediate or advance usage.", "votes": 2 }, { "id": 591970, "postDate": "2019-08-04T15:18:42.047Z", "content": "For me, there were a few reasons:\nDataloading/handling during the training phase is easier in pytorch IMO, you only need to specify like 2 things for datasets(len and forward) and you're good. No need to return stuff in a specific format because you can handle it however you want in the training loop.\nEasier debugging. This has already been mentioned\nMore research implementations available in pytorch than in keras.
", "rawMarkdown": "For me, there were a few reasons:\nDataloading/handling during the training phase is easier in pytorch IMO, you only need to specify like 2 things for datasets(__len__ and forward) and you're good. No need to return stuff in a specific format because you can handle it however you want in the training loop.\nEasier debugging. This has already been mentioned\nMore research implementations available in pytorch than in keras.", "votes": 2 }, { "id": 591179, "postDate": "2019-08-03T09:18:17.367Z", "content": "Here's a recent tweet from Jeremy \n> That's quite astonishing. If I'm reading this right, PyTorch grew 194% in research use last year (based on arxiv mentions) and TensorFlow only grew 23%, meaning they're now neck and neck. In another year TensorFlow may just be a small minority player in research
\n\n", "rawMarkdown": "Here's a recent tweet from Jeremy \n> That's quite astonishing. If I'm reading this right, PyTorch grew 194% in research use last year (based on arxiv mentions) and TensorFlow only grew 23%, meaning they're now neck and neck. In another year TensorFlow may just be a small minority player in research\n\n[tweet link](https://twitter.com/jeremyphoward/status/1155043840966815745)\n", "votes": 2 }, { "id": 591080, "postDate": "2019-08-03T05:31:59.180Z", "content": "I switched to Pytorch from Keras/TF, from my personal perspective Pytorch is more flexible, makes debugging easier and has abundant documentation. It has grown more and more popular in the research community in recent years.
", "rawMarkdown": "I switched to Pytorch from Keras/TF, from my personal perspective Pytorch is more flexible, makes debugging easier and has abundant documentation. It has grown more and more popular in the research community in recent years.", "votes": 2 }, { "id": 591297, "postDate": "2019-08-03T13:04:04.407Z", "rawMarkdown": "", "votes": 2, "isDeleted": true } ], "comments": [ { "id": 591265, "author_name": "Hamish", "author_url": "", "post_date": "2019-08-03T12:24:24.687000", "content": "Hmm so this reply is longer than I originally intended, but it was nice to write because I'm one of these people who has moved from keras to pytorch over the past year and it's nice to think about why that actually happened :)
\n\nIt turns out (for me) to be a lot of reasons:
\n\nFirstly, and maybe this is just me, but the approach I've taken to work (and kaggle when I have time) has changed. Until maybe a year ago I was quite inexperienced with machine learning and spent a lot of time trying different model architectures. That made keras perfect, I could try lots of things quickly and just do .fit on it to train it
Now I think I've been around a bit longer, I spend less time playing with model architectures and more time thinking about how to train a model differently (and I also spend much much longer doing cleaning, but that's less relevant here), for that I needed a tool that was a bit more flexible and low level than keras' out the box functions but still let me define a model easily. PyTorch has kind of perfectly hit that sweet spot and it became usable right when I needed it
\n\nAnother big factor in all of this is I find pytorch much easier to debug than tf&keras. When trying out an idea or trying to understand something someone's put in a kernel I tend to print out lots of different things, from full tensors to their shape, etc
\n\nSecondly, although I'd been \"using\" pytorch for a while, the Quora Insincere Questions challenge and the solutions people shared afterwards was a big turning point. With the same model architecture, those using pytorch managed to get much better scores by doing the training differently (eg bucketing similar length sentences into batches - this would be possible but fiddly in keras, but is easy in pytorch)
\n\nDuring the Jigsaw Toxic Comments challenge, the pytorch pretrained models were a big help - you can fine tune BERT for example with very little ceremony
\n\nThe pytorch forums are amazing, so if you're ever stuck then it feels like there's always someone there to help
\n\nfast.ai is another big factor, lots of people are now learning deep learning with pytorch
\n\nFinally, I think all these reasons mean it's easier for people to try new ideas and share them. Because of that we get lots of shared pytorch kernels in competitions, which means more people fork pytorch kernels, improve upon them and share...
", "votes": 18, "replies": [ { "id": 591302, "author_name": "[ods.ai] Kyrylo", "author_url": "", "post_date": "2019-08-03T13:16:21.923000", "content": "Thanks for your detailed answer!
", "votes": 1, "replies": [] } ] }, { "id": 591257, "author_name": "Filemon", "author_url": "", "post_date": "2019-08-03T12:11:49.713000", "content": "I switched from Keras to PyTorch. From my recent experience, most of the top teams here on Kaggle are using PyTorch.
", "votes": 3, "replies": [] }, { "id": 590927, "author_name": "Borys Tymchenko", "author_url": "", "post_date": "2019-08-02T20:51:02.423000", "content": "I switched from Keras to Catalyst and now life is easier for me (especially, when it comes to custom layers and debugging)
", "votes": 3, "replies": [] }, { "id": 592215, "author_name": "哈尔的移动城堡", "author_url": "", "post_date": "2019-08-05T02:55:51.357000", "content": "If you use Tensorflow 2.0, I think it's not a matter which you use ,but if you use Tensorflow 1.X, oh my god , please just give up it , it's the world's disaster .
", "votes": 4, "replies": [ { "id": 593451, "author_name": "[ods.ai] Kyrylo", "author_url": "", "post_date": "2019-08-06T16:09:42.643000", "content": "Exactly! When i tried to learned tf 1.X i gave up and decided to master Keras
", "votes": 1, "replies": [] } ] }, { "id": 591972, "author_name": "Aptha K S", "author_url": "", "post_date": "2019-08-04T15:21:07.167000", "content": "Most people say that and I quote \"Experiment with Pytorch and use TensorFlow for deployment\".
", "votes": 1, "replies": [ { "id": 592268, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2019-08-05T05:17:44.213000", "content": "Yet now. Thats not a problem, pytorch models can be easily converted to ONNX format which can be used with Tensorflow Serving.
", "votes": 2, "replies": [] } ] }, { "id": 592631, "author_name": "CC Joshua ", "author_url": "", "post_date": "2019-08-05T15:35:49.833000", "content": "I started with keras because it close to sklearn implementation.\nI use pytorch now because of fast.ai
", "votes": 2, "replies": [] }, { "id": 592473, "author_name": "Eugene Khvedchenya", "author_url": "", "post_date": "2019-08-05T10:34:39.003000", "content": "It's worth trying at least to see how PyTorch ideology fits your needs and style of coding. It's better to be diverse and play with TF/Keras/PyTorch/etc. Each lib has it's pros & cons.
", "votes": 2, "replies": [] }, { "id": 592432, "author_name": "Sabbir Ahmed", "author_url": "", "post_date": "2019-08-05T09:17:01.957000", "content": "Keras is really good for straightforward simple model training but you will have to face a nightmare in terms of struggle if the computation graph or training pipeline is a bit complicated. I think for beginners, Keras is a good choice and Pytorch is better for intermediate or advance usage.
", "votes": 2, "replies": [] }, { "id": 591970, "author_name": "sh", "author_url": "", "post_date": "2019-08-04T15:18:42.047000", "content": "For me, there were a few reasons:\nDataloading/handling during the training phase is easier in pytorch IMO, you only need to specify like 2 things for datasets(len and forward) and you're good. No need to return stuff in a specific format because you can handle it however you want in the training loop.\nEasier debugging. This has already been mentioned\nMore research implementations available in pytorch than in keras.
", "votes": 2, "replies": [] }, { "id": 591179, "author_name": "Rishabh Agrahari", "author_url": "", "post_date": "2019-08-03T09:18:17.367000", "content": "Here's a recent tweet from Jeremy \n> That's quite astonishing. If I'm reading this right, PyTorch grew 194% in research use last year (based on arxiv mentions) and TensorFlow only grew 23%, meaning they're now neck and neck. In another year TensorFlow may just be a small minority player in research
\n\n", "votes": 2, "replies": [] }, { "id": 591080, "author_name": "Rishabh Agrahari", "author_url": "", "post_date": "2019-08-03T05:31:59.180000", "content": "I switched to Pytorch from Keras/TF, from my personal perspective Pytorch is more flexible, makes debugging easier and has abundant documentation. It has grown more and more popular in the research community in recent years.
", "votes": 2, "replies": [] }, { "id": 591297, "author_name": "", "author_url": "", "post_date": "2019-08-03T13:04:04.407000", "content": "", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "591265": "Hmm so this reply is longer than I originally intended, but it was nice to write because I'm one of these people who has moved from keras to pytorch over the past year and it's nice to think about why that actually happened :)\n\nIt turns out (for me) to be a lot of reasons:\n\nFirstly, and maybe this is just me, but the approach I've taken to work (and kaggle when I have time) has changed. Until maybe a year ago I was quite inexperienced with machine learning and spent a lot of time trying different model architectures. That made keras perfect, I could try lots of things quickly and just do `.fit` on it to train it\n\nNow I think I've been around a bit longer, I spend less time playing with model architectures and more time thinking about how to train a model differently (and I also spend much much longer doing cleaning, but that's less relevant here), for that I needed a tool that was a bit more flexible and low level than keras' out the box functions but still let me define a model easily. PyTorch has kind of perfectly hit that sweet spot and it became usable right when I needed it\n\nAnother big factor in all of this is I find pytorch much easier to debug than tf&keras. When trying out an idea or trying to understand something someone's put in a kernel I tend to print out lots of different things, from full tensors to their shape, etc\n\nSecondly, although I'd been \"using\" pytorch for a while, the Quora Insincere Questions challenge and the solutions people shared afterwards was a big turning point. With the same model architecture, those using pytorch managed to get much better scores by doing the training differently (eg bucketing similar length sentences into batches - this would be possible but fiddly in keras, but is easy in pytorch)\n\nDuring the Jigsaw Toxic Comments challenge, the pytorch pretrained models were a big help - you can fine tune BERT for example with very little ceremony\n\nThe [pytorch forums](https://discuss.pytorch.org/) are amazing, so if you're ever stuck then it feels like there's always someone there to help\n\n[fast.ai](https://www.fast.ai/) is another big factor, lots of people are now learning deep learning with pytorch\n\nFinally, I think all these reasons mean it's easier for people to try new ideas and share them. Because of that we get lots of shared pytorch kernels in competitions, which means more people fork pytorch kernels, improve upon them and share...", "590900": "Lately, i started to recognize that more and more people are using Pytorch. Before(previous competitions), i saw more kernels with Tensorflow or Keras but now Pytorch appears to be more popular. Is it worth to switch from Keras to Pytorch in the future or pytorch is just popular for this contest?", "591257": "I switched from Keras to PyTorch. From my recent experience, most of the top teams here on Kaggle are using PyTorch.", "590927": "I switched from Keras to Catalyst and now life is easier for me (especially, when it comes to custom layers and debugging)", "592215": "If you use Tensorflow 2.0, I think it's not a matter which you use ,but if you use Tensorflow 1.X, oh my god , please just give up it , it's the world's disaster . ", "591972": "Most people say that and I quote \"Experiment with Pytorch and use TensorFlow for deployment\".", "592631": "I started with keras because it close to sklearn implementation.\nI use pytorch now because of fast.ai ", "592473": "It's worth trying at least to see how PyTorch ideology fits your needs and style of coding. It's better to be diverse and play with TF/Keras/PyTorch/etc. Each lib has it's pros & cons.", "592432": "Keras is really good for straightforward simple model training but you will have to face a nightmare in terms of struggle if the computation graph or training pipeline is a bit complicated. I think for beginners, Keras is a good choice and Pytorch is better for intermediate or advance usage.", "591970": "For me, there were a few reasons:\nDataloading/handling during the training phase is easier in pytorch IMO, you only need to specify like 2 things for datasets(__len__ and forward) and you're good. No need to return stuff in a specific format because you can handle it however you want in the training loop.\nEasier debugging. This has already been mentioned\nMore research implementations available in pytorch than in keras.", "591179": "Here's a recent tweet from Jeremy \n> That's quite astonishing. If I'm reading this right, PyTorch grew 194% in research use last year (based on arxiv mentions) and TensorFlow only grew 23%, meaning they're now neck and neck. In another year TensorFlow may just be a small minority player in research\n\n[tweet link](https://twitter.com/jeremyphoward/status/1155043840966815745)\n", "591080": "I switched to Pytorch from Keras/TF, from my personal perspective Pytorch is more flexible, makes debugging easier and has abundant documentation. It has grown more and more popular in the research community in recent years.", "591297": "" } }