Are most people running their codes on their desktop computers? My computer has an i7-4790 CPU but no graphics card or GPUs. I'm guessing this would be fine for doing this competition given that there are 4 months remaining, but since I'm still a bit new to machine learning I'm guessing I will have to tweak my code and rerun it more often than most competitors. I'm thinking about using AWS to speed up the time for my iterations, but have never used it before. It doesn't seem too difficult to learn, but I'm wondering: is this something people would recommend and what sort of hardware and environments is everyone else using for this competition?
", "messages": [ { "id": "67376", "postDate": "03/20/2015 06:35:23", "content": "Are most people running their codes on their desktop computers? My computer has an i7-4790 CPU but no graphics card or GPUs. I'm guessing this would be fine for doing this competition given that there are 4 months remaining, but since I'm still a bit new to machine learning I'm guessing I will have to tweak my code and rerun it more often than most competitors. I'm thinking about using AWS to speed up the time for my iterations, but have never used it before. It doesn't seem too difficult to learn, but I'm wondering: is this something people would recommend and what sort of hardware and environments is everyone else using for this competition?
", "rawMarkdown": "", "votes": null }, { "id": "67379", "postDate": "03/20/2015 07:03:14", "content": "https://timdettmers.wordpress.com/2015/03/09/deep-learning-hardware-guide/
", "rawMarkdown": "", "votes": null }, { "id": "67380", "postDate": "03/20/2015 07:22:37", "content": "Hi old-ufo,
\nI saw that guide posted on the main Kaggle forum. I was asking for this problem specifically to see how other people were handling this dataset and if they were able to find ways to reduce the computational load on their computer for this problem specifically so that they didn't need (well, it's not necessary, obviously, so much as beneficial) computational power that I don't currently have or if it was generally accepted that this problem might need more computational power for someone running a number of iterations such as myself.
", "rawMarkdown": "", "votes": null }, { "id": "67388", "postDate": "03/20/2015 10:08:01", "content": "J Kolb - it's going to depend on what kind of model you're aiming to use. If deep learning, I think the GPU is pretty much essential. It's not just about absolute model estimation time, it's also about the development cycle. As this is a pretty chunky image analysis problem it's probably going to require a fairly serious desktop at the minimum (I've not fully committed to participating yet myself).
", "rawMarkdown": "", "votes": null }, { "id": "67656", "postDate": "03/22/2015 23:52:06", "content": "I have a laptop (HP EliteBook 2760p) with Intel HD Graphics 3000. Is this going to be useless for running convnets? According to the link posted it seems GPU is essential for deep learning.
\nThe laptop has a i7-2620M CPU @2.70Ghz and 8GB RAM.
", "rawMarkdown": "", "votes": null }, { "id": "67674", "postDate": "03/23/2015 03:43:19", "content": "I just switched from an Amazon g2.2xlarge to a GTX 980 - the training times are now cut in half. Spot price for g2.2xlarge is usually around 6.8 cents per hour (does that even cover the cost of their electricity?). Amazon is a good option if you can handle the headache of the cuda installation, or find a suitable AMI.
", "rawMarkdown": "", "votes": null }, { "id": "67724", "postDate": "03/23/2015 16:38:36", "content": "I'm surprised you got it that cheap; it was ~13-14 cents when I switched over to using them. Good to know the 980 was that that much faster; slow model training was a serious development bottleneck in the previous competition,so I will probably invest in one.
\nOlivier Grisel just released an AMI : https://twitter.com/ogrisel/status/569985979492241408
", "rawMarkdown": "", "votes": null }, { "id": "67749", "postDate": "03/23/2015 18:26:24", "content": "[quote=Torgos;67724]
\n...Good to know the 980 was that that much faster; slow model training was a serious development bottleneck in the previous competition,so I will probably invest in one.
\nOlivier Grisel just released an AMI : https://twitter.com/ogrisel/status/569985979492241408
\n[/quote]
\nI'm happy with the 980. There is almost no noise, I can't even tell from the sound if it's running or not. A word of caution if you've never installed a gpu before; be sure everything is backed up, because if you lose all your graphics a reinstall of the OS may be the only way to get them back. I considered installing the gpu headless and continuing to drive the monitors with the onboard graphics, but I'm glad I didn't because now I have a lot of cool video games to play.
", "rawMarkdown": "", "votes": null }, { "id": "67944", "postDate": "03/24/2015 11:31:54", "content": "[quote=James King;67674]
\nI just switched from an Amazon g2.2xlarge to a GTX 980 - the training times are now cut in half. Spot price for g2.2xlarge is usually around 6.8 cents per hour (does that even cover the cost of their electricity?). Amazon is a good option if you can handle the headache of the cuda installation, or find a suitable AMI.
\n[/quote]
\n\nJames
\ndid you have a chance to evaluate your electricity costs yet with the GTX980 ?
", "rawMarkdown": "", "votes": null }, { "id": "67954", "postDate": "03/24/2015 12:41:04", "content": "[quote=knarfben;67944]
\nJames
\ndid you have a chance to evaluate your electricity costs yet with the GTX980 ?
\n[/quote]
\nNo the bill has not come yet.
", "rawMarkdown": "", "votes": null }, { "id": "68036", "postDate": "03/24/2015 18:10:20", "content": "@James what OS are u using and why do u need to reinstall the OS? Can't you just reinstall cuda + driver?
", "rawMarkdown": "", "votes": null }, { "id": "68047", "postDate": "03/24/2015 19:16:08", "content": "Ubuntu. The graphics didn't come back when I reinstalled the NVIDIA driver, or any of the other video drivers I tried. I'm not sure what I screwed up to lose the graphics in the first place.
\nAlso I had to turn off the onboard intel drivers before the gpu driven graphics would come up correctly.
", "rawMarkdown": "", "votes": null }, { "id": "68084", "postDate": "03/24/2015 21:23:53", "content": "For Ubuntu there are several forums on this topic if you just google it. You shouldn't have reinstalled the OS. The problem is on the Nvidia driver version + cuda version and ubuntu loading the nouveau driver by default.
\nSo, all you got to do was uninstall/purge anything related to nvidia, reinstall the correct version of the graphics driver + cuda (see this document for driver + cuda version matches: http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/#axzz3VLACaxRg), and use nvidia's prime-select package to switch between the intel and the nvidia graphics if you happen to use a discrete + integrated graphics.
\nNote: Nvidia doesn't still Officially target Linux and there might still be some issues. For instance, I tried to add 4 monitors using driver version 347.25 and that shuts down the system immediately. Apparently, I cannot add more than 3 monitors although my 980 can support up to 5 monitors.
\nI hope this helps someone who runs in to the same issue.
", "rawMarkdown": "", "votes": null }, { "id": "68180", "postDate": "03/25/2015 09:32:06", "content": "This is how all this sound to me : 中国似乎对我
", "rawMarkdown": "", "votes": null }, { "id": "68465", "postDate": "03/26/2015 19:36:08", "content": "James, as Deep says a reinstall shouldn't be necessary. I've spent time than I care to admit sparring with Ubuntu/Mint and nvidia drivers and in my experience a graphics failure usually 'just' requires some combination of:
\n( - back up your working X11, blacklist, video config files! )
\n- Drop into a VT console as root
\n- apt-get purge all the nvidia stuff
\n- reinstall from the nvidia installer, not the package manager.
\n- double check that the kernel module has been loaded correctly
\n- mod probe for extraneous nvidia versions and any mentions of nouveau and blacklist them.
\n- compile and run the cuda hardware tests to see if it's working properly.
\nIn my experiences the problem has almost always been some small breakage introduced by combining your specific kernel, the version of the cudatoolkit installed, and the minor version of the nvidia driver. For some combinations the nvidia installer works, for some the package manager works, and for some neither will work entirely but you can do things like manually force the kernel module to load etc. That said, if you don't have a lot of other stuff configured on the box it could very well be easier to start from scratch.
\nFor what it's worth wrt to the thread topic, I'm running (although I haven't really dug in yet) on g2.2xlarge instances using an AMI that I forked from one of the public ubuntu+caffe instances (I like to use a different python stack, as well as some of the theano based libs, as opposed to just caffe). I'm choosing to use EC2 for this challenge because the data size is fairly large and I'm predicting that I will spend a long time on preprocessing steps. For these purposes I can scale out CPU intensive tasks to a cluster of big beefy 36 core boxes and have things done in a fraction of the time compared to any local setup I have available.
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 67379, "author_name": "oldufo", "author_url": "", "post_date": "03/20/2015 07:03:14", "content": "https://timdettmers.wordpress.com/2015/03/09/deep-learning-hardware-guide/
", "votes": null, "replies": [] }, { "id": 67380, "author_name": "jessekolb", "author_url": "", "post_date": "03/20/2015 07:22:37", "content": "Hi old-ufo,
\nI saw that guide posted on the main Kaggle forum. I was asking for this problem specifically to see how other people were handling this dataset and if they were able to find ways to reduce the computational load on their computer for this problem specifically so that they didn't need (well, it's not necessary, obviously, so much as beneficial) computational power that I don't currently have or if it was generally accepted that this problem might need more computational power for someone running a number of iterations such as myself.
", "votes": null, "replies": [] }, { "id": 67388, "author_name": "mpearce", "author_url": "", "post_date": "03/20/2015 10:08:01", "content": "J Kolb - it's going to depend on what kind of model you're aiming to use. If deep learning, I think the GPU is pretty much essential. It's not just about absolute model estimation time, it's also about the development cycle. As this is a pretty chunky image analysis problem it's probably going to require a fairly serious desktop at the minimum (I've not fully committed to participating yet myself).
", "votes": null, "replies": [] }, { "id": 67656, "author_name": "siddjain", "author_url": "", "post_date": "03/22/2015 23:52:06", "content": "I have a laptop (HP EliteBook 2760p) with Intel HD Graphics 3000. Is this going to be useless for running convnets? According to the link posted it seems GPU is essential for deep learning.
\nThe laptop has a i7-2620M CPU @2.70Ghz and 8GB RAM.
", "votes": null, "replies": [] }, { "id": 67674, "author_name": "jfkingiii", "author_url": "", "post_date": "03/23/2015 03:43:19", "content": "I just switched from an Amazon g2.2xlarge to a GTX 980 - the training times are now cut in half. Spot price for g2.2xlarge is usually around 6.8 cents per hour (does that even cover the cost of their electricity?). Amazon is a good option if you can handle the headache of the cuda installation, or find a suitable AMI.
", "votes": null, "replies": [] }, { "id": 67724, "author_name": "telser", "author_url": "", "post_date": "03/23/2015 16:38:36", "content": "I'm surprised you got it that cheap; it was ~13-14 cents when I switched over to using them. Good to know the 980 was that that much faster; slow model training was a serious development bottleneck in the previous competition,so I will probably invest in one.
\nOlivier Grisel just released an AMI : https://twitter.com/ogrisel/status/569985979492241408
", "votes": null, "replies": [] }, { "id": 67749, "author_name": "jfkingiii", "author_url": "", "post_date": "03/23/2015 18:26:24", "content": "[quote=Torgos;67724]
\n...Good to know the 980 was that that much faster; slow model training was a serious development bottleneck in the previous competition,so I will probably invest in one.
\nOlivier Grisel just released an AMI : https://twitter.com/ogrisel/status/569985979492241408
\n[/quote]
\nI'm happy with the 980. There is almost no noise, I can't even tell from the sound if it's running or not. A word of caution if you've never installed a gpu before; be sure everything is backed up, because if you lose all your graphics a reinstall of the OS may be the only way to get them back. I considered installing the gpu headless and continuing to drive the monitors with the onboard graphics, but I'm glad I didn't because now I have a lot of cool video games to play.
", "votes": null, "replies": [] }, { "id": 67944, "author_name": "knarfben", "author_url": "", "post_date": "03/24/2015 11:31:54", "content": "[quote=James King;67674]
\nI just switched from an Amazon g2.2xlarge to a GTX 980 - the training times are now cut in half. Spot price for g2.2xlarge is usually around 6.8 cents per hour (does that even cover the cost of their electricity?). Amazon is a good option if you can handle the headache of the cuda installation, or find a suitable AMI.
\n[/quote]
\n\nJames
\ndid you have a chance to evaluate your electricity costs yet with the GTX980 ?
", "votes": null, "replies": [] }, { "id": 67954, "author_name": "jfkingiii", "author_url": "", "post_date": "03/24/2015 12:41:04", "content": "[quote=knarfben;67944]
\nJames
\ndid you have a chance to evaluate your electricity costs yet with the GTX980 ?
\n[/quote]
\nNo the bill has not come yet.
", "votes": null, "replies": [] }, { "id": 68036, "author_name": "deepcnn", "author_url": "", "post_date": "03/24/2015 18:10:20", "content": "@James what OS are u using and why do u need to reinstall the OS? Can't you just reinstall cuda + driver?
", "votes": null, "replies": [] }, { "id": 68047, "author_name": "jfkingiii", "author_url": "", "post_date": "03/24/2015 19:16:08", "content": "Ubuntu. The graphics didn't come back when I reinstalled the NVIDIA driver, or any of the other video drivers I tried. I'm not sure what I screwed up to lose the graphics in the first place.
\nAlso I had to turn off the onboard intel drivers before the gpu driven graphics would come up correctly.
", "votes": null, "replies": [] }, { "id": 68084, "author_name": "deepcnn", "author_url": "", "post_date": "03/24/2015 21:23:53", "content": "For Ubuntu there are several forums on this topic if you just google it. You shouldn't have reinstalled the OS. The problem is on the Nvidia driver version + cuda version and ubuntu loading the nouveau driver by default.
\nSo, all you got to do was uninstall/purge anything related to nvidia, reinstall the correct version of the graphics driver + cuda (see this document for driver + cuda version matches: http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/#axzz3VLACaxRg), and use nvidia's prime-select package to switch between the intel and the nvidia graphics if you happen to use a discrete + integrated graphics.
\nNote: Nvidia doesn't still Officially target Linux and there might still be some issues. For instance, I tried to add 4 monitors using driver version 347.25 and that shuts down the system immediately. Apparently, I cannot add more than 3 monitors although my 980 can support up to 5 monitors.
\nI hope this helps someone who runs in to the same issue.
", "votes": null, "replies": [] }, { "id": 68180, "author_name": "kazanova", "author_url": "", "post_date": "03/25/2015 09:32:06", "content": "This is how all this sound to me : 中国似乎对我
", "votes": null, "replies": [] }, { "id": 68465, "author_name": "kyzyl295079", "author_url": "", "post_date": "03/26/2015 19:36:08", "content": "James, as Deep says a reinstall shouldn't be necessary. I've spent time than I care to admit sparring with Ubuntu/Mint and nvidia drivers and in my experience a graphics failure usually 'just' requires some combination of:
\n( - back up your working X11, blacklist, video config files! )
\n- Drop into a VT console as root
\n- apt-get purge all the nvidia stuff
\n- reinstall from the nvidia installer, not the package manager.
\n- double check that the kernel module has been loaded correctly
\n- mod probe for extraneous nvidia versions and any mentions of nouveau and blacklist them.
\n- compile and run the cuda hardware tests to see if it's working properly.
\nIn my experiences the problem has almost always been some small breakage introduced by combining your specific kernel, the version of the cudatoolkit installed, and the minor version of the nvidia driver. For some combinations the nvidia installer works, for some the package manager works, and for some neither will work entirely but you can do things like manually force the kernel module to load etc. That said, if you don't have a lot of other stuff configured on the box it could very well be easier to start from scratch.
\nFor what it's worth wrt to the thread topic, I'm running (although I haven't really dug in yet) on g2.2xlarge instances using an AMI that I forked from one of the public ubuntu+caffe instances (I like to use a different python stack, as well as some of the theano based libs, as opposed to just caffe). I'm choosing to use EC2 for this challenge because the data size is fairly large and I'm predicting that I will spend a long time on preprocessing steps. For these purposes I can scale out CPU intensive tasks to a cluster of big beefy 36 core boxes and have things done in a fraction of the time compared to any local setup I have available.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "67376": "", "67379": "", "67380": "", "67388": "", "67656": "", "67674": "", "67724": "", "67749": "", "67944": "", "67954": "", "68036": "", "68047": "", "68084": "", "68180": "", "68465": "" }, "source": "meta" }