Hi guys,
\n\nWhat is the minimum hardware config needed to participate in this competition? I haven't dealt with a 4gb data before. But I really feel this competition is very interesting and want to participate. I want to make appropriate hardware arrangements but I fear I will fall short and hence the question.
\n\nRegards,\nChaitanya.
", "messages": [ { "id": "120182", "postDate": "05/16/2016 06:20:45", "content": "Hi guys,
\n\nWhat is the minimum hardware config needed to participate in this competition? I haven't dealt with a 4gb data before. But I really feel this competition is very interesting and want to participate. I want to make appropriate hardware arrangements but I fear I will fall short and hence the question.
\n\nRegards,\nChaitanya.
", "rawMarkdown": "Hi guys,\r\n\r\nWhat is the minimum hardware config needed to participate in this competition? I haven't dealt with a 4gb data before. But I really feel this competition is very interesting and want to participate. I want to make appropriate hardware arrangements but I fear I will fall short and hence the question.\r\n\r\nRegards,\r\nChaitanya.", "votes": null }, { "id": "120187", "postDate": "05/16/2016 07:35:19", "content": "You certainly need GPU.\nI was using my macbook pro with 8GB RAM and I couldn't load and process the data.
\n\nI am now using 16GB RAM with NVIDIA 960M, works like charm, but still I have to evaluate in batches because even my 4 GB GPU runs out of memory.
", "rawMarkdown": "You certainly need GPU.\r\nI was using my macbook pro with 8GB RAM and I couldn't load and process the data.\r\n\r\nI am now using 16GB RAM with NVIDIA 960M, works like charm, but still I have to evaluate in batches because even my 4 GB GPU runs out of memory.", "votes": null }, { "id": "120191", "postDate": "05/16/2016 08:57:28", "content": "@Asymptote, thanks for the reply.
\n\nGPU's are quite costly. NVIDIA 960M goes upto 780$. I will surely buy a GPU as and when possible but are there any other options available like computing on AWS cloud?
", "rawMarkdown": "Asymptote, thanks for the reply.\r\n\r\nGPU's are quite costly. NVIDIA 960M goes upto 780$. I will surely buy a GPU as and when possible but are there any other options available like computing on AWS cloud?", "votes": null }, { "id": "120194", "postDate": "05/16/2016 09:05:42", "content": "GPUs are necessary for deep-learning as image datasets can get quite large. I have not used AWS.
\n\nHere are a few links:\nhttps://www.kaggle.com/forums/f/208/getting-started/t/11505/an-aws-ami-mainly-for-deep-learning
\n\nhttps://www.kaggle.com/c/facial-keypoints-detection/details/deep-learning-tutorial
", "rawMarkdown": "GPUs are necessary for deep-learning as image datasets can get quite large. I have not used AWS.\r\n\r\nHere are a few links:\r\nhttps://www.kaggle.com/forums/f/208/getting-started/t/11505/an-aws-ami-mainly-for-deep-learning\r\n\r\nhttps://www.kaggle.com/c/facial-keypoints-detection/details/deep-learning-tutorial", "votes": null }, { "id": "120224", "postDate": "05/16/2016 14:59:59", "content": "The bigger machines are needed for the VGG-16 pretrained model.\nI have 4GB of RAM (the other 4 went bad :( ) a 12 GB swap file and a GTX 760 I found used for $200. With very little effort I will probably make it to an LB score of 0.5 using a meta tuned CV'd stacked ensemble.\nI obviously won't be able to run the VGG-16 enough times for good tuning though. VGG-16 is theoretically not needed though. This competition only has a few thousand examples of 10 classes. not 10k of 1k classes. Whatever model wins this competition probably won't be compressed but I'd be it could be compressed into a tiny fraction of the size of VGG-16. VGG-16 is just a training shortcut. $400 is probably enough to build an entire rig and have fun goofing around in this competition.
", "rawMarkdown": "The bigger machines are needed for the VGG-16 pretrained model.\r\nI have 4GB of RAM (the other 4 went bad :( ) a 12 GB swap file and a GTX 760 I found used for $200. With very little effort I will probably make it to an LB score of 0.5 using a meta tuned CV'd stacked ensemble.\r\nI obviously won't be able to run the VGG-16 enough times for good tuning though. VGG-16 is theoretically not needed though. This competition only has a few thousand examples of 10 classes. not 10k of 1k classes. Whatever model wins this competition probably won't be compressed but I'd be it could be compressed into a tiny fraction of the size of VGG-16. VGG-16 is just a training shortcut. $400 is probably enough to build an entire rig and have fun goofing around in this competition.", "votes": null }, { "id": "121052", "postDate": "05/23/2016 09:21:13", "content": "Using MacBook Pro i5 4GB ram... Got LB ~1.50 after training for 30 min in keras...
", "rawMarkdown": "Using MacBook Pro i5 4GB ram... Got LB ~1.50 after training for 30 min in keras...", "votes": null }, { "id": "121125", "postDate": "05/24/2016 07:14:15", "content": "No GPU?
", "rawMarkdown": "No GPU?", "votes": null }, { "id": "121347", "postDate": "05/25/2016 19:09:43", "content": "[quote=ChaitanyaGokhale;121125]
\n\nNo GPU?
\n\n[/quote]
\n\nThat seems quite reasonable. 1.5 is never going to be a winning score. It corresponds to pretty low accuracies too - e.g. 0.3 or so. But it does demonstrate some ML has learned something about the problem in a relatively short period of time.
\n\nGetting better scores gets progressively harder, and without a GPU to at least explore the possibilities, I doubt anyone will post something competitive on just a CPU.
\n\nI have got a score of 0.89 without GPU, starting from ZFTurbo's script. It took ~36 hours to train that, though and I've shelved the competition for now, as I don't have easy access to GPU and don't want to pay for AWS.
", "rawMarkdown": "[quote=ChaitanyaGokhale;121125]\r\n\r\nNo GPU?\r\n\r\n[/quote]\r\n\r\nThat seems quite reasonable. 1.5 is never going to be a winning score. It corresponds to pretty low accuracies too - e.g. 0.3 or so. But it does demonstrate some ML has learned *something* about the problem in a relatively short period of time.\r\n\r\nGetting better scores gets progressively harder, and without a GPU to at least explore the possibilities, I doubt anyone will post something competitive on just a CPU.\r\n\r\nI have got a score of 0.89 without GPU, starting from ZFTurbo's script. It took ~36 hours to train that, though and I've shelved the competition for now, as I don't have easy access to GPU and don't want to pay for AWS.", "votes": null }, { "id": "121413", "postDate": "05/26/2016 07:02:43", "content": "@Neil
\n\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.
\n\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?
", "rawMarkdown": "Neil\r\n\r\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.\r\n\r\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?", "votes": null }, { "id": "121561", "postDate": "05/27/2016 11:43:23", "content": "[quote=ChaitanyaGokhale;121413]
\n\n@Neil
\n\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.
\n\n[/quote]
\n\n4 years old MacBook Pro i7, 8GB RAM.
\n\nThe basic script does run nice and fast, which is a good decision by ZFTurbo - it shows the format of a usable script, does a lot of hard work in designing/writing the script, and you can focus instead on the model and additional extras you want to try.
\n\nYou will want larger network, more iterations, perhaps to augment the data. Each of these things is computationally expensive to add. Exponentially so. Every time you gain some small amount in the score, the time to train will likely double . . .
\n\n[quote=ChaitanyaGokhale;121413]
\n\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?
\n\n[/quote]
\n\nProbably yes, but depends what is published. It may be possible to find some good meta-params that train in reasonable time using GPU-assisted NNs to search for the best meta-params, and then publish that script. A similar search on a CPU rig could take weeks or months.
\n\nPointers: Try variations in number of feature maps, layers, pooling layers. Neural networks tend very much to over-fit, and that is not helped by the small number of drivers (so the classes we want to predict don't have much variation compared to target population of all drivers). So look at the options for regularisation. Plot and try to get a feel for the learning curves (training logloss and validation logloss plotted against epoch number) as you vary meta-params.
", "rawMarkdown": "[quote=ChaitanyaGokhale;121413]\r\n\r\n@Neil\r\n\r\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.\r\n\r\n[/quote]\r\n\r\n4 years old MacBook Pro i7, 8GB RAM.\r\n\r\nThe basic script does run nice and fast, which is a good decision by ZFTurbo - it shows the format of a usable script, does a lot of hard work in designing/writing the script, and you can focus instead on the model and additional extras you want to try.\r\n\r\nYou will want larger network, more iterations, perhaps to augment the data. Each of these things is computationally expensive to add. Exponentially so. Every time you gain some small amount in the score, the time to train will likely double . . .\r\n\r\n[quote=ChaitanyaGokhale;121413]\r\n\r\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?\r\n\r\n[/quote]\r\n\r\nProbably yes, but depends what is published. It may be possible to *find* some good meta-params that train in reasonable time using GPU-assisted NNs to search for the best meta-params, and then publish that script. A similar search on a CPU rig could take weeks or months.\r\n\r\nPointers: Try variations in number of feature maps, layers, pooling layers. Neural networks tend very much to over-fit, and that is not helped by the small number of drivers (so the classes we want to predict don't have much variation compared to target population of all drivers). So look at the options for regularisation. Plot and try to get a feel for the learning curves (training logloss and validation logloss plotted against epoch number) as you vary meta-params.", "votes": null } ], "comments": [ { "id": 120187, "author_name": "asymptote", "author_url": "", "post_date": "05/16/2016 07:35:19", "content": "You certainly need GPU.\nI was using my macbook pro with 8GB RAM and I couldn't load and process the data.
\n\nI am now using 16GB RAM with NVIDIA 960M, works like charm, but still I have to evaluate in batches because even my 4 GB GPU runs out of memory.
", "votes": null, "replies": [] }, { "id": 120191, "author_name": "statchaitya", "author_url": "", "post_date": "05/16/2016 08:57:28", "content": "@Asymptote, thanks for the reply.
\n\nGPU's are quite costly. NVIDIA 960M goes upto 780$. I will surely buy a GPU as and when possible but are there any other options available like computing on AWS cloud?
", "votes": null, "replies": [] }, { "id": 120194, "author_name": "asymptote", "author_url": "", "post_date": "05/16/2016 09:05:42", "content": "GPUs are necessary for deep-learning as image datasets can get quite large. I have not used AWS.
\n\nHere are a few links:\nhttps://www.kaggle.com/forums/f/208/getting-started/t/11505/an-aws-ami-mainly-for-deep-learning
\n\nhttps://www.kaggle.com/c/facial-keypoints-detection/details/deep-learning-tutorial
", "votes": null, "replies": [] }, { "id": 120224, "author_name": "bschumacher", "author_url": "", "post_date": "05/16/2016 14:59:59", "content": "The bigger machines are needed for the VGG-16 pretrained model.\nI have 4GB of RAM (the other 4 went bad :( ) a 12 GB swap file and a GTX 760 I found used for $200. With very little effort I will probably make it to an LB score of 0.5 using a meta tuned CV'd stacked ensemble.\nI obviously won't be able to run the VGG-16 enough times for good tuning though. VGG-16 is theoretically not needed though. This competition only has a few thousand examples of 10 classes. not 10k of 1k classes. Whatever model wins this competition probably won't be compressed but I'd be it could be compressed into a tiny fraction of the size of VGG-16. VGG-16 is just a training shortcut. $400 is probably enough to build an entire rig and have fun goofing around in this competition.
", "votes": null, "replies": [] }, { "id": 121052, "author_name": "shahnawazakhtar", "author_url": "", "post_date": "05/23/2016 09:21:13", "content": "Using MacBook Pro i5 4GB ram... Got LB ~1.50 after training for 30 min in keras...
", "votes": null, "replies": [] }, { "id": 121125, "author_name": "statchaitya", "author_url": "", "post_date": "05/24/2016 07:14:15", "content": "No GPU?
", "votes": null, "replies": [] }, { "id": 121347, "author_name": "slobo777", "author_url": "", "post_date": "05/25/2016 19:09:43", "content": "[quote=ChaitanyaGokhale;121125]
\n\nNo GPU?
\n\n[/quote]
\n\nThat seems quite reasonable. 1.5 is never going to be a winning score. It corresponds to pretty low accuracies too - e.g. 0.3 or so. But it does demonstrate some ML has learned something about the problem in a relatively short period of time.
\n\nGetting better scores gets progressively harder, and without a GPU to at least explore the possibilities, I doubt anyone will post something competitive on just a CPU.
\n\nI have got a score of 0.89 without GPU, starting from ZFTurbo's script. It took ~36 hours to train that, though and I've shelved the competition for now, as I don't have easy access to GPU and don't want to pay for AWS.
", "votes": null, "replies": [] }, { "id": 121413, "author_name": "statchaitya", "author_url": "", "post_date": "05/26/2016 07:02:43", "content": "@Neil
\n\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.
\n\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?
", "votes": null, "replies": [] }, { "id": 121561, "author_name": "slobo777", "author_url": "", "post_date": "05/27/2016 11:43:23", "content": "[quote=ChaitanyaGokhale;121413]
\n\n@Neil
\n\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.
\n\n[/quote]
\n\n4 years old MacBook Pro i7, 8GB RAM.
\n\nThe basic script does run nice and fast, which is a good decision by ZFTurbo - it shows the format of a usable script, does a lot of hard work in designing/writing the script, and you can focus instead on the model and additional extras you want to try.
\n\nYou will want larger network, more iterations, perhaps to augment the data. Each of these things is computationally expensive to add. Exponentially so. Every time you gain some small amount in the score, the time to train will likely double . . .
\n\n[quote=ChaitanyaGokhale;121413]
\n\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?
\n\n[/quote]
\n\nProbably yes, but depends what is published. It may be possible to find some good meta-params that train in reasonable time using GPU-assisted NNs to search for the best meta-params, and then publish that script. A similar search on a CPU rig could take weeks or months.
\n\nPointers: Try variations in number of feature maps, layers, pooling layers. Neural networks tend very much to over-fit, and that is not helped by the small number of drivers (so the classes we want to predict don't have much variation compared to target population of all drivers). So look at the options for regularisation. Plot and try to get a feel for the learning curves (training logloss and validation logloss plotted against epoch number) as you vary meta-params.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "120182": "Hi guys,\r\n\r\nWhat is the minimum hardware config needed to participate in this competition? I haven't dealt with a 4gb data before. But I really feel this competition is very interesting and want to participate. I want to make appropriate hardware arrangements but I fear I will fall short and hence the question.\r\n\r\nRegards,\r\nChaitanya.", "120187": "You certainly need GPU.\r\nI was using my macbook pro with 8GB RAM and I couldn't load and process the data.\r\n\r\nI am now using 16GB RAM with NVIDIA 960M, works like charm, but still I have to evaluate in batches because even my 4 GB GPU runs out of memory.", "120191": "Asymptote, thanks for the reply.\r\n\r\nGPU's are quite costly. NVIDIA 960M goes upto 780$. I will surely buy a GPU as and when possible but are there any other options available like computing on AWS cloud?", "120194": "GPUs are necessary for deep-learning as image datasets can get quite large. I have not used AWS.\r\n\r\nHere are a few links:\r\nhttps://www.kaggle.com/forums/f/208/getting-started/t/11505/an-aws-ami-mainly-for-deep-learning\r\n\r\nhttps://www.kaggle.com/c/facial-keypoints-detection/details/deep-learning-tutorial", "120224": "The bigger machines are needed for the VGG-16 pretrained model.\r\nI have 4GB of RAM (the other 4 went bad :( ) a 12 GB swap file and a GTX 760 I found used for $200. With very little effort I will probably make it to an LB score of 0.5 using a meta tuned CV'd stacked ensemble.\r\nI obviously won't be able to run the VGG-16 enough times for good tuning though. VGG-16 is theoretically not needed though. This competition only has a few thousand examples of 10 classes. not 10k of 1k classes. Whatever model wins this competition probably won't be compressed but I'd be it could be compressed into a tiny fraction of the size of VGG-16. VGG-16 is just a training shortcut. $400 is probably enough to build an entire rig and have fun goofing around in this competition.", "121052": "Using MacBook Pro i5 4GB ram... Got LB ~1.50 after training for 30 min in keras...", "121125": "No GPU?", "121347": "[quote=ChaitanyaGokhale;121125]\r\n\r\nNo GPU?\r\n\r\n[/quote]\r\n\r\nThat seems quite reasonable. 1.5 is never going to be a winning score. It corresponds to pretty low accuracies too - e.g. 0.3 or so. But it does demonstrate some ML has learned *something* about the problem in a relatively short period of time.\r\n\r\nGetting better scores gets progressively harder, and without a GPU to at least explore the possibilities, I doubt anyone will post something competitive on just a CPU.\r\n\r\nI have got a score of 0.89 without GPU, starting from ZFTurbo's script. It took ~36 hours to train that, though and I've shelved the competition for now, as I don't have easy access to GPU and don't want to pay for AWS.", "121413": "Neil\r\n\r\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.\r\n\r\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?", "121561": "[quote=ChaitanyaGokhale;121413]\r\n\r\n@Neil\r\n\r\nWhat hardware are you using? I am able to run ZFTurbo's basic script pretty fast (10-15 mins) on Windows 7 64-bit i5 8GB RAM.\r\n\r\n[/quote]\r\n\r\n4 years old MacBook Pro i7, 8GB RAM.\r\n\r\nThe basic script does run nice and fast, which is a good decision by ZFTurbo - it shows the format of a usable script, does a lot of hard work in designing/writing the script, and you can focus instead on the model and additional extras you want to try.\r\n\r\nYou will want larger network, more iterations, perhaps to augment the data. Each of these things is computationally expensive to add. Exponentially so. Every time you gain some small amount in the score, the time to train will likely double . . .\r\n\r\n[quote=ChaitanyaGokhale;121413]\r\n\r\nBesides the computation, could you give some hints on how to proceed after I fully understand ZFTurbo's keras script. Some pointers? This is the first time I am dealing with Neural Nets. I haven't understood them fully, but I am hoping that as I keep playing the competition I'll get more intuitive about it. My realistic target for the competition is to at least do better than the top public script. I feel that will surely be possible on a CPU, won't it?\r\n\r\n[/quote]\r\n\r\nProbably yes, but depends what is published. It may be possible to *find* some good meta-params that train in reasonable time using GPU-assisted NNs to search for the best meta-params, and then publish that script. A similar search on a CPU rig could take weeks or months.\r\n\r\nPointers: Try variations in number of feature maps, layers, pooling layers. Neural networks tend very much to over-fit, and that is not helped by the small number of drivers (so the classes we want to predict don't have much variation compared to target population of all drivers). So look at the options for regularisation. Plot and try to get a feel for the learning curves (training logloss and validation logloss plotted against epoch number) as you vary meta-params." }, "source": "meta" }