I know it's a little bit off-topic, hope it's still okay to post this here.
\n\nSince i ran into quite many hardware limitations in this competition, i would like to ask if you have any recommendations for similar workloads or other competitions on kaggle.
\n\nCurrent System: - Core i5 4590, 16 GB DDR3 RAM, some cheap HDD, GTX 1080\n (no space for dual boot ubuntu on current 128 gb ssd)
\n\nThose problems were e.g. running out of ram when normalizing the data and oversampling (oversampling with float train data), slowly loading datasets/neural nets and high computational times when optimizing features with parametersweeps or similar.
\n\nSo the idea was to upgrade cpu, ram and get a faster harddrive after ryzen 3000 is out and benchmarked. New GPU is not needed. The plan is as follows:
\n\nDoes that kind of build make sense for such tasks? Is a fast M.2 SSD worth it or better get something cheaper? Do RAM timings have any noticable effect? Any Board Recommendations? ...
\n\nThank you for any input!
", "messages": [ { "id": "544444", "postDate": "06/05/2019 14:14:37", "content": "I know it's a little bit off-topic, hope it's still okay to post this here.
\n\nSince i ran into quite many hardware limitations in this competition, i would like to ask if you have any recommendations for similar workloads or other competitions on kaggle.
\n\nCurrent System: - Core i5 4590, 16 GB DDR3 RAM, some cheap HDD, GTX 1080\n (no space for dual boot ubuntu on current 128 gb ssd)
\n\nThose problems were e.g. running out of ram when normalizing the data and oversampling (oversampling with float train data), slowly loading datasets/neural nets and high computational times when optimizing features with parametersweeps or similar.
\n\nSo the idea was to upgrade cpu, ram and get a faster harddrive after ryzen 3000 is out and benchmarked. New GPU is not needed. The plan is as follows:
\n\nDoes that kind of build make sense for such tasks? Is a fast M.2 SSD worth it or better get something cheaper? Do RAM timings have any noticable effect? Any Board Recommendations? ...
\n\nThank you for any input!
", "rawMarkdown": "I know it's a little bit off-topic, hope it's still okay to post this here.\n\nSince i ran into quite many hardware limitations in this competition, i would like to ask if you have any recommendations for similar workloads or other competitions on kaggle. \n\nCurrent System: - Core i5 4590, 16 GB DDR3 RAM, some cheap HDD, GTX 1080\n (no space for dual boot ubuntu on current 128 gb ssd)\n\nThose problems were e.g. running out of ram when normalizing the data and oversampling (oversampling with float train data), slowly loading datasets/neural nets and high computational times when optimizing features with parametersweeps or similar. \n\nSo the idea was to upgrade cpu, ram and get a faster harddrive after ryzen 3000 is out and benchmarked. New GPU is not needed. The plan is as follows:\n\n- CPU: Ryzen 3700/3700x (if good), or a cheap used 2700/2700x\n- RAM: 2x 16 GB DDR4 3000/3200 MHz, cl 16 (64 GB upgrade still possible if needed)\n- SSD: some M.2 SSD \n- Mainboard: some 450/470/550/570 (not sure yet)\n\nDoes that kind of build make sense for such tasks? Is a fast M.2 SSD worth it or better get something cheaper? Do RAM timings have any noticable effect? Any Board Recommendations? ...\n\nThank you for any input!", "votes": null }, { "id": "544841", "postDate": "06/06/2019 00:24:32", "content": "Ah. First things first. You won't need any benchmarks on ryzen 3000. 13% IPC improvement just places it at the level of Intel IPC. The benefit is upgrading to Ryzen 3700 is that is will be better price/performance ratio when compared to Intel. I would not worry too much about single core performance improvements too much. Your main issue is increasing RAM. With 16GB, and GTX 1080, you might as well be running kaggle kernels. 32GB is a good place to start. Doesn't matter what SSD you have, doesn't need to be M.2. If you can afford M.2 go for it. To be honest, I've been upgrading my parts (M.2 ssd, now 64GB RAM, RTX 2070). I hardly use any of it anymore. Once I realized that kaggle kernels get the job done I just stopped worrying about my hardware. I'm currently competing in Analytics Vidhya Computer Vision Hackathon and I'm using Kaggle GPU. Sure there are things I can't do given the RAM limitation (5-fold CV, ZCA whitening, etc.), but I've learned to manage memory and utilize every bit I have more effectively. The only downside with using Kaggle Kernels is peak utilization. But you're going to want to have decent hdd or ssd space to store and read datasets from. It's definitely exciting to have new equipment and hardware, but it fades over time. RTX was a nightmare to setup with tensorflow when I first got it. I put in a ticket to the tensorflow team on GitHub 4-5 months ago and it's still open. Finally, none of the overclocking, timings, GPU overclocking matters for machine learning or deep learning. I performed benchmarks on it a while back and I even posted my findings with the methodology online. The improvements aren't even worth mentioning. I've also had considerably less frustration over hardware and installation issues, software/packages included since I started working on kaggle kernels. The funny thing is, Kaggle GPU has more VRAM than my RTX 2070. I've hardly used RTX since I purchased it for training neural networks. If you do still plan on upgrading, just go with the most cores and most RAM you can afford. You'll end up using GBM alot so the more cores the faster you'll finish training. Anyways, I hope what I'm saying helps.
", "rawMarkdown": "Ah. First things first. You won't need any benchmarks on ryzen 3000. 13% IPC improvement just places it at the level of Intel IPC. The benefit is upgrading to Ryzen 3700 is that is will be better price/performance ratio when compared to Intel. I would not worry too much about single core performance improvements too much. Your main issue is increasing RAM. With 16GB, and GTX 1080, you might as well be running kaggle kernels. 32GB is a good place to start. Doesn't matter what SSD you have, doesn't need to be M.2. If you can afford M.2 go for it. To be honest, I've been upgrading my parts (M.2 ssd, now 64GB RAM, RTX 2070). I hardly use any of it anymore. Once I realized that kaggle kernels get the job done I just stopped worrying about my hardware. I'm currently competing in Analytics Vidhya Computer Vision Hackathon and I'm using Kaggle GPU. Sure there are things I can't do given the RAM limitation (5-fold CV, ZCA whitening, etc.), but I've learned to manage memory and utilize every bit I have more effectively. The only downside with using Kaggle Kernels is peak utilization. But you're going to want to have decent hdd or ssd space to store and read datasets from. It's definitely exciting to have new equipment and hardware, but it fades over time. RTX was a nightmare to setup with tensorflow when I first got it. I put in a ticket to the tensorflow team on GitHub 4-5 months ago and it's still open. Finally, none of the overclocking, timings, GPU overclocking matters for machine learning or deep learning. I performed benchmarks on it a while back and I even posted my findings with the methodology online. The improvements aren't even worth mentioning. I've also had considerably less frustration over hardware and installation issues, software/packages included since I started working on kaggle kernels. The funny thing is, Kaggle GPU has more VRAM than my RTX 2070. I've hardly used RTX since I purchased it for training neural networks. If you do still plan on upgrading, just go with the most cores and most RAM you can afford. You'll end up using GBM alot so the more cores the faster you'll finish training. Anyways, I hope what I'm saying helps.", "votes": null }, { "id": "546027", "postDate": "06/06/2019 07:10:46", "content": "Get M2 drive. Some of them are actually not so expensive. Don't buy crucial P1 drive or intel 660p. These drive slow down to HD speed once the buffer runs out. You can set up large swap space to overcome ram limitation. But Swap is slower than RAM, but your program won't crush. A faster SSD makes swap run faster. I have Ryzen 2600X, and 1080ti, 32GB ram, 30GB Swap (another M2 has 50GB swap, I should change it to 100GB+).
", "rawMarkdown": "Get M2 drive. Some of them are actually not so expensive. Don't buy crucial P1 drive or intel 660p. These drive slow down to HD speed once the buffer runs out. You can set up large swap space to overcome ram limitation. But Swap is slower than RAM, but your program won't crush. A faster SSD makes swap run faster. I have Ryzen 2600X, and 1080ti, 32GB ram, 30GB Swap (another M2 has 50GB swap, I should change it to 100GB+).", "votes": null }, { "id": "546469", "postDate": "06/06/2019 15:52:46", "content": "I agree. Do not buy the the Crucial P1 or Intel 660p. I have Samsung 970 Evo. It works, but I probably could have gone with a non M.2 drive. Storing large datasets are a problem over time. For example the Microsoft Malware 2015 Kaggle challenge is half a Terabyte uncompressed for train and test. I downloaded it and quickly deleted it after realizing I didn't have enough space to work with it.
", "rawMarkdown": "I agree. Do not buy the the Crucial P1 or Intel 660p. I have Samsung 970 Evo. It works, but I probably could have gone with a non M.2 drive. Storing large datasets are a problem over time. For example the Microsoft Malware 2015 Kaggle challenge is half a Terabyte uncompressed for train and test. I downloaded it and quickly deleted it after realizing I didn't have enough space to work with it.", "votes": null }, { "id": "546480", "postDate": "06/06/2019 16:01:17", "content": "Buy AWS. And a crappy comp if u have to
", "rawMarkdown": "Buy AWS. And a crappy comp if u have to", "votes": null }, { "id": "546700", "postDate": "06/06/2019 20:25:27", "content": "I bought a different brand (ADATA XPG SX8200 Pro 1TB ) cheaper than 970 EVO. But this seems to have good reviews, even cheaper and longer endurance rating. I also bought 970 EVO 512GB last year.
\n\n-pla--solid+state+disk-_-N82E16820236477&gclid=Cj0KEQjw8-LnBRCyxtfMl-Cbu48BEiQA6eUMGmgGmNpjNGN0knB6TCWRjtwZJwRhdezwHX2SJFJDJ24aAlMN8P8HAQ&gclsrc=aw.ds\">https://www.newegg.com/corsair-force-mp510-960gb/p/N82E16820236477?item=N82E16820236477&ignorebbr=1&source=region&nm_mc=knc-googleadwords-pc&cm_mmc=knc-googleadwords-pc--pla--solid+state+disk-_-N82E16820236477&gclid=Cj0KEQjw8-LnBRCyxtfMl-Cbu48BEiQA6eUMGmgGmNpjNGN0knB6TCWRjtwZJwRhdezwHX2SJFJDJ24aAlMN8P8HAQ&gclsrc=aw.ds
", "rawMarkdown": "I bought a different brand (ADATA XPG SX8200 Pro 1TB ) cheaper than 970 EVO. But this seems to have good reviews, even cheaper and longer endurance rating. I also bought 970 EVO 512GB last year.\n\nhttps://www.newegg.com/corsair-force-mp510-960gb/p/N82E16820236477?item=N82E16820236477&ignorebbr=1&source=region&nm_mc=knc-googleadwords-pc&cm_mmc=knc-googleadwords-pc-_-pla-_-solid+state+disk-_-N82E16820236477&gclid=Cj0KEQjw8-LnBRCyxtfMl-Cbu48BEiQA6eUMGmgGmNpjNGN0knB6TCWRjtwZJwRhdezwHX2SJFJDJ24aAlMN8P8HAQ&gclsrc=aw.ds", "votes": null }, { "id": "546707", "postDate": "06/06/2019 20:35:33", "content": "It helps a lot, thanks. Maybe i should try kaggle kernels first in the next competition, i honestly haven't really looked at them yet. I don't have to upgrade right away and can wait, but a new SSD is coming sooner or later anyway and when upgrading might as well swap every outdated part and be settled for the next 3-5 years.
", "rawMarkdown": "It helps a lot, thanks. Maybe i should try kaggle kernels first in the next competition, i honestly haven't really looked at them yet. I don't have to upgrade right away and can wait, but a new SSD is coming sooner or later anyway and when upgrading might as well swap every outdated part and be settled for the next 3-5 years.", "votes": null }, { "id": "546711", "postDate": "06/06/2019 20:39:13", "content": "That swap space seems like a good idea, now i just have to read up how it works. I've lost count of how many times my spyder crashed in this comp, that'll definitely help :). I think i actually added something like that when setting up ubuntu, but it was my first time and maybe i missed something there.
", "rawMarkdown": "That swap space seems like a good idea, now i just have to read up how it works. I've lost count of how many times my spyder crashed in this comp, that'll definitely help :). I think i actually added something like that when setting up ubuntu, but it was my first time and maybe i missed something there.", "votes": null }, { "id": "546717", "postDate": "06/06/2019 20:46:41", "content": "Thing is, I use this computer anyway for other stuff like gaming, surfing, office etc. I don't really want to spend extra money for server costs as long as it's not absolutely necessary, but maybe i overestimate aws costs, haven't tried it yet. \nAnd why crappy? If you want to make donations, i'll gladly give you my paypal lol
", "rawMarkdown": "Thing is, I use this computer anyway for other stuff like gaming, surfing, office etc. I don't really want to spend extra money for server costs as long as it's not absolutely necessary, but maybe i overestimate aws costs, haven't tried it yet. \nAnd why crappy? If you want to make donations, i'll gladly give you my paypal lol", "votes": null }, { "id": "546887", "postDate": "06/07/2019 01:56:31", "content": "The other thing you may want to consider is have two machines. One to prototype and another to heavy lifting. I have access to more than 1 machine, sometimes I run variation of different scripts/notebooks at the same time. I found that I can run more scripts using Kaggle kernels much easier and all for free. Don't need multiple computers. The upside to having a decent machine is running scripts for > 9hrs which is Kaggle's current limit. A lot of the computer vision competitions on here require quite a bit of hardware which is a pretty big barrier to entry for many people. Certainly having the best hardware won't mean winning, but it can allow you to work on projects that most can't, even if it is just to learn and have fun.
", "rawMarkdown": "The other thing you may want to consider is have two machines. One to prototype and another to heavy lifting. I have access to more than 1 machine, sometimes I run variation of different scripts/notebooks at the same time. I found that I can run more scripts using Kaggle kernels much easier and all for free. Don't need multiple computers. The upside to having a decent machine is running scripts for > 9hrs which is Kaggle's current limit. A lot of the computer vision competitions on here require quite a bit of hardware which is a pretty big barrier to entry for many people. Certainly having the best hardware won't mean winning, but it can allow you to work on projects that most can't, even if it is just to learn and have fun.", "votes": null }, { "id": "547190", "postDate": "06/07/2019 11:38:33", "content": "Its free for the first year. You should try it. It depends on your needs at the end of the day\nI meant if you are going to spend money on an expensive hardware, than you could make a trade off between less expensive (=crappy) comp and enough AWS credits.
", "rawMarkdown": "Its free for the first year. You should try it. It depends on your needs at the end of the day\nI meant if you are going to spend money on an expensive hardware, than you could make a trade off between less expensive (=crappy) comp and enough AWS credits.", "votes": null } ], "comments": [ { "id": 544841, "author_name": "teeyee314", "author_url": "", "post_date": "06/06/2019 00:24:32", "content": "Ah. First things first. You won't need any benchmarks on ryzen 3000. 13% IPC improvement just places it at the level of Intel IPC. The benefit is upgrading to Ryzen 3700 is that is will be better price/performance ratio when compared to Intel. I would not worry too much about single core performance improvements too much. Your main issue is increasing RAM. With 16GB, and GTX 1080, you might as well be running kaggle kernels. 32GB is a good place to start. Doesn't matter what SSD you have, doesn't need to be M.2. If you can afford M.2 go for it. To be honest, I've been upgrading my parts (M.2 ssd, now 64GB RAM, RTX 2070). I hardly use any of it anymore. Once I realized that kaggle kernels get the job done I just stopped worrying about my hardware. I'm currently competing in Analytics Vidhya Computer Vision Hackathon and I'm using Kaggle GPU. Sure there are things I can't do given the RAM limitation (5-fold CV, ZCA whitening, etc.), but I've learned to manage memory and utilize every bit I have more effectively. The only downside with using Kaggle Kernels is peak utilization. But you're going to want to have decent hdd or ssd space to store and read datasets from. It's definitely exciting to have new equipment and hardware, but it fades over time. RTX was a nightmare to setup with tensorflow when I first got it. I put in a ticket to the tensorflow team on GitHub 4-5 months ago and it's still open. Finally, none of the overclocking, timings, GPU overclocking matters for machine learning or deep learning. I performed benchmarks on it a while back and I even posted my findings with the methodology online. The improvements aren't even worth mentioning. I've also had considerably less frustration over hardware and installation issues, software/packages included since I started working on kaggle kernels. The funny thing is, Kaggle GPU has more VRAM than my RTX 2070. I've hardly used RTX since I purchased it for training neural networks. If you do still plan on upgrading, just go with the most cores and most RAM you can afford. You'll end up using GBM alot so the more cores the faster you'll finish training. Anyways, I hope what I'm saying helps.
", "votes": null, "replies": [ { "id": 546707, "author_name": "svenhinderer", "author_url": "", "post_date": "06/06/2019 20:35:33", "content": "It helps a lot, thanks. Maybe i should try kaggle kernels first in the next competition, i honestly haven't really looked at them yet. I don't have to upgrade right away and can wait, but a new SSD is coming sooner or later anyway and when upgrading might as well swap every outdated part and be settled for the next 3-5 years.
", "votes": null, "replies": [] } ] }, { "id": 546027, "author_name": "joejeo1", "author_url": "", "post_date": "06/06/2019 07:10:46", "content": "Get M2 drive. Some of them are actually not so expensive. Don't buy crucial P1 drive or intel 660p. These drive slow down to HD speed once the buffer runs out. You can set up large swap space to overcome ram limitation. But Swap is slower than RAM, but your program won't crush. A faster SSD makes swap run faster. I have Ryzen 2600X, and 1080ti, 32GB ram, 30GB Swap (another M2 has 50GB swap, I should change it to 100GB+).
", "votes": null, "replies": [ { "id": 546469, "author_name": "teeyee314", "author_url": "", "post_date": "06/06/2019 15:52:46", "content": "I agree. Do not buy the the Crucial P1 or Intel 660p. I have Samsung 970 Evo. It works, but I probably could have gone with a non M.2 drive. Storing large datasets are a problem over time. For example the Microsoft Malware 2015 Kaggle challenge is half a Terabyte uncompressed for train and test. I downloaded it and quickly deleted it after realizing I didn't have enough space to work with it.
", "votes": null, "replies": [] }, { "id": 546711, "author_name": "svenhinderer", "author_url": "", "post_date": "06/06/2019 20:39:13", "content": "That swap space seems like a good idea, now i just have to read up how it works. I've lost count of how many times my spyder crashed in this comp, that'll definitely help :). I think i actually added something like that when setting up ubuntu, but it was my first time and maybe i missed something there.
", "votes": null, "replies": [] } ] }, { "id": 546480, "author_name": "zikazika", "author_url": "", "post_date": "06/06/2019 16:01:17", "content": "Buy AWS. And a crappy comp if u have to
", "votes": null, "replies": [ { "id": 546717, "author_name": "svenhinderer", "author_url": "", "post_date": "06/06/2019 20:46:41", "content": "Thing is, I use this computer anyway for other stuff like gaming, surfing, office etc. I don't really want to spend extra money for server costs as long as it's not absolutely necessary, but maybe i overestimate aws costs, haven't tried it yet. \nAnd why crappy? If you want to make donations, i'll gladly give you my paypal lol
", "votes": null, "replies": [] }, { "id": 547190, "author_name": "zikazika", "author_url": "", "post_date": "06/07/2019 11:38:33", "content": "Its free for the first year. You should try it. It depends on your needs at the end of the day\nI meant if you are going to spend money on an expensive hardware, than you could make a trade off between less expensive (=crappy) comp and enough AWS credits.
", "votes": null, "replies": [] } ] }, { "id": 546700, "author_name": "joejeo1", "author_url": "", "post_date": "06/06/2019 20:25:27", "content": "I bought a different brand (ADATA XPG SX8200 Pro 1TB ) cheaper than 970 EVO. But this seems to have good reviews, even cheaper and longer endurance rating. I also bought 970 EVO 512GB last year.
\n\n-pla--solid+state+disk-_-N82E16820236477&gclid=Cj0KEQjw8-LnBRCyxtfMl-Cbu48BEiQA6eUMGmgGmNpjNGN0knB6TCWRjtwZJwRhdezwHX2SJFJDJ24aAlMN8P8HAQ&gclsrc=aw.ds\">https://www.newegg.com/corsair-force-mp510-960gb/p/N82E16820236477?item=N82E16820236477&ignorebbr=1&source=region&nm_mc=knc-googleadwords-pc&cm_mmc=knc-googleadwords-pc--pla--solid+state+disk-_-N82E16820236477&gclid=Cj0KEQjw8-LnBRCyxtfMl-Cbu48BEiQA6eUMGmgGmNpjNGN0knB6TCWRjtwZJwRhdezwHX2SJFJDJ24aAlMN8P8HAQ&gclsrc=aw.ds
", "votes": null, "replies": [] }, { "id": 546887, "author_name": "teeyee314", "author_url": "", "post_date": "06/07/2019 01:56:31", "content": "The other thing you may want to consider is have two machines. One to prototype and another to heavy lifting. I have access to more than 1 machine, sometimes I run variation of different scripts/notebooks at the same time. I found that I can run more scripts using Kaggle kernels much easier and all for free. Don't need multiple computers. The upside to having a decent machine is running scripts for > 9hrs which is Kaggle's current limit. A lot of the computer vision competitions on here require quite a bit of hardware which is a pretty big barrier to entry for many people. Certainly having the best hardware won't mean winning, but it can allow you to work on projects that most can't, even if it is just to learn and have fun.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "544444": "I know it's a little bit off-topic, hope it's still okay to post this here.\n\nSince i ran into quite many hardware limitations in this competition, i would like to ask if you have any recommendations for similar workloads or other competitions on kaggle. \n\nCurrent System: - Core i5 4590, 16 GB DDR3 RAM, some cheap HDD, GTX 1080\n (no space for dual boot ubuntu on current 128 gb ssd)\n\nThose problems were e.g. running out of ram when normalizing the data and oversampling (oversampling with float train data), slowly loading datasets/neural nets and high computational times when optimizing features with parametersweeps or similar. \n\nSo the idea was to upgrade cpu, ram and get a faster harddrive after ryzen 3000 is out and benchmarked. New GPU is not needed. The plan is as follows:\n\n- CPU: Ryzen 3700/3700x (if good), or a cheap used 2700/2700x\n- RAM: 2x 16 GB DDR4 3000/3200 MHz, cl 16 (64 GB upgrade still possible if needed)\n- SSD: some M.2 SSD \n- Mainboard: some 450/470/550/570 (not sure yet)\n\nDoes that kind of build make sense for such tasks? Is a fast M.2 SSD worth it or better get something cheaper? Do RAM timings have any noticable effect? Any Board Recommendations? ...\n\nThank you for any input!", "544841": "Ah. First things first. You won't need any benchmarks on ryzen 3000. 13% IPC improvement just places it at the level of Intel IPC. The benefit is upgrading to Ryzen 3700 is that is will be better price/performance ratio when compared to Intel. I would not worry too much about single core performance improvements too much. Your main issue is increasing RAM. With 16GB, and GTX 1080, you might as well be running kaggle kernels. 32GB is a good place to start. Doesn't matter what SSD you have, doesn't need to be M.2. If you can afford M.2 go for it. To be honest, I've been upgrading my parts (M.2 ssd, now 64GB RAM, RTX 2070). I hardly use any of it anymore. Once I realized that kaggle kernels get the job done I just stopped worrying about my hardware. I'm currently competing in Analytics Vidhya Computer Vision Hackathon and I'm using Kaggle GPU. Sure there are things I can't do given the RAM limitation (5-fold CV, ZCA whitening, etc.), but I've learned to manage memory and utilize every bit I have more effectively. The only downside with using Kaggle Kernels is peak utilization. But you're going to want to have decent hdd or ssd space to store and read datasets from. It's definitely exciting to have new equipment and hardware, but it fades over time. RTX was a nightmare to setup with tensorflow when I first got it. I put in a ticket to the tensorflow team on GitHub 4-5 months ago and it's still open. Finally, none of the overclocking, timings, GPU overclocking matters for machine learning or deep learning. I performed benchmarks on it a while back and I even posted my findings with the methodology online. The improvements aren't even worth mentioning. I've also had considerably less frustration over hardware and installation issues, software/packages included since I started working on kaggle kernels. The funny thing is, Kaggle GPU has more VRAM than my RTX 2070. I've hardly used RTX since I purchased it for training neural networks. If you do still plan on upgrading, just go with the most cores and most RAM you can afford. You'll end up using GBM alot so the more cores the faster you'll finish training. Anyways, I hope what I'm saying helps.", "546027": "Get M2 drive. Some of them are actually not so expensive. Don't buy crucial P1 drive or intel 660p. These drive slow down to HD speed once the buffer runs out. You can set up large swap space to overcome ram limitation. But Swap is slower than RAM, but your program won't crush. A faster SSD makes swap run faster. I have Ryzen 2600X, and 1080ti, 32GB ram, 30GB Swap (another M2 has 50GB swap, I should change it to 100GB+).", "546469": "I agree. Do not buy the the Crucial P1 or Intel 660p. I have Samsung 970 Evo. It works, but I probably could have gone with a non M.2 drive. Storing large datasets are a problem over time. For example the Microsoft Malware 2015 Kaggle challenge is half a Terabyte uncompressed for train and test. I downloaded it and quickly deleted it after realizing I didn't have enough space to work with it.", "546480": "Buy AWS. And a crappy comp if u have to", "546700": "I bought a different brand (ADATA XPG SX8200 Pro 1TB ) cheaper than 970 EVO. But this seems to have good reviews, even cheaper and longer endurance rating. I also bought 970 EVO 512GB last year.\n\nhttps://www.newegg.com/corsair-force-mp510-960gb/p/N82E16820236477?item=N82E16820236477&ignorebbr=1&source=region&nm_mc=knc-googleadwords-pc&cm_mmc=knc-googleadwords-pc-_-pla-_-solid+state+disk-_-N82E16820236477&gclid=Cj0KEQjw8-LnBRCyxtfMl-Cbu48BEiQA6eUMGmgGmNpjNGN0knB6TCWRjtwZJwRhdezwHX2SJFJDJ24aAlMN8P8HAQ&gclsrc=aw.ds", "546707": "It helps a lot, thanks. Maybe i should try kaggle kernels first in the next competition, i honestly haven't really looked at them yet. I don't have to upgrade right away and can wait, but a new SSD is coming sooner or later anyway and when upgrading might as well swap every outdated part and be settled for the next 3-5 years.", "546711": "That swap space seems like a good idea, now i just have to read up how it works. I've lost count of how many times my spyder crashed in this comp, that'll definitely help :). I think i actually added something like that when setting up ubuntu, but it was my first time and maybe i missed something there.", "546717": "Thing is, I use this computer anyway for other stuff like gaming, surfing, office etc. I don't really want to spend extra money for server costs as long as it's not absolutely necessary, but maybe i overestimate aws costs, haven't tried it yet. \nAnd why crappy? If you want to make donations, i'll gladly give you my paypal lol", "546887": "The other thing you may want to consider is have two machines. One to prototype and another to heavy lifting. I have access to more than 1 machine, sometimes I run variation of different scripts/notebooks at the same time. I found that I can run more scripts using Kaggle kernels much easier and all for free. Don't need multiple computers. The upside to having a decent machine is running scripts for > 9hrs which is Kaggle's current limit. A lot of the computer vision competitions on here require quite a bit of hardware which is a pretty big barrier to entry for many people. Certainly having the best hardware won't mean winning, but it can allow you to work on projects that most can't, even if it is just to learn and have fun.", "547190": "Its free for the first year. You should try it. It depends on your needs at the end of the day\nI meant if you are going to spend money on an expensive hardware, than you could make a trade off between less expensive (=crappy) comp and enough AWS credits." }, "source": "meta" }