![\"awsec\"](\"https://s7.wampi.ru/2018/08/14/Screenshot-from-2018-08-14-00-57-52.png\")
Just crossing fingers xD\n
Update: Got it! xD
", "messages": [ { "id": "369845", "postDate": "08/13/2018 22:04:34", "content": "Just crossing fingers xD\n
Update: Got it! xD
", "rawMarkdown": "Just crossing fingers xD\n![awsec][1]\n\n\n [1]: https://s7.wampi.ru/2018/08/14/Screenshot-from-2018-08-14-00-57-52.png\n\nUpdate: Got it! xD", "votes": null }, { "id": "369872", "postDate": "08/13/2018 23:02:02", "content": "Good luck!
", "rawMarkdown": "Good luck!", "votes": null }, { "id": "369881", "postDate": "08/13/2018 23:30:16", "content": "Good luck! 🍀
", "rawMarkdown": "Good luck! 🍀", "votes": null }, { "id": "369885", "postDate": "08/13/2018 23:58:08", "content": "Good luck ))
", "rawMarkdown": "Good luck ))", "votes": null }, { "id": "369887", "postDate": "08/13/2018 23:59:27", "content": "Thanks :)
", "rawMarkdown": "Thanks :)", "votes": null }, { "id": "369889", "postDate": "08/14/2018 00:00:22", "content": "Well, I believe I already have! Gap from 104 to 63 on Public LB. One AWS instance per event. I'm crazy:)
", "rawMarkdown": "Well, I believe I already have! Gap from 104 to 63 on Public LB. One AWS instance per event. I'm crazy:)", "votes": null }, { "id": "369890", "postDate": "08/14/2018 00:03:13", "content": "поздравляю Sergey !
", "rawMarkdown": "поздравляю Sergey !", "votes": null }, { "id": "369891", "postDate": "08/14/2018 00:05:55", "content": "Congrats. :) Same here 128 cores altogether. I moved from 200+ to 58 in the last few days. Hope I had more time to work on this (couldn't even submit my last submission :) ).
", "rawMarkdown": "Congrats. :) Same here 128 cores altogether. I moved from 200+ to 58 in the last few days. Hope I had more time to work on this (couldn't even submit my last submission :) ).", "votes": null }, { "id": "369895", "postDate": "08/14/2018 00:09:37", "content": "Congrats @Sergey! One instance per event is pretty crazy! Happy it worked out for you. :)
", "rawMarkdown": "Congrats @Sergey! One instance per event is pretty crazy! Happy it worked out for you. :)", "votes": null }, { "id": "369906", "postDate": "08/14/2018 00:24:46", "content": "@Sergey Kabanov congrats making it to 75 and beating moi :-)
\n\n@Jack Vial, I ran all my models for this competition on my not so great 12GB, 2.7GHz windows solid state device. It takes 3 days to make one submission. Which one is crazier? :-)
", "rawMarkdown": "Sergey Kabanov congrats making it to 75 and beating moi :-)\n\n@Jack Vial, I ran all my models for this competition on my not so great 12GB, 2.7GHz windows solid state device. It takes 3 days to make one submission. Which one is crazier? :-)", "votes": null }, { "id": "369921", "postDate": "08/14/2018 00:39:40", "content": "@YaGana Sheriff-Hussaini I admire your determination and patience! I think it might be a tie between Sergey and you! XD I estimated my final model would have take about 3 days on my local setup (Core i5-8400 2.8GHz 6-Core) even with parallelization.
", "rawMarkdown": "YaGana Sheriff-Hussaini I admire your determination and patience! I think it might be a tie between Sergey and you! XD I estimated my final model would have take about 3 days on my local setup (Core i5-8400 2.8GHz 6-Core) even with parallelization.", "votes": null }, { "id": "369922", "postDate": "08/14/2018 00:39:42", "content": "Thank you guys. My summer plan is to get three medals in TrackML, Santander and Home Credit competitions. But I really joined to TrackML in last few days :( I thought that knowlege of ten-volume series \"Landau and Lifshitz\" will help me. Xa-Xa Naive boy. Just tons of AWS instances... and more z-shifting. And I learned a lot from it!
", "rawMarkdown": "Thank you guys. My summer plan is to get three medals in TrackML, Santander and Home Credit competitions. But I really joined to TrackML in last few days :( I thought that knowlege of **ten-volume series \"Landau and Lifshitz\"** will help me. Xa-Xa Naive boy. **Just tons of AWS instances...** and more z-shifting. And I learned a lot from it!", "votes": null }, { "id": "369940", "postDate": "08/14/2018 00:59:30", "content": "\"Landau and Lifshitz \"....wow ! You aimed too much high level :)
\n\nMay be a rather very elementary physics book like the one by Yakov Perelman* is sufficient ( at least in background knowledge of physics)
\n\n*I read it in french back in high school, I don't know the original russian title.
", "rawMarkdown": "\"Landau and Lifshitz \"....wow ! You aimed too much high level :)\n\nMay be a rather very elementary physics book like the one by Yakov Perelman* is sufficient ( at least in background knowledge of physics)\n\n*I read it in french back in high school, I don't know the original russian title.", "votes": null }, { "id": "370331", "postDate": "08/14/2018 16:32:10", "content": "Almost 95% of the people have no prior knowledge in Physics and gave solution ))) that's funny how ML works.
", "rawMarkdown": "Almost 95% of the people have no prior knowledge in Physics and gave solution ))) that's funny how ML works.", "votes": null }, { "id": "371127", "postDate": "08/16/2018 03:49:11", "content": "Thanks @Jack Vial.
\n\nUsing the low performing HW may have hindered my performance but I really did not have a lot of time to spend on the competition so I spend a couple of hours at a time on how to improve my model. List them up and then run the script for 5 training events, and any change that made significant improvement in the mean scores, I ran it on the test data and waited. I used dbscan, with pre-processing and track extension. Code optimization to run faster is part of the solution but my HW is a CPU hence not as good as a GPU machine like others :-)
", "rawMarkdown": "Thanks @Jack Vial. \n\nUsing the low performing HW may have hindered my performance but I really did not have a lot of time to spend on the competition so I spend a couple of hours at a time on how to improve my model. List them up and then run the script for 5 training events, and any change that made significant improvement in the mean scores, I ran it on the test data and waited. I used dbscan, with pre-processing and track extension. Code optimization to run faster is part of the solution but my HW is a CPU hence not as good as a GPU machine like others :-)", "votes": null }, { "id": "371288", "postDate": "08/16/2018 13:10:25", "content": "@YaGana Sheriff-Hussaini For development and running validation I done everything on my local machine. I ran validation on 6 events using my 6 cores in parallel and then got the mean score so I think we had a very similar process there. Only after getting a validation score that I thought could move me up the LB a few places did I process the test events on AWS. My hardware knowledge is pretty basic but 12Ghz sounds like a very fast CPU, did you mean 1.2Ghz? I had looked into trying to get my code to run on GPU but couldn't figure it out. I haven't heard of anyone in this competition running their solution code on GPU but maybe I missed that in the forums somewhere. It certainly would have been useful being able to iterate on on ideas quicker. Code optimization isn't something I had time to look into, was still trying to follow Yuval's breadcrumbs to figure out good features up to the last day. Do you mind my asking what kind of code optimization you done?
", "rawMarkdown": "YaGana Sheriff-Hussaini For development and running validation I done everything on my local machine. I ran validation on 6 events using my 6 cores in parallel and then got the mean score so I think we had a very similar process there. Only after getting a validation score that I thought could move me up the LB a few places did I process the test events on AWS. My hardware knowledge is pretty basic but 12Ghz sounds like a very fast CPU, did you mean 1.2Ghz? I had looked into trying to get my code to run on GPU but couldn't figure it out. I haven't heard of anyone in this competition running their solution code on GPU but maybe I missed that in the forums somewhere. It certainly would have been useful being able to iterate on on ideas quicker. Code optimization isn't something I had time to look into, was still trying to follow Yuval's breadcrumbs to figure out good features up to the last day. Do you mind my asking what kind of code optimization you done?", "votes": null }, { "id": "371503", "postDate": "08/16/2018 23:27:15", "content": "12Ghz ? I wish :-)
\n\n@Jack Vial, that was meant to be 12GB physical memory and processor is 2.7GHz
\n\nFor optimization of my code, I used pool which I think was also recommended by @CPMP in another thread to do the 5 events training evaluation on Google colab with GPU. Because using pool on my labtop gives me problems, I used my code with only the predict part using pool on Colab. Then ran the improved model on my labtop on the test data. On my locally run script my motivation to optimize the code was to improve the processing time per event on the test data. My final time per event was about 27 minutes.
\n\nI was also following Yuval's suggestions until the last day but was not successful in getting any new features with constant values with the test he shared. Only one of my features does that and could not find any additional one.
", "rawMarkdown": "12Ghz ? I wish :-)\n\n@Jack Vial, that was meant to be 12GB physical memory and processor is 2.7GHz\n\nFor optimization of my code, I used pool which I think was also recommended by @CPMP in another thread to do the 5 events training evaluation on Google colab with GPU. Because using pool on my labtop gives me problems, I used my code with only the predict part using pool on Colab. Then ran the improved model on my labtop on the test data. On my locally run script my motivation to optimize the code was to improve the processing time per event on the test data. My final time per event was about 27 minutes.\n\nI was also following Yuval's suggestions until the last day but was not successful in getting any new features with constant values with the test he shared. Only one of my features does that and could not find any additional one.", "votes": null }, { "id": "371530", "postDate": "08/17/2018 02:25:27", "content": "@YaGana Sheriff-Hussaini Same here! XD
\n\nOh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab? 27 minutes is certainly much faster than my solution, it took about 1hr 30 mins for each event.
\n\nI too was unable to find them but it was a good learning experience.
", "rawMarkdown": "YaGana Sheriff-Hussaini Same here! XD\n\nOh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab? 27 minutes is certainly much faster than my solution, it took about 1hr 30 mins for each event.\n\nI too was unable to find them but it was a good learning experience.", "votes": null }, { "id": "371537", "postDate": "08/17/2018 02:57:17", "content": "@Jack Vial, I could run my code on GPU with no modifications. I only added pool to parallelize my code.
\n\nThe 27 minutes per event time is for test events on my labtop. Since you are higher up on the LB than me, you may be doing more in your code.
", "rawMarkdown": "Jack Vial, I could run my code on GPU with no modifications. I only added pool to parallelize my code.\n\nThe 27 minutes per event time is for test events on my labtop. Since you are higher up on the LB than me, you may be doing more in your code.", "votes": null }, { "id": "371571", "postDate": "08/17/2018 06:18:05", "content": "\n\n\nI could run my code on GPU with no modifications
\n
What part of your code did you run on gpu? Are you using a deep learning model?
\n\n\n\n\nmy HW is a CPU hence not as good as a GPU machine like others :-)
\n
FYI, I am not using any GPU, and I don't think solutions #1, #3, #5, and #7 use any GPU either from their description.
\n\nI am reacting because it would be great to get GPU acceleration for numpy operations or for DBSCAN, but I didn't find anything that I could use.
", "rawMarkdown": "> I could run my code on GPU with no modifications\n\nWhat part of your code did you run on gpu? Are you using a deep learning model? \n\n> my HW is a CPU hence not as good as a GPU machine like others :-)\n\nFYI, I am not using any GPU, and I don't think solutions #1, #3, #5, and #7 use any GPU either from their description. \n\nI am reacting because it would be great to get GPU acceleration for numpy operations or for DBSCAN, but I didn't find anything that I could use.", "votes": null }, { "id": "371590", "postDate": "08/17/2018 07:10:18", "content": "\n\n\nOh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab?
\n
@CPMP thanks for the clarification. I misunderstood @Jack's question, I did not use any GPU acceleration for dbscan. I was responding to the above quote. And I did not use a deep learning model.
", "rawMarkdown": "> Oh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab?\n\n@CPMP thanks for the clarification. I misunderstood @Jack's question, I did not use any GPU acceleration for dbscan. I was responding to the above quote. And I did not use a deep learning model.", "votes": null }, { "id": "371805", "postDate": "08/17/2018 16:24:08", "content": "Good luck
", "rawMarkdown": "Good luck", "votes": null }, { "id": "372307", "postDate": "08/18/2018 22:34:24", "content": "@YaGana Sheriff-Hussaini I get what you meant now. @CPMP Thanks for the clarification.
\n\nGPU acceleration for DBSCAN would be great, but since one call of DBSCAN is already quite fast I am thinking the whole DBSCAN ensemble and merging model would need to be GPU optimized because of the GPU data transfer overhead.
", "rawMarkdown": "YaGana Sheriff-Hussaini I get what you meant now. @CPMP Thanks for the clarification.\n\nGPU acceleration for DBSCAN would be great, but since one call of DBSCAN is already quite fast I am thinking the whole DBSCAN ensemble and merging model would need to be GPU optimized because of the GPU data transfer overhead.", "votes": null } ], "comments": [ { "id": 369872, "author_name": "alexanderliao", "author_url": "", "post_date": "08/13/2018 23:02:02", "content": "Good luck!
", "votes": null, "replies": [] }, { "id": 369881, "author_name": "jackvial", "author_url": "", "post_date": "08/13/2018 23:30:16", "content": "Good luck! 🍀
", "votes": null, "replies": [ { "id": 369885, "author_name": "alexanderkireev", "author_url": "", "post_date": "08/13/2018 23:58:08", "content": "Good luck ))
", "votes": null, "replies": [] }, { "id": 369887, "author_name": "jackvial", "author_url": "", "post_date": "08/13/2018 23:59:27", "content": "Thanks :)
", "votes": null, "replies": [] }, { "id": 369889, "author_name": "sggpls", "author_url": "", "post_date": "08/14/2018 00:00:22", "content": "Well, I believe I already have! Gap from 104 to 63 on Public LB. One AWS instance per event. I'm crazy:)
", "votes": null, "replies": [] }, { "id": 369890, "author_name": "serigne", "author_url": "", "post_date": "08/14/2018 00:03:13", "content": "поздравляю Sergey !
", "votes": null, "replies": [] }, { "id": 369891, "author_name": "akilaw", "author_url": "", "post_date": "08/14/2018 00:05:55", "content": "Congrats. :) Same here 128 cores altogether. I moved from 200+ to 58 in the last few days. Hope I had more time to work on this (couldn't even submit my last submission :) ).
", "votes": null, "replies": [] }, { "id": 369895, "author_name": "jackvial", "author_url": "", "post_date": "08/14/2018 00:09:37", "content": "Congrats @Sergey! One instance per event is pretty crazy! Happy it worked out for you. :)
", "votes": null, "replies": [] }, { "id": 369906, "author_name": "sheriytm", "author_url": "", "post_date": "08/14/2018 00:24:46", "content": "@Sergey Kabanov congrats making it to 75 and beating moi :-)
\n\n@Jack Vial, I ran all my models for this competition on my not so great 12GB, 2.7GHz windows solid state device. It takes 3 days to make one submission. Which one is crazier? :-)
", "votes": null, "replies": [] }, { "id": 369921, "author_name": "jackvial", "author_url": "", "post_date": "08/14/2018 00:39:40", "content": "@YaGana Sheriff-Hussaini I admire your determination and patience! I think it might be a tie between Sergey and you! XD I estimated my final model would have take about 3 days on my local setup (Core i5-8400 2.8GHz 6-Core) even with parallelization.
", "votes": null, "replies": [] }, { "id": 369922, "author_name": "sggpls", "author_url": "", "post_date": "08/14/2018 00:39:42", "content": "Thank you guys. My summer plan is to get three medals in TrackML, Santander and Home Credit competitions. But I really joined to TrackML in last few days :( I thought that knowlege of ten-volume series \"Landau and Lifshitz\" will help me. Xa-Xa Naive boy. Just tons of AWS instances... and more z-shifting. And I learned a lot from it!
", "votes": null, "replies": [] }, { "id": 369940, "author_name": "serigne", "author_url": "", "post_date": "08/14/2018 00:59:30", "content": "\"Landau and Lifshitz \"....wow ! You aimed too much high level :)
\n\nMay be a rather very elementary physics book like the one by Yakov Perelman* is sufficient ( at least in background knowledge of physics)
\n\n*I read it in french back in high school, I don't know the original russian title.
", "votes": null, "replies": [] }, { "id": 370331, "author_name": "muhakabartay", "author_url": "", "post_date": "08/14/2018 16:32:10", "content": "Almost 95% of the people have no prior knowledge in Physics and gave solution ))) that's funny how ML works.
", "votes": null, "replies": [] }, { "id": 371127, "author_name": "sheriytm", "author_url": "", "post_date": "08/16/2018 03:49:11", "content": "Thanks @Jack Vial.
\n\nUsing the low performing HW may have hindered my performance but I really did not have a lot of time to spend on the competition so I spend a couple of hours at a time on how to improve my model. List them up and then run the script for 5 training events, and any change that made significant improvement in the mean scores, I ran it on the test data and waited. I used dbscan, with pre-processing and track extension. Code optimization to run faster is part of the solution but my HW is a CPU hence not as good as a GPU machine like others :-)
", "votes": null, "replies": [] }, { "id": 371288, "author_name": "jackvial", "author_url": "", "post_date": "08/16/2018 13:10:25", "content": "@YaGana Sheriff-Hussaini For development and running validation I done everything on my local machine. I ran validation on 6 events using my 6 cores in parallel and then got the mean score so I think we had a very similar process there. Only after getting a validation score that I thought could move me up the LB a few places did I process the test events on AWS. My hardware knowledge is pretty basic but 12Ghz sounds like a very fast CPU, did you mean 1.2Ghz? I had looked into trying to get my code to run on GPU but couldn't figure it out. I haven't heard of anyone in this competition running their solution code on GPU but maybe I missed that in the forums somewhere. It certainly would have been useful being able to iterate on on ideas quicker. Code optimization isn't something I had time to look into, was still trying to follow Yuval's breadcrumbs to figure out good features up to the last day. Do you mind my asking what kind of code optimization you done?
", "votes": null, "replies": [] }, { "id": 371503, "author_name": "sheriytm", "author_url": "", "post_date": "08/16/2018 23:27:15", "content": "12Ghz ? I wish :-)
\n\n@Jack Vial, that was meant to be 12GB physical memory and processor is 2.7GHz
\n\nFor optimization of my code, I used pool which I think was also recommended by @CPMP in another thread to do the 5 events training evaluation on Google colab with GPU. Because using pool on my labtop gives me problems, I used my code with only the predict part using pool on Colab. Then ran the improved model on my labtop on the test data. On my locally run script my motivation to optimize the code was to improve the processing time per event on the test data. My final time per event was about 27 minutes.
\n\nI was also following Yuval's suggestions until the last day but was not successful in getting any new features with constant values with the test he shared. Only one of my features does that and could not find any additional one.
", "votes": null, "replies": [] }, { "id": 371530, "author_name": "jackvial", "author_url": "", "post_date": "08/17/2018 02:25:27", "content": "@YaGana Sheriff-Hussaini Same here! XD
\n\nOh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab? 27 minutes is certainly much faster than my solution, it took about 1hr 30 mins for each event.
\n\nI too was unable to find them but it was a good learning experience.
", "votes": null, "replies": [] }, { "id": 371537, "author_name": "sheriytm", "author_url": "", "post_date": "08/17/2018 02:57:17", "content": "@Jack Vial, I could run my code on GPU with no modifications. I only added pool to parallelize my code.
\n\nThe 27 minutes per event time is for test events on my labtop. Since you are higher up on the LB than me, you may be doing more in your code.
", "votes": null, "replies": [] }, { "id": 371571, "author_name": "cpmpml", "author_url": "", "post_date": "08/17/2018 06:18:05", "content": "\n\n\nI could run my code on GPU with no modifications
\n
What part of your code did you run on gpu? Are you using a deep learning model?
\n\n\n\n\nmy HW is a CPU hence not as good as a GPU machine like others :-)
\n
FYI, I am not using any GPU, and I don't think solutions #1, #3, #5, and #7 use any GPU either from their description.
\n\nI am reacting because it would be great to get GPU acceleration for numpy operations or for DBSCAN, but I didn't find anything that I could use.
", "votes": null, "replies": [] }, { "id": 371590, "author_name": "sheriytm", "author_url": "", "post_date": "08/17/2018 07:10:18", "content": "\n\n\nOh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab?
\n
@CPMP thanks for the clarification. I misunderstood @Jack's question, I did not use any GPU acceleration for dbscan. I was responding to the above quote. And I did not use a deep learning model.
", "votes": null, "replies": [] }, { "id": 371805, "author_name": "tuanflash", "author_url": "", "post_date": "08/17/2018 16:24:08", "content": "Good luck
", "votes": null, "replies": [] }, { "id": 372307, "author_name": "jackvial", "author_url": "", "post_date": "08/18/2018 22:34:24", "content": "@YaGana Sheriff-Hussaini I get what you meant now. @CPMP Thanks for the clarification.
\n\nGPU acceleration for DBSCAN would be great, but since one call of DBSCAN is already quite fast I am thinking the whole DBSCAN ensemble and merging model would need to be GPU optimized because of the GPU data transfer overhead.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "369845": "Just crossing fingers xD\n![awsec][1]\n\n\n [1]: https://s7.wampi.ru/2018/08/14/Screenshot-from-2018-08-14-00-57-52.png\n\nUpdate: Got it! xD", "369872": "Good luck!", "369881": "Good luck! 🍀", "369885": "Good luck ))", "369887": "Thanks :)", "369889": "Well, I believe I already have! Gap from 104 to 63 on Public LB. One AWS instance per event. I'm crazy:)", "369890": "поздравляю Sergey !", "369891": "Congrats. :) Same here 128 cores altogether. I moved from 200+ to 58 in the last few days. Hope I had more time to work on this (couldn't even submit my last submission :) ).", "369895": "Congrats @Sergey! One instance per event is pretty crazy! Happy it worked out for you. :)", "369906": "Sergey Kabanov congrats making it to 75 and beating moi :-)\n\n@Jack Vial, I ran all my models for this competition on my not so great 12GB, 2.7GHz windows solid state device. It takes 3 days to make one submission. Which one is crazier? :-)", "369921": "YaGana Sheriff-Hussaini I admire your determination and patience! I think it might be a tie between Sergey and you! XD I estimated my final model would have take about 3 days on my local setup (Core i5-8400 2.8GHz 6-Core) even with parallelization.", "369922": "Thank you guys. My summer plan is to get three medals in TrackML, Santander and Home Credit competitions. But I really joined to TrackML in last few days :( I thought that knowlege of **ten-volume series \"Landau and Lifshitz\"** will help me. Xa-Xa Naive boy. **Just tons of AWS instances...** and more z-shifting. And I learned a lot from it!", "369940": "\"Landau and Lifshitz \"....wow ! You aimed too much high level :)\n\nMay be a rather very elementary physics book like the one by Yakov Perelman* is sufficient ( at least in background knowledge of physics)\n\n*I read it in french back in high school, I don't know the original russian title.", "370331": "Almost 95% of the people have no prior knowledge in Physics and gave solution ))) that's funny how ML works.", "371127": "Thanks @Jack Vial. \n\nUsing the low performing HW may have hindered my performance but I really did not have a lot of time to spend on the competition so I spend a couple of hours at a time on how to improve my model. List them up and then run the script for 5 training events, and any change that made significant improvement in the mean scores, I ran it on the test data and waited. I used dbscan, with pre-processing and track extension. Code optimization to run faster is part of the solution but my HW is a CPU hence not as good as a GPU machine like others :-)", "371288": "YaGana Sheriff-Hussaini For development and running validation I done everything on my local machine. I ran validation on 6 events using my 6 cores in parallel and then got the mean score so I think we had a very similar process there. Only after getting a validation score that I thought could move me up the LB a few places did I process the test events on AWS. My hardware knowledge is pretty basic but 12Ghz sounds like a very fast CPU, did you mean 1.2Ghz? I had looked into trying to get my code to run on GPU but couldn't figure it out. I haven't heard of anyone in this competition running their solution code on GPU but maybe I missed that in the forums somewhere. It certainly would have been useful being able to iterate on on ideas quicker. Code optimization isn't something I had time to look into, was still trying to follow Yuval's breadcrumbs to figure out good features up to the last day. Do you mind my asking what kind of code optimization you done?", "371503": "12Ghz ? I wish :-)\n\n@Jack Vial, that was meant to be 12GB physical memory and processor is 2.7GHz\n\nFor optimization of my code, I used pool which I think was also recommended by @CPMP in another thread to do the 5 events training evaluation on Google colab with GPU. Because using pool on my labtop gives me problems, I used my code with only the predict part using pool on Colab. Then ran the improved model on my labtop on the test data. On my locally run script my motivation to optimize the code was to improve the processing time per event on the test data. My final time per event was about 27 minutes.\n\nI was also following Yuval's suggestions until the last day but was not successful in getting any new features with constant values with the test he shared. Only one of my features does that and could not find any additional one.", "371530": "YaGana Sheriff-Hussaini Same here! XD\n\nOh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab? 27 minutes is certainly much faster than my solution, it took about 1hr 30 mins for each event.\n\nI too was unable to find them but it was a good learning experience.", "371537": "Jack Vial, I could run my code on GPU with no modifications. I only added pool to parallelize my code.\n\nThe 27 minutes per event time is for test events on my labtop. Since you are higher up on the LB than me, you may be doing more in your code.", "371571": "> I could run my code on GPU with no modifications\n\nWhat part of your code did you run on gpu? Are you using a deep learning model? \n\n> my HW is a CPU hence not as good as a GPU machine like others :-)\n\nFYI, I am not using any GPU, and I don't think solutions #1, #3, #5, and #7 use any GPU either from their description. \n\nI am reacting because it would be great to get GPU acceleration for numpy operations or for DBSCAN, but I didn't find anything that I could use.", "371590": "> Oh, I didn't realize processing events could run on GPU. Was there anything specific you had to do to get your code to run on GPU on Colab?\n\n@CPMP thanks for the clarification. I misunderstood @Jack's question, I did not use any GPU acceleration for dbscan. I was responding to the above quote. And I did not use a deep learning model.", "371805": "Good luck", "372307": "YaGana Sheriff-Hussaini I get what you meant now. @CPMP Thanks for the clarification.\n\nGPU acceleration for DBSCAN would be great, but since one call of DBSCAN is already quite fast I am thinking the whole DBSCAN ensemble and merging model would need to be GPU optimized because of the GPU data transfer overhead." }, "source": "meta" }