Hi all, \nI have tried running [Cervix segmentation (GMM)][1] as a starting point. I downloaded the notebook as Python script and ran it on Colfax's compute node. \nAll required modules such as opencv-python, scikit-learn, scikit-image are installed in a conda environment, there was no error but it took about over 400 seconds per image to process. \nIs there anyone have the same problem? Any suggestion to solve?\nThank you.
\n\nUpdate: Tensorflow has better performance than Theano in normal installation and configuration.
\n\n[1]: https://www.kaggle.com/chattob/intel-mobileodt-cervical-cancer-screening/cervix-segmentation-gmm \"
", "messages": [ { "id": 176710, "postDate": "2017-04-21T16:01:51.357Z", "content": "My jupyter notebook running on Colfax keeps dying and restarting after processing about 40 images
", "rawMarkdown": "My jupyter notebook running on Colfax keeps dying and restarting after processing about 40 images ", "votes": 1, "replies": [ { "id": 176718, "postDate": "2017-04-21T17:01:15.833Z", "content": "When you log on Colfax Cluster, you are in Login Node. \nIf you run a heavy task, it will terminate your process. You should not run on Login Node, submit job to compute instead.
", "rawMarkdown": "When you log on Colfax Cluster, you are in Login Node. \nIf you run a heavy task, it will terminate your process. You should not run on Login Node, submit job to compute instead. " }, { "id": 177144, "postDate": "2017-04-23T22:36:29.677Z", "content": "The page explains how to run jupyter notebook on the compute node instead of the login node:\nhttps://colfaxresearch.com/discussion/topic/connecting-jupyter-notebook-on-compute-node/
\n\nGive that a try.
", "rawMarkdown": "The page explains how to run jupyter notebook on the compute node instead of the login node:\nhttps://colfaxresearch.com/discussion/topic/connecting-jupyter-notebook-on-compute-node/\n\nGive that a try.", "votes": 1 }, { "id": 180278, "postDate": "2017-05-04T18:02:01.723Z", "content": "Thanks!
", "rawMarkdown": "Thanks!" } ] }, { "id": 175612, "postDate": "2017-04-16T13:48:44.077Z", "content": "Hi all, \nI have tried running [Cervix segmentation (GMM)][1] as a starting point. I downloaded the notebook as Python script and ran it on Colfax's compute node. \nAll required modules such as opencv-python, scikit-learn, scikit-image are installed in a conda environment, there was no error but it took about over 400 seconds per image to process. \nIs there anyone have the same problem? Any suggestion to solve?\nThank you.
\n\nUpdate: Tensorflow has better performance than Theano in normal installation and configuration.
\n\n[1]: https://www.kaggle.com/chattob/intel-mobileodt-cervical-cancer-screening/cervix-segmentation-gmm \"
", "rawMarkdown": "Hi all, \nI have tried running [Cervix segmentation (GMM)][1] as a starting point. I downloaded the notebook as Python script and ran it on Colfax's compute node. \nAll required modules such as opencv-python, scikit-learn, scikit-image are installed in a conda environment, there was no error but it took about over 400 seconds per image to process. \nIs there anyone have the same problem? Any suggestion to solve?\nThank you.\n\n\nUpdate: Tensorflow has better performance than Theano in normal installation and configuration.\n\n\n [1]: https://www.kaggle.com/chattob/intel-mobileodt-cervical-cancer-screening/cervix-segmentation-gmm \"", "votes": 1 }, { "id": 178830, "postDate": "2017-04-28T23:42:09.127Z", "content": "I have tried to run [Cervix segmentation (GMM)] on Colfax cluster. The script works fine on the login node but it keeps dying. On the compute node I get the error [module 'sklearn.mixture' has no attribute 'GaussianMixture']. Does anyone know how to solve this problem?
", "rawMarkdown": "I have tried to run [Cervix segmentation (GMM)] on Colfax cluster. The script works fine on the login node but it keeps dying. On the compute node I get the error [module 'sklearn.mixture' has no attribute 'GaussianMixture']. Does anyone know how to solve this problem?" }, { "id": 176841, "postDate": "2017-04-22T09:15:19.067Z", "content": "I ran some tests comparing the performance of colfax to my laptop (mid-2012 macbook pro). My laptop is a lot faster even when running NNs in keras/theano. I use the precompiled theano library provided on the cluster. I don't understand why this ultra high performance compute cluster is 10 times slower than my old laptop. I am not using the laptop's GPU, just CPU. I guess the Theano they provide is not optimized at all. Does anyone have better results? How can I run theano quickly on the colfax cluster?
\n\nA simple dense network ran in 21 seconds on my laptop but took 225 seconds on the cluster.
", "rawMarkdown": "I ran some tests comparing the performance of colfax to my laptop (mid-2012 macbook pro). My laptop is a lot faster even when running NNs in keras/theano. I use the precompiled theano library provided on the cluster. I don't understand why this ultra high performance compute cluster is 10 times slower than my old laptop. I am not using the laptop's GPU, just CPU. I guess the Theano they provide is not optimized at all. Does anyone have better results? How can I run theano quickly on the colfax cluster?\n\nA simple dense network ran in 21 seconds on my laptop but took 225 seconds on the cluster.", "replies": [ { "id": 177015, "postDate": "2017-04-23T05:19:02.870Z", "content": "which framework have you use, Keras, Caffe, mxnet..?
", "rawMarkdown": "which framework have you use, Keras, Caffe, mxnet..?" }, { "id": 177192, "postDate": "2017-04-24T07:36:15.800Z", "content": "Keras and Theano but Keras + TF is also giving me trouble. I wonder if anyone has been successful using one of the common python DL frameworks on colfax. If so I would greatly appreciate how to set up a good environment. I'm now using floydhub instead and that just works perfectly straight out of the box.
", "rawMarkdown": "Keras and Theano but Keras + TF is also giving me trouble. I wonder if anyone has been successful using one of the common python DL frameworks on colfax. If so I would greatly appreciate how to set up a good environment. I'm now using floydhub instead and that just works perfectly straight out of the box." } ] }, { "id": 176449, "postDate": "2017-04-20T18:00:44.383Z", "content": "There is currently an online seminar going on by Colfax.
", "rawMarkdown": "There is currently an online seminar going on by Colfax. ", "replies": [ { "id": 176719, "postDate": "2017-04-21T17:01:47.787Z", "content": "Could you show me link of the seminar, \nthanks.
", "rawMarkdown": "Could you show me link of the seminar, \nthanks. " } ] }, { "id": 176368, "postDate": "2017-04-20T07:54:33.787Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 176716, "postDate": "2017-04-21T16:56:49.633Z", "content": "I tried to compile OpenCv from source but it was not success because of lacking some library. \nI would try to parallelism the code. \nThank you.
", "rawMarkdown": "I tried to compile OpenCv from source but it was not success because of lacking some library. \nI would try to parallelism the code. \nThank you. " } ] } ], "comments": [ { "id": 176710, "author_name": "Jeff Law", "author_url": "", "post_date": "2017-04-21T16:01:51.357000", "content": "My jupyter notebook running on Colfax keeps dying and restarting after processing about 40 images
", "votes": 1, "replies": [ { "id": 176718, "author_name": " Van-Cao Nguyen ", "author_url": "", "post_date": "2017-04-21T17:01:15.833000", "content": "When you log on Colfax Cluster, you are in Login Node. \nIf you run a heavy task, it will terminate your process. You should not run on Login Node, submit job to compute instead.
", "votes": 0, "replies": [] }, { "id": 177144, "author_name": "PhillipChin", "author_url": "", "post_date": "2017-04-23T22:36:29.677000", "content": "The page explains how to run jupyter notebook on the compute node instead of the login node:\nhttps://colfaxresearch.com/discussion/topic/connecting-jupyter-notebook-on-compute-node/
\n\nGive that a try.
", "votes": 1, "replies": [] }, { "id": 180278, "author_name": "Jeff Law", "author_url": "", "post_date": "2017-05-04T18:02:01.723000", "content": "Thanks!
", "votes": 0, "replies": [] } ] }, { "id": 178830, "author_name": "kxx", "author_url": "", "post_date": "2017-04-28T23:42:09.127000", "content": "I have tried to run [Cervix segmentation (GMM)] on Colfax cluster. The script works fine on the login node but it keeps dying. On the compute node I get the error [module 'sklearn.mixture' has no attribute 'GaussianMixture']. Does anyone know how to solve this problem?
", "votes": 0, "replies": [] }, { "id": 176841, "author_name": "alutrin", "author_url": "", "post_date": "2017-04-22T09:15:19.067000", "content": "I ran some tests comparing the performance of colfax to my laptop (mid-2012 macbook pro). My laptop is a lot faster even when running NNs in keras/theano. I use the precompiled theano library provided on the cluster. I don't understand why this ultra high performance compute cluster is 10 times slower than my old laptop. I am not using the laptop's GPU, just CPU. I guess the Theano they provide is not optimized at all. Does anyone have better results? How can I run theano quickly on the colfax cluster?
\n\nA simple dense network ran in 21 seconds on my laptop but took 225 seconds on the cluster.
", "votes": 0, "replies": [ { "id": 177015, "author_name": " Van-Cao Nguyen ", "author_url": "", "post_date": "2017-04-23T05:19:02.870000", "content": "which framework have you use, Keras, Caffe, mxnet..?
", "votes": 0, "replies": [] }, { "id": 177192, "author_name": "alutrin", "author_url": "", "post_date": "2017-04-24T07:36:15.800000", "content": "Keras and Theano but Keras + TF is also giving me trouble. I wonder if anyone has been successful using one of the common python DL frameworks on colfax. If so I would greatly appreciate how to set up a good environment. I'm now using floydhub instead and that just works perfectly straight out of the box.
", "votes": 0, "replies": [] } ] }, { "id": 176449, "author_name": "Joseph Chui", "author_url": "", "post_date": "2017-04-20T18:00:44.383000", "content": "There is currently an online seminar going on by Colfax.
", "votes": 0, "replies": [ { "id": 176719, "author_name": " Van-Cao Nguyen ", "author_url": "", "post_date": "2017-04-21T17:01:47.787000", "content": "Could you show me link of the seminar, \nthanks.
", "votes": 0, "replies": [] } ] }, { "id": 176368, "author_name": "", "author_url": "", "post_date": "2017-04-20T07:54:33.787000", "content": "", "votes": 0, "replies": [ { "id": 176716, "author_name": " Van-Cao Nguyen ", "author_url": "", "post_date": "2017-04-21T16:56:49.633000", "content": "I tried to compile OpenCv from source but it was not success because of lacking some library. \nI would try to parallelism the code. \nThank you.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "176710": "My jupyter notebook running on Colfax keeps dying and restarting after processing about 40 images ", "175612": "Hi all, \nI have tried running [Cervix segmentation (GMM)][1] as a starting point. I downloaded the notebook as Python script and ran it on Colfax's compute node. \nAll required modules such as opencv-python, scikit-learn, scikit-image are installed in a conda environment, there was no error but it took about over 400 seconds per image to process. \nIs there anyone have the same problem? Any suggestion to solve?\nThank you.\n\n\nUpdate: Tensorflow has better performance than Theano in normal installation and configuration.\n\n\n [1]: https://www.kaggle.com/chattob/intel-mobileodt-cervical-cancer-screening/cervix-segmentation-gmm \"", "178830": "I have tried to run [Cervix segmentation (GMM)] on Colfax cluster. The script works fine on the login node but it keeps dying. On the compute node I get the error [module 'sklearn.mixture' has no attribute 'GaussianMixture']. Does anyone know how to solve this problem?", "176841": "I ran some tests comparing the performance of colfax to my laptop (mid-2012 macbook pro). My laptop is a lot faster even when running NNs in keras/theano. I use the precompiled theano library provided on the cluster. I don't understand why this ultra high performance compute cluster is 10 times slower than my old laptop. I am not using the laptop's GPU, just CPU. I guess the Theano they provide is not optimized at all. Does anyone have better results? How can I run theano quickly on the colfax cluster?\n\nA simple dense network ran in 21 seconds on my laptop but took 225 seconds on the cluster.", "176449": "There is currently an online seminar going on by Colfax. ", "176368": "" } }