My models running locally are taking a lot of time, so I thought I'd explore using my GCP credit. I got a VM running, setup the SSH keys, installed Anaconda, Tensorflow and Jupyter. But I'm stuck getting Jupyter to display locally in a browser, or getting X11 running so I can login to Kaggle and use wget to download the competition files. I'm using Windows 10 to communicate with the GCP.
\n\n1) I tried to follow these instructions to use Jupyter with GCP:\nhttps://hackernoon.com/launch-a-gpu-backed-google-compute-engine-instance-and-setup-tensorflow-keras-and-jupyter-902369ed5272
\n\nI'm trying to follow the step under \"Runing the lab\":
\n\nOn your local machine, open your browser and head to: http://[EXTERNAL_IP_ADDRESS]:8888
\n\nBut when I use the external IP address for my GCP instance, all I see is \"This site can't be reached. [my external IP address] took too long to respond.\"
\n\n2) I also tried to follow these instructions to use Jupyter with GCP:
\nhttps://haroldsoh.com/2016/04/28/set-up-anaconda-ipython-tensorflow-julia-on-a-google-compute-engine-vm/ These instructions are over a year old, so perhaps some things have changed.
I was able to get Jupyter started on the server, and authenticate myself using \"gcloud init\" under Windows. But I'm not sure what to do for the \"initiate a SSH tunnel from your local machine to the server. The ssh-flags in the article do not work in my hands on Windows 10.
\n\nThis page\nhttps://cloud.google.com/sdk/gcloud/reference/compute/ssh\nseemed to suggest something like:\ngcloud compute ssh example-instance --zone us-central1-a
\n\nThe logs me in fine into a PuTTY terminal. But the next Windows command starts a Chrome browser, but does not show Jupyter on the server:\n\"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe\" -proxy-server=\"http://localhost:8123/?token=f1c8d2d98988143ff8c6ba0b48660336cee4d1f3b1bb8beb \" -host-resolver-rules=\"MAP * 0.0.0.0 , EXCLUDE localhost\" -user-data-dir=\\tmp\\
\n\n3) X11. I often use PuTTY/XMing to connect with local Linux machines from Windows. I configured PuTTY to talk to the GCP after getting the SSH key setup. In the configuration under SSH, then X11, I enabled X11 forwarding and setup X display location as localhost:0.0. I also tried using the external IP address for the GCP server as the X display location. I've always had problems with the X11 DISPLAY variable.
\n\nI'd like to run firefox to login to Kaggle and then perhaps wget would download the real test/train files instead of an html error notice that I'm not logged in?
\n\nAny help or advice would be appreciated.
", "messages": [ { "id": "265905", "postDate": "01/07/2018 03:48:19", "content": "My models running locally are taking a lot of time, so I thought I'd explore using my GCP credit. I got a VM running, setup the SSH keys, installed Anaconda, Tensorflow and Jupyter. But I'm stuck getting Jupyter to display locally in a browser, or getting X11 running so I can login to Kaggle and use wget to download the competition files. I'm using Windows 10 to communicate with the GCP.
\n\n1) I tried to follow these instructions to use Jupyter with GCP:\nhttps://hackernoon.com/launch-a-gpu-backed-google-compute-engine-instance-and-setup-tensorflow-keras-and-jupyter-902369ed5272
\n\nI'm trying to follow the step under \"Runing the lab\":
\n\nOn your local machine, open your browser and head to: http://[EXTERNAL_IP_ADDRESS]:8888
\n\nBut when I use the external IP address for my GCP instance, all I see is \"This site can't be reached. [my external IP address] took too long to respond.\"
\n\n2) I also tried to follow these instructions to use Jupyter with GCP:
\nhttps://haroldsoh.com/2016/04/28/set-up-anaconda-ipython-tensorflow-julia-on-a-google-compute-engine-vm/ These instructions are over a year old, so perhaps some things have changed.
I was able to get Jupyter started on the server, and authenticate myself using \"gcloud init\" under Windows. But I'm not sure what to do for the \"initiate a SSH tunnel from your local machine to the server. The ssh-flags in the article do not work in my hands on Windows 10.
\n\nThis page\nhttps://cloud.google.com/sdk/gcloud/reference/compute/ssh\nseemed to suggest something like:\ngcloud compute ssh example-instance --zone us-central1-a
\n\nThe logs me in fine into a PuTTY terminal. But the next Windows command starts a Chrome browser, but does not show Jupyter on the server:\n\"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe\" -proxy-server=\"http://localhost:8123/?token=f1c8d2d98988143ff8c6ba0b48660336cee4d1f3b1bb8beb \" -host-resolver-rules=\"MAP * 0.0.0.0 , EXCLUDE localhost\" -user-data-dir=\\tmp\\
\n\n3) X11. I often use PuTTY/XMing to connect with local Linux machines from Windows. I configured PuTTY to talk to the GCP after getting the SSH key setup. In the configuration under SSH, then X11, I enabled X11 forwarding and setup X display location as localhost:0.0. I also tried using the external IP address for the GCP server as the X display location. I've always had problems with the X11 DISPLAY variable.
\n\nI'd like to run firefox to login to Kaggle and then perhaps wget would download the real test/train files instead of an html error notice that I'm not logged in?
\n\nAny help or advice would be appreciated.
", "rawMarkdown": "My models running locally are taking a lot of time, so I thought I'd explore using my GCP credit. I got a VM running, setup the SSH keys, installed Anaconda, Tensorflow and Jupyter. But I'm stuck getting Jupyter to display locally in a browser, or getting X11 running so I can login to Kaggle and use wget to download the competition files. I'm using Windows 10 to communicate with the GCP.\n\n**1) I tried to follow these instructions to use Jupyter with GCP:**\nhttps://hackernoon.com/launch-a-gpu-backed-google-compute-engine-instance-and-setup-tensorflow-keras-and-jupyter-902369ed5272\n\nI'm trying to follow the step under \"Runing the lab\":\n\nOn your local machine, open your browser and head to: http://[EXTERNAL_IP_ADDRESS]:8888\n\nBut when I use the external IP address for my GCP instance, all I see is \"This site can't be reached. [my external IP address] took too long to respond.\"\n\n**2) I also tried to follow these instructions to use Jupyter with GCP:** \nhttps://haroldsoh.com/2016/04/28/set-up-anaconda-ipython-tensorflow-julia-on-a-google-compute-engine-vm/ These instructions are over a year old, so perhaps some things have changed. \n\nI was able to get Jupyter started on the server, and authenticate myself using \"gcloud init\" under Windows. But I'm not sure what to do for the \"initiate a SSH tunnel from your local machine to the server. The ssh-flags in the article do not work in my hands on Windows 10.\n\nThis page\nhttps://cloud.google.com/sdk/gcloud/reference/compute/ssh\nseemed to suggest something like:\ngcloud compute ssh example-instance --zone us-central1-a\n\nThe logs me in fine into a PuTTY terminal. But the next Windows command starts a Chrome browser, but does not show Jupyter on the server:\n\"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe\" -proxy-server=\"http://localhost:8123/?token=f1c8d2d98988143ff8c6ba0b48660336cee4d1f3b1bb8beb \" -host-resolver-rules=\"MAP * 0.0.0.0 , EXCLUDE localhost\" -user-data-dir=\\tmp\\\n\n**3) X11.** I often use PuTTY/XMing to connect with local Linux machines from Windows. I configured PuTTY to talk to the GCP after getting the SSH key setup. In the configuration under SSH, then X11, I enabled X11 forwarding and setup X display location as localhost:0.0. I also tried using the external IP address for the GCP server as the X display location. I've always had problems with the X11 DISPLAY variable. \n\nI'd like to run firefox to login to Kaggle and then perhaps wget would download the real test/train files instead of an html error notice that I'm not logged in? \n\nAny help or advice would be appreciated.", "votes": null }, { "id": "265971", "postDate": "01/07/2018 09:28:53", "content": "Hi, you can download data without X using only command line: use https://github.com/floydwch/kaggle-cli
\n\nFor jupyter you are missing port forwarding. I haven't tried this on Windows, but I guess the steps are the same as in Linux:
\n\n1) ssh login to VM
\n\n2) run \"jupyter notebook\" on VM
\n\n3) Now port forward local (for VM) jupyter to you local PC by doing the following command on you PC:
\n\n\nssh -N -f -L localhost:8888:localhost:8888 username@[VM EXTERNAL IP]\n
so now go to localhost:8888 - it should work
", "rawMarkdown": "Hi, you can download data without X using only command line: use https://github.com/floydwch/kaggle-cli\n\nFor jupyter you are missing port forwarding. I haven't tried this on Windows, but I guess the steps are the same as in Linux:\n\n1) ssh login to VM\n\n2) run \"jupyter notebook\" on VM\n\n3) Now port forward local (for VM) jupyter to you local PC by doing the following command on you PC:\n\n\n```\nssh -N -f -L localhost:8888:localhost:8888 username@[VM EXTERNAL IP]\n```\n\nso now go to localhost:8888 - it should work", "votes": null }, { "id": "266100", "postDate": "01/07/2018 20:49:11", "content": "By default Jupyter runs on localhost only, and google firewall denies the access. With these instructions, you can change it to run in all IPs, create a password to protect unauthorized access, and enable the network access through GCP.
\n\nIf you don’t already have one, create a config file for the notebook using the following command:
\n\njupyter notebook --generate-config\nCreate a password:
\n\njupyter notebook password\nEdit file ~/.jupyter/jupyter_notebook_config.py and config/uncomment the following options:
\n\nc.NotebookApp.ip = '*'\nc.NotebookApp.open_browser = False\nRestart your jupyter notebook server
\n\njupyter notebook\nThe last item is to enable the network access via the GCP:
\n\nGo to cloud.google.com, my Console, choose you Project, choose Networking > VPC network, choose \"Firewalls rules\", choose Create Firewall Rule.
\n\nallow-jupyterAll instances in the network0.0.0.0/0Specified protocols and ports and write tcp:8888 in the textbox below.Now you can access your jupyter remotely from your home, via http://[EXTERNAL_IP_ADDRESS]:8888
", "rawMarkdown": "By default Jupyter runs on localhost only, and google firewall denies the access. With these instructions, you can change it to run in all IPs, create a password to protect unauthorized access, and enable the network access through GCP.\n\nIf you don’t already have one, create a config file for the notebook using the following command:\n\n jupyter notebook --generate-config\n\nCreate a password:\n\n jupyter notebook password\n\nEdit file ~/.jupyter/jupyter_notebook_config.py and config/uncomment the following options:\n\n c.NotebookApp.ip = '*'\n c.NotebookApp.open_browser = False\n\nRestart your jupyter notebook server\n\n jupyter notebook\n\nThe last item is to enable the network access via the GCP:\n\nGo to cloud.google.com, my Console, choose you Project, choose Networking > VPC network, choose \"Firewalls rules\", choose Create Firewall Rule.\n\n - In **Name** write `allow-jupyter`\n - In **Targets** dropdown choose `All instances in the network`\n - In **Source IP ranges** write `0.0.0.0/0`\n - In **Protocols and ports** choose `Specified protocols and ports` and write `tcp:8888` in the textbox below.\n - Click **Create**.\n\nNow you can access your jupyter remotely from your home, via http://[EXTERNAL_IP_ADDRESS]:8888", "votes": null } ], "comments": [ { "id": 265971, "author_name": "heyt0ny", "author_url": "", "post_date": "01/07/2018 09:28:53", "content": "Hi, you can download data without X using only command line: use https://github.com/floydwch/kaggle-cli
\n\nFor jupyter you are missing port forwarding. I haven't tried this on Windows, but I guess the steps are the same as in Linux:
\n\n1) ssh login to VM
\n\n2) run \"jupyter notebook\" on VM
\n\n3) Now port forward local (for VM) jupyter to you local PC by doing the following command on you PC:
\n\n\nssh -N -f -L localhost:8888:localhost:8888 username@[VM EXTERNAL IP]\n
so now go to localhost:8888 - it should work
", "votes": null, "replies": [] }, { "id": 266100, "author_name": "theofilos", "author_url": "", "post_date": "01/07/2018 20:49:11", "content": "By default Jupyter runs on localhost only, and google firewall denies the access. With these instructions, you can change it to run in all IPs, create a password to protect unauthorized access, and enable the network access through GCP.
\n\nIf you don’t already have one, create a config file for the notebook using the following command:
\n\njupyter notebook --generate-config\nCreate a password:
\n\njupyter notebook password\nEdit file ~/.jupyter/jupyter_notebook_config.py and config/uncomment the following options:
\n\nc.NotebookApp.ip = '*'\nc.NotebookApp.open_browser = False\nRestart your jupyter notebook server
\n\njupyter notebook\nThe last item is to enable the network access via the GCP:
\n\nGo to cloud.google.com, my Console, choose you Project, choose Networking > VPC network, choose \"Firewalls rules\", choose Create Firewall Rule.
\n\nallow-jupyterAll instances in the network0.0.0.0/0Specified protocols and ports and write tcp:8888 in the textbox below.Now you can access your jupyter remotely from your home, via http://[EXTERNAL_IP_ADDRESS]:8888
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "265905": "My models running locally are taking a lot of time, so I thought I'd explore using my GCP credit. I got a VM running, setup the SSH keys, installed Anaconda, Tensorflow and Jupyter. But I'm stuck getting Jupyter to display locally in a browser, or getting X11 running so I can login to Kaggle and use wget to download the competition files. I'm using Windows 10 to communicate with the GCP.\n\n**1) I tried to follow these instructions to use Jupyter with GCP:**\nhttps://hackernoon.com/launch-a-gpu-backed-google-compute-engine-instance-and-setup-tensorflow-keras-and-jupyter-902369ed5272\n\nI'm trying to follow the step under \"Runing the lab\":\n\nOn your local machine, open your browser and head to: http://[EXTERNAL_IP_ADDRESS]:8888\n\nBut when I use the external IP address for my GCP instance, all I see is \"This site can't be reached. [my external IP address] took too long to respond.\"\n\n**2) I also tried to follow these instructions to use Jupyter with GCP:** \nhttps://haroldsoh.com/2016/04/28/set-up-anaconda-ipython-tensorflow-julia-on-a-google-compute-engine-vm/ These instructions are over a year old, so perhaps some things have changed. \n\nI was able to get Jupyter started on the server, and authenticate myself using \"gcloud init\" under Windows. But I'm not sure what to do for the \"initiate a SSH tunnel from your local machine to the server. The ssh-flags in the article do not work in my hands on Windows 10.\n\nThis page\nhttps://cloud.google.com/sdk/gcloud/reference/compute/ssh\nseemed to suggest something like:\ngcloud compute ssh example-instance --zone us-central1-a\n\nThe logs me in fine into a PuTTY terminal. But the next Windows command starts a Chrome browser, but does not show Jupyter on the server:\n\"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe\" -proxy-server=\"http://localhost:8123/?token=f1c8d2d98988143ff8c6ba0b48660336cee4d1f3b1bb8beb \" -host-resolver-rules=\"MAP * 0.0.0.0 , EXCLUDE localhost\" -user-data-dir=\\tmp\\\n\n**3) X11.** I often use PuTTY/XMing to connect with local Linux machines from Windows. I configured PuTTY to talk to the GCP after getting the SSH key setup. In the configuration under SSH, then X11, I enabled X11 forwarding and setup X display location as localhost:0.0. I also tried using the external IP address for the GCP server as the X display location. I've always had problems with the X11 DISPLAY variable. \n\nI'd like to run firefox to login to Kaggle and then perhaps wget would download the real test/train files instead of an html error notice that I'm not logged in? \n\nAny help or advice would be appreciated.", "265971": "Hi, you can download data without X using only command line: use https://github.com/floydwch/kaggle-cli\n\nFor jupyter you are missing port forwarding. I haven't tried this on Windows, but I guess the steps are the same as in Linux:\n\n1) ssh login to VM\n\n2) run \"jupyter notebook\" on VM\n\n3) Now port forward local (for VM) jupyter to you local PC by doing the following command on you PC:\n\n\n```\nssh -N -f -L localhost:8888:localhost:8888 username@[VM EXTERNAL IP]\n```\n\nso now go to localhost:8888 - it should work", "266100": "By default Jupyter runs on localhost only, and google firewall denies the access. With these instructions, you can change it to run in all IPs, create a password to protect unauthorized access, and enable the network access through GCP.\n\nIf you don’t already have one, create a config file for the notebook using the following command:\n\n jupyter notebook --generate-config\n\nCreate a password:\n\n jupyter notebook password\n\nEdit file ~/.jupyter/jupyter_notebook_config.py and config/uncomment the following options:\n\n c.NotebookApp.ip = '*'\n c.NotebookApp.open_browser = False\n\nRestart your jupyter notebook server\n\n jupyter notebook\n\nThe last item is to enable the network access via the GCP:\n\nGo to cloud.google.com, my Console, choose you Project, choose Networking > VPC network, choose \"Firewalls rules\", choose Create Firewall Rule.\n\n - In **Name** write `allow-jupyter`\n - In **Targets** dropdown choose `All instances in the network`\n - In **Source IP ranges** write `0.0.0.0/0`\n - In **Protocols and ports** choose `Specified protocols and ports` and write `tcp:8888` in the textbox below.\n - Click **Create**.\n\nNow you can access your jupyter remotely from your home, via http://[EXTERNAL_IP_ADDRESS]:8888" }, "source": "meta" }