{
  "id": 125923,
  "title": "Notebooks on GCP - change path",
  "url": "/competitions/tensorflow2-question-answering/discussion/125923",
  "author_name": "",
  "post_date": "2020-01-14T14:46:05.554913400Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I'm trying to use the GCP's AI Notebooks (<a href=\"https://console.cloud.google.com/ai-platform/notebooks/\">https://console.cloud.google.com/ai-platform/notebooks/</a>). However, when you %pip install a package, it gets installed outside of the system's path you can't call it from the shell. Tried to change the default path but I'm doing something wrong. Would appreciate suggestions.</p>\n\n<p>Here's an example:</p>\n\n<p>```\n[1] %pip install kaggle --user\n ... Installing collected packages: kaggle\n  WARNING: The script kaggle is installed in '/home/jupyter/.local/bin' which is not on PATH.\n  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\nSuccessfully installed kaggle-1.5.6\nNote: you may need to restart the kernel to use updated packages.</p>\n\n<h1>restarted the kernel</h1>\n\n<p>[2] import kaggle\n #works\n[3] !pip show kaggle\nName: kaggle\nVersion: 1.5.6\n...\nLocation: /home/jupyter/.local/lib/python3.5/site-packages\n...\n[4] !kaggle -v #doesn't work\n/bin/sh: 1: kaggle: not found</p>\n\n<p>[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !echo \"export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin\" &gt;&gt; ~/.profile </p>\n\n<h1>no output</h1>\n\n<p>[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games</p>\n\n<h1>path is unchanged</h1>\n\n<p>```</p>",
  "messages": [
    {
      "id": "718572",
      "postDate": "01/14/2020 14:46:05",
      "content": "<p>I'm trying to use the GCP's AI Notebooks (<a href=\"https://console.cloud.google.com/ai-platform/notebooks/\">https://console.cloud.google.com/ai-platform/notebooks/</a>). However, when you %pip install a package, it gets installed outside of the system's path you can't call it from the shell. Tried to change the default path but I'm doing something wrong. Would appreciate suggestions.</p>\n\n<p>Here's an example:</p>\n\n<p>```\n[1] %pip install kaggle --user\n ... Installing collected packages: kaggle\n  WARNING: The script kaggle is installed in '/home/jupyter/.local/bin' which is not on PATH.\n  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\nSuccessfully installed kaggle-1.5.6\nNote: you may need to restart the kernel to use updated packages.</p>\n\n<h1>restarted the kernel</h1>\n\n<p>[2] import kaggle\n #works\n[3] !pip show kaggle\nName: kaggle\nVersion: 1.5.6\n...\nLocation: /home/jupyter/.local/lib/python3.5/site-packages\n...\n[4] !kaggle -v #doesn't work\n/bin/sh: 1: kaggle: not found</p>\n\n<p>[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !echo \"export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin\" &gt;&gt; ~/.profile </p>\n\n<h1>no output</h1>\n\n<p>[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games</p>\n\n<h1>path is unchanged</h1>\n\n<p>```</p>",
      "rawMarkdown": "I'm trying to use the GCP's AI Notebooks (https://console.cloud.google.com/ai-platform/notebooks/). However, when you %pip install a package, it gets installed outside of the system's path you can't call it from the shell. Tried to change the default path but I'm doing something wrong. Would appreciate suggestions.\n\nHere's an example:\n\n\n```\n[1] %pip install kaggle --user\n ... Installing collected packages: kaggle\n  WARNING: The script kaggle is installed in '/home/jupyter/.local/bin' which is not on PATH.\n  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\nSuccessfully installed kaggle-1.5.6\nNote: you may need to restart the kernel to use updated packages.\n#restarted the kernel\n[2] import kaggle\n #works\n[3] !pip show kaggle\nName: kaggle\nVersion: 1.5.6\n...\nLocation: /home/jupyter/.local/lib/python3.5/site-packages\n...\n[4] !kaggle -v #doesn't work\n/bin/sh: 1: kaggle: not found\n\n[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !echo \"export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin\" &gt;&gt; ~/.profile \n#no output\n[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n# path is unchanged\n```",
      "votes": null
    },
    {
      "id": "718598",
      "postDate": "01/14/2020 15:13:00",
      "content": "<p>Two things that I might try:</p>\n\n<ul>\n<li><p>Don't append to \"~/.profile\" but just run: <code>!export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin</code>. This should (at least) change $PATH ;)</p></li>\n<li><p>Where does the package get installed without <code>--user</code>? Maybe with <code>sudo</code>: <code>sudo pip install --upgrade kaggle</code>. I don't use notebooks but this allows my to use the kaggle-cli in GCP:</p></li>\n</ul>\n\n<p><code>\nsudo pip install --upgrade kaggle\nmkdir .kaggle\necho '{\"username\":\"MYUSER\",\"key\":\"MYKEY\"}' &gt; ~/.kaggle/kaggle.json\nchmod 600 ~/.kaggle/kaggle.json\nkaggle competitions download -c tensorflow2-question-answering\n</code></p>",
      "rawMarkdown": "Two things that I might try:\n\n* Don't append to \"~/.profile\" but just run: `!export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin`. This should (at least) change $PATH ;)\n\n* Where does the package get installed without `--user`? Maybe with `sudo`: `sudo pip install --upgrade kaggle`. I don't use notebooks but this allows my to use the kaggle-cli in GCP:\n\n```\nsudo pip install --upgrade kaggle\nmkdir .kaggle\necho '{\"username\":\"MYUSER\",\"key\":\"MYKEY\"}' &gt; ~/.kaggle/kaggle.json\nchmod 600 ~/.kaggle/kaggle.json\nkaggle competitions download -c tensorflow2-question-answering\n```",
      "votes": null
    },
    {
      "id": "718652",
      "postDate": "01/14/2020 16:03:41",
      "content": "<p>Thanks. Suggestion #1 didn't work for me:</p>\n\n<p>```\n[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !export PATH=$PATH:/home/jupyter/.local/lib/python3.5/site-packages </p>\n\n<h1>no output</h1>\n\n<p>[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games</p>\n\n<h1>path is unchanged</h1>\n\n<p>```</p>\n\n<p>Suggestion #2 also didn't work because I get a permission denied error when trying to install the package without the --user option. The kaggle package is successfully installed and I can import it into python - I can even call it from the terminal, just not from the notebook, which has the rest of my code.</p>\n\n<p>Curious: If you don't use notebooks, how/where do you debug your code?</p>",
      "rawMarkdown": "Thanks. Suggestion #1 didn't work for me:\n\n```\n[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !export PATH=$PATH:/home/jupyter/.local/lib/python3.5/site-packages \n#no output\n[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n# path is unchanged\n```\n\nSuggestion #2 also didn't work because I get a permission denied error when trying to install the package without the --user option. The kaggle package is successfully installed and I can import it into python - I can even call it from the terminal, just not from the notebook, which has the rest of my code.\n\nCurious: If you don't use notebooks, how/where do you debug your code?",
      "votes": null
    },
    {
      "id": "718664",
      "postDate": "01/14/2020 16:16:26",
      "content": "<p>Hm, <code>sudo</code> should do the trick, like I wrote. Anyway, maybe someone else is more familiar with notebooks. I mostly use python scripts and an IDE :)</p>",
      "rawMarkdown": "Hm, `sudo` should do the trick, like I wrote. Anyway, maybe someone else is more familiar with notebooks. I mostly use python scripts and an IDE :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 718598,
      "author_name": "seesee",
      "author_url": "",
      "post_date": "01/14/2020 15:13:00",
      "content": "<p>Two things that I might try:</p>\n\n<ul>\n<li><p>Don't append to \"~/.profile\" but just run: <code>!export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin</code>. This should (at least) change $PATH ;)</p></li>\n<li><p>Where does the package get installed without <code>--user</code>? Maybe with <code>sudo</code>: <code>sudo pip install --upgrade kaggle</code>. I don't use notebooks but this allows my to use the kaggle-cli in GCP:</p></li>\n</ul>\n\n<p><code>\nsudo pip install --upgrade kaggle\nmkdir .kaggle\necho '{\"username\":\"MYUSER\",\"key\":\"MYKEY\"}' &gt; ~/.kaggle/kaggle.json\nchmod 600 ~/.kaggle/kaggle.json\nkaggle competitions download -c tensorflow2-question-answering\n</code></p>",
      "votes": null,
      "replies": [
        {
          "id": 718652,
          "author_name": "alonbochman",
          "author_url": "",
          "post_date": "01/14/2020 16:03:41",
          "content": "<p>Thanks. Suggestion #1 didn't work for me:</p>\n\n<p>```\n[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !export PATH=$PATH:/home/jupyter/.local/lib/python3.5/site-packages </p>\n\n<h1>no output</h1>\n\n<p>[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games</p>\n\n<h1>path is unchanged</h1>\n\n<p>```</p>\n\n<p>Suggestion #2 also didn't work because I get a permission denied error when trying to install the package without the --user option. The kaggle package is successfully installed and I can import it into python - I can even call it from the terminal, just not from the notebook, which has the rest of my code.</p>\n\n<p>Curious: If you don't use notebooks, how/where do you debug your code?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 718664,
          "author_name": "seesee",
          "author_url": "",
          "post_date": "01/14/2020 16:16:26",
          "content": "<p>Hm, <code>sudo</code> should do the trick, like I wrote. Anyway, maybe someone else is more familiar with notebooks. I mostly use python scripts and an IDE :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "718572": "I'm trying to use the GCP's AI Notebooks (https://console.cloud.google.com/ai-platform/notebooks/). However, when you %pip install a package, it gets installed outside of the system's path you can't call it from the shell. Tried to change the default path but I'm doing something wrong. Would appreciate suggestions.\n\nHere's an example:\n\n\n```\n[1] %pip install kaggle --user\n ... Installing collected packages: kaggle\n  WARNING: The script kaggle is installed in '/home/jupyter/.local/bin' which is not on PATH.\n  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\nSuccessfully installed kaggle-1.5.6\nNote: you may need to restart the kernel to use updated packages.\n#restarted the kernel\n[2] import kaggle\n #works\n[3] !pip show kaggle\nName: kaggle\nVersion: 1.5.6\n...\nLocation: /home/jupyter/.local/lib/python3.5/site-packages\n...\n[4] !kaggle -v #doesn't work\n/bin/sh: 1: kaggle: not found\n\n[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !echo \"export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin\" &gt;&gt; ~/.profile \n#no output\n[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n# path is unchanged\n```",
    "718598": "Two things that I might try:\n\n* Don't append to \"~/.profile\" but just run: `!export PATH=\\$PATH:$(python -c \"import site; print(site.getuserbase())\")/bin`. This should (at least) change $PATH ;)\n\n* Where does the package get installed without `--user`? Maybe with `sudo`: `sudo pip install --upgrade kaggle`. I don't use notebooks but this allows my to use the kaggle-cli in GCP:\n\n```\nsudo pip install --upgrade kaggle\nmkdir .kaggle\necho '{\"username\":\"MYUSER\",\"key\":\"MYKEY\"}' &gt; ~/.kaggle/kaggle.json\nchmod 600 ~/.kaggle/kaggle.json\nkaggle competitions download -c tensorflow2-question-answering\n```",
    "718652": "Thanks. Suggestion #1 didn't work for me:\n\n```\n[5] !echo $PATH\n /usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n[6] !export PATH=$PATH:/home/jupyter/.local/lib/python3.5/site-packages \n#no output\n[7] !echo $PATH\n/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games\n# path is unchanged\n```\n\nSuggestion #2 also didn't work because I get a permission denied error when trying to install the package without the --user option. The kaggle package is successfully installed and I can import it into python - I can even call it from the terminal, just not from the notebook, which has the rest of my code.\n\nCurious: If you don't use notebooks, how/where do you debug your code?",
    "718664": "Hm, `sudo` should do the trick, like I wrote. Anyway, maybe someone else is more familiar with notebooks. I mostly use python scripts and an IDE :)"
  },
  "source": "meta"
}