{
  "id": 316140,
  "title": "Cannot access to GCS from colab ",
  "url": "/competitions/happy-whale-and-dolphin/discussion/316140",
  "author_name": "",
  "post_date": "2022-03-31T12:49:55.275661400Z",
  "votes": 7,
  "comment_count": 10,
  "views": 0,
  "content": "<p>I am using google colab pro+ TPU and today I cannot connect from colab to GCS. I have been able to do this until yesterday and now suddenly I can't. Authorizing permissions to the TPU service account doesn't work. Is anyone else experiencing the same problem?</p>",
  "messages": [
    {
      "id": "1741007",
      "postDate": "03/31/2022 12:49:55",
      "content": "<p>I am using google colab pro+ TPU and today I cannot connect from colab to GCS. I have been able to do this until yesterday and now suddenly I can't. Authorizing permissions to the TPU service account doesn't work. Is anyone else experiencing the same problem?</p>",
      "rawMarkdown": "I am using google colab pro+ TPU and today I cannot connect from colab to GCS. I have been able to do this until yesterday and now suddenly I can't. Authorizing permissions to the TPU service account doesn't work. Is anyone else experiencing the same problem?",
      "votes": null
    },
    {
      "id": "1741037",
      "postDate": "03/31/2022 13:20:46",
      "content": "<p>I am not sure if the GCS expires, you need getting new one.</p>",
      "rawMarkdown": "I am not sure if the GCS expires, you need getting new one.",
      "votes": null
    },
    {
      "id": "1741038",
      "postDate": "03/31/2022 13:23:12",
      "content": "<p>Hi!<br>\nNow i am using TPU on colab pro but no problem.<br>\nwhen TPU can not use , push runtime of setting bar then formatting runtime.<br>\n(In Japanese is written with ''ランタイムを出荷時にリセット'')</p>\n<p>I wish your problem is solved by my suggest.</p>",
      "rawMarkdown": "Hi!\nNow i am using TPU on colab pro but no problem.\nwhen TPU can not use , push runtime of setting bar then formatting runtime.\n(In Japanese is written with ''ランタイムを出荷時にリセット'')\n\nI wish your problem is solved by my suggest.",
      "votes": null
    },
    {
      "id": "1741074",
      "postDate": "03/31/2022 13:49:55",
      "content": "<p>Thank you for reply. I tried it, but the problem still remains…  <br>\nI can connect to GCS correctly for a dataset I created a few weeks ago, but it doesn't seem to be working for a newly created dataset like the one I created yesterday. </p>",
      "rawMarkdown": "Thank you for reply. I tried it, but the problem still remains...  \nI can connect to GCS correctly for a dataset I created a few weeks ago, but it doesn't seem to be working for a newly created dataset like the one I created yesterday.",
      "votes": null
    },
    {
      "id": "1741081",
      "postDate": "03/31/2022 13:59:30",
      "content": "<p>Thank you for your reply. I tried it, but the problem still remains…<br>\nMay I ask you a question? You may run this cell when you connect to GCS from colab</p>\n<pre><code>from google.colab import auth\nauth.authenticate_user()\n</code></pre>\n<p>which window appears when run this code.<br>\n<img src=\"https://user-images.githubusercontent.com/62830735/161073585-edab12b9-bb36-4f5d-b4de-82721f0ead2b.png\" alt=\"\"><br>\nor<br>\n<img src=\"https://i0.wp.com/neptune.ai/wp-content/uploads/colab-code.png?resize=636%2C221&amp;ssl=1\" alt=\"\"></p>",
      "rawMarkdown": "Thank you for your reply. I tried it, but the problem still remains…\nMay I ask you a question? You may run this cell when you connect to GCS from colab\n```\nfrom google.colab import auth\nauth.authenticate_user()\n```\nwhich window appears when run this code.\n![](https://user-images.githubusercontent.com/62830735/161073585-edab12b9-bb36-4f5d-b4de-82721f0ead2b.png)\nor\n![](https://i0.wp.com/neptune.ai/wp-content/uploads/colab-code.png?resize=636%2C221&ssl=1)",
      "votes": null
    },
    {
      "id": "1741132",
      "postDate": "03/31/2022 14:38:07",
      "content": "<p>I am not sure if that colab is testing something.</p>\n<p>GCS not working for me a day ago for  a while,  error message that over limit of notebook public gcs access.  but later It recovered.  I access public dataset on kaggle.</p>",
      "rawMarkdown": "I am not sure if that colab is testing something.\n\nGCS not working for me a day ago for  a while,  error message that over limit of notebook public gcs access.  but later It recovered.  I access public dataset on kaggle.",
      "votes": null
    },
    {
      "id": "1741150",
      "postDate": "03/31/2022 14:51:22",
      "content": "<p>Same to me.</p>\n<p>I directly add permission to TPU to access to buckets.</p>\n<p><a href=\"https://cloud.google.com/tpu/docs/storage-buckets\" target=\"_blank\">https://cloud.google.com/tpu/docs/storage-buckets</a></p>",
      "rawMarkdown": "Same to me.\n\nI directly add permission to TPU to access to buckets.\n\nhttps://cloud.google.com/tpu/docs/storage-buckets",
      "votes": null
    },
    {
      "id": "1741875",
      "postDate": "04/01/2022 08:26:47",
      "content": "<p>Thank you for teaching me. I tried this but not work. I found that only the datasets I made don't work, and there may be something wrong with my process to get gcs path…</p>",
      "rawMarkdown": "Thank you for teaching me. I tried this but not work. I found that only the datasets I made don't work, and there may be something wrong with my process to get gcs path...",
      "votes": null
    },
    {
      "id": "1741993",
      "postDate": "04/01/2022 11:22:40",
      "content": "<p>Same to me.<br>\nPlease add permission to buckets. <br>\n<img src=\"https://imgur.com/a/6AXMT70\" alt=\"\"><br>\nHonestly idk how to get tpu service but reading from 403 error in the colab notebook.</p>",
      "rawMarkdown": "Same to me.\nPlease add permission to buckets. \n![](https://imgur.com/a/6AXMT70)\nHonestly idk how to get tpu service but reading from 403 error in the colab notebook.",
      "votes": null
    },
    {
      "id": "1742544",
      "postDate": "04/02/2022 02:45:18",
      "content": "<p>(You already mentioned you already tried,, my bad)<br>\nIs the GCS of the same account? In my case, it is solved by giving the GCS reading permission to the TPU service account in the Colab error message.</p>",
      "rawMarkdown": "~~Well, I solved it by giving the GCS reading permission to the TPU service account in the error message. Not sure this is a recommended way though,,~~(You already mentioned you already tried,, my bad)\nIs the GCS of the same account? In my case, it is solved by giving the GCS reading permission to the TPU service account in the Colab error message.",
      "votes": null
    },
    {
      "id": "1742666",
      "postDate": "04/02/2022 06:44:17",
      "content": "<p>Thanks for help. I am not familiar with GCS, but it worked if I put the dataset directly in GCS instead of getting the GCS path via kaggle dataset. Perhaps the problem was caused by google recently changing the authentication method from colab to GCS.</p>",
      "rawMarkdown": "Thanks for help. I am not familiar with GCS, but it worked if I put the dataset directly in GCS instead of getting the GCS path via kaggle dataset. Perhaps the problem was caused by google recently changing the authentication method from colab to GCS.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1741037,
      "author_name": "dragonzhang",
      "author_url": "",
      "post_date": "03/31/2022 13:20:46",
      "content": "<p>I am not sure if the GCS expires, you need getting new one.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1741074,
          "author_name": "ryutadokoro",
          "author_url": "",
          "post_date": "03/31/2022 13:49:55",
          "content": "<p>Thank you for reply. I tried it, but the problem still remains…  <br>\nI can connect to GCS correctly for a dataset I created a few weeks ago, but it doesn't seem to be working for a newly created dataset like the one I created yesterday. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1741132,
          "author_name": "dragonzhang",
          "author_url": "",
          "post_date": "03/31/2022 14:38:07",
          "content": "<p>I am not sure if that colab is testing something.</p>\n<p>GCS not working for me a day ago for  a while,  error message that over limit of notebook public gcs access.  but later It recovered.  I access public dataset on kaggle.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1741038,
      "author_name": "ishizakireo",
      "author_url": "",
      "post_date": "03/31/2022 13:23:12",
      "content": "<p>Hi!<br>\nNow i am using TPU on colab pro but no problem.<br>\nwhen TPU can not use , push runtime of setting bar then formatting runtime.<br>\n(In Japanese is written with ''ランタイムを出荷時にリセット'')</p>\n<p>I wish your problem is solved by my suggest.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1741081,
          "author_name": "ryutadokoro",
          "author_url": "",
          "post_date": "03/31/2022 13:59:30",
          "content": "<p>Thank you for your reply. I tried it, but the problem still remains…<br>\nMay I ask you a question? You may run this cell when you connect to GCS from colab</p>\n<pre><code>from google.colab import auth\nauth.authenticate_user()\n</code></pre>\n<p>which window appears when run this code.<br>\n<img src=\"https://user-images.githubusercontent.com/62830735/161073585-edab12b9-bb36-4f5d-b4de-82721f0ead2b.png\" alt=\"\"><br>\nor<br>\n<img src=\"https://i0.wp.com/neptune.ai/wp-content/uploads/colab-code.png?resize=636%2C221&amp;ssl=1\" alt=\"\"></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1741150,
      "author_name": "ren4yu",
      "author_url": "",
      "post_date": "03/31/2022 14:51:22",
      "content": "<p>Same to me.</p>\n<p>I directly add permission to TPU to access to buckets.</p>\n<p><a href=\"https://cloud.google.com/tpu/docs/storage-buckets\" target=\"_blank\">https://cloud.google.com/tpu/docs/storage-buckets</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1741875,
          "author_name": "ryutadokoro",
          "author_url": "",
          "post_date": "04/01/2022 08:26:47",
          "content": "<p>Thank you for teaching me. I tried this but not work. I found that only the datasets I made don't work, and there may be something wrong with my process to get gcs path…</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1741993,
      "author_name": "locbaop",
      "author_url": "",
      "post_date": "04/01/2022 11:22:40",
      "content": "<p>Same to me.<br>\nPlease add permission to buckets. <br>\n<img src=\"https://imgur.com/a/6AXMT70\" alt=\"\"><br>\nHonestly idk how to get tpu service but reading from 403 error in the colab notebook.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1742544,
      "author_name": "jihunlorenzopark",
      "author_url": "",
      "post_date": "04/02/2022 02:45:18",
      "content": "<p>(You already mentioned you already tried,, my bad)<br>\nIs the GCS of the same account? In my case, it is solved by giving the GCS reading permission to the TPU service account in the Colab error message.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1742666,
          "author_name": "ryutadokoro",
          "author_url": "",
          "post_date": "04/02/2022 06:44:17",
          "content": "<p>Thanks for help. I am not familiar with GCS, but it worked if I put the dataset directly in GCS instead of getting the GCS path via kaggle dataset. Perhaps the problem was caused by google recently changing the authentication method from colab to GCS.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1741007": "I am using google colab pro+ TPU and today I cannot connect from colab to GCS. I have been able to do this until yesterday and now suddenly I can't. Authorizing permissions to the TPU service account doesn't work. Is anyone else experiencing the same problem?",
    "1741037": "I am not sure if the GCS expires, you need getting new one.",
    "1741038": "Hi!\nNow i am using TPU on colab pro but no problem.\nwhen TPU can not use , push runtime of setting bar then formatting runtime.\n(In Japanese is written with ''ランタイムを出荷時にリセット'')\n\nI wish your problem is solved by my suggest.",
    "1741074": "Thank you for reply. I tried it, but the problem still remains...  \nI can connect to GCS correctly for a dataset I created a few weeks ago, but it doesn't seem to be working for a newly created dataset like the one I created yesterday.",
    "1741081": "Thank you for your reply. I tried it, but the problem still remains…\nMay I ask you a question? You may run this cell when you connect to GCS from colab\n```\nfrom google.colab import auth\nauth.authenticate_user()\n```\nwhich window appears when run this code.\n![](https://user-images.githubusercontent.com/62830735/161073585-edab12b9-bb36-4f5d-b4de-82721f0ead2b.png)\nor\n![](https://i0.wp.com/neptune.ai/wp-content/uploads/colab-code.png?resize=636%2C221&ssl=1)",
    "1741132": "I am not sure if that colab is testing something.\n\nGCS not working for me a day ago for  a while,  error message that over limit of notebook public gcs access.  but later It recovered.  I access public dataset on kaggle.",
    "1741150": "Same to me.\n\nI directly add permission to TPU to access to buckets.\n\nhttps://cloud.google.com/tpu/docs/storage-buckets",
    "1741875": "Thank you for teaching me. I tried this but not work. I found that only the datasets I made don't work, and there may be something wrong with my process to get gcs path...",
    "1741993": "Same to me.\nPlease add permission to buckets. \n![](https://imgur.com/a/6AXMT70)\nHonestly idk how to get tpu service but reading from 403 error in the colab notebook.",
    "1742544": "~~Well, I solved it by giving the GCS reading permission to the TPU service account in the error message. Not sure this is a recommended way though,,~~(You already mentioned you already tried,, my bad)\nIs the GCS of the same account? In my case, it is solved by giving the GCS reading permission to the TPU service account in the Colab error message.",
    "1742666": "Thanks for help. I am not familiar with GCS, but it worked if I put the dataset directly in GCS instead of getting the GCS path via kaggle dataset. Perhaps the problem was caused by google recently changing the authentication method from colab to GCS."
  },
  "source": "meta"
}