Hi, have you encountered problem with access to Kaggle Dataset from Colab (TPU)?
\nLast week something has changed and I can't read images from TFRecords. Till this day there was no problem to use Colab and priv Kaggle DS.
Configuration:
\nPermissionDeniedError: Error executing an HTTP request: HTTP response code 403 with body '{\n \"error\": {\n \"code\": 403,\n \"message\": \"service-495559152420@cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.\",\n \"errors\": [\n {\n \"message\": \"service-495559152420@cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.\",\n \"domain\": \"global\",\n \"reason\": \"forbidden\"\n }\n ]\n }\n}\nHi, have you encountered problem with access to Kaggle Dataset from Colab (TPU)?
\nLast week something has changed and I can't read images from TFRecords. Till this day there was no problem to use Colab and priv Kaggle DS.
Configuration:
\nPermissionDeniedError: Error executing an HTTP request: HTTP response code 403 with body '{\n \"error\": {\n \"code\": 403,\n \"message\": \"service-495559152420@cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.\",\n \"errors\": [\n {\n \"message\": \"service-495559152420@cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.\",\n \"domain\": \"global\",\n \"reason\": \"forbidden\"\n }\n ]\n }\n}\nHi, Remek. The problem you are having is probably the same problem I have raised in this discussion. I am not very familiar with GCS, but I think the error is probably due to a change in google colab authentication method, as mentioned in this link(https://github.com/googlecolab/colabtools/issues/2710#issuecomment-1086328194). I haven't found a fundamental solution, but if you put the data directly into GCS Instead of getting the GCS path from the kaggle dataset, you should be able to load the images.
", "rawMarkdown": "Hi, Remek. The problem you are having is probably the same problem I have raised in this [discussion](https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/316140). I am not very familiar with GCS, but I think the error is probably due to a change in google colab authentication method, as mentioned in this [link](url)(https://github.com/googlecolab/colabtools/issues/2710#issuecomment-1086328194). I haven't found a fundamental solution, but if you put the data directly into GCS Instead of getting the GCS path from the kaggle dataset, you should be able to load the images. ", "votes": 3, "replies": [ { "id": 1745968, "postDate": "2022-04-05T12:47:16.730Z", "content": "Thank you!!! ๐ช๐ช๐ค๐๐
", "rawMarkdown": "Thank you!!! ๐ช๐ช๐ค๐๐", "votes": 1 } ] }, { "id": 1746303, "postDate": "2022-04-05T16:40:32.210Z", "content": "I just encountered this error in the afternoon. The first time was caused by my GCS writing wrong or expired. The second time I encountered it, it disappeared after I refreshed the page, but I don't know what caused it. of๐
", "rawMarkdown": "I just encountered this error in the afternoon. The first time was caused by my GCS writing wrong or expired. The second time I encountered it, it disappeared after I refreshed the page, but I don't know what caused it. of๐", "votes": 1 }, { "id": 1746019, "postDate": "2022-04-05T13:11:17.603Z", "content": "I'd rather use google cloud storage. but it should be careful about billing
\nthe private dataset in kaggle can be connected from google colab? i know that it can't
Yes. We used Kaggle private dataset as a gcp bucket all the time. Something had changed last week and now public one is accessible only :)
", "rawMarkdown": "Yes. We used Kaggle private dataset as a gcp bucket all the time. Something had changed last week and now public one is accessible only :)", "votes": 1 } ] }, { "id": 1752172, "postDate": "2022-04-11T13:43:51.107Z", "content": "I got the same error and access behavior with normal Colab and my own GCS bucket. I solved it by changing the bucket access permissions: Added a \"principal\" with that email \"service-495559152420@cloud-tpu.iam.gserviceaccount.com\" and assigned the role \"Storage Object Viewer\".
\nNow, that obviously won't work with a private kaggle dataset. Anyone found a solution here?
", "rawMarkdown": "I got the same error and access behavior with normal Colab and my own GCS bucket. I solved it by changing the bucket access permissions: Added a \"principal\" with that email \"service-495559152420@cloud-tpu.iam.gserviceaccount.com\" and assigned the role \"Storage Object Viewer\".\n\nNow, that obviously won't work with a private kaggle dataset. Anyone found a solution here?" }, { "id": 1746055, "postDate": "2022-04-05T13:30:17.220Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1745966, "author_name": "kurokami", "author_url": "", "post_date": "2022-04-05T12:45:20.837000", "content": "Hi, Remek. The problem you are having is probably the same problem I have raised in this discussion. I am not very familiar with GCS, but I think the error is probably due to a change in google colab authentication method, as mentioned in this link(https://github.com/googlecolab/colabtools/issues/2710#issuecomment-1086328194). I haven't found a fundamental solution, but if you put the data directly into GCS Instead of getting the GCS path from the kaggle dataset, you should be able to load the images.
", "votes": 3, "replies": [ { "id": 1745968, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-04-05T12:47:16.730000", "content": "Thank you!!! ๐ช๐ช๐ค๐๐
", "votes": 1, "replies": [] } ] }, { "id": 1746303, "author_name": "lyu", "author_url": "", "post_date": "2022-04-05T16:40:32.210000", "content": "I just encountered this error in the afternoon. The first time was caused by my GCS writing wrong or expired. The second time I encountered it, it disappeared after I refreshed the page, but I don't know what caused it. of๐
", "votes": 1, "replies": [] }, { "id": 1746019, "author_name": "kaggler", "author_url": "", "post_date": "2022-04-05T13:11:17.603000", "content": "I'd rather use google cloud storage. but it should be careful about billing
\nthe private dataset in kaggle can be connected from google colab? i know that it can't
Yes. We used Kaggle private dataset as a gcp bucket all the time. Something had changed last week and now public one is accessible only :)
", "votes": 1, "replies": [] } ] }, { "id": 1752172, "author_name": "Marius Wanko", "author_url": "", "post_date": "2022-04-11T13:43:51.107000", "content": "I got the same error and access behavior with normal Colab and my own GCS bucket. I solved it by changing the bucket access permissions: Added a \"principal\" with that email \"service-495559152420@cloud-tpu.iam.gserviceaccount.com\" and assigned the role \"Storage Object Viewer\".
\nNow, that obviously won't work with a private kaggle dataset. Anyone found a solution here?
", "votes": 0, "replies": [] }, { "id": 1746055, "author_name": "", "author_url": "", "post_date": "2022-04-05T13:30:17.220000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1745912": "Hi, have you encountered problem with access to Kaggle Dataset from Colab (TPU)?\nLast week something has changed and I can't read images from TFRecords. Till this day there was no problem to use Colab and priv Kaggle DS.\n\nConfiguration:\n- Colab Pro+ TPU\n- Kaggle Dataset is private\n- I can read TFRecords using gcp (bucket - gs://) - show length of TFRecord but can't access files inside\n- I get and error\n\n```\nPermissionDeniedError: Error executing an HTTP request: HTTP response code 403 with body '{\n \"error\": {\n \"code\": 403,\n \"message\": \"service-495559152420@cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.\",\n \"errors\": [\n {\n \"message\": \"service-495559152420@cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.\",\n \"domain\": \"global\",\n \"reason\": \"forbidden\"\n }\n ]\n }\n}\n```", "1745966": "Hi, Remek. The problem you are having is probably the same problem I have raised in this [discussion](https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/316140). I am not very familiar with GCS, but I think the error is probably due to a change in google colab authentication method, as mentioned in this [link](url)(https://github.com/googlecolab/colabtools/issues/2710#issuecomment-1086328194). I haven't found a fundamental solution, but if you put the data directly into GCS Instead of getting the GCS path from the kaggle dataset, you should be able to load the images. ", "1746303": "I just encountered this error in the afternoon. The first time was caused by my GCS writing wrong or expired. The second time I encountered it, it disappeared after I refreshed the page, but I don't know what caused it. of๐", "1746019": "I'd rather use google cloud storage. but it should be careful about billing \nthe private dataset in kaggle can be connected from google colab? i know that it can't", "1752172": "I got the same error and access behavior with normal Colab and my own GCS bucket. I solved it by changing the bucket access permissions: Added a \"principal\" with that email \"service-495559152420@cloud-tpu.iam.gserviceaccount.com\" and assigned the role \"Storage Object Viewer\".\n\nNow, that obviously won't work with a private kaggle dataset. Anyone found a solution here?", "1746055": "" } }