{
  "id": 37026,
  "title": "Reading data from kaggle-tsa-stage1 bucket",
  "url": "/competitions/passenger-screening-algorithm-challenge/discussion/37026",
  "author_name": "",
  "post_date": "2017-07-26T03:47:42.315231Z",
  "votes": 2,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Hello Google Cloud Advocates,</p>\n\n<p>I was wondering if it is possible to run a job on Google Cloud ML-engine and read the training data directly from gs://kaggle-tsa-stage1? Or would I need specific permissions to the TSA storage bucket to do that? I'm trying to save money by not re-uploading the training data to my own cloud storage bucket for training.</p>",
  "messages": [
    {
      "id": "207224",
      "postDate": "07/26/2017 03:47:42",
      "content": "<p>Hello Google Cloud Advocates,</p>\n\n<p>I was wondering if it is possible to run a job on Google Cloud ML-engine and read the training data directly from gs://kaggle-tsa-stage1? Or would I need specific permissions to the TSA storage bucket to do that? I'm trying to save money by not re-uploading the training data to my own cloud storage bucket for training.</p>",
      "rawMarkdown": "Hello Google Cloud Advocates,\n\nI was wondering if it is possible to run a job on Google Cloud ML-engine and read the training data directly from gs://kaggle-tsa-stage1? Or would I need specific permissions to the TSA storage bucket to do that? I'm trying to save money by not re-uploading the training data to my own cloud storage bucket for training.",
      "votes": null
    },
    {
      "id": "207230",
      "postDate": "07/26/2017 04:24:31",
      "content": "<p>ML Engine and Datalab access data with a service account, not with your Google account. that means you need copy the data in the TSA storage bucket to your own bucket, and add permission to let the service account accessing data. Sorry for inconvenience, but it's recommended not to use any specific personal account when building a data analytic system to make it secure and portable. If you create your GCS bucket in US region, copying the entire 3TB data should take less than 30 minutes (please use -m option for parallel copy: eg. \"gsutil -m cp ...\").</p>",
      "rawMarkdown": "ML Engine and Datalab access data with a service account, not with your Google account. that means you need copy the data in the TSA storage bucket to your own bucket, and add permission to let the service account accessing data. Sorry for inconvenience, but it's recommended not to use any specific personal account when building a data analytic system to make it secure and portable. If you create your GCS bucket in US region, copying the entire 3TB data should take less than 30 minutes (please use -m option for parallel copy: eg. \"gsutil -m cp ...\").",
      "votes": null
    },
    {
      "id": "207539",
      "postDate": "07/26/2017 20:56:02",
      "content": "<p>Ah good to know, thank you for the answer.</p>",
      "rawMarkdown": "Ah good to know, thank you for the answer.",
      "votes": null
    },
    {
      "id": "207822",
      "postDate": "07/27/2017 15:40:34",
      "content": "<p>Hi dojochu - Make sure to apply for free cloud credits to help cover the cost of storage. Read more about it here: <a href=\"https://www.kaggle.com/c/passenger-screening-algorithm-challenge/discussion/37104\">Apply for FREE Google Cloud Credits from Kaggle!</a></p>",
      "rawMarkdown": "Hi dojochu - Make sure to apply for free cloud credits to help cover the cost of storage. Read more about it here: [Apply for FREE Google Cloud Credits from Kaggle!](https://www.kaggle.com/c/passenger-screening-algorithm-challenge/discussion/37104)",
      "votes": null
    },
    {
      "id": "207828",
      "postDate": "07/27/2017 15:54:31",
      "content": "<p>Thanks!!</p>",
      "rawMarkdown": "Thanks!!",
      "votes": null
    },
    {
      "id": "208408",
      "postDate": "07/29/2017 17:39:56",
      "content": "<p>When I try to copy to my bucket, the system says:</p>\n\n<p>\"You don't have permissions to perform the action on the selected resource\"</p>\n\n<p>How might I obtain such permissions.</p>",
      "rawMarkdown": "When I try to copy to my bucket, the system says:\n\n\"You don't have permissions to perform the action on the selected resource\"\n\nHow might I obtain such permissions.",
      "votes": null
    },
    {
      "id": "208443",
      "postDate": "07/29/2017 20:01:17",
      "content": "<p>Hi Pete, go to this <a href=\"https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data\">https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data</a> and read the instructions under Data size and access</p>",
      "rawMarkdown": "Hi Pete, go to this https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data and read the instructions under Data size and access",
      "votes": null
    },
    {
      "id": "208493",
      "postDate": "07/30/2017 00:52:52",
      "content": "<p>Thanks. I see that my mistake was not registering and verifying that I had read the rules. Mistake corrected! Thanks.</p>",
      "rawMarkdown": "Thanks. I see that my mistake was not registering and verifying that I had read the rules. Mistake corrected! Thanks.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 207230,
      "author_name": "kazunori279",
      "author_url": "",
      "post_date": "07/26/2017 04:24:31",
      "content": "<p>ML Engine and Datalab access data with a service account, not with your Google account. that means you need copy the data in the TSA storage bucket to your own bucket, and add permission to let the service account accessing data. Sorry for inconvenience, but it's recommended not to use any specific personal account when building a data analytic system to make it secure and portable. If you create your GCS bucket in US region, copying the entire 3TB data should take less than 30 minutes (please use -m option for parallel copy: eg. \"gsutil -m cp ...\").</p>",
      "votes": null,
      "replies": [
        {
          "id": 207539,
          "author_name": "dojochu",
          "author_url": "",
          "post_date": "07/26/2017 20:56:02",
          "content": "<p>Ah good to know, thank you for the answer.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 207822,
          "author_name": "stevegreenberg",
          "author_url": "",
          "post_date": "07/27/2017 15:40:34",
          "content": "<p>Hi dojochu - Make sure to apply for free cloud credits to help cover the cost of storage. Read more about it here: <a href=\"https://www.kaggle.com/c/passenger-screening-algorithm-challenge/discussion/37104\">Apply for FREE Google Cloud Credits from Kaggle!</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 207828,
          "author_name": "dojochu",
          "author_url": "",
          "post_date": "07/27/2017 15:54:31",
          "content": "<p>Thanks!!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 208408,
      "author_name": "petewills",
      "author_url": "",
      "post_date": "07/29/2017 17:39:56",
      "content": "<p>When I try to copy to my bucket, the system says:</p>\n\n<p>\"You don't have permissions to perform the action on the selected resource\"</p>\n\n<p>How might I obtain such permissions.</p>",
      "votes": null,
      "replies": [
        {
          "id": 208443,
          "author_name": "dojochu",
          "author_url": "",
          "post_date": "07/29/2017 20:01:17",
          "content": "<p>Hi Pete, go to this <a href=\"https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data\">https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data</a> and read the instructions under Data size and access</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 208493,
          "author_name": "petewills",
          "author_url": "",
          "post_date": "07/30/2017 00:52:52",
          "content": "<p>Thanks. I see that my mistake was not registering and verifying that I had read the rules. Mistake corrected! Thanks.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "207224": "Hello Google Cloud Advocates,\n\nI was wondering if it is possible to run a job on Google Cloud ML-engine and read the training data directly from gs://kaggle-tsa-stage1? Or would I need specific permissions to the TSA storage bucket to do that? I'm trying to save money by not re-uploading the training data to my own cloud storage bucket for training.",
    "207230": "ML Engine and Datalab access data with a service account, not with your Google account. that means you need copy the data in the TSA storage bucket to your own bucket, and add permission to let the service account accessing data. Sorry for inconvenience, but it's recommended not to use any specific personal account when building a data analytic system to make it secure and portable. If you create your GCS bucket in US region, copying the entire 3TB data should take less than 30 minutes (please use -m option for parallel copy: eg. \"gsutil -m cp ...\").",
    "207539": "Ah good to know, thank you for the answer.",
    "207822": "Hi dojochu - Make sure to apply for free cloud credits to help cover the cost of storage. Read more about it here: [Apply for FREE Google Cloud Credits from Kaggle!](https://www.kaggle.com/c/passenger-screening-algorithm-challenge/discussion/37104)",
    "207828": "Thanks!!",
    "208408": "When I try to copy to my bucket, the system says:\n\n\"You don't have permissions to perform the action on the selected resource\"\n\nHow might I obtain such permissions.",
    "208443": "Hi Pete, go to this https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data and read the instructions under Data size and access",
    "208493": "Thanks. I see that my mistake was not registering and verifying that I had read the rules. Mistake corrected! Thanks."
  },
  "source": "meta"
}