Can any one help i have got some 100 gb of data on my GCP VM instance and i want to download that to my another non gcp machine hosted on other server where i train the model .
", "messages": [ { "id": "771782", "postDate": "03/14/2020 16:00:33", "content": "Can any one help i have got some 100 gb of data on my GCP VM instance and i want to download that to my another non gcp machine hosted on other server where i train the model .
", "rawMarkdown": "Can any one help i have got some 100 gb of data on my GCP VM instance and i want to download that to my another non gcp machine hosted on other server where i train the model .", "votes": null }, { "id": "771798", "postDate": "03/14/2020 16:33:45", "content": "You can try to upload the data to Google bucket first, then download from Google bucket to GCP VM, you can read this
", "rawMarkdown": "You can try to upload the data to Google bucket first, then download from Google bucket to GCP VM, you can read [this](https://towardsdatascience.com/quickly-transfer-a-kaggle-dataset-into-a-google-bucket-ac21aefceb15)", "votes": null }, { "id": "771800", "postDate": "03/14/2020 16:37:50", "content": "@jaideepvalani \n1st option\n1. Attach external disk to your VM. copy all data then unmount and detach.\n2. Same external drive you connect to your GCP VM.\nhttps://www.cloudbooklet.com/attach-and-mount-disks-to-vm-instance-in-google-cloud/\nI did same in AWS. it will work in GCP VM too.
\n\n2nd option copy data to google bucket storage
", "rawMarkdown": "jaideepvalani \n1st option\n1. Attach external disk to your VM. copy all data then unmount and detach.\n2. Same external drive you connect to your GCP VM.\nhttps://www.cloudbooklet.com/attach-and-mount-disks-to-vm-instance-in-google-cloud/\nI did same in AWS. it will work in GCP VM too.\n\n\n2nd option copy data to google bucket storage", "votes": null }, { "id": "772141", "postDate": "03/15/2020 04:57:38", "content": "@seshurajup thanks \nMy req is to send the data out of gcp storage or vM . \nI think storage bucket shld work \n1. How to upload vM to storage bucket \n2. Once done how to download the storage bucket data to another machine where i work that is not gcp machine .
", "rawMarkdown": "seshurajup thanks \nMy req is to send the data out of gcp storage or vM . \nI think storage bucket shld work \n1. How to upload vM to storage bucket \n2. Once done how to download the storage bucket data to another machine where i work that is not gcp machine .", "votes": null }, { "id": "772143", "postDate": "03/15/2020 05:00:15", "content": "@noxuslol thanks \n1. How I can upload vM data to gcp bucket \n2. My req is to download gcp bucket data to my another machine which is non gcp
", "rawMarkdown": "noxuslol thanks \n1. How I can upload vM data to gcp bucket \n2. My req is to download gcp bucket data to my another machine which is non gcp", "votes": null }, { "id": "772153", "postDate": "03/15/2020 05:14:44", "content": "\n\n1) Uploading to a bucket: please refer to https://www.kaggle.com/c/deepfake-detection-challenge/discussion/128864
\n\n2) If you upload as a zip file to the bucket, you can download by right click in the bucket portal. Or, while downloading you can use Chrome CurlWGet extension to get wget link, then stop the download. Use `wget to download in local Linux VM. The above discussion has also mentioned using this extension.
@jaideepvalani Once you uploaded data to bucket, you need to download again. you can use from bucket\n1. create google storage bucket (https://cloud.google.com/storage/docs/creating-buckets)\n2. gcloud auth login\n3. gsutil -m cp -r [DATA FOLDER] gs://[BUCKET NAME]
", "rawMarkdown": "jaideepvalani Once you uploaded data to bucket, you need to download again. you can use from bucket\n1. create google storage bucket (https://cloud.google.com/storage/docs/creating-buckets)\n2. gcloud auth login\n3. gsutil -m cp -r [DATA FOLDER] gs://[BUCKET NAME]", "votes": null }, { "id": "772215", "postDate": "03/15/2020 07:10:13", "content": "@jaideepvalani this flow working faster
\n\nthanku @seshurajup in your second method \n1) i have to run the command on target machine\n2) ,i dont have to create a bucket and can copy the data via scp or how it is ?
\n\nSorry for q i have not dont before
", "rawMarkdown": "thanku @seshurajup in your second method \n1) i have to run the command on target machine\n2) ,i dont have to create a bucket and can copy the data via scp or how it is ?\n\nSorry for q i have not dont before", "votes": null } ], "comments": [ { "id": 771798, "author_name": "noxuslol", "author_url": "", "post_date": "03/14/2020 16:33:45", "content": "You can try to upload the data to Google bucket first, then download from Google bucket to GCP VM, you can read this
", "votes": null, "replies": [ { "id": 772143, "author_name": "jaideepvalani", "author_url": "", "post_date": "03/15/2020 05:00:15", "content": "@noxuslol thanks \n1. How I can upload vM data to gcp bucket \n2. My req is to download gcp bucket data to my another machine which is non gcp
", "votes": null, "replies": [] } ] }, { "id": 771800, "author_name": "seshurajup", "author_url": "", "post_date": "03/14/2020 16:37:50", "content": "@jaideepvalani \n1st option\n1. Attach external disk to your VM. copy all data then unmount and detach.\n2. Same external drive you connect to your GCP VM.\nhttps://www.cloudbooklet.com/attach-and-mount-disks-to-vm-instance-in-google-cloud/\nI did same in AWS. it will work in GCP VM too.
\n\n2nd option copy data to google bucket storage
", "votes": null, "replies": [ { "id": 772141, "author_name": "jaideepvalani", "author_url": "", "post_date": "03/15/2020 04:57:38", "content": "@seshurajup thanks \nMy req is to send the data out of gcp storage or vM . \nI think storage bucket shld work \n1. How to upload vM to storage bucket \n2. Once done how to download the storage bucket data to another machine where i work that is not gcp machine .
", "votes": null, "replies": [] }, { "id": 772158, "author_name": "seshurajup", "author_url": "", "post_date": "03/15/2020 05:30:30", "content": "@jaideepvalani Once you uploaded data to bucket, you need to download again. you can use from bucket\n1. create google storage bucket (https://cloud.google.com/storage/docs/creating-buckets)\n2. gcloud auth login\n3. gsutil -m cp -r [DATA FOLDER] gs://[BUCKET NAME]
", "votes": null, "replies": [] }, { "id": 772215, "author_name": "seshurajup", "author_url": "", "post_date": "03/15/2020 07:10:13", "content": "@jaideepvalani this flow working faster
\n\nthanku @seshurajup in your second method \n1) i have to run the command on target machine\n2) ,i dont have to create a bucket and can copy the data via scp or how it is ?
\n\nSorry for q i have not dont before
", "votes": null, "replies": [] } ] }, { "id": 772153, "author_name": "debanga", "author_url": "", "post_date": "03/15/2020 05:14:44", "content": "\n\n1) Uploading to a bucket: please refer to https://www.kaggle.com/c/deepfake-detection-challenge/discussion/128864
\n\n2) If you upload as a zip file to the bucket, you can download by right click in the bucket portal. Or, while downloading you can use Chrome CurlWGet extension to get wget link, then stop the download. Use `wget to download in local Linux VM. The above discussion has also mentioned using this extension.