{
  "id": 43259,
  "title": "How do I access data remotely?",
  "url": "/competitions/cdiscount-image-classification-challenge/discussion/43259",
  "author_name": "",
  "post_date": "2017-11-12T11:15:18.053182800Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hey everybody,</p>\n\n<p>This is my first time dealing with big data like this. The dataset is huge and I don't have space on my computer to store it. However I do have enough RAM and GPUs...</p>\n\n<p>For what I see, running kernels on Kaggle will take hours and wont lead to any good results. Can you give me any ideas on how to access data remotely so that I don't need to download the dataset directly?</p>\n\n<p>Thanks in advance</p>\n\n<p>Edit: I could store it in somewhere else and access it via IP. But is this viable?</p>",
  "messages": [
    {
      "id": "242678",
      "postDate": "11/12/2017 11:15:18",
      "content": "<p>Hey everybody,</p>\n\n<p>This is my first time dealing with big data like this. The dataset is huge and I don't have space on my computer to store it. However I do have enough RAM and GPUs...</p>\n\n<p>For what I see, running kernels on Kaggle will take hours and wont lead to any good results. Can you give me any ideas on how to access data remotely so that I don't need to download the dataset directly?</p>\n\n<p>Thanks in advance</p>\n\n<p>Edit: I could store it in somewhere else and access it via IP. But is this viable?</p>",
      "rawMarkdown": "Hey everybody,\n\nThis is my first time dealing with big data like this. The dataset is huge and I don't have space on my computer to store it. However I do have enough RAM and GPUs...\n\nFor what I see, running kernels on Kaggle will take hours and wont lead to any good results. Can you give me any ideas on how to access data remotely so that I don't need to download the dataset directly?\n\nThanks in advance\n\nEdit: I could store it in somewhere else and access it via IP. But is this viable?",
      "votes": null
    },
    {
      "id": "242701",
      "postDate": "11/12/2017 12:14:03",
      "content": "<p>Why don't just buy another HDD? HDDs are super cheap these days...</p>",
      "rawMarkdown": "Why don't just buy another HDD? HDDs are super cheap these days...",
      "votes": null
    },
    {
      "id": "242703",
      "postDate": "11/12/2017 12:18:26",
      "content": "<p>I do no have access by ssh to that pc I talked about but it is not mine.  What about that solution of storing data somewhere else and access it by ssh?</p>",
      "rawMarkdown": "I do no have access by ssh to that pc I talked about but it is not mine.  What about that solution of storing data somewhere else and access it by ssh?",
      "votes": null
    },
    {
      "id": "243894",
      "postDate": "11/15/2017 05:14:59",
      "content": "<p>Hey, perhaps you can consider training using an Amazon Web Service p2 instance. <a href=\"https://aws.amazon.com/ec2/instance-types/\">https://aws.amazon.com/ec2/instance-types/</a>\nor any other cloud service. You can then download the data on the instance and train on it.</p>",
      "rawMarkdown": "Hey, perhaps you can consider training using an Amazon Web Service p2 instance. https://aws.amazon.com/ec2/instance-types/\nor any other cloud service. You can then download the data on the instance and train on it.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 242701,
      "author_name": "timjoseph",
      "author_url": "",
      "post_date": "11/12/2017 12:14:03",
      "content": "<p>Why don't just buy another HDD? HDDs are super cheap these days...</p>",
      "votes": null,
      "replies": [
        {
          "id": 242703,
          "author_name": "skinish",
          "author_url": "",
          "post_date": "11/12/2017 12:18:26",
          "content": "<p>I do no have access by ssh to that pc I talked about but it is not mine.  What about that solution of storing data somewhere else and access it by ssh?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 243894,
      "author_name": "jzerox2",
      "author_url": "",
      "post_date": "11/15/2017 05:14:59",
      "content": "<p>Hey, perhaps you can consider training using an Amazon Web Service p2 instance. <a href=\"https://aws.amazon.com/ec2/instance-types/\">https://aws.amazon.com/ec2/instance-types/</a>\nor any other cloud service. You can then download the data on the instance and train on it.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "242678": "Hey everybody,\n\nThis is my first time dealing with big data like this. The dataset is huge and I don't have space on my computer to store it. However I do have enough RAM and GPUs...\n\nFor what I see, running kernels on Kaggle will take hours and wont lead to any good results. Can you give me any ideas on how to access data remotely so that I don't need to download the dataset directly?\n\nThanks in advance\n\nEdit: I could store it in somewhere else and access it via IP. But is this viable?",
    "242701": "Why don't just buy another HDD? HDDs are super cheap these days...",
    "242703": "I do no have access by ssh to that pc I talked about but it is not mine.  What about that solution of storing data somewhere else and access it by ssh?",
    "243894": "Hey, perhaps you can consider training using an Amazon Web Service p2 instance. https://aws.amazon.com/ec2/instance-types/\nor any other cloud service. You can then download the data on the instance and train on it."
  },
  "source": "meta"
}