{
  "id": 54985,
  "title": "92 GiB is too much...",
  "url": "/competitions/cvpr-2018-autonomous-driving/discussion/54985",
  "author_name": "",
  "post_date": "2018-04-20T15:58:54.386882500Z",
  "votes": 4,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I don't have acess to any GPU hardware of my own. I seek to train my models on online free platforms like floydhub, neptune or google colab. The dataset of 92 GiB is way much bigger dataset to train any such platforms. Help me get across this problem. Suggest some techniques or hacks to train my model.</p>",
  "messages": [
    {
      "id": "317062",
      "postDate": "04/20/2018 15:58:54",
      "content": "<p>I don't have acess to any GPU hardware of my own. I seek to train my models on online free platforms like floydhub, neptune or google colab. The dataset of 92 GiB is way much bigger dataset to train any such platforms. Help me get across this problem. Suggest some techniques or hacks to train my model.</p>",
      "rawMarkdown": "I don't have acess to any GPU hardware of my own. I seek to train my models on online free platforms like floydhub, neptune or google colab. The dataset of 92 GiB is way much bigger dataset to train any such platforms. Help me get across this problem. Suggest some techniques or hacks to train my model.",
      "votes": null
    },
    {
      "id": "317938",
      "postDate": "04/22/2018 21:47:17",
      "content": "<p>I ran into the same issue, I made a kernel which focuses on the 1500 most interesting images from a vehicle perspective and trains a model on them (<a href=\"https://www.kaggle.com/kmader/vehicle-unet-fcl-segmentation\">https://www.kaggle.com/kmader/vehicle-unet-fcl-segmentation</a>). You probably want some empty images as well, but taking random samples with the different features you want to classify is probably more efficient than powering through the whole 92gb</p>",
      "rawMarkdown": "I ran into the same issue, I made a kernel which focuses on the 1500 most interesting images from a vehicle perspective and trains a model on them (https://www.kaggle.com/kmader/vehicle-unet-fcl-segmentation). You probably want some empty images as well, but taking random samples with the different features you want to classify is probably more efficient than powering through the whole 92gb",
      "votes": null
    },
    {
      "id": "318072",
      "postDate": "04/23/2018 06:13:35",
      "content": "<p>Thanks for the suggestion. But just trying to understand how you have taken consideration of interesting images? Is it that you have randomly picked up 1500 images?</p>",
      "rawMarkdown": "Thanks for the suggestion. But just trying to understand how you have taken consideration of interesting images? Is it that you have randomly picked up 1500 images?",
      "votes": null
    },
    {
      "id": "318091",
      "postDate": "04/23/2018 07:13:55",
      "content": "<p>No, if you look at the code and the kernel it imports (<a href=\"https://www.kaggle.com/kmader/label-analysis\">https://www.kaggle.com/kmader/label-analysis</a>) it takes the instanceIds image to pick the 1500 images with the most vehicles (sum over multiple label types).</p>",
      "rawMarkdown": "No, if you look at the code and the kernel it imports (https://www.kaggle.com/kmader/label-analysis) it takes the instanceIds image to pick the 1500 images with the most vehicles (sum over multiple label types).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 317938,
      "author_name": "kmader",
      "author_url": "",
      "post_date": "04/22/2018 21:47:17",
      "content": "<p>I ran into the same issue, I made a kernel which focuses on the 1500 most interesting images from a vehicle perspective and trains a model on them (<a href=\"https://www.kaggle.com/kmader/vehicle-unet-fcl-segmentation\">https://www.kaggle.com/kmader/vehicle-unet-fcl-segmentation</a>). You probably want some empty images as well, but taking random samples with the different features you want to classify is probably more efficient than powering through the whole 92gb</p>",
      "votes": null,
      "replies": [
        {
          "id": 318072,
          "author_name": "manishhag",
          "author_url": "",
          "post_date": "04/23/2018 06:13:35",
          "content": "<p>Thanks for the suggestion. But just trying to understand how you have taken consideration of interesting images? Is it that you have randomly picked up 1500 images?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 318091,
          "author_name": "kmader",
          "author_url": "",
          "post_date": "04/23/2018 07:13:55",
          "content": "<p>No, if you look at the code and the kernel it imports (<a href=\"https://www.kaggle.com/kmader/label-analysis\">https://www.kaggle.com/kmader/label-analysis</a>) it takes the instanceIds image to pick the 1500 images with the most vehicles (sum over multiple label types).</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "317062": "I don't have acess to any GPU hardware of my own. I seek to train my models on online free platforms like floydhub, neptune or google colab. The dataset of 92 GiB is way much bigger dataset to train any such platforms. Help me get across this problem. Suggest some techniques or hacks to train my model.",
    "317938": "I ran into the same issue, I made a kernel which focuses on the 1500 most interesting images from a vehicle perspective and trains a model on them (https://www.kaggle.com/kmader/vehicle-unet-fcl-segmentation). You probably want some empty images as well, but taking random samples with the different features you want to classify is probably more efficient than powering through the whole 92gb",
    "318072": "Thanks for the suggestion. But just trying to understand how you have taken consideration of interesting images? Is it that you have randomly picked up 1500 images?",
    "318091": "No, if you look at the code and the kernel it imports (https://www.kaggle.com/kmader/label-analysis) it takes the instanceIds image to pick the 1500 images with the most vehicles (sum over multiple label types)."
  },
  "source": "meta"
}