{
  "id": 35221,
  "title": "Minimum number of tiles to train U-net from scratch?",
  "url": "/competitions/noaa-fisheries-steller-sea-lion-population-count/discussion/35221",
  "author_name": "",
  "post_date": "2017-06-24T13:34:50.563867700Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>What is a minimum number of tiles to train U-net from scratch? I want to avoid creating large DB and waiting for too long, just want to try if it work at all.</p>",
  "messages": [
    {
      "id": "195675",
      "postDate": "06/24/2017 13:34:50",
      "content": "<p>What is a minimum number of tiles to train U-net from scratch? I want to avoid creating large DB and waiting for too long, just want to try if it work at all.</p>",
      "rawMarkdown": "What is a minimum number of tiles to train U-net from scratch? I want to avoid creating large DB and waiting for too long, just want to try if it work at all.",
      "votes": null
    },
    {
      "id": "195680",
      "postDate": "06/24/2017 14:12:22",
      "content": "<p>Which tile size are you considering?</p>",
      "rawMarkdown": "Which tile size are you considering?",
      "votes": null
    },
    {
      "id": "195703",
      "postDate": "06/24/2017 15:45:11",
      "content": "<p>I'm trying to use 256x256, but in case of training U-net from scratch I can use any tile size that can fit in 8Gb GPU memory.</p>",
      "rawMarkdown": "I'm trying to use 256x256, but in case of training U-net from scratch I can use any tile size that can fit in 8Gb GPU memory.",
      "votes": null
    },
    {
      "id": "195704",
      "postDate": "06/24/2017 15:51:41",
      "content": "<p>I even try to overfeat to one single sample but seems on channels where we have no objects network outputs garbage.</p>\n\n<p><img src=\"https://habrastorage.org/web/508/e9d/850/508e9d8506b849ebb51677d8750ed4f5.png\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://habrastorage.org/web/0db/e63/5b5/0dbe635b56f54b01b4fe867a378e5b4c.png\" alt=\"enter image description here\" title=\"\"></p>",
      "rawMarkdown": "I even try to overfeat to one single sample but seems on channels where we have no objects network outputs garbage.\n\n![enter image description here][1]\n![enter image description here][2]\n\n\n  [1]: https://habrastorage.org/web/508/e9d/850/508e9d8506b849ebb51677d8750ed4f5.png\n  [2]: https://habrastorage.org/web/0db/e63/5b5/0dbe635b56f54b01b4fe867a378e5b4c.png",
      "votes": null
    },
    {
      "id": "195941",
      "postDate": "06/25/2017 20:30:08",
      "content": "<p>being a relative late-comer to this competition, my initial results using u-net from scratch look pretty decent.  one model has a bit over one million images(includes augmentation) and the other has ~700k images. first model is 196x196 and the other is 92x92. both operate patch based on resized versions of the images (half height and half width).  However, in the home stretch, it will take 25 hours for my first model to do its thing...likely around the same time for the next!  I'm wishing I divided the images by 3 or 4 instead!  Nonetheless, learned a lot which is why I'm playing.\nas far as a DB...you're right...storing 1M images in raw form takes a lot of space.  If you are not up to speed on real-time augmentation, then Caffe allows for storing jpgs in folders instead. I've used that method in the past and it worked quite well.  For this, I'm using the generator features in keras...it's worth learning but I feel still has limitations for problems like this one that I had to work around.</p>",
      "rawMarkdown": "being a relative late-comer to this competition, my initial results using u-net from scratch look pretty decent.  one model has a bit over one million images(includes augmentation) and the other has ~700k images. first model is 196x196 and the other is 92x92. both operate patch based on resized versions of the images (half height and half width).  However, in the home stretch, it will take 25 hours for my first model to do its thing...likely around the same time for the next!  I'm wishing I divided the images by 3 or 4 instead!  Nonetheless, learned a lot which is why I'm playing.\nas far as a DB...you're right...storing 1M images in raw form takes a lot of space.  If you are not up to speed on real-time augmentation, then Caffe allows for storing jpgs in folders instead. I've used that method in the past and it worked quite well.  For this, I'm using the generator features in keras...it's worth learning but I feel still has limitations for problems like this one that I had to work around.",
      "votes": null
    },
    {
      "id": "195988",
      "postDate": "06/26/2017 00:42:06",
      "content": "<p>it's possible the matplotlib just scales range each image, your result may be ok but values range of 0 - 0.001 for example can be scaled to 0..1.0 range for display.</p>\n\n<p>You can try to specify the range when plotting to 0 .. 1.0, or add a point of 1.0 value to the corner of your image</p>",
      "rawMarkdown": "it's possible the matplotlib just scales range each image, your result may be ok but values range of 0 - 0.001 for example can be scaled to 0..1.0 range for display.\n\nYou can try to specify the range when plotting to 0 .. 1.0, or add a point of 1.0 value to the corner of your image",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 195680,
      "author_name": "firolino",
      "author_url": "",
      "post_date": "06/24/2017 14:12:22",
      "content": "<p>Which tile size are you considering?</p>",
      "votes": null,
      "replies": [
        {
          "id": 195703,
          "author_name": "mrgloom",
          "author_url": "",
          "post_date": "06/24/2017 15:45:11",
          "content": "<p>I'm trying to use 256x256, but in case of training U-net from scratch I can use any tile size that can fit in 8Gb GPU memory.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 195704,
          "author_name": "mrgloom",
          "author_url": "",
          "post_date": "06/24/2017 15:51:41",
          "content": "<p>I even try to overfeat to one single sample but seems on channels where we have no objects network outputs garbage.</p>\n\n<p><img src=\"https://habrastorage.org/web/508/e9d/850/508e9d8506b849ebb51677d8750ed4f5.png\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://habrastorage.org/web/0db/e63/5b5/0dbe635b56f54b01b4fe867a378e5b4c.png\" alt=\"enter image description here\" title=\"\"></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 195988,
          "author_name": "dmytropoplavskiy",
          "author_url": "",
          "post_date": "06/26/2017 00:42:06",
          "content": "<p>it's possible the matplotlib just scales range each image, your result may be ok but values range of 0 - 0.001 for example can be scaled to 0..1.0 range for display.</p>\n\n<p>You can try to specify the range when plotting to 0 .. 1.0, or add a point of 1.0 value to the corner of your image</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 195941,
      "author_name": "zerozero",
      "author_url": "",
      "post_date": "06/25/2017 20:30:08",
      "content": "<p>being a relative late-comer to this competition, my initial results using u-net from scratch look pretty decent.  one model has a bit over one million images(includes augmentation) and the other has ~700k images. first model is 196x196 and the other is 92x92. both operate patch based on resized versions of the images (half height and half width).  However, in the home stretch, it will take 25 hours for my first model to do its thing...likely around the same time for the next!  I'm wishing I divided the images by 3 or 4 instead!  Nonetheless, learned a lot which is why I'm playing.\nas far as a DB...you're right...storing 1M images in raw form takes a lot of space.  If you are not up to speed on real-time augmentation, then Caffe allows for storing jpgs in folders instead. I've used that method in the past and it worked quite well.  For this, I'm using the generator features in keras...it's worth learning but I feel still has limitations for problems like this one that I had to work around.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "195675": "What is a minimum number of tiles to train U-net from scratch? I want to avoid creating large DB and waiting for too long, just want to try if it work at all.",
    "195680": "Which tile size are you considering?",
    "195703": "I'm trying to use 256x256, but in case of training U-net from scratch I can use any tile size that can fit in 8Gb GPU memory.",
    "195704": "I even try to overfeat to one single sample but seems on channels where we have no objects network outputs garbage.\n\n![enter image description here][1]\n![enter image description here][2]\n\n\n  [1]: https://habrastorage.org/web/508/e9d/850/508e9d8506b849ebb51677d8750ed4f5.png\n  [2]: https://habrastorage.org/web/0db/e63/5b5/0dbe635b56f54b01b4fe867a378e5b4c.png",
    "195941": "being a relative late-comer to this competition, my initial results using u-net from scratch look pretty decent.  one model has a bit over one million images(includes augmentation) and the other has ~700k images. first model is 196x196 and the other is 92x92. both operate patch based on resized versions of the images (half height and half width).  However, in the home stretch, it will take 25 hours for my first model to do its thing...likely around the same time for the next!  I'm wishing I divided the images by 3 or 4 instead!  Nonetheless, learned a lot which is why I'm playing.\nas far as a DB...you're right...storing 1M images in raw form takes a lot of space.  If you are not up to speed on real-time augmentation, then Caffe allows for storing jpgs in folders instead. I've used that method in the past and it worked quite well.  For this, I'm using the generator features in keras...it's worth learning but I feel still has limitations for problems like this one that I had to work around.",
    "195988": "it's possible the matplotlib just scales range each image, your result may be ok but values range of 0 - 0.001 for example can be scaled to 0..1.0 range for display.\n\nYou can try to specify the range when plotting to 0 .. 1.0, or add a point of 1.0 value to the corner of your image"
  },
  "source": "meta"
}