{
  "id": 45787,
  "title": "Transfer learning with NasNet and 2h Epochs",
  "url": "/competitions/cdiscount-image-classification-challenge/discussion/45787",
  "author_name": "danber",
  "post_date": "2017-12-15T21:49:28.456000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<p>My idea was to train a single model with NasNet (to have a small chance I thought I should use the best what I can get). As I jumped in very late (only about one week before deadline) and I have limited resources, my choice was to use a pretrained net and train only the upper layers of large nasnet.</p>\n\n<p>To fasten things up, I calculated the bottleneck values (features of images before final prediction layer) and saved them to tfrecords files (~250GB). For prediction I tried to predict the different category levels and used those predictions as additional input of the upper next layer. </p>\n\n<p>Training needed about 2h per Epoch on Nvidia Tesla K80. For this I used Google CloudML. The creation of the bottleneck images took about 40h. In sum everything costed about 100€ (from the 300$ free credits for signing up).  But I needed about 50€ extra for some trial and error and mainly because the jobs were stopped by the server sometimes in the middle of the training for reasons unkown to me. I also could not submit them in time, as I has problems creating the predictions on CloudML. </p>\n\n<p>With my results I am a little bit disappointed, I only got 0.46329 on private leaderboard. I am not sure whether this is my fault, or due to the small input size of 90*90 pixels I used for time reasons (The image size only affects the time needed in the first run, as the bottleneck features I used had fixes size of 4032 values).  Network was trained for about 5 Epochs. No data augmentation was used. Input for prediction layer are features from four images, if product did not have four images zeros were used instead.</p>\n\n<p>Full code, written for tensorflow, with commands to run in cloudml, you can find under:</p>\n\n<pre><code>https://github.com/DanBmh/DiscountProducts\n</code></pre>\n\n<p>Maybe it can be some help to you for future Competitions:)</p>",
  "messages": [
    {
      "id": 258276,
      "postDate": "2017-12-15T21:49:28.457Z",
      "content": "<p>My idea was to train a single model with NasNet (to have a small chance I thought I should use the best what I can get). As I jumped in very late (only about one week before deadline) and I have limited resources, my choice was to use a pretrained net and train only the upper layers of large nasnet.</p>\n\n<p>To fasten things up, I calculated the bottleneck values (features of images before final prediction layer) and saved them to tfrecords files (~250GB). For prediction I tried to predict the different category levels and used those predictions as additional input of the upper next layer. </p>\n\n<p>Training needed about 2h per Epoch on Nvidia Tesla K80. For this I used Google CloudML. The creation of the bottleneck images took about 40h. In sum everything costed about 100€ (from the 300$ free credits for signing up).  But I needed about 50€ extra for some trial and error and mainly because the jobs were stopped by the server sometimes in the middle of the training for reasons unkown to me. I also could not submit them in time, as I has problems creating the predictions on CloudML. </p>\n\n<p>With my results I am a little bit disappointed, I only got 0.46329 on private leaderboard. I am not sure whether this is my fault, or due to the small input size of 90*90 pixels I used for time reasons (The image size only affects the time needed in the first run, as the bottleneck features I used had fixes size of 4032 values).  Network was trained for about 5 Epochs. No data augmentation was used. Input for prediction layer are features from four images, if product did not have four images zeros were used instead.</p>\n\n<p>Full code, written for tensorflow, with commands to run in cloudml, you can find under:</p>\n\n<pre><code>https://github.com/DanBmh/DiscountProducts\n</code></pre>\n\n<p>Maybe it can be some help to you for future Competitions:)</p>",
      "rawMarkdown": "My idea was to train a single model with NasNet (to have a small chance I thought I should use the best what I can get). As I jumped in very late (only about one week before deadline) and I have limited resources, my choice was to use a pretrained net and train only the upper layers of large nasnet.\n\nTo fasten things up, I calculated the bottleneck values (features of images before final prediction layer) and saved them to tfrecords files (~250GB). For prediction I tried to predict the different category levels and used those predictions as additional input of the upper next layer. \n\nTraining needed about 2h per Epoch on Nvidia Tesla K80. For this I used Google CloudML. The creation of the bottleneck images took about 40h. In sum everything costed about 100€ (from the 300$ free credits for signing up).  But I needed about 50€ extra for some trial and error and mainly because the jobs were stopped by the server sometimes in the middle of the training for reasons unkown to me. I also could not submit them in time, as I has problems creating the predictions on CloudML. \n\nWith my results I am a little bit disappointed, I only got 0.46329 on private leaderboard. I am not sure whether this is my fault, or due to the small input size of 90*90 pixels I used for time reasons (The image size only affects the time needed in the first run, as the bottleneck features I used had fixes size of 4032 values).  Network was trained for about 5 Epochs. No data augmentation was used. Input for prediction layer are features from four images, if product did not have four images zeros were used instead.\n\nFull code, written for tensorflow, with commands to run in cloudml, you can find under:\n\n    https://github.com/DanBmh/DiscountProducts\n\nMaybe it can be some help to you for future Competitions:)\n\n",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "258276": "My idea was to train a single model with NasNet (to have a small chance I thought I should use the best what I can get). As I jumped in very late (only about one week before deadline) and I have limited resources, my choice was to use a pretrained net and train only the upper layers of large nasnet.\n\nTo fasten things up, I calculated the bottleneck values (features of images before final prediction layer) and saved them to tfrecords files (~250GB). For prediction I tried to predict the different category levels and used those predictions as additional input of the upper next layer. \n\nTraining needed about 2h per Epoch on Nvidia Tesla K80. For this I used Google CloudML. The creation of the bottleneck images took about 40h. In sum everything costed about 100€ (from the 300$ free credits for signing up).  But I needed about 50€ extra for some trial and error and mainly because the jobs were stopped by the server sometimes in the middle of the training for reasons unkown to me. I also could not submit them in time, as I has problems creating the predictions on CloudML. \n\nWith my results I am a little bit disappointed, I only got 0.46329 on private leaderboard. I am not sure whether this is my fault, or due to the small input size of 90*90 pixels I used for time reasons (The image size only affects the time needed in the first run, as the bottleneck features I used had fixes size of 4032 values).  Network was trained for about 5 Epochs. No data augmentation was used. Input for prediction layer are features from four images, if product did not have four images zeros were used instead.\n\nFull code, written for tensorflow, with commands to run in cloudml, you can find under:\n\n    https://github.com/DanBmh/DiscountProducts\n\nMaybe it can be some help to you for future Competitions:)\n\n"
  }
}