{
  "id": 20466,
  "title": "VGG_16 Keras",
  "url": "/competitions/state-farm-distracted-driver-detection/discussion/20466",
  "author_name": "",
  "post_date": "2016-04-26T23:08:56.473Z",
  "votes": 1,
  "comment_count": 17,
  "views": 3804,
  "content": "<p>I was trying to learn how to use pre-trained <a href=\"https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3\">VGG_16</a> model in this competition. Here is the code after the changes:</p>\n\n<pre><code>def VGG_16(weights_path=None):\n\n# dimensions of the generated pictures for each filter.\nimg_width = 120\nimg_height = 120\n\n# path to the model weights file.\nweights_path = 'vgg16_weights.h5'\n\nmodel = Sequential()\nmodel.add(ZeroPadding2D((1, 1), batch_input_shape=(1, 3, img_width, img_height)))\n\nmodel.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_2'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_2'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_2'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_3'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_2'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_3'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_2'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_3'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nassert os.path.exists(weights_path), 'Model weights not found (see &quot;weights_path&quot; variable in script).'\nf = h5py.File(weights_path)\nfor k in range(f.attrs['nb_layers']):\n    if k &gt;= len(model.layers):\n        # we don't look at the last (fully-connected) layers in the savefile\n        break\n    g = f['layer_{}'.format(k)]\n    weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]\n    model.layers[k].set_weights(weights)\nf.close()\nprint('Model loaded.')\n\n\nmodel.add(Flatten())\nmodel.add(Dense(4096, activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(4096, activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10, activation='softmax'))\n\n\nsgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)\nmodel.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])\nreturn model\n</code></pre>\n\n<p>But It seemed that it didn't work for me from the output since the losses are very high. </p>\n\n<pre><code>Epoch 1/10\n18807/18807 [==============================] - 118s - loss: 14.5545 - acc: 0.0961 - val_loss: 14.6877 - val_acc: 0.0887\nEpoch 2/10 18807/18807 [==============================] - 121s - loss: 14.5446 - acc: 0.0976 - val_loss: 14.6877 - val_acc: 0.0887\nEpoch 3/10 18807/18807 [==============================] - 120s - loss: 14.5677 - acc: 0.0962 - val_loss: 14.6877 - val_acc: 0.0887\n</code></pre>\n\n<p>Did anyone else have the same issue? Any idea how to solve this? </p>",
  "messages": [
    {
      "id": "117026",
      "postDate": "04/26/2016 23:08:56",
      "content": "<p>I was trying to learn how to use pre-trained <a href=\"https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3\">VGG_16</a> model in this competition. Here is the code after the changes:</p>\n\n<pre><code>def VGG_16(weights_path=None):\n\n# dimensions of the generated pictures for each filter.\nimg_width = 120\nimg_height = 120\n\n# path to the model weights file.\nweights_path = 'vgg16_weights.h5'\n\nmodel = Sequential()\nmodel.add(ZeroPadding2D((1, 1), batch_input_shape=(1, 3, img_width, img_height)))\n\nmodel.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_2'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_2'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_2'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_3'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_2'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_3'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_1'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_2'))\nmodel.add(ZeroPadding2D((1, 1)))\nmodel.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_3'))\nmodel.add(MaxPooling2D((2, 2), strides=(2, 2)))\n\nassert os.path.exists(weights_path), 'Model weights not found (see &quot;weights_path&quot; variable in script).'\nf = h5py.File(weights_path)\nfor k in range(f.attrs['nb_layers']):\n    if k &gt;= len(model.layers):\n        # we don't look at the last (fully-connected) layers in the savefile\n        break\n    g = f['layer_{}'.format(k)]\n    weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]\n    model.layers[k].set_weights(weights)\nf.close()\nprint('Model loaded.')\n\n\nmodel.add(Flatten())\nmodel.add(Dense(4096, activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(4096, activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10, activation='softmax'))\n\n\nsgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)\nmodel.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])\nreturn model\n</code></pre>\n\n<p>But It seemed that it didn't work for me from the output since the losses are very high. </p>\n\n<pre><code>Epoch 1/10\n18807/18807 [==============================] - 118s - loss: 14.5545 - acc: 0.0961 - val_loss: 14.6877 - val_acc: 0.0887\nEpoch 2/10 18807/18807 [==============================] - 121s - loss: 14.5446 - acc: 0.0976 - val_loss: 14.6877 - val_acc: 0.0887\nEpoch 3/10 18807/18807 [==============================] - 120s - loss: 14.5677 - acc: 0.0962 - val_loss: 14.6877 - val_acc: 0.0887\n</code></pre>\n\n<p>Did anyone else have the same issue? Any idea how to solve this? </p>",
      "rawMarkdown": "I was trying to learn how to use pre-trained [VGG_16][1] model in this competition. Here is the code after the changes:\r\n\r\n    def VGG_16(weights_path=None):\r\n    \r\n    # dimensions of the generated pictures for each filter.\r\n    img_width = 120\r\n    img_height = 120\r\n    \r\n    # path to the model weights file.\r\n    weights_path = 'vgg16_weights.h5'\r\n    \r\n    model = Sequential()\r\n    model.add(ZeroPadding2D((1, 1), batch_input_shape=(1, 3, img_width, img_height)))\r\n    \r\n    model.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_2'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_2'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_2'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_3'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_2'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_3'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_2'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_3'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n\r\n    assert os.path.exists(weights_path), 'Model weights not found (see \"weights_path\" variable in script).'\r\n    f = h5py.File(weights_path)\r\n    for k in range(f.attrs['nb_layers']):\r\n        if k >= len(model.layers):\r\n            # we don't look at the last (fully-connected) layers in the savefile\r\n            break\r\n        g = f['layer_{}'.format(k)]\r\n        weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]\r\n        model.layers[k].set_weights(weights)\r\n    f.close()\r\n    print('Model loaded.')\r\n\r\n    \r\n    model.add(Flatten())\r\n    model.add(Dense(4096, activation='relu'))\r\n    model.add(Dropout(0.5))\r\n    model.add(Dense(4096, activation='relu'))\r\n    model.add(Dropout(0.5))\r\n    model.add(Dense(10, activation='softmax'))\r\n\r\n  \r\n    sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)\r\n    model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])\r\n    return model\r\n\r\nBut It seemed that it didn't work for me from the output since the losses are very high. \r\n\r\n    Epoch 1/10\r\n    18807/18807 [==============================] - 118s - loss: 14.5545 - acc: 0.0961 - val_loss: 14.6877 - val_acc: 0.0887\r\n    Epoch 2/10 18807/18807 [==============================] - 121s - loss: 14.5446 - acc: 0.0976 - val_loss: 14.6877 - val_acc: 0.0887\r\n    Epoch 3/10 18807/18807 [==============================] - 120s - loss: 14.5677 - acc: 0.0962 - val_loss: 14.6877 - val_acc: 0.0887\r\n\r\nDid anyone else have the same issue? Any idea how to solve this? \r\n\r\n  [1]: https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3",
      "votes": null
    },
    {
      "id": "117032",
      "postDate": "04/27/2016 00:02:33",
      "content": "<p>I have some guesses:</p>\n\n<ul>\n<li>The weights you're using were obtained when training on ImageNet 224x224x3 images, so using them for 120x120x3 images might not work. This can be seen as a bad initialization, thus preventing the network to learn at all.</li>\n<li>That is quite a big network, in terms of both depth and width, so it'll be pretty hard to train it as a whole, as you seem to be doing, using such a small dataset (~20000 samples).</li>\n<li>Finally, when fine-tuning, you might want to use smaller learning rates.</li>\n</ul>",
      "rawMarkdown": "I have some guesses:\r\n\r\n - The weights you're using were obtained when training on ImageNet 224x224x3 images, so using them for 120x120x3 images might not work. This can be seen as a bad initialization, thus preventing the network to learn at all.\r\n - That is quite a big network, in terms of both depth and width, so it'll be pretty hard to train it as a whole, as you seem to be doing, using such a small dataset (~20000 samples).\r\n - Finally, when fine-tuning, you might want to use smaller learning rates.",
      "votes": null
    },
    {
      "id": "117048",
      "postDate": "04/27/2016 03:23:50",
      "content": "<p>@pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models. </p>",
      "rawMarkdown": "pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models.",
      "votes": null
    },
    {
      "id": "117795",
      "postDate": "04/30/2016 23:47:48",
      "content": "<p>Ctrl+W if you don't mind me asking. How did you fix the high loss Ctrl+W? </p>",
      "rawMarkdown": "Ctrl+W if you don't mind me asking. How did you fix the high loss Ctrl+W?",
      "votes": null
    },
    {
      "id": "117800",
      "postDate": "05/01/2016 00:49:06",
      "content": "<p>@ucisee I haven't figured it out yet. :(</p>",
      "rawMarkdown": "ucisee I haven't figured it out yet. :(",
      "votes": null
    },
    {
      "id": "117864",
      "postDate": "05/01/2016 09:07:03",
      "content": "<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>",
      "rawMarkdown": "I've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.",
      "votes": null
    },
    {
      "id": "117890",
      "postDate": "05/01/2016 13:58:28",
      "content": "<p>I would recommend <em>really</em> decreasing the learning rate. That has helped me a lot with some of these large neural networks. I was planning on implementing VGG-16 in Keras as well, so when I do I'll let you know if that helped. </p>",
      "rawMarkdown": "I would recommend *really* decreasing the learning rate. That has helped me a lot with some of these large neural networks. I was planning on implementing VGG-16 in Keras as well, so when I do I'll let you know if that helped.",
      "votes": null
    },
    {
      "id": "117895",
      "postDate": "05/01/2016 14:24:31",
      "content": "<p>[quote=Ctrl+W;117048]</p>\n\n<p>@pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models. </p>\n\n<p>[/quote]</p>\n\n<p>Usually, in finetuning, lr is small, say 1e-4, and your can start tuning only the last layer and keep the other dense layers unchanged </p>",
      "rawMarkdown": "[quote=Ctrl+W;117048]\r\n\r\n@pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models. \r\n\r\n[/quote]\r\n\r\nUsually, in finetuning, lr is small, say 1e-4, and your can start tuning only the last layer and keep the other dense layers unchanged",
      "votes": null
    },
    {
      "id": "117953",
      "postDate": "05/02/2016 01:48:31",
      "content": "<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. </p>\n\n<p>[quote=ZFTurbo;117864]</p>\n\n<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>\n\n<p>[/quote]</p>",
      "rawMarkdown": "I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n\r\n[quote=ZFTurbo;117864]\r\n\r\nI've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.\r\n\r\n[/quote]",
      "votes": null
    },
    {
      "id": "117954",
      "postDate": "05/02/2016 02:06:02",
      "content": "<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>",
      "rawMarkdown": "[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?",
      "votes": null
    },
    {
      "id": "117957",
      "postDate": "05/02/2016 02:42:52",
      "content": "<p>Yep ~ You will see when I post my code and thoughts :)</p>\n\n<p>[quote=Bojan Tunguz;117954]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>\n\n<p>[/quote]</p>",
      "rawMarkdown": "Yep ~ You will see when I post my code and thoughts :)\r\n\r\n[quote=Bojan Tunguz;117954]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?\r\n\r\n[/quote]",
      "votes": null
    },
    {
      "id": "117958",
      "postDate": "05/02/2016 02:45:13",
      "content": "<p>[quote=Jiao Dong;117957]</p>\n\n<p>Yep ~ You will see when I post my code and thoughts :)</p>\n\n<p>[quote=Bojan Tunguz;117954]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>Wow, I think I will just drop everything I've been doing and wait for your code. It will save me a few $$$ in electricity bill. :)</p>",
      "rawMarkdown": "[quote=Jiao Dong;117957]\r\n\r\nYep ~ You will see when I post my code and thoughts :)\r\n\r\n[quote=Bojan Tunguz;117954]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nWow, I think I will just drop everything I've been doing and wait for your code. It will save me a few $$$ in electricity bill. :)",
      "votes": null
    },
    {
      "id": "117964",
      "postDate": "05/02/2016 04:06:36",
      "content": "<p>[quote=Jiao Dong;117957]</p>\n\n<p>Yep ~ You will see when I post my code and thoughts :)</p>\n\n<p>[quote=Bojan Tunguz;117954]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>@Jiao, looking forward to seeing your code.</p>",
      "rawMarkdown": "[quote=Jiao Dong;117957]\r\n\r\nYep ~ You will see when I post my code and thoughts :)\r\n\r\n[quote=Bojan Tunguz;117954]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\n@Jiao, looking forward to seeing your code.",
      "votes": null
    },
    {
      "id": "117966",
      "postDate": "05/02/2016 04:23:37",
      "content": "<p>[quote=Jiao Dong;117953]\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. \n[/quote]</p>\n\n<p>I really look forward to seeing this. Thanks for sharing!</p>",
      "rawMarkdown": "[quote=Jiao Dong;117953]\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n[/quote]\r\n\r\nI really look forward to seeing this. Thanks for sharing!",
      "votes": null
    },
    {
      "id": "118442",
      "postDate": "05/03/2016 18:14:48",
      "content": "<p>Appreciate the sharing Jiao Dong.  Looking forward.</p>",
      "rawMarkdown": "Appreciate the sharing Jiao Dong.  Looking forward.",
      "votes": null
    },
    {
      "id": "118477",
      "postDate": "05/03/2016 21:02:54",
      "content": "<p>The input images have to be at least 224 by 224 for VGG_16 and standardized by the mean values that they used to train the CNN.  Otherwise it won't work properly. That's probably why you are getting such a high loss. </p>",
      "rawMarkdown": "The input images have to be at least 224 by 224 for VGG_16 and standardized by the mean values that they used to train the CNN.  Otherwise it won't work properly. That's probably why you are getting such a high loss.",
      "votes": null
    },
    {
      "id": "118790",
      "postDate": "05/05/2016 10:53:26",
      "content": "<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. </p>\n\n<p>[quote=ZFTurbo;117864]</p>\n\n<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>I used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?</p>",
      "rawMarkdown": "[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n\r\n[quote=ZFTurbo;117864]\r\n\r\nI've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nI used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?",
      "votes": null
    },
    {
      "id": "118798",
      "postDate": "05/05/2016 11:32:39",
      "content": "<p>[quote=Abhijay Arora;118790]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. </p>\n\n<p>[quote=ZFTurbo;117864]</p>\n\n<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>I used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?</p>\n\n<p>[/quote]</p>\n\n<p>Loss 2.3 - it seems that NN is not trained at all. You can try larger image, smaller LR, try to change batch size.</p>",
      "rawMarkdown": "[quote=Abhijay Arora;118790]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n\r\n[quote=ZFTurbo;117864]\r\n\r\nI've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nI used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?\r\n\r\n[/quote]\r\n\r\nLoss 2.3 - it seems that NN is not trained at all. You can try larger image, smaller LR, try to change batch size.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 117032,
      "author_name": "pennacchio",
      "author_url": "",
      "post_date": "04/27/2016 00:02:33",
      "content": "<p>I have some guesses:</p>\n\n<ul>\n<li>The weights you're using were obtained when training on ImageNet 224x224x3 images, so using them for 120x120x3 images might not work. This can be seen as a bad initialization, thus preventing the network to learn at all.</li>\n<li>That is quite a big network, in terms of both depth and width, so it'll be pretty hard to train it as a whole, as you seem to be doing, using such a small dataset (~20000 samples).</li>\n<li>Finally, when fine-tuning, you might want to use smaller learning rates.</li>\n</ul>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117048,
      "author_name": "zhugds",
      "author_url": "",
      "post_date": "04/27/2016 03:23:50",
      "content": "<p>@pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117795,
      "author_name": "notaapple",
      "author_url": "",
      "post_date": "04/30/2016 23:47:48",
      "content": "<p>Ctrl+W if you don't mind me asking. How did you fix the high loss Ctrl+W? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117800,
      "author_name": "zhugds",
      "author_url": "",
      "post_date": "05/01/2016 00:49:06",
      "content": "<p>@ucisee I haven't figured it out yet. :(</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117864,
      "author_name": "zfturbo",
      "author_url": "",
      "post_date": "05/01/2016 09:07:03",
      "content": "<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117890,
      "author_name": "tunguz",
      "author_url": "",
      "post_date": "05/01/2016 13:58:28",
      "content": "<p>I would recommend <em>really</em> decreasing the learning rate. That has helped me a lot with some of these large neural networks. I was planning on implementing VGG-16 in Keras as well, so when I do I'll let you know if that helped. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117895,
      "author_name": "usixuz",
      "author_url": "",
      "post_date": "05/01/2016 14:24:31",
      "content": "<p>[quote=Ctrl+W;117048]</p>\n\n<p>@pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models. </p>\n\n<p>[/quote]</p>\n\n<p>Usually, in finetuning, lr is small, say 1e-4, and your can start tuning only the last layer and keep the other dense layers unchanged </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117953,
      "author_name": "jiaodong",
      "author_url": "",
      "post_date": "05/02/2016 01:48:31",
      "content": "<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. </p>\n\n<p>[quote=ZFTurbo;117864]</p>\n\n<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>\n\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117954,
      "author_name": "tunguz",
      "author_url": "",
      "post_date": "05/02/2016 02:06:02",
      "content": "<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117957,
      "author_name": "jiaodong",
      "author_url": "",
      "post_date": "05/02/2016 02:42:52",
      "content": "<p>Yep ~ You will see when I post my code and thoughts :)</p>\n\n<p>[quote=Bojan Tunguz;117954]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>\n\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117958,
      "author_name": "tunguz",
      "author_url": "",
      "post_date": "05/02/2016 02:45:13",
      "content": "<p>[quote=Jiao Dong;117957]</p>\n\n<p>Yep ~ You will see when I post my code and thoughts :)</p>\n\n<p>[quote=Bojan Tunguz;117954]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>Wow, I think I will just drop everything I've been doing and wait for your code. It will save me a few $$$ in electricity bill. :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117964,
      "author_name": "zhugds",
      "author_url": "",
      "post_date": "05/02/2016 04:06:36",
      "content": "<p>[quote=Jiao Dong;117957]</p>\n\n<p>Yep ~ You will see when I post my code and thoughts :)</p>\n\n<p>[quote=Bojan Tunguz;117954]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>[/quote]</p>\n\n<p>Is that with a single model on the original images?</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>@Jiao, looking forward to seeing your code.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117966,
      "author_name": "gauss256",
      "author_url": "",
      "post_date": "05/02/2016 04:23:37",
      "content": "<p>[quote=Jiao Dong;117953]\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. \n[/quote]</p>\n\n<p>I really look forward to seeing this. Thanks for sharing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 118442,
      "author_name": "drpatrickchan",
      "author_url": "",
      "post_date": "05/03/2016 18:14:48",
      "content": "<p>Appreciate the sharing Jiao Dong.  Looking forward.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 118477,
      "author_name": "notaapple",
      "author_url": "",
      "post_date": "05/03/2016 21:02:54",
      "content": "<p>The input images have to be at least 224 by 224 for VGG_16 and standardized by the mean values that they used to train the CNN.  Otherwise it won't work properly. That's probably why you are getting such a high loss. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 118790,
      "author_name": "abhijayvuyyuru",
      "author_url": "",
      "post_date": "05/05/2016 10:53:26",
      "content": "<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. </p>\n\n<p>[quote=ZFTurbo;117864]</p>\n\n<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>I used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 118798,
      "author_name": "obaiev",
      "author_url": "",
      "post_date": "05/05/2016 11:32:39",
      "content": "<p>[quote=Abhijay Arora;118790]</p>\n\n<p>[quote=Jiao Dong;117953]</p>\n\n<p>I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). </p>\n\n<p>I would like to share my script later, sometime this week. </p>\n\n<p>[quote=ZFTurbo;117864]</p>\n\n<p>I've tried this model too, with no success. And I used 224x224x3 with image normalization: </p>\n\n<pre><code>im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\nim[:,:,0] -= 103.939\nim[:,:,1] -= 116.779\nim[:,:,2] -= 123.68\nim = im.transpose((2,0,1))\n</code></pre>\n\n<p>Would be good if someone find the way in Keras to successfully retrain existed models.</p>\n\n<p>[/quote]</p>\n\n<p>[/quote]</p>\n\n<p>I used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?</p>\n\n<p>[/quote]</p>\n\n<p>Loss 2.3 - it seems that NN is not trained at all. You can try larger image, smaller LR, try to change batch size.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "117026": "I was trying to learn how to use pre-trained [VGG_16][1] model in this competition. Here is the code after the changes:\r\n\r\n    def VGG_16(weights_path=None):\r\n    \r\n    # dimensions of the generated pictures for each filter.\r\n    img_width = 120\r\n    img_height = 120\r\n    \r\n    # path to the model weights file.\r\n    weights_path = 'vgg16_weights.h5'\r\n    \r\n    model = Sequential()\r\n    model.add(ZeroPadding2D((1, 1), batch_input_shape=(1, 3, img_width, img_height)))\r\n    \r\n    model.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_2'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_2'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_2'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_3'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_2'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_3'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n    \r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_1'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_2'))\r\n    model.add(ZeroPadding2D((1, 1)))\r\n    model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_3'))\r\n    model.add(MaxPooling2D((2, 2), strides=(2, 2)))\r\n\r\n    assert os.path.exists(weights_path), 'Model weights not found (see \"weights_path\" variable in script).'\r\n    f = h5py.File(weights_path)\r\n    for k in range(f.attrs['nb_layers']):\r\n        if k >= len(model.layers):\r\n            # we don't look at the last (fully-connected) layers in the savefile\r\n            break\r\n        g = f['layer_{}'.format(k)]\r\n        weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]\r\n        model.layers[k].set_weights(weights)\r\n    f.close()\r\n    print('Model loaded.')\r\n\r\n    \r\n    model.add(Flatten())\r\n    model.add(Dense(4096, activation='relu'))\r\n    model.add(Dropout(0.5))\r\n    model.add(Dense(4096, activation='relu'))\r\n    model.add(Dropout(0.5))\r\n    model.add(Dense(10, activation='softmax'))\r\n\r\n  \r\n    sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)\r\n    model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])\r\n    return model\r\n\r\nBut It seemed that it didn't work for me from the output since the losses are very high. \r\n\r\n    Epoch 1/10\r\n    18807/18807 [==============================] - 118s - loss: 14.5545 - acc: 0.0961 - val_loss: 14.6877 - val_acc: 0.0887\r\n    Epoch 2/10 18807/18807 [==============================] - 121s - loss: 14.5446 - acc: 0.0976 - val_loss: 14.6877 - val_acc: 0.0887\r\n    Epoch 3/10 18807/18807 [==============================] - 120s - loss: 14.5677 - acc: 0.0962 - val_loss: 14.6877 - val_acc: 0.0887\r\n\r\nDid anyone else have the same issue? Any idea how to solve this? \r\n\r\n  [1]: https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3",
    "117032": "I have some guesses:\r\n\r\n - The weights you're using were obtained when training on ImageNet 224x224x3 images, so using them for 120x120x3 images might not work. This can be seen as a bad initialization, thus preventing the network to learn at all.\r\n - That is quite a big network, in terms of both depth and width, so it'll be pretty hard to train it as a whole, as you seem to be doing, using such a small dataset (~20000 samples).\r\n - Finally, when fine-tuning, you might want to use smaller learning rates.",
    "117048": "pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models.",
    "117795": "Ctrl+W if you don't mind me asking. How did you fix the high loss Ctrl+W?",
    "117800": "ucisee I haven't figured it out yet. :(",
    "117864": "I've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.",
    "117890": "I would recommend *really* decreasing the learning rate. That has helped me a lot with some of these large neural networks. I was planning on implementing VGG-16 in Keras as well, so when I do I'll let you know if that helped.",
    "117895": "[quote=Ctrl+W;117048]\r\n\r\n@pennacchio Thank you for your help. I am suspecting if I am doing the right thing with pre-trained models. \r\n\r\n[/quote]\r\n\r\nUsually, in finetuning, lr is small, say 1e-4, and your can start tuning only the last layer and keep the other dense layers unchanged",
    "117953": "I have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n\r\n[quote=ZFTurbo;117864]\r\n\r\nI've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.\r\n\r\n[/quote]",
    "117954": "[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?",
    "117957": "Yep ~ You will see when I post my code and thoughts :)\r\n\r\n[quote=Bojan Tunguz;117954]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?\r\n\r\n[/quote]",
    "117958": "[quote=Jiao Dong;117957]\r\n\r\nYep ~ You will see when I post my code and thoughts :)\r\n\r\n[quote=Bojan Tunguz;117954]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nWow, I think I will just drop everything I've been doing and wait for your code. It will save me a few $$$ in electricity bill. :)",
    "117964": "[quote=Jiao Dong;117957]\r\n\r\nYep ~ You will see when I post my code and thoughts :)\r\n\r\n[quote=Bojan Tunguz;117954]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\n[/quote]\r\n\r\nIs that with a single model on the original images?\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\n@Jiao, looking forward to seeing your code.",
    "117966": "[quote=Jiao Dong;117953]\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n[/quote]\r\n\r\nI really look forward to seeing this. Thanks for sharing!",
    "118442": "Appreciate the sharing Jiao Dong.  Looking forward.",
    "118477": "The input images have to be at least 224 by 224 for VGG_16 and standardized by the mean values that they used to train the CNN.  Otherwise it won't work properly. That's probably why you are getting such a high loss.",
    "118790": "[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n\r\n[quote=ZFTurbo;117864]\r\n\r\nI've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nI used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?",
    "118798": "[quote=Abhijay Arora;118790]\r\n\r\n[quote=Jiao Dong;117953]\r\n\r\nI have tried vgg_16 pre-trained model based on your sample code, the best LB score I got is 0.23800 ( Currently Ranked #8 ). \r\n\r\nI would like to share my script later, sometime this week. \r\n\r\n[quote=ZFTurbo;117864]\r\n\r\nI've tried this model too, with no success. And I used 224x224x3 with image normalization: \r\n\r\n    im = cv2.resize(cv2.imread('cat.jpg'), (224, 224)).astype(np.float32)\r\n    im[:,:,0] -= 103.939\r\n    im[:,:,1] -= 116.779\r\n    im[:,:,2] -= 123.68\r\n    im = im.transpose((2,0,1))\r\n\r\nWould be good if someone find the way in Keras to successfully retrain existed models.\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nI used VGG_16 pre-trained model, with image size of 48 X 64 X 3(Larger image sizes slow down the computer). I got a LB score of 2.3. Any tips to improve the score?\r\n\r\n[/quote]\r\n\r\nLoss 2.3 - it seems that NN is not trained at all. You can try larger image, smaller LR, try to change batch size."
  },
  "source": "meta"
}