{
  "id": 49370,
  "title": "Any hints on how the high validation accuracy is achieved?",
  "url": "/competitions/sp-society-camera-model-identification/discussion/49370",
  "author_name": "A0198918E_MaZhaoyang",
  "post_date": "2018-02-09T23:38:22.969000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I am just keeping wondering how most winners achieved a so high validation accuracy? Any hints what I have missed? </p>\n\n<p>The batch size is 4 and input image size is 512x512.</p>\n\n<p>Here is part of my training record:</p>\n\n<p><strong>For training Xception model even with 0 dropout:</strong> </p>\n\n<p>Epoch 62/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2975 - acc: 0.9416 - val_loss: 0.9382 - val_acc: 0.8661</p>\n\n<p>Epoch 63/200\n1975/1975 [==============================] - 1229s 622ms/step - loss: 0.2690 - acc: 0.9467 - val_loss: 0.7668 - val_acc: 0.8755</p>\n\n<p>Epoch 64/200\n1975/1975 [==============================] - 1228s 622ms/step - loss: 0.2804 - acc: 0.9430 - val_loss: 0.9324 - val_acc: 0.8620</p>\n\n<p>Epoch 65/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2798 - acc: 0.9449 - val_loss: 0.7955 - val_acc: 0.8833</p>\n\n<p>Epoch 66/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2454 - acc: 0.9492 - val_loss: 0.8227 - val_acc: 0.8703</p>\n\n<p>Epoch 67/200\n1975/1975 [==============================] - 1221s 618ms/step - loss: 0.2855 - acc: 0.9394 - val_loss: 0.7439 - val_acc: 0.8698</p>\n\n<p>Epoch 68/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2234 - acc: 0.9522 - val_loss: 0.8842 - val_acc: 0.8719</p>\n\n<p>Epoch 69/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2362 - acc: 0.9490 - val_loss: 0.8605 - val_acc: 0.8656</p>\n\n<p>Epoch 70/200\n1975/1975 [==============================] - 1223s 619ms/step - loss: 0.2501 - acc: 0.9501 - val_loss: 0.7569 - val_acc: 0.8745</p>\n\n<p><strong>With 0.3 dropout, for InceptionResNet:</strong></p>\n\n<p>Epoch 129/200\n1895/1895 [==============================] - 1422s 751ms/step - loss: 0.6473 - acc: 0.8711 - val_loss: 0.5347 - val_acc: 0.8856</p>\n\n<p>Epoch 130/200\n1895/1895 [==============================] - 1421s 750ms/step - loss: 0.6573 - acc: 0.8701 - val_loss: 0.7332 - val_acc: 0.8728</p>\n\n<p>Epoch 131/200\n1895/1895 [==============================] - 1451s 766ms/step - loss: 0.6721 - acc: 0.8652 - val_loss: 0.7261 - val_acc: 0.8603</p>\n\n<p>Epoch 132/200\n1895/1895 [==============================] - 1460s 770ms/step - loss: 0.6722 - acc: 0.8636 - val_loss: 0.6263 - val_acc: 0.8634</p>\n\n<p>Epoch 133/200\n1895/1895 [==============================] - 1464s 773ms/step - loss: 0.7025 - acc: 0.8578 - val_loss: 0.6221 - val_acc: 0.8775</p>\n\n<p>Epoch 134/200\n1895/1895 [==============================] - 1461s 771ms/step - loss: 0.6852 - acc: 0.8617 - val_loss: 0.6017 - val_acc: 0.8762</p>\n\n<p>Epoch 135/200\n1895/1895 [==============================] - 1444s 762ms/step - loss: 0.6174 - acc: 0.8770 - val_loss: 0.5433 - val_acc: 0.8950</p>\n\n<p>Epoch 136/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5713 - acc: 0.8868 - val_loss: 0.6895 - val_acc: 0.8731</p>\n\n<p>Epoch 137/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5565 - acc: 0.8902 - val_loss: 0.6082 - val_acc: 0.8812</p>\n\n<p>Epoch 138/200\n1895/1895 [==============================] - 1441s 760ms/step - loss: 0.5229 - acc: 0.9001 - val_loss: 0.6242 - val_acc: 0.8859</p>\n\n<p>Epoch 139/200\n1895/1895 [==============================] - 1445s 763ms/step - loss: 0.6404 - acc: 0.8701 - val_loss: 0.5816 - val_acc: 0.8928</p>\n\n<p>Epoch 140/200\n1895/1895 [==============================] - 1442s 761ms/step - loss: 0.6649 - acc: 0.8679 - val_loss: 0.6693 - val_acc: 0.8784</p>\n\n<p>Epoch 141/200\n1895/1895 [==============================] - 1439s 759ms/step - loss: 0.6482 - acc: 0.8694 - val_loss: 0.6046 - val_acc: 0.8941</p>\n\n<p>Epoch 142/200\n1895/1895 [==============================] - 1423s 751ms/step - loss: 0.6133 - acc: 0.8772 - val_loss: 0.6777 - val_acc: 0.8728</p>\n\n<p>Epoch 143/200\n1895/1895 [==============================] - 1425s 752ms/step - loss: 0.6591 - acc: 0.8701 - val_loss: 0.6410 - val_acc: 0.8834</p>\n\n<p>Epoch 144/200\n1895/1895 [==============================] - 1432s 756ms/step - loss: 0.6336 - acc: 0.8686 - val_loss: 0.7510 - val_acc: 0.8741</p>",
  "messages": [
    {
      "id": 280419,
      "postDate": "2018-02-09T23:38:22.970Z",
      "content": "<p>I am just keeping wondering how most winners achieved a so high validation accuracy? Any hints what I have missed? </p>\n\n<p>The batch size is 4 and input image size is 512x512.</p>\n\n<p>Here is part of my training record:</p>\n\n<p><strong>For training Xception model even with 0 dropout:</strong> </p>\n\n<p>Epoch 62/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2975 - acc: 0.9416 - val_loss: 0.9382 - val_acc: 0.8661</p>\n\n<p>Epoch 63/200\n1975/1975 [==============================] - 1229s 622ms/step - loss: 0.2690 - acc: 0.9467 - val_loss: 0.7668 - val_acc: 0.8755</p>\n\n<p>Epoch 64/200\n1975/1975 [==============================] - 1228s 622ms/step - loss: 0.2804 - acc: 0.9430 - val_loss: 0.9324 - val_acc: 0.8620</p>\n\n<p>Epoch 65/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2798 - acc: 0.9449 - val_loss: 0.7955 - val_acc: 0.8833</p>\n\n<p>Epoch 66/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2454 - acc: 0.9492 - val_loss: 0.8227 - val_acc: 0.8703</p>\n\n<p>Epoch 67/200\n1975/1975 [==============================] - 1221s 618ms/step - loss: 0.2855 - acc: 0.9394 - val_loss: 0.7439 - val_acc: 0.8698</p>\n\n<p>Epoch 68/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2234 - acc: 0.9522 - val_loss: 0.8842 - val_acc: 0.8719</p>\n\n<p>Epoch 69/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2362 - acc: 0.9490 - val_loss: 0.8605 - val_acc: 0.8656</p>\n\n<p>Epoch 70/200\n1975/1975 [==============================] - 1223s 619ms/step - loss: 0.2501 - acc: 0.9501 - val_loss: 0.7569 - val_acc: 0.8745</p>\n\n<p><strong>With 0.3 dropout, for InceptionResNet:</strong></p>\n\n<p>Epoch 129/200\n1895/1895 [==============================] - 1422s 751ms/step - loss: 0.6473 - acc: 0.8711 - val_loss: 0.5347 - val_acc: 0.8856</p>\n\n<p>Epoch 130/200\n1895/1895 [==============================] - 1421s 750ms/step - loss: 0.6573 - acc: 0.8701 - val_loss: 0.7332 - val_acc: 0.8728</p>\n\n<p>Epoch 131/200\n1895/1895 [==============================] - 1451s 766ms/step - loss: 0.6721 - acc: 0.8652 - val_loss: 0.7261 - val_acc: 0.8603</p>\n\n<p>Epoch 132/200\n1895/1895 [==============================] - 1460s 770ms/step - loss: 0.6722 - acc: 0.8636 - val_loss: 0.6263 - val_acc: 0.8634</p>\n\n<p>Epoch 133/200\n1895/1895 [==============================] - 1464s 773ms/step - loss: 0.7025 - acc: 0.8578 - val_loss: 0.6221 - val_acc: 0.8775</p>\n\n<p>Epoch 134/200\n1895/1895 [==============================] - 1461s 771ms/step - loss: 0.6852 - acc: 0.8617 - val_loss: 0.6017 - val_acc: 0.8762</p>\n\n<p>Epoch 135/200\n1895/1895 [==============================] - 1444s 762ms/step - loss: 0.6174 - acc: 0.8770 - val_loss: 0.5433 - val_acc: 0.8950</p>\n\n<p>Epoch 136/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5713 - acc: 0.8868 - val_loss: 0.6895 - val_acc: 0.8731</p>\n\n<p>Epoch 137/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5565 - acc: 0.8902 - val_loss: 0.6082 - val_acc: 0.8812</p>\n\n<p>Epoch 138/200\n1895/1895 [==============================] - 1441s 760ms/step - loss: 0.5229 - acc: 0.9001 - val_loss: 0.6242 - val_acc: 0.8859</p>\n\n<p>Epoch 139/200\n1895/1895 [==============================] - 1445s 763ms/step - loss: 0.6404 - acc: 0.8701 - val_loss: 0.5816 - val_acc: 0.8928</p>\n\n<p>Epoch 140/200\n1895/1895 [==============================] - 1442s 761ms/step - loss: 0.6649 - acc: 0.8679 - val_loss: 0.6693 - val_acc: 0.8784</p>\n\n<p>Epoch 141/200\n1895/1895 [==============================] - 1439s 759ms/step - loss: 0.6482 - acc: 0.8694 - val_loss: 0.6046 - val_acc: 0.8941</p>\n\n<p>Epoch 142/200\n1895/1895 [==============================] - 1423s 751ms/step - loss: 0.6133 - acc: 0.8772 - val_loss: 0.6777 - val_acc: 0.8728</p>\n\n<p>Epoch 143/200\n1895/1895 [==============================] - 1425s 752ms/step - loss: 0.6591 - acc: 0.8701 - val_loss: 0.6410 - val_acc: 0.8834</p>\n\n<p>Epoch 144/200\n1895/1895 [==============================] - 1432s 756ms/step - loss: 0.6336 - acc: 0.8686 - val_loss: 0.7510 - val_acc: 0.8741</p>",
      "rawMarkdown": "I am just keeping wondering how most winners achieved a so high validation accuracy? Any hints what I have missed? \n\nThe batch size is 4 and input image size is 512x512.\n\nHere is part of my training record:\n\n**For training Xception model even with 0 dropout:** \n\nEpoch 62/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2975 - acc: 0.9416 - val_loss: 0.9382 - val_acc: 0.8661\n\nEpoch 63/200\n1975/1975 [==============================] - 1229s 622ms/step - loss: 0.2690 - acc: 0.9467 - val_loss: 0.7668 - val_acc: 0.8755\n\nEpoch 64/200\n1975/1975 [==============================] - 1228s 622ms/step - loss: 0.2804 - acc: 0.9430 - val_loss: 0.9324 - val_acc: 0.8620\n\nEpoch 65/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2798 - acc: 0.9449 - val_loss: 0.7955 - val_acc: 0.8833\n\nEpoch 66/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2454 - acc: 0.9492 - val_loss: 0.8227 - val_acc: 0.8703\n\nEpoch 67/200\n1975/1975 [==============================] - 1221s 618ms/step - loss: 0.2855 - acc: 0.9394 - val_loss: 0.7439 - val_acc: 0.8698\n\nEpoch 68/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2234 - acc: 0.9522 - val_loss: 0.8842 - val_acc: 0.8719\n\nEpoch 69/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2362 - acc: 0.9490 - val_loss: 0.8605 - val_acc: 0.8656\n\nEpoch 70/200\n1975/1975 [==============================] - 1223s 619ms/step - loss: 0.2501 - acc: 0.9501 - val_loss: 0.7569 - val_acc: 0.8745\n\n\n\n\n**With 0.3 dropout, for InceptionResNet:**\n\nEpoch 129/200\n1895/1895 [==============================] - 1422s 751ms/step - loss: 0.6473 - acc: 0.8711 - val_loss: 0.5347 - val_acc: 0.8856\n\nEpoch 130/200\n1895/1895 [==============================] - 1421s 750ms/step - loss: 0.6573 - acc: 0.8701 - val_loss: 0.7332 - val_acc: 0.8728\n\nEpoch 131/200\n1895/1895 [==============================] - 1451s 766ms/step - loss: 0.6721 - acc: 0.8652 - val_loss: 0.7261 - val_acc: 0.8603\n\nEpoch 132/200\n1895/1895 [==============================] - 1460s 770ms/step - loss: 0.6722 - acc: 0.8636 - val_loss: 0.6263 - val_acc: 0.8634\n\nEpoch 133/200\n1895/1895 [==============================] - 1464s 773ms/step - loss: 0.7025 - acc: 0.8578 - val_loss: 0.6221 - val_acc: 0.8775\n\nEpoch 134/200\n1895/1895 [==============================] - 1461s 771ms/step - loss: 0.6852 - acc: 0.8617 - val_loss: 0.6017 - val_acc: 0.8762\n\nEpoch 135/200\n1895/1895 [==============================] - 1444s 762ms/step - loss: 0.6174 - acc: 0.8770 - val_loss: 0.5433 - val_acc: 0.8950\n\nEpoch 136/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5713 - acc: 0.8868 - val_loss: 0.6895 - val_acc: 0.8731\n\nEpoch 137/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5565 - acc: 0.8902 - val_loss: 0.6082 - val_acc: 0.8812\n\nEpoch 138/200\n1895/1895 [==============================] - 1441s 760ms/step - loss: 0.5229 - acc: 0.9001 - val_loss: 0.6242 - val_acc: 0.8859\n\nEpoch 139/200\n1895/1895 [==============================] - 1445s 763ms/step - loss: 0.6404 - acc: 0.8701 - val_loss: 0.5816 - val_acc: 0.8928\n\nEpoch 140/200\n1895/1895 [==============================] - 1442s 761ms/step - loss: 0.6649 - acc: 0.8679 - val_loss: 0.6693 - val_acc: 0.8784\n\nEpoch 141/200\n1895/1895 [==============================] - 1439s 759ms/step - loss: 0.6482 - acc: 0.8694 - val_loss: 0.6046 - val_acc: 0.8941\n\nEpoch 142/200\n1895/1895 [==============================] - 1423s 751ms/step - loss: 0.6133 - acc: 0.8772 - val_loss: 0.6777 - val_acc: 0.8728\n\nEpoch 143/200\n1895/1895 [==============================] - 1425s 752ms/step - loss: 0.6591 - acc: 0.8701 - val_loss: 0.6410 - val_acc: 0.8834\n\nEpoch 144/200\n1895/1895 [==============================] - 1432s 756ms/step - loss: 0.6336 - acc: 0.8686 - val_loss: 0.7510 - val_acc: 0.8741"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "280419": "I am just keeping wondering how most winners achieved a so high validation accuracy? Any hints what I have missed? \n\nThe batch size is 4 and input image size is 512x512.\n\nHere is part of my training record:\n\n**For training Xception model even with 0 dropout:** \n\nEpoch 62/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2975 - acc: 0.9416 - val_loss: 0.9382 - val_acc: 0.8661\n\nEpoch 63/200\n1975/1975 [==============================] - 1229s 622ms/step - loss: 0.2690 - acc: 0.9467 - val_loss: 0.7668 - val_acc: 0.8755\n\nEpoch 64/200\n1975/1975 [==============================] - 1228s 622ms/step - loss: 0.2804 - acc: 0.9430 - val_loss: 0.9324 - val_acc: 0.8620\n\nEpoch 65/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2798 - acc: 0.9449 - val_loss: 0.7955 - val_acc: 0.8833\n\nEpoch 66/200\n1975/1975 [==============================] - 1227s 621ms/step - loss: 0.2454 - acc: 0.9492 - val_loss: 0.8227 - val_acc: 0.8703\n\nEpoch 67/200\n1975/1975 [==============================] - 1221s 618ms/step - loss: 0.2855 - acc: 0.9394 - val_loss: 0.7439 - val_acc: 0.8698\n\nEpoch 68/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2234 - acc: 0.9522 - val_loss: 0.8842 - val_acc: 0.8719\n\nEpoch 69/200\n1975/1975 [==============================] - 1222s 619ms/step - loss: 0.2362 - acc: 0.9490 - val_loss: 0.8605 - val_acc: 0.8656\n\nEpoch 70/200\n1975/1975 [==============================] - 1223s 619ms/step - loss: 0.2501 - acc: 0.9501 - val_loss: 0.7569 - val_acc: 0.8745\n\n\n\n\n**With 0.3 dropout, for InceptionResNet:**\n\nEpoch 129/200\n1895/1895 [==============================] - 1422s 751ms/step - loss: 0.6473 - acc: 0.8711 - val_loss: 0.5347 - val_acc: 0.8856\n\nEpoch 130/200\n1895/1895 [==============================] - 1421s 750ms/step - loss: 0.6573 - acc: 0.8701 - val_loss: 0.7332 - val_acc: 0.8728\n\nEpoch 131/200\n1895/1895 [==============================] - 1451s 766ms/step - loss: 0.6721 - acc: 0.8652 - val_loss: 0.7261 - val_acc: 0.8603\n\nEpoch 132/200\n1895/1895 [==============================] - 1460s 770ms/step - loss: 0.6722 - acc: 0.8636 - val_loss: 0.6263 - val_acc: 0.8634\n\nEpoch 133/200\n1895/1895 [==============================] - 1464s 773ms/step - loss: 0.7025 - acc: 0.8578 - val_loss: 0.6221 - val_acc: 0.8775\n\nEpoch 134/200\n1895/1895 [==============================] - 1461s 771ms/step - loss: 0.6852 - acc: 0.8617 - val_loss: 0.6017 - val_acc: 0.8762\n\nEpoch 135/200\n1895/1895 [==============================] - 1444s 762ms/step - loss: 0.6174 - acc: 0.8770 - val_loss: 0.5433 - val_acc: 0.8950\n\nEpoch 136/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5713 - acc: 0.8868 - val_loss: 0.6895 - val_acc: 0.8731\n\nEpoch 137/200\n1895/1895 [==============================] - 1440s 760ms/step - loss: 0.5565 - acc: 0.8902 - val_loss: 0.6082 - val_acc: 0.8812\n\nEpoch 138/200\n1895/1895 [==============================] - 1441s 760ms/step - loss: 0.5229 - acc: 0.9001 - val_loss: 0.6242 - val_acc: 0.8859\n\nEpoch 139/200\n1895/1895 [==============================] - 1445s 763ms/step - loss: 0.6404 - acc: 0.8701 - val_loss: 0.5816 - val_acc: 0.8928\n\nEpoch 140/200\n1895/1895 [==============================] - 1442s 761ms/step - loss: 0.6649 - acc: 0.8679 - val_loss: 0.6693 - val_acc: 0.8784\n\nEpoch 141/200\n1895/1895 [==============================] - 1439s 759ms/step - loss: 0.6482 - acc: 0.8694 - val_loss: 0.6046 - val_acc: 0.8941\n\nEpoch 142/200\n1895/1895 [==============================] - 1423s 751ms/step - loss: 0.6133 - acc: 0.8772 - val_loss: 0.6777 - val_acc: 0.8728\n\nEpoch 143/200\n1895/1895 [==============================] - 1425s 752ms/step - loss: 0.6591 - acc: 0.8701 - val_loss: 0.6410 - val_acc: 0.8834\n\nEpoch 144/200\n1895/1895 [==============================] - 1432s 756ms/step - loss: 0.6336 - acc: 0.8686 - val_loss: 0.7510 - val_acc: 0.8741"
  }
}