{"nbformat": 4, "cells": [{"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "# **Use keras to count Sea Lions**\n\nThis kernel is a lite version of my approach.\n\n[for more information...][1]\n\n\n  [1]: https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count/discussion/35408", "metadata": {"_uuid": "8637295fd48b02cc36592459e72ac525ebbeee76", "_cell_guid": "e29888aa-8eb8-e86c-f7ea-335b3721bcfa", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "import numpy as np\nimport cv2\nimport matplotlib.pyplot as plt\nimport skimage.feature\nimport keras\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Activation, Flatten, Conv2D, MaxPooling2D", "metadata": {"_execution_state": "idle", "_uuid": "367a29f6cf2e4f9bb3ed97148520b9de537e9242", "_cell_guid": "6ebfb9d2-30a4-2a86-ee32-8199c410d84c", "trusted": false, "_active": false, "collapsed": false}}, {"execution_count": null, "cell_type": "markdown", "outputs": [], "source": "**Scale and patch**", "metadata": {"_execution_state": "idle", "_uuid": "83ab09b6a18d1ec3f13bc5a166d222b82b98d257", "_cell_guid": "fdca7b86-a47f-46d5-9778-48449a682035", "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "r = 0.4     #scale down\nwidth = 100 #patch size ", "metadata": {"_execution_state": "idle", "_uuid": "da12d4183701191cb6f25144750152a17447bdd5", "_cell_guid": "f988a32b-f725-3a21-2ae1-64ad6b8b140d", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "**Get dot coordinates and cut image to patches :** (thanks to Radu Stoicescu)", "metadata": {"_uuid": "0cd131409ffa821e90d063c119edf27b49ef538d", "_cell_guid": "155679de-9f9b-5e2b-a0ca-914e65bdea29", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "def GetData(filename):\n    # read the Train and Train Dotted images\n    image_1 = cv2.imread(\"../input/TrainDotted/\" + filename)\n    image_2 = cv2.imread(\"../input/Train/\" + filename)\n    img1 = cv2.GaussianBlur(image_1,(5,5),0)\n\n    # absolute difference between Train and Train Dotted\n    image_3 = cv2.absdiff(image_1,image_2)\n    mask_1 = cv2.cvtColor(image_1, cv2.COLOR_BGR2GRAY)\n    mask_1[mask_1 < 50] = 0\n    mask_1[mask_1 > 0] = 255\n    image_4 = cv2.bitwise_or(image_3, image_3, mask=mask_1)\n\n    # convert to grayscale to be accepted by skimage.feature.blob_log\n    image_6 = np.max(image_4,axis=2)\n\n    # detect blobs\n    blobs = skimage.feature.blob_log(image_6, min_sigma=3, max_sigma=7, num_sigma=1, threshold=0.05)\n\n    h,w,d = image_2.shape\n\n    res=np.zeros((int((w*r)//width)+1,int((h*r)//width)+1,5), dtype='int16')\n\n    for blob in blobs:\n        # get the coordinates for each blob\n        y, x, s = blob\n        # get the color of the pixel from Train Dotted in the center of the blob\n        b,g,R = img1[int(y)][int(x)][:]\n        x1 = int((x*r)//width)\n        y1 = int((y*r)//width)\n        # decision tree to pick the class of the blob by looking at the color in Train Dotted\n        if R > 225 and b < 25 and g < 25: # RED\n            res[x1,y1,0]+=1\n        elif R > 225 and b > 225 and g < 25: # MAGENTA\n            res[x1,y1,1]+=1\n        elif R < 75 and b < 50 and 150 < g < 200: # GREEN\n            res[x1,y1,4]+=1\n        elif R < 75 and  150 < b < 200 and g < 75: # BLUE\n            res[x1,y1,3]+=1\n        elif 60 < R < 120 and b < 50 and g < 75:  # BROWN\n            res[x1,y1,2]+=1\n\n    ma = cv2.cvtColor((1*(np.sum(image_1, axis=2)>20)).astype('uint8'), cv2.COLOR_GRAY2BGR)\n    img = cv2.resize(image_2 * ma, (int(w*r),int(h*r)))\n    h1,w1,d = img.shape\n\n    trainX = []\n    trainY = []\n\n    for i in range(int(w1//width)):\n        for j in range(int(h1//width)):\n            trainY.append(res[i,j,:])\n            trainX.append(img[j*width:j*width+width,i*width:i*width+width,:])\n\n    return np.array(trainX), np.array(trainY)\n\ndef rmse(predictions, targets):\n    return np.sqrt(((predictions - targets) ** 2).mean())", "metadata": {"_execution_state": "idle", "_uuid": "f8c46f4c18f7026255158bbdfcfb6dd879d271c0", "_cell_guid": "6e343a6d-c109-2f3c-10ba-ab218a95c8de", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "**Use only 1 image, split to train/test.**\n\nIn my real approach:\n\n - r = 1 to 0.6561 (0.9^0, 0.9^1 ... 0.9^4)\n   \n - patch size = 300x300\n   \n - cut whole training set to patches, number of positive(all) vs\n   background(random) = 1 : 3\n   \n - 95% for training, 5% for validation\n\n - data augmentation by flip, rotate, change saturation, brightness, contrast", "metadata": {"_uuid": "4884873c2988fd3559027ae649085452287e33bd", "_cell_guid": "1233be31-b216-19e3-e61b-9fb6f5cb8e30", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "trainX, trainY = GetData(\"0.jpg\")\n\nnp.random.seed(1004)\nrandomize = np.arange(len(trainX))\nnp.random.shuffle(randomize)\ntrainX = trainX[randomize]\ntrainY = trainY[randomize]\n\nn_train = int(len(trainX) * 0.7)\ntestX = trainX[n_train:]\ntestY = trainY[n_train:]\ntrainX = trainX[:n_train]\ntrainY = trainY[:n_train]\n\nprint(trainY.shape, trainY[0])\nprint(testY.shape, testY[0])", "metadata": {"_execution_state": "idle", "_uuid": "c85cb08f407de25a913de27b7d2d1c11f728cf60", "_cell_guid": "fcf3edc8-c64f-cf4e-d0dd-a58329f6f7b9", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "**Patches looks like :**", "metadata": {"_uuid": "cc0e989a461c29dd6bc732b29b86ef08d55239fc", "_cell_guid": "8ad7f45a-e8b6-d63d-d781-8499574bf167", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "fig = plt.figure(figsize=(12,12))\nfor i in range(4):\n    ax = fig.add_subplot(1,4,i+1)\n    plt.imshow(cv2.cvtColor(trainX[i], cv2.COLOR_BGR2RGB))\nprint(trainY[:4])", "metadata": {"_execution_state": "idle", "_uuid": "cf67e28d99778598876e6b7263148d8d2c3bd9dd", "_cell_guid": "2838a615-57f2-6eae-1b75-c80ec3422941", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "**Keras CNN model, for example**", "metadata": {"_uuid": "cba4811d592bfc78d40dd41b9d5d3dbe56d54507", "_cell_guid": "7dbff9b6-db78-a721-9245-03ceaa7085c0", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "model = Sequential()\n\nmodel.add(Conv2D(32, (3, 3), activation='relu', padding='same', input_shape=(width,width,3)))\nmodel.add(Conv2D(32, (3, 3), activation='relu', padding='same'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\nmodel.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Conv2D(128, (3, 3), activation='relu', padding='same'))\nmodel.add(Conv2D(128, (3, 3), activation='relu', padding='same'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Flatten())\nmodel.add(Dense(256, activation='relu'))\nmodel.add(Dense(5, activation='linear'))\n\n#model.summary()", "metadata": {"_execution_state": "idle", "_uuid": "f010ffccc71c8d75dd89df083e382bd583436212", "_cell_guid": "15676918-1a5c-3afd-3857-037bd69d1546", "trusted": false, "_active": false, "collapsed": false}}, {"execution_count": null, "cell_type": "markdown", "outputs": [], "source": "full version model:\n\n    initial_model = applications.VGG16(weights=\"imagenet\", include_top=False, input_shape=(300,300,3))\n    last = initial_model.output\n    x = Flatten()(last)\n    x = Dense(1024)(x)\n    x = LeakyReLU(alpha=.1)(x)\n    preds = Dense(5, activation='linear')(x)\n    model = Model(initial_model.input, preds)", "metadata": {"_execution_state": "idle", "_uuid": "6a33077d5e96d7b3d94d22b329b9b965545f56d8", "_cell_guid": "2905d6b2-6db0-448e-892e-9d6489ee8408", "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "**Start training slowly :**", "metadata": {"_uuid": "111ff271d4d4f00d3435de4b3e018bf43bdd1e39", "_cell_guid": "be8659b0-304d-c197-b417-5183a81974f7", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "optim = keras.optimizers.SGD(lr=1e-5, momentum=0.2)\nmodel.compile(loss='mean_squared_error', optimizer=optim)\nmodel.fit(trainX, trainY, epochs=8, verbose=2)", "metadata": {"_execution_state": "idle", "_uuid": "a68c572bcefb5db3426285f6a28060a66057bfbe", "_cell_guid": "2048e2d0-f473-cb4f-7525-bbc3075717c2", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "**Then speed up :**", "metadata": {"_uuid": "b0b83d9adbfb63591eb78a55b15fff104de083d2", "_cell_guid": "9c85e582-becd-ed9f-a60d-3528e44911f5", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "optim = keras.optimizers.SGD(lr=1e-4, momentum=0.9)\nmodel.compile(loss='mean_squared_error', optimizer=optim)\nmodel.fit(trainX, trainY, epochs=30, verbose=2)", "metadata": {"_execution_state": "idle", "_uuid": "8013099dbab26a4a57178bd95dd76aefdd45a0d1", "_cell_guid": "470ee712-89e8-3149-6a6a-863a1cb2c1be", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "# The kernel was killed for running longer than 1200 seconds ...\nmodel.fit(trainX, trainY, epochs=20, verbose=2)", "metadata": {"_execution_state": "idle", "_uuid": "b0c40bdac3c531ec252d484027e217f9e675d512", "_cell_guid": "ca8a06b5-7b77-74d9-d4f9-21e482a594ad", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "**Test :**", "metadata": {"_uuid": "d0715bf60eb4dcb1f87ff7f4b336fb47c29ff76e", "_cell_guid": "e26e2efe-0b9d-8aef-6002-b9608acc8fd0", "_active": false, "collapsed": false}}, {"cell_type": "code", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "result = model.predict(trainX)\nprint('Training set --')\nprint('    ground truth: ', np.sum(trainY, axis=0))\nprint('  evaluate count: ', np.sum(result*(result>0.3), axis=0).astype('int'))\n\nresult = model.predict(testX)\nprint('Testing set --')\nprint('    ground truth: ', np.sum(testY, axis=0))\nprint('   predict count: ', np.sum(result*(result>0.3), axis=0).astype('int'))", "metadata": {"_execution_state": "idle", "_uuid": "1eb5b41bcf6d6cdc4647d88d4469d62cb3e42d31", "_cell_guid": "1cad34c2-c304-0b45-00f8-b4e892673f0c", "trusted": false, "_active": false, "collapsed": false}}, {"cell_type": "markdown", "execution_state": "idle", "execution_count": null, "outputs": [], "source": "## Experience ##\n\nThe challenge is scale problem. They distinguish sea lion by size. In different images, one juveniles is larger than adult_females in another.\n\nI can't handle it well, so I decided to fit LB score:\n\n - scale down testing image get better score\n - more juveniles (less adult_females) get better score\n\nThe final submission is made by:\n\n - testing image scale: 0.48\n - add 50% juveniles, and subtract adult_females with the same amount\n - add 20% pups\n\n**Post processing details:**\n\nThese lucky variables are according to patch level regression.\n\nThe relationship between adult_females and juveniles in patches is:\n\n![juveniles regression][1]\n\n - value in table = average of juveniles# / (adult_females# + juveniles#) @ juveniles number range in patches\n\n - r#.# means image scale\n\n - *#.# means juveniles increase ratio\n\n  [1]: http://i.imgur.com/IkucSf6.gif", "metadata": {"_uuid": "ccb46c486dcb7dbe64a8b83b7e91f12f196e53e4", "_cell_guid": "0ff05b53-abbf-cdb7-ea9b-635f1822da22", "_active": false, "collapsed": false}}], "nbformat_minor": 0, "metadata": {"kernelspec": {"name": "python3", "display_name": "Python 3", "language": "python"}, "language_info": {"name": "python", "file_extension": ".py", "mimetype": "text/x-python", "codemirror_mode": {"name": "ipython", "version": 3}, "version": "3.6.1", "pygments_lexer": "ipython3", "nbconvert_exporter": "python"}}}