{"cells":[{"metadata":{"_uuid":"8637295fd48b02cc36592459e72ac525ebbeee76","_cell_guid":"e29888aa-8eb8-e86c-f7ea-335b3721bcfa","_active":false},"cell_type":"markdown","source":"# **Use keras to count Sea Lions**\n\nThis kernel is a lite version of my approach.\n\n[for more information...][1]\n\n\n  [1]: https://www.kaggle.com/c/noaa-fisheries-steller-sea-lion-population-count/discussion/35408"},{"metadata":{"_execution_state":"idle","_uuid":"367a29f6cf2e4f9bb3ed97148520b9de537e9242","_cell_guid":"6ebfb9d2-30a4-2a86-ee32-8199c410d84c","trusted":false,"_active":false},"cell_type":"code","source":"import numpy as np\nimport cv2\nimport matplotlib.pyplot as plt\nimport skimage.feature\nimport keras\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Activation, Flatten, Conv2D, MaxPooling2D","execution_count":null,"outputs":[]},{"metadata":{"_execution_state":"idle","_uuid":"83ab09b6a18d1ec3f13bc5a166d222b82b98d257","_cell_guid":"fdca7b86-a47f-46d5-9778-48449a682035"},"cell_type":"markdown","source":"**Scale and patch**"},{"metadata":{"_execution_state":"idle","_uuid":"da12d4183701191cb6f25144750152a17447bdd5","_cell_guid":"f988a32b-f725-3a21-2ae1-64ad6b8b140d","trusted":false,"_active":false},"cell_type":"code","source":"r = 0.4     #scale down\nwidth = 100 #patch size ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0cd131409ffa821e90d063c119edf27b49ef538d","_cell_guid":"155679de-9f9b-5e2b-a0ca-914e65bdea29","_active":false},"cell_type":"markdown","source":"**Get dot coordinates and cut image to patches :** (thanks to Radu Stoicescu)"},{"metadata":{"_execution_state":"idle","_uuid":"f8c46f4c18f7026255158bbdfcfb6dd879d271c0","_cell_guid":"6e343a6d-c109-2f3c-10ba-ab218a95c8de","trusted":false,"_active":false},"cell_type":"code","source":"def GetData(filename):\n    # read the Train and Train Dotted images\n    image_1 = cv2.imread(\"../input/TrainDotted/\" + filename)\n    image_2 = cv2.imread(\"../input/Train/\" + filename)\n    img1 = cv2.GaussianBlur(image_1,(5,5),0)\n\n    # absolute difference between Train and Train Dotted\n    image_3 = cv2.absdiff(image_1,image_2)\n    mask_1 = cv2.cvtColor(image_1, cv2.COLOR_BGR2GRAY)\n    mask_1[mask_1 < 50] = 0\n    mask_1[mask_1 > 0] = 255\n    image_4 = cv2.bitwise_or(image_3, image_3, mask=mask_1)\n\n    # convert to grayscale to be accepted by skimage.feature.blob_log\n    image_6 = np.max(image_4,axis=2)\n\n    # detect blobs\n    blobs = skimage.feature.blob_log(image_6, min_sigma=3, max_sigma=7, num_sigma=1, threshold=0.05)\n\n    h,w,d = image_2.shape\n\n    res=np.zeros((int((w*r)//width)+1,int((h*r)//width)+1,5), dtype='int16')\n\n    for blob in blobs:\n        # get the coordinates for each blob\n        y, x, s = blob\n        # get the color of the pixel from Train Dotted in the center of the blob\n        b,g,R = img1[int(y)][int(x)][:]\n        x1 = int((x*r)//width)\n        y1 = int((y*r)//width)\n        # decision tree to pick the class of the blob by looking at the color in Train Dotted\n        if R > 225 and b < 25 and g < 25: # RED\n            res[x1,y1,0]+=1\n        elif R > 225 and b > 225 and g < 25: # MAGENTA\n            res[x1,y1,1]+=1\n        elif R < 75 and b < 50 and 150 < g < 200: # GREEN\n            res[x1,y1,4]+=1\n        elif R < 75 and  150 < b < 200 and g < 75: # BLUE\n            res[x1,y1,3]+=1\n        elif 60 < R < 120 and b < 50 and g < 75:  # BROWN\n            res[x1,y1,2]+=1\n\n    ma = cv2.cvtColor((1*(np.sum(image_1, axis=2)>20)).astype('uint8'), cv2.COLOR_GRAY2BGR)\n    img = cv2.resize(image_2 * ma, (int(w*r),int(h*r)))\n    h1,w1,d = img.shape\n\n    trainX = []\n    trainY = []\n\n    for i in range(int(w1//width)):\n        for j in range(int(h1//width)):\n            trainY.append(res[i,j,:])\n            trainX.append(img[j*width:j*width+width,i*width:i*width+width,:])\n\n    return np.array(trainX), np.array(trainY)\n\ndef rmse(predictions, targets):\n    return np.sqrt(((predictions - targets) ** 2).mean())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4884873c2988fd3559027ae649085452287e33bd","_cell_guid":"1233be31-b216-19e3-e61b-9fb6f5cb8e30","_active":false},"cell_type":"markdown","source":"**Use only 1 image, split to train/test.**\n\nIn my real approach:\n\n - r = 1 to 0.6561 (0.9^0, 0.9^1 ... 0.9^4)\n   \n - patch size = 300x300\n   \n - cut whole training set to patches, number of positive(all) vs\n   background(random) = 1 : 3\n   \n - 95% for training, 5% for validation\n\n - data augmentation by flip, rotate, change saturation, brightness, contrast"},{"metadata":{"_execution_state":"idle","_uuid":"c85cb08f407de25a913de27b7d2d1c11f728cf60","_cell_guid":"fcf3edc8-c64f-cf4e-d0dd-a58329f6f7b9","trusted":false,"_active":false},"cell_type":"code","source":"trainX, trainY = GetData(\"0.jpg\")\n\nnp.random.seed(1004)\nrandomize = np.arange(len(trainX))\nnp.random.shuffle(randomize)\ntrainX = trainX[randomize]\ntrainY = trainY[randomize]\n\nn_train = int(len(trainX) * 0.7)\ntestX = trainX[n_train:]\ntestY = trainY[n_train:]\ntrainX = trainX[:n_train]\ntrainY = trainY[:n_train]\n\nprint(trainY.shape, trainY[0])\nprint(testY.shape, testY[0])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cc0e989a461c29dd6bc732b29b86ef08d55239fc","_cell_guid":"8ad7f45a-e8b6-d63d-d781-8499574bf167","_active":false},"cell_type":"markdown","source":"**Patches looks like :**"},{"metadata":{"_execution_state":"idle","_uuid":"cf67e28d99778598876e6b7263148d8d2c3bd9dd","_cell_guid":"2838a615-57f2-6eae-1b75-c80ec3422941","trusted":false,"_active":false},"cell_type":"code","source":"fig = plt.figure(figsize=(12,12))\nfor i in range(4):\n    ax = fig.add_subplot(1,4,i+1)\n    plt.imshow(cv2.cvtColor(trainX[i], cv2.COLOR_BGR2RGB))\nprint(trainY[:4])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cba4811d592bfc78d40dd41b9d5d3dbe56d54507","_cell_guid":"7dbff9b6-db78-a721-9245-03ceaa7085c0","_active":false},"cell_type":"markdown","source":"**Keras CNN model, for example**"},{"metadata":{"_execution_state":"idle","_uuid":"f010ffccc71c8d75dd89df083e382bd583436212","_cell_guid":"15676918-1a5c-3afd-3857-037bd69d1546","trusted":false,"_active":false},"cell_type":"code","source":"model = Sequential()\n\nmodel.add(Conv2D(32, (3, 3), activation='relu', padding='same', input_shape=(width,width,3)))\nmodel.add(Conv2D(32, (3, 3), activation='relu', padding='same'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\nmodel.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Conv2D(128, (3, 3), activation='relu', padding='same'))\nmodel.add(Conv2D(128, (3, 3), activation='relu', padding='same'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Flatten())\nmodel.add(Dense(256, activation='relu'))\nmodel.add(Dense(5, activation='linear'))\n\n#model.summary()","execution_count":null,"outputs":[]},{"metadata":{"_execution_state":"idle","_uuid":"6a33077d5e96d7b3d94d22b329b9b965545f56d8","_cell_guid":"2905d6b2-6db0-448e-892e-9d6489ee8408"},"cell_type":"markdown","source":"full version model:\n\n    initial_model = applications.VGG16(weights=\"imagenet\", include_top=False, input_shape=(300,300,3))\n    last = initial_model.output\n    x = Flatten()(last)\n    x = Dense(1024)(x)\n    x = LeakyReLU(alpha=.1)(x)\n    preds = Dense(5, activation='linear')(x)\n    model = Model(initial_model.input, preds)"},{"metadata":{"_uuid":"111ff271d4d4f00d3435de4b3e018bf43bdd1e39","_cell_guid":"be8659b0-304d-c197-b417-5183a81974f7","_active":false},"cell_type":"markdown","source":"**Start training slowly :**"},{"metadata":{"_execution_state":"idle","_uuid":"a68c572bcefb5db3426285f6a28060a66057bfbe","_cell_guid":"2048e2d0-f473-cb4f-7525-bbc3075717c2","trusted":false,"_active":false},"cell_type":"code","source":"optim = keras.optimizers.SGD(lr=1e-5, momentum=0.2)\nmodel.compile(loss='mean_squared_error', optimizer=optim)\nmodel.fit(trainX, trainY, epochs=8, verbose=2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b0b83d9adbfb63591eb78a55b15fff104de083d2","_cell_guid":"9c85e582-becd-ed9f-a60d-3528e44911f5","_active":false},"cell_type":"markdown","source":"**Then speed up :**"},{"metadata":{"_execution_state":"idle","_uuid":"8013099dbab26a4a57178bd95dd76aefdd45a0d1","_cell_guid":"470ee712-89e8-3149-6a6a-863a1cb2c1be","trusted":false,"_active":false},"cell_type":"code","source":"optim = keras.optimizers.SGD(lr=1e-4, momentum=0.9)\nmodel.compile(loss='mean_squared_error', optimizer=optim)\nmodel.fit(trainX, trainY, epochs=30, verbose=2)","execution_count":null,"outputs":[]},{"metadata":{"_execution_state":"idle","_uuid":"b0c40bdac3c531ec252d484027e217f9e675d512","_cell_guid":"ca8a06b5-7b77-74d9-d4f9-21e482a594ad","trusted":false,"_active":false},"cell_type":"code","source":"# The kernel was killed for running longer than 1200 seconds ...\nmodel.fit(trainX, trainY, epochs=20, verbose=2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d0715bf60eb4dcb1f87ff7f4b336fb47c29ff76e","_cell_guid":"e26e2efe-0b9d-8aef-6002-b9608acc8fd0","_active":false},"cell_type":"markdown","source":"**Test :**"},{"metadata":{"_execution_state":"idle","_uuid":"1eb5b41bcf6d6cdc4647d88d4469d62cb3e42d31","_cell_guid":"1cad34c2-c304-0b45-00f8-b4e892673f0c","trusted":false,"_active":false},"cell_type":"code","source":"result = model.predict(trainX)\nprint('Training set --')\nprint('    ground truth: ', np.sum(trainY, axis=0))\nprint('  evaluate count: ', np.sum(result*(result>0.3), axis=0).astype('int'))\n\nresult = model.predict(testX)\nprint('Testing set --')\nprint('    ground truth: ', np.sum(testY, axis=0))\nprint('   predict count: ', np.sum(result*(result>0.3), axis=0).astype('int'))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ccb46c486dcb7dbe64a8b83b7e91f12f196e53e4","_cell_guid":"0ff05b53-abbf-cdb7-ea9b-635f1822da22","_active":false},"cell_type":"markdown","source":"## Experience ##\n\nThe challenge is scale problem. They distinguish sea lion by size. In different images, one juveniles is larger than adult_females in another.\n\nI can't handle it well, so I decided to fit LB score:\n\n - scale down testing image get better score\n - more juveniles (less adult_females) get better score\n\nThe final submission is made by:\n\n - testing image scale: 0.48\n - add 50% juveniles, and subtract adult_females with the same amount\n - add 20% pups\n\n**Post processing details:**\n\nThese lucky variables are according to patch level regression.\n\nThe relationship between adult_females and juveniles in patches is:\n\n![juveniles regression][1]\n\n - value in table = average of juveniles# / (adult_females# + juveniles#) @ juveniles number range in patches\n\n - r#.# means image scale\n\n - *#.# means juveniles increase ratio\n\n  [1]: http://i.imgur.com/IkucSf6.gif"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}