{"cells":[{"cell_type":"markdown","metadata":{"_cell_guid":"1848f0f6-cd34-e41c-cd20-5a1d651546d3"},"source":"## Welcome to the base Tensorflow kernel ##\n\n\n----------\nLets get right into it:\n\n 1. First, we will be getting the coordinates with the method used in [this kernel][1]\n 2. We need to then crop the images so that they are classified and only 32 x 32 per lion\n 3. a set of 32 x 32 images **per sea-lion**, **Per: image_id**\n 4. One hot encode these\n 5. Use Tensorflow to build a model\n 6. Train the model\n 7. Plot the results\n\n  [1]: https://www.kaggle.com/radustoicescu/noaa-fisheries-steller-sea-lion-population-count/use-keras-to-classify-sea-lions-0-91-accuracy"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"9ecbdd29-a86f-925e-f630-aaa50b73fc65"},"outputs":[],"source":"from tensorflow.contrib.learn.python.learn.estimators import model_fn as model_fn_lib\nfrom tensorflow.contrib.learn.python import SKCompat\nfrom sklearn.preprocessing import label_binarize\nfrom tensorflow.contrib import learn\nfrom subprocess import check_output\nimport tensorflow as tf\nimport skimage.feature\nimport numpy as np \nimport pandas as pd \nimport cv2\nimport os\n# if anyone knows me, they know my imports\n# have to be ascending order\n\nprint(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))\nprint('# File sizes')\nfor f in os.listdir('../input'):\n    if not os.path.isdir('../input/' + f):\n        print(f.ljust(30) + str(round(os.path.getsize('../input/' + f) / 1000000, 2)) + 'MB')\n    else:\n        sizes = [os.path.getsize('../input/'+f+'/'+x)/1000000 for x in os.listdir('../input/' + f)]\n        print(f.ljust(30) + str(round(sum(sizes), 2)) + 'MB' + ' ({} files)'.format(len(sizes)))"},{"cell_type":"markdown","metadata":{"_cell_guid":"cbde99ff-e82d-f9b8-8770-bcb0c6d195e3"},"source":"As you can see we don't exactly have all the training data."},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"42f615f8-c681-f4e7-4442-e94f12dd04e9"},"outputs":[],"source":"# CREDITS GO TO:  Radu Stoicescu\n# classes = [\"adult_males\", \"subadult_males\", \"adult_females\", \"juveniles\", \"pups\"]\nclasses = ['0','1','2','3','4']\nfile_names = os.listdir(\"../input/Train/\")\nfile_names = sorted(file_names, key=lambda \n                    item: (int(item.partition('.')[0]) if item[0].isdigit() else float('inf'), item)) \n# select a subset of files to run on\nfile_names = file_names[0:3] #INCREASE FOR YOUR OWN MACHINE\nprint(file_names)\n# dataframe to store results in\ncoordinates_df = pd.DataFrame(index=file_names, columns=classes)"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"7d01af6e-b5e4-2480-3e7b-b35fb57ef5b4"},"outputs":[],"source":"# CREDITS GO TO:  Radu Stoicescu\nfor filename in file_names:\n    # read the Train and Train Dotted images\n    image_1 = cv2.imread(\"../input/TrainDotted/\" + filename)\n    image_2 = cv2.imread(\"../input/Train/\" + filename)\n    cut = np.copy(image_2)\n    # absolute difference between Train and Train Dotted\n    image_3 = cv2.absdiff(image_1,image_2)\n    # mask out blackened regions from Train Dotted\n    mask_1 = cv2.cvtColor(image_1, cv2.COLOR_BGR2GRAY)\n    mask_1[mask_1 < 20] = 0\n    mask_1[mask_1 > 0] = 255\n    mask_2 = cv2.cvtColor(image_2, cv2.COLOR_BGR2GRAY)\n    mask_2[mask_2 < 20] = 0\n    mask_2[mask_2 > 0] = 255\n    image_3 = cv2.bitwise_or(image_3, image_3, mask=mask_1)\n    image_3 = cv2.bitwise_or(image_3, image_3, mask=mask_2) \n    # convert to grayscale to be accepted by skimage.feature.blob_log\n    image_3 = cv2.cvtColor(image_3, cv2.COLOR_BGR2GRAY)\n    # detect blobs\n    blobs = skimage.feature.blob_log(image_3, min_sigma=3, max_sigma=4, num_sigma=1, threshold=0.02)\n    adult_males = []\n    subadult_males = []\n    pups = []\n    juveniles = []\n    adult_females = [] \n    image_circles = image_1.copy()\n    for blob in blobs:\n        # get the coordinates for each blob\n        y, x, s = blob\n        # get the color of the pixel from Train Dotted in the center of the blob\n        g,b,r = image_1[int(y)][int(x)][:]\n        # decision tree to pick the class of the blob by looking at the color in Train Dotted\n        if r > 200 and g < 50 and b < 50: # RED\n            adult_males.append((int(x),int(y)))\n            cv2.circle(image_circles, (int(x),int(y)), 20, (0,0,255), 10) \n        elif r > 200 and g > 200 and b < 50: # MAGENTA\n            subadult_males.append((int(x),int(y))) \n            cv2.circle(image_circles, (int(x),int(y)), 20, (250,10,250), 10)\n        elif r < 100 and g < 100 and 150 < b < 200: # GREEN\n            pups.append((int(x),int(y)))\n            cv2.circle(image_circles, (int(x),int(y)), 20, (20,180,35), 10)\n        elif r < 100 and  100 < g and b < 100: # BLUE\n            juveniles.append((int(x),int(y))) \n            cv2.circle(image_circles, (int(x),int(y)), 20, (180,60,30), 10)\n        elif r < 150 and g < 50 and b < 100:  # BROWN\n            adult_females.append((int(x),int(y)))\n            cv2.circle(image_circles, (int(x),int(y)), 20, (0,42,84), 10)  \n        cv2.rectangle(cut, (int(x)-112,int(y)-112),(int(x)+112,int(y)+112), 0,-1)\n    coordinates_df[\"0\"][filename] = adult_males\n    coordinates_df[\"1\"][filename] = subadult_males\n    coordinates_df[\"2\"][filename] = adult_females\n    coordinates_df[\"3\"][filename] = juveniles\n    coordinates_df[\"4\"][filename] = pups"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"12f06fe2-1153-95ef-5612-a5b2bb04f2b4"},"outputs":[],"source":"# CREDITS TO HIM\nx = []\ny = []\nfor filename in file_names:    \n    image = cv2.imread(\"../input/Train/\" + filename)\n    for lion_class in classes:\n        for coordinates in coordinates_df[lion_class][filename]:\n            thumb = image[coordinates[1]-16:coordinates[1]+16,coordinates[0]-16:coordinates[0]+16,:]\n            if np.shape(thumb) == (32, 32, 3):\n                x.append(thumb)\n                y.append(lion_class)\n# Add negs\nfor i in range(0,np.shape(cut)[0],224):\n    for j in range(0,np.shape(cut)[1],224):                \n        thumb = cut[i:i+32,j:j+32,:]\n        if np.amin(cv2.cvtColor(thumb, cv2.COLOR_BGR2GRAY)) != 0:\n            if np.shape(thumb) == (32,32,3):\n                x.append(thumb)\n                y.append(\"5\") \nclasses.append(\"5\")\nx = np.array(x, dtype=np.float32)\ny = np.array(y, dtype=np.float32)"},{"cell_type":"markdown","metadata":{"_cell_guid":"0931d62f-179f-b3f3-7b63-3fab594bee7d"},"source":"## Here is where we switch to Tensorflow ##\n\nWe need to convert our labels to numerical values so TF can validate off of those."},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"9ae33b4f-4f17-060d-1912-d2aaf5644610"},"outputs":[],"source":"#y = label_binarize(y, classes=classes) # Guy in comments gave me this one\ntf.logging.set_verbosity(tf.logging.INFO)"},{"cell_type":"markdown","metadata":{"_cell_guid":"b2e85e33-9842-f4c0-6352-b41d8bef3980"},"source":"Now that we've binarized our labels, we need to create a tensorflow model.\n------------------------------------------------------------------------\n\nOur layers will consist of:\n\n 1. Convolutional (32 5x5 filters) `conv2d()`\n 2. Pooling (Max pooling 2x2, stride of 2) `max_pooling2d()`\n 3. Convolution (64 5x5 layers) `conv2d()`\n 4. Pooling (Max pooling 5x5 filters) `max_pooling2d()`\n 5. Dense 1 (512 number of neurons, dropout at 0.5) `dense()`\n 6. Dense 2 (6 neurons for each class, classes{adult male to pups} plus negative class) `dense()`\n\n*Each of these methods accepts a tensor as input and returns a transformed tensor as output. This makes it easy to connect one layer to another: just take the output from one layer-creation method and supply it as input to another.*\n\n### Tis easy as: ###\n\n![cnn][1]\n\n\n  [1]: https://s18.postimg.org/xlbv52ujd/Screen_Shot_2017-04-21_at_1.17.40_AM.png"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"92521042-eabb-b289-44a1-92682370f062"},"outputs":[],"source":"def CNN_NOAA(features, labels, mode): # create a function to pass to main run\n    # Input Layer\n    input_layer = tf.reshape(features, [-1, 32, 32, 3]) #batch, pixles=32x32x3\n    #Note that we've indicated -1 for batch size, \n    # which specifies that this dimension should be dynamically\n    # computed based on the number of input values in features\n\n    # Convolutional Layer #1\n    conv1 = tf.layers.conv2d(\n      inputs=input_layer,\n      filters=32, # feature map 32 x 32 x 32\n      kernel_size=[5, 5],\n      padding=\"same\",\n      activation=tf.nn.relu) # still 32 x 32 x 32 \n\n    # Pooling Layer #1\n    pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)# 16x16x32\n\n    # Convolutional Layer #2 \n    conv2 = tf.layers.conv2d(\n      inputs=pool1,\n      filters=64, # 16x16x64\n      kernel_size=[5, 5],\n      padding=\"same\",\n      activation=tf.nn.relu)\n\n    # Pooling Layer #2\n    pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2) # 8x8x64\n\n    # Dense Layer\n    pool2_flat = tf.reshape(pool2, [-1, 8 * 8 * 64])\n    dense = tf.layers.dense(inputs=pool2_flat, units=1024, activation=tf.nn.relu)\n    dropout = tf.layers.dropout(\n      inputs=dense, rate=0.4, training=mode == learn.ModeKeys.TRAIN)\n\n    # Logits Layer\n    logits = tf.layers.dense(inputs=dropout, units=6)\n\n    loss = None # starts at none, then gets computed each time\n    train_op = None # starts at none, then gets computed each time\n\n    # Calculate Loss (for TRAIN mode)\n    if mode != learn.ModeKeys.INFER:\n        onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=6)\n        loss = tf.losses.softmax_cross_entropy(\n        onehot_labels=onehot_labels, logits=logits)\n\n    # Configure the Training Op (for TRAIN mode)\n    if mode == learn.ModeKeys.TRAIN:\n        train_op = tf.contrib.layers.optimize_loss(\n        loss=loss,\n        global_step=tf.contrib.framework.get_global_step(),\n        learning_rate=0.01, # i know its high as fuck, im on kaggle not my machine\n        optimizer=\"SGD\")\n\n    # Generate Predictions\n    predictions = {\n      \"classes\": tf.argmax(\n          input=logits, axis=1),\n      \"probabilities\": tf.nn.softmax(\n          logits, name=\"softmax_tensor\")\n    }\n    # Return a ModelFnOps object\n    return model_fn_lib.ModelFnOps(\n        mode=mode, predictions=predictions, loss=loss, train_op=train_op)"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"89d377ad-dfe7-a486-16f6-cc49efea431c"},"outputs":[],"source":"# Load training from x and y\ntrain_data = x # Returns image arrays\ntrain_labels = y # the label array\n\n#for evaluation we want to grab a photo from the train and test how well we did on it\neval_data = train_data #\neval_label = train_labels #"},{"cell_type":"markdown","metadata":{"_cell_guid":"5e887a0a-0484-7e33-2543-39e6407efd7e"},"source":"## Estimator ##\n\n**a TensorFlow class for performing high-level model training, evaluation, and inference) for our model.**"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"c264950c-d09d-7d98-4ea5-1d9a48488d6a"},"outputs":[],"source":"# its going to write to a model directory (output)\nnoaa_classifier = SKCompat(learn.Estimator(model_fn=CNN_NOAA))"},{"cell_type":"markdown","metadata":{"_cell_guid":"d3f9208e-0657-2757-9731-989b16c245df"},"source":"## Add what we need to log ##"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"774c0641-ae5a-a960-b023-7bd54e6872fb"},"outputs":[],"source":"# Set up logging for predictions\ntensors_to_log = {\"probabilities\": \"softmax_tensor\"}\nlogging_hook = tf.train.LoggingTensorHook(\n  tensors=tensors_to_log, every_n_iter=500)"},{"cell_type":"markdown","metadata":{"_cell_guid":"6d71c12c-8646-e973-a8c0-9dda403f1812"},"source":"## Train the model ##\nWe call `fit()` on noaa_classifier"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"1cbec54c-7472-646f-aee3-635dc1e324cf"},"outputs":[],"source":"noaa_classifier.fit(\n    train_data,\n    train_labels,\n    batch_size=512,\n    steps=200,  \n    monitors=[logging_hook]) # Again, on kaggle machine"},{"cell_type":"markdown","metadata":{"_cell_guid":"e8e4d987-0464-1f89-f51b-eb570357285c"},"source":"## I planned on doing 20,000 steps ##\nIf I had the time and the computer power I would to 20,000 steps. Anything over 200 on Kaggle just hangs.."},{"cell_type":"markdown","metadata":{"_cell_guid":"0b2d5eac-bdd9-c717-025b-55f8e8600ff1"},"source":"## Evaluating our Model ##\n**How well did it do?**\n\n    metric_fn.\n \nThe function that calculates and returns the value of our metric\n\n    prediction_key\n\nThe key of the tensor that contains the predictions returned by the model function."},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"41e3dd35-0cc8-f0d4-b57b-85923ef29893"},"outputs":[],"source":"metrics = {\n    \"accuracy\": # What we're tracking\n        learn.MetricSpec( # calculation function\n            metric_fn=tf.metrics.accuracy, prediction_key=\"classes\"), # returns class predctions\n}"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"681b59bc-89cd-de85-bf92-95102f95d355"},"outputs":[],"source":"# Evaluate the model and print results\neval_results = noaa_classifier.predict(\n    eval_data[0], batch_size=128)\nprint(eval_results)"},{"cell_type":"markdown","metadata":{"_cell_guid":"bda9fc42-4dc6-3944-82ff-92f116724e05"},"source":"**It calculates the probability for a sea-lion type in a cropped 32x32 picture of one that is passed through noaa_classifier.predict**\n\n**Under \"class\" it shows 2**, from 0, 1 , 2 :  \"2\"  is **adult female**\n\n----------\nThings to improve:\n\n 1. Using a larger subset of the train images\n 2. more steps in training \n 3. choose a model out_dir\n 4. Decode the probabilities to give a better response.\n"}],"metadata":{"_change_revision":0,"_is_fork":false,"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.0"}},"nbformat":4,"nbformat_minor":0}