{"cells":[{"metadata":{"_uuid":"674fbce0e6b3db0d88c081aa7f5742d6e407f36e"},"cell_type":"markdown","source":"Hello  everyone!  \nThis is a very quick tutorial on using transfer learning  to quickly use existing models and architecture for your own applications.  \nWe'll be using the ResNet network in this kernel, but similar if not identical methods works for other neural nets aswell."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport keras\nimport os\nfrom sklearn import preprocessing\nfrom sklearn.model_selection import train_test_split\nfrom keras.models import Model\n\nfrom tqdm import tqdm\nimport cv2\n\nfrom keras.applications.vgg19 import VGG19\nfrom keras.layers import Dense, Flatten,Dropout\nfrom keras.models import Sequential\n\nfrom keras.layers import (AveragePooling2D, Convolution2D, Dense, Dropout, Flatten, Input, MaxPooling2D) \nfrom keras.models import Model, Sequential\n\nprint(os.listdir(\"../input/\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3012fe883145f5216c1381cc9508561ca42378dc"},"cell_type":"markdown","source":"After we checked out our input, we then proceed to read and save the data in a format that is more usefull to us."},{"metadata":{"trusted":true,"_uuid":"c2fe747e9f28f96d003130c16a608da13b8b7e06"},"cell_type":"code","source":"df_train = pd.read_csv('../input/dog-breed-identification/labels.csv')\ntrain= ('../input/dog-breed-identification/train')\ndf_test = pd.read_csv('../input/dog-breed-identification/sample_submission.csv')\ntest= ('../input/dog-breed-identification/test')\nresnet_weight_paths=(\"../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels.h5\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e273145d64722aba89978cc921de388dea161048"},"cell_type":"code","source":"df_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"95ea14b7eefd76d4ff971ef3600d8d2f53c5b739"},"cell_type":"code","source":"df_train.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ecaa55d3fd883550567f20757ce52147157c57a8"},"cell_type":"markdown","source":"As we cab see the test and train datasets are very different in terms of content.  \nIf you're confused, the test dataset holds the possibilities of the dog belonging to that id.  \nAnd the train set gives us the breed of each dog with their id attached. We'll be using these after a bit of formatting to train or neural net."},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"8e8d2afa5cef67a8671a0f34fbff38ed996b3cab"},"cell_type":"code","source":"len(df_train[\"breed\"].unique())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"68982912d71382edb95d5afa2060a54c3b275b5c"},"cell_type":"markdown","source":"Here I found how many different breeds of dogs we had in the dataset.  \nWe'll be using this later down the line to specify our top layer."},{"metadata":{"trusted":true,"_uuid":"dca9e3145e0a8f8c1e7e9cc7eebb67278d027c92"},"cell_type":"code","source":"num_classes = 120","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bacfbe29fbe9ab05bd8ffd5777ad353659540264"},"cell_type":"markdown","source":"Next we'll read and save all the images in the train set. We'll be using several loops to sort out the data in a meaningfull manner.  \nWe'll be resizing them to 90x90 images and turning them from images to pixel values out computer can actually use to train the model."},{"metadata":{"trusted":true,"_uuid":"b85a73e39076a887fccb25f0c592b52239a712b3"},"cell_type":"code","source":"target_labels = df_train[\"breed\"]\none_hot = pd.get_dummies(target_labels, sparse = True)\none_hot_labels = np.asarray(one_hot)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2be2dc0e543b7bcb3ebccf4198bc6c7a6ef05ff4"},"cell_type":"markdown","source":"What we did here is called \"one hot encoding\".  \nThis means is that we transformed the different labels into an array of 1s and 0s.  \nThis is done in a way such that every label is now held, but instead of by name, we store them by 1s and 0s.  \nHere is a little demo of how we modified our dataset."},{"metadata":{"trusted":true,"_uuid":"9d64d42b26fd6474f955411cc7a8dd98e99ab2ad"},"cell_type":"code","source":"demo = [\"a\",\"b\",\"c\",\"d\",\"e\"]\ndemo_df = pd.DataFrame(demo)\nprint(demo_df)\none_hot_demo = pd.get_dummies(demo_df,sparse = True)\none_hot_demo = np.asarray(one_hot_demo)\none_hot_demo","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"302fa86d1c5405b63f1f603f0e27179ca34be289"},"cell_type":"markdown","source":"Here we have the loop to sort our data into two different arrays without changing the order.  \nWe use a number to keep track of the index to make sure our data doesn't get misallaigned."},{"metadata":{"trusted":true,"_uuid":"bf0019c58223f8c42d689101fd679a78d5b338ae"},"cell_type":"code","source":"img_size = 90\nx_train=[]\ny_train=[]\ni=0\nfor f in tqdm(df_train.id):\n    img = cv2.imread('../input/dog-breed-identification/train/{0}.jpg'.format(f))\n    label = one_hot_labels[i]\n    x_train.append(cv2.resize(img, (img_size, img_size)))\n    y_train.append(label)\n    i += 1","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5dcf7c3a5cadbd148deb8042170e415263c83ab0"},"cell_type":"markdown","source":"Here the images are read, resized and saved into arrays."},{"metadata":{"trusted":true,"_uuid":"2423711ca6b5d2d4d5d7d9776cfab3359abf9418"},"cell_type":"code","source":"x_test = []\nfor f in tqdm(df_test.id):\n    img = cv2.imread('../input/dog-breed-identification/test/{0}.jpg'.format(f))\n    x_test.append(cv2.resize(img, (img_size,img_size))) ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7f26100acd3c6fda8a8cd182dc0de4d411af7857"},"cell_type":"markdown","source":"All the pictures are then reformatted, and divided by 255. to reduce computational load."},{"metadata":{"trusted":true,"_uuid":"add498e06d18bb345dbf504cd51d5a2d57b586be"},"cell_type":"code","source":"x_train_raw = np.array(x_train).astype(\"float32\")/255.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5e226f74c6922f4380aacd25f17083c92bafb202"},"cell_type":"code","source":"y_train_raw = np.array(y_train).astype(\"uint8\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"014c9965b7299652ddbb65a9cd50129ad78c3967"},"cell_type":"code","source":"x_test = np.array(x_test, np.float32) / 255.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"13cfe55d380a137f3f8c8171f6ee855aa1e6762f"},"cell_type":"code","source":"print(x_train_raw.shape)\nprint(y_train_raw.shape)\nprint(x_test.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8c31751dfe2616629bf682004ffb8845d6754570"},"cell_type":"markdown","source":"Alright, now we are certain that our data is in the format we want it in, we can split this data into train and test datasets to validate our results."},{"metadata":{"trusted":true,"_uuid":"1922d003d776741ef5a2f237b4daa557b8f20360"},"cell_type":"code","source":"train_X,test_X,train_Y,test_Y = train_test_split(x_train_raw,y_train_raw,test_size = 0.3, random_state=3)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5b30c591165712018f4280e8884a7ad7b922889e"},"cell_type":"markdown","source":"This is the first way of using transfer learning.  \nWith this method we copy the architecture of the VGG19 model to then train ourselves."},{"metadata":{"trusted":true,"_uuid":"5d91e3dbe8422e83cb2e780d3dd35c36d2c6bc68"},"cell_type":"code","source":"base_model = VGG19(weights = None, include_top=False, input_shape=(img_size, img_size, 3))\n\n# Add a new top layer\nx = base_model.output\nx = Flatten()(x)\npredictions = Dense(num_classes, activation='softmax')(x)\n\n# This is the model we will train\nmodel = Model(inputs=base_model.input, outputs=predictions)\n\n# First: train only the top layers (which were randomly initialized)\nfor layer in base_model.layers:\n    layer.trainable = False\n    \nmodel.compile(loss='categorical_crossentropy', \n              optimizer='adam', \n              metrics=['accuracy'])\n#Take a look at the architecture.\ncallbacks_list = [keras.callbacks.EarlyStopping(monitor='val_acc', patience=3, verbose=1)]\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"122759a423a3f758ae12df898d34c6cf0f4ed59f"},"cell_type":"markdown","source":"[Here](http://keras.io/layers/convolutional/) you can find what each of these layers actually do and how they interact. "},{"metadata":{"trusted":true,"_uuid":"6358dbc190786d20f3bfff3de110824b0ef161c8"},"cell_type":"code","source":"model.fit(train_X, train_Y, epochs=1, validation_data=(test_X, test_Y), verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5f06397a8957fb999b43d539dcecdd6af592a0be"},"cell_type":"code","source":"predictions = model.predict(x_test)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"be59e684eb5bac57dbaf537475e71a3d07a4cc14"},"cell_type":"markdown","source":"Now for the easier method, here we'll simply use the already existing weights trained on the official VGG19 network."},{"metadata":{"trusted":true,"_uuid":"fc8c1a9c72017f5cf7d7d9c2692c925f7e231507"},"cell_type":"code","source":"weights = \"../input/resnet50/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5\"\nnew_model = vgg19(weights = weights, include_top=False, input_shape=(img_size, img_size, 3))\nnew_model.compile(loss='categorical_crossentropy', \n              optimizer='adam', \n              metrics=['accuracy'])\nnew_model.fit(train_X, train_Y, epochs=1, validation_data=(test_X, test_Y), verbose=1)\nnew_predicts = new_model.predict ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3233264f64d223a330f20a84d3b05f1e91f99df1"},"cell_type":"code","source":"#sub = pd.DataFrame(new_predict)\n# Use the previous one_hot_encoded values to give the name to the columns\n#col_names = one_hot.columns.values\n#sub.columns = col_names\n\n#sub.insert(0, 'id', df_test['id'])\n#sub.head(5)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}