{"cells":[{"metadata":{"_uuid":"11a56a4a04d7af2d7c03560cbb3038be5419f22c"},"cell_type":"markdown","source":"Working with images is pretty memory consuming, especially if you read and preprocess all of them at the same time. The following approach avoids this problem in Keras, leaving more space in memory to use augmentation and/or loading pre-trained models. I hope this helps Kagglers to work on their networks in Kaggle kernels without worrying (or at least worrying less) with the 14GB of memory provided using GPU environment. I tried to comment the comment the code as clearer as I could, but if some part was not well-explained or if you have doubts, ask on the comments :) \n\n# Contents\n___\n1. [(Very) brief data exploration](#exploration)\n2. [Training models using generators and flow_from_dataframe](#flow)"},{"metadata":{"trusted":true,"_uuid":"0d9c73ad23e6c2eae3028255ee00c3254fe66401"},"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport os\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mplimg\nfrom matplotlib.pyplot import imshow\nfrom skimage.io import imread\nimport cv2\n\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.preprocessing import OneHotEncoder\n\nfrom keras import layers\nfrom keras.preprocessing import image\nfrom keras.applications.imagenet_utils import preprocess_input\nfrom keras.layers import Input, Dense, Activation, BatchNormalization, Flatten, Conv2D\nfrom keras.layers import AveragePooling2D, MaxPooling2D, Dropout, GlobalMaxPooling2D, GlobalAveragePooling2D, Concatenate\nfrom keras.models import Model\nfrom keras.applications.vgg16 import VGG16\nfrom keras.callbacks import EarlyStopping, ModelCheckpoint\n\nimport keras.backend as K\nfrom keras.models import Sequential\n\nimport warnings\nwarnings.simplefilter(\"ignore\", category=DeprecationWarning)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2cea35de3530cc898be5b85063b84e875401d092"},"cell_type":"code","source":"os.listdir(\"../input/\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"46a8839e13a14eb8d16ea6823de9927ea63d5001"},"cell_type":"code","source":"train_df = pd.read_csv(\"../input/train.csv\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"67ae1f5ec310a945ab45d763a182b0018f883cff"},"cell_type":"markdown","source":"# (Very) brief data exploration \n___ \n\nLet's see some whale images that we have."},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"65a7b542f911fb4a0a2f614cecac205aa5a6358b"},"cell_type":"code","source":"img_path = '../input/train/'\n\n#get the first 5 whale images\nimages = [(whale_img, whale_label) for (whale_img, whale_label) in zip(train_df.Image[:5], train_df.Id[:5])]\n\nfig, m_axs = plt.subplots(1, len(images), figsize = (20, 10))\n#show the images and label them\nfor ii, c_ax in enumerate(m_axs):\n c_ax.imshow(imread(os.path.join(img_path,images[ii][0])))\n c_ax.set_title(images[ii][1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c792fed9e252e589df2fa9961a55cfe9b9855741"},"cell_type":"markdown","source":"At first the task seems 'simple' to begin with as this is a classification problem. The problem in this dataset is that there are 5005 kinds of whales (including the category new_whale). Since there are about 25k images, this gives us about 5 images of each whale on average, which is an extremely low number to train a model. But let's confirm that first."},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"251dfe2fddcd12a2f2c6d5ccb6e66cd5d1410438"},"cell_type":"code","source":"#how many photos of each whale\ntrain_df.Id.value_counts()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"184ceb314a97e701f513c3fea32fea2a8ce41279"},"cell_type":"markdown","source":"Now here we need to be careful. Several whales have been spotted considerably more times than others. Moreover, 9664 whales in the training dataset are actually non-identified whales. There are also many classes with only one image. I believe that correctly identifying (i.e. guessing a photo is one of them with a higher probability) those ones will define the top Kagglers in this competition. Now the fun begins. \n\n# Training models using generators and flow_from_dataframe \n____\n\nIn order to avoid using consuming so much memory, we will make use of the `fit_generator` method in our model. To use it, we have three options to get and train our data: `flow`, `flow_from_dataframe` and `flow_from_directory`. [Here](https://keras.io/preprocessing/image/#imagedatagenerator-class/) you can check the full documentation and their arguments. \n\nThe first method, `flow` is used when we already have all the data we need available in memory. Here, however, we only have the image's location in memory right now, and we don't want to load all images at the same time to avoid extra memory consumption. Therefore we won't use `flow`. \n\nThe second one, `flow_from_dataframe` will be very useful to us. It checks the path available on the dataframe and then automatically search for the image in train directory. Then it make the desired preprocessing steps available in `ImageDataGenerator`. \n\nThe last method, `flow_from_directory`, will be used during testing, since we don't have a dataframe containing all the image's paths to the test set. This method looks inside the directory and get `batch_size` images to make the classification. \n\nIn order to use `flow_from_directory` in our test data, we can't just use \n```python \n'../input/test/' \n``` \nbecause the the folders need to be arranged in the following format:\n\n\n[Source](https://medium.com/@vijayabhaskar96/tutorial-image-classification-with-keras-flow-from-directory-and-generators-95f75ebe5720)\n\nSo we will need to create a folder that contains a folder that contains our images. This will make our generator recognize that there is one class of images in the test dataset (think of it as an \"unlabeled\" class) so then we can get this data and use our classifier to predict their classes. Let's do this now.\n"},{"metadata":{"trusted":true,"_uuid":"7dc9a691b988465dd989e5f5e26c6927729daa74","_kg_hide-output":true},"cell_type":"code","source":"import os\nfrom distutils.dir_util import copy_tree\n\n#create directories\nos.mkdir('test_folder')\nos.mkdir('test_folder/test_images')\n\n# copy subdirectory example\nfromDirectory = \"../input/test\"\ntoDirectory = \"test_folder/test_images\"\n\ncopy_tree(fromDirectory, toDirectory, verbose=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b556eed9fa5adf2ca828b3ee850e5e69b9d90dea","scrolled":true},"cell_type":"code","source":"''' \nmake sure all preprocessing done in the training \nimage generator is done in test generator as well\n'''\n\n# validation_split sets the percentage of data generated\n# to be used in validation phase\ndatagen=image.ImageDataGenerator(rescale=1./255, validation_split = 0.1)\ntest_datagen = image.ImageDataGenerator(rescale=1./255) \n\n'''\nComments:\n- ImageDataGenerator will resize all images to target_size\n- x_col is the column where the images' names are\n- y_col is the column where the labels are\n- has_ext means that the images' names include a file extension, e.g. image_name.jpg\n- Here you can change the targe_size to resize all images to different shapes.\nMaybe larger images help in getting a better accuracy\n'''\n\n# since the datagen is splitted in training and validation,\n# make sure to set subsets correctly\n\ntrain_generator=datagen.flow_from_dataframe(\n dataframe=train_df, directory=\"../input/train/\", \n x_col=\"Image\", y_col=\"Id\", has_ext=True, seed = 42,\n class_mode=\"categorical\", target_size=(100,100), batch_size=32, subset = \"training\")\n\nvalidation_generator = datagen.flow_from_dataframe(dataframe=train_df, directory=\"../input/train/\", \n x_col=\"Image\", y_col=\"Id\", has_ext=True, seed = 42,\n class_mode=\"categorical\", target_size=(100,100), batch_size=32, subset = \"validation\")\n\n# make sure shuffle is set to false, so the predictions are done on the same order\n# as they appear on the directory. batch_size should be 1 to make the\n# predictions image by image\n\ntest_generator = test_datagen.flow_from_directory(directory=\"test_folder\", \n seed = 42, class_mode=None, target_size=(100,100), batch_size=1, shuffle = False)\n\nSTEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size\nSTEP_SIZE_VALID=validation_generator.n//validation_generator.batch_size","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"962d6da478a301d615354486af46a4f208d952e1"},"cell_type":"markdown","source":"Next let's build, compile and train the model. Here I'll use a simple Convolutional layer, Dropout and GlobalMaxPooling for the sake of simplicity. You can plug your model right here and test it! Just make sure `input_shape` from Conv2D is the same that `target_size` in the generators."},{"metadata":{"trusted":true,"_uuid":"16a9ccbecde62d9c1a057efa50281f2eb7df14b1"},"cell_type":"code","source":"from keras.metrics import top_k_categorical_accuracy\n\n''' the function top_5_accuracy is from Peter's kernel:\n https://www.kaggle.com/pestipeti/keras-cnn-starter\n'''\ndef top_5_accuracy(y_true, y_pred):\n return top_k_categorical_accuracy(y_true, y_pred, k=5)\n\nmodel = Sequential()\nmodel.add(Conv2D(filters = 16, kernel_size = 3, padding = 'same', activation = 'relu', input_shape = (100, 100, 3)))\nmodel.add(Dropout(0.5))\nmodel.add(GlobalMaxPooling2D()) \n# model.add(Flatten())\nmodel.add(Dense(5005, activation = 'softmax'))\n# model.summary()\n\nmodel.compile(optimizer= 'adam', loss='categorical_crossentropy', metrics=['accuracy', top_5_accuracy])\n\n# early_stopping = EarlyStopping(monitor = 'val_loss', patience = 5)\ncheckpointer = ModelCheckpoint(filepath='weights.hdf5', \n verbose=1, save_best_only=True)\n\nearly_stopping = EarlyStopping(monitor = 'val_loss', patience = 5)\n\nmodel.fit_generator(generator=train_generator,\n steps_per_epoch=STEP_SIZE_TRAIN,\n validation_data=validation_generator,\n validation_steps=STEP_SIZE_VALID,\n epochs=2, callbacks = [checkpointer, early_stopping])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"9e62c51a5ece0501aeed8f3bde7489e6955a3fa7"},"cell_type":"code","source":"#we need to use .reset() here otherwise\n#the other of predictions will be different\n#then the expected\ntest_generator.reset()\npred = model.predict_generator(test_generator,verbose = 1,steps=7960)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"88d6231548b14af35ae5bd576225160d05dfef33","scrolled":true},"cell_type":"code","source":"'''This filters only the top 5 possible ids of an image'''\npred_sorted = np.argsort(-pred, axis = 1)[:,:5]\npred_sorted","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9310827f1787a20556c9333a93bc965dbfe8f0f7"},"cell_type":"markdown","source":"Note that our model has clearly overfit. Indexes 3, 265 and 166 are the most common indexes in our training dataset (3 is new_whale). In fact, this overfit was expected because I didn't augment the data yet and the network was kept really really simple just to use it as a proxy for a real well-thought network. \nSome basic augmentation can be done within `ImagedataGenerator`. Check the [documentation](https://keras.io/preprocessing/image/) for some ideas!\n\nNow let's proceed to the final steps. "},{"metadata":{"trusted":true,"_uuid":"c2d7ae2815556f6109757e23f19a7f12dc112c87"},"cell_type":"code","source":"'''\nNow we generate a map of each \nindex to an Id on the format \n{\n0: 'w_f48451c',\n1: 'w_c3d896a',\n2: 'w_20df2c5',\n...\n}\n'''\nlabels = (train_generator.class_indices)\nlabels = dict((v,k) for k,v in labels.items())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"10cede985340f4d2c18534085fe77370ef866d04"},"cell_type":"code","source":"'''\nHere we prepare pred_ids, which is a list of lists containing \nthe top 5 ids by name. For example, w_13ae3d4. \n'''\nfrom tqdm import tqdm\n#create empty list\npred_ids = list()\nfor i,row in enumerate(tqdm(pred_sorted)):\n #create a temporary list to store the ids for a given image\n temp_list = []\n for j,value in enumerate(row):\n #for each index in pred_sorted, append the real Id in temp_list\n temp_list.append(labels[row[j]])\n #append all 5 ids for a given image to pred_ids\n #effectively creating a similar list to pred_sorted\n #but with the real ids\n pred_ids.append(temp_list)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"df9cc9d2cac8e33f8b22d4661e90bdf5489f2511"},"cell_type":"code","source":"'''create the final predictions by using all ids in a single string'''\nfinal_preds = []\nfor i,top_5_ids in enumerate(pred_ids):\n final_preds.append(' '.join(pred_ids[i]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5daa8430d7bc301110623e0db49de10cdb63a003"},"cell_type":"code","source":"'''delete the files on disk - otherwise the Kaggle kernel will throw an error'''\nimport shutil\nshutil.rmtree('test_folder', ignore_errors=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"47cc8b430b81f400f685d7f03ecfe20cc08035e5"},"cell_type":"code","source":"submission = pd.DataFrame({\"Image\": os.listdir('../input/test'), \"Id\": final_preds})\nsubmission.to_csv(\"submission.csv\", index = False)\nsubmission.head()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}