{"cells":[{"metadata":{},"cell_type":"markdown","source":"<img src=\"https://storage.googleapis.com/kaggle-competitions/kaggle/5408/media/bigleaves.jpg\" width=\"600\"></img>"},{"metadata":{},"cell_type":"markdown","source":"### This kernel is base on [Alexander Teplyuk](https://www.kaggle.com/ateplyuk/inat2019-starter-keras/output) here I applied Data Augmentation technic from [Udacity](https://colab.research.google.com/github/tensorflow/examples/blob/master/courses/udacity_intro_to_tensorflow_for_deep_learning/l05c04_exercise_flowers_with_data_augmentation_solution.ipynb#scrollTo=UOoVpxFwVrWy) as following:\n* random 45 degree rotation\n* random zoom of up to 50%\n* random horizontal flip\n* width shift of 0.15\n* height shfit of 0.15"},{"metadata":{},"cell_type":"markdown","source":"> * Model: vgg16  \n Apply transfer learning skill from pretrained model using Keras.\n> Using vgg16 with a flatten layer followed by 2 fully connected layer with 1024 units and 1010 units. The output class probabilities based on 1010 classes which is done by the softmax activation function. Add a layer use a relu activation function. Add Dropout layers with a probability of 50%, where appropriate."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import os\nimport cv2\nimport matplotlib.pyplot as plt\nimport pandas as pd\nimport numpy as np\nimport json\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Flatten, Activation, Dropout, Conv2D,MaxPooling2D\nfrom keras.optimizers import Adam\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.applications import VGG16\nfrom keras.applications import ResNet50\nfrom keras import optimizers","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Train data"},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"ann_file = '../input/train2019.json'\nwith open(ann_file) as data_file:\n        train_anns = json.load(data_file)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_anns_df = pd.DataFrame(train_anns['annotations'])[['image_id','category_id']]\ntrain_img_df = pd.DataFrame(train_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})\ndf_train_file_cat = pd.merge(train_img_df, train_anns_df, on='image_id')\ndf_train_file_cat['category_id']=df_train_file_cat['category_id'].astype(str)\ndf_train_file_cat.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(df_train_file_cat['category_id'].unique())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Example of images for category_id = 400\nimg_names = df_train_file_cat[df_train_file_cat['category_id']=='400']['file_name'][:30]\n\nplt.figure(figsize=[15,15])\ni = 1\nfor img_name in img_names:\n    img = cv2.imread(\"../input/train_val2019/%s\" % img_name)[...,[2, 1, 0]]\n    plt.subplot(6, 5, i)\n    plt.imshow(img)\n    i += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Validation data"},{"metadata":{"trusted":true},"cell_type":"code","source":"valid_ann_file = '../input/val2019.json'\nwith open(valid_ann_file) as data_file:\n        valid_anns = json.load(data_file)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"valid_anns_df = pd.DataFrame(valid_anns['annotations'])[['image_id','category_id']]\nvalid_anns_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"valid_img_df = pd.DataFrame(valid_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})\nvalid_img_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_valid_file_cat = pd.merge(valid_img_df, valid_anns_df, on='image_id')\ndf_valid_file_cat['category_id']=df_valid_file_cat['category_id'].astype(str)\ndf_valid_file_cat.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"nb_classes = 1010\nbatch_size = 128\nimg_size = 128\nnb_epochs = 10","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### In the cell below, use ImageDataGenerator to create a transformation that rescales the images by 255 here I applied Data Augmentation as following:\n* random 45 degree rotation\n* random zoom of up to 50%\n* random horizontal flip\n* width shift of 0.15\n* height shfit of 0.15"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntrain_datagen=ImageDataGenerator(rescale=1./255, rotation_range=45, \n                    width_shift_range=.15, \n                    height_shift_range=.15, \n                    horizontal_flip=True, \n                    zoom_range=0.5)\n\ntrain_generator=train_datagen.flow_from_dataframe(\n    dataframe=df_train_file_cat,\n    directory=\"../input/train_val2019\",\n    x_col=\"file_name\",\n    y_col=\"category_id\",\n    batch_size=batch_size,\n    shuffle=True,\n    class_mode=\"sparse\",    \n    target_size=(img_size,img_size))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# udacity_intro_to_tensorflow_for_deep_learning/l05c04_exercise_flowers_with_data_augmentation_solution.ipynb#scrollTo=jqb9OGoVKIOi\n# This function will plot images in the form of a grid with 1 row and 5 columns where images are placed in each column.\ndef plotImages(images_arr):\n    fig, axes = plt.subplots(1, 5, figsize=(20,20))\n    axes = axes.flatten()\n    for img, ax in zip( images_arr, axes):\n        ax.imshow(img)\n    plt.tight_layout()\n    plt.show()\n    \n    \naugmented_images = [train_generator[0][0][0] for i in range(5)]\nplotImages(augmented_images)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntest_datagen = ImageDataGenerator(rescale=1./255)\n\nvalid_generator=test_datagen.flow_from_dataframe(\n    dataframe=df_valid_file_cat,\n    directory=\"../input/train_val2019\",\n    x_col=\"file_name\",\n    y_col=\"category_id\",\n    batch_size=batch_size,\n    class_mode=\"sparse\",    \n    target_size=(img_size,img_size))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"vgg16_net = VGG16(weights='imagenet', \n                  include_top=False, \n                  input_shape=(img_size, img_size, 3))\nvgg16_net.trainable = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"resnet = ResNet50(include_top=False, weights='imagenet',\n               input_shape=(img_size,img_size,3))\nresnet.trainable = False","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = Sequential()\nmodel.add(vgg16_net)\nmodel.add(Flatten())\nmodel.add(Dense(1024))\nmodel.add(Activation('relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(nb_classes, activation='softmax'))\n\nmodel.compile(optimizers.rmsprop(lr=0.0001, decay=1e-6),loss='sparse_categorical_crossentropy',metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nhistory = model.fit_generator(generator=train_generator, \n                              \n                    steps_per_epoch=500,\n                              \n                    validation_data=valid_generator, \n                              \n                    validation_steps=100,\n                              \n                    epochs=nb_epochs,\n                    verbose=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('history.json', 'w') as f:\n    json.dump(history.history, f)\n\nhistory_df = pd.DataFrame(history.history)\nhistory_df[['loss', 'val_loss']].plot()\nhistory_df[['acc', 'val_acc']].plot()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"acc = history.history['acc']\nval_acc = history.history['val_acc']\n\nloss = history.history['loss']\nval_loss = history.history['val_loss']\n\nepochs_range = range(nb_epochs)\n\nplt.figure(figsize=(8, 8))\nplt.subplot(1, 2, 1)\nplt.plot(epochs_range, acc, label='Training Accuracy')\nplt.plot(epochs_range, val_acc, label='Validation Accuracy')\nplt.legend(loc='lower right')\nplt.title('Training and Validation Accuracy')\n\nplt.subplot(1, 2, 2)\nplt.plot(epochs_range, loss, label='Training Loss')\nplt.plot(epochs_range, val_loss, label='Validation Loss')\nplt.legend(loc='upper right')\nplt.title('Training and Validation Loss')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Test data"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_ann_file = '../input/test2019.json'\nwith open(test_ann_file) as data_file:\n        test_anns = json.load(data_file)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_img_df = pd.DataFrame(test_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})\ntest_img_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntest_datagen = ImageDataGenerator(rescale=1./255.)\ntest_generator = test_datagen.flow_from_dataframe(      \n    \n        dataframe=test_img_df,    \n    \n        directory = \"../input/test2019\",    \n        x_col=\"file_name\",\n        target_size = (img_size,img_size),\n        batch_size = 1,\n        shuffle=False,\n        class_mode = None\n        )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Prediction"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntest_generator.reset()\npredict=model.predict_generator(test_generator, steps = len(test_generator.filenames),verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(predict)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predicted_class_indices=np.argmax(predict,axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels = (train_generator.class_indices)\nlabels = dict((v,k) for k,v in labels.items())\npredictions = [labels[k] for k in predicted_class_indices]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sam_sub_df = pd.read_csv('../input/kaggle_sample_submission.csv')\nsam_sub_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"filenames=test_generator.filenames\nresults=pd.DataFrame({\"file_name\":filenames,\n                      \"predicted\":predictions})\ndf_res = pd.merge(test_img_df, results, on='file_name')[['image_id','predicted']]\\\n    .rename(columns={'image_id':'id'})\n\ndf_res.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_res.to_csv(\"submission.csv\",index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Hope you like it and finds this kernel helpful :)!"},{"metadata":{},"cell_type":"markdown","source":"# Reference\n[Alexander Teplyuk](https://www.kaggle.com/ateplyuk/inat2019-starter-keras/output)  \n[Udacity](https://colab.research.google.com/github/tensorflow/examples/blob/master/courses/udacity_intro_to_tensorflow_for_deep_learning/l05c04_exercise_flowers_with_data_augmentation_solution.ipynb#scrollTo=08rRJ0sn3Tb1)  \nhttps://medium.com/@vijayabhaskar96/tutorial-on-keras-flow-from-dataframe-1fd4493d237c"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}