According to the World Health Organization (WHO) survey, 1.3 million people worldwide die in traffic accidents each year, making them the eighth leading cause of death and an additional 20-50 millions are injured/ disabled. As per the report of National Crime Research Bureau (NCRB), Govt. of India, Indian roads account for the highest fatalities in the world. There has been a continuous increase in road crash deaths in India since 2006. The report also states that the total number of deaths have risen to 1.46 lakhs in 2015 and driver error is the most common cause behind these traffic accidents.The number of accidents because of distracted driver has been increasing since few years.
\n\nNational Highway Traffic Safety Administrator of United States (NHTSA) reports deaths of 3477 people and injuries to 391000 people in motor vehicle crashes because of distracted drivers in 2015. In the United States, everyday approximately 9 people are killed and more than 1,000 are injured in road crashes that are reported to involve a distracted driver. NTHSA describes distracted driving as “any activity that diverts attention of the driver from the task of driving” which can be classified into Manual, Visual or Cognitive distraction. As per the definitions of Center for Disease Control and Prevention (CDC), cognitive distraction is basically “driver’s mind is off the driving”.
\n\nn other words, even though the driver is in safe driving posture, he is mentally distracted from the task of driving. He might be lost in thoughts, daydreaming etc. Distraction because of inattention, sleepiness, fatigue or drowsiness falls into visual distraction class where “drivers’s eyes are off the road”. Manual distractions are concerned with various activities where “driver’s hands are off the wheel”. Such distractions include talking or texting using mobile phones, eating and drinking, talking to passengers in the vehicle, adjusting the radio, makeup etc.Nowadays, Advanced Driver Assistance Systems (ADAS) are being developed to prevent accidents by offering technologies that alert the driver to potential problems and to keep the car’s driver and occupants safe if an accident does occur. But even today’s latest autonomous vehicles require the driver to be attentive and ready to take the control of the wheel back in case of emergency.
\n\nTesla autopilot’s crash with the white truck-trailor in Williston, Florida in May 2016 was the first fatal crash in testing of autonomous vehicle. Recently in March 2018, Uber’s self driving car with an emergency backup driver behind the wheel struck and killed a pedestrian in Arizona. In both of these fatalities, the safety driver could have avoided the crashes but evidences reveal that he was clearly distracted. This makes detection of distracted driver an essential part of the self driving cars as well. We believe that distracted driver detection is utmost important for further preventive measures. If the vehicle could detect such distractions and then warn the driver against it, number of road crashes can be reduced. In this projects, we focus on detecting manual distractions where driver is engaged in other activities than safe driving and also identify the cause of distraction. We present a Convolutional Neural Network based approach for this problem. We also attempt to reduce the computational complexity and memory requirement while maintaining good accuracy which is desirable in real time applications.
"},{"metadata":{},"cell_type":"markdown","source":"\n## Problem Statment \n\nGiven a dataset of 2D dashboard camera images, an algorithm needs to be developed to classify each driver's behaviour and determine if they are driving attentively, wearing their seatbelt, or taking a selfie with their friends in the backseat etc..? This can then be used to automatically detect drivers engaging in distracted behaviours from dashboard cameras.\n\nFollowing are needed tasks for the development of the algorithm:\n1. Download and preprocess the driver images.\n1. Build and train the model to classify the driver images.\n1. Test the model and further improve the model using different techniques."},{"metadata":{},"cell_type":"markdown","source":"\n\n## Tools\n\nThe project utilizes the following dependencies:\n\n- Python 3.5: Tensorflow, Keras, Numpy, Scipy, seaborn, Matplotlib.\n- NVIDIA Geforce GTX1060 GPU, CUDA, CuDNN."},{"metadata":{},"cell_type":"markdown","source":"\n\n## Dataset\nThe provided data set has driver images, each taken in a car with a driver doing something in the car (texting, eating, talking on the phone, makeup, reaching behind, etc). This dataset is obtained from [Kaggle](https://www.kaggle.com/c/state-farm-distracted-driver-detection/data).(State Farm Distracted Driver Detection competition).\n\nFollowing are the file descriptions and URL’s from which the data can be obtained :\n\n* imgs.zip - zipped folder of all (train/test) images\n* sample_submission.csv - a sample submission file in the correct format\n* driver_imgs_list.csv - a list of training images, their subject (driver) id, and class id"},{"metadata":{},"cell_type":"markdown","source":"\n## The 10 classes to predict\n\n- c0: safe driving **\n- c1: texting - right\n- c2: talking on the phone - right\n- c3: texting - left\n- c4: talking on the phone - left\n- c5: operating the radio\n- c6: drinking\n- c7: reaching behind\n- c8: hair and makeup\n- c9: talking to passenger\n\nThere are 22423 train images and 79725 test images."},{"metadata":{},"cell_type":"markdown","source":"\n\n## Implementation\n\nThis project aims to develop a machine learning system that can detect and classify different distracted states of car drivers. The main approach is to apply deep convolutional neural networks (CNNs). We will explore and experiment various CNN architectures, leveraged pre-trained networks (learning transfer).\n\n#### Transfer Learning\n\nMost of the time you won't want to train a whole convolutional network yourself. Modern ConvNets training on huge datasets like ImageNet take weeks on multiple GPUs. \n\nInstead, most people use a pretrained network either as a fixed feature extractor, or as an initial network to fine tune. \n\nIn this notebook, you'll be using [VGGNet] trained on the [ImageNet dataset] as a feature extractor. Below is a diagram of the VGGNet architecture, with a series of convolutional and maxpooling layers, then three fully-connected layers at the end that classify the 1000 classes found in the ImageNet database.\n\n![](\"../input/vgg-pic/vgg_16_architecture.png\")
\n\n#!img[VGG Architecture]\nVGGNet is great because it's simple and has great performance, coming in second in the ImageNet competition. The idea here is that we keep all the convolutional layers, but **replace the final fully-connected layer** with our own classifier. This way we can use VGGNet as a *fixed feature extractor* for our images then easily train a simple classifier on top of that. \n\n\n\n"},{"metadata":{},"cell_type":"markdown","source":"\n\n# PREPARING"},{"metadata":{},"cell_type":"markdown","source":"\n\n## Import libraries that will be used"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow import keras \nfrom tensorflow.keras import layers\n\nfrom keras.models import Sequential, Model\nfrom keras.layers.core import Flatten, Dense, Dropout\nfrom keras.layers.convolutional import Convolution2D, MaxPooling2D, ZeroPadding2D\nfrom keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization, GlobalAveragePooling2D\nfrom keras.optimizers import SGD\nfrom keras.applications.vgg16 import VGG16\n\nfrom keras.utils import np_utils\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.preprocessing import image\nfrom keras.callbacks import ModelCheckpoint, EarlyStopping\n\nfrom IPython.display import FileLink,display, Image\nfrom PIL import Image as I\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.datasets import load_files \nfrom sklearn.utils import shuffle\nfrom sklearn.metrics import log_loss\n\nfrom random import sample\nimport random\n\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nimport seaborn as sns\n\nimport numpy as np\nimport pandas as pd\n\nfrom tqdm import tqdm\nfrom glob import glob\nimport pickle\nimport zipfile\nimport os\nimport cv2\nimport timeit\nimport time\nimport h5py","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create folder\nmodels_dir = \"saved_models\"\nif not os.path.exists(models_dir):\n os.makedirs(models_dir)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3b048fe573382e8dad5903b8757af5e6fe0689e6"},"cell_type":"code","source":"# check if CUDA is available\nif tf.test.is_gpu_available(cuda_only=True):\n print('CUDA is available! Training on GPU ...')\n\nIMG_SIZE = 244 if tf.test.is_gpu_available(cuda_only=True) else 160\nCOLOR_TYPE = 3\nCLASSES = 10\nEPOCHS = 30\nBATCHES = 50\nIMG_SIZE = 224\nTEST_SIZE = 10\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n## Exploration\n\nData exploration, preprocessing and analy will be conducted in great details to gain as much information about the dataset as possible. All steps of a machine learning pipeline are included and a summary is provided at the end of each section.\n"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"df = pd.read_csv(\"../input/state-farm-distracted-driver-detection/driver_imgs_list.csv\")\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set()\n\nplt.figure(figsize = (10,5))\n# Count the number of images per category\nsns.countplot(x = 'classname', color = '#169DE3',data = df)\n\nplt.title('Categories Distribution'.title(),size=22 , color = '#169DE3')\nplt.xlabel('classname',size=17 , color = '#169DE3')\nplt.ylabel('Count',size=17 , color = '#169DE3')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n# PREPROCESSING"},{"metadata":{},"cell_type":"markdown","source":"\n \n## Load Training data"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Load the dataset from Kaggle\ndef get_cv2_image(path, img_size, color_type):\n # Loading as Grayscale image\n if color_type == 1:\n img = cv2.imread(path, cv2.IMREAD_GRAYSCALE)\n # Loading as color image\n elif color_type == 3:\n img = cv2.imread(path, cv2.IMREAD_COLOR)\n # Reduce size\n img = cv2.resize(img[:500], (img_size, img_size)) \n return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_trainning_data(img_size , color_type):\n start_time = time.time()\n training_images = []\n training_labels = []\n\n # Loop over the training folder \n for class_ in tqdm(range(CLASSES)):\n \n print('Loading directory c{}'.format(class_))\n \n files = glob(os.path.join('../input/state-farm-distracted-driver-detection/imgs/train', 'c' + str(class_), '*.jpg'))\n \n for file in files:\n img = get_cv2_image(file, img_size , color_type)\n training_images.append(img)\n training_labels.append(class_) \n \n print(\"Data Loaded in {} Min\".format((time.time() - start_time)/60))\n return training_images, training_labels ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X, y = load_trainning_data( IMG_SIZE , COLOR_TYPE)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X[0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Convert Categorical data to numerical\ny = np_utils.to_categorical(y, CLASSES)\ny[0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# splitting train data to train and validation\nX_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size=0.15,shuffle=True, random_state=2021)\nprint(X_train[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# import gc\n# del X\n# gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# del y\n# gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# convert data to numpy array\nX_train = np.array(X_train, dtype=np.uint8).reshape(-1,IMG_SIZE,IMG_SIZE,COLOR_TYPE)\nX_valid = np.array(X_valid, dtype=np.uint8).reshape(-1,IMG_SIZE,IMG_SIZE,COLOR_TYPE)\n\nprint('Train shape :', X_train.shape)\nprint('Number of train samples : ',X_train.shape[0])\n\nprint('Validation shape :', X_valid.shape)\nprint('Number of Validation samples : ',X_valid.shape[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#shffle training data \n\nrandom.shuffle(X_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Normalization\n#X_train = np.divide(X_train,255)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# X_valid = np.divide(X_valid,255)\n# print(X_train[0],'/n/n')\n# print(X_valid[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# pickle_out = open(\"X_train.pickle\",\"wb\")\n# pickle.dump(X_train,pickle_out)\n# pickle_out.close()\n\n# pickle_out = open(\"y_train.pickle\",\"wb\")\n# pickle.dump(y_train,pickle_out)\n# pickle_out.close()\n\n# pickle_out = open(\"X_valid.pickle\",\"wb\")\n# pickle.dump(X_valid,pickle_out)\n# pickle_out.close()\n\n# pickle_out = open(\"y_valid.pickle\",\"wb\")\n# pickle.dump(y_valid,pickle_out)\n# pickle_out.close()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n## Load test data"},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_testing_data(test_size,img_size, color_type):\n\n files = sorted(glob(os.path.join('../input/state-farm-distracted-driver-detection/imgs/test', '*.jpg')))\n testing_image = []\n testing_image_id = []\n \n total = 0\n files_size = len(files)\n \n for file in tqdm(files):\n \n if total == test_size:\n break\n \n file_base = os.path.basename(file)\n img = get_cv2_image(file, img_size, color_type)\n testing_image.append(img)\n testing_image_id.append(file_base)\n \n total += 1\n return testing_image, testing_image_id\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data, test_ids = load_testing_data(TEST_SIZE, IMG_SIZE, COLOR_TYPE)\ntest_data = np.array(test_data, dtype=np.uint8)\ntest_data = test_data.reshape(-1,IMG_SIZE,IMG_SIZE,COLOR_TYPE)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Test shape:', test_data.shape)\nprint(test_data.shape[0], 'Test samples')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n## Visualize and preparing image dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"# mapping categotical\nCAT_MAP = {'c0': 'Safe driving', \n 'c1': 'Texting - right', \n 'c2': 'Talking on the phone - right', \n 'c3': 'Texting - left', \n 'c4': 'Talking on the phone - left', \n 'c5': 'Operating the radio', \n 'c6': 'Drinking', \n 'c7': 'Reaching behind', \n 'c8': 'Hair and makeup', \n 'c9': 'Talking to passenger'}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize = (12, 20))\n#image_count = 1\nDIR = '../input/state-farm-distracted-driver-detection/imgs/train/'\n\nfor directory in os.listdir(DIR):\n \n if directory[0] != '.':\n for i, file in enumerate(os.listdir(DIR + directory)):\n if i == 2:\n break\n else:\n #fig = plt.subplot(2, 2, image_count)\n #image_count += 1\n image = mpimg.imread(DIR + directory + '/' + file)\n plt.imshow(image)\n plt.title(CAT_MAP[directory])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n## Convert data to batches of tensors"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Preparing data augmentation\n'''Generate batches of tensor image data with real-time data augmentation.\n The data will be looped over (in batches).'''\n\ntrain_gen = ImageDataGenerator(rescale = 1.0/255,\n height_shift_range=0.5,\n width_shift_range = 0.5,\n rotation_range=30,\n validation_split = 0.2)\n\nvalid_gen = ImageDataGenerator(rescale=1.0/ 255, validation_split = 0.2)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'''Takes the dataframe and the path to a directory + generates batches.\n\n The generated batches contain => augmented/normalized data.'''\n\ntraining_generator = train_gen.flow_from_directory('../input/state-farm-distracted-driver-detection/imgs/train', \n target_size = (IMG_SIZE, IMG_SIZE), \n batch_size = BATCHES,\n shuffle=True,\n class_mode='categorical', subset=\"training\")\n\n\nvalidation_generator = valid_gen.flow_from_directory('../input/state-farm-distracted-driver-detection/imgs/train', \n target_size = (IMG_SIZE, IMG_SIZE), \n batch_size = BATCHES,\n shuffle=False,\n class_mode='categorical', subset=\"validation\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n# MODEL1\nTo define a model for training we'll follow these steps:\n1. Load in a pre-trained VGG16 model.\n2. \"Freeze\" all the parameters, so the net acts as a fixed feature extractor.\n4. Replace the last layer with a linear classifier of our own .\n\n> > Freezing simply means that the parameters in the pre-trained model will not change during training."},{"metadata":{},"cell_type":"markdown","source":"**Failed Model**"},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\n# def VGG_16(weights_path=None):\n \n# model = Sequential()\n# # Layer 1 (CONV)\n# model.add(ZeroPadding2D((1,1),input_shape=(COLOR_TYPE,IMG_SIZE,IMG_SIZE)))\n# model.add(Convolution2D(64, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(64, 3, 3, activation='relu'))\n# model.add(MaxPooling2D((2,2), strides=(2,2)))\n# # Layer 2 (CONV)\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(128, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(128, 3, 3, activation='relu'))\n# model.add(MaxPooling2D((2,2), strides=(2,2)))\n# # Layer 3 (CONV)\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(256, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(256, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(256, 3, 3, activation='relu'))\n# model.add(MaxPooling2D((2,2), strides=(2,2)))\n# # Layer 4 (CONV)\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(512, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(512, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(512, 3, 3, activation='relu'))\n# model.add(MaxPooling2D((2,2), strides=(2,2)))\n# # Layer 5 (CONV)\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(512, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(512, 3, 3, activation='relu'))\n# model.add(ZeroPadding2D((1,1)))\n# model.add(Convolution2D(512, 3, 3, activation='relu'))\n# model.add(MaxPooling2D((2,2), strides=(2,2)))\n# # Layer 6 (MPL)\n# model.add(Flatten())\n# model.add(Dense(4096, activation='relu'))\n# model.add(Dropout(0.5))\n# model.add(Dense(4096, activation='relu'))\n# model.add(Dropout(0.5))\n# model.add(Dense(10, activation='softmax'))\n\n# if weights_path:\n# model.load_weights(weights_path)\n\n# return model\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"''' This callback will stop the training when there is no improvement in\n the validation loss for three consecutive epochs.'''\n\n# Model weights are saved at the end of every epoch\nearly_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=1)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'''ModelCheckpoint callback is used in conjunction with training using model.fit() \n to save a model or weights (in a checkpoint file) at some interval, so the model \n or weights can be loaded later to continue the training from the state saved.'''\n\ncheckpoint = ModelCheckpoint(filepath='saved_models/weights_best_vgg16_model.hdf5', \n monitor='val_loss', mode='max',\n verbose=1, save_best_only=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_samples = 17943\nvalid_samples = 4481","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n## Define model 1\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"!rm -f input/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels.h5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\ndef VGG16_MODEL(img_rows=IMG_SIZE, img_cols=IMG_SIZE, color_type=3):\n # Remove fully connected layer and replace\n # with softmax for classifying 10 classes\n model_vgg16_1 = VGG16(weights=\"imagenet\", include_top=False)\n\n # Freeze all layers of the pre-trained model\n for layer in model_vgg16_1.layers:\n layer.trainable = False\n \n x = model_vgg16_1.output\n x = GlobalAveragePooling2D()(x)\n x = Dense(1024, activation='relu')(x)\n predictions = Dense(CLASSES, activation = 'softmax')(x)\n\n model = Model(input = model_vgg16_1.input, output = predictions)\n \n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Model 1 network...\")\nmodel_vgg16_1 = VGG16_MODEL(img_rows=IMG_SIZE, img_cols=IMG_SIZE)\n\nmodel_vgg16_1.summary()\n\nmodel_vgg16_1.compile(loss='categorical_crossentropy',\n optimizer='rmsprop',\n metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n## Train model 1"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nhistory = model_vgg16_1.fit_generator(training_generator,\n steps_per_epoch = train_samples // BATCHES,\n epochs = EPOCHS, \n callbacks=[early_stopping, checkpoint],\n verbose = 1,\n class_weight='auto',\n validation_data = validation_generator,\n validation_steps = valid_samples // BATCHES)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model_vgg16_1.load_weights('saved_models/weights_best_vgg16_model.hdf5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n## Visualize model 1 results\n**from backcalls**"},{"metadata":{},"cell_type":"markdown","source":" > **Plotting the accuracy and loss of the train and validation data to improve our model**"},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_train_history(history):\n # Summarize history for accuracy\n plt.figure(figsize = (8, 5))\n #plt.xticks(np.arange(0, 10))\n #plt.yticks(np.arange(0, 100))\n plt.plot(history.history['acc'])\n plt.plot(history.history['val_acc'])\n plt.title('Model Accuracy')\n plt.ylabel('Accuracy')\n plt.xlabel('Epoch')\n plt.legend(['train', 'validation'], loc='upper left')\n plt.show()\n\n # Summarize history for loss\n plt.figure(figsize = (8, 5))\n #plt.xticks(np.arange(0, 10))\n #plt.yticks(np.arange(0, 100))\n plt.plot(history.history['loss'])\n plt.plot(history.history['val_loss'])\n plt.title('Model loss')\n plt.ylabel('loss')\n plt.xlabel('epoch')\n plt.legend(['train', 'validation'], loc='lower left')\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_train_history(history)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Evaluate the performance of the new model\nscore = model_vgg16_1.evaluate_generator(validation_generator, valid_samples // BATCHES, verbose = 1)\nprint(\"validation Score:\", score[0])\nprint(\"validation Accuracy:\", score[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n# MODEL 2"},{"metadata":{},"cell_type":"markdown","source":"\n## Define model 2\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"!rm -f input/vggweights/vgg16_weights.h5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\ndef VGG16_MODEL(img_rows=IMG_SIZE, img_cols=IMG_SIZE, color_type=3):\n # Remove fully connected layer and replace\n # with softmax for classifying 10 classes\n vgg16_model_2 = VGG16(weights=\"imagenet\", include_top=False)\n\n # Freeze all layers of the pre-trained model\n for layer in vgg16_model_2.layers:\n layer.trainable = False\n \n x = vgg16_model_2.output\n x = GlobalAveragePooling2D()(x)\n x = Dense(1024, activation='relu')(x)\n predictions = Dense(CLASSES, activation = 'softmax')(x)\n\n model = Model(input = vgg16_model_2.input, output = predictions)\n \n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Loading network...\")\nmodel_vgg16_2 = VGG16_MODEL(img_rows=IMG_SIZE, img_cols=IMG_SIZE)\n\nmodel_vgg16_2.summary()\n\nmodel_vgg16_2.compile(loss='categorical_crossentropy',\n optimizer='rmsprop',\n metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n## Train model 2\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nhistory = model_vgg16_2.fit_generator(training_generator,\n steps_per_epoch = train_samples // BATCHES,\n epochs = EPOCHS, \n callbacks=[early_stopping, checkpoint],\n verbose = 1,\n class_weight='auto',\n validation_data = validation_generator,\n validation_steps = valid_samples // BATCHES)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n# Visualize model 2 results\n**from backcalls**"},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_train_history(history):\n # Summarize history for accuracy\n plt.figure(figsize = (8, 5))\n #plt.xticks(np.arange(0, 10))\n #plt.yticks(np.arange(0, 100))\n plt.plot(history.history['acc'])\n plt.plot(history.history['val_acc'])\n plt.title('Model Accuracy')\n plt.ylabel('Accuracy')\n plt.xlabel('Epoch')\n plt.legend(['train', 'validation'], loc='upper left')\n plt.show()\n\n # Summarize history for loss\n plt.figure(figsize = (8, 5))\n #plt.xticks(np.arange(0, 10))\n #plt.yticks(np.arange(0, 100))\n plt.plot(history.history['loss'])\n plt.plot(history.history['val_loss'])\n plt.title('Model loss')\n plt.ylabel('loss')\n plt.xlabel('epoch')\n plt.legend(['train', 'validation'], loc='lower left')\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_train_history(history)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Evaluate the performance of the new model\nscore = model_vgg16_2.evaluate_generator(validation_generator, valid_samples // BATCHES, verbose = 1)\nprint(\"Validation Score:\", score[0])\nprint(\"Validation Accuracy:\", score[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n# TESTING FINAL MODEL "},{"metadata":{"trusted":true},"cell_type":"code","source":"def prediction():\n for i in np.arange(10):\n img_brute = test_data[i]\n\n im = cv2.resize(cv2.cvtColor(img_brute, cv2.COLOR_BGR2RGB), (IMG_SIZE,IMG_SIZE)).astype(np.float32) / 255.0\n im = np.expand_dims(im, axis =0)\n\n img_display = cv2.resize(img_brute,(IMG_SIZE,IMG_SIZE))\n plt.imshow(img_display, cmap='gray')\n\n y_preds = model_vgg16_1.predict(im, batch_size=BATCHES, verbose=1)\n print(y_preds)\n y_prediction = np.argmax(y_preds)\n print('Y Prediction: {}'.format(y_prediction))\n print('Predicted as: {}'.format(CAT_MAP.get('c{}'.format(y_prediction))))\n\n plt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"prediction()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}