According to the World Health Organization (WHO) survey, 1.3 million people worldwide die in traffic accidents each year, making them the eighth leading cause of death and an additional 20-50 millions are injured/ disabled. As per the report of National Crime Research Bureau (NCRB), Govt. of India, Indian roads account for the highest fatalities in the world. There has been a continuous increase in road crash deaths in India since 2006. The report also states that the total number of deaths have risen to 1.46 lakhs in 2015 and driver error is the most common cause behind these traffic accidents.The number of accidents because of distracted driver has been increasing since few years.
\n\nNational Highway Traffic Safety Administrator of United States (NHTSA) reports deaths of 3477 people and injuries to 391000 people in motor vehicle crashes because of distracted drivers in 2015. In the United States, everyday approximately 9 people are killed and more than 1,000 are injured in road crashes that are reported to involve a distracted driver. NTHSA describes distracted driving as “any activity that diverts attention of the driver from the task of driving” which can be classified into Manual, Visual or Cognitive distraction. As per the definitions of Center for Disease Control and Prevention (CDC), cognitive distraction is basically “driver’s mind is off the driving”.
\n\nn other words, even though the driver is in safe driving posture, he is mentally distracted from the task of driving. He might be lost in thoughts, daydreaming etc. Distraction because of inattention, sleepiness, fatigue or drowsiness falls into visual distraction class where “drivers’s eyes are off the road”. Manual distractions are concerned with various activities where “driver’s hands are off the wheel”. Such distractions include talking or texting using mobile phones, eating and drinking, talking to passengers in the vehicle, adjusting the radio, makeup etc.Nowadays, Advanced Driver Assistance Systems (ADAS) are being developed to prevent accidents by offering technologies that alert the driver to potential problems and to keep the car’s driver and occupants safe if an accident does occur. But even today’s latest autonomous vehicles require the driver to be attentive and ready to take the control of the wheel back in case of emergency.
\n\nTesla autopilot’s crash with the white truck-trailor in Williston, Florida in May 2016 was the first fatal crash in testing of autonomous vehicle. Recently in March 2018, Uber’s self driving car with an emergency backup driver behind the wheel struck and killed a pedestrian in Arizona. In both of these fatalities, the safety driver could have avoided the crashes but evidences reveal that he was clearly distracted. This makes detection of distracted driver an essential part of the self driving cars as well. We believe that distracted driver detection is utmost important for further preventive measures. If the vehicle could detect such distractions and then warn the driver against it, number of road crashes can be reduced. In this projects, we focus on detecting manual distractions where driver is engaged in other activities than safe driving and also identify the cause of distraction. We present a Convolutional Neural Network based approach for this problem. We also attempt to reduce the computational complexity and memory requirement while maintaining good accuracy which is desirable in real time applications.
","metadata":{}},{"cell_type":"markdown","source":"\n## Problem Statment \n\nGiven a dataset of 2D dashboard camera images, an algorithm needs to be developed to classify each driver's behaviour and determine if they are driving attentively, wearing their seatbelt, or taking a selfie with their friends in the backseat etc..? This can then be used to automatically detect drivers engaging in distracted behaviours from dashboard cameras.\n\nFollowing are needed tasks for the development of the algorithm:\n1. Download and preprocess the driver images.\n1. Build and train the model to classify the driver images.\n1. Test the model and further improve the model using different techniques.","metadata":{}},{"cell_type":"markdown","source":"\n\n## Tools\n\nThe project utilizes the following dependencies:\n\n- Python 3.5: Tensorflow, Keras, Numpy, Scipy, seaborn, Matplotlib.\n- NVIDIA Geforce GTX1060 GPU, CUDA, CuDNN.","metadata":{}},{"cell_type":"markdown","source":"\n\n## Dataset\nThe provided data set has driver images, each taken in a car with a driver doing something in the car (texting, eating, talking on the phone, makeup, reaching behind, etc). This dataset is obtained from [Kaggle](https://www.kaggle.com/c/state-farm-distracted-driver-detection/data).(State Farm Distracted Driver Detection competition).\n\nFollowing are the file descriptions and URL’s from which the data can be obtained :\n\n* imgs.zip - zipped folder of all (train/test) images\n* sample_submission.csv - a sample submission file in the correct format\n* driver_imgs_list.csv - a list of training images, their subject (driver) id, and class id","metadata":{}},{"cell_type":"markdown","source":"\n## The 10 classes to predict\n\n- c0: safe driving\n- c1: texting - right\n- c2: talking on the phone - right\n- c3: texting - left\n- c4: talking on the phone - left\n- c5: operating the radio\n- c6: drinking\n- c7: reaching behind\n- c8: hair and makeup\n- c9: talking to passenger\n\nThere are 22423 train images and 79725 test images.","metadata":{}},{"cell_type":"markdown","source":"\n\n## Implementation\n\nThis project aims to develop a machine learning system that can detect and classify different distracted states of car drivers. The main approach is to apply deep convolutional neural networks (CNNs). We will explore and experiment various CNN architectures, leveraged pre-trained networks (learning transfer).\n\n#### Transfer Learning\n\nMost of the time you won't want to train a whole convolutional network yourself. Modern ConvNets training on huge datasets like ImageNet take weeks on multiple GPUs. \n\nInstead, most people use a pretrained network either as a fixed feature extractor, or as an initial network to fine tune. \n\nIn this notebook, you'll be using [VGGNet] trained on the [ImageNet dataset] as a feature extractor. Below is a diagram of the VGGNet architecture, with a series of convolutional and maxpooling layers, then three fully-connected layers at the end that classify the 1000 classes found in the ImageNet database.\n\n![](\"../input/vgg-pic/vgg_16_architecture.png\")
\n\n#!img[VGG Architecture]\nVGGNet is great because it's simple and has great performance, coming in second in the ImageNet competition. The idea here is that we keep all the convolutional layers, but **replace the final fully-connected layer** with our own classifier. This way we can use VGGNet as a *fixed feature extractor* for our images then easily train a simple classifier on top of that. \n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"\n\n# PREPARING","metadata":{}},{"cell_type":"markdown","source":"\n\n## Import libraries that will be used","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport pickle\nimport numpy as np\nimport seaborn as sns\nfrom sklearn.datasets import load_files\nfrom keras.utils import np_utils\nimport matplotlib.pyplot as plt\nfrom keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D\nfrom keras.layers import Dropout, Flatten, Dense\nfrom keras.models import Sequential\nfrom keras.utils.vis_utils import plot_model\nfrom keras.callbacks import ModelCheckpoint\nfrom keras.utils import to_categorical\nfrom sklearn.metrics import confusion_matrix\nfrom keras.preprocessing import image \nfrom tqdm import tqdm\n\nimport seaborn as sns\nfrom sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:17.615018Z","iopub.execute_input":"2021-07-08T14:53:17.615479Z","iopub.status.idle":"2021-07-08T14:53:17.623515Z","shell.execute_reply.started":"2021-07-08T14:53:17.615449Z","shell.execute_reply":"2021-07-08T14:53:17.622407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Pretty display for notebooks\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:17.625779Z","iopub.execute_input":"2021-07-08T14:53:17.626219Z","iopub.status.idle":"2021-07-08T14:53:17.636496Z","shell.execute_reply.started":"2021-07-08T14:53:17.626175Z","shell.execute_reply":"2021-07-08T14:53:17.635289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Defining the train,test and model directories\nWe will create the directories for train,test and model training paths if not present","metadata":{}},{"cell_type":"code","source":"DATA_DIR = \"../input/state-farm-distracted-driver-detection/imgs\"\nTEST_DIR = os.path.join(DATA_DIR,\"test\")\nTRAIN_DIR = os.path.join(DATA_DIR,\"train\")\nRANDOM_DIR = '../input/driverdistractionimages'\n\nMODEL_PATH = os.path.join(os.getcwd(),\"model\",\"trained\")\nPICKLE_DIR = os.path.join(os.getcwd(),\"pickle_files\")\nCSV_DIR = os.path.join(os.getcwd(),\"csv_files\")","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:55:35.670649Z","iopub.execute_input":"2021-07-08T15:55:35.671000Z","iopub.status.idle":"2021-07-08T15:55:35.676813Z","shell.execute_reply.started":"2021-07-08T15:55:35.670965Z","shell.execute_reply":"2021-07-08T15:55:35.675567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if not os.path.exists(TRAIN_DIR):\n print(\"Training data does not exists\")\nif not os.path.exists(TEST_DIR):\n print(\"Testing data does not exists\")\nif not os.path.exists(RANDOM_DIR):\n print(\"Random data does not exists\")\n############################################ \nif not os.path.exists(MODEL_PATH):\n print(\"Model path does not exists\")\n os.makedirs(MODEL_PATH)\n print(\"Model path created\")\n############################################\nif not os.path.exists(PICKLE_DIR):\n os.makedirs(PICKLE_DIR)\n############################################\nif not os.path.exists(CSV_DIR):\n os.makedirs(CSV_DIR)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:55:38.569564Z","iopub.execute_input":"2021-07-08T15:55:38.569874Z","iopub.status.idle":"2021-07-08T15:55:38.582966Z","shell.execute_reply.started":"2021-07-08T15:55:38.569844Z","shell.execute_reply":"2021-07-08T15:55:38.582089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Data Preparation","metadata":{}},{"cell_type":"markdown","source":"We will create a csv file having the location of the files present for training and test images and their associated class if present so that it is easily traceable.","metadata":{}},{"cell_type":"code","source":"def create_csv(DATA_DIR,filename):\n class_names = os.listdir(DATA_DIR) # train , test\n data = list()\n \n if(os.path.isdir(os.path.join(DATA_DIR,class_names[0]))): # if in train file\n for class_name in class_names: # which class in the train file\n file_names = os.listdir(os.path.join(DATA_DIR,class_name)) # images in this class (c0, c1, ...., c9)\n for file in file_names: # create a dic of the image path and its class name(c0, c1, ...., c9)\n data.append({\n \"Filename\":os.path.join(DATA_DIR,class_name,file),\n \"ClassName\":class_name\n })\n else: # if in test file\n class_name = \"test\"\n file_names = os.listdir(DATA_DIR) # images in the test file\n for file in file_names: # create a dic of the image path and its class name (test)\n data.append(({\n \"FileName\":os.path.join(DATA_DIR,file),\n \"ClassName\":class_name\n }))\n \n data = pd.DataFrame(data) # create a DataFrame using Pandas\n data.to_csv(os.path.join(os.getcwd(),\"csv_files\",filename),index=False) # convert the DataFrame to CSV file","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:58:55.094519Z","iopub.execute_input":"2021-07-08T15:58:55.094843Z","iopub.status.idle":"2021-07-08T15:58:55.102599Z","shell.execute_reply.started":"2021-07-08T15:58:55.094814Z","shell.execute_reply":"2021-07-08T15:58:55.101491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"create_csv(TRAIN_DIR,\"train.csv\")\ncreate_csv(TEST_DIR,\"test.csv\")","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:59:00.210985Z","iopub.execute_input":"2021-07-08T15:59:00.211295Z","iopub.status.idle":"2021-07-08T15:59:01.577522Z","shell.execute_reply.started":"2021-07-08T15:59:00.211265Z","shell.execute_reply":"2021-07-08T15:59:01.576591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_train = pd.read_csv(os.path.join(os.getcwd(),\"csv_files\",\"train.csv\"))\ndata_train.info()","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:25.934236Z","iopub.execute_input":"2021-07-08T14:53:25.934585Z","iopub.status.idle":"2021-07-08T14:53:25.983070Z","shell.execute_reply.started":"2021-07-08T14:53:25.934550Z","shell.execute_reply":"2021-07-08T14:53:25.982138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_train['ClassName'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:25.984785Z","iopub.execute_input":"2021-07-08T14:53:25.985138Z","iopub.status.idle":"2021-07-08T14:53:26.000864Z","shell.execute_reply.started":"2021-07-08T14:53:25.985105Z","shell.execute_reply":"2021-07-08T14:53:25.999878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_train.describe()","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2021-07-08T14:53:26.004081Z","iopub.execute_input":"2021-07-08T14:53:26.004392Z","iopub.status.idle":"2021-07-08T14:53:26.056540Z","shell.execute_reply.started":"2021-07-08T14:53:26.004361Z","shell.execute_reply":"2021-07-08T14:53:26.055722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.set()\n\ncount_img_in_each_class= data_train['ClassName'].value_counts(sort=False)\ncategories = data_train['ClassName'].value_counts(sort=False).index.tolist()\n\ny = np.array(count_img_in_each_class)\nwidth = 1/1.5\nN = len(y)\nx = range(N)\n\nfig = plt.figure(figsize=(20,15))\nay = fig.add_subplot(211)\n\nplt.xticks(x, categories, size=15)\nplt.yticks(size=15)\n\nay.bar(x, y, width, color=\"#169DE3\")\n\nplt.title('Bar Chart',size=25)\nplt.xlabel('classname',size=15)\nplt.ylabel('Count',size=15)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.059503Z","iopub.execute_input":"2021-07-08T14:53:26.059816Z","iopub.status.idle":"2021-07-08T14:53:26.299049Z","shell.execute_reply.started":"2021-07-08T14:53:26.059786Z","shell.execute_reply":"2021-07-08T14:53:26.298237Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Observation\n1. There are total 22424 training samples\n2. There are total 79726 testing samples\n3. The training dataset is equally balanced to a great extent and hence we need not do any downsampling of the data","metadata":{}},{"cell_type":"markdown","source":"\n## Converting into numerical values","metadata":{}},{"cell_type":"code","source":"labels_list = list(set(data_train['ClassName'].values.tolist()))\ndata_train","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.300529Z","iopub.execute_input":"2021-07-08T14:53:26.300871Z","iopub.status.idle":"2021-07-08T14:53:26.313311Z","shell.execute_reply.started":"2021-07-08T14:53:26.300834Z","shell.execute_reply":"2021-07-08T14:53:26.312236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import re\n\nfor label in labels_list : \n\n for id in re.findall(r'\\d+',label) : \n data_train['ClassName'].replace(label,id,inplace=True)\n\ndata_train","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.314901Z","iopub.execute_input":"2021-07-08T14:53:26.315263Z","iopub.status.idle":"2021-07-08T14:53:26.344886Z","shell.execute_reply.started":"2021-07-08T14:53:26.315231Z","shell.execute_reply":"2021-07-08T14:53:26.344249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Use pickle to save a dictionary","metadata":{}},{"cell_type":"code","source":"with open(os.path.join(os.getcwd(),\"pickle_files\",\"labels_list.pkl\"),\"wb\") as handle:\n pickle.dump(id,handle)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.346126Z","iopub.execute_input":"2021-07-08T14:53:26.346461Z","iopub.status.idle":"2021-07-08T14:53:26.351318Z","shell.execute_reply.started":"2021-07-08T14:53:26.346428Z","shell.execute_reply":"2021-07-08T14:53:26.350244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Converting into categorical values","metadata":{}},{"cell_type":"code","source":"labels = to_categorical(data_train['ClassName'])\nprint(labels.shape)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.352601Z","iopub.execute_input":"2021-07-08T14:53:26.353251Z","iopub.status.idle":"2021-07-08T14:53:26.363610Z","shell.execute_reply.started":"2021-07-08T14:53:26.353215Z","shell.execute_reply":"2021-07-08T14:53:26.362870Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Splitting datasets","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nxtrain,xvalid,ytrain,yvalid = train_test_split(data_train.iloc[:,0],labels,test_size = 0.2,random_state=2021)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.364987Z","iopub.execute_input":"2021-07-08T14:53:26.365833Z","iopub.status.idle":"2021-07-08T14:53:26.384800Z","shell.execute_reply.started":"2021-07-08T14:53:26.365797Z","shell.execute_reply":"2021-07-08T14:53:26.384203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xtrain.shape","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.386213Z","iopub.execute_input":"2021-07-08T14:53:26.386544Z","iopub.status.idle":"2021-07-08T14:53:26.392154Z","shell.execute_reply.started":"2021-07-08T14:53:26.386511Z","shell.execute_reply":"2021-07-08T14:53:26.391265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Converting into 224*224 images \n**NOTE:**\nYou can substitute 64,64 to 224,224 for better results only if ram is >32gb","metadata":{}},{"cell_type":"code","source":"from keras.preprocessing.image import img_to_array, array_to_img\nfrom keras.preprocessing import image\n\ndef path_to_tensor(img_path):\n # loads RGB image as PIL.Image.Image type\n img = image.load_img(img_path,color_mode = 'grayscale', target_size=(224, 224))\n # convert PIL.Image.Image type to 3D tensor with shape (1, 224, 224, 1)\n x = image.img_to_array(img)\n # convert 3D tensor to 4D tensor with shape (1, 224, 224, 1) and return 4D tensor\n return np.expand_dims(x, axis=0)\n\ndef paths_to_tensor(img_paths):\n list_of_tensors = [path_to_tensor(img_path) for img_path in tqdm(img_paths)]\n return np.vstack(list_of_tensors)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.393990Z","iopub.execute_input":"2021-07-08T14:53:26.394381Z","iopub.status.idle":"2021-07-08T14:53:26.402885Z","shell.execute_reply.started":"2021-07-08T14:53:26.394344Z","shell.execute_reply":"2021-07-08T14:53:26.401918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Load training and validation images","metadata":{}},{"cell_type":"code","source":"\nfrom PIL import ImageFile, Image \nImageFile.LOAD_TRUNCATED_IMAGES = True \n\n# pre-process the data for Keras\ntrain_tensors = paths_to_tensor(xtrain).astype('float32')/255 - 0.5","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:53:26.404649Z","iopub.execute_input":"2021-07-08T14:53:26.405043Z","iopub.status.idle":"2021-07-08T14:57:39.033107Z","shell.execute_reply.started":"2021-07-08T14:53:26.405010Z","shell.execute_reply":"2021-07-08T14:57:39.032108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"valid_tensors = paths_to_tensor(xvalid).astype('float32')/255 - 0.5","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:57:39.034384Z","iopub.execute_input":"2021-07-08T14:57:39.034713Z","iopub.status.idle":"2021-07-08T14:58:39.761874Z","shell.execute_reply.started":"2021-07-08T14:57:39.034678Z","shell.execute_reply":"2021-07-08T14:58:39.761055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# MODEL","metadata":{}},{"cell_type":"markdown","source":"\n\n## Defining the Model","metadata":{}},{"cell_type":"code","source":"model = Sequential()\n\nmodel.add(Conv2D(filters=64, kernel_size=2, padding='same', activation='relu', input_shape=(224,224,1), kernel_initializer='glorot_normal'))\nmodel.add(MaxPooling2D(pool_size=2))\nmodel.add(Conv2D(filters=128, kernel_size=2, padding='same', activation='relu', kernel_initializer='glorot_normal'))\nmodel.add(MaxPooling2D(pool_size=2))\nmodel.add(Conv2D(filters=256, kernel_size=2, padding='same', activation='relu', kernel_initializer='glorot_normal'))\nmodel.add(MaxPooling2D(pool_size=2))\nmodel.add(Conv2D(filters=512, kernel_size=2, padding='same', activation='relu', kernel_initializer='glorot_normal'))\nmodel.add(MaxPooling2D(pool_size=2))\nmodel.add(Dropout(0.5))\nmodel.add(Flatten())\nmodel.add(Dense(500, activation='relu', kernel_initializer='glorot_normal'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10, activation='softmax', kernel_initializer='glorot_normal'))\n\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:58:39.763331Z","iopub.execute_input":"2021-07-08T14:58:39.763659Z","iopub.status.idle":"2021-07-08T14:58:41.907954Z","shell.execute_reply.started":"2021-07-08T14:58:39.763622Z","shell.execute_reply":"2021-07-08T14:58:41.906992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Plotting the Model architecture","metadata":{}},{"cell_type":"code","source":"plot_model(model,to_file=os.path.join(MODEL_PATH,\"model_distracted_driver.png\"),show_shapes=True,show_layer_names=True)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:58:41.909240Z","iopub.execute_input":"2021-07-08T14:58:41.909545Z","iopub.status.idle":"2021-07-08T14:58:42.706965Z","shell.execute_reply.started":"2021-07-08T14:58:41.909512Z","shell.execute_reply":"2021-07-08T14:58:42.705897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Compiling the Model","metadata":{}},{"cell_type":"code","source":"model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:58:42.708316Z","iopub.execute_input":"2021-07-08T14:58:42.708654Z","iopub.status.idle":"2021-07-08T14:58:42.727320Z","shell.execute_reply.started":"2021-07-08T14:58:42.708612Z","shell.execute_reply":"2021-07-08T14:58:42.726470Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Callbacks","metadata":{}},{"cell_type":"code","source":"filepath = os.path.join(MODEL_PATH,\"CNN-model-{epoch:02d}-{val_accuracy:.2f}.hdf5\")\ncheckpoint = ModelCheckpoint(filepath, monitor='val_accuracy', verbose=1, save_best_only=True, mode='max',period=1)\ncallbacks_list = [checkpoint]","metadata":{"execution":{"iopub.status.busy":"2021-07-08T14:58:42.729869Z","iopub.execute_input":"2021-07-08T14:58:42.732081Z","iopub.status.idle":"2021-07-08T14:58:42.738520Z","shell.execute_reply.started":"2021-07-08T14:58:42.732043Z","shell.execute_reply":"2021-07-08T14:58:42.737564Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Training the Model","metadata":{}},{"cell_type":"code","source":"model_history = model.fit(train_tensors,ytrain,validation_data = (valid_tensors, yvalid),epochs=25, batch_size=40, shuffle=True,callbacks=callbacks_list)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2021-07-08T14:58:42.739910Z","iopub.execute_input":"2021-07-08T14:58:42.740352Z","iopub.status.idle":"2021-07-08T15:18:05.657283Z","shell.execute_reply.started":"2021-07-08T14:58:42.740318Z","shell.execute_reply":"2021-07-08T15:18:05.655856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Model_path =os.path.join(MODEL_PATH,\"the-complete-model.hdf5\")\nmodel.save(Model_path)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:05.659091Z","iopub.execute_input":"2021-07-08T15:18:05.659450Z","iopub.status.idle":"2021-07-08T15:18:06.178454Z","shell.execute_reply.started":"2021-07-08T15:18:05.659415Z","shell.execute_reply":"2021-07-08T15:18:06.177658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # SAVE THE SESSION\n# import keras\n# sess_path = os.path.join(MODEL_PATH,'session.hdf5')\n# saver = tf.train.Saver()\n# sess = keras.backend.get_session()\n# saver.save(sess, sess_path)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:06.179982Z","iopub.execute_input":"2021-07-08T15:18:06.180377Z","iopub.status.idle":"2021-07-08T15:18:06.184400Z","shell.execute_reply.started":"2021-07-08T15:18:06.180339Z","shell.execute_reply":"2021-07-08T15:18:06.183537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Visulaizing the Model history","metadata":{}},{"cell_type":"code","source":"def plot_train_history(history = model_history):\n # Summarize history for accuracy\n plt.figure(figsize = (8, 5))\n #plt.xticks(np.arange(0, 10))\n #plt.yticks(np.arange(0, 100))\n plt.plot(history.history['accuracy'])\n plt.plot(history.history['val_accuracy'])\n plt.title('Model Accuracy')\n plt.ylabel('Accuracy')\n plt.xlabel('Epoch')\n plt.legend(['train', 'validation'], loc='lower right')\n plt.show()\n\n # Summarize history for loss\n plt.figure(figsize = (8, 5))\n #plt.xticks(np.arange(0, 10))\n #plt.yticks(np.arange(0, 100))\n plt.plot(history.history['loss'])\n plt.plot(history.history['val_loss'])\n plt.title('Model loss')\n plt.ylabel('loss')\n plt.xlabel('epoch')\n plt.legend(['train', 'validation'], loc='upper right')\n plt.show()\n\nplot_train_history(model_history)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:06.185620Z","iopub.execute_input":"2021-07-08T15:18:06.186112Z","iopub.status.idle":"2021-07-08T15:18:06.571868Z","shell.execute_reply.started":"2021-07-08T15:18:06.186077Z","shell.execute_reply":"2021-07-08T15:18:06.571035Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# ANALYZING THE MODEL","metadata":{}},{"cell_type":"markdown","source":"\n\n## Finding the Confusion matrix","metadata":{}},{"cell_type":"code","source":"def print_confusion_matrix(confusion_matrix, class_names, figsize = (10,7), fontsize=14):\n \n df_cm = pd.DataFrame(\n confusion_matrix, index=class_names, columns=class_names, \n ) \n fig = plt.figure(figsize=figsize)\n \n try:\n heatmap = sns.heatmap(df_cm, annot=True, fmt=\"d\") #Plot rectangular data as a color-encoded matrix.\n except ValueError:\n raise ValueError(\"Confusion matrix values must be integers.\")\n \n heatmap.yaxis.set_ticklabels(heatmap.yaxis.get_ticklabels(), rotation=0, ha='right', fontsize=fontsize)\n heatmap.xaxis.set_ticklabels(heatmap.xaxis.get_ticklabels(), rotation=45, ha='right', fontsize=fontsize)\n plt.ylabel('True label')\n plt.xlabel('Predicted label')\n fig.savefig(os.path.join(MODEL_PATH,\"confusion_matrix.png\"))\n \n return fig","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:06.573467Z","iopub.execute_input":"2021-07-08T15:18:06.573798Z","iopub.status.idle":"2021-07-08T15:18:06.581064Z","shell.execute_reply.started":"2021-07-08T15:18:06.573761Z","shell.execute_reply":"2021-07-08T15:18:06.580242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def print_heatmap(n_labels, n_predictions, class_names):\n labels = n_labels \n predictions = n_predictions \n\n matrix = confusion_matrix(labels.argmax(axis=1),predictions.argmax(axis=1))\n row_sum = np.sum(matrix, axis = 1)\n w, h = matrix.shape\n\n c_m = np.zeros((w, h))\n\n for i in range(h):\n c_m[i] = matrix[i] * 100 / row_sum[i]\n c = c_m.astype(dtype = np.uint8)\n \n heatmap = print_confusion_matrix(c, class_names, figsize=(18,10), fontsize=20)\n","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:06.582376Z","iopub.execute_input":"2021-07-08T15:18:06.582730Z","iopub.status.idle":"2021-07-08T15:18:06.595085Z","shell.execute_reply.started":"2021-07-08T15:18:06.582696Z","shell.execute_reply":"2021-07-08T15:18:06.594174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# class_names = list()\n# for name,idx in labels_list:\n# class_names.append(name)\n# print(class_names)\n\nypred_valid = model.predict(valid_tensors)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:06.597657Z","iopub.execute_input":"2021-07-08T15:18:06.598059Z","iopub.status.idle":"2021-07-08T15:18:10.486706Z","shell.execute_reply.started":"2021-07-08T15:18:06.598002Z","shell.execute_reply":"2021-07-08T15:18:10.485820Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print_heatmap(yvalid,ypred_valid,labels_list)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2021-07-08T15:18:10.488152Z","iopub.execute_input":"2021-07-08T15:18:10.488498Z","iopub.status.idle":"2021-07-08T15:18:11.484242Z","shell.execute_reply.started":"2021-07-08T15:18:10.488463Z","shell.execute_reply":"2021-07-08T15:18:11.483330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## Finding the Precision ,Recall and F1 score","metadata":{}},{"cell_type":"code","source":"ypred_class = np.argmax(ypred_valid,axis=1)\nyvalid = np.argmax(yvalid,axis=1)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:11.485851Z","iopub.execute_input":"2021-07-08T15:18:11.486218Z","iopub.status.idle":"2021-07-08T15:18:11.490997Z","shell.execute_reply.started":"2021-07-08T15:18:11.486170Z","shell.execute_reply":"2021-07-08T15:18:11.489985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"accuracy = accuracy_score(yvalid,ypred_class)\nprint('Accuracy: %f' % accuracy)\n# precision tp / (tp + fp)\nprecision = precision_score(yvalid, ypred_class,average='weighted')\nprint('Precision: %f' % precision)\n# recall: tp / (tp + fn)\nrecall = recall_score(yvalid,ypred_class,average='weighted')\nprint('Recall: %f' % recall)\n# f1: 2 tp / (2 tp + fp + fn)\nf1 = f1_score(yvalid,ypred_class,average='weighted')\nprint('F1 score: %f' % f1)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:11.492743Z","iopub.execute_input":"2021-07-08T15:18:11.493126Z","iopub.status.idle":"2021-07-08T15:18:11.513586Z","shell.execute_reply.started":"2021-07-08T15:18:11.493088Z","shell.execute_reply":"2021-07-08T15:18:11.512651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# Testing The Un-labedled Dataset","metadata":{}},{"cell_type":"code","source":"data_test = pd.read_csv(os.path.join(os.getcwd(),\"csv_files\",\"test.csv\"))\nprint('Data Test Shape (n_ imgs, classes) = ',data_test.shape, '\\n\\n')\ndata_test.head()","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:11.516533Z","iopub.execute_input":"2021-07-08T15:18:11.516801Z","iopub.status.idle":"2021-07-08T15:18:11.642044Z","shell.execute_reply.started":"2021-07-08T15:18:11.516777Z","shell.execute_reply":"2021-07-08T15:18:11.641158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xtest = data_test['FileName'].tolist()\ntest_tensors = paths_to_tensor(xtest[:20]).astype('float32')/255 - 0.5","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:11.643266Z","iopub.execute_input":"2021-07-08T15:18:11.643591Z","iopub.status.idle":"2021-07-08T15:18:11.972820Z","shell.execute_reply.started":"2021-07-08T15:18:11.643556Z","shell.execute_reply":"2021-07-08T15:18:11.971926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_tensors.shape","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:11.977079Z","iopub.execute_input":"2021-07-08T15:18:11.977331Z","iopub.status.idle":"2021-07-08T15:18:11.982544Z","shell.execute_reply.started":"2021-07-08T15:18:11.977305Z","shell.execute_reply":"2021-07-08T15:18:11.981692Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.set_style(\"whitegrid\", {'axes.grid' : False})\n\nimport matplotlib.image as mpimg\n# mapping categotical\n\nclasses = [ 'safe driving', ' texting - right', 'talking on the phone - right',\n 'texting - left', 'talking on the phone - left', 'operating the radio',\n 'drinking', 'reaching behind', 'hair and makeup', 'talking to passenger']\n\n\nPredictions = model.predict(test_tensors)\nPredicted_Labels = np.argmax(Predictions, axis = 1)\nfig = plt.figure(figsize = (20,40))\n\nfor i, LAB in enumerate(Predicted_Labels):\n \n fig.add_subplot(10, 5, i+1)\n plt.imshow(test_tensors[i][:][:])\n plt.axis('off')\n plt.title('Predicted Label: \\n C{}->{}'.format(LAB , classes[LAB]))\n ","metadata":{"execution":{"iopub.status.busy":"2021-07-08T16:09:42.972479Z","iopub.execute_input":"2021-07-08T16:09:42.972802Z","iopub.status.idle":"2021-07-08T16:09:44.830980Z","shell.execute_reply.started":"2021-07-08T16:09:42.972771Z","shell.execute_reply":"2021-07-08T16:09:44.822812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nfrom keras.models import load_model\nmodel_saved = load_model(Model_path)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:18:13.948890Z","iopub.execute_input":"2021-07-08T15:18:13.949366Z","iopub.status.idle":"2021-07-08T15:18:14.344965Z","shell.execute_reply.started":"2021-07-08T15:18:13.949328Z","shell.execute_reply":"2021-07-08T15:18:14.344182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# create_csv(RANDOM_DIR,\"random.csv\")\n\nclass_name = \"random\"\nfile_names = os.listdir(RANDOM_DIR)\ndata = list() \n\nfor file in file_names: # create a dic of the image path and its class name (test)\n data.append(({\n \"FileName\":os.path.join(RANDOM_DIR,file),\n \"ClassName\":class_name\n }))\n\ndata = pd.DataFrame(data) # create a DataFrame using Pandas\ndata.to_csv(os.path.join(os.getcwd(),\"csv_files\",'random.csv'),index=False) # convert the DataFrame to CSV file","metadata":{"execution":{"iopub.status.busy":"2021-07-08T16:09:18.018258Z","iopub.execute_input":"2021-07-08T16:09:18.018575Z","iopub.status.idle":"2021-07-08T16:09:18.028502Z","shell.execute_reply.started":"2021-07-08T16:09:18.018546Z","shell.execute_reply":"2021-07-08T16:09:18.027665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_random = pd.read_csv(os.path.join(os.getcwd(),\"csv_files\",\"random.csv\"))\nprint('Data Test Shape (n_ imgs, classes) = ',data_random.shape, '\\n\\n')\ndata_random.head(5)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T16:09:20.416832Z","iopub.execute_input":"2021-07-08T16:09:20.417176Z","iopub.status.idle":"2021-07-08T16:09:20.432363Z","shell.execute_reply.started":"2021-07-08T16:09:20.417147Z","shell.execute_reply":"2021-07-08T16:09:20.431160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xrandom = data_random['FileName'].tolist()\nrandom_tensors = paths_to_tensor(xrandom).astype('float32')/255 - 0.5","metadata":{"execution":{"iopub.status.busy":"2021-07-08T16:09:24.040107Z","iopub.execute_input":"2021-07-08T16:09:24.040434Z","iopub.status.idle":"2021-07-08T16:09:24.714970Z","shell.execute_reply.started":"2021-07-08T16:09:24.040405Z","shell.execute_reply":"2021-07-08T16:09:24.713185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import glob\n\n# images_paths = list(glob.glob(os.path.join('../input/driverdistractionimages','*.*')))\n# tensors = paths_to_tensor(images_paths)","metadata":{"execution":{"iopub.status.busy":"2021-07-08T15:38:28.388834Z","iopub.execute_input":"2021-07-08T15:38:28.389203Z","iopub.status.idle":"2021-07-08T15:38:29.133444Z","shell.execute_reply.started":"2021-07-08T15:38:28.389174Z","shell.execute_reply":"2021-07-08T15:38:29.132518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.set_style(\"whitegrid\", {'axes.grid' : False})\n\nimport matplotlib.image as mpimg\n# mapping categotical\n\nclasses = [ 'safe driving', ' texting - right', 'talking on the phone - right',\n 'texting - left', 'talking on the phone - left', 'operating the radio',\n 'drinking', 'reaching behind', 'hair and makeup', 'talking to passenger']\n\n\nPredictions = model.predict(random_tensors)\nPredicted_Labels = np.argmax(Predictions, axis = 1)\nfig = plt.figure(figsize = (20,40))\n\nfor i, LAB in enumerate(Predicted_Labels):\n \n fig.add_subplot(10, 6, i+1)\n plt.imshow(random_tensors[i][:][:])\n plt.axis('off')\n plt.title('Predicted Label: \\n C{}->{}'.format(LAB , classes[LAB]))\n ","metadata":{"execution":{"iopub.status.busy":"2021-07-08T16:10:34.376456Z","iopub.execute_input":"2021-07-08T16:10:34.376767Z","iopub.status.idle":"2021-07-08T16:10:38.629447Z","shell.execute_reply.started":"2021-07-08T16:10:34.376738Z","shell.execute_reply":"2021-07-08T16:10:38.621090Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}