![\"happy-whale\"](\"https://i.ibb.co/HTtzjh2/happy-whale.jpg\")
Humpback Whale Identification
","metadata":{"_uuid":"799553e03d37b0f2885c3b038e27fa2a2d757139"}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"Story about dataset\n\n1. After centuries of intense whaling, recovering whale populations still have a hard time adapting to warming oceans and struggle to compete every day with the industrial fishing industry for food.\n\n2. To aid whale conservation efforts, scientists use photo surveillance systems to monitor ocean activity. They use the shape of whales’ tails and unique markings found in footage to identify what species of whale they’re analyzing and meticulously log whale pod dynamics and movements. For the past 40 years, most of this work has been done manually by individual scientists, leaving a huge trove of data untapped and underutilized.\n\n3. We'd like to thank Happywhale for providing this data and problem. [Happywhale](https://happywhale.com/) is a platform that uses image process algorithms to let anyone to submit their whale photo and have it automatically identified.","metadata":{}},{"cell_type":"markdown","source":"📌 **Algorithms used:** \n * CNN\n \n \n📌**Tools used:-** \n* Google Colab\n\n📌**Libraries used:-** \n* Numpy\n* pandas\n* Matplotlib\n* flask\n* scikit-learn\n* Keras\n\n","metadata":{}},{"cell_type":"markdown","source":" Introduction
\nImporting the libraries
","metadata":{}},{"cell_type":"markdown","source":"1. NumPy is a Python library used for working with arrays. It also has functions for working in domain of linear algebra, fourier transform, and matrices.\n2. Pandas is mainly used for data analysis. Pandas allows importing data from various file formats such as comma-separated values, JSON, SQL database tables or queries, and Microsoft Excel.\n3. matplotlib. pyplot is a collection of functions that make matplotlib work like MATLAB. Each pyplot function makes some change to a figure: e.g., creates a figure, creates a plotting area in a figure, plots some lines in a plotting area, decorates the plot with labels, etc.\n4. The OS module in Python provides functions for interacting with the operating system. OS comes under Python's standard utility modules.\n5. Warning messages are typically issued in situations where it is useful to alert the user of some condition in a program, where that condition (normally) doesn't warrant raising an exception and terminating the program.\n","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# import warnings\nimport warnings\n# filter warnings\nwarnings.filterwarnings('ignore')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-01-10T12:01:28.406962Z","iopub.execute_input":"2022-01-10T12:01:28.407310Z","iopub.status.idle":"2022-01-10T12:01:28.416242Z","shell.execute_reply.started":"2022-01-10T12:01:28.407253Z","shell.execute_reply":"2022-01-10T12:01:28.415165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nImporting the required dataset
","metadata":{}},{"cell_type":"code","source":"#Reading the csv file for train dataset\ntrain = pd.read_csv(\"../input/train.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.417459Z","iopub.execute_input":"2022-01-10T12:01:28.417729Z","iopub.status.idle":"2022-01-10T12:01:28.451360Z","shell.execute_reply.started":"2022-01-10T12:01:28.417682Z","shell.execute_reply":"2022-01-10T12:01:28.450664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# The info() method prints information about the DataFrame.\n# The information contains the number of columns, column labels, column data types,\n# memory usage, range index, and the number of cells in each column (non-null values).\ntrain.info()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.452292Z","iopub.execute_input":"2022-01-10T12:01:28.452495Z","iopub.status.idle":"2022-01-10T12:01:28.470596Z","shell.execute_reply.started":"2022-01-10T12:01:28.452454Z","shell.execute_reply":"2022-01-10T12:01:28.468266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# The describe() method is used for calculating some statistical data like percentile,\n# mean and std of the numerical values of the Series or DataFrame.\n# It analyzes both numeric and object series and also the DataFrame column sets of mixed data types.\ntrain.describe()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.471833Z","iopub.execute_input":"2022-01-10T12:01:28.472320Z","iopub.status.idle":"2022-01-10T12:01:28.530611Z","shell.execute_reply.started":"2022-01-10T12:01:28.472272Z","shell.execute_reply":"2022-01-10T12:01:28.529810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **There are 5005 different classes in our dataset**","metadata":{}},{"cell_type":"code","source":"# shape gives number of rows and columns in a tuple\ntrain.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.531790Z","iopub.execute_input":"2022-01-10T12:01:28.532278Z","iopub.status.idle":"2022-01-10T12:01:28.537282Z","shell.execute_reply.started":"2022-01-10T12:01:28.532229Z","shell.execute_reply":"2022-01-10T12:01:28.536525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# head funtion gives the first 5 rows of datasets\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.538619Z","iopub.execute_input":"2022-01-10T12:01:28.539114Z","iopub.status.idle":"2022-01-10T12:01:28.555640Z","shell.execute_reply.started":"2022-01-10T12:01:28.539064Z","shell.execute_reply":"2022-01-10T12:01:28.555062Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# tail funtion gives the last 5 rows of datasets\ntrain.tail()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.556733Z","iopub.execute_input":"2022-01-10T12:01:28.556935Z","iopub.status.idle":"2022-01-10T12:01:28.571054Z","shell.execute_reply.started":"2022-01-10T12:01:28.556894Z","shell.execute_reply":"2022-01-10T12:01:28.570141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We need to identify the Id of whale. So, our output variable will be Id. We need to seperate feature columns and output column.**","metadata":{}},{"cell_type":"code","source":"# put labels into y_train variable\ny_train = train[\"Id\"]\n# Drop the 'Id' column\nxtrain = train.drop(labels = [\"Id\"], axis = 1)\ny_train.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.572144Z","iopub.execute_input":"2022-01-10T12:01:28.572569Z","iopub.status.idle":"2022-01-10T12:01:28.581181Z","shell.execute_reply.started":"2022-01-10T12:01:28.572521Z","shell.execute_reply":"2022-01-10T12:01:28.580321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Indicates sum of values in our data\ntrain.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.582385Z","iopub.execute_input":"2022-01-10T12:01:28.582879Z","iopub.status.idle":"2022-01-10T12:01:28.598653Z","shell.execute_reply.started":"2022-01-10T12:01:28.582832Z","shell.execute_reply":"2022-01-10T12:01:28.597970Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**So, no row or column data is missing means we don't have to preprocess the data.**","metadata":{}},{"cell_type":"markdown","source":"\nImage Preprocessing
","metadata":{}},{"cell_type":"code","source":"# importing the libraries for image preprocessing\nfrom keras.preprocessing import image\nfrom keras.applications.imagenet_utils import preprocess_input","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.599739Z","iopub.execute_input":"2022-01-10T12:01:28.600023Z","iopub.status.idle":"2022-01-10T12:01:28.604492Z","shell.execute_reply.started":"2022-01-10T12:01:28.599973Z","shell.execute_reply":"2022-01-10T12:01:28.603815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prepareImages(train, shape, path):\n \n x_train = np.zeros((shape, 100, 100, 3))\n count = 0\n \n for fig in train['Image']:\n \n #load images into images of size 100x100x3\n img = image.load_img(\"../input/\"+path+\"/\"+fig, target_size=(100, 100, 3))\n x = image.img_to_array(img)\n x = preprocess_input(x)\n\n x_train[count] = x\n if (count%500 == 0):\n print(\"Processing image: \", count+1, \", \", fig)\n count += 1\n \n return x_train","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:01:28.606001Z","iopub.execute_input":"2022-01-10T12:01:28.606451Z","iopub.status.idle":"2022-01-10T12:01:28.615679Z","shell.execute_reply.started":"2022-01-10T12:01:28.606404Z","shell.execute_reply":"2022-01-10T12:01:28.615099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x_train = prepareImages(train, train.shape[0], \"train\")","metadata":{"_uuid":"9f4dcb4353af7c7fb2aba26405d332b04f83a05e","execution":{"iopub.status.busy":"2022-01-10T12:01:28.617537Z","iopub.execute_input":"2022-01-10T12:01:28.618007Z","iopub.status.idle":"2022-01-10T12:07:27.622013Z","shell.execute_reply.started":"2022-01-10T12:01:28.617766Z","shell.execute_reply":"2022-01-10T12:07:27.621270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nNormalize the Data
","metadata":{"_uuid":"9b7332cb21da0e745c02f6bd4e6ef079295f5b0a"}},{"cell_type":"code","source":"x_train = x_train / 255.0 \n# rescaling the dataset \n# dividing an image by 255 simply rescales the image from 0-255 to 0-1. \n# (Converting it to float from int makes computation convenient too) \nprint(\"xtrain shape: \",x_train.shape)","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:27.623221Z","iopub.execute_input":"2022-01-10T12:07:27.623477Z","iopub.status.idle":"2022-01-10T12:07:31.441057Z","shell.execute_reply.started":"2022-01-10T12:07:27.623434Z","shell.execute_reply":"2022-01-10T12:07:31.440319Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Checking example input image\nplt.imshow(x_train[0][:,:,0], cmap=\"gray\")\nplt.title(plt.title(train.iloc[0,0]))\nplt.axis(\"off\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.442219Z","iopub.execute_input":"2022-01-10T12:07:31.442490Z","iopub.status.idle":"2022-01-10T12:07:31.531200Z","shell.execute_reply.started":"2022-01-10T12:07:31.442441Z","shell.execute_reply":"2022-01-10T12:07:31.530245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nLabel Encoding
","metadata":{}},{"cell_type":"markdown","source":"1. Encode target labels with value between 0 and n_classes-1.\n2. This transformer should be used to encode target values, i.e. y, and not the input X.\n3. Note:- Label encoding converts the data in machine-readable form, but it assigns a unique number(starting from 0) to each class of data. This may lead to the generation of priority issues in the training of data sets. A label with a high value may be considered to have high priority than a label having a lower value.","metadata":{}},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nlabel_encoder = LabelEncoder()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.532541Z","iopub.execute_input":"2022-01-10T12:07:31.533032Z","iopub.status.idle":"2022-01-10T12:07:31.537607Z","shell.execute_reply.started":"2022-01-10T12:07:31.532982Z","shell.execute_reply":"2022-01-10T12:07:31.536603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Fit label encoder and return encoded labels.\ny_train = label_encoder.fit_transform(y_train)","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.538930Z","iopub.execute_input":"2022-01-10T12:07:31.539536Z","iopub.status.idle":"2022-01-10T12:07:31.568049Z","shell.execute_reply.started":"2022-01-10T12:07:31.539466Z","shell.execute_reply":"2022-01-10T12:07:31.567275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# let's look at first 10 values\ny_train[0:10] ","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.572262Z","iopub.execute_input":"2022-01-10T12:07:31.574622Z","iopub.status.idle":"2022-01-10T12:07:31.584555Z","shell.execute_reply.started":"2022-01-10T12:07:31.574561Z","shell.execute_reply":"2022-01-10T12:07:31.583559Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# finding shape of y_train data\ny_train.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.589780Z","iopub.execute_input":"2022-01-10T12:07:31.592152Z","iopub.status.idle":"2022-01-10T12:07:31.600315Z","shell.execute_reply.started":"2022-01-10T12:07:31.592092Z","shell.execute_reply":"2022-01-10T12:07:31.599285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert to one-hot-encoding\n# we got 5005 classes from the function train.Id.describe()\n\nfrom keras.utils.np_utils import to_categorical\ny_train = to_categorical(y_train, num_classes = 5005)","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.603752Z","iopub.execute_input":"2022-01-10T12:07:31.604492Z","iopub.status.idle":"2022-01-10T12:07:31.668765Z","shell.execute_reply.started":"2022-01-10T12:07:31.604431Z","shell.execute_reply":"2022-01-10T12:07:31.667914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_train ","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.670002Z","iopub.execute_input":"2022-01-10T12:07:31.670270Z","iopub.status.idle":"2022-01-10T12:07:31.679746Z","shell.execute_reply.started":"2022-01-10T12:07:31.670225Z","shell.execute_reply":"2022-01-10T12:07:31.678994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nImplementation with Keras using CNN(Convolutional Neural Network)
","metadata":{}},{"cell_type":"markdown","source":"**Convolutional layer**\n\n1. This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs. If use_bias is True, a bias vector is created and added to the outputs. Finally, if activation is not None, it is applied to the outputs as well.\n\n2. When using this layer as the first layer in a model, provide the keyword argument input_shape (tuple of integers or None, does not include the sample axis), e.g. input_shape=(128, 128, 3) for 128x128 RGB pictures in data_format=\"channels_last\". You can use None when a dimension has variable size.\n\n3. filters: Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution).\n\n4. kernel_size: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.\n\n5. strides: An integer or tuple/list of 2 integers, specifying the strides of the convolution along the height and width. Can be a single integer to specify the same value for all spatial dimensions. Specifying any stride value != 1 is incompatible with specifying any dilation_rate value != 1.\n\n6. padding: one of \"valid\" or \"same\" (case-insensitive). \"valid\" means no padding. \"same\" results in padding with zeros evenly to the left/right or up/down of the input. When padding=\"same\" and strides=1, the output has the same size as the input.\n\n7. kernel_initializer: Initializer for the kernel weights matrix (see keras.initializers). Defaults to 'glorot_uniform'.","metadata":{}},{"cell_type":"markdown","source":"![\"uwHol\"](\"https://i.ibb.co/XttLpSy/uwHol.gif\")
","metadata":{}},{"cell_type":"markdown","source":"**Padding**\n\n1. When padding == ”VALID”, the input image is not padded. When padding == \"VALID\", there can be a loss of information and (Input Size != Output Size)\n\n2. When padding == “SAME”, the output size is the same as the input size(when stride=1). Normally, padding is set to \"SAME\" while training the model. Output size is mathematically convenient for further computation.\n3. Size of each feature map = [N-f+2P /S] + 1\n","metadata":{}},{"cell_type":"markdown","source":"**Visualize how padding works**\n![\"padding\"](\"https://i.ibb.co/yXYD7hM/padding.gif\")
","metadata":{}},{"cell_type":"markdown","source":"**Maxpooling2D**\n\n1. Downsamples the input along its spatial dimensions (height and width) by taking the maximum value over an input window (of size defined by pool_size) for each channel of the input. The window is shifted by strides along each dimension.\n\n2. The resulting output, when using the \"valid\" padding option, has a spatial shape (number of rows or columns) of: output_shape = math.floor((input_shape - pool_size) / strides) + 1 (when input_shape >= pool_size)\n\n3. The resulting output shape when using the \"same\" padding option is: output_shape = math.floor((input_shape - 1) / strides) + 1","metadata":{}},{"cell_type":"markdown","source":"**Dropout layer**\n\n1. The Dropout layer randomly sets input units to 0 with a frequency of rate at each step during training time, which helps prevent overfitting. Inputs not set to 0 are scaled up by 1/(1 - rate) such that the sum over all inputs is unchanged.\n\n2. Note that the Dropout layer only applies when training is set to True such that no values are dropped during inference. When using model.fit, training will be appropriately set to True automatically, and in other contexts, you can set the kwarg explicitly to True when calling the layer.\n\n","metadata":{}},{"cell_type":"markdown","source":"**Flatten layer**\n\n1. Flattens the input.\n2. If inputs are shaped (batch,) without a feature axis, then flattening adds an extra channel dimension and output shape is (batch, 1).","metadata":{}},{"cell_type":"code","source":"from keras.utils.np_utils import to_categorical # convert to one-hot-encoding\nfrom keras.models import Sequential # to create a cnn model\nfrom keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D\nfrom keras.optimizers import RMSprop,Adam\nfrom keras.layers.normalization import BatchNormalization\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.callbacks import ReduceLROnPlateau\n\nmodel = Sequential()\n\nmodel.add(Conv2D(filters = 16, kernel_size = (3,3), padding = 'Same', activation = 'relu', input_shape = (100,100,3)))\nmodel.add(Conv2D(filters = 16, kernel_size = (3,3), padding = 'Same', activation = 'relu'))\nmodel.add(MaxPool2D(pool_size = (2,2)))\nmodel.add(BatchNormalization())\n\nmodel.add(Conv2D(filters = 32, kernel_size = (3,3), padding = 'Same', activation = 'relu'))\nmodel.add(Conv2D(filters = 32, kernel_size = (3,3), padding = 'Same', activation = 'relu'))\nmodel.add(MaxPool2D(pool_size = (2,2), strides=(2,2)))\nmodel.add(Dropout(0.25))\n\nmodel.add(Conv2D(filters = 64, kernel_size = (3,3), padding = 'Same', activation = 'relu'))\nmodel.add(Conv2D(filters = 64, kernel_size = (3,3), padding = 'Same', activation = 'relu'))\nmodel.add(MaxPool2D(pool_size = (2,2), strides=(2,2)))\nmodel.add(BatchNormalization())\n\n# fully connected\nmodel.add(Flatten())\nmodel.add(Dense(256, activation = 'relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dense(y_train.shape[1], activation = \"softmax\"))","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.681477Z","iopub.execute_input":"2022-01-10T12:07:31.681990Z","iopub.status.idle":"2022-01-10T12:07:31.919183Z","shell.execute_reply.started":"2022-01-10T12:07:31.681721Z","shell.execute_reply":"2022-01-10T12:07:31.918527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Provides the summary of model we created\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.920540Z","iopub.execute_input":"2022-01-10T12:07:31.920952Z","iopub.status.idle":"2022-01-10T12:07:31.932895Z","shell.execute_reply.started":"2022-01-10T12:07:31.920771Z","shell.execute_reply":"2022-01-10T12:07:31.932237Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\nCompile the Model
","metadata":{}},{"cell_type":"markdown","source":" **Benefits of Adam:**\n* Straightforward to implement.\n* Computationally efficient.\n* Little memory requirements.\n* Invariant to diagonal rescale of the gradients.\n* Well suited for problems that are large in terms of data and/or parameters.\n* Appropriate for non-stationary objectives.\n* Appropriate for problems with very noisy/or sparse gradients.\n* Hyper-parameters have intuitive interpretation and typically require little tuning.\n\n\n[Watch Adam paper here](https://arxiv.org/pdf/1412.6980.pdf/) \n![\"adam-algo\"](\"https://i.ibb.co/7zxThZJ/adam-algo.jpg\")
","metadata":{}},{"cell_type":"code","source":"# Define the optimizer\noptimizer = Adam(lr = 0.001, beta_1 = 0.9, beta_2 = 0.999)","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.934669Z","iopub.execute_input":"2022-01-10T12:07:31.935189Z","iopub.status.idle":"2022-01-10T12:07:31.955806Z","shell.execute_reply.started":"2022-01-10T12:07:31.935001Z","shell.execute_reply":"2022-01-10T12:07:31.955229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Learning rate scheduler**\n\n\nAt the beginning of every epoch, this callback gets the updated learning rate value from schedule function provided at __init__, with the current epoch and current learning rate, and applies the updated learning rate on the optimizer.\n\n1. **schedule:** a function that takes an epoch index (integer, indexed from 0) and current learning rate (float) as inputs and returns a new learning rate as output (float).\n\n2. **verbose:** int value 0: quiet , 1: update messages.","metadata":{}},{"cell_type":"markdown","source":"**ReduceLROnPlateau terms**\n\n1. monitor\tquantity to be monitored.\n2. factor\tfactor by which the learning rate will be reduced. new_lr = lr * factor.\n3. patience\tnumber of epochs with no improvement after which learning rate will be reduced.\n4. verbose\tint. 0: quiet, 1: update messages.\n5. min_delta\tthreshold for measuring the new optimum, to only focus on significant changes.\n6. min_lr\tlower bound on the learning rate.","metadata":{}},{"cell_type":"markdown","source":"![\"lr-on-plateau\"](\"https://i.ibb.co/tDYtRNS/lr-on-plateau.png\")
","metadata":{}},{"cell_type":"code","source":"# Set a learning rate scheduler\nlearning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', \n patience=3, \n verbose=1, \n factor=0.5, \n min_lr=0.00001)","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.957220Z","iopub.execute_input":"2022-01-10T12:07:31.957692Z","iopub.status.idle":"2022-01-10T12:07:31.962728Z","shell.execute_reply.started":"2022-01-10T12:07:31.957643Z","shell.execute_reply":"2022-01-10T12:07:31.961841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Compiler terms**\n\n1. optimizer: String (name of optimizer) or optimizer instance.\n\n2. loss: A loss function is any callable with the signature loss = fn(y_true, y_pred), where y_true are the ground truth values, and y_pred are the model's predictions. y_true should have shape (batch_size, d0, .. dN) (except in the case of sparse loss functions such as sparse categorical crossentropy which expects integer arrays of shape (batch_size, d0, .. dN-1)). y_pred should have shape (batch_size, d0, .. dN).\n\n3. metrics: List of metrics to be evaluated by the model during training and testing.\n\n4. loss_weights: Optional list or dictionary specifying scalar coefficients (Python floats) to weight the loss contributions of different model outputs. The loss value that will be minimized by the model will then be the weighted sum of all individual losses, weighted by the loss_weights coefficients. \n\n5. run_eagerly: Bool. Defaults to False.\n\n6. steps_per_execution: (Default value of steps_er_epochs=1). The number of batches to run during each tf.function call. Running multiple batches inside a single tf.function call can greatly improve performance on small models. At most, one full epoch will be run each execution. If a number larger than the size of the epoch is passed, the execution will be truncated to the size of the epoch.\n\n7. **kwargs: Arguments supported for backwards compatibility only.\n","metadata":{}},{"cell_type":"code","source":"model.compile(optimizer = optimizer, loss = \"categorical_crossentropy\", metrics=[\"accuracy\"])","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.964183Z","iopub.execute_input":"2022-01-10T12:07:31.964690Z","iopub.status.idle":"2022-01-10T12:07:31.994899Z","shell.execute_reply.started":"2022-01-10T12:07:31.964436Z","shell.execute_reply":"2022-01-10T12:07:31.994164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nData Augmentation
","metadata":{}},{"cell_type":"markdown","source":"Lets use Data augmentation:-\n1. Data augmentation are techniques used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data. \n2. It acts as a regularizer and helps reduce overfitting when training a machine learning model.","metadata":{}},{"cell_type":"code","source":"# With data augmentation to prevent overfitting\n\ndatagen = ImageDataGenerator(\n featurewise_center=False, # set input mean to 0 over the dataset\n samplewise_center=False, # set each sample mean to 0\n featurewise_std_normalization=False, # divide inputs by std of the dataset\n samplewise_std_normalization=False, # divide each input by its std\n zca_whitening=False, # apply ZCA whitening\n rotation_range=10, # randomly rotate images in the range (degrees, 0 to 180)\n zoom_range = 0.1, # Randomly zoom image \n width_shift_range=0.1, # randomly shift images horizontally (fraction of total width)\n height_shift_range=0.1, # randomly shift images vertically (fraction of total height)\n horizontal_flip=False, # randomly flip images horizontally\n vertical_flip=False) # randomly flip images vertically\n\n\ndatagen.fit(x_train)","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:31.996164Z","iopub.execute_input":"2022-01-10T12:07:31.996439Z","iopub.status.idle":"2022-01-10T12:07:36.695172Z","shell.execute_reply.started":"2022-01-10T12:07:31.996370Z","shell.execute_reply":"2022-01-10T12:07:36.694278Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Epochs and Batch Size**","metadata":{}},{"cell_type":"markdown","source":"1. Stochastic gradient descent is an iterative learning algorithm that uses a training dataset to update a model.\n2. The batch size is a hyperparameter of gradient descent that controls the number of training samples to work through before the model’s internal parameters are updated.\n3. The number of epochs is a hyperparameter of gradient descent that controls the number of complete passes through the training dataset.","metadata":{}},{"cell_type":"code","source":"epochs = 50 # for better result increase the number of epochs\nbatch_size = 64","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:07:36.696430Z","iopub.execute_input":"2022-01-10T12:07:36.696718Z","iopub.status.idle":"2022-01-10T12:07:36.701639Z","shell.execute_reply.started":"2022-01-10T12:07:36.696671Z","shell.execute_reply":"2022-01-10T12:07:36.700876Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nFit the Model
\n\nWhen we call the .fit_generator() function it makes assumptions:\n1. Keras is first calling the generator function(dataAugmentaion)\n2. Generator function(dataAugmentaion) provides a batch_size of 32 to our .fit_generator() function.\n3. our .fit_generator() function first accepts a batch of the dataset, then performs backpropagation on it, and then updates the weights in our model.\n4. For the number of epochs specified the process is repeated.\n5. fit_generator is used when either we have a huge dataset to fit into our memory or when data augmentation needs to be applied.","metadata":{}},{"cell_type":"code","source":"history = model.fit_generator(datagen.flow(x_train, y_train, batch_size=batch_size),\n epochs=50, verbose = 2, \n steps_per_epoch=x_train.shape[0] // batch_size,\n callbacks=[learning_rate_reduction]) ","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:08:52.360147Z","iopub.execute_input":"2022-01-10T12:08:52.360433Z","iopub.status.idle":"2022-01-10T12:54:22.053639Z","shell.execute_reply.started":"2022-01-10T12:08:52.360376Z","shell.execute_reply":"2022-01-10T12:54:22.052721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nEvaluate the model
\n\n","metadata":{}},{"cell_type":"code","source":"# Plot the loss curve for training\nplt.plot(history.history['loss'], color='r', label=\"Train Loss\")\nplt.title(\"Train Loss\")\nplt.xlabel(\"Number of Epochs\")\nplt.ylabel(\"Loss\")\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:56:37.424984Z","iopub.execute_input":"2022-01-10T12:56:37.425271Z","iopub.status.idle":"2022-01-10T12:56:37.578663Z","shell.execute_reply.started":"2022-01-10T12:56:37.425218Z","shell.execute_reply":"2022-01-10T12:56:37.577822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot the accuracy curve for training\nplt.plot(history.history['acc'], color='g', label=\"Train Accuracy\")\nplt.title(\"Train Accuracy\")\nplt.xlabel(\"Number of Epochs\")\nplt.ylabel(\"Accuracy\")\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:56:38.673722Z","iopub.execute_input":"2022-01-10T12:56:38.673997Z","iopub.status.idle":"2022-01-10T12:56:38.826200Z","shell.execute_reply.started":"2022-01-10T12:56:38.673949Z","shell.execute_reply":"2022-01-10T12:56:38.825325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# finding the training accuracy \nprint('Train accuracy of the model: ',history.history['acc'][-1])","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:56:39.180159Z","iopub.execute_input":"2022-01-10T12:56:39.180424Z","iopub.status.idle":"2022-01-10T12:56:39.184908Z","shell.execute_reply.started":"2022-01-10T12:56:39.180369Z","shell.execute_reply":"2022-01-10T12:56:39.184118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# finding the training loss \nprint('Train loss of the model: ',history.history['loss'][-1])","metadata":{"execution":{"iopub.status.busy":"2022-01-10T12:58:13.386676Z","iopub.execute_input":"2022-01-10T12:58:13.386953Z","iopub.status.idle":"2022-01-10T12:58:13.394600Z","shell.execute_reply.started":"2022-01-10T12:58:13.386905Z","shell.execute_reply":"2022-01-10T12:58:13.390591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nThe End
\n\n","metadata":{}}]}