{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <center>👨🏻‍💻 | Getting Started with Computer Vision </center>","metadata":{}},{"cell_type":"markdown","source":"This is my **first** time working on a  **computer vision** project and developing a **neural network**!<br><br>\nIn order to start with computer vision, I'm using Kaggle's <a href=\"https://www.kaggle.com/competitions/digit-recognizer/overview/description\">Digit Recognizer</a> competition to put on practice the knowledge I've acquired after studying the principles of computer vision and deep learning.<br><br>\nThe MNIST (\"Modified National Institute of Standards and Technology\") is the **de facto “hello world” dataset of computer vision**. Since its release in 1999, this classic dataset of handwritten images has served as the basis for benchmarking classification algorithms. As new machine learning techniques emerge, MNIST remains a reliable resource for researchers and learners alike.<br><br>\nThe **goal of this competition is to take images of handwritten digits and create a machine learning model that can correctly identify what digit is in the image**. The metrics used to evaluate how well the model performs is the accuracy of predicitions, that is the percentage of images that our model can label correctly.<br><br>\n\n# Why Convolutional Neural Network?\nAccording to <a href = \"https://www.ibm.com/cloud/learn/convolutional-neural-networks\">this IBM article</a>, convolutional neural networks are distinguished from other neural networks by their **superior performance with image, speech and audio inputs**, which makes it ideal for a getting started project working with image classification. ","metadata":{}},{"cell_type":"markdown","source":"<h1 style = \"border-bottom: 3px solid black; padding: 8px\"><b>1 | Importing Libraries</b></h1>","metadata":{}},{"cell_type":"code","source":"import numpy as np,pandas as pd, plotly.express as plt, tensorflow as tf, matplotlib.pyplot as plt\nfrom tensorflow.keras.layers import Dense, Conv2D, MaxPool2D, Flatten\nfrom tensorflow.keras.models import Sequential\nfrom sklearn.model_selection import train_test_split\ntrain = pd.read_csv('../input/digit-recognizer/train.csv')\ntest = pd.read_csv('../input/digit-recognizer/test.csv')","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:15:16.403203Z","iopub.execute_input":"2022-08-13T01:15:16.403856Z","iopub.status.idle":"2022-08-13T01:15:20.198527Z","shell.execute_reply.started":"2022-08-13T01:15:16.403821Z","shell.execute_reply":"2022-08-13T01:15:20.197572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from plotly.offline import init_notebook_mode\ninit_notebook_mode(connected=True)","metadata":{"execution":{"iopub.status.busy":"2022-08-13T15:49:14.065558Z","iopub.execute_input":"2022-08-13T15:49:14.066105Z","iopub.status.idle":"2022-08-13T15:49:14.182223Z","shell.execute_reply.started":"2022-08-13T15:49:14.065997Z","shell.execute_reply":"2022-08-13T15:49:14.180839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<h1 style = \"border-bottom: 3px solid black; padding: 8px\"><b>2 | Exploring Data</b></h1>","metadata":{}},{"cell_type":"code","source":"# Seeing train dataset\ntrain","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:15:20.200780Z","iopub.execute_input":"2022-08-13T01:15:20.201543Z","iopub.status.idle":"2022-08-13T01:15:20.228487Z","shell.execute_reply.started":"2022-08-13T01:15:20.201496Z","shell.execute_reply":"2022-08-13T01:15:20.227257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.label.value_counts() # Counting labels ","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:15:20.230107Z","iopub.execute_input":"2022-08-13T01:15:20.230737Z","iopub.status.idle":"2022-08-13T01:15:20.239483Z","shell.execute_reply.started":"2022-08-13T01:15:20.230692Z","shell.execute_reply":"2022-08-13T01:15:20.238680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Let's see if we have any missing data in any of both datasets**","metadata":{}},{"cell_type":"code","source":"train.isnull().sum().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:15:20.241459Z","iopub.execute_input":"2022-08-13T01:15:20.242334Z","iopub.status.idle":"2022-08-13T01:15:20.304217Z","shell.execute_reply.started":"2022-08-13T01:15:20.242290Z","shell.execute_reply":"2022-08-13T01:15:20.303080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.isna().sum().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:15:20.305448Z","iopub.execute_input":"2022-08-13T01:15:20.305785Z","iopub.status.idle":"2022-08-13T01:15:20.342447Z","shell.execute_reply.started":"2022-08-13T01:15:20.305755Z","shell.execute_reply":"2022-08-13T01:15:20.341367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualizing Label counts\nfig = px.histogram(train, x = 'label', nbins=20, template = 'plotly_dark',\n                  title = 'Count for Every Label in the Dataset')\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:15:20.343911Z","iopub.execute_input":"2022-08-13T01:15:20.344259Z","iopub.status.idle":"2022-08-13T01:15:20.414859Z","shell.execute_reply.started":"2022-08-13T01:15:20.344228Z","shell.execute_reply":"2022-08-13T01:15:20.413759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Tensorflow** can only get array values as input. Since our data is currently in a csv format, we must convert it into array.","metadata":{}},{"cell_type":"code","source":"labels = np.array(train.label) # Converting labels into arrays\ntrain.drop(['label'], axis = 1, inplace = True) # Removing labels from dataframe\n\n# Converting images into 28x28 matrix (original proportions)\ntrain = np.array(train).reshape(train.shape[0], 28, 28)\ntest = np.array(test).reshape(test.shape[0], 28,28)\n\ntrain = np.array(train).reshape((-1, 28, 28, 1))\ntest = np.array(test).reshape((-1, 28,28,1))","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:15:20.416398Z","iopub.execute_input":"2022-08-13T01:15:20.416755Z","iopub.status.idle":"2022-08-13T01:15:20.864244Z","shell.execute_reply.started":"2022-08-13T01:15:20.416724Z","shell.execute_reply":"2022-08-13T01:15:20.863230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualizing images for labels\nplt.figure(figsize=(20,15))\nfor i in range(20): # Running loop for each digit to be printed\n    plt.subplot(5,5,i+1)\n    index = np.random.randint(0,42000)\n    plt.imshow(train[index], cmap = 'Greys')\n    plt.title(labels[index])","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:16:09.294268Z","iopub.execute_input":"2022-08-13T01:16:09.294948Z","iopub.status.idle":"2022-08-13T01:16:11.216021Z","shell.execute_reply.started":"2022-08-13T01:16:09.294911Z","shell.execute_reply":"2022-08-13T01:16:11.215057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<h1 style = \"border-bottom: 3px solid black; padding: 8px\"><b>3 | Building a Convolutional Neural Network model using TensorFlow</b></h1>","metadata":{}},{"cell_type":"code","source":"# Creating model\n\nmodel = Sequential() # Defining a Sequential model, ideal to  connect together a list of layers\n# Conv2D layers creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs\nmodel.add(Conv2D(filters=64, kernel_size=(5,5), activation = 'relu', input_shape=(28,28,1)))\nmodel.add(Conv2D(filters = 32, kernel_size=(3,3), activation = 'relu'))\n\n# MaxPool2D layers downsamples the input along its spatial dimensionas (height and width)\nmodel.add(MaxPool2D(pool_size=(2,2)))\nmodel.add(Conv2D(filters = 32, kernel_size =(3,3), activation = 'relu'))\nmodel.add(Conv2D(filters = 16, kernel_size =(3,3), activation = 'relu'))\nmodel.add(MaxPool2D(pool_size=(2,2)))\n\n# Flatten layers are responsible for flattening input\nmodel.add(Flatten())\n\n# Dense layers receives input from all neurons from previous layer. It's used to classify image based on output from convolutional layers\nmodel.add(Dense(64, activation = 'relu'))\nmodel.add(Dense(10, activation = 'Softmax', name = 'Output_layer'))\n\n# Compiling model and defining the loss function, optimizers and metrics for prediction.\nmodel.compile(loss = 'sparse_categorical_crossentropy', # Defining a loss function\n             optimizer = 'adam', # Defining a loss function\n             metrics = ['accuracy']) # Defining evaluation metric. Accuracy, as it is the main metric defined on the competition description \n\nmodel.summary() # Getting summary of our model","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:28:06.646214Z","iopub.execute_input":"2022-08-13T01:28:06.646627Z","iopub.status.idle":"2022-08-13T01:28:06.731633Z","shell.execute_reply.started":"2022-08-13T01:28:06.646594Z","shell.execute_reply":"2022-08-13T01:28:06.730354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Using Keras' *plot_model*, we can visualize how our neural network is structured.","metadata":{}},{"cell_type":"code","source":"# Visualizing model\ntf.keras.utils.plot_model(model)","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:28:27.169801Z","iopub.execute_input":"2022-08-13T01:28:27.170318Z","iopub.status.idle":"2022-08-13T01:28:28.433143Z","shell.execute_reply.started":"2022-08-13T01:28:27.170272Z","shell.execute_reply":"2022-08-13T01:28:28.431540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Splitting data\ntrain_image, test_image, train_labels, test_labels = train_test_split(train,labels, train_size = 0.75, random_state = 123)","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:30:28.082476Z","iopub.execute_input":"2022-08-13T01:30:28.083546Z","iopub.status.idle":"2022-08-13T01:30:28.497440Z","shell.execute_reply.started":"2022-08-13T01:30:28.083490Z","shell.execute_reply":"2022-08-13T01:30:28.496420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Training model\nhistory = model.fit(train_image, train_labels, validation_data = (test_image, test_labels), epochs = 20)","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:31:22.095607Z","iopub.execute_input":"2022-08-13T01:31:22.096123Z","iopub.status.idle":"2022-08-13T01:41:45.322319Z","shell.execute_reply.started":"2022-08-13T01:31:22.096084Z","shell.execute_reply":"2022-08-13T01:41:45.321229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plotting Model Accuracy \nplt.figure(figsize=(15,8))\nplt.plot(history.history['accuracy'])\nplt.plot(history.history['val_accuracy'])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:52:32.913013Z","iopub.execute_input":"2022-08-13T01:52:32.913892Z","iopub.status.idle":"2022-08-13T01:52:33.132444Z","shell.execute_reply.started":"2022-08-13T01:52:32.913849Z","shell.execute_reply":"2022-08-13T01:52:33.131261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Creating submission file\npredictions = model.predict(test)\npredictions = tf.math.argmax(predictions, axis = -1)\npredictions = pd.Series(predictions, name='Label')\nimage_id = pd.Series(range(1,28001), name = 'ImageId')\nimage_id.isnull().sum()\npredictions = pd.concat([image_id, predictions], axis = 1)\npredictions.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-13T01:55:45.742078Z","iopub.execute_input":"2022-08-13T01:55:45.742529Z","iopub.status.idle":"2022-08-13T01:55:53.524024Z","shell.execute_reply.started":"2022-08-13T01:55:45.742493Z","shell.execute_reply":"2022-08-13T01:55:53.522917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<h1 style = \"border-bottom: 3px solid black; padding: 8px\"><b>4 | Conclusion</b></h1>","metadata":{}},{"cell_type":"markdown","source":"For some reason, I'm having trouble to make submissions attached to the notebook, however, this model achieved an accuracy score of **98.09%** on the test dataset, which is a very good result!<br><br>\n\nFor a first attempt at developing a deep learning model, it was really fun participating in this competition.<br>\nFeel free to leave comments and suggestions, and if you liked this notebook, leave an upvote! <br><br>\n\nThank you so much!<br><br><br>\n\n*Luís Fernando Torres*","metadata":{}}]}