{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

✨ Handwritten Digit Recognition with CNN✨

\n
","metadata":{}},{"cell_type":"markdown","source":"

1 | Introduction 🎯

\n

✅ Competition Description

\n\n* In this competition, our goal is to correctly identify digits from a dataset of tens of thousands of handwritten images. \n\n

✅ Data Description

\n\n* The data files train.csv and test.csv contain gray-scale images of hand-drawn digits, from zero through nine.\n\n* Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\n\n* The training data set, (train.csv), has 785 columns. The first column, called \"label\", is the digit that was drawn by the user. The rest of the columns contain the pixel-values of the associated image.\n\n* Each pixel column in the training set has a name like pixelx, where x is an integer between 0 and 783, inclusive. To locate this pixel on the image, suppose that we have decomposed x as x = i * 28 + j, where i and j are integers between 0 and 27, inclusive. Then pixelx is located on row i and column j of a 28 x 28 matrix, (indexing by zero).\n\n

✅ Model choice

\n\nIn this kernel, I have created a model based on Deep Learning : Convolutional Neural Network (CNN) to recognize different handwritten digits and classify them. The dataset used here is actually from Digit Recognition Competition. Let's get started.\n\n
","metadata":{}},{"cell_type":"markdown","source":"

2 | Import Necessary Librairies 📚

","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport colorama\nfrom colorama import Fore\nfrom keras.utils.np_utils import to_categorical\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Conv2D, Flatten\nimport tensorflow as tf\nfrom tensorflow.keras import datasets, layers, models\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))\n\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-29T22:15:12.308357Z","iopub.execute_input":"2022-07-29T22:15:12.308810Z","iopub.status.idle":"2022-07-29T22:15:23.643695Z","shell.execute_reply.started":"2022-07-29T22:15:12.308713Z","shell.execute_reply":"2022-07-29T22:15:23.642599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

3 | Data Loading and Overview 📅

","metadata":{}},{"cell_type":"code","source":"train_df=pd.read_csv(\"../input/digit-recognizer/train.csv\")\ntest_df=pd.read_csv(\"../input/digit-recognizer/test.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:23.645452Z","iopub.execute_input":"2022-07-29T22:15:23.646008Z","iopub.status.idle":"2022-07-29T22:15:28.645545Z","shell.execute_reply.started":"2022-07-29T22:15:23.645979Z","shell.execute_reply":"2022-07-29T22:15:28.644598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:28.646943Z","iopub.execute_input":"2022-07-29T22:15:28.647239Z","iopub.status.idle":"2022-07-29T22:15:28.676941Z","shell.execute_reply.started":"2022-07-29T22:15:28.647212Z","shell.execute_reply":"2022-07-29T22:15:28.675957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(Fore.BLUE +f\"The train set has\",Fore.GREEN+f\"{train_df.shape[0]}\",Fore.BLUE +\"rows and\",Fore.GREEN+f\"{train_df.shape[1]}\",Fore.BLUE +\"columns\")\nprint(Fore.BLUE +f\"The test set has\",Fore.GREEN+f\"{test_df.shape[0]}\",Fore.BLUE +\"rows and\",Fore.GREEN+f\"{test_df.shape[1]}\",Fore.BLUE +\"columns\")","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:28.679448Z","iopub.execute_input":"2022-07-29T22:15:28.679800Z","iopub.status.idle":"2022-07-29T22:15:28.702672Z","shell.execute_reply.started":"2022-07-29T22:15:28.679763Z","shell.execute_reply":"2022-07-29T22:15:28.701528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_na=train_df.isna().sum().sum()\ntest_na=test_df.isna().sum().sum()\nprint(Fore.BLUE +f\"The number of missing values in train set is :\", Fore.GREEN+f\"{train_na}\")\nprint(Fore.BLUE +f\"The number of missing values in test set is :\", Fore.GREEN+f\"{test_na}\")","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:28.706335Z","iopub.execute_input":"2022-07-29T22:15:28.706677Z","iopub.status.idle":"2022-07-29T22:15:28.786422Z","shell.execute_reply.started":"2022-07-29T22:15:28.706647Z","shell.execute_reply":"2022-07-29T22:15:28.785313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

4 | Visualization of the target variable 👁‍🗨👀

","metadata":{}},{"cell_type":"code","source":"X = train_df.drop(\"label\", axis=1)\nY=train_df[\"label\"]","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:28.787708Z","iopub.execute_input":"2022-07-29T22:15:28.788008Z","iopub.status.idle":"2022-07-29T22:15:28.919230Z","shell.execute_reply.started":"2022-07-29T22:15:28.787983Z","shell.execute_reply":"2022-07-29T22:15:28.918292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Y.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:28.920508Z","iopub.execute_input":"2022-07-29T22:15:28.920920Z","iopub.status.idle":"2022-07-29T22:15:28.935748Z","shell.execute_reply.started":"2022-07-29T22:15:28.920891Z","shell.execute_reply":"2022-07-29T22:15:28.934856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"result=Y.value_counts().sort_values(ascending=False)\nresult.plot.barh(color=sns.color_palette(\"Spectral\", 9),width = 0.6, figsize=(12,5))\nplt.title(\"Number of items in each category\")\nplt.ylabel('Number of items')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:28.936647Z","iopub.execute_input":"2022-07-29T22:15:28.936994Z","iopub.status.idle":"2022-07-29T22:15:29.191165Z","shell.execute_reply.started":"2022-07-29T22:15:28.936964Z","shell.execute_reply":"2022-07-29T22:15:29.189883Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

5 | Visualization of the features variables 📈📊

","metadata":{}},{"cell_type":"code","source":"X = X.values.reshape(-1,28,28,1)\nX_test = test_df.values.reshape(-1,28,28,1)\nX.shape, X.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:29.192654Z","iopub.execute_input":"2022-07-29T22:15:29.192963Z","iopub.status.idle":"2022-07-29T22:15:29.200324Z","shell.execute_reply.started":"2022-07-29T22:15:29.192933Z","shell.execute_reply":"2022-07-29T22:15:29.199433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(3, 3, figsize=(13, 8))\nax[0,0].imshow(X[0], cmap = 'Greens', interpolation='bicubic')\nax[0,1].imshow(X[50], cmap = 'viridis', interpolation='bicubic')\nax[0,2].imshow(X[100], cmap = 'viridis', interpolation='bicubic')\nax[1,0].imshow(X[150], cmap = 'viridis', interpolation='bicubic')\nax[1,1].imshow(X[200], cmap = 'Greens', interpolation='bicubic')\nax[1,2].imshow(X[250], cmap = 'viridis', interpolation='bicubic')\nax[2,0].imshow(X[300], cmap = 'viridis', interpolation='bicubic')\nax[2,1].imshow(X[350], cmap = 'viridis', interpolation='bicubic')\nax[2,2].imshow(X[400], cmap = 'Greens', interpolation='bicubic')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:29.203646Z","iopub.execute_input":"2022-07-29T22:15:29.204004Z","iopub.status.idle":"2022-07-29T22:15:30.092666Z","shell.execute_reply.started":"2022-07-29T22:15:29.203965Z","shell.execute_reply":"2022-07-29T22:15:30.091558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

6 | Data preprocessing ♻

","metadata":{}},{"cell_type":"code","source":"# Data normalization\nX = X/255\nX_test = X_test/255","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:30.093939Z","iopub.execute_input":"2022-07-29T22:15:30.094316Z","iopub.status.idle":"2022-07-29T22:15:30.340009Z","shell.execute_reply.started":"2022-07-29T22:15:30.094282Z","shell.execute_reply":"2022-07-29T22:15:30.338911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# one-hot encode target column\nY = to_categorical(Y, num_classes = 10)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:30.341345Z","iopub.execute_input":"2022-07-29T22:15:30.341661Z","iopub.status.idle":"2022-07-29T22:15:30.346728Z","shell.execute_reply.started":"2022-07-29T22:15:30.341634Z","shell.execute_reply":"2022-07-29T22:15:30.345633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Spliting Data in training and testing set\nfrom sklearn.model_selection import train_test_split\nX_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size = 0.3, random_state=42)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:30.348053Z","iopub.execute_input":"2022-07-29T22:15:30.348870Z","iopub.status.idle":"2022-07-29T22:15:31.037134Z","shell.execute_reply.started":"2022-07-29T22:15:30.348839Z","shell.execute_reply":"2022-07-29T22:15:31.035960Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

7 | Building a CNN model ⚔

","metadata":{}},{"cell_type":"code","source":"#create model\nmodel = models.Sequential()\n# Add layers\nmodel.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))\nmodel.add(layers.MaxPooling2D((2, 2)))\nmodel.add(layers.Conv2D(64, (3, 3), activation='relu'))\nmodel.add(layers.MaxPooling2D((2, 2)))\nmodel.add(layers.Conv2D(64, (3, 3), activation='relu'))\nmodel.add(layers.MaxPooling2D((2, 2)))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:31.049820Z","iopub.execute_input":"2022-07-29T22:15:31.050534Z","iopub.status.idle":"2022-07-29T22:15:31.290963Z","shell.execute_reply.started":"2022-07-29T22:15:31.050487Z","shell.execute_reply":"2022-07-29T22:15:31.290000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.summary()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:31.293332Z","iopub.execute_input":"2022-07-29T22:15:31.293654Z","iopub.status.idle":"2022-07-29T22:15:31.300915Z","shell.execute_reply.started":"2022-07-29T22:15:31.293626Z","shell.execute_reply":"2022-07-29T22:15:31.299411Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.add(layers.Flatten())\nmodel.add(layers.Dense(128, activation='relu'))\nmodel.add(layers.Dense(10, activation='softmax'))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:31.302580Z","iopub.execute_input":"2022-07-29T22:15:31.303248Z","iopub.status.idle":"2022-07-29T22:15:31.338825Z","shell.execute_reply.started":"2022-07-29T22:15:31.303203Z","shell.execute_reply":"2022-07-29T22:15:31.337794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.summary()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:31.340159Z","iopub.execute_input":"2022-07-29T22:15:31.340484Z","iopub.status.idle":"2022-07-29T22:15:31.348107Z","shell.execute_reply.started":"2022-07-29T22:15:31.340441Z","shell.execute_reply":"2022-07-29T22:15:31.346839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualization of the model\ntf.keras.utils.plot_model(model)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:31.349539Z","iopub.execute_input":"2022-07-29T22:15:31.350745Z","iopub.status.idle":"2022-07-29T22:15:32.597865Z","shell.execute_reply.started":"2022-07-29T22:15:31.350699Z","shell.execute_reply":"2022-07-29T22:15:32.595807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Compile the model\nmodel.compile(optimizer='adam',\n loss='categorical_crossentropy',\n metrics=['accuracy'])\nhistory = model.fit(X_train, Y_train, epochs=10, \n validation_data=(X_val, Y_val))","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:15:32.600606Z","iopub.execute_input":"2022-07-29T22:15:32.601698Z","iopub.status.idle":"2022-07-29T22:17:38.277806Z","shell.execute_reply.started":"2022-07-29T22:15:32.601646Z","shell.execute_reply":"2022-07-29T22:17:38.276764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

8 | Model evaluation & accuracy

","metadata":{}},{"cell_type":"code","source":"plt.plot(history.history['accuracy'], label='accuracy')\nplt.plot(history.history['val_accuracy'], label = 'val_accuracy')\nplt.xlabel('Epoch')\nplt.ylabel('Accuracy')\nplt.legend(loc='lower right')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:17:38.279674Z","iopub.execute_input":"2022-07-29T22:17:38.279998Z","iopub.status.idle":"2022-07-29T22:17:38.464465Z","shell.execute_reply.started":"2022-07-29T22:17:38.279971Z","shell.execute_reply":"2022-07-29T22:17:38.463656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'], label='loss')\nplt.plot(history.history['val_loss'], label = 'val_loss')\nplt.xlabel('Epoch')\nplt.ylabel('Accuracy')\nplt.legend(loc='lower right')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:17:38.465643Z","iopub.execute_input":"2022-07-29T22:17:38.466087Z","iopub.status.idle":"2022-07-29T22:17:38.646101Z","shell.execute_reply.started":"2022-07-29T22:17:38.466059Z","shell.execute_reply":"2022-07-29T22:17:38.644982Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_loss, train_accuracy = model.evaluate(X_train, Y_train)\nprint('Train loss: ', train_loss)\nprint('Train accuracy: ', train_accuracy)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:17:38.647795Z","iopub.execute_input":"2022-07-29T22:17:38.648600Z","iopub.status.idle":"2022-07-29T22:17:42.628138Z","shell.execute_reply.started":"2022-07-29T22:17:38.648556Z","shell.execute_reply":"2022-07-29T22:17:42.627137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"validation_loss, validation_accuracy = model.evaluate(X_val, Y_val)\nprint('Validation loss: ', validation_loss)\nprint('Validation accuracy: ', validation_accuracy)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:17:42.629340Z","iopub.execute_input":"2022-07-29T22:17:42.630001Z","iopub.status.idle":"2022-07-29T22:17:44.143698Z","shell.execute_reply.started":"2022-07-29T22:17:42.629970Z","shell.execute_reply":"2022-07-29T22:17:44.142801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

9 | Prediction & submission

","metadata":{}},{"cell_type":"code","source":"# predict results\nresults = model.predict(X_test)\n\n# select the indix with the maximum probability\nresults = np.argmax(results,axis = 1)\nresults = pd.Series(results,name=\"Label\")","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:17:44.145006Z","iopub.execute_input":"2022-07-29T22:17:44.145302Z","iopub.status.idle":"2022-07-29T22:17:47.172529Z","shell.execute_reply.started":"2022-07-29T22:17:44.145275Z","shell.execute_reply":"2022-07-29T22:17:47.171601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.concat([pd.Series(range(1,28001),name = \"ImageId\"),results],axis = 1)\n\nsubmission.to_csv(\"submission.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-29T22:17:47.173717Z","iopub.execute_input":"2022-07-29T22:17:47.174006Z","iopub.status.idle":"2022-07-29T22:17:47.212256Z","shell.execute_reply.started":"2022-07-29T22:17:47.173980Z","shell.execute_reply":"2022-07-29T22:17:47.211512Z"},"trusted":true},"execution_count":null,"outputs":[]}]}