{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Digit Recognizer

\n","metadata":{"id":"6249cd9e-8f87-424b-bb2c-3c637e23f66b"}},{"cell_type":"markdown","source":"

Competition Guidelines

\n\n\n> The dataset is called MNIST and refers to handwritten digit recognition.\nThe dataset provides 70,000 images (28x28 pixels) of handwritten digits (1 digit per image).The goal is to write an algorithm that detects which digit is written. Since there are only 10 digits (0, 1, 2, 3, 4, 5, 6, 7, 8, 9), this is a classification problem with 10 classes.Our goal would be to build a neural network with 2 hidden layer\n![](https://www.researchgate.net/profile/Hugo-Larochelle/publication/200744481/figure/fig1/AS:668968306098181@1536505881710/Samples-from-the-MNIST-digit-recognition-data-set-Here-a-black-pixel-corresponds-to-an.png)","metadata":{"id":"17prH9dyWFH1"}},{"cell_type":"markdown","source":"

The objective is to identify digits from a dataset of tens of thousands of hand written images.

\n\n>
In this model, Convolutional Neural Network model has been developed using TensorFlow framework to recognition of handwritten digit.
\n\n\n\n>
Sequential Keras model which has two layers Convolutional2D and MaxPooling2D layer has been used. MaxPooling layer act as a sort of downsampling using max values instead of averaging. The last layer has a dense layer.
\n","metadata":{"id":"2d957986-8caa-476e-8f6a-84ea5fbe69fb"}},{"cell_type":"markdown","source":"

Importing Libraries

","metadata":{"id":"b6987ce6-2e93-4360-a12a-ff6c5cb7d17e"}},{"cell_type":"markdown","source":"

Importing important libraries

\n\n\n\n\n\n\n","metadata":{"id":"iDMKB7ImCuSK"}},{"cell_type":"markdown","source":"

Importing Libraries

\n\n","metadata":{"id":"zciKk65SRpuD"}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sns\nimport tensorflow as tf","metadata":{"id":"Cp-RsBnpGoOD","execution":{"iopub.status.busy":"2022-08-06T01:12:30.014094Z","iopub.execute_input":"2022-08-06T01:12:30.015022Z","iopub.status.idle":"2022-08-06T01:12:30.027740Z","shell.execute_reply.started":"2022-08-06T01:12:30.014965Z","shell.execute_reply":"2022-08-06T01:12:30.026084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train=pd.read_csv(\"../input/digit-recognizer/train.csv\")\ntest=pd.read_csv(\"../input/digit-recognizer/test.csv\")","metadata":{"id":"ox5BU49cRR6S","execution":{"iopub.status.busy":"2022-08-06T01:12:30.044874Z","iopub.execute_input":"2022-08-06T01:12:30.046126Z","iopub.status.idle":"2022-08-06T01:12:34.444152Z","shell.execute_reply.started":"2022-08-06T01:12:30.046067Z","shell.execute_reply":"2022-08-06T01:12:34.442719Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Overview

","metadata":{"id":"EJunEkt-R6b_"}},{"cell_type":"code","source":"print(train.shape)\nprint(test.shape)","metadata":{"id":"i_Y_IZD8iVhk","outputId":"7054b9f0-c1d1-4102-e8b1-5656a3ebc6bd","execution":{"iopub.status.busy":"2022-08-06T01:12:34.447478Z","iopub.execute_input":"2022-08-06T01:12:34.448011Z","iopub.status.idle":"2022-08-06T01:12:34.455895Z","shell.execute_reply.started":"2022-08-06T01:12:34.447961Z","shell.execute_reply":"2022-08-06T01:12:34.454548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Normalization

","metadata":{"id":"QCXC2vEkSG0c"}},{"cell_type":"code","source":"#Normalizing the value of pixcel of image\ntrain_x=train.drop(['label'],axis=1).astype('int32')\ntrain_y=train['label'].astype('float32')\ntest_x=test.astype('float32')\ntrain_x.shape,train_y.shape,test_x.shape\n#train=train.astype('float32')/255.0\n#test=test.astype('float32')/255.0","metadata":{"id":"Kf7-xPGSixFI","outputId":"204ba9d3-8cb0-4361-aff1-ee7217a9eee6","execution":{"iopub.status.busy":"2022-08-06T01:12:34.457905Z","iopub.execute_input":"2022-08-06T01:12:34.458682Z","iopub.status.idle":"2022-08-06T01:12:34.674622Z","shell.execute_reply.started":"2022-08-06T01:12:34.458614Z","shell.execute_reply":"2022-08-06T01:12:34.673169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_x= train_x.values.reshape(-1,28,28,1)\ntrain_x=train_x/255.0\ntest_x=test_x.values.reshape(-1,28,28,1)\ntest_x=test_x/255.0\ntrain_x.shape,test_x.shape ","metadata":{"id":"6x8M4jgom_C5","outputId":"79d77db0-6676-4490-a08b-018d73eeeb83","execution":{"iopub.status.busy":"2022-08-06T01:12:34.678766Z","iopub.execute_input":"2022-08-06T01:12:34.679309Z","iopub.status.idle":"2022-08-06T01:12:34.810520Z","shell.execute_reply.started":"2022-08-06T01:12:34.679258Z","shell.execute_reply":"2022-08-06T01:12:34.809537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Visualization

","metadata":{"id":"u2DjfA_6SXs3"}},{"cell_type":"code","source":"fig=plt.figure(figsize=(5,3))\nfor i in range(15):\n ax=fig.add_subplot(2,10,i+1,xticks=[],yticks=[])\n ax.imshow(np.squeeze(train_x[i]),cmap='gray')\n ax.set_title(train_y[i])","metadata":{"id":"k7_BKwQWjJjL","outputId":"33fb3576-5f4b-4703-d719-81eb7028735a","execution":{"iopub.status.busy":"2022-08-06T01:12:34.811883Z","iopub.execute_input":"2022-08-06T01:12:34.813057Z","iopub.status.idle":"2022-08-06T01:12:35.445275Z","shell.execute_reply.started":"2022-08-06T01:12:34.813015Z","shell.execute_reply":"2022-08-06T01:12:35.443842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Verify the normalization work\nplt.matshow(train_x[0,:,:,0],cmap='gray')\nplt.colorbar()\nplt.show()","metadata":{"id":"Pti_sxfez32K","outputId":"c202f6b3-3385-4002-f90c-93ab163a3e96","execution":{"iopub.status.busy":"2022-08-06T01:12:35.446835Z","iopub.execute_input":"2022-08-06T01:12:35.447235Z","iopub.status.idle":"2022-08-06T01:12:35.820881Z","shell.execute_reply.started":"2022-08-06T01:12:35.447200Z","shell.execute_reply":"2022-08-06T01:12:35.819533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

One Hot Enconder

","metadata":{"id":"PSghb8uU37qr"}},{"cell_type":"code","source":"train_y=tf.keras.utils.to_categorical(train_y,10)\ntrain_y.shape","metadata":{"id":"1dEdfDdW4HHN","outputId":"7eca169c-cef9-467d-b755-aa7559c2f0d3","execution":{"iopub.status.busy":"2022-08-06T01:12:35.822792Z","iopub.execute_input":"2022-08-06T01:12:35.823581Z","iopub.status.idle":"2022-08-06T01:12:35.833479Z","shell.execute_reply.started":"2022-08-06T01:12:35.823527Z","shell.execute_reply":"2022-08-06T01:12:35.832261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#One hot encoder of same label\nprint(train_y[0:5,:])","metadata":{"id":"4srC9GY64f6o","outputId":"019d974f-d6fd-4541-fe14-a6f57dc09865","execution":{"iopub.status.busy":"2022-08-06T01:12:35.835291Z","iopub.execute_input":"2022-08-06T01:12:35.836054Z","iopub.status.idle":"2022-08-06T01:12:35.845783Z","shell.execute_reply.started":"2022-08-06T01:12:35.836006Z","shell.execute_reply":"2022-08-06T01:12:35.844523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Defining the model","metadata":{"id":"QZl8JtQS4v1l"}},{"cell_type":"code","source":"model=tf.keras.models.Sequential([tf.keras.layers.Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)),\n tf.keras.layers.Conv2D(32,(3,3),activation='relu'),\n tf.keras.layers.MaxPooling2D(2,2),\n tf.keras.layers.Conv2D(64,(3,3),activation='relu',padding='Same'),\n tf.keras.layers.Conv2D(64,(3,3),activation='relu',padding='Same'),\n tf.keras.layers.MaxPool2D(pool_size=(2,2),strides=(2,2)),\n tf.keras.layers.Dropout(0.25),\n tf.keras.layers.Conv2D(64,(3,3),activation='relu',padding='Same'),\n tf.keras.layers.Conv2D(64,(3,3),activation='relu',padding='Same'),\n tf.keras.layers.MaxPool2D(pool_size=(2,2),strides=(2,2)),\n tf.keras.layers.Dropout(0.25),\n tf.keras.layers.Flatten(),\n tf.keras.layers.Dense(256,activation='relu'),\n tf.keras.layers.Dense(256,activation='relu'),\n tf.keras.layers.Dropout(0.50),\n tf.keras.layers.Dense(10,activation='softmax')\n ])\nmodel.summary()","metadata":{"id":"EvhFMfkX5CSE","outputId":"c7cd76b7-8660-4594-e4e1-dff3f4696c11","execution":{"iopub.status.busy":"2022-08-06T01:12:35.847934Z","iopub.execute_input":"2022-08-06T01:12:35.848713Z","iopub.status.idle":"2022-08-06T01:12:35.975125Z","shell.execute_reply.started":"2022-08-06T01:12:35.848664Z","shell.execute_reply":"2022-08-06T01:12:35.973575Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tf.keras.utils.plot_model(model,'model.jpg',show_shapes=True)","metadata":{"id":"xr2SIpfB9JoO","outputId":"573b88fb-b98d-4533-8866-bd575a7cfaee","execution":{"iopub.status.busy":"2022-08-06T01:12:35.978293Z","iopub.execute_input":"2022-08-06T01:12:35.979199Z","iopub.status.idle":"2022-08-06T01:12:36.850641Z","shell.execute_reply.started":"2022-08-06T01:12:35.979112Z","shell.execute_reply":"2022-08-06T01:12:36.849110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class myCallback(tf.keras.callbacks.Callback):\n def on_epoch_end(self,epoch,logs={}):\n if(logs.get('accuracy')>0.999):\n print(\"\\nTraining is cancelled because you reached 99.9%\")\n self.model.stop_training = True\n\ncallbacks=myCallback()","metadata":{"id":"z0OEOLKoANdC","execution":{"iopub.status.busy":"2022-08-06T01:12:36.853247Z","iopub.execute_input":"2022-08-06T01:12:36.853764Z","iopub.status.idle":"2022-08-06T01:12:36.863259Z","shell.execute_reply.started":"2022-08-06T01:12:36.853710Z","shell.execute_reply":"2022-08-06T01:12:36.861764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005,\n beta_1=0.9,\n beta_2=0.999,\n epsilon=1e-07,\n name='Adam')\nmodel.compile(optimizer=Optimizer,loss='categorical_crossentropy',metrics=['accuracy'])\nhistory=model.fit(train_x,train_y,batch_size=50,epochs=20,callbacks=[callbacks])","metadata":{"id":"Ll96Jh4N-EfY","outputId":"c78de0e2-8023-48a5-ebbb-4bb72ca49288","execution":{"iopub.status.busy":"2022-08-06T01:12:36.864920Z","iopub.execute_input":"2022-08-06T01:12:36.865320Z","iopub.status.idle":"2022-08-06T01:30:38.874965Z","shell.execute_reply.started":"2022-08-06T01:12:36.865285Z","shell.execute_reply":"2022-08-06T01:30:38.873967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Model Curves

","metadata":{"id":"N9681QFPB8dv"}},{"cell_type":"code","source":"print(history.params)","metadata":{"id":"HHL0wbxBCIhc","outputId":"e6f233ad-3ec7-4de7-ed98-45ce95413713","execution":{"iopub.status.busy":"2022-08-06T01:30:38.876275Z","iopub.execute_input":"2022-08-06T01:30:38.876670Z","iopub.status.idle":"2022-08-06T01:30:38.883290Z","shell.execute_reply.started":"2022-08-06T01:30:38.876635Z","shell.execute_reply":"2022-08-06T01:30:38.882016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nplt.plot(history.history['accuracy'],label='training set')\n#plt.plot(history.history['loss'],label='validation set')\nplt.ylabel('Accuracy')\nplt.xlabel('Epoch')\nplt.legend(['Training data','validation data'],loc='lower right')\nplt.legend()","metadata":{"id":"qIyz_fDCFUwE","outputId":"5327820a-f0d5-4f50-8e5c-8298a1a907bd","execution":{"iopub.status.busy":"2022-08-06T01:30:38.884534Z","iopub.execute_input":"2022-08-06T01:30:38.884882Z","iopub.status.idle":"2022-08-06T01:30:39.135669Z","shell.execute_reply.started":"2022-08-06T01:30:38.884849Z","shell.execute_reply":"2022-08-06T01:30:39.134426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nplt.plot(history.history['loss'],label='validation set')\nplt.ylabel('Accuracy')\nplt.xlabel('Epoch')\nplt.legend(['Training data','validation data'],loc='lower right')\nplt.legend()","metadata":{"outputId":"eceae58f-b5a2-4443-e64d-3892f2c43f5e","id":"Jv36UUtzssQQ","execution":{"iopub.status.busy":"2022-08-06T01:30:39.137248Z","iopub.execute_input":"2022-08-06T01:30:39.138168Z","iopub.status.idle":"2022-08-06T01:30:39.373734Z","shell.execute_reply.started":"2022-08-06T01:30:39.138119Z","shell.execute_reply":"2022-08-06T01:30:39.372513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results=model.predict(test_x)\nresults=np.argmax(results,axis=1)\nresults=pd.Series(results,name='Label')\nprint(results)","metadata":{"id":"7t9SitwVwg1l","execution":{"iopub.status.busy":"2022-08-06T01:53:56.921172Z","iopub.execute_input":"2022-08-06T01:53:56.922380Z","iopub.status.idle":"2022-08-06T01:54:08.948672Z","shell.execute_reply.started":"2022-08-06T01:53:56.922325Z","shell.execute_reply":"2022-08-06T01:54:08.947395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission=pd.concat([pd.Series(range(1,28001),name='ImageId'),results],axis=1)\nsubmission.to_csv(\"submission.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-06T18:01:14.657157Z","iopub.execute_input":"2022-08-06T18:01:14.657635Z","iopub.status.idle":"2022-08-06T18:01:14.752799Z","shell.execute_reply.started":"2022-08-06T18:01:14.657516Z","shell.execute_reply":"2022-08-06T18:01:14.751117Z"},"trusted":true},"execution_count":null,"outputs":[]}]}