{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-23T04:03:41.794288Z","iopub.execute_input":"2022-07-23T04:03:41.795286Z","iopub.status.idle":"2022-07-23T04:03:41.805748Z","shell.execute_reply.started":"2022-07-23T04:03:41.795238Z","shell.execute_reply":"2022-07-23T04:03:41.804990Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

MNIST DIGITS CLASSIFICATION

","metadata":{}},{"cell_type":"markdown","source":"![](https://thumbs.gfycat.com/AffectionateMemorableGreyhounddog-size_restricted.gif)","metadata":{}},{"cell_type":"markdown","source":"## **Index:**\n\n- [Importing Essential libraries](#import)\n- [Importing the data](#import)\n- [Explore Data Analysis(EDA)](#eda)\n - [Normalization of data](#normalize)\n- [Data Visualization](#visualize)\n- [Model Creation and Evalutation](#model)\n- [Submission](#sv)","metadata":{}},{"cell_type":"markdown","source":"
\n\n

PROBLEM STATMENT

","metadata":{}},{"cell_type":"markdown","source":"**Competition Description**\n\nMNIST (\"Modified National Institute of Standards and Technology\") is the de facto “hello world” dataset of computer vision. Since its release in 1999, this classic dataset of handwritten images has served as the basis for benchmarking classification algorithms. As new machine learning techniques emerge, MNIST remains a reliable resource for researchers and learners alike.\n\nIn this competition, your goal is to correctly identify digits from a dataset of tens of thousands of handwritten images. We’ve curated a set of tutorial-style kernels which cover everything from regression to neural networks. We encourage you to experiment with different algorithms to learn first-hand what works well and how techniques compare.","metadata":{}},{"cell_type":"markdown","source":"
\n

Importing relevant Libraries

","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\nimport os\nimport tensorflow as tf\nfrom glob import glob\nimport sys\nimport sklearn.metrics as metrics\nfrom sklearn.metrics import accuracy_score,classification_report,confusion_matrix\nimport seaborn as sns\nimport pandas as pd\nfrom sklearn.model_selection import train_test_split","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:41.807534Z","iopub.execute_input":"2022-07-23T04:03:41.808143Z","iopub.status.idle":"2022-07-23T04:03:41.817715Z","shell.execute_reply.started":"2022-07-23T04:03:41.808111Z","shell.execute_reply":"2022-07-23T04:03:41.816586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow.keras\nfrom tensorflow.keras.models import Sequential, Model, load_model\nfrom tensorflow.keras.applications.vgg16 import VGG16,preprocess_input\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator,load_img, img_to_array\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Dropout, Input, Flatten, Activation\nfrom tensorflow.keras.optimizers import Adam, SGD, RMSprop\nfrom tensorflow.keras.callbacks import Callback, EarlyStopping\nfrom tensorflow.keras.utils import to_categorical\nfrom sklearn.metrics import confusion_matrix\nfrom tensorflow.keras import backend as K\nfrom tensorflow.keras.layers import Input, Lambda, Dense, Flatten\nfrom tensorflow.keras.applications.inception_v3 import InceptionV3\nfrom tensorflow.keras.applications.inception_v3 import preprocess_input\nfrom tensorflow.keras.preprocessing import image\nfrom tensorflow.keras.layers import Input, Lambda, Dense, Flatten, BatchNormalization","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:41.987559Z","iopub.execute_input":"2022-07-23T04:03:41.987938Z","iopub.status.idle":"2022-07-23T04:03:41.997215Z","shell.execute_reply.started":"2022-07-23T04:03:41.987906Z","shell.execute_reply":"2022-07-23T04:03:41.996063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv(\"/kaggle/input/digit-recognizer/train.csv\")\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:41.998885Z","iopub.execute_input":"2022-07-23T04:03:41.999477Z","iopub.status.idle":"2022-07-23T04:03:44.546224Z","shell.execute_reply.started":"2022-07-23T04:03:41.999445Z","shell.execute_reply":"2022-07-23T04:03:44.545012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test = pd.read_csv(\"/kaggle/input/digit-recognizer/test.csv\")\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:44.549396Z","iopub.execute_input":"2022-07-23T04:03:44.549984Z","iopub.status.idle":"2022-07-23T04:03:45.870973Z","shell.execute_reply.started":"2022-07-23T04:03:44.549949Z","shell.execute_reply":"2022-07-23T04:03:45.869863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Explore Data Analysis(EDA)

\n","metadata":{}},{"cell_type":"markdown","source":"### **Splitting Data**","metadata":{}},{"cell_type":"markdown","source":"![](https://cdn-images-1.medium.com/max/1000/1*Owa2rsDG6Rwv1IM_RdsL3A.gif)","metadata":{}},{"cell_type":"code","source":"x_Train = train.drop(['label'], axis=1).astype('float32').values.reshape(-1,28,28,1)\ny_Train = train['label'].astype('int32')\nx_Test = test.values.reshape(-1,28,28,1)\ny_Test = test.values.reshape(-1,28,28,1)\nx_Train.shape, y_Train.shape, x_Test.shape, y_Test","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:45.872158Z","iopub.execute_input":"2022-07-23T04:03:45.872465Z","iopub.status.idle":"2022-07-23T04:03:46.065674Z","shell.execute_reply.started":"2022-07-23T04:03:45.872436Z","shell.execute_reply":"2022-07-23T04:03:46.064485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets check for null values\ntrain.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.069204Z","iopub.execute_input":"2022-07-23T04:03:46.069849Z","iopub.status.idle":"2022-07-23T04:03:46.125999Z","shell.execute_reply.started":"2022-07-23T04:03:46.069806Z","shell.execute_reply":"2022-07-23T04:03:46.124933Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.129138Z","iopub.execute_input":"2022-07-23T04:03:46.129448Z","iopub.status.idle":"2022-07-23T04:03:46.165849Z","shell.execute_reply.started":"2022-07-23T04:03:46.129419Z","shell.execute_reply":"2022-07-23T04:03:46.165011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sum(train.isnull().sum())","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.166928Z","iopub.execute_input":"2022-07-23T04:03:46.167635Z","iopub.status.idle":"2022-07-23T04:03:46.220117Z","shell.execute_reply.started":"2022-07-23T04:03:46.167602Z","shell.execute_reply":"2022-07-23T04:03:46.219166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sum(test.isnull().sum())","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.221440Z","iopub.execute_input":"2022-07-23T04:03:46.221996Z","iopub.status.idle":"2022-07-23T04:03:46.258711Z","shell.execute_reply.started":"2022-07-23T04:03:46.221952Z","shell.execute_reply":"2022-07-23T04:03:46.257591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets check for null values\ntest.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.259972Z","iopub.execute_input":"2022-07-23T04:03:46.260447Z","iopub.status.idle":"2022-07-23T04:03:46.298133Z","shell.execute_reply.started":"2022-07-23T04:03:46.260417Z","shell.execute_reply":"2022-07-23T04:03:46.297074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Data Pre-processing ⌛

","metadata":{}},{"cell_type":"markdown","source":"![](https://cdn.dribbble.com/users/2017910/screenshots/5102683/ai_trends_dribbble_shot.gif)","metadata":{}},{"cell_type":"markdown","source":"### ***Normalization of data*** ","metadata":{}},{"cell_type":"code","source":"print('Max value in Training data:', np.amax(x_Train), 'Min value in Training data:', np.amin(x_Train))\nprint('Max value in Testing data:', np.amax(x_Test), 'Min value in Testing data:', np.amin(x_Test))","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.301553Z","iopub.execute_input":"2022-07-23T04:03:46.302123Z","iopub.status.idle":"2022-07-23T04:03:46.379765Z","shell.execute_reply.started":"2022-07-23T04:03:46.302088Z","shell.execute_reply":"2022-07-23T04:03:46.378572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Data normalization\nx_Train= x_Train/255\nx_Test = x_Test/255","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.381382Z","iopub.execute_input":"2022-07-23T04:03:46.382089Z","iopub.status.idle":"2022-07-23T04:03:46.499684Z","shell.execute_reply.started":"2022-07-23T04:03:46.382047Z","shell.execute_reply":"2022-07-23T04:03:46.498447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### **Sample data instance**","metadata":{}},{"cell_type":"code","source":"index = np.random.randint(1, x_Train.shape[0]+1)\nplt.imshow(x_Train[index][:,:,0], )\nplt.title(y_Train[index]);\nprint('y_Train Label: ', y_Train[index])","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.501132Z","iopub.execute_input":"2022-07-23T04:03:46.501458Z","iopub.status.idle":"2022-07-23T04:03:46.686440Z","shell.execute_reply.started":"2022-07-23T04:03:46.501427Z","shell.execute_reply":"2022-07-23T04:03:46.685307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Model Creation and Evalutation

","metadata":{}},{"cell_type":"markdown","source":"![](https://miro.medium.com/max/1080/1*82NoZku9Ki3F-AM6U7TORA.gif)","metadata":{}},{"cell_type":"markdown","source":"**CNN Model**\n\nCNN is a type of deep learning model for processing data that has a grid pattern, such as images, which is inspired by the organization of animal visual cortex [13, 14] and designed to automatically and adaptively learn spatial hierarchies of features, from low- to high-level patterns.","metadata":{}},{"cell_type":"code","source":"model = tf.keras.models.Sequential([\ntf.keras.layers.Conv2D(32, (2, 2), activation='relu', input_shape=x_Train.shape[1:]),\ntf.keras.layers.Activation(\"relu\"),\ntf.keras.layers.MaxPooling2D(pool_size=(2, 2)),\ntf.keras.layers.Conv2D(32, (4, 4), activation='relu'),\ntf.keras.layers.MaxPooling2D(pool_size=(2, 2)),\ntf.keras.layers.Flatten(),\ntf.keras.layers.Dense(128, activation='relu'),\ntf.keras.layers.Dense(10, activation='softmax')\n])\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.687855Z","iopub.execute_input":"2022-07-23T04:03:46.690018Z","iopub.status.idle":"2022-07-23T04:03:46.753715Z","shell.execute_reply.started":"2022-07-23T04:03:46.689973Z","shell.execute_reply":"2022-07-23T04:03:46.752654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tf.keras.utils.plot_model(model)","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.757865Z","iopub.execute_input":"2022-07-23T04:03:46.758237Z","iopub.status.idle":"2022-07-23T04:03:46.900963Z","shell.execute_reply.started":"2022-07-23T04:03:46.758206Z","shell.execute_reply":"2022-07-23T04:03:46.899744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.compile(optimizer='adam', loss= 'sparse_categorical_crossentropy', metrics=['accuracy'])\nmodel.fit(x_Train, y_Train, epochs=10)","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:03:46.902893Z","iopub.execute_input":"2022-07-23T04:03:46.903839Z","iopub.status.idle":"2022-07-23T04:05:56.288196Z","shell.execute_reply.started":"2022-07-23T04:03:46.903665Z","shell.execute_reply":"2022-07-23T04:05:56.287108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Submission file

\n","metadata":{}},{"cell_type":"markdown","source":"![](https://dashtechinc.com/wp-content/uploads/2020/01/Machine-Learning-Hero-Banner.png)","metadata":{}},{"cell_type":"code","source":"y_Test = model.predict(x_Test)\n\ny_Test = np.argmax(y_Test, axis=1)\nplt.imshow(x_Test[5])\nf'Predicted label is: {y_Test[5]}'","metadata":{"execution":{"iopub.status.busy":"2022-07-23T04:05:56.289909Z","iopub.execute_input":"2022-07-23T04:05:56.290190Z","iopub.status.idle":"2022-07-23T04:05:58.910099Z","shell.execute_reply.started":"2022-07-23T04:05:56.290164Z","shell.execute_reply":"2022-07-23T04:05:58.909198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

YOUR FEEDBACKS IS SO VALUABLE FOR ME

","metadata":{}},{"cell_type":"markdown","source":"![](https://st2.depositphotos.com/1006899/7664/i/600/depositphotos_76643019-stock-photo-thank-you-words.jpg)\n","metadata":{}}]}