{"cells":[{"metadata":{},"cell_type":"markdown","source":"In this notebook, some image processing techniques have been shown step by step. These image processing techniques remove the noises from images and make the images more clear. Let's know about these techniques step by step and after that we will implement these techniques. I used these techniques for [bengali handwritten digit recognition challenge in 2018](https://www.kaggle.com/shawon10/bangla-handwritten-digit-recognizer/notebook) and got better accuracy.\n\n### Interpolation When Resizing Images\nImages can lose much important information due to resizing.  Inter-area interpolation is preferred method for image decimation. Interpolation works by using known data to estimate values at unknown points. We use inter-area interpolation after resizing images.\n\n### Removing Blur From Images\nWe have used Gaussian blur to add blur at first and then subtract the blurred image from the original image. Then we have added a weighted portion of the mask to get de-blurred image.\n\n                    g_mask(x,y)=f(x,y)-f'(x,y)                      \n              \n                    g(x,y)=f(x,y)+ k*g_mask(x,y)      \n\nHere f'(x,y) is the blurred image and k is a weight for generality.\n\n\n### Sharpening The Images\nThere are many filters for sharpening images. In this notebook, we have used the Laplacian filter. Our filer is a 3×3 matrix. Laplacian filter makes the images more sharp and clear.\n                                     \n              \n                    -1 -1 -1 \n                    -1  9 -1\n                    -1 -1 -1\n \n\n\n\n### Global Thresholding (Otsu Method)\nImages can be more specified by applying segmentation or thresholding.  Otsu’s method is used for global threshold selection method.\n"},{"metadata":{},"cell_type":"markdown","source":"## Implementation "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport cv2\nimport matplotlib.image as mpimg\nfrom matplotlib import pyplot as plt\nimport seaborn as sns\nimport albumentations as A\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/bengaliai-cv19/train.csv')\ntest_df = pd.read_csv('/kaggle/input/bengaliai-cv19/test.csv')\nclass_map_df = pd.read_csv('/kaggle/input/bengaliai-cv19/class_map.csv')\nsample_sub_df = pd.read_csv('/kaggle/input/bengaliai-cv19/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = train_df.drop(['grapheme'], axis=1, inplace=False)\ntrain_df[['grapheme_root', 'vowel_diacritic', 'consonant_diacritic']] = train_df[['grapheme_root', 'vowel_diacritic', 'consonant_diacritic']].astype('uint8')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Image Pre Processing by OpenCV Library\n\n"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"def imageProcessing(df, size=256):\n    imageProcessed = {}\n    \n    for i in (range(df.shape[0])):\n        #Interpolation\n        image = cv2.resize(df.loc[df.index[i]].values.reshape(137,236),(size,size),interpolation=cv2.INTER_AREA)\n        #Noise Removing\n        image=cv2.fastNlMeansDenoising(image)\n        #Gaussian Blur\n        gaussian_3 = cv2.GaussianBlur(image, (9,9), 10.0) #unblur\n        image = cv2.addWeighted(image, 1.5, gaussian_3, -0.5, 0, image)\n        #Laplacian Filter\n        kernel = np.array([[-1,-1,-1], [-1,9,-1], [-1,-1,-1]]) #filter\n        image = cv2.filter2D(image, -1, kernel)\n        #Otsu Method for Thresholding\n        ret,image = cv2.threshold(image, 128, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)\n        imageProcessed[df.index[i]] = image.reshape(-1)\n   \n    imageProcessed = pd.DataFrame(imageProcessed).T\n    return imageProcessed","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Preprocessed Train Image Data 1"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_0.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=4, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(imageProcessing(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(256, 256))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Preprocessed Train Image Data 2"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_1.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=3, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(imageProcessing(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(256, 256))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Preprocessed Train Image Data 3"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_2.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=3, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(imageProcessing(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(256, 256))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Preprocessed Train Image Data 4"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_3.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=3, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(imageProcessing(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(256, 256))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Cropping to (64*64) and Finding Region of Interest\n\nWe have used albumentations pre-processing library here instead of opencv"},{"metadata":{"trusted":true},"cell_type":"code","source":"def crop_images(df, size=64):\n    resized = {}\n    resize_size=64\n    \n    for i in range(df.shape[0]):\n        #image = cv2.resize(df.loc[df.index[i]].values.reshape(137,236),(size,size),None,fx=0.5,fy=0.5,interpolation=cv2.INTER_AREA)\n        image=df.loc[df.index[i]].values.reshape(137,236)\n        #Removing Blur\n        aug = A.GaussianBlur(p=1.0)\n        image = aug(image=image)['image']\n        #Noise Removing\n        #augNoise=A.MultiplicativeNoise(p=1.0)\n        #image = augNoise(image=image)['image']\n        #Removing Distortion\n        #augDist=A.ElasticTransform(sigma=50, alpha=1, alpha_affine=10, p=1.0)\n        #image = augDist(image=image)['image']\n        #Brightness\n        augBright=A.RandomBrightnessContrast(p=1.0)\n        image = augBright(image=image)['image']\n        #Thresholding\n        ret, image = cv2.threshold(image, 128, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)\n        contours, _ = cv2.findContours(image,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)[-2:]\n\n        idx = 0 \n        ls_xmin = []\n        ls_ymin = []\n        ls_xmax = []\n        ls_ymax = []\n        for cnt in contours:\n            idx += 1\n            x,y,w,h = cv2.boundingRect(cnt)\n            ls_xmin.append(x)\n            ls_ymin.append(y)\n            ls_xmax.append(x + w)\n            ls_ymax.append(y + h)\n        xmin = min(ls_xmin)\n        ymin = min(ls_ymin)\n        xmax = max(ls_xmax)\n        ymax = max(ls_ymax)\n\n        roi = image[ymin:ymax,xmin:xmax]\n        resized_roi = cv2.resize(roi, (resize_size, resize_size),interpolation=cv2.INTER_AREA)\n        #image=affine_image(image)\n        #image= crop_resize(image)\n        #image = cv2.resize(image,(size,size),interpolation=cv2.INTER_AREA)\n        #image=resize_image(image,(64,64))\n        #image = cv2.resize(image,(size,size),interpolation=cv2.INTER_AREA)\n        #gaussian_3 = cv2.GaussianBlur(image, (5,5), cv2.BORDER_DEFAULT) #unblur\n        #image = cv2.addWeighted(image, 1.5, gaussian_3, -0.5, 0, image)\n        #kernel = np.array([[-1,-1,-1], [-1,9,-1], [-1,-1,-1]]) #filter\n        #image = cv2.filter2D(image, -1, kernel)\n        #ret,image = cv2.threshold(image, 128, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)\n        resized[df.index[i]] = resized_roi.reshape(-1)\n    resized = pd.DataFrame(resized).T\n    return resized","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_0.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=4, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(crop_images(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(64, 64))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_1.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=4, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(crop_images(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(64, 64))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_2.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=4, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(crop_images(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(64, 64))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_images = pd.merge(pd.read_parquet(f'/kaggle/input/bengaliai-cv19/train_image_data_3.parquet'), train_df, on='image_id').drop(['image_id'], axis=1)\n\n# Visualize few samples of current training dataset\nfig, ax = plt.subplots(nrows=4, ncols=4, figsize=(20, 10))\ncount=0\nfor row in ax:\n    for col in row:\n        col.imshow(crop_images(train_images.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'], axis=1).iloc[[count]]).values.reshape(64, 64))\n        count += 1\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Thanks. Stay Tuned..."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}