{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# import libraries\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# import data visualization\nimport matplotlib.pyplot as plt\nfrom matplotlib.font_manager import FontProperties\n\n# import image processing and computer vision\nimport cv2 as cv\n\n# import data augmentation\nimport albumentations as albu","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"![image](https://github.com/Lexie88rus/Bengali_AI_Competition/raw/master/assets/samples.png)"},{"metadata":{},"cell_type":"markdown","source":"# Bengali.AI EDA"},{"metadata":{},"cell_type":"markdown","source":"## Introduction\n\n__Bengali is the 5th most spoken language in the world__ with hundreds of million of speakers. It’s the official language of Bangladesh and the second most spoken language in India. Considering its reach, there’s significant business and educational interest in developing AI that can optically recognize images of the language handwritten. This challenge hopes to improve on approaches to Bengali recognition.\n\nOptical character recognition is particularly challenging for Bengali. While Bengali has 49 letters (to be more specific 11 vowels and 38 consonants) in its [alphabet](https://en.wikipedia.org/wiki/Bengali_alphabet), there are also 18 potential [diacritics](https://en.wikipedia.org/wiki/Diacritic), or [accents](https://www.computerhope.com/jargon/a/accent.htm). This means that there are many more graphemes, or the smallest units in a written language. The added complexity results in ~13,000 different grapheme variations (compared to English’s 250 graphemic units)."},{"metadata":{},"cell_type":"markdown","source":"## Dataset Statistics"},{"metadata":{},"cell_type":"markdown","source":"The list of files:"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"! ls ../input/bengaliai-cv19/","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# setup the input data folder\nDATA_PATH = '../input/bengaliai-cv19/'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Load the labels and explore target classes and distribution of images between them:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# load the dataframes with labels\ntrain_labels = pd.read_csv(DATA_PATH + 'train.csv')\ntest_labels = pd.read_csv(DATA_PATH + 'test.csv')\nclass_map = pd.read_csv(DATA_PATH + 'class_map.csv')\nsample_submission = pd.read_csv(DATA_PATH + 'sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('There are {} train images.'.format(len(train_labels)))\nprint('There are {} test images.'.format(len(test_labels.groupby(by=['image_id']).count())))\nprint('There are {} classes for all 3 targets.'.format(len(class_map)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the number of classes for each target\nn_classes = class_map.groupby(by=['component_type']).count()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# source: https://stackoverflow.com/questions/28931224/adding-value-labels-on-a-matplotlib-bar-chart\ndef add_value_labels(ax, spacing=5):\n    \"\"\"Add labels to the end of each bar in a bar chart.\n\n    Arguments:\n        ax (matplotlib.axes.Axes): The matplotlib object containing the axes\n            of the plot to annotate.\n        spacing (int): The distance between the labels and the bars.\n    \"\"\"\n\n    # For each bar: Place a label\n    for rect in ax.patches:\n        # Get X and Y placement of label from rect.\n        y_value = rect.get_height()\n        x_value = rect.get_x() + rect.get_width() / 2\n\n        # Number of points between bar and label. Change to your liking.\n        space = spacing\n        # Vertical alignment for positive values\n        va = 'bottom'\n\n        # If value of bar is negative: Place label below bar\n        if y_value < 0:\n            # Invert space to place label below\n            space *= -1\n            # Vertically align label at top\n            va = 'top'\n\n        # Use Y value as label and format number with one decimal place\n        label = \"{:.0f}\".format(y_value)\n\n        # Create annotation\n        ax.annotate(\n            label,                      # Use `label` as label\n            (x_value, y_value),         # Place label at end of the bar\n            xytext=(0, space),          # Vertically shift label by `space`\n            textcoords=\"offset points\", # Interpret `xytext` as offset in points\n            ha='center',                # Horizontally center label\n            va=va)                      # Vertically align label differently for\n                                        # positive and negative values.","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# plot bar chart with the number of classes to predict\nfig, ax = plt.subplots(1,1,figsize=(7,7))\nplt.bar(range(0,len(n_classes.label.values)), n_classes.label.values, color=['#41EAD4', '#F71735', '#FF9F1C'])\nplt.xticks(range(0,len(n_classes.label.values)), n_classes.index.values)\nplt.title('Number of target classes', fontsize=16)\nadd_value_labels(ax)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's look at the number of images for each class:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the number of images for each grapheme_root\ngrapheme_root_img = train_labels.groupby(by=['grapheme_root']).count().reset_index()[['grapheme_root', 'image_id']]\\\n.sort_values(by=['image_id'], ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# plot bar chart with the number of images per each grapheme root\nn_classes = len(grapheme_root_img)\nfig, ax = plt.subplots(1,1,figsize=(20,7))\nplt.bar(range(0, n_classes), grapheme_root_img.image_id.values, color='#F71735')\nplt.xticks(range(0, n_classes), grapheme_root_img.grapheme_root.values, rotation=90)\nplt.title('Number of images per grapheme root', fontsize=16)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# plot bar chart with the top popular grapheme root class\nn_classes = 10\nfig, ax = plt.subplots(1,1,figsize=(10,7))\nplt.bar(range(0, n_classes), grapheme_root_img.image_id.values[:n_classes], color='#F71735')\nlabels = [class_map[class_map.label == val].component.values[0] for val in grapheme_root_img.grapheme_root.values[:n_classes]]\nprop = FontProperties()\nprop.set_file('../input/bengaliaiutils/kalpurush.ttf')\nplt.xticks(range(0, n_classes), labels, fontproperties=prop, fontsize=24)\nplt.title('Number of images per grapheme root (TOP-10 popular)', fontsize=16)\nadd_value_labels(ax)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# plot bar chart with the top popular grapheme root class\nn_classes = 10\nfig, ax = plt.subplots(1,1,figsize=(10,7))\nplt.bar(range(0, n_classes), grapheme_root_img.image_id.values[-n_classes:], color='#F71735')\nlabels = [class_map[class_map.label == val].component.values[0] for val in grapheme_root_img.grapheme_root.values[-n_classes:]]\nprop = FontProperties()\nprop.set_file('../input/bengaliaiutils/kalpurush.ttf')\nplt.xticks(range(0, n_classes), labels, fontproperties=prop, fontsize=24)\nplt.title('Number of images per grapheme root (TOP-10 least popular)', fontsize=16)\nadd_value_labels(ax)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the number of images for each vowel_diacritic\nvowel_diacritic_img = train_labels.groupby(by=['vowel_diacritic']).count().reset_index()[['vowel_diacritic', 'image_id']]\\\n.sort_values(by=['image_id'], ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# plot bar chart with the number of images per each vowel_diacritic\nn_classes = len(vowel_diacritic_img)\nfig, ax = plt.subplots(1,1,figsize=(10,7))\nplt.bar(range(0, n_classes), vowel_diacritic_img.image_id.values, color='#F71735')\nlabels = [class_map[class_map.label == val].component.values[0] for val in vowel_diacritic_img.vowel_diacritic.values]\nplt.xticks(range(0, n_classes), labels, fontproperties=prop, fontsize=24)\nplt.title('Number of images per vowel diacritic', fontsize=16)\nadd_value_labels(ax)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the number of images for each consonant_diacritic\nconsonant_diacritic_img = train_labels.groupby(by=['consonant_diacritic']).count().reset_index()[['consonant_diacritic', 'image_id']]\\\n.sort_values(by=['image_id'], ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# plot bar chart with the number of images per each consonant_diacritic\nn_classes = len(consonant_diacritic_img)\nfig, ax = plt.subplots(1,1,figsize=(10,7))\nplt.bar(range(0, n_classes), consonant_diacritic_img.image_id.values, color='#F71735')\nlabels = [class_map[class_map.label == val].component.values[0] for val in consonant_diacritic_img.consonant_diacritic.values]\nplt.xticks(range(0, n_classes), labels, fontproperties=prop, fontsize=24)\nplt.title('Number of images per consonant diacritic', fontsize=16)\nadd_value_labels(ax)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__Conclusions:__\n* We have a __lot of target classes__. There are 168 classes for one of the target variables.\n* The __distribution of images between the classes varies greately__, especially for `grapheme root` and `consonant diacritic`"},{"metadata":{},"cell_type":"markdown","source":"## Visualize Sample Images"},{"metadata":{},"cell_type":"markdown","source":"Load the first data file ([parquet format wiki page](https://en.wikipedia.org/wiki/Apache_Parquet)) and visualize sample image:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# load the 1st data file\ntrain_df_1 = pd.read_parquet(DATA_PATH + 'train_image_data_0.parquet')\ntrain_df_1.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# setup image hight and width\nHEIGHT = 137\nWIDTH = 236\n\n# function to visualize the images\ndef visualize_image(df, image_id):\n    '''\n    Helper function to visualize the image from dataframe by image_id\n    '''\n    img = df[df.image_id == image_id].values[:, 1:].reshape(HEIGHT, WIDTH)\n    plt.imshow(img.astype(float), cmap='gray')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_image(train_df_1, 'Train_7')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_image(train_df_1, 'Train_11')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that:\n* there is no noise on the images,\n* graphemes are not centered exactly."},{"metadata":{},"cell_type":"markdown","source":"Load all images:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def load_images():\n    '''\n    Helper function to load all train and test images\n    '''\n    train_list = []\n    for i in range(0,4):\n        train_list.append(pd.read_parquet(DATA_PATH + 'train_image_data_{}.parquet'.format(i)))\n    train = pd.concat(train_list, ignore_index=True)\n    \n    test_list = []\n    for i in range(0,4):\n        test_list.append(pd.read_parquet(DATA_PATH + 'test_image_data_{}.parquet'.format(i)))\n    test = pd.concat(test_list, ignore_index=True)\n    \n    return train, test\n\n# function to visualize the images\ndef visualize_image_label(df, image_id, train_labels):\n    '''\n    Helper function to visualize the image from dataframe by image_id\n    '''\n    grapheme = train_labels[train_labels.image_id == image_id].grapheme.values[0]\n    img = df[df.image_id == image_id].values[:, 1:].reshape(HEIGHT, WIDTH)\n    plt.imshow(img.astype(float), cmap='gray')\n    \n    prop = FontProperties()\n    prop.set_file('../input/bengaliaiutils/kalpurush.ttf')\n    \n    plt.title(grapheme, fontproperties=prop, fontsize=20)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# load the images\ntrain, test = load_images()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize an image with label\nvisualize_image_label(train_df_1, 'Train_11', train_labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize an image with label\nvisualize_image_label(train_df_1, 'Train_9', train_labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize an image with label\nvisualize_image_label(train_df_1, 'Train_56', train_labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This grapheme didn't even fit in the image.[](http://)"},{"metadata":{},"cell_type":"markdown","source":"Let's visualize sample images for the most popular graphemes:"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def class_to_component(label):\n    '''\n    Helper function to get the component by label\n    '''\n    return class_map[class_map.label == label].component.values[0]\n\ndef visualize_sample_graphemes(classes_list, label_type = 'grapheme_root', img_per_class=5):\n    '''\n    Helper function to plot the sample images for a list of class labels of specified type\n    '''\n    img_per_class = img_per_class+1\n    fig, axs = plt.subplots(len(classes_list), img_per_class, figsize=(20,10))\n    for row in range(len(classes_list)):\n        # get sample ids for the class from the list\n        all_img_ids = train_labels[train_labels[label_type] == classes_list[row]].image_id.values\n        # get random indices\n        np.random.seed(123)\n        idx = np.random.randint(len(all_img_ids), size=img_per_class)\n        for col in range(img_per_class):\n            if col > 0:\n                # get the id of the image\n                img_id = all_img_ids[idx[col-1]]\n                # get the image by id\n                img = train[train.image_id == img_id].values[:, 1:].reshape(HEIGHT, WIDTH).astype(float)\n                axs[row, col].imshow(img, cmap='gray')\n                axs[row, col].axis('off')\n            else:\n                # plot the component as text\n                component = class_to_component(classes_list[row])\n                axs[row, col].text(0, 0, component, fontproperties=prop, fontsize=48)\n                axs[row, col].axis('off')\n    plt.suptitle('Sample Images for {}'.format(label_type), fontsize=16)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# get top grapheme_root classes\nn_classes = 10\ntop_grapheme_root = grapheme_root_img.grapheme_root.values[:n_classes]\n# visualize sample image\nvisualize_sample_graphemes(top_grapheme_root, label_type = 'grapheme_root', img_per_class=10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize sample images for vowel_diacritic\nvisualize_sample_graphemes(vowel_diacritic_img.vowel_diacritic.values, label_type = 'vowel_diacritic', img_per_class=10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize sample images for consonant_diacritic\nvisualize_sample_graphemes(consonant_diacritic_img.consonant_diacritic.values, label_type = 'consonant_diacritic', img_per_class=10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__Conclusions:__\n* The background is not clear on all of the images. That is why the initial preprocessing is required to threshold the images and eliminate the noise."},{"metadata":{},"cell_type":"markdown","source":"## Preprocess the Images\n\nLet's try to preprocess the images by removing the background and noise with thresholding in OpenCV. I chose the adaptive thresholding with [Otsu's method](https://en.wikipedia.org/wiki/Otsu%27s_method)."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import PIL\ndef threshold_image(img):\n    '''\n    Helper function for thresholding the images\n    '''\n    gray = PIL.Image.fromarray(np.uint8(img), 'L')\n    ret,th = cv.threshold(np.array(gray),0,255,cv.THRESH_BINARY+cv.THRESH_OTSU)\n    return th\n\ndef visualize_thresholded(df, image_id, train_labels):\n    '''\n    Function to visualize raw and thresholded images\n    INPUT:\n        df - train images dataframe\n        image_id - id of the image\n        train_labels - dataframe containing the training labels\n    '''\n    grapheme = train_labels[train_labels.image_id == image_id].grapheme.values[0]\n    img = df[df.image_id == image_id].values[:, 1:].reshape(HEIGHT, WIDTH)\n    img_th = threshold_image(img)\n    \n    fig, axs = plt.subplots(1,2,figsize=(7,4))\n    \n    axs[0].imshow(img.astype(float), cmap='gray')\n    axs[0].set_title('original image')\n    \n    axs[1].imshow(img_th.astype(float), cmap='gray')\n    axs[1].set_title('thresholded image')\n    \n    prop = FontProperties()\n    prop.set_file('../input/bengaliaiutils/kalpurush.ttf')\n    \n    plt.suptitle(grapheme, fontproperties=prop, fontsize=20)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_thresholded(train_df_1, 'Train_56', train_labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_thresholded(train_df_1, 'Train_100', train_labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_thresholded(train_df_1, 'Train_555', train_labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data Augmentation\n\nData augmentation is essential when building deep learning models. Good augmentation pipeline can improve the performance of the model dramatically! I suggest to try the following augmentations:\n* random crop and resize;\n* slight rotations;\n* slight distortions.\n\nI am using the [albumentations](https://github.com/albumentations-team/albumentations) library. This library is efficient and easy to use with most deep learning frameworks."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def apply_augmentation(img, augmentation):\n    '''\n    Helper function to apply the augmentation\n    '''\n    img_aug = augmentation(image=img)['image']\n    return img_aug\n\ndef demonstrate_augmentation(aug, n_rows=5, n_cols=5, title=''):\n    '''\n    Helper function to plot the augmented images\n    INPUT:\n        aug - augmentation\n        n_rows, n_cols - number of rows and cols for image grid\n        title - title for a plot (for example, the name of visualization)\n    '''\n    # calculate the number of sample images\n    n_img = n_rows * n_cols\n    \n    # get the random ids of sample images\n    all_img_ids = train_labels.image_id.values\n    np.random.seed(123)\n    idx = np.random.randint(len(all_img_ids), size=n_img)\n    image_ids = [all_img_ids[i] for i in idx]\n    \n    fig, axs = plt.subplots(n_rows, n_cols, figsize=(7,5))\n    for row in range(n_rows):\n        for col in range(n_cols):\n            count = row + col*n_cols\n            # get the image by id\n            img = train[train.image_id == image_ids[count]].values[:, 1:].reshape(HEIGHT, WIDTH).astype(float)\n            \n            # preprocess the image\n            img_th = threshold_image(img)\n            \n            # apply the augmentation\n            img_aug = apply_augmentation(img_th, aug)\n            \n            # plot the image\n            axs[row, col].imshow(img_aug, cmap='gray')\n            axs[row, col].axis('off')\n            \n    plt.suptitle('Augmentation demo {}'.format(title), fontsize=16)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Crop a random part of the input and rescale it to some size.\naug = albu.RandomSizedCrop(min_max_height=(int(HEIGHT // 1.1), HEIGHT), height = HEIGHT, width = WIDTH, p=1.0)\n\n# demonstrate the augmentation on sample images\ndemonstrate_augmentation(aug, n_rows=5, n_cols=5, title='RandomSizedCrop')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Rotate the input by an angle selected randomly from the uniform distribution.\naug = albu.Rotate(limit=5, p=1.0)\n\n# demonstrate the augmentation on sample images\ndemonstrate_augmentation(aug, n_rows=5, n_cols=5, title='Rotate')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can even create a pipeline of these augmentations!"},{"metadata":{"trusted":true},"cell_type":"code","source":"def pipeline_augmentations(p=.5):\n    '''\n    Function returns the pipeline of augmentations\n    '''\n    return albu.Compose([\n        # compose the random cropping and random rotation\n        albu.RandomSizedCrop(min_max_height=(int(HEIGHT // 1.1), HEIGHT), height = HEIGHT, width = WIDTH, p=0.5),\n        albu.Rotate(limit=5, p=0.5)\n    ], p=p)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# demonstrate the augmentation pipeline on sample images\ndemonstrate_augmentation(pipeline_augmentations(p=.9), n_rows=5, n_cols=5, title='Rotate')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Research Papers\nYou might find useful these research papers/repositories:\n\n1. [Bengali Handwritten Character Classification using Transfer Learning on Deep Convolutional Neural Network](https://arxiv.org/html/1902.11133), [repo](https://github.com/swagato-c/bangla-hwcr-present): In this paper, we propose a solution which uses state-of-the-art techniques in Deep Learning to tackle the problem of Bengali Handwritten Character Recognition ( HCR ). Our method uses lesser iterations to train than most other comparable methods. We employ Transfer Learning on ResNet 50, a state-of-the-art deep Convolutional Neural Network Model, pretrained on ImageNet dataset. We also use other techniques like a modified version of One Cycle Policy, varying the input image sizes etc. to ensure that our training occurs fast. We use the BanglaLekha-Isolated Dataset for evaluation of our technique which consists of 84 classes (50 Basic, 10 Numerals and 24 Compound Characters). We are able to achieve 96.12% accuracy in just 47 epochs on BanglaLekha-Isolated dataset.\n2. [A New Benchmark on the Recognition of Handwritten Bangla and Farsi Numeral Characters](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.544.9065&rep=rep1&type=pdf): In this work, we applied some advanced character recognition methods to handwritten Bangla and Farsi numeral recognition and evaluated the performance on three public databases: ISI Bangla numerals, CENPARMI Farsi numerals, and IFHCDB Farsi numerals.\n3. [Hand Written Bangla Numerals Recognition for Automated Postal System ](https://www.researchgate.net/publication/325064193_Hand_Written_Bangla_Numerals_Recognition_for_Automated_Postal_System)(the full pdf is available for download): This paper delineate a robust hybrid system for recognition of handwritten Bangla numerals for the automated postal system, which performed feature extraction using k-means clustering, Baye’s theorem and Maximum a Posteriori, then the recognition is performed using Support Vector Machine . Recognition of handwritten numerals, such as postal codes, reveal all kinds of local and global deformations: distortions, different writing styles, thickness variations, wide variety of scales, limited amount of rotation, added noise, occlusion and missing parts.\n4. [An Improved Feature Descriptor for Recognition of Handwritten Bangla Alphabet](https://arxiv.org/pdf/1501.05497.pdf): In the present work we have identified a new variation of feature set which significantly outperforms on handwritten Bangla alphabet from the previously used feature set. 132 number of features in all viz. modified shadow features, octant and centroid features, distance based features, quad tree based longest run features are used here. Using this feature set the recognition performance increases sharply from the 75.05% observed in our previous work, to 85.40% on 50 character classes with MLP based classifier on the same dataset. \n5. [Handwritten Bangla Character Recognition Using The State-of-Art Deep Convolutional Neural Networks](https://arxiv.org/pdf/1712.09872.pdf): application of the state-of-the-art Deep Convolutional Neural Networks (DCNN) including VGG Network, All Convolution Network (All-Conv Net), Network in Network (NiN), Residual Network, FractalNet, and DenseNet for HBCR. The deep learning approaches have the advantage of extracting and using feature information, improving the recognition of 2D shapes with a high degree of invariance to translation, scaling and other distortions.\n6. [Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification](https://arxiv.org/pdf/1309.0261.pdf): Deep and wide max-pooling convolutional neural networks (MPCNN) on GPU embody the current state of the art in stationary pattern recognition. They outperformed other methods on image classification, object detection, and image segmentation. Through output averaging, several independently trained deep NN (DNN) can form a Multi-Column DNN (MCDNN) with error rates 20-40% below those of single DNN.\n7. [Multi-column Deep Neural Networks for Image Classification](https://arxiv.org/pdf/1202.2745.pdf): Small (often minimal) receptive fields of convolutional winner-take-all neurons yield large network depth, resulting in roughly as many sparsely connected neural layers as found in mammals between retina and visual cortex. Only winner neurons are trained. Several deep neural columns become experts on inputs preprocessed in different ways; their predictions are averaged. Graphics cards allow for fast training. On the very competitive MNIST handwriting benchmark, our method is the first to achieve near-human performance. "},{"metadata":{},"cell_type":"markdown","source":"## Modelling Suggestions\nThese are some resources which might be useful for modelling:\n\n1. [Shake-shake regularization](https://arxiv.org/pdf/1705.07485.pdf) paper\n2. [MixUp training](https://arxiv.org/pdf/1710.09412.pdf) paper\n3. [Label Smoothing](https://arxiv.org/pdf/1906.02629.pdf) paper\n4. [Small CNN architectures](https://towardsdatascience.com/3-small-but-powerful-convolutional-networks-27ef86faa42d) Medium article"},{"metadata":{},"cell_type":"markdown","source":"## References and Useful Links\n1. [Bengali fonts](https://www.omicronlab.com/bangla-fonts.html) to use for matplotlib diagrams\n2. [Thresholding tutorial](https://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html) in OpenCV\n3. [Common Bengali Handwritten Graphemes in Context](https://bengali.ai/wp-content/uploads/CV19-COCO-Grapheme.pdf) Introductory Booklet\n4. [Consonant Conjuncts vs Consonant Diacritics](https://www.kaggle.com/c/bengaliai-cv19/discussion/123002) discussion"},{"metadata":{},"cell_type":"markdown","source":"__Please, feel free to leave comments and/or questions! Your feedback is very welcome!__ "}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}