{"cells":[{"metadata":{},"cell_type":"markdown","source":"<h1>Bengali.AI Handwritten Grapheme - Getting Started</h1>\n\n# Introduction\n\nBengali is the 5th most spoken language in the world with hundreds of million of speakers. Optical character recognition is particularly challenging for Bengali. While Bengali has 49 letters (to be more specific 11 vowels and 38 consonants) in its alphabet, there are also 18 potential diacritics, or accents. This means that there are many more graphemes, or the smallest units in a written language. The added complexity results in ~13,000 different grapheme variations (compared to English’s 250 graphemic units).\n\nBangladesh-based non-profit Bengali.AI is focused on helping to solve this problem. They build and release crowdsourced, metadata-rich datasets and open source them through research competitions. Through this work, Bengali.AI hopes to democratize and accelerate research in Bengali language technologies and to promote machine learning education.\n\nFor this competition, we are given the image of a handwritten Bengali grapheme and are challenged to separately classify three constituent elements in the image: grapheme root, vowel diacritics, and consonant diacritics."},{"metadata":{},"cell_type":"markdown","source":"# Prepare for data analysis\n\n## Load packages"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport PIL.Image\nimport time\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm_notebook\n%matplotlib inline ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Check the data\n\nWe verify what data is available."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"for dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We have both csv files and parquet files.  \nWe will start by exploring csv files and will follow with parquet files."},{"metadata":{},"cell_type":"markdown","source":"# Data exploration\n\nWe start with the few csv files."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"DATA_FOLDER = '/kaggle/input/bengaliai-cv19/'\ntrain_df = pd.read_csv(os.path.join(DATA_FOLDER, 'train.csv'))\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"test_df = pd.read_csv(os.path.join(DATA_FOLDER, 'test.csv'))\ntest_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"test_df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"class_map_df = pd.read_csv(os.path.join(DATA_FOLDER, 'class_map.csv'))\nclass_map_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class_map_df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission_df = pd.read_csv(os.path.join(DATA_FOLDER, 'sample_submission.csv'))\nsample_submission_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sample_submission_df.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We follow how with the parquet files. We will read only two of the parquet files for now, first train file."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"start_time = time.time()\ntrain_0_df = pd.read_parquet(os.path.join(DATA_FOLDER,'train_image_data_0.parquet'))\nprint(f\"`train_image_data_0` read in {round(time.time()-start_time,2)} sec.\")                               ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_0_df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_0_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"start_time = time.time()\ntrain_1_df = pd.read_parquet(os.path.join(DATA_FOLDER,'train_image_data_1.parquet'))\nprint(f\"`train_image_data_1` read in {round(time.time()-start_time,2)} sec.\")  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_1_df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_1_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Each `train_image_data_x` (x = 0...3) contains **50210** rows and **32333** columns - size of each image being: **(137, 236)**. Totally there are **50210** x **4** = **200840** rows in the training set.  \n\nWe also read one of the test files.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"start_time = time.time()\ntest_0_df = pd.read_parquet(os.path.join(DATA_FOLDER,'test_image_data_0.parquet'))\nprint(f\"`test_image_data_0` read in {round(time.time()-start_time,2)} sec.\")  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_0_df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_0_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Unique values\n\nWe look here to the distribution of grapheme roots, vowel diacritics and consonant diacritics."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"print(f\"Train: unique grapheme roots: {train_df.grapheme_root.nunique()}\")\nprint(f\"Train: unique vowel diacritics: {train_df.vowel_diacritic.nunique()}\")\nprint(f\"Train: unique consonant diacritics: {train_df.consonant_diacritic.nunique()}\")\nprint(f\"Train: total unique elements: {train_df.grapheme_root.nunique() + train_df.vowel_diacritic.nunique() + train_df.consonant_diacritic.nunique()}\")\nprint(f\"Class map: unique elements: \\n{class_map_df.component_type.value_counts()}\")\nprint(f\"Total combinations: {pd.DataFrame(train_df.groupby(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic'])).shape[0]}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data distribution\n\nLet's start by viewing each grapheme.\n\nLet's show the grapheme roots first."},{"metadata":{"trusted":true},"cell_type":"code","source":"cm_gr = class_map_df.loc[(class_map_df.component_type=='grapheme_root'), 'component'].values\ncm_vd = class_map_df.loc[(class_map_df.component_type=='vowel_diacritic'), 'component'].values  \ncm_cd = class_map_df.loc[(class_map_df.component_type=='consonant_diacritic'), 'component'].values   \n\nprint(f\"grapheme root:\\n{15*'-'}\\n{cm_gr}\\n\\n vowel discritic:\\n{18*'-'}\\n{cm_vd}\\n\\n consonant diacritic:\\n{20*'-'}\\n {cm_cd}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's follow by investigating the most frequent values."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def most_frequent_values(data):\n    total = data.count()\n    tt = pd.DataFrame(total)\n    tt.columns = ['Total']\n    items = []\n    vals = []\n    for col in data.columns:\n        itm = data[col].value_counts().index[0]\n        val = data[col].value_counts().values[0]\n        items.append(itm)\n        vals.append(val)\n    tt['Most frequent item'] = items\n    tt['Frequence'] = vals\n    tt['Percent from total'] = np.round(vals / total * 100, 3)\n    return(np.transpose(tt))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Most frequent train values."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"most_frequent_values(train_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Most frequent test values."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"most_frequent_values(test_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's look now to the distribution of class values."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_count(feature, title, df, size=1):\n    '''\n    Plot count of classes of selected feature; feature is a categorical value\n    param: feature - the feature for which we present the distribution of classes\n    param: title - title to show in the plot\n    param: df - dataframe \n    param: size - size (from 1 to n), multiplied with 4 - size of plot\n    '''\n    f, ax = plt.subplots(1,1, figsize=(4*size,4))\n    total = float(len(df))\n    g = sns.countplot(df[feature], order = df[feature].value_counts().index[:20], palette='Set3')\n    g.set_title(\"Number and percentage of {}\".format(title))\n    if(size > 2):\n        plt.xticks(rotation=90, size=8)\n    for p in ax.patches:\n        height = p.get_height()\n        ax.text(p.get_x()+p.get_width()/2.,\n                height + 3,\n                '{:1.2f}%'.format(100*height/total),\n                ha=\"center\") \n    plt.show() ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_count('grapheme_root', 'grapheme_root (first most frequent 20 values - train)', train_df, size=4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_count('vowel_diacritic', 'vowel_diacritic (train)', train_df, size=3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_count('consonant_diacritic', 'consonant_diacritic (train)', train_df, size=3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's show now distribution of combinations of features. We create a function to show a heatmap."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def plot_count_heatmap(feature1, feature2, df, size=1):  \n    '''\n    Heatmap showing the distribution of couple of features\n    param: feature1 - ex: vowel_diacritic\n    param: feature2 - ex: consonant_diacritic\n    '''\n    tmp = train_df.groupby([feature1, feature2])['grapheme'].count()\n    df = tmp.reset_index()\n    df\n    df_m = df.pivot(feature1, feature2, \"grapheme\")\n    f, ax = plt.subplots(figsize=(9, size * 4))\n    sns.heatmap(df_m, annot=True, fmt='3.0f', linewidths=.5, ax=ax)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see first what consonant diacritics and vowel diacritics appears together."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_count_heatmap('vowel_diacritic','consonant_diacritic', train_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We look now to the combinations of consonant diacritic and grapheme roots."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_count_heatmap('grapheme_root','consonant_diacritic', train_df, size=8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here is the combinations of vowel diacritic and grapheme roots."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_count_heatmap('grapheme_root','vowel_diacritic', train_df, size=8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Inspect grapheme images\n\n\nWe define a function to show a sample of size * size (ex: 5 x 5 = 25) handwritten graphemes."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def display_image_from_data(data_df, size=5):\n    '''\n    Display grapheme images from sample data\n    param: data_df - sample of data\n    param: size - sqrt(sample size of data)\n    '''\n    plt.figure()\n    fig, ax = plt.subplots(size,size,figsize=(12,12))\n    # we show grapheme images for a selection of size x size samples\n    for i, index in enumerate(data_df.index):\n        image_id = data_df.iloc[i]['image_id']\n        flattened_image = data_df.iloc[i].drop('image_id').values.astype(np.uint8)\n        unpacked_image = PIL.Image.fromarray(flattened_image.reshape(137, 236))\n\n        ax[i//size, i%size].imshow(unpacked_image)\n        ax[i//size, i%size].set_title(image_id)\n        ax[i//size, i%size].axis('on')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"display_image_from_data(train_0_df.sample(25))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We show also a sample from the second set of images and with fewer samples size (16)."},{"metadata":{"trusted":true},"cell_type":"code","source":"display_image_from_data(train_1_df.sample(16), size = 4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's apply this function, this time to show not random graphemes, but the same grapheme, with different writing.   \n\nFor this we create a second function, to perform the sampling (based on variation of grapheme root, vowel diacritic and consonant diacritic, as parameters to the function)."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def display_writting_variety(data_df=train_0_df, grapheme_root=72, vowel_diacritic=0,\\\n                             consonant_diacritic=0, size=5):\n    '''\n    This function get a set of grapheme root, vowel diacritic and consonant diacritic\n    and display a sample of 25 images for this grapheme\n    param: data_df - the dataset used as source of data\n    param: grapheme_root - the grapheme root label\n    param: vowel_diacritic - the vowel diacritic label\n    param: consonant_diacritic - the consonant diacritic label \n    param: size - sqrt(number of images to show)\n    '''\n    sample_train_df = train_df.loc[(train_df.grapheme_root == grapheme_root) & \\\n                                  (train_df.vowel_diacritic == vowel_diacritic) & \\\n                                  (train_df.consonant_diacritic == consonant_diacritic)]\n    print(f\"total: {sample_train_df.shape}\")\n    sample_df = data_df.merge(sample_train_df.image_id, how='inner')\n    print(f\"total: {sample_df.shape}\")\n    gr = sample_train_df.iloc[0]['grapheme']\n    cm_gr = class_map_df.loc[(class_map_df.component_type=='grapheme_root')& \\\n                             (class_map_df.label==grapheme_root), 'component'].values[0]\n    cm_vd = class_map_df.loc[(class_map_df.component_type=='vowel_diacritic')& \\\n                             (class_map_df.label==vowel_diacritic), 'component'].values[0]    \n    cm_cd = class_map_df.loc[(class_map_df.component_type=='consonant_diacritic')& \\\n                             (class_map_df.label==consonant_diacritic), 'component'].values[0]    \n    \n    print(f\"grapheme: {gr}, grapheme root: {cm_gr}, vowel discritic: {cm_vd}, consonant diacritic: {cm_cd}\")\n    sample_df = sample_df.sample(size * size)\n    display_image_from_data(sample_df, size=size)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We apply the function for few combinations of grapheme root, vowel diacritic and consonant diacritic."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"display_writting_variety(train_0_df,72,1,1,4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"display_writting_variety(train_0_df,64,1,2,4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"display_writting_variety(train_1_df,13,0,0,4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"display_writting_variety(train_1_df,23,3,2,4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can observe there is a large variety of writting for the selected graphemes."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}