{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# _Bengali.AI Handwritten Grapheme Classification_\n\nEn el siguiente trabajo se aborda la resolución del problema planteado para la clasificación de los componentes de grafemas benagalíes. \n\nPara más información [ingrese aquí](https://www.kaggle.com/c/bengaliai-cv19)\n\n## 1. Importar módulos y cargar _dataset_","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport PIL.Image as Image, PIL.ImageDraw as ImageDraw, PIL.ImageFont as ImageFont\nimport seaborn as sns\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Conv2D, MaxPool2D\nfrom tensorflow.keras import layers\nfrom tensorflow.keras.layers import Dense, Dropout, Flatten\nfrom tensorflow.keras.utils import plot_model\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nimport cv2\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-08-27T07:35:50.580917Z","iopub.execute_input":"2021-08-27T07:35:50.581334Z","iopub.status.idle":"2021-08-27T07:35:55.971899Z","shell.execute_reply.started":"2021-08-27T07:35:50.581237Z","shell.execute_reply":"2021-08-27T07:35:55.970326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"En la lista podemos ver los archivos que conforman el _dataset_ propuesto para este problema. En nuestro caso sólo utilizaremos el archivo `train_image_data_0.parquet` y el archivo `train.csv` para el entrenamiento.","metadata":{}},{"cell_type":"code","source":"train_image_0_df = pd.read_parquet(f'../input/bengaliai-cv19/train_image_data_0.parquet')","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:38:24.663850Z","iopub.execute_input":"2021-08-27T07:38:24.664222Z","iopub.status.idle":"2021-08-27T07:39:28.760593Z","shell.execute_reply.started":"2021-08-27T07:38:24.664190Z","shell.execute_reply":"2021-08-27T07:39:28.759671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_tabular_df = pd.read_csv('../input/bengaliai-cv19/train.csv')","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:39:58.273293Z","iopub.execute_input":"2021-08-27T07:39:58.273637Z","iopub.status.idle":"2021-08-27T07:39:58.626822Z","shell.execute_reply.started":"2021-08-27T07:39:58.273603Z","shell.execute_reply":"2021-08-27T07:39:58.625931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Análisis exploratorio\n\nPara conseguir un entendimiento del problema, primero vamos a avanzar analizando las generalidades de los _dataframes_ que se cargaron en memoria. \n\n### 2.1. Información de los _dataframes_","metadata":{}},{"cell_type":"code","source":"train_image_0_df.info()","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:40:31.398195Z","iopub.execute_input":"2021-08-27T07:40:31.398544Z","iopub.status.idle":"2021-08-27T07:40:33.139108Z","shell.execute_reply.started":"2021-08-27T07:40:31.398511Z","shell.execute_reply":"2021-08-27T07:40:33.138092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_tabular_df.info()","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:41:24.194378Z","iopub.execute_input":"2021-08-27T07:41:24.194756Z","iopub.status.idle":"2021-08-27T07:41:24.246822Z","shell.execute_reply.started":"2021-08-27T07:41:24.194718Z","shell.execute_reply":"2021-08-27T07:41:24.245836Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_image_0_df.shape","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:48:15.774473Z","iopub.execute_input":"2021-08-27T07:48:15.775081Z","iopub.status.idle":"2021-08-27T07:48:15.787209Z","shell.execute_reply.started":"2021-08-27T07:48:15.775031Z","shell.execute_reply":"2021-08-27T07:48:15.786003Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_tabular_df.shape","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:48:18.381577Z","iopub.execute_input":"2021-08-27T07:48:18.381929Z","iopub.status.idle":"2021-08-27T07:48:18.390967Z","shell.execute_reply.started":"2021-08-27T07:48:18.381891Z","shell.execute_reply":"2021-08-27T07:48:18.390121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"En los comandos anteriores es posible observar que `train_image_0_df` posee 50210 imágenes. Entre sus columnas se observa el `image_id` y los píxeles de cada una de las imágenes.\n\nSi restamos de las 32333 columnas el `image_id`, sólo restan las columnas que se corresponden a los 137x236 píxeles de cada imagen, tal como se informa en la seción [data](https://www.kaggle.com/c/bengaliai-cv19/data) de la documentación.","metadata":{}},{"cell_type":"code","source":"train_image_0_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:47:27.077496Z","iopub.execute_input":"2021-08-27T07:47:27.077927Z","iopub.status.idle":"2021-08-27T07:47:27.112913Z","shell.execute_reply.started":"2021-08-27T07:47:27.077893Z","shell.execute_reply":"2021-08-27T07:47:27.111892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_tabular_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:47:33.869940Z","iopub.execute_input":"2021-08-27T07:47:33.870288Z","iopub.status.idle":"2021-08-27T07:47:33.881109Z","shell.execute_reply.started":"2021-08-27T07:47:33.870258Z","shell.execute_reply":"2021-08-27T07:47:33.880205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.2. Valores únicos por columna\n","metadata":{}},{"cell_type":"code","source":"train_tabular_df.nunique()","metadata":{"execution":{"iopub.status.busy":"2021-08-27T08:00:19.333949Z","iopub.execute_input":"2021-08-27T08:00:19.334279Z","iopub.status.idle":"2021-08-27T08:00:19.436945Z","shell.execute_reply.started":"2021-08-27T08:00:19.334237Z","shell.execute_reply":"2021-08-27T08:00:19.435934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Cabe destacar la cantidad de valores únicos para cada una de las columnas. Pues esto definirá el tipo y cantidad de salidas de la red neuronal.","metadata":{}},{"cell_type":"markdown","source":"## 3. Ingeniería de _features_\n\n### 3.1. Fusión de los _dataframes_\n\nEs necesario contar con un único conjunto de datos, hasta ahora se tienen las imágenes y las etiquetas en tablas diferentes. Además, debido a que no se están utilizando todas las imágenes (por falta de memoria), es necesario descartar las etiquetas que no perteneces a una imagen dentro de `train_image_0_df`","metadata":{}},{"cell_type":"code","source":"train_data =  pd.merge(train_image_0_df, train_tabular_df, on='image_id').drop(['image_id'], axis=1)","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:51:57.259962Z","iopub.execute_input":"2021-08-27T07:51:57.260277Z","iopub.status.idle":"2021-08-27T07:52:08.380555Z","shell.execute_reply.started":"2021-08-27T07:51:57.260249Z","shell.execute_reply":"2021-08-27T07:52:08.379691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data.shape","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:52:17.067907Z","iopub.execute_input":"2021-08-27T07:52:17.068228Z","iopub.status.idle":"2021-08-27T07:52:17.074548Z","shell.execute_reply.started":"2021-08-27T07:52:17.068201Z","shell.execute_reply":"2021-08-27T07:52:17.073621Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 3.2. Separar las variables independientes (_features_) de las salidas (_labels_)","metadata":{}},{"cell_type":"code","source":"train_labels = train_data[['grapheme_root', 'vowel_diacritic', 'consonant_diacritic','grapheme']]\ntrain_labels.shape","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:53:34.772268Z","iopub.execute_input":"2021-08-27T07:53:34.772719Z","iopub.status.idle":"2021-08-27T07:53:34.808430Z","shell.execute_reply.started":"2021-08-27T07:53:34.772677Z","shell.execute_reply":"2021-08-27T07:53:34.807683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data = train_data.drop(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic','grapheme'], axis=1)\ntrain_data.shape","metadata":{"execution":{"iopub.status.busy":"2021-08-27T07:53:38.178983Z","iopub.execute_input":"2021-08-27T07:53:38.179294Z","iopub.status.idle":"2021-08-27T07:53:39.688451Z","shell.execute_reply.started":"2021-08-27T07:53:38.179266Z","shell.execute_reply":"2021-08-27T07:53:39.687477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 3.3. Redimensionar las imágenes\n\nLamentablemente, es necesario redimiensionar las imágenes para que sea posible procesarlas con el _hardware_ disponible. Además, esto acelera el proceso de ensayo. Es probable que, una vez definido el modelo, valga la pena hace pruebas con imágenes más grandes.","metadata":{}},{"cell_type":"code","source":"def resize(df, size=64, need_progress_bar=True):\n    resized = {}\n    for i in range(df.shape[0]):\n        image = cv2.resize(df.loc[df.index[i]].values.reshape(137,236),(size,size))\n        resized[df.index[i]] = image.reshape(-1)\n    resized = pd.DataFrame(resized).T\n    return resized","metadata":{"execution":{"iopub.status.busy":"2021-08-27T08:04:21.970114Z","iopub.execute_input":"2021-08-27T08:04:21.970484Z","iopub.status.idle":"2021-08-27T08:04:21.977129Z","shell.execute_reply.started":"2021-08-27T08:04:21.970448Z","shell.execute_reply":"2021-08-27T08:04:21.975941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Al momento de llamar el método `resize()`, también se divide por 255 para obtener valores decimales en cada píxel. Esto permite que, durante el entrenamiento, se interprete correctamente la naturaleza del número como indicador de la intensidad de cada píxel.","metadata":{}},{"cell_type":"code","source":"train_data = resize(train_data, size=64)/255\ntrain_data = train_data.values.reshape(-1, 64, 64, 1)","metadata":{"execution":{"iopub.status.busy":"2021-08-27T08:07:14.350181Z","iopub.execute_input":"2021-08-27T08:07:14.350524Z","iopub.status.idle":"2021-08-27T08:07:38.509193Z","shell.execute_reply.started":"2021-08-27T08:07:14.350492Z","shell.execute_reply":"2021-08-27T08:07:38.508275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Modelo\n\nDebido a que los grafemas están compuestos por 3 componentes, se decidió entrenar 3 modelos con la misma estructura. El entrenamiento de cada uno de los modelos está enfocado a cada uno de las componentes del grafema.\n\nLa única diferencia que hay entre los modelos es la capa de salida. Si bien todos terminan con una activación _softmax_, cada uno lo hace con la cantidad de salidas correspondientes al número de valores únicos que tiene cada componente. ","metadata":{}},{"cell_type":"code","source":"model_dict = {\n    'grapheme_root': Sequential(),\n    'vowel_diacritic': Sequential(),\n    'consonant_diacritic': Sequential()\n}\nfor model_type, model in model_dict.items():\n    model.add(Conv2D(input_shape=(64,64,1),filters=64,kernel_size=(3,3),padding=\"same\", activation=\"relu\"))\n    model.add(Conv2D(filters=64,kernel_size=(3,3),padding=\"same\", activation=\"relu\"))\n    model.add(layers.BatchNormalization(momentum=0.15))\n    model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))\n    model.add(Conv2D(128, 3, activation=\"relu\", padding=\"same\"))\n    model.add(Conv2D(128, 3, activation=\"relu\", padding=\"same\"))\n    model.add(MaxPool2D(2))\n    model.add(Conv2D(256, 3, activation=\"relu\", padding=\"same\"))\n    model.add(Conv2D(256, 3, activation=\"relu\", padding=\"same\"))\n    model.add(MaxPool2D(2))\n    model.add(Flatten())\n    model.add(Dense(1024, activation=\"relu\"))\n    model.add(Dropout(0.5))\n    model.add(Dense(512, activation=\"relu\"))\n    model.add(Dropout(0.5))\n    if model_type == 'grapheme_root':\n        model.add(layers.Dense(168, activation='softmax', name='root_out'))\n    elif model_type == 'vowel_diacritic':\n        model.add(layers.Dense(11, activation='softmax', name='vowel_out'))\n    elif model_type == 'consonant_diacritic':\n        model.add(layers.Dense(7, activation='softmax', name='consonant_out'))\n    model.compile(optimizer=\"adam\", loss=['categorical_crossentropy'], metrics=['accuracy'])\n    \nplot_model(model_dict['grapheme_root'])","metadata":{"execution":{"iopub.status.busy":"2021-08-27T08:01:59.710399Z","iopub.execute_input":"2021-08-27T08:01:59.710747Z","iopub.status.idle":"2021-08-27T08:02:02.659512Z","shell.execute_reply.started":"2021-08-27T08:01:59.710717Z","shell.execute_reply":"2021-08-27T08:02:02.658515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 4.1. Entrenamiento de los modelos\n\nIterando sobre el diccionario de modelos es posible realizar el entrenamiento de los 3 modelos para detectar las componentes de cada grafema.\n\nCabe señalar algunos aspectos importantes del entrenamiento:\n\n1. Se utiliza un `ImageDataGenerator` para hacer _data augmentation_.\n1. Antes de entrenar a cada modelo se divide el _dataset_ en _train_ y _test_, de forma aleatoria, dejando el 10% de las filas para _test_.\n1. El histórico de la evolución de las métricas se guarda en una lista.","metadata":{}},{"cell_type":"code","source":"batch_size = 32\nepochs = 10\nhistory_list = []\nmodel_types = ['grapheme_root', 'vowel_diacritic', 'consonant_diacritic']\nfor target in model_types:\n    Y_train = train_labels[target]\n    Y_train = pd.get_dummies(Y_train).values\n    x_train, x_test, y_train, y_test = train_test_split(train_data, Y_train, test_size=0.1, random_state=123)\n    datagen = ImageDataGenerator()\n    datagen.fit(x_train)\n    history = model_dict[target].fit(datagen.flow(x_train, y_train, batch_size=batch_size), \n                                               epochs = epochs, validation_data = (x_test, y_test))\n    history_list.append(history)","metadata":{"execution":{"iopub.status.busy":"2021-08-27T08:09:48.376062Z","iopub.execute_input":"2021-08-27T08:09:48.376421Z","iopub.status.idle":"2021-08-27T08:22:46.772164Z","shell.execute_reply.started":"2021-08-27T08:09:48.376390Z","shell.execute_reply":"2021-08-27T08:22:46.771251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 4.2. Visualización del avance de las métricas en entrenamiento","metadata":{}},{"cell_type":"code","source":"for history in history_list:\n    # summarize history for accuracy\n    plt.plot(history.history['accuracy'])\n    plt.plot(history.history['val_accuracy'])\n    plt.title('model accuracy')\n    plt.ylabel('accuracy')\n    plt.xlabel('epoch')\n    plt.legend(['train', 'test'], loc='upper left')\n    plt.show()\n    # summarize history for loss\n    plt.plot(history.history['loss'])\n    plt.plot(history.history['val_loss'])\n    plt.title('model loss')\n    plt.ylabel('loss')\n    plt.xlabel('epoch')\n    plt.legend(['train', 'test'], loc='upper left')\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-27T08:22:46.774116Z","iopub.execute_input":"2021-08-27T08:22:46.774492Z","iopub.status.idle":"2021-08-27T08:22:47.600331Z","shell.execute_reply.started":"2021-08-27T08:22:46.774450Z","shell.execute_reply":"2021-08-27T08:22:47.599571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5. Conclusiones\n\nDespués de hacer repetidas pruebas, se concluye que:\n\n1. El modelo da mejores resultados cuando se entrena para decidir entre menos categorías. Para el caso de los 168 _grapheme roots_, el modelo tiene su peor _performance_.\n1. La capa `BatchNormalization` tiene un gran impacto en el _accuracy_.\n1. VGG16 no llega a los resultados obtenidos con esta red.\n1. 64x64 parece ser un tamaño razonable para las imágenes. Las pruebas con 96x96 y 128x128 resultaron en memoria insuficiente.\n1. Trabajar en colab no fue posible por la cantidad de memoria necesaria.\n1. Trabajar en _hardware_ local (sin GPU) resulta en entrenamientos de casi 3 horas, lo que dificulta el desarrollo.","metadata":{}}]}