{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Visualization of squeezed features: Where sSE Block looks?"},{"metadata":{},"cell_type":"markdown","source":" I suppose that **sSE Block** plays an important role as well as RandomErasing in [my solution](https://www.kaggle.com/c/bengaliai-cv19/discussion/136815).  \nThis notebook shows you where on images **sSE Block** pays its attention, by visualizing squeezed features for some images in train dataset.\n \nUsed model is cycle2 (70 epoch) model, which achived the best single private score(0.9499) in my submissions.\n\n<br>\n\nThank you Peter(@pestipeti)! I can visualize each component by your code and dataset in [Bengali - Quick EDA](https://www.kaggle.com/pestipeti/bengali-quick-eda). "},{"metadata":{},"cell_type":"markdown","source":"## preparation"},{"metadata":{},"cell_type":"markdown","source":"### import"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T02:40:39.115557Z","start_time":"2020-03-21T02:40:39.098658Z"},"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import sys\nimport os\nimport time\nimport gc\nimport random\nfrom pathlib import Path\nfrom collections import Counter\nfrom itertools import chain\nfrom PIL import Image, ImageDraw, ImageFont\n\nfrom typing import Union, Tuple, Dict, Optional\n\nimport pandas as pd\nimport numpy as np\nfrom scipy.sparse import coo_matrix\n\nimport chainer\n\nfrom matplotlib import pyplot as plt\nimport seaborn as sns\n\nfrom tqdm import tqdm\n\n%matplotlib inline\n\npd.options.display.max_rows = 2000\npd.options.display.max_columns = 50\npd.options.display.max_colwidth = 500","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T00:00:44.761393Z","start_time":"2020-03-21T00:00:44.755159Z"},"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# # set Path\nROOT = Path(\".\").absolute().parents[0]\nINPUT_ROOT = ROOT / \"input\"\nRAW_DATA = INPUT_ROOT / \"bengaliai-cv19\"\n\nSRC_PATH = INPUT_ROOT / \"bengali-src-final-05\"\nPROC_DATA = INPUT_ROOT / \"bengali-processed-data\"\nTRIANED_MODELS_PATH = INPUT_ROOT / \"bengali-final-serx50sse-128x224-3x35ep-all-fold\"","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T00:00:45.963345Z","start_time":"2020-03-21T00:00:45.958767Z"},"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"cd ../input/bengali-src-final-05","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T00:00:47.122463Z","start_time":"2020-03-21T00:00:46.922786Z"},"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# # import from my src\n# sys.path.append(SRC_PATH.as_posix())\n\nfrom competition_utils import utils as my_utils\n\nfrom nn_for_image_data import backborn_chains as my_backborns\nfrom nn_for_image_data import global_pooling_chains as my_global_poolings\nfrom nn_for_image_data import classifer_chains as my_classifers\nfrom training_utils import nn_training as my_nn_tr\n\nimport config as my_config","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### make dataset"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T04:08:58.829254Z","start_time":"2020-03-21T04:08:58.685497Z"},"trusted":true},"cell_type":"code","source":"# # read train label info\ntrain_df = pd.read_csv(PROC_DATA / \"train_add-4fold-index.csv\").drop([\"character_id\", \"fold\"], axis=1)\nclm = pd.read_csv(RAW_DATA / \"class_map.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I select some of graphemes to visualize, giving priority to `consonant_diacritic`."},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T04:09:04.631592Z","start_time":"2020-03-21T04:09:04.613839Z"},"_kg_hide-output":true,"scrolled":true,"trusted":true},"cell_type":"code","source":"train_df = train_df.drop_duplicates(\n    subset=[\"vowel_diacritic\", \"consonant_diacritic\"]\n).sort_values(by=[\"consonant_diacritic\", \"vowel_diacritic\"]).reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"add str of each component"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T05:38:23.758497Z","start_time":"2020-03-21T05:38:23.739845Z"},"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# # add `grapheme_root_str`\ntrain_df = train_df.merge(\n    clm.query(\"component_type == 'grapheme_root'\")[[\"component\", \"label\"]].rename(\n        columns={\"component\": 'grapheme_root_str', \"label\": \"grapheme_root\"}),\n    on=\"grapheme_root\", how=\"left\")\n# # add `vowel_diacritic_str`\ntrain_df = train_df.merge(\n    clm.query(\"component_type == 'vowel_diacritic'\")[[\"component\", \"label\"]].rename(\n        columns={\"component\": 'vowel_diacritic_str', \"label\": \"vowel_diacritic\"}),\n    on=\"vowel_diacritic\", how=\"left\")\n# # add `consonant_diacritic`\ntrain_df = train_df.merge(\n    clm.query(\"component_type == 'consonant_diacritic'\")[[\"component\", \"label\"]].rename(\n        columns={\"component\": 'consonant_diacritic_str', \"label\": \"consonant_diacritic\"}),\n    on=\"consonant_diacritic\", how=\"left\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T05:38:31.027238Z","start_time":"2020-03-21T05:38:31.011871Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"train_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### make labels arr"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T00:00:54.747386Z","start_time":"2020-03-21T00:00:54.740854Z"},"trusted":true},"cell_type":"code","source":"train_labels_arr = train_df[['grapheme_root', 'vowel_diacritic', 'consonant_diacritic']].values.astype(\"i\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### make chainer dataset for extract squeezed features"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T00:01:56.815014Z","start_time":"2020-03-21T00:01:56.803355Z"},"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"## prepare a dataset for visualize to feed into the trained model.\next_dataset = chainer.datasets.LabeledImageDataset(\n    pairs=list(\n        zip((train_df[\"image_id\"] + \".png\").tolist(),  train_labels_arr)),\n    root=(PROC_DATA / \"train\").as_posix())\n\n# # set transforms\n# # # Note: `CustomTranspose` just changes position of channel. This is because I use chainer datasets class and albumentations.\n# # # Albumentations requires [H, W, C] format images while chainer datasets class output [C, H, W] format images.\next_dataset = chainer.datasets.TransformDataset(\n    ext_dataset,\n    my_nn_tr.ImageTransformer([\n        # # change format from [C, H, W] to [H, W, C]\n        [\"CustomTranspose\", {\"always_apply\": True, \"axis\": [1, 2, 0]}],\n        # # Pad\n        [\"PadIfNeeded\", {\n            \"always_apply\": True, \"min_height\": 140, \"min_width\": 245, \"border_mode\": 0, \"value\": 253}],\n        # # Resize\n        [\"Resize\", {\"always_apply\": True, \"height\": 128, \"width\": 224}],\n        # # Normalize\n        [\"Normalize\", {\n            \"always_apply\": True, \"mean\": [0.946967259411315,], \"std\": [0.06580952928802959,]}],\n        # # change format from [H, W, C] to [C, H, W]\n        [\"CustomTranspose\", {\"always_apply\": True, \"axis\": [2, 0, 1]}],\n    ])\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### init and load model\n#### make new class which inherits ImageClassificationModels\nAdd new method for extract features squeezed  by sSE Block output."},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T03:08:47.559066Z","start_time":"2020-03-21T03:08:47.547597Z"},"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"class CustomICM(my_nn_tr.ImageClassificationModel):\n    \"\"\"Custom Class\"\"\"\n    \n    def __init__(\n        self, extractor: chainer.Chain,\n        global_pooling: Optional[chainer.Chain], classifier: chainer.Chain\n    ) -> None:\n        \"\"\"Initialization.\"\"\"\n        super(CustomICM, self).__init__(extractor, global_pooling, classifier)\n        \n    def extract_squeezed_features(self, x: chainer.Variable) -> Tuple[chainer.Variable]:\n        \"\"\"New method for extraction,\"\"\"\n        # # x: (bs, 1, H, W) => feature_map: (bs, 2048, H // 32, W // 32)\n        # # # [in this case] x: (bs, 1,128, 224) => feature_map: (bs, 2048, 4, 7)\n        feature_map = self.extractor(x)\n\n        # # feature_map: (bs, 2048, H // 32, W // 32) => sqfeat_XXX: (bs, 1, H // 32, W // 32) (the same shape)\n        # # # for root\n        sqfeat_root = chainer.functions.sigmoid(\n            self.global_pooling.pool1.sse.channel_squeeze(feature_map))\n        # # # for vowel\n        sqfeat_vowel = chainer.functions.sigmoid(\n            self.global_pooling.pool2.sse.channel_squeeze(feature_map))\n        # # # for consonant\n        sqfeat_consonant = chainer.functions.sigmoid(\n                self.global_pooling.pool3.sse.channel_squeeze(feature_map))\n        \n        return (sqfeat_root , sqfeat_vowel, sqfeat_consonant)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### init and load model"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T03:09:44.159319Z","start_time":"2020-03-21T03:09:42.404444Z"},"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"# # init model\nmodel = CustomICM(\n    # # backborn\n    extractor=getattr(my_backborns, \"ImageFeatureExtractor\")(\n        backborn_model=\"SEResNeXt50Conv1MeanTo1ch\", pretrained_model_path=None, extract_layers=[\"res5\"]),\n    # # global pooling\n    global_pooling=getattr(my_global_poolings, \"TripleHeadPoolingLayer\")(\n        pooling_layer=\"sSEAvgPool\", pooling_kwargs={}),\n    # # head classifier\n    classifier=getattr(my_classifers, \"TripleInOutClassificationLayer\")(\n        n_classes=[168, 11, 7], classification_layer=\"LADL\")\n)\n\n# # load the model of cycle2(70 epoch), which is my best single model at private LB (0.9499). \nchainer.serializers.load_npz(\n    TRIANED_MODELS_PATH / \"model_snapshot_70.npz\", model)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## extract"},{"metadata":{},"cell_type":"markdown","source":"for extract squeezed features, rewrite inference loop function."},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T03:09:49.557312Z","start_time":"2020-03-21T03:09:49.543924Z"},"_kg_hide-input":true,"hide_input":true,"trusted":true},"cell_type":"code","source":"def loop_for_extraction_of_squeezed_features(\n    model: chainer.Chain, test_iter: chainer.iterators.MultiprocessIterator, gpu_device: int=-1\n# ) -> Tuple[np.ndarray]:\n) -> Tuple[Tuple[np.ndarray], np.ndarray]:\n    \"\"\"Inference roop for extraction of squeezed_features for each component\"\"\"\n#     test_pred_list = []\n    sqfeat_root_list = []\n    sqfeat_vowel_list = []\n    sqfeat_consonant_list = []\n\n    test_label_list = []\n    \n    iter_num = 0\n    epoch_test_start = time.time()\n\n    while True:\n        test_batch = test_iter.next()\n        iter_num += 1\n        print(\"\\rtmp_iteration: {:0>5}\".format(iter_num), end=\"\")\n        in_arrays = chainer.dataset.concat_examples(test_batch, gpu_device)\n\n        # Forward the test data\n        with chainer.no_backprop_mode() and chainer.using_config(\"train\", False):\n#             prediction_test = model.inference(*in_arrays[:-1])\n#             test_pred_list.append(prediction_test)\n            (test_sqfeat_root, test_sqfeat_vowel, test_sqfeat_consonant) = model.extract_squeezed_features(*in_arrays[:-1])\n            sqfeat_root_list.append(test_sqfeat_root)\n            sqfeat_vowel_list.append(test_sqfeat_vowel)\n            sqfeat_consonant_list.append(test_sqfeat_consonant)\n\n            test_label_list.append(in_arrays[-1])\n#             prediction_test.unchain_backward()\n            test_sqfeat_root.unchain_backward()\n            test_sqfeat_vowel.unchain_backward()\n            test_sqfeat_consonant.unchain_backward()\n\n        if test_iter.is_new_epoch:\n            print(\" => test end: {:.2f} sec\".format(time.time() - epoch_test_start))\n            test_iter.reset()\n            break\n\n#     test_pred_all = chainer.cuda.to_cpu(functions.concat(test_pred_list, axis=0).data)\n    sqfeat_root_all = chainer.cuda.to_cpu(chainer.functions.concat(sqfeat_root_list, axis=0).data)\n    sqfeat_vowel_all = chainer.cuda.to_cpu(chainer.functions.concat(sqfeat_vowel_list, axis=0).data)\n    sqfeat_consonant_all = chainer.cuda.to_cpu(chainer.functions.concat(sqfeat_consonant_list, axis=0).data)\n\n    test_label_all = chainer.cuda.to_cpu(chainer.functions.concat(test_label_list, axis=0).data)\n#     del test_pred_list\n    del sqfeat_root_list\n    del sqfeat_vowel_list\n    del sqfeat_consonant_list\n    del test_label_list\n#     return test_pred_all, test_label_all\n    return (sqfeat_root_all, sqfeat_vowel_all, sqfeat_consonant_all), test_label_all","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"extact"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T03:10:01.562879Z","start_time":"2020-03-21T03:09:52.737636Z"},"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# # create iterator\next_iter = chainer.iterators.MultiprocessIterator(\n    ext_dataset, batch_size=64, repeat=False, shuffle=False, n_processes=2)\n\n# # extract\n# model.to_gpu(0)\n(\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr\n), label_arr = loop_for_extraction_of_squeezed_features(model, ext_iter, gpu_device=-1)\n# model.to_cpu()\n\n# # check shapes\nprint((sqfeat_root_arr.shape, sqfeat_vowel_arr.shape, sqfeat_consonant_arr.shape), label_arr.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualization"},{"metadata":{},"cell_type":"markdown","source":"prepare a dataset for visualization."},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:04:10.371131Z","start_time":"2020-03-21T06:04:10.363043Z"},"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"grapheme_arr = train_df[\n    [\"image_id\", \"grapheme\", \"grapheme_root_str\", \"vowel_diacritic_str\", \"consonant_diacritic_str\"]].values","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T03:10:11.683048Z","start_time":"2020-03-21T03:10:11.668629Z"},"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"viz_dataset = chainer.datasets.LabeledImageDataset(\n    pairs=list(\n        zip((train_df[\"image_id\"] + \".png\").tolist(),  train_labels_arr)),\n    root=(PROC_DATA / \"train\").as_posix())\n\n# # Now I want to get original images, not apply `Normalize`.\nviz_dataset = chainer.datasets.TransformDataset(\n    viz_dataset,\n    my_nn_tr.ImageTransformer([\n        # # change format from [C, H, W] to [H, W, C]\n        [\"CustomTranspose\", {\"always_apply\": True, \"axis\": [1, 2, 0]}],\n        # # Pad\n        [\"PadIfNeeded\", {\n            \"always_apply\": True, \"min_height\": 140, \"min_width\": 245, \"border_mode\": 0, \"value\": 253}],\n        # # Resize\n        [\"Resize\", {\"always_apply\": True, \"height\": 128, \"width\": 224}],\n        # # Normalize\n#         [\"Normalize\", {\n#             \"always_apply\": True, \"mean\": [0.946967259411315,], \"std\": [0.06580952928802959,]}],\n        # # change format from [H, W, C] to [C, H, W]\n        [\"CustomTranspose\", {\"always_apply\": True, \"axis\": [2, 0, 1]}],\n    ])\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"define visualization function"},{"metadata":{"trusted":true},"cell_type":"code","source":"def image_from_char(char, width=224, height=128):\n    \"\"\"\n    Make image from char.\n    reference: https://www.kaggle.com/pestipeti/bengali-quick-eda\n    \"\"\"\n    image = Image.new('RGB', (width, height))\n    draw = ImageDraw.Draw(image)\n    myfont = ImageFont.truetype('/kaggle/input/bengaliai/hind_siliguri_normal_500.ttf', 120)\n    w, h = draw.textsize(char, font=myfont)\n    draw.text(((width - w) / 2,(height - h) / 2), char, font=myfont)\n\n    return image\n\ndef get_sqfeat_image(idx, sqfeat_arr, img_size, cmap, size=(224, 128)):\n    \"\"\"Create sqeezed feature image.\"\"\"\n    # # min_max normalize for clear visualization\n    arr = (sqfeat_arr[idx][0] - sqfeat_arr[idx][0].min()) / (sqfeat_arr[idx][0].max() - sqfeat_arr[idx][0].min())\n    # # convert to RGB color image by color map\n    img =  Image.fromarray((cmap(arr)*255).astype(\"uint8\")).resize(size).convert(\"RGB\")\n    return img","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:15:06.225366Z","start_time":"2020-03-21T06:15:06.209274Z"},"hide_input":false,"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def visualize_squeezed_features(\n    img_idxs: np.ndarray, img_dataset: chainer.datasets.LabeledImageDataset,\n    root_arr: np.ndarray, vowel_arr: np.ndarray, consonant_arr: np.ndarray,\n    label_arr: np.ndarray, grapheme_arr:np.ndarray\n):\n    \"\"\"Vidualize squeezed features by blending with original image.\"\"\"\n    cmap_autumn = plt.get_cmap(\"autumn_r\")\n    num_img = len(img_idxs)\n    fig = plt.figure(figsize=(16, 2 * 3 * num_img))\n    \n    for i, idx in enumerate(img_idxs):\n        image_id, grapheme_str, root_str, vowel_str, consonant_str = grapheme_arr[idx]\n        root_label, vowel_label, consonant_label = label_arr[idx]\n        img_org = Image.fromarray(img_dataset[idx][0][0].astype(\"uint8\")).convert(\"RGB\")\n        img_org_font = image_from_char(grapheme_str)\n        img_root = get_sqfeat_image(idx, root_arr, img_org.size, cmap_autumn)\n        img_root_font = image_from_char(root_str)\n        img_vowel = get_sqfeat_image(idx, vowel_arr, img_org.size, cmap_autumn)\n        img_vowel_font = image_from_char(vowel_str)\n        img_consonant = get_sqfeat_image(idx, consonant_arr, img_org.size, cmap_autumn)\n        img_consonant_font = image_from_char(consonant_str)\n    \n        ax1_1 = fig.add_subplot(num_img * 2, 4, 4 * 2 * i + 1)\n        ax2_1 = fig.add_subplot(num_img * 2, 4, 4 * (2 * i + 1) + 1)\n        ax1_2 = fig.add_subplot(num_img * 2, 4, 4 * 2 * i + 2)\n        ax2_2 = fig.add_subplot(num_img * 2, 4, 4 * (2 * i + 1) + 2)\n        ax1_3 = fig.add_subplot(num_img * 2, 4, 4 * 2 * i + 3)\n        ax2_3 = fig.add_subplot(num_img * 2, 4, 4 * (2 * i + 1) + 3)\n        ax1_4 = fig.add_subplot(num_img * 2, 4, 4 * 2 * i + 4)\n        ax2_4 = fig.add_subplot(num_img * 2, 4, 4 * (2 * i + 1) + 4)\n\n        ax1_1.imshow(img_org)\n        ax1_1.xticks = None\n        ax2_1.imshow(img_org_font)\n        ax1_2.imshow(Image.blend(img_org, img_root, 0.7))\n        ax2_2.imshow(img_root_font)\n        ax1_3.imshow(Image.blend(img_org, img_vowel, 0.7))\n        ax2_3.imshow(img_vowel_font)\n        ax1_4.imshow(Image.blend(img_org, img_consonant, 0.7))\n        ax2_4.imshow(img_consonant_font)\n        \n        ax1_1.set_title(\"[grapheme] {}\".format(image_id), fontsize=14)\n        ax1_1.tick_params(labelbottom=False, labelleft=False)\n        ax2_1.tick_params(labelbottom=False, labelleft=False)\n        ax1_2.set_title(\"[grapheme_root] id: {}\".format(root_label), fontsize=14)\n        ax1_2.tick_params(labelbottom=False, labelleft=False)\n        ax2_2.tick_params(labelbottom=False, labelleft=False)\n        ax1_3.set_title(\"[vowel_diacritic]  id: {}\".format(vowel_label), fontsize=14)\n        ax1_3.tick_params(labelbottom=False, labelleft=False)\n        ax2_3.tick_params(labelbottom=False, labelleft=False)\n        ax1_4.set_title(\"[consonant_diacritic] id: {}\".format(consonant_label), fontsize=14)\n        ax1_4.tick_params(labelbottom=False, labelleft=False)\n        ax2_4.tick_params(labelbottom=False, labelleft=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### for consonant_diacrtic == 0 (nothing)"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:17:46.403503Z","start_time":"2020-03-21T06:17:46.388556Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"train_df.query(\"consonant_diacritic == 0\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:17:52.572957Z","start_time":"2020-03-21T06:17:48.737313Z"},"scrolled":true,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"visualize_squeezed_features(\n    train_df.query(\"consonant_diacritic == 0\").index.values, viz_dataset,\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr, label_arr, grapheme_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### for consonant_diacrtic == 1"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:19:02.052352Z","start_time":"2020-03-21T06:19:02.037300Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true,"_kg_hide-input":false},"cell_type":"code","source":"train_df.query(\"consonant_diacritic == 1\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:19:14.560729Z","start_time":"2020-03-21T06:19:12.329099Z"},"scrolled":true,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"visualize_squeezed_features(\n    train_df.query(\"consonant_diacritic == 1\").index.values, viz_dataset,\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr, label_arr, grapheme_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### for consonant_diacrtic == 2"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:20:03.902296Z","start_time":"2020-03-21T06:20:03.887524Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"train_df.query(\"consonant_diacritic == 2\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:20:24.053896Z","start_time":"2020-03-21T06:20:20.882520Z"},"scrolled":true,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"visualize_squeezed_features(\n    train_df.query(\"consonant_diacritic == 2\").index.values, viz_dataset,\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr, label_arr, grapheme_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### for consonant_diacrtic == 3"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:21:32.576217Z","start_time":"2020-03-21T06:21:32.559265Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true,"_kg_hide-input":false},"cell_type":"code","source":"train_df.query(\"consonant_diacritic == 3\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:21:44.213649Z","start_time":"2020-03-21T06:21:43.877175Z"},"scrolled":true,"trusted":true,"_kg_hide-output":false,"_kg_hide-input":true},"cell_type":"code","source":"visualize_squeezed_features(\n    train_df.query(\"consonant_diacritic == 3\").index.values, viz_dataset,\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr, label_arr, grapheme_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### for consonant_diacrtic == 4"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:22:19.473578Z","start_time":"2020-03-21T06:22:19.454012Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"train_df.query(\"consonant_diacritic == 4\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:22:32.611739Z","start_time":"2020-03-21T06:22:30.062100Z"},"scrolled":true,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"visualize_squeezed_features(\n    train_df.query(\"consonant_diacritic == 4\").index.values, viz_dataset,\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr, label_arr, grapheme_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### for consonant_diacrtic == 5"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:23:09.792727Z","start_time":"2020-03-21T06:23:09.779290Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"train_df.query(\"consonant_diacritic == 5\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:23:26.973864Z","start_time":"2020-03-21T06:23:23.529675Z"},"scrolled":false,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"visualize_squeezed_features(\n    train_df.query(\"consonant_diacritic == 5\").index.values, viz_dataset,\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr, label_arr, grapheme_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### for consonant_diacrtic == 6"},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:24:21.872705Z","start_time":"2020-03-21T06:24:21.858027Z"},"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"train_df.query(\"consonant_diacritic == 6\")","execution_count":null,"outputs":[]},{"metadata":{"ExecuteTime":{"end_time":"2020-03-21T06:24:30.693840Z","start_time":"2020-03-21T06:24:30.029094Z"},"scrolled":false,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"visualize_squeezed_features(\n    train_df.query(\"consonant_diacritic == 6\").index.values, viz_dataset,\n    sqfeat_root_arr, sqfeat_vowel_arr, sqfeat_consonant_arr, label_arr, grapheme_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Discussion\n\nFor me, difference between squeezed features of each component looks not so large.\n\nBut this result gives me some findings.\n\n### In most cases, All the sSE Blocks look the grapheme, **_not the white space_**.\nI think this behavior contributes to model performance. Simple GAP doesn't do this.\n\n### sSE Blocks of each component look different areas.\n\nAs you see above, this difference seems to be just a little. But there is **certainly** a difference.  \nRoughly, sSE Block of `grapheme_root` looks relatively wide area, and one of `consonant_diacritic` relatively narrow area.\n\n<br>\n\nWe know **_the devils are in the details_**, I think these small differences contribute to my solo gold place solution.\n\nThank you for reading! I'm glad to share this somewhat interesting result."}],"metadata":{"hide_input":false,"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.9"},"latex_envs":{"LaTeX_envs_menu_present":true,"autoclose":false,"autocomplete":true,"bibliofile":"biblio.bib","cite_by":"apalike","current_citInitial":1,"eqLabelWithNumbers":true,"eqNumInitial":1,"hotkeys":{"equation":"Ctrl-E","itemize":"Ctrl-I"},"labels_anchors":false,"latex_user_defs":false,"report_style_numbering":false,"user_envs_cfg":false},"toc":{"base_numbering":1,"nav_menu":{},"number_sections":true,"sideBar":true,"skip_h1_title":false,"title_cell":"Table of Contents","title_sidebar":"Contents","toc_cell":false,"toc_position":{},"toc_section_display":true,"toc_window_display":false}},"nbformat":4,"nbformat_minor":4}