{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":11848,"databundleVersionId":862157,"sourceType":"competition"}],"dockerImageVersionId":31192,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Histopathologic Image Classification Using Convolutiona Neural Network (in PyTorch)\n---\n\n# Overview\nThe notebook documents a computer vision binary classification problem. Specifically, the dataset composes of 220,025x microscopies of 96x96 pixels with corresponding labels indicating whether the center 32x32 region contains metastatic cancer cells. A convolutional neural network of 4x convolutional layers was built using PyTorch and training is done over 10x epochs with various hyperparameters.\n\nADAM (Adaptive Moment Estimation) was used to optimized the training along with Negative Log Likelihood (NLL) as the loss metric. The rationale for using ADAM and NLL is for their versatility to and ability to adapt to more general classification problems beyond a binary classification problem for future investigations.\n\nThe notebook has yet to implemented the desired tuning tool (Ray Tune) as a hyperparameter tuning tool and thus tuning was done manually and thus there are plenty of room for the model's performance. However, undocumented are (1) the experimentation of different CNN filter-size / layers, and (2) data batch-size, prefetch size, number of workers using the PyTorch Dataloader wrapper.","metadata":{}},{"cell_type":"code","source":"################################################################################\n## INSTALL AND UPGRADE LIBRARIES\n################################################################################\n\n## Intall and Upgrade libraries\nprint(\"Installing / Upgrading packages...\")\n!pip install --no-input 'torchinfo==1.8' \n\nprint(\"#\"*80)\nprint(\"Done installing! >>> Remember to restart the kernel!!! Else the updates wont be reflected.\")\nprint(\"#\"*80)\n\n## NOTE:\n################################################################################\n## - When \"Save and Run All\" using Kaggle, we don't need to restart the kernel, \n##   it seems that Kaggle would miraculously load the installed / upgraded \n##   package. \n## - Just for precaution, there are package version assertions to make sure that\n##   the kernel is loading the version needed.\n\n# ## IGNORE - Trick to restart kernel from keyboard\n# # Src: https://stackoverflow.com/a/62744676/13697827\n# import IPython\n# IPython.Application.instance().kernel.do_shutdown(True) # Automatically restarts kernel\n\n\n################################################################################\n## Import Libraries\n################################################################################\nimport torch\nfrom torch.utils.data import DataLoader\nimport torchvision\nfrom torchvision.transforms import transforms \nfrom torchvision.transforms import v2 # PyTorch recommends using v2 - https://pytorch.org/vision/main/transforms.html#v1-or-v2-which-one-should-i-use\nimport torchinfo\nimport torch.nn as nn\nimport torch.nn.functional as func\nfrom pathlib import Path\nimport pandas as pd\nfrom PIL import Image\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom tqdm.autonotebook import tqdm\nimport datetime as dt\nimport os, sys, psutil, time, pickle, json\n\n## IGNORE - Checks\nprint(f\"Current working directory: {os.getcwd()}\")\nprint(f\"CPU Core count (physical): {psutil.cpu_count(logical=False)}\")\nprint(f\"CUDA availability: {torch.cuda.is_available()}\")\n\n## Silence warnings\ntorchvision.disable_beta_transforms_warning()\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:55:49.801395Z","iopub.execute_input":"2023-11-20T14:55:49.801940Z","iopub.status.idle":"2023-11-20T14:56:07.351048Z","shell.execute_reply.started":"2023-11-20T14:55:49.801912Z","shell.execute_reply":"2023-11-20T14:56:07.350058Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## HELPER FUNCTION - Plot the training and validation loss for each epoch\n################################################################################\n\ndef plot_results(config, training_loss, validation_loss, filename_path):\n    \"\"\"Helper Function: Plot and save the train/validation loss average over epochs.\"\"\"\n    print_textbox = False\n    if print_textbox: textbox_text = \"\\n\".join(  \"=\".join((key, str(value))) for (key, value) in config.items()  )\n    \n    training_loss = np.mean(training_loss, axis=1)\n    validation_loss = np.mean(validation_loss, axis=1)\n    \n    fig, ax = plt.subplots(1, 1)\n    ax.plot(training_loss, \n            linestyle=\"dashed\", \n            linewidth=2, \n            alpha=0.5,\n            label=\"Training Loss (avg per epoch)\")\n    ax.scatter(x=range(len(training_loss)),\n               y=training_loss)\n    ax.plot(validation_loss, \n            linestyle=\"dashed\", \n            linewidth=2, \n            alpha=0.5,\n            label=\"Validation Loss (avg per epoch)\")\n    ax.scatter(x=range(len(validation_loss)), \n               y=validation_loss)\n\n    ax.set_title(f\"Training/Validation Loss of Trial: {config['trial_name']}\")\n    ax.set_xlabel(\"Epoch\")\n    ax.set_ylabel(\"Loss\")\n    if print_textbox: \n        ax.text(0.9, 0.1, textbox_text, fontsize=12, \n            transform=ax.transAxes,\n            verticalalignment=\"bottom\",\n            horizontalalignment=\"right\", \n            bbox=dict(boxstyle=\"round\", facecolor='wheat', alpha=0.5))\n    ax.legend()\n    \n    fig.show()\n    fig.savefig(Path(filename_path))\n    return\n\n################################################################################\n## HELPER FUNCTION - Calculate conv layer input and outputs dimensions\n################################################################################\n\ndef calc_dim(height:int, width:int, conv_layer:torch.nn):\n    \"\"\"Calculate the output dimensions for convolutional layers.\"\"\"\n    \n    ## Case checking - Sometimes the values can be returned as int instead of a tuple\n    kernel_h, kernel_w = (\n        (conv_layer.kernel_size, conv_layer.kernel_size) if isinstance(conv_layer.kernel_size, int) \n        else conv_layer.kernel_size\n    )\n    \n    padding_h, padding_w = (\n        (conv_layer.padding, conv_layer.padding) if isinstance(conv_layer.padding, int) \n        else conv_layer.padding\n    )\n    \n    stride_h, stride_w = (\n        (conv_layer.stride, conv_layer.stride) if isinstance(conv_layer.stride, int) \n        else conv_layer.stride\n    )\n    \n    dilation_h, dilation_w = (\n        (conv_layer.dilation, conv_layer.dilation) if isinstance(conv_layer.dilation, int) \n        else conv_layer.dilation\n    )\n    \n    ## Calculate the output dimensions - Src: https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html\n    output_height = int( (height + 2*padding_h - dilation_h*(kernel_h - 1) - 1)/stride_h + 1 )\n    output_width = int( (width + 2*padding_w - dilation_w*(kernel_w-1) - 1)/stride_w +1 )\n    \n    return (output_height, output_width)\n    \n\n## IGNORE - Tests\ntest_conv_layer = nn.Conv2d(3, 3, 5, padding=1)\nassert (3, 3) == calc_dim(5, 5, test_conv_layer)","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:07.353199Z","iopub.execute_input":"2023-11-20T14:56:07.353827Z","iopub.status.idle":"2023-11-20T14:56:07.394438Z","shell.execute_reply.started":"2023-11-20T14:56:07.353793Z","shell.execute_reply":"2023-11-20T14:56:07.393491Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Exploratory Data Analysis (EDA)\nDue to the dataset being image data instead of tabular data with dimensions, there is not much we can do in terms of exploratory data analysis without having to process the image using some kind of computer vision techniques.\n\nInstead, the only thing we can do is to analyze the balance of the labels (non-cancer vs cancer). Which we found that there are approximately 46.9% more non-cancerous images than cancerous ones (see bar-chart below).\n\n# Image Processing (Transformation)\nThe dataset contains images of 96x96 pixels, however, the labels only indicate whether the center 32x32 region contains at least one metastatic cancer cell. The additional pixels ignored in the labeling have the same effect as adding padding around the input images for each CNN convolution layers.\n\nDue to the size of this modified dataset being relatively large (~7GB) for the limited amount of training resource we have on hand, during the image processing (transformation) stage, the image is center-cropped to a 46x46 square, still providing a 14-pixel around the region of interest (32x32), but removes the extra pixels that would only have increased the I/O reading time of our pipeline.\n\nThe image processing is done using a pipeline of transformers. The pipeline converts the image to a PyTorch Tensor and normalizes each of the three RGB layers pixel values from range 0-255 to 0-1 for a smoother error surface that facilitates better model convergence.\n\nIn future improvements of the model, additional histology images can be generated from this dataset by flipping/rotating the existing images. Since histology images are not limited to specific orientation and can be slice from an organ/organism at any orientation, we would like to ensure that the model is trained on all possible permutation. \n\nIt would also be interesting and beneficial if we are able to provide additional context to the model such as the age of the subject, organ of origin","metadata":{}},{"cell_type":"code","source":"################################################################################\n## Wrap the TRAIN dataset with PyTorch utility wrapper\n################################################################################\n\nclass DatasetWrapper_Train(torch.utils.data.Dataset):\n    # TODO: Redo the function so that data can be downloaded without being on Kaggle\n    \n    def __init__(self):\n        \"\"\"PyTorch utility wrapper that provides a consistent interface for our data.\"\"\"\n                \n        ## Construct an image transformer\n        to_tensor_transformer = transforms.ToTensor()\n        center_crop_transformer = v2.CenterCrop(size=(46, 46))\n        composite_transformer = transforms.Compose([to_tensor_transformer,  \n                                                    center_crop_transformer,\n                                                   ])\n        self.transformer = composite_transformer\n        \n        # Gather training data\n        cwd = Path.cwd()\n        input_path = Path(\"/kaggle/input/\")\n        dataset_name = \"histopathologic-cancer-detection\"\n        data_dir = Path(input_path, dataset_name, \"train\")\n        self.list_of_filenames = [file.name for file in data_dir.iterdir()]\n        self.list_of_fullpaths = list(data_dir.iterdir())\n        self.dataset_size = len(self.list_of_fullpaths)\n        \n        # Gather labels\n        labels_path = Path(input_path, dataset_name, \"train_labels.csv\")\n        labels_df = pd.read_csv(labels_path).set_index(\"id\")\n        ## Matching the label with each file name\n        self.labels = [labels_df.loc[name[:-4], \"label\"] for name in self.list_of_filenames]\n        self.labels_count = len(self.labels)\n        \n        \n    def __len__(self):\n        \"\"\"Returns the number of data entries.\"\"\"\n        return self.dataset_size\n    \n    def __getitem__(self, idx): \n        \"\"\"Get the i-th entry of transformed data.\n        \n        Args: \n            idx (int): Index of the image and label to get.\n        \n        Returns:\n            (img, label): Transformed image as PyTorch Tensor, label as an int.\n        \"\"\"\n        # Src: https://pytorch.org/vision/main/auto_examples/transforms/plot_transforms_illustrations.html#sphx-glr-auto-examples-transforms-plot-transforms-illustrations-py\n        with Image.open(self.list_of_fullpaths[idx]) as image: \n            image_transformed = self.transformer(image)\n            label = self.labels[idx]\n        return (image_transformed, label)\n    \n    def get_untransformed(self, idx): \n        \"\"\"Get the i-th entry of untransformed data.\n        \n        Args: \n            idx (int): Index of the image and label to get.\n        \n        Returns:\n            (img, label): Untransformed image as a PIL image, label as an int.\n        \"\"\"\n        image = Image.open(self.list_of_fullpaths[idx])\n        label = self.labels[idx]\n        \n        return (image, label)\n\n    \n## Gather and transform the image\ntrain_dataset = DatasetWrapper_Train()\nprint(f\"training_data_tensor size: {sys.getsizeof(train_dataset)} bytes\")\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:07.395768Z","iopub.execute_input":"2023-11-20T14:56:07.396441Z","iopub.status.idle":"2023-11-20T14:56:15.595236Z","shell.execute_reply.started":"2023-11-20T14:56:07.396407Z","shell.execute_reply":"2023-11-20T14:56:15.594237Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Wrap the TEST dataset with PyTorch utility wrapper\n################################################################################\n\nclass DatasetWrapper_Test(torch.utils.data.Dataset):\n    # TODO: Redo the function so that data can be downloaded without being on Kaggle\n    \n    def __init__(self):\n        \"\"\"PyTorch utility wrapper that provides a consistent interface for our data.\"\"\"\n                \n        ## Construct an image transformer\n        to_tensor_transformer = transforms.ToTensor()\n        center_crop_transformer = v2.CenterCrop(size=(46, 46))\n        composite_transformer = transforms.Compose([to_tensor_transformer,  \n                                                    center_crop_transformer,\n                                                   ])\n        self.transformer = composite_transformer\n        \n        # Gather training data\n        cwd = Path.cwd()\n        input_path = Path(\"/kaggle/input/\")\n        dataset_name = \"histopathologic-cancer-detection\"\n        data_dir = Path(input_path, dataset_name, \"test\")\n        self.list_of_filenames = [file.name for file in data_dir.iterdir()]\n#         self.list_of_fullpaths = list(data_dir.iterdir())\n        self.list_of_fullpaths = [Path(data_dir, filename) for filename in self.list_of_filenames]\n        self.dataset_size = len(self.list_of_fullpaths)\n        \n        \n    def __len__(self):\n        \"\"\"Returns the number of data entries.\"\"\"\n        return self.dataset_size\n    \n    def __getitem__(self, idx): \n        \"\"\"Get the i-th entry of transformed data.\n        \n        Args: \n            idx (int): Index of the image and label to get.\n        \n        Returns:\n            img: Transformed image as PyTorch Tensor\n        \"\"\"\n        # Src: https://pytorch.org/vision/main/auto_examples/transforms/plot_transforms_illustrations.html#sphx-glr-auto-examples-transforms-plot-transforms-illustrations-py\n        with Image.open(self.list_of_fullpaths[idx]) as image: \n            image_transformed = self.transformer(image)\n        file_id = self.list_of_filenames[idx][:-4]\n        return (file_id, image_transformed)\n    \n    def get_untransformed(self, idx): \n        \"\"\"Get the i-th entry of untransformed data.\n        \n        Args: \n            idx (int): Index of the image and label to get.\n        \n        Returns:\n            (img, label): Untransformed image as a PIL image, label as an int.\n        \"\"\"\n        image = Image.open(self.list_of_fullpaths[idx])\n        file_id = self.list_of_filenames[idx][:-4]\n        \n        return (file_id, image)\n\n    \n## Gather and transform the image\ntest_dataset = DatasetWrapper_Test()\nprint(f\"testing_data_tensor size: {sys.getsizeof(test_dataset)} bytes\")\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:15.598204Z","iopub.execute_input":"2023-11-20T14:56:15.598609Z","iopub.status.idle":"2023-11-20T14:56:17.167002Z","shell.execute_reply.started":"2023-11-20T14:56:15.598563Z","shell.execute_reply":"2023-11-20T14:56:17.166024Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Preview what the transformed images look like\n################################################################################\n\ntraining_set_wrapped = DatasetWrapper_Train()\nprint(f\"Dataset size: {len(training_set_wrapped)}\")\nborder_width = 3\nn_row, n_col = 5, 10\ncolors_set = [\"#249A41\", \"#E92E18\"]\n\n## Create the figure\nfig, ax = plt.subplots(n_row, n_col, figsize=(n_col*1.3, n_row*1.3), facecolor=\"#EFEFEF\")\nfig.suptitle(\"UNTRANSFORMED Image Preview\\n(Green=No-Cancer, Red=Cancer)\")\n\n\nfor i in range(n_row):\n    for j in range(n_col): \n        idx = n_row * i + j\n        image, label = training_set_wrapped.get_untransformed(idx)\n        \n        ## Show the image\n        ax[i, j].imshow(image)\n        \n        ## Configure the shown image\n        ax[i, j].spines[:].set_color(colors_set[label])  # SpinesProxy broadcasts the method call to all spines\n        ax[i, j].spines[:].set_linewidth(border_width)   # SpinesProxy broadcasts the method call to all spines\n        ax[i, j].set_xticklabels([])\n        ax[i, j].set_xticks([])\n        ax[i, j].set_yticklabels([])\n        ax[i, j].set_yticks([])\n        \n        ## Add a rectangle patch\n        left, bottom, width, height = (32, 32, 32, 32)\n        rectangle = plt.Rectangle((left, bottom), width, height, \n                                  facecolor=\"yellow\", alpha=0.2)\n        ax[i, j].add_patch(rectangle)\n\n## Matplotlib show\nfig.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:17.168319Z","iopub.execute_input":"2023-11-20T14:56:17.168733Z","iopub.status.idle":"2023-11-20T14:56:25.593642Z","shell.execute_reply.started":"2023-11-20T14:56:17.168693Z","shell.execute_reply":"2023-11-20T14:56:25.592550Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Preview what the transformed images look like\n################################################################################\n\ntraining_set_iterator = iter(DatasetWrapper_Train())\nborder_width = 3\nn_row, n_col = 5, 10\ncolors_set = [\"#249A41\", \"#E92E18\"]\n\n## Create the figure\nfig, ax = plt.subplots(n_row, n_col, figsize=(n_col*1.3, n_row*1.3), facecolor=\"#EFEFEF\")\nfig.suptitle(\"TRANSFORMED Image Preview\\n(Green=No-Cancer, Red=Cancer)\")\n\n\nfor i in range(n_row):\n    for j in range(n_col): \n        image, label = next(training_set_iterator)  # Fetch the next image\n        image = np.transpose(image, [1, 2, 0])      # PyTorch Tensor is in a different ordering, thus need to transpose\n        \n        ## Show the image\n        ax[i, j].imshow(image)\n        \n        ## Configure the shown image\n        ax[i, j].spines[:].set_color(colors_set[label])  # SpinesProxy broadcasts the method call to all spines\n        ax[i, j].spines[:].set_linewidth(border_width)   # SpinesProxy broadcasts the method call to all spines\n        ax[i, j].set_xticklabels([])\n        ax[i, j].set_xticks([])\n        ax[i, j].set_yticklabels([])\n        ax[i, j].set_yticks([])\n        \n        ## Add a rectangle patch\n        left, bottom, width, height = (7, 7, 32, 32)\n        rectangle = plt.Rectangle((left, bottom), width, height, \n                                  facecolor=\"yellow\", alpha=0.2)\n        ax[i, j].add_patch(rectangle)\n\n## Matplotlib show\nfig.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:25.594801Z","iopub.execute_input":"2023-11-20T14:56:25.595109Z","iopub.status.idle":"2023-11-20T14:56:34.105825Z","shell.execute_reply.started":"2023-11-20T14:56:25.595083Z","shell.execute_reply":"2023-11-20T14:56:34.104819Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Analyze the balance of labels\n################################################################################\n\n## Get an instance of the dataloader\ntraining_set = DatasetWrapper_Train()\n\n## Count the number of cancer / no-cancer labels\ncount_no_cancer = np.count_nonzero(np.equal(training_set.labels, 0))\ncount_cancer = np.count_nonzero(np.equal(training_set.labels, 1))\n\n## Check\nprint(\"No-cancer count: \", count_no_cancer)\nprint(\"Cancer count: \", count_cancer)\n\n## Plot bar chart of cancer vs no cancer \nfig, ax = plt.subplots(figsize=(3, 5))\nbarchart = ax.bar(\n    [\"no_cancer\", \"cancer\"], [count_no_cancer, count_cancer], \n    color=[\"#249A41\", \"#E92E18\"],\n    width=0.3,\n    align=\"center\",\n)\nax.bar_label(barchart, labels=[count_no_cancer, count_cancer], padding=1)\nax.set_title(\"Training Set Label Counts\")\nax.text(\n    0.3, 122000, f\"no_cancer:cancer = {round(count_no_cancer/count_cancer, 3)}\", \n    ha=\"left\", va=\"center\",\n    wrap=True,\n    bbox=dict(facecolor=\"#EFEFEF\", \n              alpha=0.3)\n)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:34.106920Z","iopub.execute_input":"2023-11-20T14:56:34.107222Z","iopub.status.idle":"2023-11-20T14:56:39.710296Z","shell.execute_reply.started":"2023-11-20T14:56:34.107197Z","shell.execute_reply":"2023-11-20T14:56:39.709352Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Train Test Split\n################################################################################\n\ndef train_test_split(dataset_to_split:torch.utils.data.Dataset, \n                     train_proportion:float=0.8, \n                     random_seed:int=1234, \n                     subset_size:int=None):\n    \"\"\"Split a dataset to train and validation sets.\n    \n    Args: \n        dataset_to_split (torch.utils.data.Dataset): The dataset to be split.\n        train_proportion (float): Proportion of training set, default=0.8.\n        random_seed (int): Seed for random generator for reproducibility.\n        subset_size (int): If set, the dataset to be split will be truncated to this size.\n    \n    Returns: \n        (train_set, test_set)\n    \"\"\"\n    assert ((train_proportion>0) & (train_proportion<1)), f\"Train proportion has to be (0, 1), got {train_proportion}.\"\n    \n    if subset_size != None:\n        dataset_to_split = torch.utils.data.Subset(dataset_to_split, range(subset_size))\n    \n    generator1 = torch.Generator().manual_seed(1234)  # To allow reproducibility\n\n    # Split to train/test subsets (including labels)\n    train_set, test_set = torch.utils.data.random_split(\n        dataset_to_split, \n        [train_proportion, 1-train_proportion], \n        generator=generator1\n    )\n    \n    return (train_set, test_set)\n\n## IGNORE - sanity check\ntraining_set, testing_set = train_test_split(train_dataset)\n\nprint(f\"The training set has {len(training_set)} entries.\")\nprint(f\"The testing set has {len(testing_set)} entries.\")\nprint()\n\nfor i, (image, label) in enumerate(training_set): \n    if i > 10: break  # Checking the first few entries\n    print(f\"Image: {i} | On: {image.device} | Shape: {image.shape} | Label: {label}\")\n    ","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:39.711556Z","iopub.execute_input":"2023-11-20T14:56:39.711820Z","iopub.status.idle":"2023-11-20T14:56:39.879845Z","shell.execute_reply.started":"2023-11-20T14:56:39.711798Z","shell.execute_reply":"2023-11-20T14:56:39.878763Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Model Architecture\nThe model is inspired by the LeNet model where there are multiples of (convolutional layer + pooling) as feature extraction section, followed by two linear layers with log-softmax as final output used as a multi-layer perceptron classifier. The log-softmax provides easy adaptation for when we want to adjust and train the network for more than two classification labels.\n\n## Feature Extraction Section\nThere are 4x (convolution + ReLU activation + maxpooling) layers each with no-padding and 3x3 filter kernels. Due to how small the region of interest is (32x32), I chose a smaller filter kernel and deepen the model depth as with each increased depth, the field of perception increases thus allowing the model to indirectly capture a larger invariant feature.\n\n## Classification Section\nThe classification section is composed of two linear layer with a log-softmax as the final output. The benefit of log-softmax is its ability to accomodate for any number of classes thus the model can be easily adapted to classify more than 2 categories.","metadata":{}},{"cell_type":"code","source":"################################################################################\n## Create a CNN model object\n################################################################################\n\nclass CustomCNN(nn.Module):\n    def __init__(self, input_channels:int=3, input_height:int=46, input_width:int=46):\n        super().__init__()  # nn.Module constructor\n        \n        self.input_channels = input_channels\n        self.input_height = input_height\n        self.input_width = input_width\n        \n        ## Create/Define the loss function\n        self._loss_function = nn.NLLLoss\n        \n        ## Create all the layers for the network\n        # Layer 1\n        self.conv1 = nn.Conv2d(in_channels=input_channels, \n                               out_channels=8, \n                               kernel_size=3, \n                               padding=0)\n        self.leak_relu1 = nn.LeakyReLU()  # Activation layer doesn't change the output dimensions\n        out_h, out_w = calc_dim(input_height, input_width, self.conv1)\n        self.maxpool1 = nn.MaxPool2d(2)\n        out_h, out_w = calc_dim(out_h, out_w, self.maxpool1)\n                               \n        # Layer 2\n        self.conv2 = nn.Conv2d(in_channels=self.conv1.out_channels, \n                               out_channels=16, \n                               kernel_size=3, \n                               padding=0)\n        self.leak_relu2 = nn.LeakyReLU()\n        out_h, out_w = calc_dim(out_h, out_w, self.conv2)\n        self.maxpool2 = nn.MaxPool2d(2)\n        out_h, out_w = calc_dim(out_h, out_w, self.maxpool2)\n        \n       # Layer 3\n        self.conv3 = nn.Conv2d(in_channels=self.conv2.out_channels, \n                               out_channels=32, \n                               kernel_size=3, \n                               padding=0)\n        self.leak_relu3 = nn.LeakyReLU()\n        out_h, out_w = calc_dim(out_h, out_w, self.conv3)\n        self.maxpool3 = nn.MaxPool2d(2)\n        out_h, out_w = calc_dim(out_h, out_w, self.maxpool3)\n        \n       # Layer 4\n        self.conv4 = nn.Conv2d(in_channels=self.conv3.out_channels, \n                               out_channels=64, \n                               kernel_size=3, \n                               padding=0)\n        self.leak_relu4 = nn.LeakyReLU()\n        out_h, out_w = calc_dim(out_h, out_w, self.conv4)\n        self.maxpool4 = nn.MaxPool2d(2)\n        out_h, out_w = calc_dim(out_h, out_w, self.maxpool4)\n\n        # Classification Layer 1\n        self.flattened_dim = out_h * out_w * self.conv4.out_channels\n        self.dense1 = nn.Linear(in_features = self.flattened_dim, \n                                out_features = 100)\n        self.leak_relu4 = nn.LeakyReLU()\n                               \n        # Classification Layer 2\n        self.dense2 = nn.Linear(in_features = 100, \n                                out_features = 2)  # Binary classification\n        self.logsoftmax = nn.LogSoftmax(dim=1)\n        \n    def forward(self, x):\n        \"\"\"Override nn.Module forward method.\"\"\"\n        x = self.conv1(x)\n        x = self.leak_relu1(x)\n        x = self.maxpool1(x)\n                               \n        x = self.conv2(x)\n        x = self.leak_relu2(x)\n        x = self.maxpool2(x)\n                               \n        x = self.conv3(x)\n        x = self.leak_relu3(x)\n        x = self.maxpool3(x)\n        \n        x = self.conv4(x)\n        x = self.leak_relu4(x)\n        x = self.maxpool4(x)\n\n        x = x.view(-1, self.flattened_dim)  # Flatten the tensor                       \n        x = self.dense1(x)\n        x = self.leak_relu4(x)\n        \n        x = self.dense2(x)\n        x = self.logsoftmax(x)\n        \n        return x\n    \n    def get_optimizer(self, **kwargs):\n        \"\"\"Return the optimizer used by this network.\"\"\"\n        return torch.optim.Adam(self.parameters(), **kwargs)\n    \n    def get_loss_function(self, **kwargs):\n        \"\"\"Return the loss function used by this network\"\"\"\n        return self._loss_function(**kwargs)\n    \n    def show_summary(self):\n        \"\"\"Show the model summary information.\"\"\"\n        return torchinfo.summary(\n            self, \n            input_size=(self.input_channels, self.input_height, self.input_width), \n            device=torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\").type,\n            verbose=0)\n\n## IGNORE - TEST\ntemp_model = CustomCNN()\ntemp_model.show_summary()\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:39.881061Z","iopub.execute_input":"2023-11-20T14:56:39.881364Z","iopub.status.idle":"2023-11-20T14:56:48.865659Z","shell.execute_reply.started":"2023-11-20T14:56:39.881338Z","shell.execute_reply":"2023-11-20T14:56:48.864778Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Training and Validation\nThe following function is the main training/validation loop. \nA configuration dictionary is used to specify the hyperparameters of the model along with the number of epochs as the function arguments. After each epoch, a checkpoint is saved locally for later retrieval to continue training or fail-safe if unexpected interruption were to occur.","metadata":{}},{"cell_type":"code","source":"################################################################################\n## Train the model\n################################################################################\n\ndef train_the_model(config:dict, n_epochs:int=1, verbose:bool=False, progress_print:bool=False):\n    \"\"\"Training loop that connects everything! See comments for details.\n\n    Args: \n        config (dict): Dictionary holding various parameters for the model\n        n_epochs (int): Number of epochs to train and validate.\n        verbose (bool): Whether to print out the profiling outputs.\n    \"\"\"\n\n    ## Determine whether to use CPU/GPU\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    print(f\"|| Using {device} ||\")\n    \n    ## Preparing dataset and train/test split it: \n    dataset_to_split = DatasetWrapper_Train()\n    training_set, testing_set = train_test_split(dataset_to_split, train_proportion=0.8)\n    \n    ## Instantiate a CNN model instance\n    model = CustomCNN()       # Model instance\n    model = model.to(device)  # Move model to the device\n    ## Get the optimizer from model instance\n    optimizer = model.get_optimizer(                   # Get the optimizer (ADAM)\n        lr=config[\"lr\"],                           # Get the learning rate from Ray Tune config\n        betas=(config[\"beta1\"], config[\"beta2\"]),  # Get the beta from Ray Tune config \n        weight_decay=config[\"weight_decay\"],       # Get the weight decay from Ray Tune config\n    )\n    ## Get te loss func from the model instance\n    loss_function = model.get_loss_function(reduction='sum')\n    \n    ## Create dataloader - for convenience\n    training_dataloader = torch.utils.data.DataLoader(\n        dataset=training_set, \n        batch_size=config[\"batch_size\"], \n        shuffle=True,\n        num_workers=config['dataloader_worker_count'],\n        prefetch_factor=config[\"dataloader_prefetch_factor\"],\n        drop_last=True,  # The last few data that doesn't form a full batch gets dropped.\n    )\n    validation_dataloader = torch.utils.data.DataLoader(\n        dataset=testing_set, \n        batch_size=config[\"batch_size\"], \n        shuffle=True,\n        num_workers=config['dataloader_worker_count'],\n        prefetch_factor=config[\"dataloader_prefetch_factor\"],\n        drop_last=True,  # The last few data that doesn't form a full batch gets dropped.\n    )\n    \n    training_dataset_size = len(training_dataloader.dataset)\n    training_batches_per_epoch = int(training_dataset_size / config['batch_size'])  # Round down because drop_last==True\n    \n    validation_dataset_size = len(validation_dataloader.dataset)\n    validation_batches_per_epoch = int(validation_dataset_size / config['batch_size'])  # Round down because drop_last==True\n\n    ## Various tracker for data output\n    track_training_loss     = np.zeros((n_epochs, training_batches_per_epoch), dtype=np.float64)\n    track_training_TP_count = np.zeros((n_epochs, training_batches_per_epoch), dtype=np.int64)\n    track_training_TN_count = np.zeros((n_epochs, training_batches_per_epoch), dtype=np.int64)\n    track_training_FP_count = np.zeros((n_epochs, training_batches_per_epoch), dtype=np.int64)\n    track_training_FN_count = np.zeros((n_epochs, training_batches_per_epoch), dtype=np.int64)\n    \n    track_validation_loss     = np.zeros((n_epochs, validation_batches_per_epoch), dtype=np.float64)\n    track_validation_TP_count = np.zeros((n_epochs, validation_batches_per_epoch), dtype=np.int64)\n    track_validation_TN_count = np.zeros((n_epochs, validation_batches_per_epoch), dtype=np.int64)\n    track_validation_FP_count = np.zeros((n_epochs, validation_batches_per_epoch), dtype=np.int64)\n    track_validation_FN_count = np.zeros((n_epochs, validation_batches_per_epoch), dtype=np.int64)\n\n#     loaded_checkpoint = train.get_checkpoint()  # Ray Tune keeps track of where the checkpoints are saved\n#     if loaded_checkpoint:\n#         with loaded_checkpoint.as_directory() as loaded_checkpoint_dir:\n#             checkpoint_dict = torch.load(\n#                 Path(loaded_checkpoint_dir, \"checkpoint.pt\")\n#             )\n#         ## Load the state dict into the model and optimizer\n#         model.load_state_dict(checkpoint_dict[\"model_state_dict\"])\n#         optimizer.load_state_dict(checkpoint_dict[\"optimizer_state_dict\"])\n    \n    \n    ## Main Training / Validation Loop\n    for epoch in tqdm(range(n_epochs), desc=\"Epochs...\"):  # Default to one epoch (because dataset is huge!!!)\n        if progress_print: print(f\"{epoch} / {n_epochs} \", \"#\"*50)\n        ########## TRAINING PORTION ##########\n        total_training_loop_start = time.time()  # Timer\n        for batch_idx in tqdm(range(training_batches_per_epoch), desc=\"TRAINING PORTION\"):\n            if verbose: start_time = time.time()  # Timer\n            if batch_idx == 0:  # Magic of dataloader! \n                data_loader_iter = iter(training_dataloader)\n                images, labels = next(data_loader_iter)\n            else: \n                images, labels = next(data_loader_iter)\n            if progress_print: print(f\"{batch_idx} / {training_batches_per_epoch}\", \"#\"*50)  # Timer\n            if verbose: print(f\"Timer - Load Dataloader batch (Train) : {time.time() - start_time}\")  # Timer\n            if verbose: start_time = time.time()  # Timer\n            images = images.to(device)\n            labels = labels.to(device)\n            if verbose: print(f\"Timer - Move to {device} : {time.time() - start_time}\")  # Timer\n            #print(f\"Images Device: {images.device}, Labels Device: {labels.device}\")  # Debug use\n\n            ## Set model to training mode\n            if verbose: start_time = time.time()  # Timer\n            model.train()\n            outputs = model(images)  # Inference: The model outputs are in log(proba) scale\n            max_value, max_idx = torch.max(outputs, dim=1)\n            prediction = max_idx  # Classification \n            ## Calculate metrics\n            loss = loss_function(outputs, labels)  # Calculate the loss\n            ## Backprop\n            optimizer.zero_grad()  # Zero out the loss gradient - https://pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam.zero_grad\n            loss.backward()        # Back propagate the loss\n            optimizer.step()       # Perform a single optimization step - https://pytorch.org/docs/stable/_modules/torch/optim/adam.html#Adam\n            if verbose: print(f\"Timer - Train+Backprop : {time.time() - start_time}\")  # Timer\n            ## Update variables\n            num_of_batches_trained = batch_idx + 1\n            ## Update trackers\n            if progress_print: print(\"Training loss: \", loss.item())  # Debug use\n            track_training_loss[epoch, batch_idx] = loss.item()\n            track_training_TP_count[epoch, batch_idx] = ((prediction==1) & (labels==1)).sum().item()\n            track_training_TN_count[epoch, batch_idx] = ((prediction==0) & (labels==0)).sum().item()\n            track_training_FP_count[epoch, batch_idx] = ((prediction==1) & (labels==0)).sum().item()\n            track_training_FN_count[epoch, batch_idx] = ((prediction==0) & (labels==1)).sum().item()\n        total_training_loop_time = time.time() - total_training_loop_start  # Timer\n        print(f\"Timer - ENTIRE TRAINING PORTION : {total_training_loop_time}\")\n\n        ## Validation Loop\n        print(\"Validation loop\")\n        total_validation_loop_start = time.time()  # Timer\n        for batch_idx in tqdm(range(validation_batches_per_epoch), desc=\"VALIDATION PORTION\"): \n            if verbose: start_time = time.time()  # Timer\n            if batch_idx == 0:  # Magic of Dataloader\n                data_loader_iter = iter(validation_dataloader)\n                images, labels = next(data_loader_iter)\n            else: \n                images, labels = next(data_loader_iter)\n            if progress_print: print(f\"{batch_idx} / {validation_batches_per_epoch}\", \"#\"*50)  # Timer\n            if verbose: print(f\"Timer - Load Dataloader batch (Validation) : {time.time() - start_time}\")  # Timer\n            with torch.no_grad():  # No gradient mode\n                start_time = time.time()\n                images = images.to(device)\n                labels = labels.to(device)\n                if verbose: print(f\"Timer - Move to {device} : {time.time() - start_time}\")\n                #print(f\"Images Device: {images.device}, Labels Device: {labels.device}\")  # Debug use\n                ## Model inference\n                if verbose: start_time =time.time()  # Timer\n                outputs = model(images)\n                max_value, max_idx = torch.max(outputs, dim=1)\n                prediction = max_idx\n                if verbose: print(f\"Timer - Model inference : {time.time() - start_time}\")  # Timer\n            ## Calculate metrics\n            loss = loss_function(outputs, labels)\n            ## Update variables\n            num_of_batches_validated = batch_idx + 1\n            ## Update trackers\n            if progress_print: print(\"Validation loss: \", loss.item())  # Debug use\n            track_validation_loss[epoch, batch_idx] = loss.item()\n            track_validation_TP_count[epoch, batch_idx] = ((prediction==1) & (labels==1)).sum().item()\n            track_validation_TN_count[epoch, batch_idx] = ((prediction==0) & (labels==0)).sum().item()\n            track_validation_FP_count[epoch, batch_idx] = ((prediction==1) & (labels==0)).sum().item()\n            track_validation_FN_count[epoch, batch_idx] = ((prediction==0) & (labels==1)).sum().item()\n        total_validation_loop_time = time.time() - total_validation_loop_start\n        print(f\"Timer - ENTIRE VALIDATION PORTION : {total_validation_loop_time}\")\n\n        \n         ## Collect all the items into dictionary to return\n        output = {\n            \"Training Loss\": track_training_loss, \n            \"Training TP\": track_training_TP_count, \n            \"Training FP\": track_training_FP_count, \n            \"Training TN\": track_training_TN_count,\n            \"Training FN\": track_training_FN_count,\n            \"Validation Loss\": track_validation_loss, \n            \"Validation TP\": track_validation_TP_count, \n            \"Validation FP\": track_validation_FP_count, \n            \"Validation TN\": track_validation_TN_count,\n            \"Validation FN\": track_validation_FN_count,\n            #\"loss\": 1, # Dummy loss\n        }\n        \n        ########## CHECKPOINT PORTION ##########\n        ## Create checkpoint data to be serialized\n        checkpoint_data = {\n            \"epoch\": epoch, \n            \"model_state_dict\": model.state_dict(),\n            \"optimizer_state_dict\": optimizer.state_dict(),\n        }\n        ## Define the location to save - Ray Tune changes directory during tuning, thus getcwd() won't be where the usual working dir is.\n        dir_path = Path(os.getcwd(),  config['trial_name'], \"model_checkpoint\")\n        print(f\"Checkpoint saved to {dir_path}.\")\n        os.makedirs(dir_path, exist_ok=True)    # Make sure the directory is created\n        ## Serialize and save the checkpoint data\n        checkpoint_path = Path(dir_path, \"checkpoint.pt\")\n        torch.save(checkpoint_data, checkpoint_path)  # Serialize the checkpoint data\n        \n        \n    ### ### ### ### ### IGNORE THIS - Still trying to figure out how to get this to work ### ### ### ### ###\n#         ## Create a Ray Tune session report\n#         ## Passes the checkpoint data to Ray Tune\n#         report = {\n#             \"loss\": np.mean(track_validation_loss[epoch, :]),\n#         }\n#         ## Read the serialized checkpoint data from storage - Seemingly redundant step allow us to revert back to lack checkpoint when code fails.\n#         checkpoint_from_storage = train.Checkpoint.from_directory(dir_path)  # Convert to Ray Tune checkpoint\n#         train.report(report, checkpoint = checkpoint_from_storage)\n    \n    dir_path = Path(os.getcwd(), config['trial_name'], \"output_pickle\")\n    os.makedirs(dir_path, exist_ok=True)    # Make sure the directory is created\n    with open(Path(dir_path, \"output.pkl\"), \"wb\") as f: \n        pickle.dump(pickle.dumps(output), f)\n   \n\n    plot_results(config, output['Training Loss'], output['Validation Loss'], Path(os.getcwd(), config['trial_name']))\n    \n    if True:  # Final report\n        print(f\"Timer - ENTIRE TRAINING PORTION : {total_training_loop_time}\")\n        print(f\"Timer - ENTIRE VALIDATION PORTION : {total_validation_loop_time}\")\n        print(f\"Checkpoint path: {checkpoint_path}\")\n        print(f\"Pickle path: {Path(os.getcwd(), 'output_pickle', 'output.pkl')}\")\n        \n        \n    return output","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:48.868460Z","iopub.execute_input":"2023-11-20T14:56:48.868751Z","iopub.status.idle":"2023-11-20T14:56:48.909744Z","shell.execute_reply.started":"2023-11-20T14:56:48.868726Z","shell.execute_reply":"2023-11-20T14:56:48.908854Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Hyperparmaeter Tuning\nThe follow calls to the main training/validation loop passes different configuration dictionaries to test the performance of each hyperparameter combinations. Specifically, it tested different learning rate and weight decay rates.\n\n## Thoughts on each hyperparameter (ADAM)\n\n### Learning rate: \n- Learning rate is would trade-off with training time and work in tandem with the momentum. A very small learning rate would need a larger momentum compared to a larger learning rate.\n\n### Weight decay: \n- Weight decay can be interpreted as a L2 regularization as it penalizes large model coefficients. \n- From interpretation, I prefer these values to be on the smaller end so that each filter is more unique from each other as opposed to having very similar parameters due to the penalization.\n\n### Beta 1 and Beta 2: \n- These parameters can be interpreted as the running average and sqaure of the gradient.\n- Defaults are 0.9 and 0.9, which provides a lot of momentum for the optimizer, thus best combined with a small learning rate to prevent overshooting the optimal parameter.","metadata":{}},{"cell_type":"code","source":"################################################################################\n## Hyperparameter Tuning: 1a\n################################################################################\n\nconfig = {\n    \"lr\": 0.003,\n    \"beta1\": 0.9,\n    \"beta2\": 0.9,\n    \"weight_decay\": 0,\n    \"batch_size\": 50,\n    \"dataloader_worker_count\": 10,\n    \"dataloader_prefetch_factor\": 5000,\n    \"trial_name\": \"1a\",\n}\n\noutput = train_the_model(config=config, n_epochs=10, verbose=False)\n\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T14:56:48.911754Z","iopub.execute_input":"2023-11-20T14:56:48.912154Z","iopub.status.idle":"2023-11-20T15:17:36.956392Z","shell.execute_reply.started":"2023-11-20T14:56:48.912119Z","shell.execute_reply":"2023-11-20T15:17:36.955268Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Hyperparameter Tuning: ab (weight decay = 2)\n################################################################################\n\nconfig = {\n    \"lr\": 0.003,\n    \"beta1\": 0.9,\n    \"beta2\": 0.9,\n    \"weight_decay\": 2,\n    \"batch_size\": 50,\n    \"dataloader_worker_count\": 10,\n    \"dataloader_prefetch_factor\": 5000,\n    \"trial_name\": \"1b\",\n}\n\noutput = train_the_model(config=config, n_epochs=10, verbose=False)\n\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T15:17:36.958218Z","iopub.execute_input":"2023-11-20T15:17:36.958962Z","iopub.status.idle":"2023-11-20T15:37:00.940185Z","shell.execute_reply.started":"2023-11-20T15:17:36.958932Z","shell.execute_reply":"2023-11-20T15:37:00.939034Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Hyperparameter Tuning: 1c (weight decay = 6)\n################################################################################\n\nconfig = {\n    \"lr\": 0.003,\n    \"beta1\": 0.9,\n    \"beta2\": 0.9,\n    \"weight_decay\": 6,\n    \"batch_size\": 50,\n    \"dataloader_worker_count\": 10,\n    \"dataloader_prefetch_factor\": 5000,\n    \"trial_name\": \"1c\",\n}\n\noutput = train_the_model(config=config, n_epochs=10, verbose=False)\n\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T15:37:00.941703Z","iopub.execute_input":"2023-11-20T15:37:00.941979Z","iopub.status.idle":"2023-11-20T15:56:27.932055Z","shell.execute_reply.started":"2023-11-20T15:37:00.941955Z","shell.execute_reply":"2023-11-20T15:56:27.930971Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Hyperparameter Tuning: 2a (learning rate = 0.03)\n################################################################################\n\nconfig = {\n    \"lr\": 0.03,\n    \"beta1\": 0.9,\n    \"beta2\": 0.9,\n    \"weight_decay\": 0,\n    \"batch_size\": 50,\n    \"dataloader_worker_count\": 10,\n    \"dataloader_prefetch_factor\": 5000,\n    \"trial_name\": \"2a\"\n}\n\noutput = train_the_model(config=config, n_epochs=10, verbose=False)\n\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T15:56:27.933813Z","iopub.execute_input":"2023-11-20T15:56:27.934716Z","iopub.status.idle":"2023-11-20T16:01:14.690639Z","shell.execute_reply.started":"2023-11-20T15:56:27.934677Z","shell.execute_reply":"2023-11-20T16:01:14.688930Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"################################################################################\n## Hyperparameter Tuning: 2b (learning rate = 0.3)\n################################################################################\n\nconfig = {\n    \"lr\": 0.3,\n    \"beta1\": 0.9,\n    \"beta2\": 0.9,\n    \"weight_decay\": 0,\n    \"batch_size\": 50,\n    \"dataloader_worker_count\": 10,\n    \"dataloader_prefetch_factor\": 5000,\n    \"trial_name\": \"2b\"\n}\n\noutput = train_the_model(config=config, n_epochs=10, verbose=False)\n\n","metadata":{"execution":{"iopub.status.busy":"2023-11-20T16:01:14.691646Z","iopub.status.idle":"2023-11-20T16:01:14.692018Z","shell.execute_reply.started":"2023-11-20T16:01:14.691823Z","shell.execute_reply":"2023-11-20T16:01:14.691838Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Inferencing the Kaggle Competition Data\n","metadata":{}},{"cell_type":"markdown","source":"The following plot was generated via prior training. The training and validation loss between each of the models in the hyperparameter tuning section differ drastically (in magtitudes), thus, I am confidence to say that out of all the models used for tuning, model 1a is the most optimal solution.\n\nThus, I load the trained model parameters and inferenced the the competition test set for competition submission\n\n```python\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\n\nimg = mpimg.imread(\"/kaggle/working/previous_kernel_output/1a.png\")\nplt.axis(\"off\")\nplt.imshow(img)\n```\n\n![1a.png](attachment:eabb6a18-de07-4b30-b63c-1469552452a6.png)","metadata":{},"attachments":{"eabb6a18-de07-4b30-b63c-1469552452a6.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAACSkUlEQVR4nOzdd3xT9foH8M9J2iZtk6YzHVDa0kKhFMpGQAGVjQjuAQqKG1C86s91BVwXvXqvXhd61QsuBLcoG0VAhuzZUlZpge6VpCttkvP747Rp072TNJ/365VXT05OTp42afLkO56vIIqiCCIiIiJyGTJ7B0BEREREnYsJIBEREZGLYQJIRERE5GKYABIRERG5GCaARERERC6GCSARERGRi2ECSERERORimAASERERuRgmgEREREQuhgkgERERkYthAkhERETkYpgAEhEREbkYJoBERERELoYJIBEREZGLYQJIRERE5GKYABIRERG5GCaARERERC6GCSARERGRi2ECSERERORimAASERERuRgmgEREREQuhgkgERERkYthAkhERETkYpgAEhEREbkYJoBERERELoYJIBEREZGLYQJIRERE5GKYABIRERG5GCaARERERC6GCSARERGRi2ECSERERORimAASERERuRgmgEREREQuhgkgERERkYthAkjUDubOnYvIyMhW3Xfp0qUQBKF9A7IjQRCwdOlS6/WVK1dCEARcuHChyftGRkZi7ty57RpPW54bspWVlYWbb74ZAQEBEAQBb7/9drs/RkteL7XxuSZqPiaA1KUJgtCsyx9//GHvUDvNu+++C41Gg4cffhiCIODs2bMNHvv8889DEAQcO3asEyNsufT0dCxduhRHjhyxdyhWFy5cgCAIePPNN+0dSrt5/PHHsWnTJjz77LP44osvMHny5DrHzJ07t1n/c+2d6LeHNWvWYPbs2ejVqxcEQcC4cePsHRJRh3GzdwBEHemLL76wuf75559jy5Ytdfb37du3TY/z8ccfw2KxtOq+f//73/HMM8+06fFbYt26dZg4cSLmzp2LDz/8EKtWrcLixYvrPfbrr79G//79MWDAgFY/3l133YXbb78dCoWi1edoSnp6Ol588UVERkZi4MCBNre15bkhW7///jtmzJiBJ598ssFjHnzwQYwfP956PSUlBYsXL8YDDzyAq666yro/Ojq63vt3xuulIcuXL8fBgwcxbNgw5OXldfrjE3UmJoDUpc2ePdvm+t69e7Fly5Y6+2srKSmBl5dXsx/H3d29VfEBgJubG9zcOudfsaSkBNu3b8fy5csxYsQIxMTE4Ouvv643AdyzZw9SUlLw2muvtekx5XI55HJ5m87RFm15bshWdnY2fH19Gz1m5MiRGDlypPX6gQMHsHjxYowcObLR/7vi4mJ4e3vb9fXyxRdfoFu3bpDJZIiPj7dLDESdhV3A5PLGjRuH+Ph4HDx4EGPGjIGXlxeee+45AMDPP/+MadOmISwsDAqFAtHR0Xj55ZdhNpttzlF77FHN7r///ve/iI6OhkKhwLBhw7B//36b+9Y3BlAQBCxYsAA//fQT4uPjoVAo0K9fP2zcuLFO/H/88QeGDh0KpVKJ6OhofPTRRw2OK/ztt99gNBoxZcoUAMCsWbNw6tQpHDp0qM6xq1atgiAIuOOOO1BeXo7FixdjyJAh0Gg08Pb2xlVXXYVt27Y1+fetb0yXKIp45ZVX0L17d3h5eeHqq6/GyZMn69w3Pz8fTz75JPr37w+VSgUfHx9MmTIFR48etfn9hw0bBgC45557rF2MK1euBFD/uLDi4mI88cQTCA8Ph0KhQGxsLN58802IomhzXEueh9bKzs7GvHnzEBwcDKVSiYSEBHz22Wd1jlu9ejWGDBkCtVoNHx8f9O/fH//5z3+st1dUVODFF19Er169oFQqERAQgCuvvBJbtmxpMobz58/jlltugb+/P7y8vHDFFVdg3bp11turnkNRFPH+++9b/8atVXW+7du345FHHoFWq0X37t1tbqv5emnu/2F9MjIycOrUKVRUVDR5bHh4OGSypj8WU1NT8cgjjyA2Nhaenp4ICAjALbfc0qpxi0T2whZAIgB5eXmYMmUKbr/9dsyePRvBwcEApA8jlUqFv/3tb1CpVPj999+xePFi6PV6vPHGG02ed9WqVTAYDHjwwQchCAL++c9/4sYbb8T58+ebbJn6888/8cMPP+CRRx6BWq3GO++8g5tuuglpaWkICAgAABw+fBiTJ09GaGgoXnzxRZjNZrz00ksICgqq95zr16/HkCFDrL/frFmz8OKLL2LVqlUYPHiw9Tiz2YxvvvkGV111FXr06IHc3Fx88sknuOOOO3D//ffDYDDg008/xaRJk7Bv37463a5NWbx4MV555RVMnToVU6dOxaFDhzBx4kSUl5fbHHf+/Hn89NNPuOWWWxAVFYWsrCx89NFHGDt2LBITExEWFoa+ffvipZdeqtPNOGrUqHofWxRFXH/99di2bRvmzZuHgQMHYtOmTXjqqadw+fJlvPXWWy1+HlqrtLQU48aNw9mzZ7FgwQJERUXh22+/xdy5c1FYWIjHHnsMALBlyxbccccduPbaa/H6668DAJKSkrBr1y7rMUuXLsWyZctw3333Yfjw4dDr9Thw4AAOHTqECRMmNBhDVlYWRo0ahZKSEjz66KMICAjAZ599huuvvx7fffcdbrjhBowZMwZffPEF7rrrLkyYMAF33313m37vKo888giCgoKwePFiFBcXN3hcW/4Pn332WXz22WdISUlptwki+/fvx+7du3H77beje/fuuHDhApYvX45x48YhMTGxRb0HRHYjErmQ+fPni7Vf9mPHjhUBiB9++GGd40tKSurse/DBB0UvLy+xrKzMum/OnDliRESE9XpKSooIQAwICBDz8/Ot+3/++WcRgPjLL79Y9y1ZsqROTABEDw8P8ezZs9Z9R48eFQGI7777rnXf9OnTRS8vL/Hy5cvWfWfOnBHd3NzqnFMURbFHjx7ikiVLbPYNGzZM7N69u2g2m637Nm7cKAIQP/roI1EURdFkMolGo9HmfgUFBWJwcLB477331om95mOsWLFCBCCmpKSIoiiK2dnZooeHhzht2jTRYrFYj3vuuedEAOKcOXOs+8rKymziEkXpb6tQKMSXXnrJum///v0iAHHFihV1fufaz81PP/0kAhBfeeUVm+NuvvlmURAEm795c5+H+lS9Bt54440Gj3n77bdFAOKXX35p3VdeXi6OHDlSVKlUol6vF0VRFB977DHRx8dHNJlMDZ4rISFBnDZtWqMx1WfRokUiAHHnzp3WfQaDQYyKihIjIyNt/v4AxPnz57fo/PU9N1WviSuvvLLO71T79SKKrf8/rNpX+3zN0a9fP3Hs2LH13lZfPHv27BEBiJ9//nmLHofIXtgFTARAoVDgnnvuqbPf09PTum0wGJCbm4urrroKJSUlOHXqVJPnve222+Dn52e9XtU6df78+SbvO378eJuB8gMGDICPj4/1vmazGVu3bsXMmTMRFhZmPS4mJsbaxVvTiRMnkJaWhmnTptnsnz17Ni5duoQdO3ZY961atQoeHh645ZZbAEjj+Dw8PAAAFosF+fn5MJlMGDp0aL3dx43ZunUrysvLsXDhQptuxEWLFtU5VqFQWLvkzGYz8vLyoFKpEBsb2+LHrbJ+/XrI5XI8+uijNvufeOIJiKKIDRs22Oxv6nloi/Xr1yMkJAR33HGHdZ+7uzseffRRFBUVYfv27QAAX19fFBcXN9qd6+vri5MnT+LMmTMtjmH48OG48sorrftUKhUeeOABXLhwAYmJiS38rZrv/vvvb9Z4v7b8H65cuRKiKLZreZia8VRUVCAvLw8xMTHw9fVt9euSqLMxASQC0K1bN2uCU9PJkydxww03QKPRwMfHB0FBQdaB7Dqdrsnz9ujRw+Z6VTJYUFDQ4vtW3b/qvtnZ2SgtLUVMTEyd4+rbt27dOgQHB2Po0KE2+2+//XbI5XKsWrUKAFBWVoYff/wRU6ZMsUleP/vsMwwYMMA6viwoKAjr1q1r1t+hptTUVABAr169bPYHBQXZPB4gJZtvvfUWevXqBYVCgcDAQAQFBeHYsWMtftyajx8WFga1Wm2zv2omeFV8VZp6HtoiNTUVvXr1qjPurHYsjzzyCHr37o0pU6age/fuuPfee+uMQ3zppZdQWFiI3r17o3///njqqaeaVb4nNTUVsbGxdfY39PdoT1FRUc06rq3/h+2ttLQUixcvto4hrXpdFhYW2iUeotZgAkgE22/0VQoLCzF27FgcPXoUL730En755Rds2bLFOgarOaVFGmrdEGtNNmjv+9Zn/fr1mDx5cp3B+1qtFhMmTMD333+PiooK/PLLLzAYDJg1a5b1mC+//BJz585FdHQ0Pv30U2zcuBFbtmzBNddc06ElVv7xj3/gb3/7G8aMGYMvv/wSmzZtwpYtW9CvX79OK+3S3s9Da2i1Whw5cgRr1661jl+cMmUK5syZYz1mzJgxOHfuHP73v/8hPj4en3zyCQYPHoxPPvmk0+Jsqfr+72prj//D9rZw4UK8+uqruPXWW/HNN99g8+bN2LJlCwICAlhyiJwGJ4EQNeCPP/5AXl4efvjhB4wZM8a6PyUlxY5RVdNqtVAqlfUWcq69r7CwELt378aCBQvqPdesWbOwceNGbNiwAatWrYKPjw+mT59uvf27775Dz5498cMPP9gkkEuWLGlx3BEREQCAM2fOoGfPntb9OTk5dVrVvvvuO1x99dX49NNP6/w+gYGB1ustmZEaERGBrVu3wmAw2LQCVnUlVsXXGSIiInDs2DFYLBabVsD6YvHw8MD06dMxffp0WCwWPPLII/joo4/wwgsvWFt8/f39cc899+Cee+5BUVERxowZg6VLl+K+++5rNIbk5OQ6++3x96iPI/4ffvfdd5gzZw7+9a9/WfeVlZWhsLDQbjERtRRbAIkaUNXyU7Olp7y8HB988IG9QrIhl8sxfvx4/PTTT0hPT7fuP3v2bJ1xbJs3bwYATJw4sd5zzZw5E15eXvjggw+wYcMG3HjjjVAqlTaPBdj+Lf766y/s2bOnxXGPHz8e7u7uePfdd23OV9+yYnK5vE5L27fffovLly/b7PP29gaAZn0AT506FWazGe+9957N/rfeeguCINQ7frKjTJ06FZmZmVizZo11n8lkwrvvvguVSoWxY8cCQJ2ixDKZzFqc22g01nuMSqVCTEyM9fbGYti3b5/Nc1lcXIz//ve/iIyMRFxcXOt/wXbQ1v/DlpSBaUlMtV+X7777brPK0hA5CrYAEjVg1KhR8PPzw5w5c/Doo49CEAR88cUXndr115SlS5di8+bNGD16NB5++GFrYhMfH2+zLNq6detw5ZVXQqPR1HselUqFmTNnWscB1uz+BYDrrrsOP/zwA2644QZMmzYNKSkp+PDDDxEXF4eioqIWxRwUFIQnn3wSy5Ytw3XXXYepU6fi8OHD2LBhg02rXtXjvvTSS7jnnnswatQoHD9+HF999ZVNyyEgrSrh6+uLDz/8EGq1Gt7e3hgxYkS9Y8ymT5+Oq6++Gs8//zwuXLiAhIQEbN68GT///DMWLVrU4AoVrfXbb7+hrKyszv6ZM2figQcewEcffYS5c+fi4MGDiIyMxHfffYddu3bh7bfftrZQ3nfffcjPz8c111yD7t27IzU1Fe+++y4GDhxoHasXFxeHcePGYciQIfD398eBAwfw3XffNdjqW+WZZ57B119/jSlTpuDRRx+Fv7+/tWzK999/36y6eB2prf+HLSkDs2PHDutkqJycHBQXF+OVV14BIHWxV7VAXnfddfjiiy+g0WgQFxeHPXv2YOvWrW0uC0TUmZgAEjUgICAAv/76K5544gn8/e9/h5+fH2bPno1rr70WkyZNsnd4AIAhQ4Zgw4YNePLJJ/HCCy8gPDwcL730EpKSkqxdeKIoYuPGjY0u3wVISd+qVasQGhqKa665xua2uXPnIjMzEx999BE2bdqEuLg4fPnll/j2229btY7yK6+8AqVSiQ8//BDbtm3DiBEjsHnz5jozlJ977jkUFxdj1apVWLNmDQYPHox169bVWTrP3d0dn332GZ599lk89NBDMJlMWLFiRb0JoEwmw9q1a7F48WKsWbMGK1asQGRkJN544w088cQTLf5dmrJx48Z6C0dHRkYiPj4ef/zxB5555hl89tln0Ov1iI2NxYoVK2zWyp09ezb++9//4oMPPkBhYSFCQkJw2223YenSpdYE7dFHH8XatWuxefNmGI1GRERE4JVXXsFTTz3VaHzBwcHYvXs3nn76abz77rsoKyvDgAED8Msvv9R5PuyhM/8Pf//9d7z44os2+1544QUA0nCHqgTwP//5D+RyOb766iuUlZVh9OjR2Lp1q8O8LxA1hyA6UnMGEbWLmTNnWkuC7Nu3DyNGjMDJkyft3p1HRESOgWMAiZxcaWmpzfUzZ85g/fr1GDdunHXfP/7xDyZ/RERkxRZAIicXGhqKuXPnomfPnkhNTcXy5cthNBpx+PDhOrX2iIiIAI4BJHJ6kydPxtdff43MzEwoFAqMHDkS//jHP5j8ERFRg9gCSERERORiOAaQiIiIyMUwASQiIiJyMUwAiYiIiFwMJ4G0gcViQXp6OtRqdYvWIiUiIiL7EUURBoMBYWFhdl/txl6YALZBeno6wsPD7R0GERERtcLFixfRvXt3e4dhF0wA26Bqnc6LFy/Cx8fHztEQERFRc+j1eoSHh1s/x10RE8A2qOr29fHxYQJIRETkZFx5+JZrdnwTERERuTAmgEREREQuhgkgERERkYvhGEAiciqiKMJkMsFsNts7FCJyUHK5HG5ubi49xq8pTACJyGmUl5cjIyMDJSUl9g6FiBycl5cXQkND4eHhYe9QHBITQCJyChaLBSkpKZDL5QgLC4OHhwe/3RNRHaIoory8HDk5OUhJSUGvXr1ctthzY5gAEpFTKC8vh8ViQXh4OLy8vOwdDhE5ME9PT7i7uyM1NRXl5eVQKpX2DsnhMCUmIqfCb/JE1Bx8r2gcWwAdkcUMpO4GirIAVTAQMQqQye0dFREREXURTI8dTeJa4O144LPrgO/nST/fjpf2ExFVioyMxNtvv93s4//44w8IgoDCwsIOi8kR5eXlQavV4sKFC/YOxeVcuHABgiDgyJEjDR6zceNGDBw4EBaLpfMCIwBMAB1L4lrgm7sBfbrtfn2GtJ9JIJHTEQSh0cvSpUtbdd79+/fjgQceaPbxo0aNQkZGBjQaTaser7kcLdF89dVXMWPGDERGRto7FKrH5MmT4e7ujq+++sreobgcdgE7CosZ2Pg0ALGeG0UAArDxGaDPNHYHEzmRjIwM6/aaNWuwePFiJCcnW/epVCrrtiiKMJvNcHNr+q05KCioRXF4eHggJCSkRfdxdiUlJfj000+xadMme4fSYcrLy52+zMncuXPxzjvv4K677rJ3KC6FLYCOInV33ZY/GyKgvywdR0ROIyQkxHrRaDQQBMF6/dSpU1Cr1diwYQOGDBkChUKBP//8E+fOncOMGTMQHBwMlUqFYcOGYevWrTbnrd0FLAgCPvnkE9xwww3w8vJCr169sHZtda9B7Za5lStXwtfXF5s2bULfvn2hUqkwefJkm4TVZDLh0Ucfha+vLwICAvD0009jzpw5mDlzZqv/HgUFBbj77rvh5+cHLy8vTJkyBWfOnLHenpqaiunTp8PPzw/e3t7o168f1q9fb73vrFmzEBQUBE9PT/Tq1QsrVqxo8LHWr18PhUKBK664wrrPbDZj3rx5iIqKgqenJ2JjY/Gf//zHevvmzZuhVCrrtGA+9thjuOaaa6zXP/74Y+uM9BtuuAH//ve/4evr22AsVd2hq1evxqhRo6BUKhEfH4/t27fbHHfixAlMmTIFKpUKwcHBuOuuu5Cbm2u9fdy4cViwYAEWLVqEwMBATJo0qcHH/OSTT9C3b18olUr06dMHH3zwQYvj2b59O4YPHw6FQoHQ0FA888wzMJlM1tstFgv++c9/IiYmBgqFAj169MCrr75qc47z58/j6quvhpeXFxISErBnzx6b26dPn44DBw7g3LlzDf4u1P6YADqKoqz2PY6InMYzzzyD1157DUlJSRgwYACKioowdepU/Pbbbzh8+DAmT56M6dOnIy0trdHzvPjii7j11ltx7NgxTJ06FbNmzUJ+fn6Dx5eUlODNN9/EF198gR07diAtLQ1PPvmk9fbXX38dX331FVasWIFdu3ZBr9fjp59+atPvOnfuXBw4cABr167Fnj17IIoipk6dioqKCgDA/PnzYTQasWPHDhw/fhyvv/66tZX0hRdeQGJiIjZs2ICkpCQsX74cgYGBDT7Wzp07MWTIEJt9FosF3bt3x7fffovExEQsXrwYzz33HL755hsAwLXXXgtfX198//331vuYzWasWbMGs2bNAgDs2rULDz30EB577DEcOXIEEyZMqJP0NOSpp57CE088gcOHD2PkyJGYPn068vLyAACFhYW45pprMGjQIBw4cAAbN25EVlYWbr31VptzfPbZZ/Dw8MCuXbvw4Ycf1vs4X331FRYvXoxXX30VSUlJ+Mc//oEXXngBn332WbPjuXz5MqZOnYphw4bh6NGjWL58OT799FO88sor1vs/++yzeO2116zPzapVqxAcHGzzGM8//zyefPJJHDlyBL1798Ydd9xhk0T26NEDwcHB2LlzZ7P+htRORGo1nU4nAhB1Ol3bT3Z+hygu8Wn6cn5H2x+LyAmVlpaKiYmJYmlpaZ3bDlzIFz/eca7Jy0+HL9W570+HLzXrvgcu5Lf5d1ixYoWo0Wis17dt2yYCEH/66acm79uvXz/x3XfftV6PiIgQ33rrLet1AOLf//536/WioiIRgLhhwwabxyooKLDGAkA8e/as9T7vv/++GBwcbL0eHBwsvvHGG9brJpNJ7NGjhzhjxowG46z9ODWdPn1aBCDu2rXLui83N1f09PQUv/nmG1EURbF///7i0qVL6z339OnTxXvuuafBx65txowZ4r333tvkcfPnzxdvuukm6/XHHntMvOaaa6zXN23aJCoUCuvvdNttt4nTpk2zOcesWbNsntvaUlJSRADia6+9Zt1XUVEhdu/eXXz99ddFURTFl19+WZw4caLN/S5evCgCEJOTk0VRFMWxY8eKgwYNavJ3io6OFletWmWz7+WXXxZHjhzZ7Hiee+45MTY2VrRYLNZj3n//fVGlUolms1nU6/WiQqEQP/7440Z/508++cS67+TJkyIAMSkpyebYQYMGNfi8t1Zj7xnt+vntpDgG0FFEjAJ8wqQJH/WOAxSk2yNGdXZkRA6v3GSBoczU5HEqRd23vNJyc7PuW27quFmKQ4cOtbleVFSEpUuXYt26dcjIyIDJZEJpaWmTLYADBgywbnt7e8PHxwfZ2dkNHu/l5YXo6Gjr9dDQUOvxOp0OWVlZGD58uPV2uVyOIUOGtHrGZlJSEtzc3DBixAjrvoCAAMTGxiIpKQkA8Oijj+Lhhx/G5s2bMX78eNx0003W3+vhhx/GTTfdhEOHDmHixImYOXMmRo1q+D2xtLS03gLA77//Pv73v/8hLS0NpaWlKC8vx8CBA623z5o1C1dccQXS09MRFhaGr776CtOmTbN28SYnJ+OGG26wOefw4cPx66+/Nvk3GDlypHXbzc0NQ4cOtf7uR48exbZt22zGhVY5d+4cevfuDQB1WjVrKy4uxrlz5zBv3jzcf//91v0mk6nOJKDG4klKSsLIkSNtVtwZPXo0ioqKcOnSJWRmZsJoNOLaa69tNJ6ar8vQ0FAAQHZ2Nvr06WPd7+npySUeOxm7gB2FTA5Mfr2BGyv/+Sa/xgkgRPXwcJNBrXRr8uLpUff/x9ND3qz7erh13Nult7e3zfUnn3wSP/74I/7xj39g586dOHLkCPr374/y8vJGz+Pu7m5zXRCERpO1+o4Xxfq+gHae++67D+fPn8ddd92F48ePY+jQoXj33XcBAFOmTEFqaioef/xxpKen49prr7Xpsq4tMDAQBQUFNvtWr16NJ598EvPmzcPmzZtx5MgR3HPPPTZ/22HDhiE6OhqrV69GaWkpfvzxR2v3b0cqKirC9OnTceTIEZvLmTNnMGbMGOtxtV8v9Z0HkMYp1jzPiRMnsHfv3naL19PTs1nH1XydVSWTtV+X+fn5LZ7YRG3DFkBHEnc9cOvnwIb/AwzVA7HhEyYlf3HX2y82Igc2JMIPQyL8WnXfGQO7tXM0bbdr1y7MnTvX2spUVFTU6XXsNBoNgoODsX//fmvyYTabcejQIZvWspbo27cvTCYT/vrrL2vLXV5eHpKTkxEXF2c9Ljw8HA899BAeeughPPvss/j444+xcOFCANLs5zlz5mDOnDm46qqr8NRTT+HNN9+s9/EGDRqEL7/80mbfrl27MGrUKDzyyCPWffVNPpg1axa++uordO/eHTKZDNOmTbPeFhsbi/3799scX/t6Q/bu3Wv9e5pMJhw8eBALFiwAAAwePBjff/89IiMjmzUTvCHBwcEICwvD+fPnm0xcG4unb9+++P777yGKojVx27VrF9RqNbp37w6tVgtPT0/89ttvuO+++1odb1lZGc6dO4dBgwa1+hzUckwAHU3c9VKpl81/B3SXAA8VcN2/AffmfdMiIufXq1cv/PDDD5g+fToEQcALL7xgl0K5CxcuxLJlyxATE4M+ffrg3XffRUFBgU2XYEOOHz8OtVptvS4IAhISEjBjxgzcf//9+Oijj6BWq/HMM8+gW7dumDFjBgBg0aJFmDJlCnr37o2CggJs27YNffv2BQAsXrwYQ4YMQb9+/WA0GvHrr79ab6vPpEmT8Oyzz6KgoAB+ftIXhF69euHzzz/Hpk2bEBUVhS+++AL79+9HVFSUzX1nzZqFpUuX4tVXX8XNN98MhUJh83cZM2YM/v3vf2P69On4/fffsWHDhmb9Xd5//3306tULffv2xVtvvYWCggLce++9AKQJMB9//DHuuOMO/N///R/8/f1x9uxZrF69Gp988gnk8ub3AL344ot49NFHodFoMHnyZBiNRhw4cAAFBQX429/+1qx4HnnkEbz99ttYuHAhFixYgOTkZCxZsgR/+9vfIJPJoFQq8fTTT+P//u//4OHhgdGjRyMnJwcnT57EvHnzmh3r3r17oVAobLqjqeOxC9gRyeRA1BhAGwf49gCKc+wdERF1on//+9/w8/PDqFGjMH36dEyaNAmDBw/u9Diefvpp3HHHHbj77rsxcuRIqFQqTJo0qd5xdbWNGTMGgwYNsl6qxq2tWLECQ4YMwXXXXYeRI0dCFEWsX7/e2k1oNpsxf/589O3bF5MnT0bv3r2t5Us8PDzw7LPPYsCAARgzZgzkcjlWr17dYAz9+/fH4MGDrTN8AeDBBx/EjTfeiNtuuw0jRoxAXl6eTWtglZiYGAwfPhzHjh2r04o2evRofPjhh/j3v/+NhIQEbNy4EY8//niz/i6vvfYaXnvtNSQkJODPP//E2rVrrTOZw8LCsGvXLpjNZkycOBH9+/fHokWL4Ovr2+J1be+77z588sknWLFiBfr374+xY8di5cqVdRLdxuLp1q0b1q9fj3379iEhIQEPPfQQ5s2bh7///e/W+7/wwgt44oknsHjxYvTt2xe33XZbo+NO6/P1119j1qxZ8PLyatH9qG0E0d4DPpyYXq+HRqOBTqeDj49P+548/QiQvEHajr4G6DGi0cOJurqysjKkpKQgKiqqWR+01P4sFgv69u2LW2+9FS+//LK9w2mWdevW4amnnsKJEydanES1xP33349Tp041WMrkwoULiIqKwuHDh1vdhd6eHCWe3NxcxMbG4sCBA3WS07Zq7D2jQz+/nQS7gB2VOrR6u+Z4QCKiTpKamorNmzdj7NixMBqNeO+995CSkoI777zT3qE127Rp03DmzBlcvnwZ4eHh7XbeN998ExMmTIC3tzc2bNiAzz77zKbQMjXPhQsX8MEHH7R78kdNYwLoqLyDALkbYDYxASQiu5DJZFi5ciWefPJJiKKI+Ph4bN26tdFxd45o0aJF7X7Offv24Z///CcMBgN69uyJd955p00TIVzV0KFD65RBos7BBNBRyWRAtyGA3ENqDRRFoBkDjImI2kt4eDh27dpl7zAcUs1xhc0RGRlp9xI7NTlaPNT5mAA6suhrmj6GiIiIqIU4C5iIiIjIxTABJCIiInIxTAAdncUCFGUDGcekcYBEREREbcQxgI7u5A9A7hlp27cH4Olr13CIiIjI+bEF0NGptNXbhkz7xUFERERdBhNAR2dTEDrdfnEQkV2NGzfOpp5dZGQk3n777UbvIwgCfvrppzY/dnudx5kkJycjJCQEBoPB3qG4nD/++AOCIKCwsLDBYz788ENMnz6984LqgrpsArhs2TIMGzYMarUaWq0WM2fORHJycr3HiqKIKVOmOOabXM0EUM+C0ETOZvr06Zg8eXK9t+3cuROCIODYsWMtPu/+/fvxwAMPtDU8G0uXLq13WbCMjAxMmTKlXR+rtpUrV8LX17dDH6Mlnn32WSxcuBBqtdreoVA97r33Xhw6dKjBpfeoaV02Ady+fTvmz5+PvXv3YsuWLaioqMDEiRNRXFxc59i3334bgqMWWVaoAGXlOoWGDGlSCBE5jXnz5mHLli24dOlSndtWrFiBoUOHYsCAAS0+b1BQELy8vNojxCaFhIRAoVB0ymM5grS0NPz666+YO3euvUPpMOXl5fYOoU08PDxw55134p133rF3KE6ryyaAGzduxNy5c9GvXz8kJCRg5cqVSEtLw8GDB22OO3LkCP71r3/hf//7n50ibYaqVkBzBVCSZ99YiKhFrrvuOgQFBWHlypU2+4uKivDtt99i3rx5yMvLwx133IFu3brBy8sL/fv3x9dff93oeWt3AZ85cwZjxoyBUqlEXFwctmzZUuc+Tz/9NHr37g0vLy/07NkTL7zwAioqKgBILXAvvvgijh49CkEQIAiCNebavSPHjx/HNddcA09PTwQEBOCBBx5AUVGR9fa5c+di5syZePPNNxEaGoqAgADMnz/f+litkZaWhhkzZkClUsHHxwe33norsrKyrLcfPXoUV199NdRqNXx8fDBkyBAcOHAAgLSm8fTp0+Hn5wdvb2/069cP69evb/CxvvnmGyQkJKBbt27WfU09R//9738RFhYGS60v6TNmzMC9995rvf7KK69Aq9VCrVbjvvvuwzPPPFNvq2uVqu7QdevWYcCAAVAqlbjiiitw4sQJm+P+/PNPXHXVVfD09ER4eDgeffRRmwaPyMhIvPzyy7j77rvh4+PTYOuxxWLBsmXLEBUVBU9PTyQkJOC7775rcTzff/89+vXrB4VCgcjISPzrX/+yud1oNOLpp59GeHg4FAoFYmJi8Omnn9occ/DgQQwdOhReXl4YNWpUnV686dOnY+3atSgtLW3w70cN67IJYG06nQ4A4O/vb91XUlKCO++8E++//z5CQkLsFVrTbMYBshuYyJm4ubnh7rvvxsqVK22W3vr2229hNptxxx13oKysDEOGDMG6detw4sQJPPDAA7jrrruwb9++Zj2GxWLBjTfeCA8PD/z111/48MMP8fTTT9c5Tq1WY+XKlUhMTMR//vMffPzxx3jrrbcAALfddhueeOIJ9OvXDxkZGcjIyMBtt91W5xzFxcWYNGkS/Pz8sH//fnz77bfYunUrFixYYHPctm3bcO7cOWzbtg2fffYZVq5cWScJbi6LxYIZM2YgPz8f27dvx5YtW3D+/Hmb+GbNmoXu3btj//79OHjwIJ555hm4u7sDAObPnw+j0YgdO3bg+PHjeP3116FSqRp8vJ07d9ZZn7ap5+iWW25BXl4etm3bZr1Pfn4+Nm7ciFmzZgEAvvrqK7z66qt4/fXXcfDgQfTo0QPLly9v1t/gqaeewr/+9S/s378fQUFBmD59ujWhPnfuHCZPnoybbroJx44dw5o1a/Dnn3/WeU7efPNNJCQk4PDhw3jhhRfqfZxly5bh888/x4cffoiTJ0/i8ccfx+zZs7F9+/Zmx3Pw4EHceuutuP3223H8+HEsXboUL7zwgs3zf/fdd+Prr7/GO++8g6SkJHz00Ud1npPnn38e//rXv3DgwAG4ubnZJNKAtI6wyWTCX3/91ay/IdUiugCz2SxOmzZNHD16tM3+Bx54QJw3b571OgDxxx9/bPA8ZWVlok6ns14uXrwoAhB1Ol1HhS7JTxHF3/8hXZI3duxjETmo0tJSMTExUSwtLa17Y9pforjr3aYvx76te99j3zbvvml/tTr2pKQkEYC4bds2676rrrpKnD17doP3mTZtmvjEE09Yr48dO1Z87LHHrNcjIiLEt956SxRFUdy0aZPo5uYmXr582Xr7hg0bmnxPe+ONN8QhQ4ZYry9ZskRMSEioc1zN8/z3v/8V/fz8xKKiIuvt69atE2UymZiZmSmKoijOmTNHjIiIEE0mk/WYW265RbztttsajGXFihWiRqOp97bNmzeLcrlcTEtLs+47efKkCEDct2+fKIqiqFarxZUrV9Z7//79+4tLly5t8LFrS0hIEF966aUmj6v9HM2YMUO89957rdc/+ugjMSwsTDSbzaIoiuKIESPE+fPn25xj9OjR9f7Nq2zbtk0EIK5evdq6Ly8vT/T09BTXrFkjiqIozps3T3zggQds7rdz505RJpNZ/18iIiLEmTNnNvr7lJWViV5eXuLu3btt9s+bN0+84447mh3PnXfeKU6YMMHmHE899ZQYFxcniqIoJicniwDELVu2NPo7b9261bpv3bp1IoA6//9+fn4NPu+NvWfodLrO+fx2YC7RAjh//nycOHECq1evtu5bu3Ytfv/99yZn0dW0bNkyaDQa6yU8PLwDoq2HOhSoGqOo50xgojpMRsBoaPpSXncMMMqLm3dfk7HV4fXp0wejRo2yDjU5e/Ysdu7ciXnz5gEAzGYzXn75ZfTv3x/+/v5QqVTYtGkT0tLSmnX+pKQkhIeHIywszLpv5MiRdY5bs2YNRo8ejZCQEKhUKvz9739v9mPUfKyEhAR4e3tb940ePRoWi8Wmi65fv36Qy+XW66GhocjOzm7RY9V8zPDwcJv33Li4OPj6+iIpKQkA8Le//Q333Xcfxo8fj9deew3nzp2zHvvoo4/ilVdewejRo7FkyZImJ92UlpZCqVTa7GvOczRr1ix8//33MBql18pXX32F22+/HTKZ9FGbnJyM4cOH25y39vWG1Hw+/f39ERsba/3djx49ipUrV0KlUlkvkyZNgsViQUpKivV+tVs1azt79ixKSkowYcIEm3N9/vnnNn/PpuJJSkrC6NGjbY4fPXo0zpw5A7PZjCNHjkAul2Ps2LGNxlNzbGxoqNQTVvs15OnpiZKSkkbPQ/Xr8gngggUL8Ouvv2Lbtm3o3r27df/vv/+Oc+fOwdfXF25ubnBzk2pi33TTTRg3bly953r22Weh0+msl4sXL3bGrwC4KQCvAMDDG1BquCIIUW1uCkChbvri4V33vh7ezbuvW9smQcybNw/ff/89DAYDVqxYgejoaOsH4BtvvIH//Oc/ePrpp7Ft2zYcOXIEkyZNateB+nv27MGsWbMwdepU/Prrrzh8+DCef/75DpsMUNX9WkUQhDrj49rT0qVLcfLkSUybNg2///474uLi8OOPPwIA7rvvPpw/fx533XUXjh8/jqFDh+Ldd99t8FyBgYEoKCiw2dec52j69OkQRRHr1q3DxYsXsXPnTmv3b0cqKirCgw8+iCNHjlgvR48exZkzZxAdHW09rmbS3tB5AGDdunU250pMTLQZB9hWnp6ezTqu5muoaqJm7ddQfn4+goKC2i02V9JlVwIRRRELFy7Ejz/+iD/++ANRUVE2tz/zzDO47777bPb1798fb731VoO1hRQKhf1mwg26S/oActTZykT2FD5curRG/5vbN5YG3HrrrXjsscewatUqfP7553j44YetH2q7du3CjBkzMHv2bADSh9zp06cRFxfXrHP37dsXFy9eREZGhrWlZO/evTbH7N69GxEREXj++eet+1JTU22O8fDwgNlsbvKxVq5cieLiYmtCsWvXLshkMsTGxjYr3paq+v0uXrxobQVMTExEYWGhzd+od+/e6N27Nx5//HHccccdWLFiBW644QYAQHh4OB566CE89NBDePbZZ/Hxxx9j4cKF9T7eoEGDkJiYaLOvOc+RUqnEjTfeiK+++gpnz55FbGwsBg8ebL09NjYW+/fvx913323dt3///mb9Dfbu3YsePXoAAAoKCnD69Gn07dsXADB48GAkJiYiJiamWedqSFxcHBQKBdLS0ppsnWssnr59+2LXrl02x+/atQu9e/eGXC5H//79YbFYsH37dowfP77V8Z47dw5lZWUYNGhQq8/hyrpsAjh//nysWrUKP//8M9RqNTIzpVU0NBoNPD09ERISUu/Ejx49etRJFh2Cu7LpY4jIYalUKtx222149tlnodfrbUqM9OrVC9999x12794NPz8//Pvf/0ZWVlazE8Dx48ejd+/emDNnDt544w3o9XqbRK/qMdLS0rB69WoMGzYM69ats7aQVYmMjERKSgqOHDmC7t27Q61W1/nSO2vWLCxZsgRz5szB0qVLkZOTg4ULF+Kuu+5CcHBw6/44laq6B2tSKBQYP348+vfvj1mzZuHtt9+GyWTCI488grFjx2Lo0KEoLS3FU089hZtvvhlRUVG4dOkS9u/fj5tuugkAsGjRIkyZMgW9e/dGQUEBtm3bZk1W6jNp0iTcd999MJvN1m7s5j5Hs2bNwnXXXYeTJ09ak8UqCxcuxP3334+hQ4di1KhRWLNmDY4dO4aePXs2+bd56aWXEBAQgODgYDz//PMIDAzEzJkzAUizu6+44gosWLAA9913H7y9vZGYmIgtW7bgvffea/LcVdRqNZ588kk8/vjjsFgsuPLKK6HT6bBr1y74+Phgzpw5zYrniSeewLBhw/Dyyy/jtttuw549e/Dee+/hgw8+ACC9zubMmYN7770X77zzDhISEpCamors7GzceuutzY53586d6Nmzp00rJ7WAvQchdhQA9V5WrFjR6H0aGzBdGweREnWeRieBOIndu3eLAMSpU6fa7M/LyxNnzJghqlQqUavVin//+9/Fu+++W5wxY4b1mMYmgYiiNLD+yiuvFD08PMTevXuLGzdurPOe9tRTT4kBAQGiSqUSb7vtNvGtt96ymXhRVlYm3nTTTaKvr6/N+2Xt8xw7dky8+uqrRaVSKfr7+4v333+/aDAYrLfPmTPHJnZRFMXHHntMHDt2bIN/mxUrVtT7nh0dHS2KoiimpqaK119/vejt7S2q1WrxlltusU46MRqN4u233y6Gh4eLHh4eYlhYmLhgwQLra2XBggVidHS0qFAoxKCgIPGuu+4Sc3NzG4yloqJCDAsLEzdurJ5015znSBSlSYehoaEiAPHcuXN1zv3SSy+JgYGBokqlEu+9917x0UcfFa+44ooGY6maEPHLL7+I/fr1Ez08PMThw4eLR48etTlu37594oQJE0SVSiV6e3uLAwYMEF999VXr7bVfLw2xWCzi22+/LcbGxoru7u5iUFCQOGnSJHH79u0tiue7774T4+LiRHd3d7FHjx7iG2+8YXN7aWmp+Pjjj4uhoaGih4eHGBMTI/7vf/+zeYyCggLr8YcPHxYBiCkpKdZ9EydOFJctW9bg78JJII0TRJEDylpLr9dDo9FAp9PBx8en8x7YYgFkXX74JpGNsrIypKSkICoqqs4AfaL29v7772Pt2rXYtGlThz7OhAkTEBISgi+++KLe2//44w9cffXVKCgocIiVUhwlnpMnT+Kaa67B6dOnodFo6j2msfcMu31+O5Au2wXcJV06AGQnAUVZwMgF7BYmIuogDz74IAoLC2EwGNptObiSkhJ8+OGHmDRpEuRyOb7++mts3bq13qLd1LiMjAx8/vnnDSZ/1DQmgM6kJB/QVS4nVZQJ+EXaNRwioq7Kzc2tzjjKthIEAevXr8err76KsrIyxMbG4vvvv2/TRAhXxb9Z2zEBdCbqGpNW9BlMAImInIinpye2bt3aovuMGzcOjjRSy9HiodbjQDJn4lNd5BUGFoQmIiKi1mELoAMyW0TsS8lHtqEMWrUSw6P8IZcJUjFoNw/AVC61ABIRERG1AhNAB7PxRAZe/CURGboy675QjRJLpsdhcnyotCxcQWr18lSK9hmcTOQs2P1ERM3B94rGsQvYgWw8kYGHvzxkk/wBQKauDA9/eQgbT2TYjgM0ZHZyhET2U7UsFNf9JKLmqHqvqL0sIUnYAuggzBYRL/6SiPq+r4gABAAv/pKICfeGwbq8uj4dCOzVaTES2ZNcLoevr691MXgvLy/rUmpERFVEUURJSQmys7Ph6+trXc2FbDEBdBD7UvLrtPzVJALI0JXhYL4S1hVPDRwHSK6lavnGqiSQiKghvr6+9S75ShImgA4i29Bw8ldTRpk74OENlBdLLYCiCLAVhFyEIAgIDQ2FVqtFRUWFvcMhIgfl7u7Olr8mMAF0EFp181b10Pp4AiHjADeFNCGEyAXJ5XK+uRMRtQETQAcxPMofoRolMnVl9Y4DFACEaKSSMJAFdHZ4RERE1IVwFrCDkMsELJkeB0BK9uqzZHqcVA+QiIiIqA2YADqQyfGhWD57MEI0tt3B/t7uWD57sFQHkIiIiKiN2AXsYCbHh2JCXAh2nM7BmgMX4e0hR99QH0zqV2smU2kBoLsMFGUCPa8GZBwPRURERM3DBNAByWUCru6jRX5xOdLyS2AoMyFLb7RtGTy/HchOkraD+wPqYPsES0RERE6HXcAOLDakepm301kG2xt9wqq3DemdFBERERF1BUwAHViMVmWd9HE6y2C7rmHNEjB6FoQmIiKi5mMC6MCU7nJEBHhB6S5HZIA3ys2W6htVwdUFoNkCSERERC3AMYAO7tq+wfB0l9ct/+LmAXgHAkU5QHEeYCqX9hERERE1gS2ADk6lcGu49p+6chygaAGKsjovKCIiInJqTACdmU+NcYAGjgMkIiKi5mEC6ETKKsy4XFhavUNdYyawnuMAiYiIqHk4BtBJrD+egTNZRfBwk+GBMT2lbmHvQEDmBlhMgCHT3iESERGRk2AC6CQEABZRRFmFGWn5JYgK9JZW//CLACxmqTtYFKtnBhMRERE1gAmgk+gdosapTKkYdHKmQUoAAWDArXaMioiIiJwRxwA6iQh/LyjcpafrXE4RTDVrAhIRERG1ABNAJ+EmlyE6SAUAKDdZcCGv2M4RERERkbNiAuhEYoNrrg1cZHujKAIl+dJPIiIiokZwDKATCff3gqeHHKXlZpzPKUK5yQIPNxlwbhuQfhgwGYGRjwBKjb1DJSIiIgfGFkAnIpcJ6KWVuoErzCJSciu7gQWZlPwBgJ4FoYmIiKhxTACdTO8a3cDJWdKsYPjUKAhtYEFoIiIiahy7gJ1MN19PqBRuUCnd0MPfS9qpDqk+gAWhiYiIqAlMAJ2MTCbgrpERULrLq3cq1NLFaJDWBGZBaCIiImoEu4CdkE3yV8UnVPppKgdK8jo3ICIiInIqTAC7CnWNcYB6jgMkIiKihjEBdGKiKCJLX4YMXWl1CyAgdQMTERERNYBjAJ1UsdGEbw5cRGFJBcL9vXDzgJoTQZgAEhERUcPYAuikvDyqxwFeKihBkcUN8AqQdpTpAQvXCiYiIqL6sQXQSQmCgNhgNf5KyYcoAmeyDBjUZyrg7gV4+nEWMBERETWILYBOrHdIdVHoM1lFgKY74OXP5I+IiIgaxQTQiQWqFAhUeQAALheWQl9WYeeIiIiIyBkwAXRyNZeGO1O1NBwRERFRI5gAOjmbtYEzi4CCVODcNuDI14DJaMfIiIiIyFExAXRyft4e0PooAABZ+jIUXToOpO0FCi6wHAwRERHViwlgFxBboxXwQrlv9Q2GzM4PhoiIiBweE8AuoFewGh5uMvQNVcMvuEf1DVwSjoiIiOrBOoBdgMbTHQ+O6Qk3uUwqAH3eHTBXsAuYiIiI6sUWwC7CTV75VMpkgLpyXeAyPWAssl9QRERE5JCYAHZF6prrAnMcIBEREdliAtjFlJssOF/ug9IKs7TDwHGAREREZItjALuQ1Lxi/HI0HTKjDNNKyhDh7w3oOQ6QiIiIbLEFsAvRqpUwWwCjXIWsUjlEiFILoCjaOzQiIiJyIEwAuxBPDzl6BHgCgoCLihjk+w8Eek1kAkhEREQ2mAB2MVVLw6X5jsBx5VAguJ80M5iIiIioEjODLiY6SAW5TAAAnMkqgsXC1j8iIiKyxQSwi1G6yxER4AUAKDKacLmw1M4RERERkaNhAtgFxYZUrg0sijh3MR3ISZZWCCEiIiJCF04Aly1bhmHDhkGtVkOr1WLmzJlITk62OebBBx9EdHQ0PD09ERQUhBkzZuDUqVN2irj99AxUwV0uoGf+DqgOfwTx+PdASa69wyIiIiIH0WUTwO3bt2P+/PnYu3cvtmzZgoqKCkycOBHFxcXWY4YMGYIVK1YgKSkJmzZtgiiKmDhxIsxmsx0jbzsPNxmiAlUodfdFhVmErqwC0LMgNBEREUkEUXSNGiE5OTnQarXYvn07xowZU+8xx44dQ0JCAs6ePYvo6Ogmz6nX66HRaKDT6eDj49PeIbfJ2WwDjp44jhGFGxCkVkDRYwgQO8XeYREREdmdI39+d5Yu2wJYm06nAwD4+/vXe3txcTFWrFiBqKgohIeHd2ZoHSI6SIUbxwxBd39vKNzkbAEkIiIiK5dIAC0WCxYtWoTRo0cjPj7e5rYPPvgAKpUKKpUKGzZswJYtW+Dh4VHveYxGI/R6vc3FUQmCAMFNAXgFSDuKcwFzhX2DIiIiIofgEgng/PnzceLECaxevbrObbNmzcLhw4exfft29O7dG7feeivKysrqPc+yZcug0WisF6doKfQJk36KFqAoy76xEBERkUPo8mMAFyxYgJ9//hk7duxAVFRUo8eWl5fDz88Pn3zyCe644446txuNRhiNRut1vV6P8PBwxx5DcPkQSk+uR16RESFDroNbxAh7R0RERGRXHAMIuNk7gI4iiiIWLlyIH3/8EX/88UeTyV/VfURRtEnyalIoFFAoFO0daofan6+E6WIhAEB+OQWhTACJiIhcXpdNAOfPn49Vq1bh559/hlqtRmZmJgBAo9HA09MT58+fx5o1azBx4kQEBQXh0qVLeO211+Dp6YmpU6faOfr2ExwSjouCHDLRjLzMVITaOyAiIiKyuy47BnD58uXQ6XQYN24cQkNDrZc1a9YAAJRKJXbu3ImpU6ciJiYGt912G9RqNXbv3g2tVmvn6NtP9wAVTJ5BsAhuyCx1g7G83N4hERERkZ112RbApoY2hoWFYf369Z0Ujf3IZAIQfyP2ZZQDggzd8srQN7T+Wc5ERETkGrpsCyBVi+4eAgjSU306y2DnaIiIiMjemAC6gG6+nlArpcbe1LwSlFU491J3RERE1DZMAF2AIAjoFawGAJjNFpxlKyAREZFL67JjAMlWP/lFFOf8CZUxB+c1sxDf3dfeIREREZGdsAXQRQRAj9CKS3C3lKIwKw0l5SZ7h0RERER2whZAFyH4hCFQrUBJuRlXh5ngLmfuT0RE5KqYALoKn1CE+3lJ226FABNAIiIil8UswFUo1IBCJW0bMoCuvQQ0ERERNYIJoCtRVy4EZyoHSvLtGwsRERHZDRNAV+ITBgCwiCLSUs9i99lcOwdERERE9sAE0JVUtgAmZehx8Hgi/krJh66kws5BERERUWdjAuhKKhNAXy8PqMqzAQCns1kUmoiIyNUwAXQl7krAyx8B3h7wrsiDIJqRnMkEkIiIyNWwDIyr6XEFlAAyPS0QS2XIMRiRX1wOf28Pe0dGREREnYQtgK4mNAEITUBEj0hAEAAAp7k2MBERkUthAuiiegerrNunswwQWReQiIjIZTABdFFqpTu6+XkCAPKKypFbVG7niIiIiKizMAF0ReUlQN45DLacgNwiJX7sBiYiInIdTABdUcoO4Ng36KE/BFWFVAw6OZPdwERERK6Cs4BdkU8okH4YHnIZ4tXFKA/zR+8QVdP3IyIioi6BCaArUodZN4cHGIFegXYMhoiIiDobu4BdkVcAIHeXtg2Z9o2FiIiIOh0TQFckkwHqEGm7TAeUF9s3HiIiIupUTABdVeW6wAAAfQZ0pRU4cCEfB1ML7BcTERERdQqOAXRVPtXjAE26y/j8qAiTRYS3Qo5B4b6QyQQ7BkdEREQdiS2ArqqqCxiAW3EWIgO9AQDFRjMuF5baKyoiIiLqBEwAXZXSF3CXVgKBPh2xNZaGS85kUWgiIqKujAmgqxIEQNNdagnU9kWknwIebtLL4Ux2EcwWFoUmIiLqqjgG0JXF3yQlggA8APQM9MapTAPKKsxIyy9BVGW3MBEREXUtbAF0ZYLtRI9ewWrrNruBiYiIui4mgGQVGeAFhbv0kjiXUwST2WLniIiIiKgjMAEkSXkx3AQgOkiaDFJusuBCXomdgyIiIqKOwATQ1V3cD+z5ANj1DlCSi9ga3cCns9gNTERE1BVxEoirE83ScnAAYMhAePAA9ApWITLAGzFaVeP3JSIiIqfEBNDV1VoSTh6agOsGhDV8PBERETk9dgG7OnVo9WxgQ7p9YyEiIqJOwQTQ1bl5AF4B0nZRDmCusG88RERE1OGYABLgU9nlK1qAoiwAgMUiIi2vBFsSszgZhIiIqIthAkjScnBVDJkAgEx9Gb4/dAknLuuQmK63U2BERETUEZgAEqCuMelDL40DDNUooVZKc4RS80pQWm62R2RERETUAZgAEqDSAjK5tG3IAAAIgoDelTUBLaKIs9lF9oqOiIiI2hkTQJKSP1WwtF2mB0zlAIDYkBprA3McIBERUZfBOoAkib4akLkB3lpALr0stGoFfL3cUVhSgUsFJSg2muCt4EuGiIjI2bEFkCS+PaTZwPLqBE8QBOvScKIInGE3MBERUZfABJAa1btGN/DpTHYDExERdQVMAKlRgSoFAlUeAIDLhaXQl7FQNBERkbNjAkjVDFlA2l7gxA+Asbq1r2o2MACczym2R2RERETUjjiin6plJ0oJIAAExwNBUuIXG6JGSYUZvYPVCNMo7RggERERtQe2AFI1nxoFoQ3p1k1fLw9cHatFN19PCIJgh8CIiIioPTEBpGrq0OptfYb94iAiIqIOxQSQqinUgEIlbRsypNovRERE1OUwAaRqglDdCmgyAqUFNjcXG004crEQ3+y/iLwiox0CJCIiovbABJBs2XQDp9vclJxlwLZT2bhcWMql4YiIiJwYE0Cy5VMjATTYjgPsHaxG1RyQ05kGiOwiJiIickpMAMlWIy2AKoUbuvl6AgAKSiqQY2A3MBERkTNiAki23D0BTz9puygbsJhtbo6tuTRcFtcGJmo1ixlI2Qkc/076Wet/jYioI7EQNNUVFAuUF0vdwaIFgNx6U4xWhW2ncmARRSRnGTA6JoC1AYlaKnEtsPFp21Z2nzBg8utA3PX2i4uIXAZbAKmu6KuBvtcB3YYAcnebm7w83NAjQOoG1pdWIFNfZo8IiZxX4lrgm7vrDLGAPkPan7jWPnERkUthAkgt1ktb3Q2cnMnZwETNZjFLLX+obwJV5b6Nz7A7mIg6nMMlgBcvXsSlS5es1/ft24dFixbhv//9b4vOs2zZMgwbNgxqtRparRYzZ85EcnKy9fb8/HwsXLgQsbGx8PT0RI8ePfDoo49Cp9O12+/SVcVoVZDLpG7fM1lFnA1M1Fypu+u2/NkQAf1l6Tgiog7kcAngnXfeiW3btgEAMjMzMWHCBOzbtw/PP/88XnrppWafZ/v27Zg/fz727t2LLVu2oKKiAhMnTkRxcTEAID09Henp6XjzzTdx4sQJrFy5Ehs3bsS8efM65PdySiYjUHABMFfY7Fa6yxER4AUAKC43IYdFoYmapyirfY8jImolQXSw5hs/Pz/s3bsXsbGxeOedd7BmzRrs2rULmzdvxkMPPYTz58+36rw5OTnQarXYvn07xowZU+8x3377LWbPno3i4mK4uTU9P0av10Oj0UCn08HHx6dVcTms89uBtD3ScnCDZgG+PWxuTssrQX5JOXppVfBWcC4RUbOk7AQ+u67p4+b8CkRd1fHxELmoLv353UwO98ldUVEBhUIBANi6dSuuv16aEdenTx9kZGQ0dtdGVXXt+vv7N3qMj49Pg8mf0WiE0Vjd2qXX61sdj8NTaqrXAtZn1EkAewR4oUdlKyARNVPEKGm2b4PdwIJ0e8SoTg2LiFyPw3UB9+vXDx9++CF27tyJLVu2YPLkyQCkLtuAgIBWndNisWDRokUYPXo04uPj6z0mNzcXL7/8Mh544IEGz7Ns2TJoNBrrJTw8vFXxOAWfsOptQ2Njloio2WRyqdQLhMpLPSa/Jh1HRNSBHC4BfP311/HRRx9h3LhxuOOOO5CQkAAAWLt2LYYPH96qc86fPx8nTpzA6tWr671dr9dj2rRpiIuLw9KlSxs8z7PPPgudTme9XLx4sVXxOAWvQEBe2RJqyGzycAcbSUDkuOKuB2793HbZRQBQ+AC3fMY6gETUKRyuC3jcuHHIzc2FXq+Hn5+fdf8DDzwAL6+WdzkuWLAAv/76K3bs2IHu3bvXud1gMGDy5MlQq9X48ccf4e7uXs9ZJAqFwto93eXJZIAqBNBdAkoLpcLQHt51DsvUlSEpU4+L+SWYNSLCOjuYiBoRdz3QZ5o02zdpLWA0AJpwIHyYvSMjIhfhcC2ApaWlMBqN1uQvNTUVb7/9NpKTk6HVapt9HlEUsWDBAvz444/4/fffERUVVecYvV6PiRMnwsPDA2vXroVSqWy336NLqNlC0UAr4JGLBTiSVoi8onKk5hV3UmBETixtrzQGUJBJEz0G3gkE9gZC4gGBXb9E1DkcrgVwxowZuPHGG/HQQw+hsLAQI0aMgLu7O3Jzc/Hvf/8bDz/8cLPOM3/+fKxatQo///wz1Go1MjOlBEaj0cDT09Oa/JWUlODLL7+EXq+3TuoICgqCXM43YqhrjAPUpwMB0XUO6R2sRlKGVAz6dJYBPYNUnRUdkfMpyQfOSWWuENQbiL8JCO4PhCRIre5ERJ3E4d5xDh06hKuuksoffPfddwgODkZqaio+//xzvPPOO80+z/Lly6HT6TBu3DiEhoZaL2vWrLE+zl9//YXjx48jJibG5pguPbavJZrRAhgR4A2Fu/QyOpdTjAqzpTMiI3JOWSert30qh6TI3Zj8EVGnc7gWwJKSEqjV0lJjmzdvxo033giZTIYrrrgCqampzT5PU5MSxo0bx4kLTVH6Au6eQEWpNBNYFAHBdoyfXCagl1aNE5d1KDdZcCG3GL2C1fWfj8iViSKQnShtCwKg7WvfeIjIpTnc186YmBj89NNPuHjxIjZt2oSJEycCALKzs122WKPdCAKgDpV+unsBprJ6D+sdXN3tm5zFtYGJ6mXIlLqAAWnCh7Ke9zNTuTTxioiogzlcC+DixYtx55134vHHH8c111yDkSNHApBaAwcNGmTn6FxQ70lSK6Bbw7Ofw/284OUhR0m5GSk5xTCazFC4cQwlkY3sGt2/wf3q3n5qHZBV2UI4+jHAzaNz4iIil+RwLYA333wz0tLScODAAWzatMm6/9prr8Vbb71lx8hclKdvo8kfAMhkAnpVtgKaLCJScjkbmMiGxQJkJ0nbMjkQFFv3GEEGWEzSpbD5w12IiFrD4RJAAAgJCcGgQYOQnp6OS5ek7pDhw4ejT58+do6MGtK7xri/5Ex2AxPZKEwFjEXStn9PqVW9toBe1du5ZzonLiJyWQ6XAFosFrz00kvQaDSIiIhAREQEfH198fLLL8Ni4QxTR9XN1xNqpTSiIMdghNnCCTZEVlWTP4D6u38BwC+ievWdvLPVa3ETEXUAhxsD+Pzzz+PTTz/Fa6+9htGjRwMA/vzzTyxduhRlZWV49dVX7RyhC8o7J32A6TOAfjcAqqA6hwiCgKt6BUHpLkO4nxdkXBGESGI2ATnJ0rabBxAQU/9xcnfAL0pq/SsvliaN1F4ujoionThcAvjZZ5/hk08+wfXXV6+HOWDAAHTr1g2PPPIIE0B7KM4BMk9I24b0ehNAAIgNYfkXojoEGRA3Q/oSJVdIiV5DAqKru3/zzjABJKIO43AJYH5+fr1j/fr06YP8/Hw7RERQ1/gQ0mcAoQn2i4XI2chkUmJXz0o6ddRsHcw7C0SN6bi4iMilOdwYwISEBLz33nt19r/33nsYMGCAHSIiqEOqC0AbMpp9NwvHARK1jEIt/b8BgCELKNPbNx4i6rIcrgXwn//8J6ZNm4atW7daawDu2bMHFy9exPr16+0cnYtyUwBeAUBxLlCULY1pktf/0rFYRCRl6nE6ywCLBbhpSPdODpbIyQXEVC+9mHcW6DbYvvEQUZfkcC2AY8eOxenTp3HDDTegsLAQhYWFuPHGG3Hy5El88cUX9g7PdVV1A4sWoCirwcMEAThwoQAXcktwsaAEhrKKTgqQyAGd/wPIPA6YjM2/T1U3sCoIkLMYNBF1DEF0kgVxjx49isGDB8NsNts7FCu9Xg+NRgOdTtf1l6m7fBA4vVna7jUB6D60wUN3n8vFX+fzIYoiAlUKaLzcoVUrMTzKH3LODiZXYTQAe96Xyrl4BwLD72/e/UQRMOoBpaZj4yNyYS71+d0Ah+sCJgelDqvebmIcYGywGl/tTcX20zkoMlYn7KEaJZZMj8PkeM5sJBeQfaq6ll9g7+bfTxCY/BFRh3O4LmByUCqttIQVIM0EbsT+C/lYdzzTJvkDgExdGR7+8hA2nmj+RBIip9XU2r9ERHbEBJCaRyaXkkAAKMkDKsrqPcxsEfHiL4n13lY11uDFXxK5Ugh1bSX51V+U1MFSF3BbzuUcI3WIyIk4TBfwjTfe2OjthYWFnRMINSxsEBDUVypO20Ax230p+cjQ1Z8cAlISmKErw76UfIyMDuigQInsrObSb9pWtv5lHAXS9koJ4JC5LApNRO3KYRJAjabxMS8ajQZ33313J0VD9WpGAehsQ8PJX2uOI3I6oghkVXb/CgKg7du681jMUvIHSOVgmAASUTtymARwxYoV9g6B2oFWrWzX44icjiGzOnHThAPKVs4wDIgBsEnazjsLRF3VLuEREQEcA0jtbHiUP0I1SjRU7EWANBt4eJR/Z4ZF1HlsJn/Etf48Sh9p/CAgJZVGQ9viIiKqgQkgtYzZBOguA5cOAOXFdW6WywQsmS596NVOAquuL5kex3qA1DWJolT+BZAmTgXVXde8RWqvDUxE1E6YAFLLXNgJHPocOLNFSgTrMTk+FMtnD0aIxrabN0SjxPLZg1kHkLouQQAGzQKixgBhgwF3z7adzyYBPNe2cxER1eAwYwDJSahrJG+GDCCo/gK3k+NDMSEuBPtS8pFtKLNZCSRLX4ZdZ3MxtX8olO7yTgqcqJN4+gGRo9vnXOpQwMNbam0vSAHMFQ3OwCciagkmgNQyPrUSwEbIZUKdUi+pecX45Wg6Kswi1h/PwIyB3dgdTNQQQZBaATOOSsMvClKBwJim70dE1AR2AVPLKHykFgkA0Ke3uECtj9Idcpn0skvNK8Hvp7LhJMtRE9lHYK/qbY4DJKJ2wgSQWkYQAJ/KdYFNRqC0oEV39/P2wPUDw6ytficu63AgtWXnIHJIZ7YC57cDxbnte17fCEDmBsjZYUNE7YcJILWcOqR6u4lu4Pp08/XExH7B1ut/nsnF6SyWuCAnVlEGpB8GUncDR74CLJb2O7ebhzSxZPTjQOzk9jsvEbk0JoDUcjUnguhbngACQJ8QH4yqMT5w04lMpBeWtjUyIvvIPQ1YTNK2Ng6QtfNbq08YWwCJqF0xAaSWs5kJnN7q0wyP8kdcmLRKgskiYu3RdBSWlLc1OqLOZ7P2bxuKPxMRdRImgNRyHl5SqQsAMGRJa5a2giAIGN83GOH+XgCA0nIzfjp8GSZzO3afEXU0YxFQcEHa9vStHiPbUcwmqRwMEVEbMAGk1vEJlT7sAntJk0FaSS4TcN2AUASoPCATBAyN9IebnC9LciLZSdWz4bVx0kSpjmDIBE58D+x6G8g60TGPQUQug4NKqHX6TG+3cU5KdzlmDOwGXUkFegR4tcs5iTqNzdq//TrucUQLkHNa2s47B4QN6rjHIqIujwkgtU47D3LXeLpD48kVDsjJlORXT4RSaQHvwI57LK4KQkTtiH1t5LCSMw1IytDbOwyihtWc/NGRrX9A9aogQPWqIERErcQWQGo7cwUgyNutVVAURey/UIBdZ3MhlwlQKdysE0WIHErVyhyC0Dmzf6uWhat6bC4LR0StxASQWi/zOHBxn7TywcA7Ad/wdjt1kVGa5Wi2iPjlWDpuH9YD/t4e7XZ+onYxcJY0Hq8oC1D6dPzj+UVKq4JYTFICKIodN+mEiLo0dgFT61lMQFG2NDi9FSuCNEQQBIzrrUVUoLTmsLHCgp8OX0ZJuandHoOoXcjdAW0foOfYznk8Nw/AL0LaNhqkxJOIqBWYAFLrqWvUO9O3viB0fWQyAVP6hyBIrQAA6EorsPZIOipYI5BcXUB09XZVFzQRUQsxAaTW8w6sXp6qHVsAqyjc5JgxMAwqhfQYGboybD6ZBbGq5hqRKwqoMe6PCSARtRITQGo9mRxQBUvbpYVAeUm7P4Ra6Y4Zg8Lg4Sa9VE9nGbDrbF67Pw5Ri5zZApz4AchJbvVKOK2m1EglZwCgooyrghBRqzABpLap2Q3cAa2AAKBVKzG1f6h1rPv+C/k4fknXIY9F1CSzSZoAlZMMJP0ijYHtbL0mAsPvB0Y8yFqARNQqTACpbXxCq7c7KAEEgKhAb1wdK7V6eHrI4a/ijGCyk/zz1csfBsXaJwHzDZeGYHAGMBG1EsvAUNuoaySA+o5LAAEgIdwXFWYLYrQq+HoxASQ7qbn0W2fU/iMi6gBMAKltPP0Ad6U0FsmQ3uF1yYZG+nfYuYmaZDICuZUTLzy8AL8o+8ZTxWJp9+UZiahr4zsGtY0gVLcClpdItck6kSiKOHKxEOUmloehTpCTLNW/BICgvvZNuipKgZQdwIH/AWe32C8OInJKbAGktusxEug+rHKx+s5bss1ktmBzYhaSMw1IzSvG9AFhkMk4Joo6kM3av3bu/hXkQNpeaRZyRSkgTuSYQCJqNrYAUtv5RUjFaTsx+QMAQ5kJF/KKAQDnc4qx/UxOpz4+uRhjEVBwQdr29AV8utkzGmlVEN/KVUHK9NKqPEREzcQEkJyWn7cHrusfBlllq8eRtEIcTiuwc1TUZeWcksa4AtLkD0dobWNRaCJqJSaA5NR6BHjh2r5a6/Xtp3NwLqfIjhFRl1VzucPgfvaLoyabZeHO2C8OInI6TACpfZQWAOmHgVPrgZL8Tn3o+G4ajIiSZgeLIrDheAay9GWdGgO5gLjrgWHzgJhrpRp8jsDTF1AFSdv6DKmbmoioGZgAUvvIPgUkbwQyjgK6S53+8COjAxAbogYAVJhF/HzkMvRlXCKL2plKC4QPt3cUtmp2A+efs18cRORUmABS++ikFUEaIggCJsYFo5uvJwCg2GjGz4cvw2jq5HVaiTpbQK/q7Vx2AxNR8zABpPahCqkeFF9zrFQncpPLMD0hDL5e1UtzsT4gtZnFwb9E+IRVz8AvuCCtVUxE1AQmgNQ+3JWAZ+UqHcU5dvsQ8vSQY+bAbogNUePWYeFQK+2wTit1Lef/APZ/AqTukVa8cTSCAPhXTgYRLUAxy8EQUdNYCJraj08oUJIntZgUZ0stE3bg5+2Bqf1Dmz6QqCkWi1T82VgElOwEQhPsHVH9ug2WxgL6RwFuCntHQ0ROgC2A1H7UNRI+feePA2xMucmCi/kl9g6DnI0urXpmrX/PTi923mw+YYC2D5M/Imo2JoDUfmwmgthnHGB9iowmfHvwIn46fBnphaX2DoecSVaNpd+0dl76jYioHTEBpPbjrQVkcmnbgVoAD6cVIFtvhMkiYu3RdBSWlNs7JHIGZpO0+gcAyN2BwF6NH09E5ESYAFL7kbsB3pVFaUvzHWbA/KjoQIT7S113peVm/HwkHWUVDj6zk+wv/zxgMkrbgb2lJNCRiSKQnQQk/QIc+9be0RCRg+uyCeCyZcswbNgwqNVqaLVazJw5E8nJyTbH/Pe//8W4cePg4+MDQRBQWFhon2C7koBoqaUk8ioAor2jAQDIZQKuGxCKAJUHACC/uBxrj6bDZGaJGGpE9snqbUdZ+q0xggBc+BPIPCEVhOaqIETUiC6bAG7fvh3z58/H3r17sWXLFlRUVGDixIkoLi62HlNSUoLJkyfjueees2OkXUzUGKD/zUDkaMDd097RWCnd5ZiR0A1eHlIX9eWCUmxNyoYoOkaSSg7GZARyz0rbHl6AX6Rdw2m2qm5qUeSqIETUqC5bBmbjxo0211euXAmtVouDBw9izJgxAIBFixYBAP74449Ojo7sQePljusHhuG7A5dgsohIytBD4+mOkdEB9g6NHE1OMmCprGUZ1Ld6bKujC4iR6hUCQN5Zxy1bQ0R212VbAGvT6XQAAH9//1afw2g0Qq/X21zIuYRqPDGlf4h10ZK95/OQmM7nkWoxGgBZ5ffjYCea/asOq255z0/hqiBE1CCXSAAtFgsWLVqE0aNHIz4+vtXnWbZsGTQajfUSHh7ejlF2MRYzYMiqHkTvQGK0alzVK9B6PSW3mF3BZCtyNDBqIRB3PeDTzd7RNJ9MJrUCAoC5AihMtW88ROSwXCIBnD9/Pk6cOIHVq1e36TzPPvssdDqd9XLx4sV2irCLuXQQ2Plv4MD/gELH/BsN7uGHAd01GBzhhynxIRCqmgSJqrgrpckfzvbaqEoAASCP4wCJqH5ddgxglQULFuDXX3/Fjh070L179zadS6FQQKFgpf0meXhVj58ypAOBMY0fbweCIOCaPlomftT1+EdJYxYtZiDvDCBOcL4klog6XJdtARRFEQsWLMCPP/6I33//HVFRUfYOyXWoa6wI4kAFoWurL/nTl1WgguVhXJfJ6Pzj5twUgG8PabtMDxTn2DceInJIXbYFcP78+Vi1ahV+/vlnqNVqZGZmAgA0Gg08PaVB0pmZmcjMzMTZs1K5h+PHj0OtVqNHjx5tmizi8pQaqRWwvERqARRFp2iByNSVYe3Rywjz9cS0/qFsHXRFlw4AF/cCgbHSOEBPP3tH1DoBMdIkEED6qdLaNx4icjhdtgVw+fLl0Ol0GDduHEJDQ62XNWvWWI/58MMPMWjQINx///0AgDFjxmDQoEFYu3atvcLuGgRBmo0oWqRyGoc+A1J2Sl1SDqrCbMHPRy6j2GjGmawi7DqbZ++QqLOJIpB1EjCVA5nHAcGJ3x4DYoDIK4Ehc4Hw4faOhogckCBy+mOr6fV6aDQa6HQ6+Pj42Dscx/LH68Dud4DyGqsR+IQBk1+XZlY6oJTcYvx85DKq/iPG9w1G/+4a+wZFnceQCRxYIW37hgODZts3HiLqMPz87sItgGRHiWuBP/5hm/wB0njAb+6WbndAUYHeuDq2uqvs91PZuJBb3Mg9qEvJqrH0m9aJav8REbUCE0BqXxYzsPHpBm6sbFrb+IzDdgcnhPticIQ07ssiilh3PANZujLsOZeHn49cxp5zeTBb2Gje5YgikJ0kbQsyIKiPfeMhIupgXXYSCNlJ6m5An97IASKgvywdF3VVp4XVEmN6BUJfWoGz2UVITNdh/PZzMJRVzwwN1SixZHocJseHNnIWciqFadLqHwDg31OaxNQVFOdJpWAKLgDxNwNyvuUTkYQtgNS+irLa9zg7EAQBk+NDkFtkxLrjmTbJHyDNFn74y0PYeMJxS9xQC2UnVm8H97NfHO0tbTdwbps0E1iXZu9oiMiBMAGk9qUKbt5xXoFNH2NHMkHA1qT6k9SqDuAXf0lkd3BXYDYBOaekbbm77Uoazi6gV/U2VwUhohqYAFL7ihglzfZFQzX0BMA7SPrA1V3qzMhaZF9KPrL0Da9jLALI0JVhX0p+5wVFHaMgBagok7YDewNuHvaNpz1VrQoCALlnABZ9IKJKTACpfcnkUqkXAHWTQAGACPQcB5QWAIe/BM5ulRatdzDZhrJmHZelL+3gSKjDmSuk4uVA1+r+BaRVQTTh0naZDijOtW88ROQwmABS+4u7Hrj1c8Cn1iQJnzBgxnKpQC0gtUZc3A/s/1QahO9AtGpls45LzixCXlHDLYXkBILjgCseBgbfBfhF2jua9hdYsxv4jP3iICKHwilh1DHirgf6TJNm+xZlSWMDI0ZVLlJvAS7tq1wdxFTZGvgV0G2I1DroAF1ww6P8EapRIlNXhoY6zVQKOdzlAr7cm4aEcA2u6BkApbu8U+OkdiIIgKa7vaPoGAHRwJkt0nbeWen/kIhcHlsAqePI5FKpl/43Sz+rxiLJZECPK4Ch9wKabtXHXz4I7P+keg1TO5LLBCyZLhUDbmg045R4ab1giyjicFohVu6+gOOXdLBwYgg5Ek8/wLty0pU+HShncXMiYgJI9uQdAAycDcSMr65PVqYDjq4GMo7ZNzYAk+NDsXz2YIRobLuDQzVKfDh7MP5xY3+MjA6Au1xKEUvLzdialIWv96fhciHHBjq8ijKgtNDeUXSOqpnNosjZwEQEgGsBtwnXEmxHJflA8gZpLKCHNzD8fsDd095RAQDMFhH7UvKRbSiDVq3E8Ch/yGXV7YL6sgr8eSYXyZkGm/sN7OFrs7QcOZhLB4Ezm6VW6Ohrum4XMAAUXpQmXQFAUCwQf6N94yGyM35+cwwgOQovf2DgnUD6IUDhUzf5E0VpnJYdyGUCRkYHNHi7j9IdU/uHon83DbafzkGOQZoUEuBt/7GM1IisE9JP3WVArrBvLB3NpxsQEi9NcvHvae9oiMgBMAEkxyEI0kSQ2owGqVu459VAoOMW6Q3398Kdw3vgRLoOZ7KKEB+msbndZLZALhMg2CmRpRpKC6qXLFQFSZeuTCYD+k63dxRE5ECYAJJjE0Xg9Capftnxb6U6bTHjHXatVplMwIDuvujfTVMn0fv9VDaKjCaM7R2EAFUXb3FydFk1ln7TdrHaf0REzcBJIOTYLCZAtFRfzzopzRTOSbZfTM1QO/nL1JXhZLoeqXkl+HJvGrafzkFZhdlO0bk4UbRd+1fb136xEBHZCRNAcmxyd6D/LVJNQbfKVrPyYuDED8DJH52mpEW5yQK1Umpwt4giDqUW4LPdF3Disg6ch9XJirKrV8TwDQc8fe0aTqcyVwC5Z4HkjUBxnr2jISI7YgJIjk8QgNAB0szgmqsaZJ8C9n0sdec5eBLVI8ALc0ZF4oqeAXCrnEFcUm7GlsQsfL3vItJZNqbzZJ+s3tbG2S8Oe0g/Ig2lSD8M5J62dzREZEdMAMl5KNRA/E3SKiPulbX5KkqBxJ+B0xvtG1szuMtlGBkdgLtHRaJ3sNq6P0tfhjX7L2LjiUwUGU12jNAFiGL1+D9BBgT1sW88nS0guno776z94iAiu2MCSM5FEKSJIMPul+qZVXGiNVw1nu6YNiAUNw/pjkBVdamYpAw90vJK7BiZCyhMk2aVA1I5FAedTNRhvPwBr8qSRvrLQDlfb0SuigkgOSeFSipm2+8GIGyQUw7kD/f3wqwREbi6jxZKdzlCNUr0DVU3fUdqPZlcSvwEGRDsYt2/VQJrrAqSz1VBiFwVy8CQc9P2kS61nVoP+IQBoQl2KyDdHDKZgIHhvogNVqOswlxn9vDB1HxEBargz6LS7UPTHUi4DTAWVU8qcjUBMUDaX9J23lkgpL994yEiu2ALIHU92aeAjKPS0nLH1jjFeq+eHnL41UryLuaXYMfpXHy5NxU7TufAaGLZmHajUEkzzF2RT/fqMbT55wELX1dErogJIHU9+svV2/kpUt3AywcdfqZwbYfSCgBIaxEfrCwbczKdZWOojWQywL9yMoipXBoXSUQuhwkgdT0x1wIDbpVmDQNS7bPTm4GjX0tLgDmJKfGhGNHT31o2pthoxuaTWVi9/yIydWV2js7JmIyVrV2Wpo91BQE1llTM4zhAIlfEBJC6poBoYNh9QNjA6n0FqVJr4KUDTtEa6OEmw6joQNw9MhIxWpV1f6auDF/vS8Omk5koZtmY5sk9DRxdA+x5F8hOsnc09lc1EQYA8s44xf8DEbUvJoDUdbkrgdgpQMLtgFIj7TObgDNbgMNfSq1CTkDj5Y7pCWG4abBt2ZjEdD1W7r6AwpJyO0bnJKpq/5WXAB6qxo91Be5KafJH+HBplR0icjlMAKnr848Chs0Dug2u3ufuCcida2ZtjwAv3DkiAuNig6Bwl/51g32U0Hi66GSG5iovBgouSNtKjTQTmIA+U6XhEr49HHqmPBF1DJaBIdfgpgB6T5JWfjj3u7TthB96cpmAQT38EBuixu6zeUgI961TOqbIaIJKwX9tq+xTgFg59k/b1ymfdyKi9sYWQHItfhHAkLnVE0Sq5J4BUnc7zSQBLw83jI8LRpDatpbdhdxi/O/PFPx5JpdlY6rUXPs3ON5+cRARORA2E5Drqd0CVFEq1QwsLwZykqUxUSqtfWJrA7NFxPbTOTBbROy/kI+kDD1GxwSib6gagiDAbBGxLyUf2YYyaNVKDI/yh1zWxVvDSgsAXWVZIFWQdCFbpYVSQWivAGm4BBG5BCaARAUXgIrKNVENmcDBlUDEKKDHSGnpMItZah0sygJUwdJtMrk9I66XKIropVXhQGoBzBYRRUYTNp3MxPHLhRBFEf/57SwyapSPCdUosWR6HCbHh9ox6g5WNfkDALT97BeHo9KnAwc/k7a1fZgAErkQJoBE2r6AwgdIXg8U50oJX8pOqTVQ5g5sXyZ9UFbxCQMmvw7EXW+/mOvhJpdhVEwg4sJ8sONMLs5lFwEAdpzOwbrjmXWOz9SV4eEvD2H57MFdMwkURSC7ZgLofOtFdzhViDQjuKKselUQB/xyQ0Ttj2MAiQBA0w0Ycg8QMbK6PlrKTuCXhbbJHwDoM4Bv7gYS13Z+nM3g6+WB6xPCcOPgbvD1csf20zn1HldV+e3FXxJhtnTBOnBF2VJCD0gzfz197RqOQ6q9Kojuon3jIaJOwwSQqIrcDeg5DhgyB/AOBM5uaeDAymRp4zMOvY5qRIA3emvVKDI2HKMIIENXhn0p+Z0XWGdx8wC6DwU8vIDgOHtH47hqrgqSe9Z+cRBRp2ICSFSbOgQI6AWUFzVykCitOZy6u9PCao3c4uYVu954MgOnswyoMDvHLOhm8fQDek0ARi4EQhLsHY3jslkV5CxXBSFyERwDSFSfktzmHVeU1bFxtJFWrWzWcUVlJqw7lgEPNxlitCr0CVEj3M8Lsq4wS1gmA7/rNsJdCfiGS0sllhYAJfmAd4C9oyKiDsYEkKg+quD2Pc5Ohkf5I1SjRKauDPW16wgANJ7u6ObrCQAoN1mQmK5HYroe3go5eger0SfEB8E+ijoFp6kLCYiREkBAWhuYCSBRl8evxUT1iRglzfZFI0mPOkw6zoHJZQKWTJfGv9X+Taqu/+PGeNw8JBxxYT7wcKt+Syg2mnE4rRBf70vDZ7sv4MRlXecE3VYWM3DpIGBsrAufbNQcB5jHcYBEroAJIFF9ZHKp1AuABpPAhDuqx045sMnxoVg+ezBCNLbdwSEaJZbPHoyp/cPQI8ALk/qF4IExPXHdgFBEa1U2RaILSiqcZ3xg/nngzGZgz3tA6h57R+McvPylQtAAoLskFUcnoi6NXcBEDYm7Hrj1c2Dj07alYBRqIHo8IHcHLh0AwofZL8ZmmhwfiglxIU2uBOIul6FXsBq9gtUoqzDjTFYRTmXqkaErQ+9g2+XzMnVl2H0uF7EhasRoVVC4OUj9uKzKpd9EEfDmyh/NFhQrjWkNiHaKLzZE1DaCKHLKV2vp9XpoNBrodDr4+PjYOxzqKLVXAqkqGg1Iy8rF3wQE9rJvjB2stNwMTw/bBG9bcjaOpBUCANxkAnoGqRAbokZUoLf9lpgzGYHd7wBmE+DuCYxayMLGRFQHP7/ZAkjUNJkciLrKdl9ZoZQUiiKQ+DMwaLZUPqaLqp38AUBWjWXlTBYRp7MMOJ1lgNJdjl5aKRns7ufZuZNHcs9IyR8grfzB5I+IqF5MAIlaI2qMVDIjOwkwVwDHvwUGzwGUrvNN8rZh4cjQleFUph6ns4pQWi4VnC6rMOP4ZR2OX9ZBrXTDmN5BdbqPO4zN0m8s/kxE1BAmgEStIQhAn+sAox7QXZZmnJ74Dhg4W1qBwgUIgoAwX0+E+XpibG8t0vJLkJypx9nsIlSYpZElhjITFG6248lEUeyYVsHyYiA/RdpW+kjLv1HLWczSknDmii4/tIHIlTEBJGotuZs0/u/Q50BpIWDIApLWAv1urCw+7DrkMgFRgd6ICvRGucmC87lFSM40IMdgRLifl82xSRkGnLisQ2yIGr2D1fV2L7dK9ilArJyprI2TknRqGbNJmj1dUSrNDGYCSNRlMQEkagsPb6D/LVISaDIChkypVdDT196R2Y2Hmwx9QnzQJ8QHJrOlzmoipzL1uFxYisuFpfgjOQeRgV6IDVGjZ6DKpg5hi2WfrN4O7tf687gyuZs0c7owTVoRpDiPRaGJuigmgERt5R0I9LsBSNkutf650DjAprjJbRM6s0VEceVYQQCwiCLO5xTjfE4xPNxkiA7yRmyIDyL8W7gMXWmB1BUPSM+HStse4bumgBgpAQSkotBMAIm6JCaARO3BPwrwi2S3YxPkMgGzR/RATpERyZkGJGcaYCiTZu2WmyxIyjAgKcMALw85JvULQWSgd51zmC1i3XqGbp5A74lAVqLtqhbUcgExwLnfpe28s0CPEfaNp7PULvcUMYqzyKlLYwJI1F5qJ38WC1BuAJQa+8TjoARBgFathFatxJUxgbhUUIrkTANOZxtgrJDG8JWUm+HnZTuZxmIRsTkxEy/+koiMGiVoQjVKLJkeh8nxQ4BuQ6TSPNR63gHS+L+S/OpVQdw97R1Vx0pcW7fgu0+YtBpQ3PX2i4uoA7nWSHWizmIySqVhDn/JNWkbIQgCwv29MD4uGA9c1RPTE8LQO1iNHv5e0Hi52xz7zm+n8dCXh2ySP0BakeThLw9h44mMqpN2VvhdV0C09FO0SEvrdWWJa4Fv7rZN/gBAnyHtT1xrn7iIOhgTQKKOcGaz9MFZppfKw5gr7B2Rw3OTyxCjVWHagFDcOLibzW0mswX/23Wh3vuJlZfFP5+EscJc7zHUQjW70fPO2i+OjmYxSy1/qK/VuHLfxmek44i6GCaARB2h5zhpzWBAaklI+oVdky1Qu07gzjO50FeOFbQlYrTsOMKQi2xDGZ778QTOZhtsjig3WVBkNIGrXraAJhxwU0jbeee6bgKUurtuy58NEdBflo4j6mI4BpCoIyjUwIBbpfIw5gogJxk4/wcQfbW9I3NK+rL6W1C7IRfDZMkYJktGoiUCxcYQqBS2XccXC0qw9kg6PNxk8PPygJ+XO/y8PeDv7QFfL3f4eXnAXc7vwjZkcsC/p7TSjckojQX0i7B3VO2vKKt5x6XskJYW9A7s2HiIOhETQKKOotJK5WGOfyu1/qXtlQbXhybYOzKno1Ur690fK7to3U4Vg3GVnxf8vG0TwMKScgBSS2CWvgxZetsxhACgVrohRKPEdQPC2jFqJxfYS2r9Cugl1bvsilTBzTtOdxHY9zEQOxkIG9SxMRF1EiaARB0pIBqImSCNCQSA5I3SrGC/SLuG5WyGR/kjVKNEpq7MOlpLBgt6C5cAACbRDSXqKDw7tS/kteoHqhTuiAr0Rn5xOfRlFfX2xBvKTPDyqNvF/OuxdBSWVFhbC/29PSpbET3aVrQaDZSzaUntw44W1Lfrr6gSMUqa7dtYN7CHWuoSB+r+35YWAhUlgDq0a/+dqEtiAkjU0boPAUrzgUsHpFmVJ34ABs9hgd0WkMsELJkeh4e/PAQB0vD8SCETSkFq3TsnhuH56xPqTaBiQ9SIDZHGY5rMFhSWVqCguBwFJRXILy5HYUk58kvK4V+r5RAAsvVG6EorkGMw1rlNrXSDr5cH/L3dERvig26+zS+VsvFERiPlbEKbfZ4O1VWXMyzTAyW5Uhe3TC6VevnmbtSdCFL5Wpr6hvRFrjgX8PSzPST9sNSyr/QBAmOBoN6AT/eu+7ejLkUQOTK61fR6PTQaDXQ6HXx8uPoDNcJiAU58Xz2jUtsX6DfTriE5o5qJ0xTZX4iVXYRK4Ybe4+/BmJGjWn1eURRhtog2K5dYLCK+2JuKwpIKWJp4m5wQF4z4btX1Hg1lFdh+Ogf+Xh6VSaLUgqh0l2PjiQw8/OWhhtINLJ892HGSwK4mKxE4vRGACAy7r7pGZ711ALsBk19ruA6gKAJ/fSStQlOThzcQ2BsIigV8I5gMOih+fnfhBHDZsmX44YcfcOrUKXh6emLUqFF4/fXXERsbaz2mrKwMTzzxBFavXg2j0YhJkybhgw8+QHBw88aF8AVELWIySnUBvfyBPtcB8rotTtQ0s8mE5D3roElaBXelFwIj+0N25WMdtmqD2SJCV1qjtbC4HAUlUgtiaeWydrcOC7dpAbyQW4wfD1+ucy5PDzmW/3EWutL6ZjRLSWCIRok/n77GsbqDjUXSOteBTrrKSkWpNAwjK7F6X+0vYS1dCcRiATKPAbmngYIL9c+UdveUxlJ2GwKoQ9rrt6F2wM/vLtwFvH37dsyfPx/Dhg2DyWTCc889h4kTJyIxMRHe3tKA5scffxzr1q3Dt99+C41GgwULFuDGG2/Erl277Bw9dUluCmDgnYCbkuOFWitxLeQbn0ZczZaajD+kxKSDVmyQywT4V84arq203IyCknIEqhQ2+wsqJ57UdibL0GDyB0idkBm6MuxLycfIaAcZInDyJyDnFAABGP2o860Kkp8CnFoHGGuUB9L2BXpPsj1OJgeirmr+eWUyIGygdKkok1r3c05Jj2epfI4rSoGMY1KLIBNAcjBdtgWwtpycHGi1Wmzfvh1jxoyBTqdDUFAQVq1ahZtvvhkAcOrUKfTt2xd79uzBFVdc0eQ5+Q2C2oXZBMi77Hex9lO1YkO9RXsF4NbPHWbZLktlq6HUUliOguIK5JeU488zuVh7tLG6c5L/3D4QMwZ2a/K4TnF2K3Bxv7Qddz0Q3M++8TSXuUIqvXTpQPU+N4WU+HXk72AqB/LPSaWfqoZ8jF5k+z+ekwxknQSC+kjjC90U9Z6KOg4/v7twC2BtOp0OAODv7w8AOHjwICoqKjB+/HjrMX369EGPHj0aTACNRiOMxurB4Hq9voOjpi6vJB84/p3U3RQSb+9oHFejKzZU2vgM0Gdah3UFt4RMJsDP2wN+tVoNw/28mpUAnssuss4OtruAmOoEMO+scySA+gzg1K/SxI0qfpHS60PZwR/2bh5SC6O2r5SEFufW/YKXdVJKAnOSAZkb4B8ltRIG9nK+FlZyWi6RAFosFixatAijR49GfLz0IZuZmQkPDw/4+vraHBscHIzMzMx6z7Ns2TK8+OKLHR0uuYoyvVQouqIUSF4vfTD59rB3VI6pJSs2tKQbr5PVV86mNpVCSmC/2puGnkHeGBEVgBCNHRPBqlVBTMbKVUEsjj2xwWKWJlxVdfnK3KQC7N2GdP7QC7k74FNrQo/FIo2ntF43AblnpIsgkwpuB8VKCWFXrb9IDsGB/4vbz/z583HixAmsXr26Ted59tlnodPprJeLFy82fSeihijUUhcQUPmh9YPUIkh16etOqKhXc1d2sJOqcjZA9azfKlXXJ/ULsS6Fdz6nGF/vS8NPhy8jvbC08wKtqWpVEKByVRAHf9+TyYHek6VtdTAw9B6g+1DHGXcrkwEjHgIG3iElpQpV9W2iRRpDmLwR2P2uNH6QqIN0+RbABQsW4Ndff8WOHTvQvXt36/6QkBCUl5ejsLDQphUwKysLISH1D9ZVKBRQKDhWg9qJIAC9JgBlhdKbfkWptGrI4LvZDVRFFIHM41KtteZo7soOdjQ5PhTLZw+uUwcwpLIO4Pi+wTiRrseBC/kwVK5/nJJbjJTcYvTw98KInv7o7ufVuUEHxEjLwgFSN7AjLQsnilIrWs1Z9YExQPxN0vg6BxgSUIdMJnVJ+0VK7wH6y9IEkpzTQJk0XAmiKBWprslYBFgq6tYjJGqFLpsAiqKIhQsX4scff8Qff/yBqKgom9uHDBkCd3d3/Pbbb7jpppsAAMnJyUhLS8PIkSPtETK5IpkciJsJHP5CGitUki+1BCbc7pgfXJ2pJB84vUkqseEdJNVsq/pwrEOQPiwjWl8LsDNNjg/FhLiQBlcCGRjui/gwHyRm6LH/QgH0pdJayGn5JUjLL8Go6ACM6NmJs4T9e0pfWERR6gaOubbzHrsxxiIgeYMUW/xNtq18Qb3tF1dLCAKg6S5doq+Vuodzk4GinLprD2ccAVJ2Si2bgbFSD0LtgvItLWdDLqvLJoDz58/HqlWr8PPPP0OtVlvH9Wk0Gnh6ekKj0WDevHn429/+Bn9/f/j4+GDhwoUYOXJks2YAE7UbdyXQ/xbg0GdAeQlQmCZ9qPWZ5jjdVp3JYgYu7gMu/FldTkOQAYPuBva8C1jXAqlS+Tea/JpTfdDJZUKjpV7c5DIM6O6LfmEaJGXosS8lH7rSCggCEKNVNXi/DuHhJRVG1l0CSvKk5NzLv3NjqC0nWfo/qajsGs885vzrbAuCNGaw9rjBKjmnpJ+GLOmSskNKEoMqk8HUPcCmZ2oVtA6TVjtxkBny5Di6bBkYoYEPzhUrVmDu3LkAqgtBf/311zaFoBvqAq6N08ipXekuA0dWVSc9Pcc6TYtWu9FnSBNiirKr9yl9pDFdAdGtW7Ghi7BYRJzKNCCv2IiregXZ3HYupwgCgKhA7wbf+9osbS9wbpu0HXMtED68Yx6nKRVlUmmazOPV+zy8pS9MAdH2iakzWCzAxb+kJNBQz0TFnGQg8cd67lj5enCgMkmOgJ/fXTgB7Ax8AVG7y06SCu9WGXyX1DXU1ZnKgQs7KtdLrnxLEgSg21AgaoxUWqMKu7hsmC0iPtt9AbrSCmh9FBgRFYDooA5IBItzgf2fSq/HboOlMiedrTANSPrVdihAUG/pC4IrzZgtLZRWIMk5JX1xFC3A3g+A8qIG7lA5RGLRcZf+X6mJn99duAuYyClp+0pri57fLpUz8XGQYsAd7dQv0gD4KqogIHZq3UHwQMtXbOjizucUQVc5RjBbb8QvR9MRqFZgRJQ/emlV7ZcIegXYbyUQs0n6gnBxX/UXBDcPIGYCENLf9YZKePpKLbDhw6VyN0fXNJL8Ac5SJok6FxNAIkfTY6S0iLzGRZI/AIgYDeSelcb6RV4pfbCxpaJZYrQqTE8Iw18pecjWS4Xqcw1GrDuWgQCVB4ZH+aO3Vg1ZW9cWFgT7JH8VZcCRL6VJEVV8w6X1tD19Oz8eR6NQN//vkLJdKonDKgMEJoBEjkcQunbyJ4pARYltl506BIidLBUdtvfkAicjCAJitCpEB3kjJbcYf6XkI7OyvExeUTk2HM/EXq88XBEdgD4hTtjV5aaQWh+Lcipbf8cC3Yc5djHqztbc8ke6S1JXcbchQPfh0uQecllMAImcQcEFIP0w0Pd6524ZqyrtUlEMDLnH9ndx9hmcdiYIAnoGqRAV6I20/BLsPZ+H9EIpESwoqcDF/NL2SwBFETBkAKqQjk/EBEEa42cql1b0UGk79vGcUcQoabiEPgMNLpeoUEtfsEzl0mzhSweAsEFA+AjbYtTkMpgAEjm6zBPAqXXSQG83T2kxe2cb82QxA5f2Axd2SuO5AGk8VwRrbrY3QRAQEeCNHv5euFRQak0Eh0fatqyazBYAUrmZFkk/LNWiKy8GBs1q3+ULqwp/uylt6/i5ewIJt7Xf43Q1MrlU6uWbu9FgmaQpb0gFpDOPSf+P5grpf/DyISBsoJQIdvQ6yeRQmAASOTqlT2URXkgfvl7+9ivB0RoNlXZhS06HEgQB4f5eCPf3QkFxOTRe7ja3H7+sw8HUAgyN9Ed8mE/zE0GZm5T8AdKqIO2VAJYXA6c3SpOB3D2lFi22TDVf3PVSqZc6ZZLCbMskRYySysmkH5FKTllMUmtgfgow/H7n+3JJrcYyMG3AaeTUaTJPAEm/SNuCAPS70fFXOmhJaRfqVCazBSt2XUCRUWqNVSncMDjCDwO6a+DeVCJYXgLsfkd6Tr0CgBEPtD2g3LPSl4SqxBKQlkjrPrTt53Y1zS2TZDRUJoKHpVb52MlSl7CL4Oc3WwCJnENIvFQe5sKf0gdv0s+AYnbDKwbYW945aaxfzXptjZV2oU5lNFmg9VGgKEdKAIuMJuw4nYMDF/IxJMIPA7r7wsOtgUSwPVcFMZUD536TWqOquHsCsVOk1S2o5ZpbJkmhBmLGS1UH0g8Dwf1tby/OA9J2Az1G1V1ujroEtgC2Ab9BUKcSRSBpLZCVKF1XqIDBd0tr5DqSMh2w90NpzCIgdRmytItDytaX4a+UfJzNtq0h5+khx+AefkgI10DhVs9zlroHOP+HtB0zHggf1vIH112SijqXFlTvC4iRkj92/dpf4log66TUah/URyrVpApq+n5Ogp/fAOfREzkLQQBip1WvDGIsAo5/B5iM9o2rNqWmeoyiXwQwbJ402YPJn8PR+igxPSEMs6+IQGyI2jr8q7TcjF1nc/H57lTrZBEbgb2qt/POtOxBLWap0PnhL6uTP7mb1AXZ/2Ymf47AVA4UpEjboiitULT/E+DE9/UvQ0dOiQkgkTORuwHxN1YXfi3KlsYG2rMhv7RA+lCvKfJKadB5wh2s6+cEgtQKTO0firuuiEDf0OpEMCZYVf/kEK+A6tdg4UWpWHNzVZQA6YeqX7M+YcDQedL4M05AcAxuHsCIh6SyOzVrBeacBg6sAI59Ky1BR06NXcBtwCZkspviPODQZ4BolmoD2mO8lMUslZG48CcQOVoabE5dQkFxOQ6kFuCKnv5QK6tnDxtNZhy8UICBPXzhlboNuHQAFlHEMc3VSJX3gFatxPAof8ibWnUkO0nqYoy8UhqD5gBFnc0WEftS8pFtKGv+7+EKzBXSGM2Le6Veh5r8o4C+051yHWZ+fnMSCJFz8g4A4m8C5B72mQhSu7TLhV3SOCG29nUJft4emBBXd3WJY5d0+CslH4fSCjDM1x+aTD3+PJuL/aVGbLJI3f6hGiWWTI/D5PjK12WZXnqduiurT6TtK63+4unXGb9OkzaeyMCLvyQiQ1fdklnn93BVcndpjGfYICDzKJC2V3pOAemnG5eVc1b2/9pFRK3jF9H5yZ+pHDi7VWp9rEr+BEH6cPDg2K2uzGIRcfRiIQCgwizii0Qz1p7IQ3aZHKVQWI/L1JXh4S8PYeOJDGkSwf5PgDOb657QgZK/h788ZJP8AbV+D5KGn3QbInUNx06WhgBE1NN6q8+w75AUajZ2AbcBm5DJ4Vw+JJXoUDdzbdCWqLe0i1aatcnSLi7BUFaBA6kFOHZRh493noObsRA6eKGbkIcQFMAAT5wWu8MDJtykOoFXRskhEwQUlJTjdOB4lGgqJ4/U6Fmt2cmq8XTHoB62ieHhtAJrvULp+Op71Bwy2KOy6HWVCrMFBy4U1Hts1aYoAvNXHUJecXmDv7NWrcCGx66Ct8INHnIZZA7eLdxpXdkWMwDBNgE0ZAEH/id9MY0YLc3qdtBxnfz8ZhcwUddgsQDnfpeWW1OogSFzpJ/tobxYavWrKj8DsLSLi1Ir3XF1rBYQgSKjGTFCIW6U7YQapdZjrsMeBAsFSC8LxOXC7gj380KWZzT+KlDDrC9s9PzdfD3rJIBJGQZk6ZueZCKXCTYJoNkiYu/5vEbvc6mgpNHkDwCyDUb8Y30SuvtJ5/Zwk0HhJoPCXQ5PdzluHtLd5viL+SXQlVZA6S6Dwk0uHesmh8Jdup/QgQlRp3Zl1/d/n7pL+qnPkCoUqLRSIhgU67CJoCtjAkjUFYhmQH9J2jYapDffgbPaZ7WNy4dskz+/SGk9Yo73c1n6sgpMku3Dv9yW45DYC8fFaJhEGXoI2QgR8jFMdgobLMNhMEUAcTOQb9DCnJLfoTG1Jr0ortGy2Nzjyk0WlJssMJSZ4OlRNwk6ma5HUoa+/hiFqgRSjl5aFcb0tq2rdyitAAJgkzDW3PaQN5xAVnVl1+7Sq+rKXj57cMePZ9TGSVUBqoaHFGUDJ38EvAMrE8E+DjHhhyRMAIm6Ark7EH8zcOhzqYvWkCkVjY6/qe3fvHtcIY3lMpUC0dcCIf35bd7Fab3dscT9c3jBiDGyE7hCTEQ53K1JmAwirpSdxJn45UBwNPprKhCtlcaI1hx0JNZIV0QR9a4+Mr6vFuVmS73Dymru03jarnXsLpfhhkHdbBKimiOeRABH0ryw8WRWk79vn1AfRAZ4w2gyw2iywFhhgdFkhqKeeI0mcz1nqI7XWFF1/7r1Ffecy0N5PfurCIKUHE6IC0aMtnrMbUFxOZ774Xid5A+Qfk8BwIu/JGJCXEjHzmzW9pFa+/LOStUBqmoGFucCiT8DXn9K4wa1/ZgIOgAmgERdhUIF9L8FOPy5NFkj94zULRxzbfPPYTEDhozqYtOAlFz2u0E6vxOWe6D2N1x+CnKhukXPQ7DAA7YFyf1QhCHKywCioVa625STaQmtj7Lpg+ohlwmIDGz89RoZ4I03NycjU1dWb/IkAAjRKLHwml71Jk5mS917De7hh6hAb5tE0WiyoKyiKnmUfnrVaj20WMRGkz9ASiDLKsyoHcqO0znIL6lo+H4AMnRl2HwyE5PjQzq0GxqCIBUKD4gB8s9L3cJVNQNL8qTVX9y9gIDojouBmoUJIFFXogoC4mZKXcCiRarT5+XfvEXe9elA8gZpbddh82y7eDtiUgk5LXlxdrseZy9ymYAl0+Pw8JeHIAA2SWBVirRkelyDrWb17Q+vNRmlJWYO6gajyYyyiupEUbqYra2GRpMZXh62H92ZzRgjCQA/H03HNX21Nsv7iaLYMQmhIEhJnn9PoDBVKhVVmCaNC/Tv2f6PRy3GBJCoqwmIBnpNkGbsAsDpzYDSVyraWh9TOZCyA7h8oLpP7fRGYOCdnRIuOSFVM78QNPc4O5ocH4rlswfXmTwR0sl1AGUyAVFNtFg2pE9I82axhmmUddZ23pKYhSyDEWEaJUI1ngjzVULj6d5+SaEgSOOG/SKlBLBqX01JvwCqECBsoNTjAEi9Eam7gaIs6XUUMYoTztoZE0CirqjbYKA0H7i4X2oJPPmjNCkk97TtG2rBBSnZK6sxaF2lBXqOs1fk5AwiRkmlf/QZQEOdpz5hTrM6zOT4UEyIC3HalUCu7BWIUI2ywa5sAAjw9sD1A7vV2Z9eWIqCkgrkGow4dkkq8eTpIUeoRokwX0+EapQI9lHCvb4lAVvKt0fdfbrLQOYJACeAtN1A+AhAlw5seV7qlajiEwZMfl1aYpLaBesAtgHrCJFDs1iAkz9IYwEL04Bzv0nJXxVPPyBqXPUycjI3IOoqoPswftOmpiWuBb65u/JKPZ2nt37OD+tOVDULGKi/K7u+WcAmswVrDlxEjsHYaO1mmSBgfJwW/cI07Rs0ILXynd9efT0nGUj8sZ4D2/d1xc9vJoBtwhcQOTxTObDtVWDX2w0fE3eDVNaFpV2opRLXAhufrtVS0w2Y/BqTPztobR3AcpMFWfoypBeWIkNXhgxdGcoqbGcz3zosHN18q5d9yzaUYX9KAUJ9lQjTeCJIrWh9i2lRtjRZJDsJ2PM+UF7UwIGVLcuLjrf5Syo/v9kFTNS1yeTA8W8aPyb1T+CmT6SlnohaIu56oM80jtVyEK3tyvZwk9lMXhFFEQUlFdaEMFNfBq1aYXOfSwWlOJ1lwOksAwDAXS5A6yMlg6G+SoRqlHUmqzRIpZUqDcjcG0n+AEAE9Jel11vUVc07NzWI7/hEXVnqbtvWmfoU5wBpe/iGSq0jk/O140DkMgEjowPadA5BEODv7QF/bw/Ed6u/2ze71szjCrOIywWluFxQvSqMr5c7YrQqXNUrqPbd62dq3mxmm6Es1GpMAIm6sua+UfINlYhaYGJcCAZH+CGjsAwZulKkF5ZBV2pbi7CwpAKGsrqrrRy7VAiNpzuCfZRQutdoLe5Cs8udARNAoq6Mb6hE1AFkMgFatRJatRIJ4b4ApCXzqpLBDF0psvRGhGpsC3mXmyzYdioHFlGUSgWqFAjTKBGiUSIsaCgUyhAoSzPrFLsGAIsIGL1C4Okks8sdHRNAoq6si5XrICLH5a1wQ4xWjRitGoA0y7j2YilZ+jJYKueeiiKQazBaS9CIoojtJU/iHsv3CBXyMUCWAk+hHACs53mx4m68Chk4yrTtuBgfUVcmk0u1swBUF4SA7fXJr3HQPhG1Oze5rM76zoEqBab0D8HAcF9ofRSQ1SgKfbmwFEfLw/Cu6UZssgyzuV8mAvBwxSKsLhqIfSn5oLZjCyBRVxd3vVQ7q065jjCW6yCiTuXpIUefEB/r6iVVJWgydGX45aj0/nQO3XDeHIpEMRJaFCIbvthn6QNLZZtVtqGZk0WoUUwAiVwBy3UQkQOqWYLGbBHxxd5UAIAIGfZa4uq9j1atrHc/tQwTQCJXwXIdROTAhkf5N7qknQBpjebhUSxY3x44BpCIiIjsTi4TsGS61OrXwIhlLJke5zRrNDs6JoBERETkECbHh2L57MEIqVU+JkSjrHc9Y2o9dgETERGRw2jtknbUMkwAiYiIyKG0x5J21Dh2ARMRERG5GCaARERERC6GCSARERGRi2ECSERERORimAASERERuRgmgEREREQuhgkgERERkYthAkhERETkYpgAEhEREbkYrgTSBqIoAgD0er2dIyEiIqLmqvrcrvocd0VMANvAYDAAAMLDw+0cCREREbWUwWCARqOxdxh2IYiunP62kcViQXp6OtRqNQShfRep1uv1CA8Px8WLF+Hj49Ou56aW4/PhWPh8OBY+H46Fz0fTRFGEwWBAWFgYZDLXHA3HFsA2kMlk6N69e4c+ho+PD/+BHQifD8fC58Ox8PlwLHw+GueqLX9VXDPtJSIiInJhTACJiIiIXAwTQAelUCiwZMkSKBQKe4dC4PPhaPh8OBY+H46Fzwc1ByeBEBEREbkYtgASERERuRgmgEREREQuhgkgERERkYthAkhERETkYpgAOqD3338fkZGRUCqVGDFiBPbt22fvkFzSsmXLMGzYMKjVami1WsycORPJycn2DosqvfbaaxAEAYsWLbJ3KC7t8uXLmD17NgICAuDp6Yn+/fvjwIED9g7LJZnNZrzwwguIioqCp6cnoqOj8fLLL7v0erfUMCaADmbNmjX429/+hiVLluDQoUNISEjApEmTkJ2dbe/QXM727dsxf/587N27F1u2bEFFRQUmTpyI4uJie4fm8vbv34+PPvoIAwYMsHcoLq2goACjR4+Gu7s7NmzYgMTERPzrX/+Cn5+fvUNzSa+//jqWL1+O9957D0lJSXj99dfxz3/+E++++669QyMHxDIwDmbEiBEYNmwY3nvvPQDSesPh4eFYuHAhnnnmGTtH59pycnKg1Wqxfft2jBkzxt7huKyioiIMHjwYH3zwAV555RUMHDgQb7/9tr3DcknPPPMMdu3ahZ07d9o7FAJw3XXXITg4GJ9++ql130033QRPT098+eWXdoyMHBFbAB1IeXk5Dh48iPHjx1v3yWQyjB8/Hnv27LFjZAQAOp0OAODv72/nSFzb/PnzMW3aNJv/E7KPtWvXYujQobjlllug1WoxaNAgfPzxx/YOy2WNGjUKv/32G06fPg0AOHr0KP78809MmTLFzpGRI3KzdwBULTc3F2azGcHBwTb7g4ODcerUKTtFRYDUErto0SKMHj0a8fHx9g7HZa1evRqHDh3C/v377R0KATh//jyWL///9u4mJKo1AOP4M05p42BhSjkSlVGYSt9TYraoXKRBUBhiDDEWJJaaJQX2YdlCWwQWIUwI5UqLLCwxKsoWkhBKNKZgH6sIYrRokRq5cM5dBAOD3eje6/UMnv8PDsx53xl9Xlz4zJnXo0+VlZU6ffq0ent7dfToUUVHR8vr9Zodz3Kqqqr07ds3rVy5Una7XRMTE6qtrZXH4zE7GiIQBRD4A6WlpRoYGNDz58/NjmJZHz9+VEVFhZ48eaI5c+aYHQf6+cbI7Xarrq5OkrRu3ToNDAzo2rVrFEAT3L59W83NzWppaVFGRob8fr+OHTum5ORkfh6YhAIYQRITE2W32zU0NBQ2PjQ0pKSkJJNSoaysTB0dHerq6tKiRYvMjmNZL1++1PDwsNavXx8am5iYUFdXlxoaGjQ+Pi673W5iQutxuVxKT08PG0tLS9Pdu3dNSmRtJ0+eVFVVlQoLCyVJq1at0ocPH3Tx4kUKICZhD2AEiY6O1oYNG9TZ2RkaCwaD6uzsVFZWlonJrMkwDJWVlamtrU3Pnj1TSkqK2ZEsLScnR/39/fL7/aHD7XbL4/HI7/dT/kyQnZ096dZI796905IlS0xKZG3fv39XVFT4r3W73a5gMGhSIkQyrgBGmMrKSnm9Xrndbm3atElXrlzR2NiYDhw4YHY0yyktLVVLS4vu37+vuLg4BQIBSdK8efPkcDhMTmc9cXFxk/ZfOp1OJSQksC/TJMePH9fmzZtVV1engoIC9fT0qLGxUY2NjWZHs6Rdu3aptrZWixcvVkZGhl69eqX6+nodPHjQ7GiIQNwGJgI1NDTo0qVLCgQCWrt2ra5evarMzEyzY1mOzWb75XhTU5OKioqmNwx+aevWrdwGxmQdHR06deqU3r9/r5SUFFVWVurQoUNmx7KkkZERVVdXq62tTcPDw0pOTta+fft07tw5RUdHmx0PEYYCCAAAYDHsAQQAALAYCiAAAIDFUAABAAAshgIIAABgMRRAAAAAi6EAAgAAWAwFEAAAwGIogAAwhWw2m+7du2d2DAD4LQoggBmjqKhINptt0pGbm2t2NACIKPwvYAAzSm5urpqamsLGYmJiTEoDAJGJK4AAZpSYmBglJSWFHfHx8ZJ+fjzr8/mUl5cnh8OhZcuW6c6dO2Gv7+/v1/bt2+VwOJSQkKDi4mKNjo6GPefGjRvKyMhQTEyMXC6XysrKwua/fPmiPXv2KDY2VitWrFB7e/v/u2gA+IcogAAspbq6Wvn5+err65PH41FhYaEGBwclSWNjY9qxY4fi4+PV29ur1tZWPX36NKzg+Xw+lZaWqri4WP39/Wpvb9fy5cvDvseFCxdUUFCg169fa+fOnfJ4PPr69eu0rhMAfssAgBnC6/UadrvdcDqdYUdtba1hGIYhySgpKQl7TWZmpnH48GHDMAyjsbHRiI+PN0ZHR0PzDx48MKKiooxAIGAYhmEkJycbZ86c+dsMkoyzZ8+GzkdHRw1JxsOHD6dsnQDwX7EHEMCMsm3bNvl8vrCx+fPnhx5nZWWFzWVlZcnv90uSBgcHtWbNGjmdztB8dna2gsGg3r59K5vNpk+fPiknJ+e3GVavXh167HQ6NXfuXA0PD//bJQHAlKMAAphRnE7npI9kp4rD4fij582ePTvs3GazKRgM/h+RAOBfYQ8gAEt58eLFpPO0tDRJUlpamvr6+jQ2Nhaa7+7uVlRUlFJTUxUXF6elS5eqs7NzWjMDwFTjCiCAGWV8fFyBQCBsbNasWUpMTJQktba2yu12a8uWLWpublZPT4+uX78uSfJ4PDp//ry8Xq9qamr0+fNnlZeXa//+/Vq4cKEkqaamRiUlJVqwYIHy8vI0MjKi7u5ulZeXT+9CAeA/oAACmFEePXokl8sVNpaamqo3b95I+vkXurdu3dKRI0fkcrl08+ZNpaenS5JiY2P1+PFjVVRUaOPGjYqNjVV+fr7q6+tDX8vr9erHjx+6fPmyTpw4ocTERO3du3f6FggAU8BmGIZhdggAmA42m01tbW3avXu32VEAwFTsAQQAALAYCiAAAIDFsAcQgGWw4wUAfuIKIAAAgMVQAAEAACyGAggAAGAxFEAAAACLoQACAABYDAUQAADAYiiAAAAAFkMBBAAAsBgKIAAAgMX8BQvGRmLaIuVGAAAAAElFTkSuQmCC"}}},{"cell_type":"markdown","source":"## Load Kaggle-API\n\nThe key for the API token has been replaced with dummy values because it is supposed to be kept secret.\n\n```python\n## API Token\napi_token = {\"username\":\"anthonyylee\", \n             \"key\":\"abcd\"}  # The real token key is hidden~~ It is supposed to be a secret, so I can't tell you~\n\napi_token_path = Path(\"/\", \"root\", \".kaggle\")\napi_token_file = Path(api_token_path, \"kaggle.json\")\nos.makedirs(api_token_path, exist_ok=True)\n\nprint(f\"api_token_path: {api_token_path}\")\n\n## Write api token file\nwith open(api_token_file, \"w\") as file: \n    json.dump(api_token, file)\n\n## Change permission and check permission\nos.chmod(api_token_file, 0o600)\nprint( f\"Is the token file 600 : { oct(os.stat(api_token_file).st_mode)[-3:] == '600' }\" )\n\n```","metadata":{}},{"cell_type":"markdown","source":"## Download the output from pervious kernel version\n\nThe Kaggle API only supports downloading the latest kernel version's output, thus, the actual results may differ slightly as new versions are continuously added.\n\n```python\n## Download previous output files to working directory\n!kaggle kernels output anthonyylee/cnn-histopathologic-classification-using-pytorch -p /kaggle/working/previous_kernel_output\n```","metadata":{}},{"cell_type":"markdown","source":"## Load model state and make inference on the competition test dataset\n\nThe best performing model state is loaded and used to make inference on the competition test dataset for submission.\n\n```python\n## Load model 1a and make inference using the CNN model\n\n## Load the model state from output file\nbatch_size = 50\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\nloaded_checkpoint = torch.load(\n    \"/kaggle/working/previous_kernel_output/1a/model_checkpoint/checkpoint.pt\",  \n   map_location=torch.device(device)\n)\n\n## IGNORE - checks\nassert isinstance(loaded_checkpoint, dict) == True, \"Checkpoint loading has some king of error\"\n\n\n## Create model and load the model state\ncnn_model = CustomCNN()\ncnn_model.load_state_dict(loaded_checkpoint[\"model_state_dict\"])\ncnn_model.to(device)\n\n## Fetch the optimizer (ADAM) and load the state\nmodel_optimizer = cnn_model.get_optimizer()\nmodel_optimizer.load_state_dict(loaded_checkpoint[\"optimizer_state_dict\"])\n\n## Inference with model 1a and save results\n\ntest_dataloader = torch.utils.data.DataLoader(  # Create a dataloader to make things easier\n    dataset = DatasetWrapper_Test(),  # The competition test dataset\n    batch_size = batch_size, \n    shuffle = False, \n    num_workers = 10, \n    prefetch_factor = 5000,\n    drop_last = False,\n)\n\nfile_id_holder = []  # Test set is small enough thus holding output in memory\nresults_holder = np.zeros(shape=(len(test_dataloader), batch_size) )   # Test set is small enough thus holding output in memory\nfor idx in tqdm(range( len(test_dataloader) ), desc=\"Inferencing test set\"):\n    with torch.no_grad():  # No gradient mode\n        \n        if idx == 0:  # Create an dataloader iterator\n            dataloader_iter = iter(test_dataloader)\n            \n        file_ids, images = next(dataloader_iter)  # Load the next iterator output\n        assert len(file_ids) == len(images), \"File IDs and images don't have the same number of records.\"\n        images = images.to(device)  # Transfer to GPU for faster inferencing\n        \n        ## Model inference\n        output = cnn_model(images)\n        max_value, max_idx = torch.max(output, dim=1)\n        predictions = max_idx\n        \n        ## Append results to holder\n        predictions = predictions.cpu().numpy()  # Need to copy to CPU first\n        if len(predictions) != batch_size:  # The last batch typically is not a full-sized batch\n            predictions = np.pad(predictions, (0, batch_size - len(predictions)), \"constant\", constant_values=(0))\n        results_holder[idx, :] = predictions\n        file_id_holder.append(file_ids)\n```","metadata":{}},{"cell_type":"markdown","source":"## Clean up the inference output for submission\n\n```python\n## Reshape the 2D array to an 1D array\nfile_id_holder_new = list(chain(*file_id_holder))  # Convert list of lists into just list\nresults_holder_new = results_holder.reshape((-1))[:len(file_id_holder_new)].astype(int)  # Remove the extra padded elements\n\n## IGNORE: Check\nassert ( len(results_holder_new) == len(file_id_holder_new)), \"The lengths are not the same, something is off.\"\nprint(len(results_holder_new))\nprint(len(file_id_holder_new))\n\n\n\n## Save the data in CSV for submission\ndf = pd.DataFrame({\"id\":file_id_holder_new, \"label\":results_holder_new})\ndf.to_csv(\"/kaggle/working/submission.csv\", index=False)\n\n\n## IGNORE: Check\nread_from_csv = pd.read_csv(\"/kaggle/working/submission.csv\")\nread_from_csv.head()\n# !cat /kaggle/working/submission.csv\n```","metadata":{}},{"cell_type":"code","source":"## Submit the results to Kaggle\n!kaggle competitions submit -c histopathologic-cancer-detection -f /kaggle/working/submission.csv -m \"AYL_Submission\"\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Final words + Future Improvements\n\nThe objective for this notebook for myself is to test the PyTorch framework as it is a framework that I have not utilized before, thus this is my first foray into PyTorch. Compared to TensorFlow, it is a different way of thinking on how to structure my model thus it took a lot of upfront investment in understanding the differences between the two.\n\nSecond objective is to build my first convolutional neural network. This objective is also accomplished but not without many setbacks in trying to figure out a more efficient way to doing certain task. \n\nDuring this process, I found two principle that was helpful and aim to carry to my next project. First, aim for good-enough and then revisit the code to refactor. Second, if certain task or variable would be used more than twice, create a function as this prevents errors from copying code over and over.\n\nLastly, I would like to take note of the following improvements that can be made and should be made were I to revisit this project: \n\n- Get RayTune working to systematically tune the parameters + Learning to leverage multiple GPUs/TPUs\n- Create a larger artificial dataset by rotating / mirroring images\n- Train and test on the entire PCam dataset\n- Properly profile and document pipeline bottlenecks\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"outputs":[],"execution_count":null}]}