{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"accelerator":"GPU","colab":{"gpuType":"T4","machine_shape":"hm","provenance":[]},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceType":"competition","sourceId":80670,"databundleVersionId":8695879}],"dockerImageVersionId":30839,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"vscode":{"interpreter":{"hash":"b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"}}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# HW1: Frame-Level Speech Recognition","metadata":{"id":"F9ERgBpbcMmB"}},{"cell_type":"markdown","source":"In this homework, you will be working with MFCC data consisting of 28 features at each time step/frame. Your model should be able to recognize the phoneme occured in that frame.","metadata":{"id":"CLkH6GMGcWcE"}},{"cell_type":"markdown","source":"# Dataset Description\n\nLet's start by understanding the dataset for this homework.\n\nOur data consists of 3 folders (train-clean-100, dev-clean and test-clean). The training and validation datasets (train-clean-100 and dev-clean) each contain 2 subfolders (mfcc and transcript). The 'mfcc' subfolder contains mel spectrograms (explained below and in writeup), while the 'transcript' subfolder contains their corresponding transcripts. However, the test dataset (test-clean) contains only the 'mfcc' subfolder without the corresponding transcripts, which will later be predicted by your model.\n\n\n## 1. Audio Representation.\nThe 'mfcc' subfolders contain many `*.npy` files of mel spectrograms. .npy files are used to store numpy arrays.\n\nEach .npy file represents a short speech recording. For example, one recording might be someone saying, \"This is the age of AI.\" This recording is converted into a mel spectrogram, which is used to represent all forms of audio signals in a computer. Such representation is important in signal and speech processing tasks, especially in machine learning.\n\nCompared to raw audio, mel spectrograms are better for speech processing because they capture both the timing and the frequencies of the sound. At each moment in time, they show which frequencies are present in the sound. This makes it easier for computers to understand and process speech.\n\nWhen converting raw audio to spectrograms, you do not process the whole audio at once. Instead, you process small frames at a time as you stride over the entire audio length. This means that if you have an audio file of 100 seconds, you may decide to process 10 seconds at a time, striding by one second. In this case, the frame size is 10 seconds. The frame size and the number of timesteps (seconds, milliseconds, etc.) depend on individual choice.\n\nWhen processing each frame, you extract a number of features that represent that frame's audio. For instance, in the audio recording of \"This is the age of AI,\" the frame corresponding to \"AI\" will have features that represent how \"AI\" is pronounced, the vocal tract, and the effect of the environment in which it was recorded. For clarity, when we say features, you should think of columns. One feature/column may have information about the gender of the person who made the speech. Another may have information about the age of the person. Another may have information about the environment where the speech was recorded. Basically, the main properties that make up a speech are encoded in those features, which combine in some way to make the audio.\n\nSince we want to recognize the word as it was pronounced despite the environment and other variabilities, we usually normalize to eliminate or minimize such effects.\n\nOur spectrograms contain 28 features. Essentially, the number of features may be different. They may depend on how the raw audio data was converted into mel spectrograms.\n\n## 2. Transcripts\nRemember where we mentioned frames? Well, in our dataset, audio frames have corresponding target transcripts. For instance the abbreviation \"AI\", in our example above, if present in the recordings, will have transcripts: /eɪ aɪ/. This means that you will have two frames one for  /eɪ/ and another for /aɪ/.\n\nThis way of representing pronounciation in text form is called ***phonetic transcription***, \"the conversion of spoken words the way they are pronounced instead of how they are written\"[[link]](https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://krisp.ai/blog/phonetic-transcription/%23:~:text%3Dphonetic%2520transcriptions%2520done.-,What%2520are%2520Phonetic%2520Transcriptions%253F,verbatim%2520to%2520intelligent%2520verbatim%2520transcriptions.&ved=2ahUKEwiV6LO6hrSHAxUKSvEDHcvwAAsQFnoECB0QAw&usg=AOvVaw0VqoWceOzdVwe-AvdyyWqJ). In this case letters 'A' and 'I' are pronounce /eɪ/ and /aɪ/, respectively. Both letters in different words may be pronounced differently.\n\nThe produced representation of the speech is referred to as phonemes. Various .npy files that contain recordings of the sentence **\"This is the age of AI.\"** would map to **\"/ðɪs ɪz ðə eɪdʒ əv eɪ aɪ/.\"** The phonemes representation for **Chelsea sucks** would be **/ˈtʃɛl.si sʌks/**\n\nGoing inside the .npy files. Each .npy file contains vectors which have 28 features/dimensions/columns. The number of vectors in the file corresponds to the number of frames in the recording. And each single frame has a corresponding phoneme in the transcript.\n\nFor instance the .npy file for \"This is the age of AI\" --> \"/ðɪs ɪz ðə eɪdʒ əv eɪ aɪ/\" might have 13 frames (13 vectors):\n\n- /ðɪs/ has 3 phonemes: /ð/, /ɪ/, /s/  \n- /ɪz/ has 2 phonemes: /ɪ/, /z/\n- /ðə/ has 2 phonemes: /ð/, /ə/\n- /eɪdʒ/ has 2 phonemes: /eɪ/, /dʒ/\n- /əv/ has 2 phonemes: /ə/, /v/\n- /eɪ aɪ/ has 2 phonemes: /eɪ/, /aɪ/\n\n**Chelsea sucks** --> **/ˈtʃɛl.si sʌks/** might have 8 frames (8 vectors):\n\n- /ˈtʃɛl.si/ has 4 phonemes: /tʃ/, /ɛ/, /l/, /si/\n- /sʌks/ has 4 phonemes: /s/, /ʌ/, /k/, /s/\n\nNote that recordings of different sentences may have different number of frames.\n\nThe model you will produce must take a vector of a particular frame and predict the frame's transcript as accurately as possible.\n\nTherefore, the **__getitem__** method of your dataset class must return a 28 dimensional vector of a particular frame and its corresponding phoneme transcript.\n\nThis means that, while you are doing your data preprocessing in the **__init__** method, you need stack all vectors from all recordings on top of each other. You must do this for all transcripts as well and remember to ensure the correspondance between frames and their phoneme mapping is maintained.\n\nFor our dataset of two samples above, if you stack the recordings together, you get:\n\n\n| Frame | Feature 1 | Feature 2 | ... | Feature 28 | Phoneme |\n|-------|-----------|-----------|-----|------------|---------|\n| 0     | v0_1      | v0_2      | ... | v0_28      | /ð/     |\n| 1     | v1_1      | v1_2      | ... | v1_28      | /ɪ/     |\n| 2     | v2_1      | v2_2      | ... | v2_28      | /s/     |\n| 3     | v3_1      | v3_2      | ... | v3_28      | /ɪ/     |\n| 4     | v4_1      | v4_2      | ... | v4_28      | /z/     |\n| 5     | v5_1      | v5_2      | ... | v5_28      | /ð/     |\n| 6     | v6_1      | v6_2      | ... | v6_28      | /ə/     |\n| 7     | v7_1      | v7_2      | ... | v7_28      | /eɪ/    |\n| 8     | v8_1      | v8_2      | ... | v8_28      | /dʒ/    |\n| 9     | v9_1      | v9_2      | ... | v9_28      | /ə/     |\n| 10    | v10_1     | v10_2     | ... | v10_28     | /v/     |\n| 11    | v11_1     | v11_2     | ... | v11_28     | /eɪ/    |\n| 12    | v12_1     | v12_2     | ... | v12_28     | /aɪ/    |\n| 13    | v13_1     | v13_2     | ... | v13_28     | /tʃ/    |\n| 14    | v14_1     | v14_2     | ... | v14_28     | /ɛ/     |\n| 15    | v15_1     | v15_2     | ... | v15_28     | /l/     |\n| 16    | v16_1     | v16_2     | ... | v16_28     | /si/     |\n| 17    | v17_1     | v17_2     | ... | v17_28     | /s/     |\n| 18    | v18_1     | v18_2     | ... | v18_28     | /ʌ/     |\n| 19    | v19_1     | v19_2     | ... | v19_28     | /k/     |\n| 20    | v20_1     | v20_2     | ... | v20_28     | /s/     |\n\n\nSo, if you pass index 5 to **__getitem__**, you will get back vector v5 (v5_1, v5_2, ..., v5_28) and transcript **/ð/**. Ideally, if you have a well trained model, it should take v5 and return **/ð/**. And the call to **__len__** would return 21 which the training loop would use to go through the whole dataset.\n\n## Context\n\nIn the dataset we are using, a few millisecs were used to convert raw audio to mel spectrogram and extract the 28 features.\nSince each vector represents only a few millisecs of speech, it may not be sufficient to feed only a single vector into the network at a time. Instead, it may be useful to provide the network with some “context” of size K around each vector in terms of additional vectors from the speech input.\n\nConcretely, a context of size 3 would mean that we provide an input of size (7, 28) to the network - the size 7 can be explained as: the vector to predict the label for, 3 vectors preceding this vector, and 3 vectors following it. It is worth thinking about how you would handle providing context before one of the first K frames of an utterance or after one of the last K frames.\n\nThere are several ways to implement this, but you could try the simplest one:\n- Concatenating all utterances and padding with K 0-valued vectors before and after the resulting matrix\n\nIf you use a context of 3 on the above table, you get the following table:\n\n| Frame | Feature 1 | Feature 2 | ... | Feature 28 | Phoneme | Context Vectors |\n|-------|-----------|-----------|-----|------------|---------|----------------|\n| 0     | v0_1      | v0_2      | ... | v0_28      | /ð/     | [0, 0, 0, ..., 0] (Padding), [0, 0, 0, ..., 0] (Padding), [0, 0, 0, ..., 0] (Padding), v0, v1, v2, v3 |\n| 1     | v1_1      | v1_2      | ... | v1_28      | /ɪ/     | [0, 0, 0, ..., 0] (Padding), [0, 0, 0, ..., 0] (Padding), v0, v1, v2, v3, v4 |\n| 2     | v2_1      | v2_2      | ... | v2_28      | /s/     | [0, 0, 0, ..., 0] (Padding), v0, v1, v2, v3, v4, v5 |\n| 3     | v3_1      | v3_2      | ... | v3_28      | /ɪ/     | v0, v1, v2, v3, v4, v5, v6 |\n| 4     | v4_1      | v4_2      | ... | v4_28      | /z/     | v1, v2, v3, v4, v5, v6, v7 |\n| 5     | v5_1      | v5_2      | ... | v5_28      | /ð/     | v2, v3, v4, v5, v6, v7, v8 |\n| 6     | v6_1      | v6_2      | ... | v6_28      | /ə/     | v3, v4, v5, v6, v7, v8, v9 |\n| 7     | v7_1      | v7_2      | ... | v7_28      | /eɪ/    | v4, v5, v6, v7, v8, v9, v10 |\n| 8     | v8_1      | v8_2      | ... | v8_28      | /dʒ/    | v5, v6, v7, v8, v9, v10, v11 |\n| 9     | v9_1      | v9_2      | ... | v9_28      | /ə/     | v6, v7, v8, v9, v10, v11, v12 |\n| 10    | v10_1     | v10_2     | ... | v10_28     | /v/     | v7, v8, v9, v10, v11, v12, v13 |\n| 11    | v11_1     | v11_2     | ... | v11_28     | /eɪ/    | v8, v9, v10, v11, v12, v13, v14 |\n| 12    | v12_1     | v12_2     | ... | v12_28     | /aɪ/    | v9, v10, v11, v12, v13, v14, v15 |\n| 13    | v13_1     | v13_2     | ... | v13_28     | /tʃ/    | v10, v11, v12, v13, v14, v15, v16 |\n| 14    | v14_1     | v14_2     | ... | v14_28     | /ɛ/     | v11, v12, v13, v14, v15, v16, v17 |\n| 15    | v15_1     | v15_2     | ... | v15_28     | /l/     | v12, v13, v14, v15, v16, v17, v18 |\n| 16    | v16_1     | v16_2     | ... | v16_28     | /s/     | v13, v14, v15, v16, v17, v18, v19 |\n| 17    | v17_1     | v17_2     | ... | v17_28     | /i/     | v14, v15, v16, v17, v18, v19, v20 |\n| 18    | v18_1     | v18_2     | ... | v18_28     | /s/     | v15, v16, v17, v18, v19, v20, v21 |\n| 19    | v19_1     | v19_2     | ... | v19_28     | /ʌ/     | v16, v17, v18, v19, v20, v21, [0, 0, 0, ..., 0] (Padding) |\n| 20    | v20_1     | v20_2     | ... | v20_28     | /k/     | v17, v18, v19, v20, v21, [0, 0, 0, ..., 0] (Padding), [0, 0, 0, ..., 0] (Padding) |\n| 21    | v21_1     | v21_2     | ... | v21_28     | /s/     | v18, v19, v20, v21, [0, 0, 0, ..., 0] (Padding), [0, 0, 0, ..., 0] (Padding), [0, 0, 0, ..., 0] (Padding) |\n\n\nNow, if you want to predict the output of vector v5, you won't just pass vector v5 alone. You will concatenate 3 vectors before it and 3 vectors after, which makes it 7 vectors ([v2, v3, v4, v5, v6, v7, v8 ]) . This needs to be reflected in your **__getitem__** method. Meaning it should return an array of shape (7, 28), in this example.\n\nHence your model is going to be taking a tensor (array) of shape (7, 28) in this example.","metadata":{"id":"P2vp3N7qr_5V"}},{"cell_type":"markdown","source":"# Libraries","metadata":{"id":"z4vZbDmJvMp1"}},{"cell_type":"code","source":"!pip install torchsummary >=1.1.0 wandb --quiet\n","metadata":{"id":"rwYu9sSUnSho","trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:18:43.485839Z","iopub.execute_input":"2026-05-25T04:18:43.486102Z","iopub.status.idle":"2026-05-25T04:18:47.899926Z","shell.execute_reply.started":"2026-05-25T04:18:43.486078Z","shell.execute_reply":"2026-05-25T04:18:47.899021Z"}},"outputs":[],"execution_count":1},{"cell_type":"code","source":"!pip install torchaudio --quiet","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:18:47.900964Z","iopub.execute_input":"2026-05-25T04:18:47.901346Z","iopub.status.idle":"2026-05-25T04:18:51.123879Z","shell.execute_reply.started":"2026-05-25T04:18:47.901309Z","shell.execute_reply":"2026-05-25T04:18:51.122609Z"},"id":"ijYNIOkpYrXf","trusted":true},"outputs":[],"execution_count":2},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport numpy as np\nfrom torchsummaryX import summary\nimport sklearn\nimport gc\nimport zipfile\nimport bisect\nimport pandas as pd\nfrom tqdm.auto import tqdm\nimport os\nimport datetime\nimport wandb\nimport yaml\nimport torchaudio.transforms as tat\nimport torchaudio\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\nprint(\"Device: \", device)","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:18:51.125035Z","iopub.execute_input":"2026-05-25T04:18:51.125322Z","iopub.status.idle":"2026-05-25T04:18:57.642874Z","shell.execute_reply.started":"2026-05-25T04:18:51.1253Z","shell.execute_reply":"2026-05-25T04:18:57.642051Z"},"id":"qI4qfx7tiBZt","trusted":true},"outputs":[{"name":"stdout","text":"Device:  cuda\n","output_type":"stream"}],"execution_count":3},{"cell_type":"code","source":"### PHONEME LIST\nPHONEMES = [\n            '[SIL]',   'AA',    'AE',    'AH',    'AO',    'AW',    'AY',\n            'B',     'CH',    'D',     'DH',    'EH',    'ER',    'EY',\n            'F',     'G',     'HH',    'IH',    'IY',    'JH',    'K',\n            'L',     'M',     'N',     'NG',    'OW',    'OY',    'P',\n            'R',     'S',     'SH',    'T',     'TH',    'UH',    'UW',\n            'V',     'W',     'Y',     'Z',     'ZH' ]","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:18:57.651634Z","iopub.execute_input":"2026-05-25T04:18:57.651889Z","iopub.status.idle":"2026-05-25T04:18:57.668793Z","shell.execute_reply.started":"2026-05-25T04:18:57.651869Z","shell.execute_reply":"2026-05-25T04:18:57.667887Z"},"id":"N-9qE20hmCgQ","trusted":true},"outputs":[],"execution_count":5},{"cell_type":"markdown","source":"# Kaggle","metadata":{"id":"ZIi0Big7vPa9"}},{"cell_type":"markdown","source":"This section contains code that helps you install kaggle's API, creating kaggle.json with you username and API key details. Make sure to input those in the given code to ensure you can download data from the competition successfully.","metadata":{"id":"BBCbeRhixGM7"}},{"cell_type":"code","source":"!pip install --upgrade kaggle==1.6.17 --force-reinstall --no-deps\n\n!mkdir -p /root/.kaggle\n\nwith open(\"/root/.kaggle/kaggle.json\", \"w+\") as f:\n    f.write('{\"username\":\"wardamirza11\",\"key\":\"efb31a8467749576150804dc71867af8\"}')\n\n!chmod 600 /root/.kaggle/kaggle.json","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:18:57.67113Z","iopub.execute_input":"2026-05-25T04:18:57.671325Z","iopub.status.idle":"2026-05-25T04:19:00.959153Z","shell.execute_reply.started":"2026-05-25T04:18:57.671307Z","shell.execute_reply":"2026-05-25T04:19:00.958117Z"},"id":"TPBUd7Cnl-Rx","trusted":true},"outputs":[{"name":"stdout","text":"Collecting kaggle==1.6.17\n  Downloading kaggle-1.6.17.tar.gz (82 kB)\n\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m82.7/82.7 kB\u001b[0m \u001b[31m4.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\nBuilding wheels for collected packages: kaggle\n  Building wheel for kaggle (setup.py) ... \u001b[?25l\u001b[?25hdone\n  Created wheel for kaggle: filename=kaggle-1.6.17-py3-none-any.whl size=105786 sha256=bf022838053ebf76a8320fba0cc3cc61ce2460f9c9695052c074367ded301307\n  Stored in directory: /root/.cache/pip/wheels/9f/af/22/bf406f913dc7506a485e60dce8143741abd0a92a19337d83a3\nSuccessfully built kaggle\nInstalling collected packages: kaggle\n  Attempting uninstall: kaggle\n    Found existing installation: kaggle 1.6.17\n    Uninstalling kaggle-1.6.17:\n      Successfully uninstalled kaggle-1.6.17\nSuccessfully installed kaggle-1.6.17\n","output_type":"stream"}],"execution_count":6},{"cell_type":"markdown","source":"# Parameters Configuration","metadata":{"id":"qNacQ8bpt9nw"}},{"cell_type":"markdown","source":"Storing your parameters and hyperparameters in a single configuration dictionary makes it easier to keep track of them during each experiment. It can also be used with weights and biases to log your parameters for each experiment and keep track of them across multiple experiments.","metadata":{"id":"WE7tsinAuLNy"}},{"cell_type":"code","source":"config = {\n    'Name': 'Warda Mirza', # Write your name here\n    'subset': 1.0, # Subset of dataset to use (1.0 == 100% of data)\n      'context':             30,\n    'archetype':           'diamond',\n    'activations':         'GELU',\n    'learning_rate':       0.0010,\n    'dropout':             0.25,\n    'optimizers':          'AdamW',\n    'scheduler':           'ReduceLROnPlateau',\n    'epochs':               5,\n    'batch_size':          2048,\n    'weight_decay':        0.01,\n    'weight_initialization': None,\n    'augmentations':       'Both',\n    'freq_mask_param':     4,\n    'time_mask_param':     8,\n }","metadata":{"id":"S5gMTwnSnp8K","trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:19:01.918742Z","iopub.execute_input":"2026-05-25T04:19:01.919057Z","iopub.status.idle":"2026-05-25T04:19:01.923706Z","shell.execute_reply.started":"2026-05-25T04:19:01.919035Z","shell.execute_reply":"2026-05-25T04:19:01.92297Z"}},"outputs":[],"execution_count":9},{"cell_type":"code","source":"config","metadata":{"id":"vzeqgWS9pumb","trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:19:01.924355Z","iopub.execute_input":"2026-05-25T04:19:01.924582Z","iopub.status.idle":"2026-05-25T04:19:01.93958Z","shell.execute_reply.started":"2026-05-25T04:19:01.924563Z","shell.execute_reply":"2026-05-25T04:19:01.938757Z"}},"outputs":[{"execution_count":10,"output_type":"execute_result","data":{"text/plain":"{'Name': 'Warda Mirza',\n 'subset': 1.0,\n 'context': 30,\n 'archetype': 'diamond',\n 'activations': 'GELU',\n 'learning_rate': 0.001,\n 'dropout': 0.25,\n 'optimizers': 'AdamW',\n 'scheduler': 'ReduceLROnPlateau',\n 'epochs': 20,\n 'batch_size': 2048,\n 'weight_decay': 0.01,\n 'weight_initialization': None,\n 'augmentations': 'Both',\n 'freq_mask_param': 4,\n 'time_mask_param': 8}"},"metadata":{}}],"execution_count":10},{"cell_type":"markdown","source":"# Dataset Class","metadata":{"id":"FYeyFHQ1yRi4"}},{"cell_type":"markdown","source":"This section covers the dataset/dataloader class for speech data. You will have to spend time writing code to create this class successfully. We have given you a lot of comments guiding you on what code to write at each stage, from top to bottom of the class. Please try and take your time figuring this out, as it will immensely help in creating dataset/dataloader classes for future homeworks.\n\nBefore running the following cells, please take some time to analyse the structure of data. Try loading a single MFCC and its transcipt, print out the shapes and print out the values. Do the transcripts look like phonemes?","metadata":{"id":"2_7QgMbBdgPp"}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nfrom tqdm.auto import tqdm\nimport torchaudio.transforms as tat\nfrom torch.utils.data import Dataset\n\nclass AudioDataset(Dataset):\n\n    def __init__(self, root, phonemes=PHONEMES, context=0, partition=\"train-clean-100\"):\n        self.context = context\n        self.phonemes = phonemes\n        self.subset = config[\"subset\"]\n\n        # Augmentations\n        self.freq_masking = tat.FrequencyMasking(freq_mask_param=config[\"freq_mask_param\"])\n        self.time_masking = tat.TimeMasking(time_mask_param=config[\"time_mask_param\"])\n\n        self.mfccDirectory = os.path.join(root, partition, \"mfcc\")\n        self.transcriptDirectory = os.path.join(root, partition, \"transcript\")\n\n        mfcc_Names = sorted(os.listdir(self.mfccDirectory))\n        transcript_Names = sorted(os.listdir(self.transcriptDirectory))\n\n        subsetSize = int(self.subset * len(mfcc_Names))\n        mfcc_Names = mfcc_Names[:subsetSize]\n        transcript_Names = transcript_Names[:subsetSize]\n\n        assert len(mfcc_Names) == len(transcript_Names)\n\n        self.mfccs = []\n        self.transcripts = []\n\n        phonemeToindex = {p: i for i, p in enumerate(self.phonemes)}\n\n        for i in tqdm(range(len(mfcc_Names))):\n            mfcc = np.load(os.path.join(self.mfccDirectory, mfcc_Names[i]))\n\n            mfcc = (mfcc - np.mean(mfcc, axis=0)) / (np.std(mfcc, axis=0) + 1e-8)\n            mfcc = torch.tensor(mfcc, dtype=torch.float32)\n\n            transcript = np.load(\n                os.path.join(self.transcriptDirectory, transcript_Names[i]),\n                allow_pickle=True\n            )[1:-1]\n\n            transcript = [phonemeToindex[p] for p in transcript]\n            transcript = torch.tensor(transcript, dtype=torch.long)\n\n            self.mfccs.append(mfcc)\n            self.transcripts.append(transcript)\n\n        self.mfccs = torch.cat(self.mfccs, dim=0)\n        self.transcripts = torch.cat(self.transcripts, dim=0)\n\n        self.length = len(self.transcripts)\n\n        self.mfccs = F.pad(\n            self.mfccs,\n            (0, 0, self.context, self.context)\n        )\n\n    def __len__(self):\n        return self.length\n\n    def collate_fn(self, batch):\n        x, y = zip(*batch)\n\n        x = torch.stack(x, dim=0)\n        y = torch.tensor(y, dtype=torch.long)\n\n        if np.random.rand() < 0.70:\n            x = x.transpose(1, 2)\n            x = self.freq_masking(x)\n            x = self.time_masking(x)\n            x = x.transpose(1, 2)\n\n        return x, y\n\n    def __getitem__(self, ind):\n        frames = self.mfccs[ind : ind + 2 * self.context + 1]\n        label = self.transcripts[ind]\n\n        # flatten yahan nahi karna\n        return frames, label\n\nprint(\"Done\")","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:19:01.940571Z","iopub.execute_input":"2026-05-25T04:19:01.940853Z","iopub.status.idle":"2026-05-25T04:19:01.95647Z","shell.execute_reply.started":"2026-05-25T04:19:01.940827Z","shell.execute_reply":"2026-05-25T04:19:01.955538Z"},"id":"HYU4NAH65dSb","trusted":true},"outputs":[{"name":"stdout","text":"Done\n","output_type":"stream"}],"execution_count":11},{"cell_type":"code","source":"#Function to preapare speech data for the model . it load raw MFCC and its label and give input and target \n# to model. it is foe test dataset\nclass AudioTestDataset(Dataset):\n\n    def __init__(self, root, context=0, partition=\"test-clean\"):\n        self.context = context\n\n        self.mfccDirectory = os.path.join(root, partition, \"mfcc\")\n        mfcc_Names = sorted(os.listdir(self.mfccDirectory))\n\n        self.mfccs = []\n\n        for i in tqdm(range(len(mfcc_Names))):\n            mfcc = np.load(os.path.join(self.mfccDirectory, mfcc_Names[i]))\n\n            mfcc = (mfcc - np.mean(mfcc, axis=0)) / (np.std(mfcc, axis=0) + 1e-8)\n            mfcc = torch.tensor(mfcc, dtype=torch.float32)\n\n            self.mfccs.append(mfcc)\n\n        self.mfccs = torch.cat(self.mfccs, dim=0)\n        self.length = len(self.mfccs)\n\n        self.mfccs = F.pad(\n            self.mfccs,\n            (0, 0, self.context, self.context)\n        )\n\n    def __len__(self):\n        return self.length\n\n    def __getitem__(self, ind):\n        frames = self.mfccs[ind : ind + 2 * self.context + 1]\n        frames = frames.flatten()\n        return frames\nprint(\"Done\")","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:19:01.957362Z","iopub.execute_input":"2026-05-25T04:19:01.957655Z","iopub.status.idle":"2026-05-25T04:19:01.972905Z","shell.execute_reply.started":"2026-05-25T04:19:01.957621Z","shell.execute_reply":"2026-05-25T04:19:01.972052Z"},"id":"C0rme6iT5dSb","trusted":true},"outputs":[{"name":"stdout","text":"Done\n","output_type":"stream"}],"execution_count":12},{"cell_type":"markdown","source":"# Create Datasets","metadata":{"id":"2mlwaKlDt_2c"}},{"cell_type":"code","source":"!ls /kaggle/input/11785-hw1p2-f24","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:19:01.973654Z","iopub.execute_input":"2026-05-25T04:19:01.973961Z","iopub.status.idle":"2026-05-25T04:19:02.110682Z","shell.execute_reply.started":"2026-05-25T04:19:01.973941Z","shell.execute_reply":"2026-05-25T04:19:02.109929Z"}},"outputs":[{"name":"stdout","text":"11785-f24-hw1p2\n","output_type":"stream"}],"execution_count":13},{"cell_type":"code","source":"!ls /kaggle/input/11785-hw1p2-fall","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:19:02.111527Z","iopub.execute_input":"2026-05-25T04:19:02.111829Z","iopub.status.idle":"2026-05-25T04:19:02.237992Z","shell.execute_reply.started":"2026-05-25T04:19:02.111789Z","shell.execute_reply":"2026-05-25T04:19:02.237227Z"}},"outputs":[{"name":"stdout","text":"ls: cannot access '/kaggle/input/11785-hw1p2-fall': No such file or directory\n","output_type":"stream"}],"execution_count":14},{"cell_type":"code","source":"ROOT = \"/kaggle/input/competitions/11785-hw1p2-f24/11785-f24-hw1p2\"# Define the root directory of the dataset here!ls /kaggle/input\ntrain_data = AudioDataset(\n    root=ROOT,\n    context=config['context'],\n    partition=\"train-clean-100\"\n)\n\nval_data = AudioDataset(\n    root=ROOT,\n    context=config['context'],\n    partition=\"dev-clean\"\n)\n\ntest_data = AudioTestDataset(\n    root=ROOT,\n    context=config['context'],\n    partition=\"test-clean\"\n)","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:19:02.238861Z","iopub.execute_input":"2026-05-25T04:19:02.239058Z","iopub.status.idle":"2026-05-25T04:27:07.224489Z","shell.execute_reply.started":"2026-05-25T04:19:02.23904Z","shell.execute_reply":"2026-05-25T04:27:07.223568Z"},"id":"gJvMzHhB5dSc","trusted":true},"outputs":[{"output_type":"display_data","data":{"text/plain":"  0%|          | 0/28539 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"41c30b1b58984f559ce29f302d3fa4b9"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"  0%|          | 0/2703 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"c62bb6230ccf4162b3e194e5002206dc"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"  0%|          | 0/2620 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"20afa9c025dd432a9790a7cfe223fa31"}},"metadata":{}}],"execution_count":15},{"cell_type":"code","source":"# Define dataloaders for train, val and test datasets\n# Dataloaders will yield a batch of frames and phonemes of given batch_size at every iteration\n# We shuffle train dataloader but not val & test dataloader. Why?\n\ntrain_loader = torch.utils.data.DataLoader(\n    dataset     = train_data,\n    num_workers = 4,\n    batch_size  = config['batch_size'],\n    pin_memory  = True,\n    shuffle     = True,\n    collate_fn = train_data.collate_fn\n)\n\nval_loader = torch.utils.data.DataLoader(\n    dataset     = val_data,\n    num_workers = 0,\n    batch_size  = config['batch_size'],\n    pin_memory  = True,\n    shuffle     = False\n)\n\ntest_loader = torch.utils.data.DataLoader(\n    dataset     = test_data,\n    num_workers = 2,\n    batch_size  = config['batch_size'], \n    pin_memory  = True,\n    shuffle     = False\n)\n\n\nprint(\"Batch size     : \", config['batch_size'])\nprint(\"Context        : \", config['context'])\nprint(\"Input size     : \", (2*config['context']+1)*28)\nprint(\"Output symbols : \", len(PHONEMES))\n\nprint(\"Train dataset samples = {}, batches = {}\".format(train_data.__len__(), len(train_loader)))\nprint(\"Validation dataset samples = {}, batches = {}\".format(val_data.__len__(), len(val_loader)))\nprint(\"Test dataset samples = {}, batches = {}\".format(test_data.__len__(), len(test_loader)))","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:07.225515Z","iopub.execute_input":"2026-05-25T04:27:07.225838Z","iopub.status.idle":"2026-05-25T04:27:07.235339Z","shell.execute_reply.started":"2026-05-25T04:27:07.225806Z","shell.execute_reply":"2026-05-25T04:27:07.234573Z"},"id":"4mzoYfTKu14s","trusted":true},"outputs":[{"name":"stdout","text":"Batch size     :  2048\nContext        :  30\nInput size     :  1708\nOutput symbols :  40\nTrain dataset samples = 36091157, batches = 17623\nValidation dataset samples = 1928204, batches = 942\nTest dataset samples = 1934138, batches = 945\n","output_type":"stream"}],"execution_count":16},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# Testing code to check if your data loaders are working\nfor i, data in enumerate(train_loader):\n    frames, phoneme = data\n    print(frames.shape, phoneme.shape)\n\n    # Visualize sample mfcc to inspect and verify everything is correctly done, especially augmentations\n    plt.figure(figsize=(10, 6))\n    plt.imshow(frames[0].numpy().T, aspect='auto', origin='lower', cmap='viridis')\n    plt.xlabel('Time')\n    plt.ylabel('Features')\n    plt.title('Feature Representation')\n    plt.show()\n\n    break","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:07.236253Z","iopub.execute_input":"2026-05-25T04:27:07.236538Z","iopub.status.idle":"2026-05-25T04:27:13.72999Z","shell.execute_reply.started":"2026-05-25T04:27:07.236509Z","shell.execute_reply":"2026-05-25T04:27:13.729136Z"},"id":"n-GV3UvgLSoF","trusted":true},"outputs":[{"name":"stdout","text":"torch.Size([2048, 61, 28]) torch.Size([2048])\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 1000x600 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAA0wAAAIjCAYAAAAwSJuMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABWtklEQVR4nO3deZwcdbX//3fvs++ZmUz2DQJkEQPkBsIi5CbkIhJBIoua4GUPKKIo+X1lCSoB9CIqIYBLIiogoqxKMGzJlX0PaxKyQJbJOpl9erqn+/P7g5uRISmcU0ymx5nX8/HoxyPpOafPp+pTVd2nq7s64JxzAgAAAADsIZjpAQAAAABAT0XDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAACRJixcvViAQ0Pr16zM9FADoMWiYAKCb7H4xurfb5Zdfvk9qPvPMM7r66qtVW1u7Tx7/0/j4+giHwxowYIBmz56tTZs2ZXp4Pdbf/vY3XX311Z/qMa699lrdf//9XTIeAOjtwpkeAAD0Nddcc42GDRvW4b4xY8bsk1rPPPOM5s2bp9mzZ6uoqGif1Pi0dq+PeDyu5557TosXL9Y//vEPvfnmm8rKysr08Hqcv/3tb1qwYMGnapquvfZafelLX9KMGTM63P/Vr35Vp512mmKx2KcbJAD0IjRMANDNpk+frkMOOSTTw/hUmpqalJub2yWP9dH1cfbZZ6usrEzXX3+9HnzwQc2cObNLanSGc07xeFzZ2dndVrOnCYVCCoVCmR4GAPQofCQPAHqYRx55REceeaRyc3OVn5+vE044QW+99VaHmBUrVmj27NkaPny4srKyVFlZqa9//evauXNne8zVV1+tyy67TJI0bNiw9o++rV+/XuvXr1cgENDixYv3qB8IBDqcvbj66qsVCAT09ttv64wzzlBxcbEmT57c/vff//73mjBhgrKzs1VSUqLTTjtNGzZs8L38Rx55pCRpzZo1He5/99139aUvfUklJSXKysrSIYccogcffLBDzO6P+S1fvlznnXeeSktLVVBQoK997WvatWtXh9ihQ4fq85//vB599FEdcsghys7O1m233SZJqq2t1SWXXKJBgwYpFotp5MiRuv7665VOpzs8xt13360JEyYoPz9fBQUFGjt2rH72s591iOnMY+2ej5/85Ce6/fbbNWLECMViMR166KF68cUX2+Nmz56tBQsWSFKHjzPu9pOf/ESHH364SktLlZ2drQkTJujee+/tMJ5AIKCmpib99re/bc+fPXt2h/X38e8w3XLLLTrooIMUi8VUVVWlOXPm7PExz2OOOUZjxozR22+/rc997nPKycnRgAEDdMMNNwgA/p1xhgkAulldXZ127NjR4b6ysjJJ0u9+9zvNmjVL06ZN0/XXX6/m5mYtXLhQkydP1quvvqqhQ4dKkpYuXaq1a9fqrLPOUmVlpd566y3dfvvteuutt/Tcc88pEAjo5JNP1qpVq3TXXXfppz/9aXuNfv36afv27eZxn3rqqRo1apSuvfZaOeckST/60Y90xRVXaObMmTr77LO1fft2/eIXv9BRRx2lV1991dfHAHe/WC8uLm6/76233tIRRxyhAQMG6PLLL1dubq7uuecezZgxQ3/+85/1xS9+scNjXHTRRSoqKtLVV1+tlStXauHChXr//ff11FNPdWgwVq5cqdNPP13nnXeezjnnHO2///5qbm7W0UcfrU2bNum8887T4MGD9cwzz2ju3Lmqrq7WTTfdJOnDOTj99NN13HHH6frrr5ckvfPOO3r66af1zW9+U5I6/Vi73XnnnWpoaNB5552nQCCgG264QSeffLLWrl2rSCSi8847T5s3b9bSpUv1u9/9bo9197Of/Uxf+MIXdOaZZyqRSOjuu+/WqaeeqocfflgnnHCCpA+3sbPPPluHHXaYzj33XEnSiBEjPOfj6quv1rx58zRlyhRdcMEF7evzxRdf1NNPP61IJNIeu2vXLh1//PE6+eSTNXPmTN1777363ve+p7Fjx2r69OmfNO0A0HM5AEC3WLRokZO015tzzjU0NLiioiJ3zjnndMjbsmWLKyws7HB/c3PzHo9/1113OUlu+fLl7ff9+Mc/dpLcunXrOsSuW7fOSXKLFi3a43Ekuauuuqr9/1dddZWT5E4//fQOcevXr3ehUMj96Ec/6nD/G2+84cLh8B73e62Pxx57zG3fvt1t2LDB3Xvvva5fv34uFou5DRs2tMced9xxbuzYsS4ej7ffl06n3eGHH+5GjRq1x2NOmDDBJRKJ9vtvuOEGJ8k98MAD7fcNGTLESXJLlizpMK4f/OAHLjc3161atarD/ZdffrkLhULugw8+cM45981vftMVFBS4trY2z2Xs7GPtno/S0lJXU1PTHvfAAw84Se6hhx5qv2/OnDnO6+n749tFIpFwY8aMcccee2yH+3Nzc92sWbP2yN+9/nZvL9u2bXPRaNRNnTrVpVKp9ribb77ZSXK/+c1v2u87+uijnSR3xx13tN/X2trqKisr3SmnnLLX8QLAvwM+kgcA3WzBggVaunRph5v04RmL2tpanX766dqxY0f7LRQKaeLEiXryySfbH+Oj37OJx+PasWOH/uM//kOS9Morr+yTcZ9//vkd/v+Xv/xF6XRaM2fO7DDeyspKjRo1qsN4P8mUKVPUr18/DRo0SF/60peUm5urBx98UAMHDpQk1dTU6IknntDMmTPV0NDQXmfnzp2aNm2aVq9evcdV9c4999wOZz4uuOAChcNh/e1vf+sQN2zYME2bNq3DfX/605905JFHqri4uMNyTZkyRalUSsuXL5ckFRUVqampqX3+9qazj7Xbl7/85Q5n1nZ/PHHt2rWdWpcf3S527dqluro6HXnkkb63iccee0yJREKXXHKJgsF/vmQ455xzVFBQoL/+9a8d4vPy8vSVr3yl/f/RaFSHHXZYp8cPAD0RH8kDgG522GGH7fWiD6tXr5YkHXvssXvNKygoaP93TU2N5s2bp7vvvlvbtm3rEFdXV9eFo/2nj1/Zb/Xq1XLOadSoUXuN/2jD8kkWLFig/fbbT3V1dfrNb36j5cuXd7hK23vvvSfnnK644gpdccUVe32Mbdu2acCAAe3///iY8vLy1L9//z2+m/PxZdq9XCtWrFC/fv08a0nShRdeqHvuuUfTp0/XgAEDNHXqVM2cOVPHH3+8+bF2Gzx4cIf/726ePv79Ky8PP/ywfvjDH+q1115Ta2tr+/0f/Riixfvvvy9J2n///TvcH41GNXz48Pa/7zZw4MA9ahUXF2vFihW+6gNAT0DDBAA9xO6LAPzud79TZWXlHn8Ph/95yJ45c6aeeeYZXXbZZfrMZz6jvLw8pdNpHX/88XtcmGBvvF5Ap1Ipz5yPXz0unU4rEAjokUce2euV1fLy8v7lOKSODeSMGTM0efJknXHGGVq5cmX7cknSd77znT3OBu02cuTITtX6uL1dES+dTus///M/9d3vfnevOfvtt58kqby8XK+99poeffRRPfLII3rkkUe0aNEife1rX9Nvf/tb02Pt5nWFOvd/3xn7JP/7v/+rL3zhCzrqqKN0yy23qH///opEIlq0aJHuvPPOf5nfFT7N+AGgp6JhAoAeYvcX78vLyzVlyhTPuF27dunxxx/XvHnzdOWVV7bfv/sM1Ud5NUa7z1x8/EpnHz9j8K/G65zTsGHD9njh71coFNL8+fP1uc99TjfffLMuv/xyDR8+XNKHZ6w+ab181OrVq/W5z32u/f+NjY2qrq7Wf/3Xf/3L3BEjRqixsbFTtaLRqE488USdeOKJSqfTuvDCC3Xbbbfpiiuu0MiRI02P1Vlec/rnP/9ZWVlZevTRRzucoVu0aFGnH+PjhgwZIunDi2PsngdJSiQSWrduXZcuFwD0VHyHCQB6iGnTpqmgoEDXXnutksnkHn/ffWW73e/if/xd+49fcU1S+28lfbwxKigoUFlZ2R7fobnllls6Pd6TTz5ZoVBI8+bN22MszrkOlzi3OOaYY3TYYYfppptuUjweV3l5uY455hjddtttqq6u3iN+b1f8u/322zusw4ULF6qtra1TV2qbOXOmnn32WT366KN7/K22tlZtbW2StMfyBYNBjRs3TpLaPw7X2cey8JrTUCikQCDQ4Szh+vXrdf/99+/1MT6evzdTpkxRNBrVz3/+8w5z/Otf/1p1dXXtV94DgN6MM0wA0EMUFBRo4cKF+upXv6rPfvazOu2009SvXz998MEH+utf/6ojjjhCN998swoKCnTUUUfphhtuUDKZ1IABA/T3v/9d69at2+MxJ0yYIEn6f//v/+m0005TJBLRiSeeqNzcXJ199tm67rrrdPbZZ+uQQw7R8uXLtWrVqk6Pd8SIEfrhD3+ouXPnav369ZoxY4by8/O1bt063XfffTr33HP1ne98x9e6uOyyy3Tqqadq8eLFOv/887VgwQJNnjxZY8eO1TnnnKPhw4dr69atevbZZ7Vx40a9/vrrHfITiYSOO+44zZw5UytXrtQtt9yiyZMn6wtf+EKnaj/44IP6/Oc/r9mzZ2vChAlqamrSG2+8oXvvvVfr169XWVmZzj77bNXU1OjYY4/VwIED9f777+sXv/iFPvOZz+iAAw4wPZbF7jn9xje+oWnTpikUCum0007TCSecoBtvvFHHH3+8zjjjDG3btk0LFizQyJEj9/gO0YQJE/TYY4/pxhtvVFVVlYYNG6aJEyfuUatfv36aO3eu5s2bp+OPP15f+MIX2tfnoYce2uECDwDQa2Xs+nwA0MfsvmTziy+++IlxTz75pJs2bZorLCx0WVlZbsSIEW727NnupZdeao/ZuHGj++IXv+iKiopcYWGhO/XUU93mzZv3uCS4cx9e2nrAgAEuGAx2uGR0c3Oz++///m9XWFjo8vPz3cyZM922bds8Lyu+ffv2vY73z3/+s5s8ebLLzc11ubm5bvTo0W7OnDlu5cqVvtdHKpVyI0aMcCNGjGi/bPeaNWvc1772NVdZWekikYgbMGCA+/znP+/uvffePR5z2bJl7txzz3XFxcUuLy/PnXnmmW7nzp0dagwZMsSdcMIJex1bQ0ODmzt3rhs5cqSLRqOurKzMHX744e4nP/lJ++XK7733Xjd16lRXXl7uotGoGzx4sDvvvPNcdXW1+bF2X1b8xz/+8R5j+fh8tLW1uYsvvtj169fPBQKBDpcY//Wvf+1GjRrlYrGYGz16tFu0aFH7/H3Uu+++64466iiXnZ3tJLVfYvzjlxXf7eabb3ajR492kUjEVVRUuAsuuMDt2rWrQ8zRRx/tDjrooD3GP2vWLDdkyJC9rmcA+HcQcI5vYgIAeofFixfrrLPO0osvvrjXKxECAGDFd5gAAAAAwAMNEwAAAAB4oGECAAAAAA98hwkAAAAAPHCGCQAAAAA80DABAAAAgIde/8O16XRamzdvVn5+vgKBQKaHAwAAACBDnHNqaGhQVVWVgsHOnTvq9Q3T5s2bNWjQoEwPAwAAAEAPsWHDBg0cOLBTsb2+YcrPz5ckjbjwSoViWZ3OC6SNhXxcOiPgI8f5+BBlOmKLT0XtNfwIttniQ60+inTHvPipYd2+fNaR9aRqN10Cxs+2b9Zdy9I9Zfa9nnr5n546LkmBbrhmkvPzyYieulF2x1z6WfYevI3ta92xDftl3fZ77LL00GH5eg3aXceWfVwnlYjrnTt+0N4jdEavb5h2fwwvFMvqsw1TwNgwKWav4UcwaYsP+SlCw7Tva/jQqxqmHvpkaNZTl6Onjks0TGY0TD1Oj20yRMO0r/Xlhqm9jGEb46IPAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHsKZHkB3aR7UpmB22z57/PEHvW/OeXfZcHNOa3nKnFP8esgU/+ULHzPX+OWKyeac/KezTfGNg525RnRUvTmneXuuKT5/tX03au5vXxbn4+2Nk4593hR/37OHmmtE6uwDS5TatuNDx6wx1xhXsMmcc0rBK+ack+76tine2XZHSdJZxz9hzrl3/WdM8Yf3X2+u8fhDE8w58YFJU/yRY1eaa7zw9zHmnBM+/5w55/5lh5ni08W2ZZekscPt2/Hqbf1M8elVeeYasZqAOef3F99oir/w3TPMNTZvLTLnyNmXZfSQalP8hkeGmmuM+vxqc86qHeWm+NiSAnONS799jznnxhtnmnMOnPW2Kb6mNcdcY9XmCnNOQX6zKT4nat/vmxMRc07i6VJbjcH215MuZs+577gF5pyvv/E1U/zkqrWm+ERjUm/+ypTCGSYAAAAA8ELDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA8BJxzLtOD2Jfq6+tVWFioob/+voI5WZ3OC0dSpjqtzRHr0FRY1GzOKc9rNOdsa8wzxTesKjbXKHrXnKJULGCKb8ux1wi12nNaym27RDBlWw5JilclzTkK23fVgQ+HTPH5b+0010itfM+ec8xnTfG1I2LmGjuPsK/jnMIWc86w0hpT/ICcWnONV7cPNOe0JGzHpNZ3Cs01srfat/2KF5pM8Q1Ds801Wgvt46qdZD9YBK375ObOPwftVvyOOUUyDmvXgfYSqWL7/jVk8A5T/MYVleYaqULbc7ckBVp8vHdcZFt+1xw2lwg12I7fkuQq46b4/v3qzDUKYrYaktSYsB/D0862H+9qsh8rAvZDhVnzZttrMElyIfvzfajZth2nCuz7Ss5a+2vdqH0TU8L+dGSSao1r9f/8f6qrq1NBQUGncjjDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA8ZbZjmz5+vQw89VPn5+SovL9eMGTO0cuXKDjHHHHOMAoFAh9v555+foREDAAAA6Esy2jAtW7ZMc+bM0XPPPaelS5cqmUxq6tSpamrqeBWlc845R9XV1e23G264IUMjBgAAANCX2K9z2YWWLFnS4f+LFy9WeXm5Xn75ZR111FHt9+fk5Kiy0n55UQAAAAD4NHrUd5jq6j68WHtJSUmH+//whz+orKxMY8aM0dy5c9Xc7P37Ra2traqvr+9wAwAAAAA/MnqG6aPS6bQuueQSHXHEERozZkz7/WeccYaGDBmiqqoqrVixQt/73ve0cuVK/eUvf9nr48yfP1/z5s3rrmEDAAAA6MV6TMM0Z84cvfnmm/rHP/7R4f5zzz23/d9jx45V//79ddxxx2nNmjUaMWLEHo8zd+5cXXrppe3/r6+v16BBg/bdwAEAAAD0Wj2iYbrooov08MMPa/ny5Ro4cOAnxk6cOFGS9N577+21YYrFYorFYvtknAAAAAD6low2TM45XXzxxbrvvvv01FNPadiwYf8y57XXXpMk9e/ffx+PDgAAAEBfl9GGac6cObrzzjv1wAMPKD8/X1u2bJEkFRYWKjs7W2vWrNGdd96p//qv/1JpaalWrFihb33rWzrqqKM0bty4TA4dAAAAQB+Q0YZp4cKFkj78cdqPWrRokWbPnq1oNKrHHntMN910k5qamjRo0CCdcsop+v73v5+B0QIAAADoazL+kbxPMmjQIC1btqybRgMAAAAAHfWo32ECAAAAgJ6kR1wlrzsEQ2kFQ+l99vhjhm425/xH8TpzzrqWMnPOmi39TPGBTz7xt1e5W1PmnB1jbZtf8/CkuUblwBpzTtPr5ab43M32FRapj5hzmgbat99tX/b+kee92VhT8q+DPqb0pUnmnFR2wBTfWmQuoWC9/fDWmmWfl3dfGmKKfyvH/jMHgZRtfUlSoDhhik8X2ffh2AGN5py1gwpM8alcH8ftmD2noLDFnFO/K8eWUGw/hjUNiJpzWobZ5j7nPXuNULU9Z9u6KlN8OGY/tqYKzSnmfUWSou9lm+JzttiXJR2y7/dt223j2lqaZa6xqajNnDNgyE5zzmdKN5niH2/Yz1wjEffxMni77UrMQR/H7+yRdeacpgbbXObmtZprNLfmmXNabLu9JCm6y3Y+p7W/bZtMt9iPxZxhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgAcaJgAAAADwQMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwEM40wPoqZLr80zxb23INdd4MzbEnKO8pDnl6P1X2xJGmEuo5YiIOac0FTLFv72l0lxj6/ZCc04g5kzxtRPtc+IS9vcqyqrqzDkTK943xf+1eYy5xq5j7MsfDNnWcVtt1FyjeKB9fSX/UWrOaflMsyk+++0cc42yI6vNOU0J2z65qy3fXKNxm/24F3IBU3xsm/1pygVt25cktebZ62Stj5niCyduM9fYFrYfw6IbbPtLS1XKXCMYt82jJBWusuXUDzeX0KDBO8w58Tb73O8ot63jUKvt+U6SEgX27TiVbcsZOW6jucaa1waac7a9WmHO+XvAlpO/zlxCWSH7dhxusa3jnZ+xz2PRHfbjcazE9roiHbUdvySprb85RW059uWP1Rjnxdme71JxH8c8cwYAAAAA9BE0TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgAcaJgAAAADwQMMEAAAAAB5omAAAAADAAw0TAAAAAHgIZ3oA3cWty5XLyup0fP5Bu0yPX5TTYh2SqncVmHMS1bnmnGc/GGqKL8iNm2scVLrFnBMOpE3xrbWdn7/dArGUOUcBY3yTfTcqWBUy5zRuLjPnPL3dlhM63L4dp+oi5pzoNts6K3rfmWs0bi0157hDGsw5xTmtpvhhx28w10ik7NvYpm1VpnjXYt8mww32nLZ82z4Z9jGu/A/MKXIb88w5rUW2+K0bis01hg7fZs750mGvmOJ/t36iucbWTfZlGfG1tab4FZtt27AkhYK25xVJ2lVnf1498CDbRrZ1SL65Rlk0Yc75YEuJKX79s4PMNVKl9ufVQJt9P3YDba9F6gP21wjRWnOKPvO1N03xXyp70VzjqSMPMOeUR+tN8Y9ttdfYFc825+zYad/2k0Nt275LGc//NNtf53KGCQAAAAA80DABAAAAgAcaJgAAAADwQMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA/hTA+guxwwaa0iudFOx1dmN9geP6faOiS9lj/InFPdr8Ccc0LFG6b4M/JXmmsUh3LMOdVtjab49yuzzTWeaR5lznlw8zhTfElWk7nGuxUV5pySvGZzTvWafqb4cDBtrjFy9CZzzvhJtpy0C5hr1CRyzTmtafsh8Qtlr5riNyeLzTUaU1nmnBXvDzDF579nX/ZAypyiSFPIFB9tcOYaBSttx29JCjbY96/mUWWm+HQkYq7xfsxWQ5J+/vznTfHJQvt+n7/ONo+S9FJ0iCk+ELTPfTJlH1dbg31e3n7dtizRWvv70zsqk+ackpdt+3Gj/WWIojvs6zhRZV+WrFW25/z8931sL3nmFD3/17Gm+KcGHGCuMWDoDnNObbNtfTXV2F9T5a3q/Ovo3Qri5hSlYrbnvLYS29y7uH0b5gwTAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMBDwDnnMj2Ifam+vl6FhYU67q/nKZwb63Re/+x6U53Vdf2sQ9OWXfnmHPdBrj0nYIsPtZpLqC3HvhlFBzWZ4sPhlLlGIGAfV+ixYlN8osBcQi0D7Msy+sAN5px1O0pN8a3NEXMN1dtzwk2292oCSXMJJUvS5pzCgXXmnHDIVqdfbqO5xvs1tm3Sj+Qq+4YcbjYeXCS1ltrWV7rQPvmRrDZ7zgr7sTWZbzu+JPvZxzV2P/t+39oWtsWnbPGStGGrfZsMR23HvVTK/p5uqj5qzgnXhcw5bf1s22XQuOySNLL/dnPOqjX9TfHBbPs2GQzZn1fz81rMObt25pniQzvtz0Xp8oQ5x7XZtsuwj+NRW4N9WQYM2WmKH1e62VwjZX1BKWlDk/1YURCNm+LH5NuWJd6Y1PxJj6qurk4FBZ177uMMEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADxktGGaP3++Dj30UOXn56u8vFwzZszQypUrO8TE43HNmTNHpaWlysvL0ymnnKKtW7dmaMQAAAAA+pKMNkzLli3TnDlz9Nxzz2np0qVKJpOaOnWqmpr+efW0b33rW3rooYf0pz/9ScuWLdPmzZt18sknZ3DUAAAAAPoK+7VEu9CSJUs6/H/x4sUqLy/Xyy+/rKOOOkp1dXX69a9/rTvvvFPHHnusJGnRokU64IAD9Nxzz+k//uM/9njM1tZWtbb+87rY9fW2y4MDAAAAwG496jtMdXUf/gZKSUmJJOnll19WMpnUlClT2mNGjx6twYMH69lnn93rY8yfP1+FhYXtt0GDBu37gQMAAADolXpMw5ROp3XJJZfoiCOO0JgxYyRJW7ZsUTQaVVFRUYfYiooKbdmyZa+PM3fuXNXV1bXfNmyw/+gfAAAAAEgZ/kjeR82ZM0dvvvmm/vGPf3yqx4nFYorFYl00KgAAAAB9WY84w3TRRRfp4Ycf1pNPPqmBAwe2319ZWalEIqHa2toO8Vu3blVlZWU3jxIAAABAX5PRhsk5p4suukj33XefnnjiCQ0bNqzD3ydMmKBIJKLHH3+8/b6VK1fqgw8+0KRJk7p7uAAAAAD6mIx+JG/OnDm688479cADDyg/P7/9e0mFhYXKzs5WYWGh/vu//1uXXnqpSkpKVFBQoIsvvliTJk3a6xXyAAAAAKArZbRhWrhwoSTpmGOO6XD/okWLNHv2bEnST3/6UwWDQZ1yyilqbW3VtGnTdMstt3TzSAEAAAD0RRltmJxz/zImKytLCxYs0IIFC7phRAAAAADwTz3mKnn72sa/D1EoltXp+FXl/7qZ+6hUYZt1SAo2hcw5OTsC5pymgSlTfMXY7eYa06veNuf8ddNBpvita8rMNaI77es4O22s4eO3kV3QPq53ggP/ddDHBLJtcx/JTpprRAvi5pyW9wpN8bE6+3afVWNfx3V5OeacWI5tne3Ynm+uEdkSNeeEm23rLGLbVCRJafuwlC6wHSsPHbXeXKMyy75Tvlxs/82+zZtKTPH5pU3mGomUfTsuzbLV2dhYZK5RUmxflkDA9rxalmOvMXzUTnNOdihhzqlN2o4VJVH7svgxpmizKX5TvMhcIxwwPklKakzar158YNlWU3xtIttco19WozknKNt2nB+xP0fWJOzPRZ/J32iKLww1m2tkBe2vEZ4OjDLnWNfxjmSeKT6RtC9Hj7hKHgAAAAD0RDRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeAhnegDdJXlwo1I5bZ2OryhsND3+5Iq11iEpK5g05+QEE+ac9+OlpvjnqoeYa/z+3UPNOQW5cVP84P22mmuU5zSYcxIp225RlVNnrtHYFjXn+PHGtipTfFNzzFzDuYA5p3D0TlN8v9wmc40heTXmnMPy15lzikLNpvhnGkaaa7w/ssScs6amzJxjFYt0/pi624SiHab4r5Q/a64xMbbLnPNiUaE5Z2n5GFP8hNz15hoTszaYc/qFbMewexsHm2uUhGzPkZK0pa3IFB8N2LevmrY8c87Rue+ac2rT2ab4prT92PpE/YHmnLqkbVxtafv75sGgM+ecWvmSOSc32GqK35K078OP7TzAnFOZZXtdMcDH8ah/1P66Ii9ke01VnSwy1ygO25+LD8rdZM6xzn2/cL0pvjmY0m2mDM4wAQAAAIAnGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4CHgnHOZHsS+VF9fr8LCQg286RoFs7M6nZdf2WCq0z/fFi9J+dG4OScYsE/XpsZCU3xxVou9Rp2thiTVbs03xYfqQ+YakSFN5pzjhq4yxZdGG801/rT6YHNOLNJmzmlNhk3xQ0trzDXeWTnQnGMV9jH3boB9/3Jpc4ry8m11mptj5hrprZ0/du2Wtd32flio1VxCyVx7Tluu7RiWjtiPecFkwJyTtd2eE262xbdUmEsoUZoy57iYbUMOxO3vncYqjQsvqTTfdjwOB+07ZGvKfqyIhezrOGQcW3VtgblGaqXtOVKSIvX27dgqXu7jQNnffjyORm3PefHmqLmGa7I9R0pS3lpbTjBhLqGsGvtxr2asLb6t0P6aoqSqzpwzoKDenLO92fbkkhe1reS2plYtP/EW1dXVqaCgc/smZ5gAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgAcaJgAAAADwQMMEAAAAAB7CmR5Ad8lfFVYo1vnFTa0vNj3+hkNC1iFpv37bzTnbW3LNOVvfLLfFu4C5RiBtTlHEmBNos9eI78w25zwTG2qrkYiYa7Q0xMw5hZUt5pxwyLaSW1M+DgnWiZQUzUuY4tO19u0+tD7LnJPMty9LU8iZ4tNt9vepArYSkqRkvi2ppcq+7IGk/VjhSmxz75L29VXav86cE/Sxknc15JjiI5GUuUayOWrOOXz4OlP8+voSc42J/dabc6zCQfs2ua0135xT3VxgztnZbDsmZUWT5hq7Kuw5yXzba5Fgq33/SuXZt+OqknpzzoHFW03xbc6+LGvqysw5mwuKzDlW9Qn7spSUNZji/bx2qchrNOdkhezbsfV43Ja2rS9rvMQZJgAAAADwRMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADyEMz2A7tJwYELBbEN/GHKmx48kQ8YRSQ3JmDmnJREx5zjj0HI2Bcw12vLMKWrtl7IlpO01rPMoSbt22hYmGPEzMLv65ixzTjpte0+kpjbXXCMYM86jpKxY0hSf2q/eXCMQsM/9wPxGc07/HNvYYsE2c40VJVXmnFTath/XN2aba8SM8yhJ4ZBtfxlSvMtcY3jeDnPOkfmrzDkNads+WRmuM9d4Iz7QnPPZ7PWm+NcLhphrBAP24947Tf1N8ek2+3u6Q7N3mnMmF6425/QL2/b7V5uHmms0DrS/RqhJ2I7hw3Ls+0pZuMGcUxK2H1sLgnFT/KZksbnGgXmbzTmt/W2vw5pTUXONsoh9feUEW03xKR/nTNLOnlOdKDTnlMdsy580jisRSugZUwZnmAAAAADAEw0TAAAAAHigYQIAAAAADzRMAAAAAOAhow3T8uXLdeKJJ6qqqkqBQED3339/h7/Pnj1bgUCgw+3444/PzGABAAAA9DkZbZiampo0fvx4LViwwDPm+OOPV3V1dfvtrrvu6sYRAgAAAOjLuuyy4rW1tSoqKjLlTJ8+XdOnT//EmFgspsrKyk8xMgAAAADwx9cZpuuvv15//OMf2/8/c+ZMlZaWasCAAXr99de7bHCS9NRTT6m8vFz777+/LrjgAu3c+cm/sdDa2qr6+voONwAAAADww1fDdOutt2rQoEGSpKVLl2rp0qV65JFHNH36dF122WVdNrjjjz9ed9xxhx5//HFdf/31WrZsmaZPn65UyvuHMufPn6/CwsL22+5xAgAAAICVr4/kbdmypb0RefjhhzVz5kxNnTpVQ4cO1cSJE7tscKeddlr7v8eOHatx48ZpxIgReuqpp3TcccftNWfu3Lm69NJL2/9fX19P0wQAAADAF19nmIqLi7VhwwZJ0pIlSzRlyhRJknPuE8/+fFrDhw9XWVmZ3nvvPc+YWCymgoKCDjcAAAAA8MPXGaaTTz5ZZ5xxhkaNGqWdO3e2X7jh1Vdf1ciRI7t0gB+1ceNG7dy5U/37999nNQAAAABgN18N009/+lMNHTpUGzZs0A033KC8vDxJUnV1tS688MJOP05jY2OHs0Xr1q3Ta6+9ppKSEpWUlGjevHk65ZRTVFlZqTVr1ui73/2uRo4cqWnTpvkZNgAAAACY+GqYIpGIvvOd7+xx/7e+9S3T47z00kv63Oc+1/7/3d89mjVrlhYuXKgVK1bot7/9rWpra1VVVaWpU6fqBz/4gWKxmJ9hAwAAAICJ799h+t3vfqfbbrtNa9eu1bPPPqshQ4bopptu0rBhw3TSSSd16jGOOeYYOec8//7oo4/6HR4AAAAAfGq+LvqwcOFCXXrppZo+fbpqa2vbL/RQVFSkm266qSvHBwAAAAAZ4+sM0y9+8Qv98pe/1IwZM3Tddde133/IIYfs9aN6PUI68OGtk0LZtqv9hcJp64gUC7WZc5rjUXNOusBWpzG38+tpt3Be0pyTn9Nqim96r9BcIx20vycQrLXtFkH7NCrsIycRsc99Kte2XQbj9vWVKrbPfaPLMsU7w77bntPm6/2gfa4w2tItdVJp2/KnEyFzjeYW+1NIVoFtvy+NNZlrRAL2q7XmBm3jkqQNyRJT/KjoFnONkLw/heElK2DbJ1Oy719loUZzTnbINq6WVMRcY2vCfmXctI/lr04WmeKfrxlqrlEQjZtzrPJC9hpr4uXmnOcTw8051u0l5ezzGAnYX7sFjTlrG8vMNfpl7fv9qypWa67hR8rHuRnrOh4UqzPFx9vsr1t8vaJYt26dDj744D3uj8ViamqyP7kBAAAAQE/kq2EaNmyYXnvttT3uX7JkiQ444IBPOyYAAAAA6BF8fSTv0ksv1Zw5cxSPx+Wc0wsvvKC77rpL8+fP169+9auuHiMAAAAAZISvhunss89Wdna2vv/976u5uVlnnHGGqqqq9LOf/UynnXZaV48RAAAAADLC3DC1tbXpzjvv1LRp03TmmWequblZjY2NKi+3fwkQAAAAAHoy83eYwuGwzj//fMXjH15ZJScnh2YJAAAAQK/k66IPhx12mF599dWuHgsAAAAA9Ci+vsN04YUX6tvf/rY2btyoCRMmKDc3t8Pfx40b1yWDAwAAAIBM8tUw7b6wwze+8Y32+wKBgJxzCgQCSqXsPxoIAAAAAD2Nr4Zp3bp1XT0OAAAAAOhxfDVMQ4YM6epxAAAAAECP46thuuOOOz7x71/72td8DQYAAAAAehJfDdM3v/nNDv9PJpNqbm5WNBpVTk4ODRMAAACAXsHXZcV37drV4dbY2KiVK1dq8uTJuuuuu7p6jAAAAACQEb4apr0ZNWqUrrvuuj3OPgEAAADAv6sua5gkKRwOa/PmzV35kAAAAACQMb6+w/Tggw92+L9zTtXV1br55pt1xBFHdMnAulzg/26dFIm2mR4+HLb/9tT2pjxzTmtT1JyTX9pkip9QudFcozzWYM6pSeT+66CPeGL9WHMNP8Itxvgmw4b1fwJpc4pCcXtOS7ntPZFQwr4sLXn2911cPGSLDzlzjVCTfVyNads2KUnNjTFTfCRmO7ZIUjhs32Ba4xFTvGu1r69gi20eJanVuCzb4/bjZHVzgTlnR8JeZ3VtP1P8e6UV5hrvNZSZcz4oLDHFv1tnH9fw/J3mnJV15ab4tLMfj7bV2+cxHLLvX20p2/7SvN1+bAkVJMw5BXm2J7CwjyejVzYPNOe07Mw251gFc+3H1pzcVnOOM26X1ucISQr5OOYHgrac0kLba0O/ssL2eYkGba+pG3KzTPGJZvu+5athmjFjRof/BwIB9evXT8cee6z+53/+x89DAgAAAECP46thSqd9vD0OAAAAAP9mfH2H6ZprrlFzc/Me97e0tOiaa6751IMCAAAAgJ7AV8M0b948NTY27nF/c3Oz5s2b96kHBQAAAAA9ga+GyTmnQGDPL729/vrrKimxfdkUAAAAAHoq03eYiouLFQgEFAgEtN9++3VomlKplBobG3X++ed3+SABAAAAIBNMDdNNN90k55y+/vWva968eSosLGz/WzQa1dChQzVp0qQuHyQAAAAAZIKpYZo1a5YkadiwYTr88MMVidh+5wMAAAAA/p34uqz40Ucf3f7veDyuRKLjD0AVFNh/NBAAAAAAehpfF31obm7WRRddpPLycuXm5qq4uLjDDQAAAAB6A18N02WXXaYnnnhCCxcuVCwW069+9SvNmzdPVVVVuuOOO7p6jAAAAACQEb4+kvfQQw/pjjvu0DHHHKOzzjpLRx55pEaOHKkhQ4boD3/4g84888yuHicAAAAAdDtfZ5hqamo0fPhwSR9+X6mmpkaSNHnyZC1fvrzrRgcAAAAAGeSrYRo+fLjWrVsnSRo9erTuueceSR+eeSoqKuqywQEAAABAJvlqmM466yy9/vrrkqTLL79cCxYsUFZWlr71rW/psssu69IBAgAAAECmBJxz7tM+yPvvv6+XX35ZI0eO1Lhx47piXF2mvr5ehYWFOv6RcxTJjXY6r6XN9htTrSn718Hyoq3mnM319ku2BwK2KW5szjLXyMlK/OugjxlQWGeKjwbbzDUKo3Fzzqrafqb4vIh92Sty6s058ZT9d8/yw7ZtbG1DqblGKu3rfReTDZtLzDmxXPu8JBP2/XhAv1pTfNC4P0rSltp8c06qLWSKb2v08bt6bQFzSsDZclzQx1OUj5xgrv34kk7atv1AyD6uwM7OP2/t5qx17NMoF/GxLClboXBpi7lG0MfctzbEzDmhGtv+Eq2zr+RUtn1ZQnFbnUSBj30laU5RqNW+/Mk829hSeSlzjYCP/d76miqdtB2LJSnQ4iMnYVvHkXr7c3drhX19+RErse371n043RLXxjlXq66urtM/heTrog8fFY/HNWTIEA0ZMuTTPhQAAAAA9Ci+3hpOpVL6wQ9+oAEDBigvL09r166VJF1xxRX69a9/3aUDBAAAAIBM8dUw/ehHP9LixYt1ww03KBr958cFxowZo1/96lddNjgAAAAAyCRfDdMdd9yh22+/XWeeeaZCoX9+znL8+PF69913u2xwAAAAAJBJvhqmTZs2aeTIkXvcn06nlUz6+DYgAAAAAPRAvhqmAw88UP/7v/+7x/333nuvDj744E89KAAAAADoCXxdJe/KK6/UrFmztGnTJqXTaf3lL3/RypUrdccdd+jhhx/u6jECAAAAQEaYzjCtXbtWzjmddNJJeuihh/TYY48pNzdXV155pd555x099NBD+s///M99NVYAAAAA6FamM0yjRo1SdXW1ysvLdeSRR6qkpERvvPGGKioq9tX4AAAAACBjTGeYnOv468aPPPKImpqaunRAAAAAANBT+Lrow24fb6AAAAAAoDcxNUyBQECBQGCP+wAAAACgNzJ9h8k5p9mzZysWi0mS4vG4zj//fOXm5naI+8tf/tJ1IwQAAACADDE1TLNmzerw/6985StdOhgAAAAA6ElMDdOiRYv21TgAAAAAoMf5VBd9AAAAAIDezHSG6d/ZiLztiuVFOh3/Rm2V6fGbElHrkFQfj5lzUml7j5uf1WqKb4slzTXCobQ5p7oh3xR/TNV75hoH5mw250wqXGOKr0tlm2sUhlrMOUkXMuc8vWukKX7TzkJzjWRL5/er3bLyEqb4SFabuUZOlq2GJO1qsO+Tm7YXmeJdW/e8T+VStgvyBBL2cbmw/Uqpke22OoG0/cJCqSz7uFLZKXOOWm37ZGS7fR/OqrEvf0u5bfnbCuzHb/lIidTY5j68Mc9cI5lvn/uwj1dCbQW27aWtzT73wYR97gPGQ2WkqXsu3BWK+8mxjS0Zt69j1dhzknm2bSxab1/Hbbn27Th7m23/cj6einLX2XeWeJmP43G9bd+3PhWl4/bjPWeYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgAcaJgAAAADwQMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8BDO9AC6ywNvjlcwO6vT8ZHqqOnxU1nOOiS5mD0nVNxqzrGKRdrMObvqcs05qV0xU/xftn3WXOO++KHmHKtQg/19h7ZS+zpWyL69BBpsu3hsp31ZcneYU+TCtrlvLbHXqCvMNudEWgL2nMZ9f6xo7W/fXvLKmmzxWfZjS1OrbdklqTFZYIqP1Nm3yZCPw2Rbs/3pMBi3jS2UsG9fSfuhVWnjooRLW+xFnI9lyQuZ4lNbbccJSQqmzClKZafNOeGChCne2TZ7SVJrq219SVJoh22fjNbb5zGYNKco1fmXX+3ilcbjXsQ+j6Fs+wbj4rZ5SeSbSyjQYp/75gG2ZYnttNdIFNjXcdrH/pVbYXv+atqRY4pPt9ifUznDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA8ZbZiWL1+uE088UVVVVQoEArr//vs7/N05pyuvvFL9+/dXdna2pkyZotWrV2dmsAAAAAD6nIw2TE1NTRo/frwWLFiw17/fcMMN+vnPf65bb71Vzz//vHJzczVt2jTF4/FuHikAAACAviijlxWfPn26pk+fvte/Oed000036fvf/75OOukkSdIdd9yhiooK3X///TrttNP2mtfa2qrW1n9eU7a+vr7rBw4AAACgT+ix32Fat26dtmzZoilTprTfV1hYqIkTJ+rZZ5/1zJs/f74KCwvbb4MGDeqO4QIAAADohXpsw7RlyxZJUkVFRYf7Kyoq2v+2N3PnzlVdXV37bcOGDft0nAAAAAB6r4x+JG9fiMViisXsvw4OAAAAAB/XY88wVVZWSpK2bt3a4f6tW7e2/w0AAAAA9qUe2zANGzZMlZWVevzxx9vvq6+v1/PPP69JkyZlcGQAAAAA+oqMfiSvsbFR7733Xvv/161bp9dee00lJSUaPHiwLrnkEv3whz/UqFGjNGzYMF1xxRWqqqrSjBkzMjdoAAAAAH1GRhuml156SZ/73Ofa/3/ppZdKkmbNmqXFixfru9/9rpqamnTuueeqtrZWkydP1pIlS5SVlZWpIQMAAADoQzLaMB1zzDFyznn+PRAI6JprrtE111zTjaMCAAAAgA/12O8wAQAAAECm9brLinsZULlL4dzOX248e3DS9PgVOfXWIal/lj0n7QLmnB2JPFP88xuHmGukdtov5e5yU6b4g4ZvMteIBtvMOQcWeP/O196EAmlzjQk568w5T9QfaM75oKnYFL9uV6m5RnleozlnVMF2U3y/aIO5xjM7hptzNuwqMuc0N9i2/ZIy+7IMymkx5/TLss1LbjhhruHHrtJsU3ww4P0pBC/RoO3YIkmjcreZczbGbftXSypirhHzcQwblrPDFD8wWmOusTVZaM7JC8VN8bvacs01grJvL83pqDlnu/F5tSpWZ65R32b/+kF2yPbaZVO8yFyjJNJkzvGj3Hjc7x/ZZa4xLmZ/XRExPufXpu2vjx5rGGPO2dWWY4p/q7a/ucbUirfNOX4cnrPaFP9I/XhTfGtjUjeaMjjDBAAAAACeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgIdwpgfQXYIBp2DAdTo+kQ6ZHn99fal1SAoZxrNbOJA25wzOrjHFlw1vNNfY1L/InFOfzDLFNyWj5hrRWJs55+36SlO8Zbva7a36/uacnfFcc05LMmKKb2iyzYkk1e2yj2t1sNwUn5PXaq4Rj9uWXZJSzfZDYqjOlrOrsdhcoyZaaM5Zm2Pb9mPZSXONtjYf77m5gCk8GfcxJ1H7cfLplpHmHCVsyx9qsa+vVLZ9WUKFtrnMzrHvX6mUj2Ux5iRb7PtwIGxfXy5l2yYlKSs3YYoPh1PmGsmkfdtPGPeXtI99ODvfvr0Eg/bnSat02j6P2THbPEpSS6vttUjAx2uE5u3259VAlm0bcz7W145G+7jys+zby0MaZ4qPt9m2+1Szj23YnAEAAAAAfQQNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgAcaJgAAAADwQMMEAAAAAB7CmR5AdxlTXK1oXmSfPf7QrJ3mnP2zNptzsgJJc876RD9T/OtNg8w1skP2ce2ft9UUPyS2w1yjJNRozokGUqb4qnCducb2VK45J+nsu+v6RJkpfk283FxjZUOFOWdnS44pPj/aaq7hR3Myas7ZWWxblkDAXEKxiH3/KsqOm+JTafv7Z0WxFnNOd2hpsx/r/eSEgmlTfH08Zq4RDduOR5IUMY4rFm4z12htsx+P0s628RdV2LevkliTOae5zb7fh43ruDFpn/uccMKck0jZ5qUmbjt+Sf62l5akff86qKTaFN+QzDLXsL4OkaTtiXxTfL9og7nGs8XDzDnWbbK+1b6+RhTaX4cl0yFzzuDsGlN8LGjbJlsbk3rNlMEZJgAAAADwRMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADyEMz2A7vLYEwcrmJXV6fh0xPb4qaI244gkOXuKQvak8E7bwgR8LEqoNWDOeao4bYp3xQlzDT9c0vg+grMvu595DIRs60uSVGeb+3Cz/T2UQNKcIhlX2TY/Ryof05KO2uclnW2bl2CefYWF8uxzX9OUY4pvaomaa7zfXGbOCQRt6ziWa9/vQz72lWQyZM5pS9hyoln2g2urj2VJp20bfyScMtdIJO07ZdhYZ1ttnrlGbnahOScYtK/jppaYKb61wRYvSbI+F0kKF9j2l0jUvk362V7SPp4nX90+0BQfDNiP3ytr+plzrIYV1ZhzcsL2414ibdsna1s6/5p4t5dbbHMiSUkfx4rXw1Wm+NLcZlN8W1OrKV7iDBMAAAAAeKJhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgAcaJgAAAADwQMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwEM40wPoLsFEQKFgoNPxbfkpW4GQM45ICrSEzDlK2lNSMePYyu1FknH7sgSybOs4HDXOiaTcnFZzTjCYNsUn2rpnN0ok7HWSnd/kP4zP8rFNpoxFJIWabO/VOD9v7ZTb5760uNGcM7Johym+pjXHXCMr1GbOaWqLmuIr8hvMNZqTthqS1JKMmOLzYvZ5LIzGzTnBgG2/l6SaeK4pPu3s+0pe1L78WSHbMbw01myu4WdZKmL15hyrSND+PBEJ2HO2tBaa4jc2F5lrvF9bbM6xzkt5vv2YV+xjexmWu9Ockx+y7ceFoRZzjZo22z4sSVlB2/6VE0yYa8Sd/fl+U6tteymJNZlrZBuPLZIUC9qfv6x1+kdrTfHxxjY9a8rgDBMAAAAAeKJhAgAAAAAPNEwAAAAA4KFHN0xXX321AoFAh9vo0aMzPSwAAAAAfUSPv+jDQQcdpMcee6z9/+Fwjx8yAAAAgF6ix3cf4XBYlZWVmR4GAAAAgD6oR38kT5JWr16tqqoqDR8+XGeeeaY++OCDT4xvbW1VfX19hxsAAAAA+NGjG6aJEydq8eLFWrJkiRYuXKh169bpyCOPVEOD9++FzJ8/X4WFhe23QYMGdeOIAQAAAPQmPbphmj59uk499VSNGzdO06ZN09/+9jfV1tbqnnvu8cyZO3eu6urq2m8bNmzoxhEDAAAA6E16/HeYPqqoqEj77bef3nvvPc+YWCymWCzWjaMCAAAA0Fv16DNMH9fY2Kg1a9aof//+mR4KAAAAgD6gRzdM3/nOd7Rs2TKtX79ezzzzjL74xS8qFArp9NNPz/TQAAAAAPQBPfojeRs3btTpp5+unTt3ql+/fpo8ebKee+459evXL9NDAwAAANAH9OiG6e677870EAAAAAD0YT36I3kAAAAAkEk9+gxTV2rt16Zgdlun46OlcdPj71+5zToklcaazDkbm4rMOdV1Bab4/GzbsktSc2vUnFOe32iKT6ZC5hojCnaYc/bL3WqKj6cj5hqrm8rNOWvqSs05iTbbOouGU+YajXH7VSmzIp3fFyUpFrbFS9LEfuvNOYOyasw5VZFdpvh3WgaYa/jZXrY255niQwFnrmHPkEpzbMc95wLmGom0/VjRmrJvx3nRVlN8U9J+nAz6mJfqJtsxv7nNPq6skH2fjPnIsdoWt233klSXyDbnNLTatpe6JnuNdNq+7cdiSVN80Mde3Ji07yvbWvPNOU0hW514xP5c/EFLsTknEkib4ptS9v0rbKwh2eclK2zbViQpnrKv4+Joszmnqc22LIWhFlN8PGU/X8QZJgAAAADwQMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACAh3CmB9BtAv9366Rk3LZq1u8qto1H0vZYrjnHOcNC/J/WeMQUHzfG+1UbTpnio+E2c41NzYXmnK3xfFP8tqY8c42aOvvcp1pD5pxwzLaOQ8Y5kaS2Nvu4UinbezU7agvMNR7YaZ97lzanaGj/nab4mqYcc426WntOaEvMFO98vH3mIs6cU12SsNVI2Y95StoXJtho345d1Lb8Lsu+fwX95ARt46oJ2Y9HaeM+LEkrkgNM8c7HPPqSsNcJtNm2y1CLjxo+jkctxs14da792BoosO3DkrQqVG7OkfH1TjBoX2F+XlMFArb9qy3p47k7Yt/v09Zl8bHsqSYfbUPY/jwRNC7/a0VVpvhUc6ukv5tyOMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkAAAAAPNAwAQAAAIAHGiYAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACAh3CmB9BdYlvDCsU6v7iprJDp8Vu2xKxDUnPImXPSUXuOy06b4oNZbfYaLmDO2VmTZ6vRbN9cQ0329wScberlY9GlsH0e/ZRpa7UtTFvUtq18mGQfWUs6aooPNvuYxwYfhzcfK3ltstwUH8lOmmsEgvbtpa0gZYoPttjXcTrHVkOSIhFbTthHjbakcSeW1GY85ktS0HgMT/vYV6Ix+/E4K2rbxsIh+34fT9r3r9ZAxBSf8vEc6Yfzczw2rrNUgY/3pwM+nu8Txu3Yx7HFtdmXJRiwb2MB4+4Sidr3laSPY4VL25bfpXw8sfh4rRcJ+3j+NnI+dsmgj20sELQtS6LNdjxKtdmfVzjDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgAcaJgAAAADwEM70ALpLuEUKpTsfH0wGTI+fzHfGEUmBlK2GZB+XJLkWW18cTNg3i6CPZZFxlaUj9nUsH8NKGxe/LT9lL+JDwPnYXuLG90Sa7e+hBNt8bJPGFOdj7tMxww7/aRiHloz7OOwmfMyLce5DrT7mMWofV7IuZouP+phHH4cK+diOU2FjIR81EiH7wiSs25ifw3fax/HIuiw+5tGl7Nuki4fsOc6WE/Ax936W31zFz9vm3bR/pa1PX1n2Y2sgy/78bd7GfLxua2vINucko8aJCfiYSD/PxX6244jtuJ9qs+2P6Wb7oDjDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMADDRMAAAAAeKBhAgAAAAAPNEwAAAAA4IGGCQAAAAA80DABAAAAgIdwpgfQXZoPjCuY3fl4l7L1koGgM46o+7iWkCk+usMWL0nOx/K3GeZDklIlSXMN+ZkW69sIfmr4yHFJH+9vGPfwQCJgLpEO7/tt38/2FUjY11fAz6K0GI8VaXuJQNo+L1ZpH88GgaR9XNZtLNDoYx5TPsblY+4DKXuOldtlnxhnXGXOx+YV8LHCzNuxj3EFfexfvo7hRn62L+s8Svbji5/jUXesL8nH8rf42Vd8HPiM6yzYZi8R9PFcbN3G/Gxf6Uj3TL6zvgy1HlrixscXZ5gAAAAAwBMNEwAAAAB4oGECAAAAAA80TAAAAADg4d+iYVqwYIGGDh2qrKwsTZw4US+88EKmhwQAAACgD+jxDdMf//hHXXrppbrqqqv0yiuvaPz48Zo2bZq2bduW6aEBAAAA6OV6fMN044036pxzztFZZ52lAw88ULfeeqtycnL0m9/8Zq/xra2tqq+v73ADAAAAAD96dMOUSCT08ssva8qUKe33BYNBTZkyRc8+++xec+bPn6/CwsL226BBg7pruAAAAAB6mR7dMO3YsUOpVEoVFRUd7q+oqNCWLVv2mjN37lzV1dW13zZs2NAdQwUAAADQC/n4ieOeLRaLKRaLZXoYAAAAAHqBHn2GqaysTKFQSFu3bu1w/9atW1VZWZmhUQEAAADoK3p0wxSNRjVhwgQ9/vjj7fel02k9/vjjmjRpUgZHBgAAAKAv6PEfybv00ks1a9YsHXLIITrssMN00003qampSWeddVamhwYAAACgl+vxDdOXv/xlbd++XVdeeaW2bNmiz3zmM1qyZMkeF4IAAAAAgK7W4xsmSbrooot00UUXZXoYAAAAAPqYf4uG6dNwzkmS0i2ttryU7etdgaAzxXcn1xIyxafjtvgPi9hT0gFjfEvSXsTPtFi/2eenhp+cpP0rh4GUbSUHksZJ6SYu5GOFOfuy+Fr6tLGGMf7DnH0/Lz5Wl1zYz7zYwn2tL+N2L0kBH4sSSNlzrJyfw7HxUOFn7v2sMOt27GdcfrYXX8djIz/bl3UeJfvy99T1JflYfh/ry886th7z1WYv4Xw8F1u3MV/H/FT3TL75uGd9PRmPf1jHdX55As4S/W9o48aN/HgtAAAAgHYbNmzQwIEDOxXb6xumdDqtzZs3Kz8/X4HAP1vQ+vp6DRo0SBs2bFBBQUEGR4juxtz3Xcx938Xc913Mfd/G/PddXnPvnFNDQ4OqqqoUDHbuNGOv/0heMBj8xO6xoKCAHaiPYu77Lua+72Lu+y7mvm9j/vuuvc19YWGh6TF69O8wAQAAAEAm0TABAAAAgIc+2zDFYjFdddVVisVimR4Kuhlz33cx930Xc993Mfd9G/Pfd3Xl3Pf6iz4AAAAAgF999gwTAAAAAPwrNEwAAAAA4IGGCQAAAAA80DABAAAAgIc+2zAtWLBAQ4cOVVZWliZOnKgXXngh00NCF1u+fLlOPPFEVVVVKRAI6P777+/wd+ecrrzySvXv31/Z2dmaMmWKVq9enZnBokvNnz9fhx56qPLz81VeXq4ZM2Zo5cqVHWLi8bjmzJmj0tJS5eXl6ZRTTtHWrVszNGJ0lYULF2rcuHHtP1Q4adIkPfLII+1/Z977huuuu06BQECXXHJJ+33Mfe919dVXKxAIdLiNHj26/e/Mfe+2adMmfeUrX1Fpaamys7M1duxYvfTSS+1/74rXe32yYfrjH/+oSy+9VFdddZVeeeUVjR8/XtOmTdO2bdsyPTR0oaamJo0fP14LFizY699vuOEG/fznP9ett96q559/Xrm5uZo2bZri8Xg3jxRdbdmyZZozZ46ee+45LV26VMlkUlOnTlVTU1N7zLe+9S099NBD+tOf/qRly5Zp8+bNOvnkkzM4anSFgQMH6rrrrtPLL7+sl156Sccee6xOOukkvfXWW5KY977gxRdf1G233aZx48Z1uJ+5790OOuggVVdXt9/+8Y9/tP+Nue+9du3apSOOOEKRSESPPPKI3n77bf3P//yPiouL22O65PWe64MOO+wwN2fOnPb/p1IpV1VV5ebPn5/BUWFfkuTuu+++9v+n02lXWVnpfvzjH7ffV1tb62KxmLvrrrsyMELsS9u2bXOS3LJly5xzH851JBJxf/rTn9pj3nnnHSfJPfvss5kaJvaR4uJi96tf/Yp57wMaGhrcqFGj3NKlS93RRx/tvvnNbzrn2Od7u6uuusqNHz9+r39j7nu3733ve27y5Mmef++q13t97gxTIpHQyy+/rClTprTfFwwGNWXKFD377LMZHBm607p167Rly5YO20FhYaEmTpzIdtAL1dXVSZJKSkokSS+//LKSyWSH+R89erQGDx7M/PciqVRKd999t5qamjRp0iTmvQ+YM2eOTjjhhA5zLLHP9wWrV69WVVWVhg8frjPPPFMffPCBJOa+t3vwwQd1yCGH6NRTT1V5ebkOPvhg/fKXv2z/e1e93utzDdOOHTuUSqVUUVHR4f6Kigpt2bIlQ6NCd9s912wHvV86ndYll1yiI444QmPGjJH04fxHo1EVFRV1iGX+e4c33nhDeXl5isViOv/883XffffpwAMPZN57ubvvvluvvPKK5s+fv8ffmPvebeLEiVq8eLGWLFmihQsXat26dTryyCPV0NDA3Pdya9eu1cKFCzVq1Cg9+uijuuCCC/SNb3xDv/3tbyV13eu9cNcNGQB6njlz5ujNN9/s8Hl29G7777+/XnvtNdXV1enee+/VrFmztGzZskwPC/vQhg0b9M1vflNLly5VVlZWpoeDbjZ9+vT2f48bN04TJ07UkCFDdM899yg7OzuDI8O+lk6ndcghh+jaa6+VJB188MF68803deutt2rWrFldVqfPnWEqKytTKBTa4+ooW7duVWVlZYZGhe62e67ZDnq3iy66SA8//LCefPJJDRw4sP3+yspKJRIJ1dbWdohn/nuHaDSqkSNHasKECZo/f77Gjx+vn/3sZ8x7L/byyy9r27Zt+uxnP6twOKxwOKxly5bp5z//ucLhsCoqKpj7PqSoqEj77bef3nvvPfb7Xq5///468MADO9x3wAEHtH8ks6te7/W5hikajWrChAl6/PHH2+9Lp9N6/PHHNWnSpAyODN1p2LBhqqys7LAd1NfX6/nnn2c76AWcc7rooot033336YknntCwYcM6/H3ChAmKRCId5n/lypX64IMPmP9eKJ1Oq7W1lXnvxY477ji98cYbeu2119pvhxxyiM4888z2fzP3fUdjY6PWrFmj/v37s9/3ckccccQePxuyatUqDRkyRFIXvt77NFem+Hd19913u1gs5hYvXuzefvttd+6557qioiK3ZcuWTA8NXaihocG9+uqr7tVXX3WS3I033uheffVV9/777zvnnLvuuutcUVGRe+CBB9yKFSvcSSed5IYNG+ZaWloyPHJ8WhdccIErLCx0Tz31lKuurm6/NTc3t8ecf/75bvDgwe6JJ55wL730kps0aZKbNGlSBkeNrnD55Ze7ZcuWuXXr1rkVK1a4yy+/3AUCAff3v//dOce89yUfvUqec8x9b/btb3/bPfXUU27dunXu6aefdlOmTHFlZWVu27Ztzjnmvjd74YUXXDgcdj/60Y/c6tWr3R/+8AeXk5Pjfv/737fHdMXrvT7ZMDnn3C9+8Qs3ePBgF41G3WGHHeaee+65TA8JXezJJ590kva4zZo1yzn34aUmr7jiCldRUeFisZg77rjj3MqVKzM7aHSJvc27JLdo0aL2mJaWFnfhhRe64uJil5OT4774xS+66urqzA0aXeLrX/+6GzJkiItGo65fv37uuOOOa2+WnGPe+5KPN0zMfe/15S9/2fXv399Fo1E3YMAA9+Uvf9m999577X9n7nu3hx56yI0ZM8bFYjE3evRod/vtt3f4e1e83gs455zv82AAAAAA0Iv1ue8wAQAAAEBn0TABAAAAgAcaJgAAAADwQMMEAAAAAB5omAAAAADAAw0TAAAAAHigYQIAAAAADzRMAAAAAOCBhgkA0CvMnj1bM2bMyPQwAAC9TDjTAwAA4F8JBAKf+PerrrpKP/vZz+Sc66YRAQD6ChomAECPV11d3f7vP/7xj7ryyiu1cuXK9vvy8vKUl5eXiaEBAHo5PpIHAOjxKisr22+FhYUKBAId7svLy9vjI3nHHHOMLr74Yl1yySUqLi5WRUWFfvnLX6qpqUlnnXWW8vPzNXLkSD3yyCMdar355puaPn268vLyVFFRoa9+9avasWNHNy8xAKCnoGECAPRav/3tb1VWVqYXXnhBF198sS644AKdeuqpOvzww/XKK69o6tSp+upXv6rm5mZJUm1trY499lgdfPDBeumll7RkyRJt3bpVM2fOzPCSAAAyhYYJANBrjR8/Xt///vc1atQozZ07V1lZWSorK9M555yjUaNG6corr9TOnTu1YsUKSdLNN9+sgw8+WNdee61Gjx6tgw8+WL/5zW/05JNPatWqVRleGgBAJvAdJgBArzVu3Lj2f4dCIZWWlmrs2LHt91VUVEiStm3bJkl6/fXX9eSTT+71+1Br1qzRfvvtt49HDADoaWiYAAC9ViQS6fD/QCDQ4b7dV99Lp9OSpMbGRp144om6/vrr93is/v3778ORAgB6KhomAAD+z2c/+1n9+c9/1tChQxUO8xQJAOA7TAAAtJszZ45qamp0+umn68UXX9SaNWv06KOP6qyzzlIqlcr08AAAGUDDBADA/6mqqtLTTz+tVCqlqVOnauzYsbrkkktUVFSkYJCnTADoiwKOn0UHAAAAgL3i7TIAAAAA8EDDBAAAAAAeaJgAAAAAwAMNEwAAAAB4oGECAAAAAA80TAAAAADggYYJAAAAADzQMAEAAACABxomAAAAAPBAwwQAAAAAHmiYAAAAAMDD/w+oPyd85eh0dwAAAABJRU5ErkJggg==\n"},"metadata":{}}],"execution_count":17},{"cell_type":"code","source":"# Testing code to check if your validation data loaders are working\nall = []\nfor i, data in enumerate(val_loader):\n    frames, phoneme = data\n    all.append(phoneme)\n    break","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:13.731015Z","iopub.execute_input":"2026-05-25T04:27:13.731249Z","iopub.status.idle":"2026-05-25T04:27:13.760545Z","shell.execute_reply.started":"2026-05-25T04:27:13.731229Z","shell.execute_reply":"2026-05-25T04:27:13.759766Z"},"id":"dJTrLe7J5dSc","trusted":true},"outputs":[],"execution_count":18},{"cell_type":"markdown","source":"# Network Architecture\n","metadata":{"id":"Nxjwve20JRJ2"}},{"cell_type":"markdown","source":"This section defines your network architecture for the homework. We have given you a sample architecture that can easily clear the very low cutoff for the early submission deadline.","metadata":{"id":"3NJzT-mRw6iy"}},{"cell_type":"code","source":"# This architecture will make you cross the very low cutoff\n# However, you need to run a lot of experiments to cross the medium or high cutoff\n\nclass Network(nn.Module):\n    def __init__(self, input_size, output_size):\n        super(Network, self).__init__()\n        self.model = nn.Sequential(\n            nn.Linear(input_size, 512),\n            nn.BatchNorm1d(512),\n            nn.GELU(),\n            nn.Dropout(config['dropout']),\n            nn.Linear(512, 768),\n            nn.BatchNorm1d(768),\n            nn.GELU(),\n            nn.Dropout(config['dropout']),\n            nn.Linear(768, 1024),\n            nn.BatchNorm1d(1024),\n            nn.GELU(),\n            nn.Dropout(config['dropout']),\n            nn.Linear(1024, 512),\n            nn.BatchNorm1d(512),\n            nn.GELU(),\n            nn.Dropout(config['dropout']),\n            nn.Linear(512, output_size),\n        )\n\n        if config['weight_initialization'] is not None:\n            self.initialize_weights()\n\n    def initialize_weights(self):\n        for m in self.modules():\n            if isinstance(m, torch.nn.Linear):\n                if config[\"weight_initialization\"] == \"xavier_normal\":\n                    torch.nn.init.xavier_normal_(m.weight)\n                elif config[\"weight_initialization\"] == \"xavier_uniform\":\n                    torch.nn.init.xavier_uniform_(m.weight)\n                elif config[\"weight_initialization\"] == \"kaiming_normal\":\n                    torch.nn.init.kaiming_normal_(m.weight, nonlinearity='relu')\n                elif config[\"weight_initialization\"] == \"kaiming_uniform\":\n                    torch.nn.init.kaiming_uniform_(m.weight, nonlinearity='relu')\n                elif config[\"weight_initialization\"] == \"uniform\":\n                    torch.nn.init.uniform_(m.weight)\n                else:\n                    raise ValueError(\"Invalid weight_initialization value\")\n\n                # Initialize bias to 0\n                m.bias.data.fill_(0)\n\n\n    def forward(self, x):\n\n        # Flatten to a 1D vector for each data point\n        x = torch.flatten(x, start_dim=1)  # Keeps batch size, flattens the rest\n\n        return self.model(x)\nprint(\"Done\")","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:13.761406Z","iopub.execute_input":"2026-05-25T04:27:13.761703Z","iopub.status.idle":"2026-05-25T04:27:13.769665Z","shell.execute_reply.started":"2026-05-25T04:27:13.761657Z","shell.execute_reply":"2026-05-25T04:27:13.768946Z"},"id":"-YsMpN-Exafq","trusted":true},"outputs":[{"name":"stdout","text":"Done\n","output_type":"stream"}],"execution_count":19},{"cell_type":"markdown","source":"# Define Model, Loss Function and Optimizer","metadata":{"id":"HejoSXe3vMVU"}},{"cell_type":"markdown","source":"Here we define the model, loss function, optimizer and optionally a learning rate scheduler.","metadata":{"id":"xAhGBH7-xxth"}},{"cell_type":"code","source":"from torchinfo import summary\n\nINPUT_SIZE = (2 * config['context'] + 1) * 28 \n\n# Initialize your network layout\nmodel = Network(INPUT_SIZE, len(train_data.phonemes)).to(device)\n\n# Grab a single batch from your dataloader to test shapes\nframes, phonemes = next(iter(train_loader))\n\n# Generate the model summary cleanly\nsummary(model, input_data=frames.to(device))","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:13.773282Z","iopub.execute_input":"2026-05-25T04:27:13.773499Z","iopub.status.idle":"2026-05-25T04:27:19.340152Z","shell.execute_reply.started":"2026-05-25T04:27:13.77348Z","shell.execute_reply":"2026-05-25T04:27:19.339214Z"},"id":"_qtrEM1ZvLje","trusted":true},"outputs":[{"execution_count":20,"output_type":"execute_result","data":{"text/plain":"==========================================================================================\nLayer (type:depth-idx)                   Output Shape              Param #\n==========================================================================================\nNetwork                                  [2048, 40]                --\n├─Sequential: 1-1                        [2048, 40]                --\n│    └─Linear: 2-1                       [2048, 512]               875,008\n│    └─BatchNorm1d: 2-2                  [2048, 512]               1,024\n│    └─GELU: 2-3                         [2048, 512]               --\n│    └─Dropout: 2-4                      [2048, 512]               --\n│    └─Linear: 2-5                       [2048, 768]               393,984\n│    └─BatchNorm1d: 2-6                  [2048, 768]               1,536\n│    └─GELU: 2-7                         [2048, 768]               --\n│    └─Dropout: 2-8                      [2048, 768]               --\n│    └─Linear: 2-9                       [2048, 1024]              787,456\n│    └─BatchNorm1d: 2-10                 [2048, 1024]              2,048\n│    └─GELU: 2-11                        [2048, 1024]              --\n│    └─Dropout: 2-12                     [2048, 1024]              --\n│    └─Linear: 2-13                      [2048, 512]               524,800\n│    └─BatchNorm1d: 2-14                 [2048, 512]               1,024\n│    └─GELU: 2-15                        [2048, 512]               --\n│    └─Dropout: 2-16                     [2048, 512]               --\n│    └─Linear: 2-17                      [2048, 40]                20,520\n==========================================================================================\nTotal params: 2,607,400\nTrainable params: 2,607,400\nNon-trainable params: 0\nTotal mult-adds (G): 5.34\n==========================================================================================\nInput size (MB): 13.99\nForward/backward pass size (MB): 92.93\nParams size (MB): 10.43\nEstimated Total Size (MB): 117.35\n=========================================================================================="},"metadata":{}}],"execution_count":20},{"cell_type":"markdown","source":"# Define Model, Loss Function and Optimizer","metadata":{"id":"uSk4YP63TBnD"}},{"cell_type":"markdown","source":"Here we define the model, loss function, optimizer and optionally a learning rate scheduler.","metadata":{"id":"cwdxl4RiTBnD"}},{"cell_type":"code","source":"#loss function is cross entropy\ncriterion = nn.CrossEntropyLoss()\n\noptimizer = torch.optim.AdamW(\n    model.parameters(),\n    lr=config[\"learning_rate\"],\n    weight_decay=config[\"weight_decay\"]\n)\nscheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(\n    optimizer,\n    mode=\"max\",\n    factor=0.5,\n    patience=2\n)\n\nscaler = torch.amp.GradScaler(\"cuda\", enabled=(device == \"cuda\"))","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:19.341889Z","iopub.execute_input":"2026-05-25T04:27:19.342132Z","iopub.status.idle":"2026-05-25T04:27:21.311076Z","shell.execute_reply.started":"2026-05-25T04:27:19.342112Z","shell.execute_reply":"2026-05-25T04:27:21.310361Z"},"id":"kZgQ7AgyTBnE","trusted":true},"outputs":[],"execution_count":21},{"cell_type":"markdown","source":"# Training and Validation Functions","metadata":{"id":"IBwunYpyugFg"}},{"cell_type":"markdown","source":"This section covers the training, and validation functions for each epoch of running your experiment with a given model architecture. The code has been provided to you, but we recommend going through the comments to understand the workflow to enable you to write these loops for future HWs.","metadata":{"id":"1JgeNhx4x2-P"}},{"cell_type":"code","source":"# CLEAR RAM!!\ntorch.cuda.empty_cache()\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:21.31184Z","iopub.execute_input":"2026-05-25T04:27:21.312257Z","iopub.status.idle":"2026-05-25T04:27:21.477342Z","shell.execute_reply.started":"2026-05-25T04:27:21.312236Z","shell.execute_reply":"2026-05-25T04:27:21.476531Z"},"id":"XblOHEVtKab2","trusted":true},"outputs":[{"execution_count":22,"output_type":"execute_result","data":{"text/plain":"107"},"metadata":{}}],"execution_count":22},{"cell_type":"code","source":"def train(model, dataloader, optimizer, criterion):\n\n    model.train()\n    tloss, tacc = 0, 0 # Monitoring loss and accuracy\n    batch_bar   = tqdm(total=len(train_loader), dynamic_ncols=True, leave=False, position=0, desc='Train')\n\n    for i, (frames, phonemes) in enumerate(dataloader):\n\n        ### Initialize Gradients\n        optimizer.zero_grad()\n\n        frames      = frames.to(device)\n        phonemes    = phonemes.to(device)\n\n        with torch.autocast(device_type=device, dtype=torch.float16):\n            ### Forward Propagation\n            logits  = model(frames)\n\n            ### Loss Calculation\n            loss    = criterion(logits, phonemes)\n\n        ### Backward Propagation\n        scaler.scale(loss).backward()\n\n        # OPTIONAL: You can add gradient clipping here, if you face issues of exploding gradients\n\n        ### Gradient Descent\n        scaler.step(optimizer)\n        scaler.update()\n\n        tloss   += loss.item()\n        tacc    += torch.sum(torch.argmax(logits, dim= 1) == phonemes).item()/logits.shape[0]\n\n        batch_bar.set_postfix(loss=\"{:.04f}\".format(float(tloss / (i + 1))),\n                              acc=\"{:.04f}%\".format(float(tacc*100 / (i + 1))))\n        batch_bar.update()\n\n        ### Release memory\n        del frames, phonemes, logits\n        torch.cuda.empty_cache()\n\n\n    batch_bar.close()\n    tloss   /= len(train_loader)\n    tacc    /= len(train_loader)\n\n\n    return tloss, tacc","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:21.478184Z","iopub.execute_input":"2026-05-25T04:27:21.478513Z","iopub.status.idle":"2026-05-25T04:27:21.485272Z","shell.execute_reply.started":"2026-05-25T04:27:21.478491Z","shell.execute_reply":"2026-05-25T04:27:21.484383Z"},"id":"8wjPz7DHqKcL","trusted":true},"outputs":[],"execution_count":23},{"cell_type":"code","source":"def eval(model, dataloader):\n\n    model.eval() # set model in evaluation mode\n    vloss, vacc = 0, 0 # Monitoring loss and accuracy\n    batch_bar   = tqdm(total=len(val_loader), dynamic_ncols=True, position=0, leave=False, desc='Val')\n\n    for i, (frames, phonemes) in enumerate(dataloader):\n\n        ### Move data to device (ideally GPU)\n        frames      = frames.to(device)\n        phonemes    = phonemes.to(device)\n\n        # makes sure that there are no gradients computed as we are not training the model now\n        with torch.inference_mode():\n            ### Forward Propagation\n            logits  = model(frames)\n            ### Loss Calculation\n            loss    = criterion(logits, phonemes)\n\n        vloss   += loss.item()\n        vacc    += torch.sum(torch.argmax(logits, dim= 1) == phonemes).item()/logits.shape[0]\n\n        # Do you think we need loss.backward() and optimizer.step() here?\n\n        batch_bar.set_postfix(loss=\"{:.04f}\".format(float(vloss / (i + 1))),\n                              acc=\"{:.04f}%\".format(float(vacc*100 / (i + 1))))\n        batch_bar.update()\n\n        ### Release memory\n        del frames, phonemes, logits\n        torch.cuda.empty_cache()\n\n    batch_bar.close()\n    vloss   /= len(val_loader)\n    vacc    /= len(val_loader)\n\n    return vloss, vacc","metadata":{"execution":{"iopub.status.busy":"2026-05-25T04:27:21.48621Z","iopub.execute_input":"2026-05-25T04:27:21.486458Z","iopub.status.idle":"2026-05-25T04:27:21.5029Z","shell.execute_reply.started":"2026-05-25T04:27:21.486437Z","shell.execute_reply":"2026-05-25T04:27:21.5021Z"},"id":"Q5npQNFH315V","trusted":true},"outputs":[],"execution_count":24},{"cell_type":"markdown","source":"# Weights and Biases Setup","metadata":{"id":"yMd_XxPku5qp"}},{"cell_type":"markdown","source":"This section is to enable logging metrics and files with Weights and Biases. Please refer to wandb documentationa and recitation 0 that covers the use of weights and biases for logging, hyperparameter tuning and monitoring your runs for your homeworks. Using this tool makes it very easy to show results when submitting your code and models for homeworks, and also extremely useful for study groups to organize and run ablations under a single team in wandb.\n\nWe have written code for you to make use of it out of the box, so that you start using wandb for all your HWs from the beginning.","metadata":{"id":"tjIbhR1wwbgI"}},{"cell_type":"code","source":"!pip uninstall -y wandb\n!pip install wandb==0.16.6","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:29:38.418574Z","iopub.execute_input":"2026-05-25T04:29:38.418979Z","iopub.status.idle":"2026-05-25T04:29:46.765423Z","shell.execute_reply.started":"2026-05-25T04:29:38.418952Z","shell.execute_reply":"2026-05-25T04:29:46.764518Z"}},"outputs":[{"name":"stdout","text":"Found existing installation: wandb 0.27.0\nUninstalling wandb-0.27.0:\n  Successfully uninstalled wandb-0.27.0\nCollecting wandb==0.16.6\n  Downloading wandb-0.16.6-py3-none-any.whl.metadata (10 kB)\nRequirement already satisfied: Click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (8.4.1)\nRequirement already satisfied: GitPython!=3.1.29,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (3.1.43)\nRequirement already satisfied: requests<3,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (2.32.3)\nRequirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (5.9.5)\nRequirement already satisfied: sentry-sdk>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (2.19.2)\nRequirement already satisfied: docker-pycreds>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (0.4.0)\nRequirement already satisfied: PyYAML in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (6.0.2)\nRequirement already satisfied: setproctitle in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (1.3.4)\nRequirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from wandb==0.16.6) (75.1.0)\nCollecting appdirs>=1.4.3 (from wandb==0.16.6)\n  Downloading appdirs-1.4.4-py2.py3-none-any.whl.metadata (9.0 kB)\nCollecting protobuf!=4.21.0,<5,>=3.19.0 (from wandb==0.16.6)\n  Downloading protobuf-4.25.9-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)\nRequirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from docker-pycreds>=0.4.0->wandb==0.16.6) (1.17.0)\nRequirement already satisfied: gitdb<5,>=4.0.1 in /usr/local/lib/python3.10/dist-packages (from GitPython!=3.1.29,>=1.0.0->wandb==0.16.6) (4.0.11)\nRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb==0.16.6) (3.4.0)\nRequirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb==0.16.6) (3.10)\nRequirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb==0.16.6) (2.2.3)\nRequirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb==0.16.6) (2024.12.14)\nRequirement already satisfied: smmap<6,>=3.0.1 in /usr/local/lib/python3.10/dist-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb==0.16.6) (5.0.1)\nDownloading wandb-0.16.6-py3-none-any.whl (2.2 MB)\n\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.2/2.2 MB\u001b[0m \u001b[31m39.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n\u001b[?25hDownloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)\nDownloading protobuf-4.25.9-cp37-abi3-manylinux2014_x86_64.whl (295 kB)\n\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m295.2/295.2 kB\u001b[0m \u001b[31m19.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n\u001b[?25hInstalling collected packages: appdirs, protobuf, wandb\n  Attempting uninstall: protobuf\n    Found existing installation: protobuf 7.35.0\n    Uninstalling protobuf-7.35.0:\n      Successfully uninstalled protobuf-7.35.0\n\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\ngoogle-api-core 1.34.1 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<4.0.0dev,>=3.19.5, but you have protobuf 4.25.9 which is incompatible.\ngoogle-cloud-bigtable 2.27.0 requires google-api-core[grpc]<3.0.0dev,>=2.16.0, but you have google-api-core 1.34.1 which is incompatible.\npandas-gbq 0.25.0 requires google-api-core<3.0.0dev,>=2.10.2, but you have google-api-core 1.34.1 which is incompatible.\ntensorflow-decision-forests 1.10.0 requires tensorflow==2.17.0, but you have tensorflow 2.17.1 which is incompatible.\u001b[0m\u001b[31m\n\u001b[0mSuccessfully installed appdirs-1.4.4 protobuf-4.25.9 wandb-0.16.6\n","output_type":"stream"}],"execution_count":29},{"cell_type":"code","source":"import wandb\nwandb.login()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:29:51.254886Z","iopub.execute_input":"2026-05-25T04:29:51.25522Z","iopub.status.idle":"2026-05-25T04:29:56.853375Z","shell.execute_reply.started":"2026-05-25T04:29:51.255194Z","shell.execute_reply":"2026-05-25T04:29:56.85128Z"}},"outputs":[{"traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mImportError\u001b[0m                               Traceback (most recent call last)","\u001b[0;32m<ipython-input-30-ae6b39ac68ed>\u001b[0m in \u001b[0;36m<cell line: 2>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mwandb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mwandb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlogin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/wandb/sdk/wandb_login.py\u001b[0m in \u001b[0;36mlogin\u001b[0;34m(anonymous, key, relogin, host, force, timeout, verify)\u001b[0m\n\u001b[1;32m     74\u001b[0m         \u001b[0mbool\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mkey\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mconfigured\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     75\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 76\u001b[0;31m     \u001b[0mRaises\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     77\u001b[0m         \u001b[0mAuthenticationError\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mapi_key\u001b[0m \u001b[0mfails\u001b[0m \u001b[0mverification\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mserver\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     78\u001b[0m         \u001b[0mUsageError\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mapi_key\u001b[0m \u001b[0mcannot\u001b[0m \u001b[0mbe\u001b[0m \u001b[0mconfigured\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mno\u001b[0m \u001b[0mtty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/wandb/sdk/wandb_setup.py\u001b[0m in \u001b[0;36msetup\u001b[0;34m(settings)\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/wandb/sdk/wandb_setup.py\u001b[0m in \u001b[0;36m_setup\u001b[0;34m(settings, _reset)\u001b[0m\n\u001b[1;32m    321\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mwl\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    322\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 323\u001b[0;31m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    324\u001b[0m def setup(\n\u001b[1;32m    325\u001b[0m     \u001b[0msettings\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mOptional\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mSettings\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/wandb/sdk/wandb_setup.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, settings)\u001b[0m\n\u001b[1;32m    299\u001b[0m         \u001b[0mpid\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mos\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetpid\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    300\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0m_WandbSetup\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_instance\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0m_WandbSetup\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_instance\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_pid\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mpid\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 301\u001b[0;31m             \u001b[0m_WandbSetup\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_instance\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_update\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msettings\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msettings\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    302\u001b[0m             \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    303\u001b[0m         \u001b[0m_WandbSetup\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_instance\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_WandbSetup__WandbSetup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msettings\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msettings\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpid\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpid\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/wandb/sdk/wandb_setup.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, pid, settings, environ)\u001b[0m\n\u001b[1;32m    113\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_check\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    114\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_setup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 115\u001b[0;31m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    116\u001b[0m         \u001b[0mtracelog_mode\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_settings\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tracelog\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    117\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mtracelog_mode\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/usr/local/lib/python3.10/dist-packages/wandb/sdk/wandb_setup.py\u001b[0m in \u001b[0;36m_setup\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    243\u001b[0m                 \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    244\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0mthreading\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_thread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m\"MainThread\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 245\u001b[0;31m             \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"bad thread2\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mthreading\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcurrent_thread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    246\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msys\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"frozen\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    247\u001b[0m             \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"frozen, could be trouble\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mImportError\u001b[0m: cannot import name 'service_connection' from 'wandb.sdk.lib' (/usr/local/lib/python3.10/dist-packages/wandb/sdk/lib/__init__.py)"],"ename":"ImportError","evalue":"cannot import name 'service_connection' from 'wandb.sdk.lib' (/usr/local/lib/python3.10/dist-packages/wandb/sdk/lib/__init__.py)","output_type":"error"}],"execution_count":30},{"cell_type":"code","source":"# Create your wandb run\nrun = wandb.init(\n    name    = \"first-run1\", ### Wandb creates random run names if you skip this field, we recommend you give useful names\n    reinit  = True, ### Allows reinitalizing runs when you re-run this cell\n    #id     = \"\", ### Insert specific run id here if you want to resume a previous run\n    #resume = \"must\", ### You need this to resume previous runs, but comment out reinit = True when using this\n    project = \"hw1p2\", ### Project should be created in your wandb account\n    config  = config ### Wandb Config for your run\n)","metadata":{"id":"xvUnYd3Bw2up","trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:29:56.854008Z","iopub.status.idle":"2026-05-25T04:29:56.854364Z","shell.execute_reply":"2026-05-25T04:29:56.854234Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"### Save your model architecture as a string with str(model)\nmodel_arch  = str(model)\n\n### Save it in a txt file\narch_file   = open(\"model_arch.txt\", \"w\")\nfile_write  = arch_file.write(model_arch)\narch_file.close()\n\n### log it in your wandb run with wandb.save()\nwandb.save('model_arch.txt')","metadata":{"id":"wft15E_IxYFi","trusted":true,"execution":{"iopub.status.busy":"2026-05-25T04:29:56.855321Z","iopub.status.idle":"2026-05-25T04:29:56.855589Z","shell.execute_reply":"2026-05-25T04:29:56.855486Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Experiment","metadata":{"id":"nclx_04fu7Dd"}},{"cell_type":"markdown","source":"Now, it is time to finally run your ablations! Have fun!","metadata":{"id":"MdLMWfEpyGOB"}},{"cell_type":"code","source":"# Clear cache\ntorch.cuda.empty_cache()\ngc.collect()\n\nbest_val_acc = 0.0  # for saving best model\n\nepochs = 4  # training for 5 epochs only\n\nfor epoch in range(epochs):\n\n    print(\"\\nEpoch {}/{}\".format(epoch+1, epochs))\n\n    curr_lr = float(optimizer.param_groups[0]['lr'])\n\n    train_loss, train_acc = train(model, train_loader, optimizer, criterion)\n    val_loss, val_acc = eval(model, val_loader)\n\n    print(\"\\tTrain Acc {:.04f}%\\tTrain Loss {:.04f}\\t Learning Rate {:.07f}\".format(\n        train_acc*100, train_loss, curr_lr))\n\n    print(\"\\tVal Acc {:.04f}%\\tVal Loss {:.04f}\".format(\n        val_acc*100, val_loss))\n\n    # Scheduler step\n    if scheduler is not None:\n        try:\n            scheduler.step(val_acc)\n        except:\n            scheduler.step()\n\n    # Save best model\n    if val_acc > best_val_acc:\n        best_val_acc = val_acc\n\n        torch.save({\n            'model_state_dict': model.state_dict(),\n            'optimizer_state_dict': optimizer.state_dict(),\n            'epoch': epoch,\n            'val_acc': val_acc\n        }, \"best_model.pth\")\n\n        print(\"Best model saved!\")","metadata":{"id":"4NNCA5DDTBnO","trusted":true,"execution":{"iopub.status.busy":"2026-05-25T05:49:37.00399Z","iopub.execute_input":"2026-05-25T05:49:37.0043Z"}},"outputs":[{"name":"stdout","text":"\nEpoch 1/4\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"Train:   0%|          | 0/17623 [00:00<?, ?it/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4b14d23eeb9f4ba7ab79f7397a1a14f7"}},"metadata":{}}],"execution_count":null},{"cell_type":"markdown","source":"# Testing and submission to Kaggle","metadata":{"id":"_kXwf5YUo_4A"}},{"cell_type":"markdown","source":"Before we get to the following code, make sure to see the format of submission given in *sample_submission.csv*. Once you have done so, it is time to fill the following function to complete your inference on test data. Refer the eval function from previous cells to get an idea of how to go about completing this function.","metadata":{"id":"WI1hSFYLpJvH"}},{"cell_type":"code","source":"def test(model, test_loader):\n\n    # evaluation mode\n    model.eval()\n\n    test_predictions = []\n\n    # disable gradients\n    with torch.inference_mode():\n\n        for i, mfccs in enumerate(tqdm(test_loader)):\n\n            mfccs = mfccs.to(device)\n\n            # forward pass\n            logits = model(mfccs)\n\n            # predicted class indices\n            predicted_indices = torch.argmax(logits, dim=1)\n\n            # convert tensor -> cpu -> numpy\n            predicted_indices = predicted_indices.cpu().numpy()\n\n            # convert indices to phoneme labels\n            predicted_phonemes = [\n                PHONEMES[idx] for idx in predicted_indices\n            ]\n\n            # store predictions\n            test_predictions.extend(predicted_phonemes)\n\n    return test_predictions","metadata":{"id":"ijLrIJFl5dSf","trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nos.environ[\"WANDB_DISABLED\"] = \"true\"\n\ntry:\n    import wandb\n    wandb.finish()\nexcept:\n    pass\n\nfor module in model.modules():\n    module._forward_hooks.clear()","metadata":{"id":"wG9v6Xmxu7wp","trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.eval()\n\npredictions = []\n\nwith torch.no_grad():\n    for mfccs in tqdm(test_loader):\n\n        mfccs = mfccs.to(device)\n\n        logits = model(mfccs)\n\n        pred = torch.argmax(logits, dim=1)\n\n        predictions.extend(pred.cpu().numpy())\n\nprint(\"Total predictions:\", len(predictions))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"### Create CSV file with predictions\nwith open(\"./submission.csv\", \"w+\") as f:\n    f.write(\"id,label\\n\")\n    for i in range(len(predictions)):\n        f.write(\"{},{}\\n\".format(i, predictions[i]))\n\nprint(\"done\")","metadata":{"id":"_I6AVEY45dSg","trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"### Finish your wandb run\nrun.finish()","metadata":{"id":"6Wf-P25TXU0N","trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\n\nsubmission = pd.DataFrame({\n    \"id\": np.arange(len(predictions)),\n    \"label\": predictions\n})\n\nsubmission.to_csv(\"/content/submission.csv\", index=False)\n\nprint(\"CSV Saved Successfully\")","metadata":{"id":"LjcammuCxMKN","trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!kaggle competitions submit -c 11785-hw1p2-f24 -f /content/submission.csv -m \"Test Submission\"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}