{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# ๐Ÿ“š NLP (Natural Language Processing) with Python\n\n***\n\nNatural language processing (NLP) is a subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data.\n\nIn this article, we will discuss a higher-level overview of the basics of Natural Language Processing, which basically consists of combining machine learning techniques with text, and using math and statistics to get that text in a format that the machine learning algorithms can understand!\n\n# ๐Ÿ“ Agenda\n\n> 1. Representing text as numerical data\n> 2. Reading a text-based dataset into pandas\n> 3. Vectorizing our dataset\n> 4. Building and evaluating a model\n> 5. Comparing models\n> 6. Examining a model for further insight\n> 7. Practicing this workflow on another dataset\n> 8. Tuning the vectorizer (discussion)\n\n---\n\n# ๐Ÿ“Œ Notebook Goals\n> In this notebook we will discuss a higher level overview of the basics of Natural Language Processing, which basically consists of combining machine learning techniques with text, and using math and statistics to get that text in a format that the machine learning algorithms can understand!","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n%matplotlib inline\nsns.set_style(\"whitegrid\")\nplt.style.use(\"fivethirtyeight\")","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:29.245566Z","iopub.execute_input":"2023-05-14T15:07:29.245911Z","iopub.status.idle":"2023-05-14T15:07:29.253611Z","shell.execute_reply.started":"2023-05-14T15:07:29.245889Z","shell.execute_reply":"2023-05-14T15:07:29.25262Z"},"trusted":true},"execution_count":7,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ” Representing text as numerical data","metadata":{}},{"cell_type":"code","source":"# example text for model training (SMS messages)\nsimple_train = ['call you tonight', 'Call me a cab', 'Please call me... PLEASE!']","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:33.142477Z","iopub.execute_input":"2023-05-14T15:07:33.142793Z","iopub.status.idle":"2023-05-14T15:07:33.148545Z","shell.execute_reply.started":"2023-05-14T15:07:33.142773Z","shell.execute_reply":"2023-05-14T15:07:33.147225Z"},"trusted":true},"execution_count":8,"outputs":[]},{"cell_type":"markdown","source":"๐Ÿ“Œ From the [scikit-learn documentation](http://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction):\n\n> Text Analysis is a major application field for machine learning algorithms. However the raw data, a sequence of symbols cannot be fed directly to the algorithms themselves as most of them expect **numerical feature vectors with a fixed size** rather than the **raw text documents with variable length**.\n\nWe will use [CountVectorizer](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html) to \"convert text into a matrix of token counts\":","metadata":{}},{"cell_type":"code","source":"# import and instantiate CountVectorizer (with the default parameters)\nfrom sklearn.feature_extraction.text import CountVectorizer\n\nvect = CountVectorizer()\n\n# learn the 'vocabulary' of the training data (occurs in-place)\nvect.fit(simple_train)\n\n# examine the fitted vocabulary\nvect.get_feature_names_out()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:35.343151Z","iopub.execute_input":"2023-05-14T15:07:35.343456Z","iopub.status.idle":"2023-05-14T15:07:35.351778Z","shell.execute_reply.started":"2023-05-14T15:07:35.343436Z","shell.execute_reply":"2023-05-14T15:07:35.350486Z"},"trusted":true},"execution_count":9,"outputs":[{"execution_count":9,"output_type":"execute_result","data":{"text/plain":"array(['cab', 'call', 'me', 'please', 'tonight', 'you'], dtype=object)"},"metadata":{}}]},{"cell_type":"code","source":"# transform training data into a 'document-term matrix'\nsimple_train_dtm = vect.transform(simple_train)\nsimple_train_dtm","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:47.582338Z","iopub.execute_input":"2023-05-14T15:07:47.582691Z","iopub.status.idle":"2023-05-14T15:07:47.589561Z","shell.execute_reply.started":"2023-05-14T15:07:47.582667Z","shell.execute_reply":"2023-05-14T15:07:47.588631Z"},"trusted":true},"execution_count":10,"outputs":[{"execution_count":10,"output_type":"execute_result","data":{"text/plain":"<3x6 sparse matrix of type ''\n\twith 9 stored elements in Compressed Sparse Row format>"},"metadata":{}}]},{"cell_type":"code","source":"# convert sparse matrix to a dense matrix\nsimple_train_dtm.toarray()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:57.29576Z","iopub.execute_input":"2023-05-14T15:07:57.296123Z","iopub.status.idle":"2023-05-14T15:07:57.304121Z","shell.execute_reply.started":"2023-05-14T15:07:57.2961Z","shell.execute_reply":"2023-05-14T15:07:57.302726Z"},"trusted":true},"execution_count":13,"outputs":[{"execution_count":13,"output_type":"execute_result","data":{"text/plain":"array([[0, 1, 0, 0, 1, 1],\n [1, 1, 1, 0, 0, 0],\n [0, 1, 1, 2, 0, 0]])"},"metadata":{}}]},{"cell_type":"code","source":"# examine the vocabulary and document-term matrix together\npd.DataFrame(simple_train_dtm.toarray(), columns=vect.get_feature_names_out())","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:13:11.789916Z","iopub.execute_input":"2023-05-14T15:13:11.79028Z","iopub.status.idle":"2023-05-14T15:13:11.831115Z","shell.execute_reply.started":"2023-05-14T15:13:11.790257Z","shell.execute_reply":"2023-05-14T15:13:11.828601Z"},"trusted":true},"execution_count":15,"outputs":[{"execution_count":15,"output_type":"execute_result","data":{"text/plain":" cab call me please tonight you\n0 0 1 0 0 1 1\n1 1 1 1 0 0 0\n2 0 1 1 2 0 0","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
cabcallmepleasetonightyou
0010011
1111000
2011200
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"๐Ÿ“Œ From the [scikit-learn documentation](http://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction):\n\n> In this scheme, features and samples are defined as follows:\n\n> - Each individual token occurrence frequency (normalized or not) is treated as a **feature**.\n> - The vector of all the token frequencies for a given document is considered a multivariate **sample**.\n\n> A **corpus of documents** can thus be represented by a matrix with **one row per document** and **one column per token** (e.g. word) occurring in the corpus.\n\n> We call **vectorization** the general process of turning a collection of text documents into numerical feature vectors. This specific strategy (tokenization, counting and normalization) is called the **Bag of Words** or \"Bag of n-grams\" representation. Documents are described by word occurrences while completely ignoring the relative position information of the words in the document.","metadata":{}},{"cell_type":"code","source":"# check the type of the document-term matrix\nprint(type(simple_train_dtm))\n\n# examine the sparse matrix contents\nprint(simple_train_dtm)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:13:42.718604Z","iopub.execute_input":"2023-05-14T15:13:42.718934Z","iopub.status.idle":"2023-05-14T15:13:42.725193Z","shell.execute_reply.started":"2023-05-14T15:13:42.718912Z","shell.execute_reply":"2023-05-14T15:13:42.723742Z"},"trusted":true},"execution_count":16,"outputs":[{"name":"stdout","text":"\n (0, 1)\t1\n (0, 4)\t1\n (0, 5)\t1\n (1, 0)\t1\n (1, 1)\t1\n (1, 2)\t1\n (2, 1)\t1\n (2, 2)\t1\n (2, 3)\t2\n","output_type":"stream"}]},{"cell_type":"markdown","source":"๐Ÿ“Œ From the [scikit-learn documentation](http://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction):\n\n> As most documents will typically use a very small subset of the words used in the corpus, the resulting matrix will have **many feature values that are zeros** (typically more than 99% of them).\n\n> For instance, a collection of 10,000 short text documents (such as emails) will use a vocabulary with a size in the order of 100,000 unique words in total while each document will use 100 to 1000 unique words individually.\n\n> In order to be able to **store such a matrix in memory** but also to **speed up operations**, implementations will typically use a **sparse representation** such as the implementations available in the `scipy.sparse` package.","metadata":{}},{"cell_type":"code","source":"# example text for model testing\nsimple_test = [\"please don't call me\"]","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:13:48.205429Z","iopub.execute_input":"2023-05-14T15:13:48.205755Z","iopub.status.idle":"2023-05-14T15:13:48.211432Z","shell.execute_reply.started":"2023-05-14T15:13:48.205734Z","shell.execute_reply":"2023-05-14T15:13:48.209981Z"},"trusted":true},"execution_count":17,"outputs":[]},{"cell_type":"markdown","source":"> In order to **make a prediction**, the new observation must have the **same features as the training observations**, both in number and meaning.","metadata":{}},{"cell_type":"code","source":"# transform testing data into a document-term matrix (using existing vocabulary)\nsimple_test_dtm = vect.transform(simple_test)\nsimple_test_dtm.toarray()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:13:55.71321Z","iopub.execute_input":"2023-05-14T15:13:55.713523Z","iopub.status.idle":"2023-05-14T15:13:55.721274Z","shell.execute_reply.started":"2023-05-14T15:13:55.713503Z","shell.execute_reply":"2023-05-14T15:13:55.719862Z"},"trusted":true},"execution_count":18,"outputs":[{"execution_count":18,"output_type":"execute_result","data":{"text/plain":"array([[0, 1, 1, 1, 0, 0]])"},"metadata":{}}]},{"cell_type":"code","source":"# examine the vocabulary and document-term matrix together\npd.DataFrame(simple_test_dtm.toarray(), columns=vect.get_feature_names_out())","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:14:00.005295Z","iopub.execute_input":"2023-05-14T15:14:00.005678Z","iopub.status.idle":"2023-05-14T15:14:00.015747Z","shell.execute_reply.started":"2023-05-14T15:14:00.005656Z","shell.execute_reply":"2023-05-14T15:14:00.014757Z"},"trusted":true},"execution_count":19,"outputs":[{"execution_count":19,"output_type":"execute_result","data":{"text/plain":" cab call me please tonight you\n0 0 1 1 1 0 0","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
cabcallmepleasetonightyou
0011100
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"## ๐Ÿ“‹ **Summary:**\n\n> - `vect.fit(train)` **learns the vocabulary** of the training data\n> - `vect.transform(train)` uses the **fitted vocabulary** to build a document-term matrix from the training data\n> - `vect.transform(test)` uses the **fitted vocabulary** to build a document-term matrix from the testing data (and **ignores tokens** it hasn't seen before)","metadata":{}},{"cell_type":"markdown","source":"# ๐Ÿ’พ Reading a text-based dataset into pandas","metadata":{}},{"cell_type":"code","source":"# read file into pandas using a relative path\nsms = pd.read_csv(\"/kaggle/input/sms-spam-collection-dataset/spam.csv\", encoding='latin-1')\nsms.dropna(how=\"any\", inplace=True, axis=1)\nsms.columns = ['label', 'message']\n\nsms.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:14:05.713941Z","iopub.execute_input":"2023-05-14T15:14:05.714295Z","iopub.status.idle":"2023-05-14T15:14:05.772028Z","shell.execute_reply.started":"2023-05-14T15:14:05.714274Z","shell.execute_reply":"2023-05-14T15:14:05.770647Z"},"trusted":true},"execution_count":20,"outputs":[{"execution_count":20,"output_type":"execute_result","data":{"text/plain":" label message\n0 ham Go until jurong point, crazy.. Available only ...\n1 ham Ok lar... Joking wif u oni...\n2 spam Free entry in 2 a wkly comp to win FA Cup fina...\n3 ham U dun say so early hor... U c already then say...\n4 ham Nah I don't think he goes to usf, he lives aro...","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
labelmessage
0hamGo until jurong point, crazy.. Available only ...
1hamOk lar... Joking wif u oni...
2spamFree entry in 2 a wkly comp to win FA Cup fina...
3hamU dun say so early hor... U c already then say...
4hamNah I don't think he goes to usf, he lives aro...
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"# ๐Ÿ” Exploratory Data Analysis (EDA)","metadata":{}},{"cell_type":"code","source":"sms.describe()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:14:38.680168Z","iopub.execute_input":"2023-05-14T15:14:38.680519Z","iopub.status.idle":"2023-05-14T15:14:38.701666Z","shell.execute_reply.started":"2023-05-14T15:14:38.680496Z","shell.execute_reply":"2023-05-14T15:14:38.700225Z"},"trusted":true},"execution_count":21,"outputs":[{"execution_count":21,"output_type":"execute_result","data":{"text/plain":" label message\ncount 5572 5572\nunique 2 5169\ntop ham Sorry, I'll call later\nfreq 4825 30","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
labelmessage
count55725572
unique25169
tophamSorry, I'll call later
freq482530
\n
"},"metadata":{}}]},{"cell_type":"code","source":"sms.groupby('label').describe()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:14:41.866017Z","iopub.execute_input":"2023-05-14T15:14:41.866377Z","iopub.status.idle":"2023-05-14T15:14:41.906897Z","shell.execute_reply.started":"2023-05-14T15:14:41.866353Z","shell.execute_reply":"2023-05-14T15:14:41.90561Z"},"trusted":true},"execution_count":22,"outputs":[{"execution_count":22,"output_type":"execute_result","data":{"text/plain":" message \n count unique top freq\nlabel \nham 4825 4516 Sorry, I'll call later 30\nspam 747 653 Please call our customer service representativ... 4","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
message
countuniquetopfreq
label
ham48254516Sorry, I'll call later30
spam747653Please call our customer service representativ...4
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"We have `4825` ham message and `747` spam message","metadata":{}},{"cell_type":"code","source":"# convert label to a numerical variable\nsms['label_num'] = sms.label.map({'ham':0, 'spam':1})\nsms.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:14:47.961217Z","iopub.execute_input":"2023-05-14T15:14:47.961728Z","iopub.status.idle":"2023-05-14T15:14:47.97343Z","shell.execute_reply.started":"2023-05-14T15:14:47.961705Z","shell.execute_reply":"2023-05-14T15:14:47.972034Z"},"trusted":true},"execution_count":23,"outputs":[{"execution_count":23,"output_type":"execute_result","data":{"text/plain":" label message label_num\n0 ham Go until jurong point, crazy.. Available only ... 0\n1 ham Ok lar... Joking wif u oni... 0\n2 spam Free entry in 2 a wkly comp to win FA Cup fina... 1\n3 ham U dun say so early hor... U c already then say... 0\n4 ham Nah I don't think he goes to usf, he lives aro... 0","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
labelmessagelabel_num
0hamGo until jurong point, crazy.. Available only ...0
1hamOk lar... Joking wif u oni...0
2spamFree entry in 2 a wkly comp to win FA Cup fina...1
3hamU dun say so early hor... U c already then say...0
4hamNah I don't think he goes to usf, he lives aro...0
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"> As we continue our analysis we want to start thinking about the features we are going to be using. This goes along with the general idea of feature engineering. The better your domain knowledge on the data, the better your ability to engineer more features from it. Feature engineering is a very large part of spam detection in general.","metadata":{}},{"cell_type":"code","source":"sms['message_len'] = sms.message.apply(len)\nsms.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:14:53.959733Z","iopub.execute_input":"2023-05-14T15:14:53.960046Z","iopub.status.idle":"2023-05-14T15:14:53.971935Z","shell.execute_reply.started":"2023-05-14T15:14:53.960026Z","shell.execute_reply":"2023-05-14T15:14:53.971021Z"},"trusted":true},"execution_count":24,"outputs":[{"execution_count":24,"output_type":"execute_result","data":{"text/plain":" label message label_num \\\n0 ham Go until jurong point, crazy.. Available only ... 0 \n1 ham Ok lar... Joking wif u oni... 0 \n2 spam Free entry in 2 a wkly comp to win FA Cup fina... 1 \n3 ham U dun say so early hor... U c already then say... 0 \n4 ham Nah I don't think he goes to usf, he lives aro... 0 \n\n message_len \n0 111 \n1 29 \n2 155 \n3 49 \n4 61 ","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
labelmessagelabel_nummessage_len
0hamGo until jurong point, crazy.. Available only ...0111
1hamOk lar... Joking wif u oni...029
2spamFree entry in 2 a wkly comp to win FA Cup fina...1155
3hamU dun say so early hor... U c already then say...049
4hamNah I don't think he goes to usf, he lives aro...061
\n
"},"metadata":{}}]},{"cell_type":"code","source":"plt.figure(figsize=(12, 8))\n\nsms[sms.label=='ham'].message_len.plot(bins=35, kind='hist', color='blue', \n label='Ham messages', alpha=0.6)\nsms[sms.label=='spam'].message_len.plot(kind='hist', color='red', \n label='Spam messages', alpha=0.6)\nplt.legend()\nplt.xlabel(\"Message Length\")","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:15:34.831787Z","iopub.execute_input":"2023-05-14T15:15:34.833128Z","iopub.status.idle":"2023-05-14T15:15:35.332854Z","shell.execute_reply.started":"2023-05-14T15:15:34.83308Z","shell.execute_reply":"2023-05-14T15:15:35.331504Z"},"trusted":true},"execution_count":25,"outputs":[{"execution_count":25,"output_type":"execute_result","data":{"text/plain":"Text(0.5, 0, 'Message Length')"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"
","image/png":"iVBORw0KGgoAAAANSUhEUgAABHoAAALQCAYAAADvmBfHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAACE0klEQVR4nOzdd3hUVeL/8c+kkYoBAiGNLotUKYYgKiC9CSJFQZAqqKC7q6zuqrDIqmv7qruUGJqAiIJSpCmiQeklNAlFOgFCIEACExJSZn5/8Mss40wgMGFCLu/X8/hgzj33nnOHHCUfTjGlpaVZBQAAAAAAgBLPo7g7AAAAAAAAgKJB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQgxvKysrS4cOHlZWVVdxdAUokxhDgGsYQ4BrGEOAaxhBKGoIeFEpeXl5xdwEo0RhDgGsYQ4BrGEOAaxhDKEkIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADMKruDsAAAAAAHA/i8WijIwMjg2/AYvFIh8fH6Wnp+vSpUvF3R0YjK+vrwICAuThUXTzcAh6AAAAAOAuY7FYdO7cOQUGBiokJEQmk6m4u3THslgsys7Olo+PT5H+MA5YrVZlZWXp3LlzKleuXJF9f/FdCgAAAAB3mYyMDAUGBsrPz4+QBygmJpNJfn5+CgwMVEZGRpE9l6AHAAAAAO4yWVlZ8vX1Le5uANDV5VtFuYSSoAcAAAAA7kLM5AHuDEU9Fgl6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgvIq7A4WxYsUK7dixQ3v37tWhQ4eUk5OjMWPGqEuXLg51o6Ojb/i8JUuWKDQ0VJJ06tQpde/evcC6Q4cO1bPPPutQnpqaqsmTJ2v9+vW6dOmSKlasqE6dOmnAgAHy8ioRHysAAAAAOJWSYlJq6p25WXNIiFWhodZbvj//Z8CYmBj95z//cVonISFBzz33nB5//HG9+uqrt9wWUBxKRCIRGxur5ORkBQcHKyQkRMnJyQXWHTp0qNPyEydO6Pvvv1fVqlVtIc+17r33XrVo0cKhvHHjxg5lqampGjRokM6cOaOWLVsqKipK27ZtU2xsrPbs2aMPPviAHewBAAAAlFipqSZNmeJT3N1watiwbJeCHsDoSkTQ8/rrrysqKkphYWGaOXOmJk6cWGBdZ7NvJOmDDz6QJD322GNOr9esWbPAe/9owoQJSklJ0auvvqonnnhCkmS1WvXmm29q5cqVWrlypdq3b1+oZwEAAAAAABSVEhH0FGY51vVcuXJFP/zwg7y9vdWpUyeXnpWRkaFVq1YpIiJCPXr0sJWbTCa98MILWrlypRYtWkTQAwAAAAAGs3fvXi1dulQJCQlKSUlRbm6uIiMj1aFDB/Xr189hG49u3bpJkubMmaMJEybol19+UUZGhmrVqqW//vWvqlWrls6ePav//ve/2rRpky5fvqz7779fo0ePVqVKlQrVpxEjRmjbtm1au3atpk6dqhUrVigtLU1Vq1bVyJEjFR0dLbPZrEmTJmn16tW6ePGiatasqZdffll16tRxeN758+f1+eefa+3atUpJSZG/v78aNWqkZ599VtWrV7ere/z4cX3++edKSEhQamqq/Pz8VKFCBTVp0kR/+ctfbCtdUlNTNXPmTK1fv15nzpyRt7e3QkJC1LBhQ40aNUqBgYGSpGPHjmnx4sXasmWLkpOTlZWVpdDQULVq1UqDBw+Wv7+/Q38PHDigiRMnaseOHTKZTGrQoIFGjRqlL774QsuWLdOiRYsUHh5ud88vv/yir7/+Wvv379eVK1cUGRmpLl266KmnnpKnp6etnsVi0XfffadFixbpxIkTunLliu655x7VrFlT/fr1c7oC6E5QIoIeV8XHx+vixYt69NFHVaZMGad1UlNTNX/+fJnNZpUtW1aNGzdWZGSkQ73ffvtN2dnZio6OdlieFRYWpsqVK2vXrl3Ky8uz+wYBAAAAAJRsixYt0tq1a9WwYUM9+OCDunLlihISEjRx4kTt2bNH7733nsM9OTk5GjlypLKzs9W2bVudP39eq1at0siRIzV16lS9+OKLCgkJUYcOHXTixAmtWbNGf/nLXzRv3ryb+pnyH//4hw4dOqRHHnlEWVlZ+uGHH/TXv/5VU6ZM0bvvvqucnBy1bt1aFy5c0KpVq/TSSy9p0aJFtpBFurrlyYgRI3TmzBk1bdpULVq00Pnz5xUfH6+NGzdq4sSJqlu3riTp7NmzGjRokDIzM9W8eXO1bdtWmZmZSkpK0jfffKMXX3xRXl5eysrK0tChQ5WcnGx7Zm5urk6dOqUVK1bo6aeftvVh9erV+u6779SkSRM1atRIVqtVu3fv1qxZs7Rt2zbFxcXZhWm///67hg8frszMTLVs2VKVKlXS3r17NWzYMN17771OP6eJEydq5syZqlChglq2bKnAwEDt2LFD//nPf7R79279+9//tqs7e/ZsRUZGqn379vL399eZM2e0c+dObd68maCnOH333XeS/pemOrNp0yZt2rTJ9rXJZFKHDh302muvyc/Pz1aelJQkSQWmq1FRUTp27JhOnz6tiIiI6/YrKyur0O9QnLKzs+1+BXBzGEOAaxhDgGsYQ3DGYrHIYrFcp4ZJ1jt2GxzrDfp+ffn3njhxQp999pnTOtfuC2v9/x+E1WrVM888o1deecUugLFarXrnnXe0ZMkSbd++XQ0aNLC7lpqaqgYNGmjcuHG2kOLee+/VxIkTNWTIEHXu3FkvvfSSbSLB+++/rwULFmj16tVq1arVDd8nv3/p6en64osvbD+/Nm3aVG+88YZtVo+z9hctWqS+ffvanjV27FidO3dOn3zyiWJiYmzlgwYN0sCBA/X2229rzpw5kqSffvpJly5d0l/+8hf16dPHrk/p6eny8PCQxWLRpk2bdOrUKT355JP685//bFfv8uXL8vLysv2etG/fXk8++aS8vb3t6k2bNk1TpkzRypUr1aFDB1v5Bx98oIyMDI0bN85uVU1cXJymT58uyf57fdOmTZo5c6ZiYmL07rvv2j4rq9Wq999/XwsXLtSqVav06KOPSpIWL16s8uXL64svvpCvr6/DO7ryffhHFovluhnBH9u/HsMHPSdPnlRCQoIqVqyopk2bOlz39fXVkCFD1KJFC0VGRspisWjfvn2KjY3VihUrlJWVZZfKms1mSVJAQIDT9vLL8+tdz6lTp5SXl3crr1UsUlJSirsLQInGGAJcwxgCXMMYwrV8fHyuG/7l5fnIYrkzf1bJy8tzKbjMycmRdDXomTZt2g3byq+fk5OjsmXLKi8vz+HnuMcee0xLlizRxo0bdd999zk8Z8SIEbJYLLZ+t2zZUhMnTlRubq4GDx5sa0OSWrVqpQULFmj//v1q3rz5Dd8nP+gZMmSIPD09bW089NBD8vLy0qVLlwpsf//+/bay33//Xb/99ps6deqkRo0a2X3GFStWVJcuXTRv3jzt27dP1apVs30G17aZz8/Pz1aWm5srSfLy8nKolx885ZcHBwfLarU61OvWrZumTJmiTZs22UKY06dPa+fOnapRo4ZatWpld0+fPn00f/58Xbp0STk5ObZr8+bNkyS9/PLLDv0eNmyYFi1apB9++EEPPfSQrdzT01O5ubnXfceikJWVpYsXLzq95unpqWrVqhX6WYYPepYsWSKr1aouXbrIw8PD4XrZsmU1fPhwu7Lo6GjVr19f/fv3V3x8vPbt26datWoVed/+uE7wTpWdna2UlBSFhobKx+fO3HkfuJMxhgDXMIYA1zCG4Ex6evp1vx88PT3l4XFnbkXh6enp0vdy/myRmJgYffLJJ07rJCQk6IUXXpCnp6e8vb2Vk5Mjb29v5ebmav78+Vq1apWOHj2qzMxMW9AiSRcuXHDoW+nSpRUVFWVXFhYWJunqipDSpUvbXatYsaKkq3vlFOY982cC1alTx6F+mTJldOXKlQLbv7aN/fv3S7r6vTFz5kyHdk6cOCHp6oSFWrVqqUWLFoqLi9PHH3+s7du3q1mzZmrYsKHDypYHHnhAISEhmjNnjg4fPqzmzZurUaNGqlKlisN2KFarVUuXLtWyZct0+PBhmc1mu1kz1/b36NGjkqQGDRo4vLePj49q1qyphIQEeXt7267v2bNHfn5++v77751+lqVKlVJSUpKtftu2bfXtt99q4MCBatu2rRo3bqy6deve1OyawvL19XV6QvitMHTQY7FYtHTpUnl4eBR42lZBfH191bFjR8XGxmrnzp22oCd/7WBGRobT+/LLr13neL02ShIfH58S12fgTsIYAlzDGAJcwxjCtS5duuT0L8L/x6Q//Ax+BzHdoO/Xd+29BT3n2vL8MMJkMukf//iH1qxZo0qVKqlt27YqU6aMvLy8ZDab9dVXXyknJ8fh3oCAAId28oOEwMBAh2v5QVReXl6h3jO/f0FBQQ7XvLy85OXlVWD717Zx6dIlSdK6deu0bt26AtvLysqSh4eHIiMjNX36dE2ZMkUbNmzQTz/9JEmqUqWKnn32WbVp00bS1aBr+vTp+uyzz7RmzRqtX79ekhQaGqpnnnlGPXv2tD37gw8+0Pz58xUaGqqHH35YISEhts9j6tSpdp9vZmampKuTN5x9TuXKlZN09fcy//rFixeVl5d33ZlcmZmZtvovv/yyIiIitGTJEs2YMUMzZsxQqVKl1Lp1a/35z39WcHBwgc+5WR4eHkX232hDBz0bNmzQmTNnFBMTY0tFb0b+b9q16+Tyk9Djx487vScpKUne3t5FlsQBAAAAAIrfnj17tGbNGsXExOjjjz+226fnt99+01dffVWMvXNd/jYkr7zyinr37l2oe6pXr65///vfys3N1d69e7VhwwZ9/fXXev3111W+fHnbfkUVK1bU2LFjZbFYdPDgQW3cuFHz5s3T+++/r6CgILVv317nz5/XN998oxo1amj69Ol2oUdqaqqmTp3qtL8XLlxw2rdz5845fUeTyaQff/yxUO/n5eWlp59+Wk8//bTOnj2rbdu2aenSpVq+fLnOnTun//73v4V6jrvdegxaAhRmE+brSUxMlPS/aW2SVLduXXl7e2vz5s12U/Skqxt2HTt2TPXr13c4Vg8AAAAAUHKdPHlSktS8eXOH07B27NhRDD0qWvmnaf322283fa+Xl5fq1aunZ599Vi+//LKsVqvWrl3rUM/Dw0M1a9bUgAEDNH78eEnSmjVrJF39fK1Wq6Kjox1mtjj7fPNP1dq1a5fDtaysLB04cMChvG7dukpPTy9w4sb1lC9fXu3bt9enn36qqKgobdmy5Y49YMmwQc+FCxe0Zs0alSlTRo888kiB9fbv3+8Q2EhXj2RftmyZSpcurQcffNBWHhgYqLZt2+rkyZNasGCBrdxqtWrixImSpO7duxfdiwAAAAAAil3+KpGdO3falR86dMjpnjYlTZ06dVS3bl2tXLnS6YwXi8Wibdu22b7eu3ev00OIzp8/L+l/y8MOHTrkdHbNH+vlT7DYtWuX3b48KSkpmjRpksP9YWFhatCggX7//XeH/s6ePdvpxsb5p4ONHz9eaWlpDtdTU1N15MgRSVf3N3MWImVmZtpOC3NlCeHtVCKmnSxatMg2mA4dOiTp6jFnCQkJkq5uvvTHcGX58uXKzc1Vx44dHY5mu9bHH3+sEydOqF69eqpQoYLt1K2dO3fKx8dHY8aMcdhvZ+TIkUpISND777+vLVu2KDIyUtu2bdPu3bv18MMPq127dkX49nevlBSTUlPdtzA4JMSq0NA79gxJAAAAAMWodu3aqlOnjlatWqXU1FTVrVtXKSkp+vXXX9W8eXP9/PPPxd1Fl40fP17PP/+8Xn/9dc2dO1e1atVSqVKldPr0af32229KS0uzzdRZsWKFFi5cqPvvv1+RkZEKCAjQkSNHtH79epUuXVpdu3aVJG3evFn/+c9/1KBBA1WqVEn33HOPTp48qTVr1qhUqVK2PXpCQkL06KOP6ueff9YzzzyjBx54QOfOndO6devUpEkT22bQ13rllVc0fPhwjRkzRj///LOioqK0b98+7d69Ww0bNtT27dvtwphmzZppyJAhmjZtmp544gk1a9ZMFStWVHp6uk6cOKEdO3ZoxIgRqlq1qq5cuaKhQ4eqUqVKqlWrlipWrKjMzEytXbtW586d09NPP33HbnBfIoKenTt3atmyZQ5l1yapfwx6Crtsq0OHDoqPj9fu3buVlpYmq9Wq8uXLq1u3burXr5+qVKnicE9ISIhmzJihyZMna926dVqzZo0qVqyo4cOHa8CAAQ47h+PWpKaaNGWK+wbOsGHZBD0AAAAAnPL09NT//d//acKECdq4caP27t2rqKgovfTSS2rWrJkhgp6IiAjNnj1bX375pX755RctWbJEnp6eKleunBo2bGg72lyS2rVrpytXrmjXrl3as2ePsrOzVaFCBfXo0UP9+/e3zYCKiYlRcnKytm/frvj4eGVmZqp8+fJq06aN+vfvb3ds+JgxYxQWFqaff/5Z8+bNU2hoqJ566ik988wzditt8v3pT39SXFycJkyYoA0bNmjjxo1q0KCBpkyZYltxk7+XT77hw4erYcOG+vrrr7VlyxZdunRJ99xzj8LDwzV06FB16NBB0tXj00eOHKktW7Zox44dunDhgoKCglS5cmU9//zzd/QED1NaWho/2eK6srKylJSUpKioKLee1JCY6OH2oKdOHcuNKwI3qbjGEGAUjCHANYwhOHP27FmVL1++wOvunl1/M9w9E99isSg7O1s+Pj537FId2MvLy1OPHj105cqVAo9Sv9PcaEzejBIxowcAAAAA4D6hoWxrgDtfbm6uzGazwzHnM2fOVHJysh5//PHi6VgxI+gBAAAAAAAlTmZmpjp37qzo6GhVqlRJubm5SkxM1J49exQSEqJhw4YVdxeLBUEPAAAAAAAocXx9ffXYY49p69at2rFjh65cuaKQkBA9/vjjGjJkiEJCQoq7i8WCoAcAAAAAAJQ43t7eevXVV4u7G3ccdpICAAAAAAAwCIIeAAAAAAAAgyDoAQAAAAAAMAiCHgAAAAAAAIMg6AEAAAAAADAIgh4AAAAAAACDIOgBAAAAAAAwCIIeAAAAAAAAgyDoAQAAAAAAMAiCHgAAAAAAAIPwKu4OAAAAAADuLKaUFJlSU4u7G05ZQ0JkDQ116RmZmZn66quv9PPPP+v48ePKzc1VmTJlFBYWpvvvv1/dunVTZGRkEfUYcC+CHgAAAACAHVNqqnymTCnubjiVPWyYS0FPRkaGhg0bpoMHDyoqKkodOnTQPffco/T0dCUmJmrmzJmKiIgg6EGJRdADAAAAALhrfPXVVzp48KC6deumf/zjHzKZTHbXT548qZycnGLqHeA6gh4AAAAAwF3jt99+kyT16tXLIeSRpIiICIey3r17S5LmzJmjCRMm6JdffpHZbFaVKlXUv39/tW/f3q7+2bNntXDhQm3cuFEnT56U2WxWSEiIHnzwQQ0bNkxly5a1qz9u3DgtW7ZMCxcuVHx8vBYuXKgzZ84oPDxcQ4YMUbt27ZSTk6MpU6bohx9+0Llz51SpUiWNHDlSDz74YKHeOy4uTlOnTtXkyZN16tQpffnllzpx4oTKli2rvn37qk+fPrJarfryyy+1cOFCnT59WqGhoRo8eLA6d+7s8LycnBzNmzdP33//vY4dOyYPDw/de++96t+/vx555BG7umazWXPmzNHPP/+s06dPy8PDQ2XKlFH9+vU1fPhwhYWFSZKuXLmib775RsuWLVNycrLy8vJUpkwZ1a5dW4MGDVLNmjVtz/v222+1fv16JSUlKS0tTcHBwXrggQc0bNgwp7Ox0tLSNGnSJP3yyy+6fPmyqlWrpoEDByojI0NvvfWWxowZoy5dutjdc+DAAX3++efatm2b0tPTFRISoocffljDhg1TcHCwXd2tW7dq9uzZOnDggNLT0xUUFKSoqCh16tRJjz/+eKF+j4oKQQ8AAAAA4K5xzz33SJKOHTtmCw4KIzc3Vy+88IIyMzPVqVMnZWZmatWqVXrzzTeVlpamPn362Opu375dc+bM0QMPPKA6derIy8tL+/fv17fffquNGzdq9uzZCgwMdGjj448/VmJioh5++GF5eHjoxx9/1JtvvqmgoCDNmzdPR44cUfPmzXXlyhWtXLlSr7zyiubNm3dTy8zmzp2rbdu26ZFHHlGTJk0UHx+vjz76SKVKldLvv/+un3/+WQ899JC8vb31448/aty4cQoLC1OjRo1sz8jOztZLL72khIQE1axZU4899phyc3O1bt06vfLKK3rllVds4ZjVatWLL76o3bt3q0GDBoqJiZGHh4eSk5O1Zs0aderUyRb0jBs3TqtWrVKNGjXUtWtXeXt7KyUlRQkJCdqzZ4/t9+vIkSOKi4tT48aN1bJlS/n6+urYsWNauXKl1q1bp9mzZ9ueKUmXL1/W8OHDdeTIEdWvX18NGzbUmTNn9MYbbygmJsbp5/Trr7/aZny1aNFCFSpU0JEjRzR//nxt3LhRM2bMUOnSpSVJa9eu1csvv6ygoCA98sgjCgkJ0YULF3TgwAEtX76coAcAAAAAgNuldevWWrFihd555x3t2bNHTZs2Va1atRxmaPxRamqqoqKiNG3aNHl7e0uSBg4cqP79++u///2vWrVqpQoVKkiSmjRpohUrVsjf39/uGcuWLdO4ceM0b948DR482KGNo0eP6ssvv1SZMmUkSV27dtWgQYP0xhtvqFq1apo7d678/PwkSTExMXr99df11Vdf6ZVXXin0++/YsUOzZ8+2zVx6+umn1aNHD/3nP/9R2bJlNXfuXFv7nTt31uDBgzVnzhy7oGfq1KlKSEjQkCFD9Oyzz9pmRmVkZOj555/Xp59+qlatWql8+fI6dOiQdu/erRYtWuiDDz6w60t2drZyc3MlXZ2l89NPP6lWrVqaMWOGPD09bfXy8vJ0+fJl29dVq1bV8uXLbaFdvq1bt2rkyJGaPn26Xn/9dVv5rFmzdOTIET3++OP6+9//bivv3LmzRo4c6fAZpaWlaezYsQoODtaUKVPsQqOVK1fqjTfe0GeffabRo0dLkpYsWSKr1apJkyY5hIdpaWkF/VbcNhyvDgAAAAC4azzyyCN66aWXZLVaNWfOHL344otq166devTooQ8++EDHjx8v8N7nn3/eFvJIUmhoqPr06aPs7Gz9+OOPtvKyZcs6hDyS1KlTJwUEBGjLli1Onz9o0CBbyCJJderUUUREhC5duqTnnnvOFvJI0qOPPiovLy8dOHDgpt7/ySeftFueFhoaqgYNGshsNju0X7duXUVERNi1YbFYtGDBAkVGRtqFPJIUEBCgoUOHKicnR/Hx8XbtlipVyqEvPj4+dp+T1WpVqVKl5OFhH1V4enoqKCjI9nVgYKBDyCNdDdiqVaumzZs325WvWLFC3t7eevbZZ+3Ko6Oj1bRpU4fnLF++3BZaXRvySFK7du1Uq1Ytu9/v673jjQLE24EZPQAAAACAu0q/fv3UvXt3bdiwQbt27dLevXuVmJio+fPn67vvvtPbb7/tsM+Mp6en6tWr5/Cshg0bSpL2799vVx4fH68FCxZo//79unTpkvLy8mzXzp4967RfzpaShYSE6OTJkw7XPD09VbZsWaWmphbupW/QxvWu7d692/b1sWPHdPHiRYWEhGiKk5PZ8mewHDt2TJJUpUoV1ahRQytXrtSZM2fUokULNW7cWDVr1rQLdAIDA9W8eXOtW7dO/fv3V+vWrdW4cWPVrl1bXl6O0UVCQoLmzp2rxMREpaWl2X2+14ZxZrNZycnJqlq1qsqVK+fwnAYNGmjTpk12Zfnvm5iYqBMnTjjcc+XKFaWlpdn2Bmrbtq3i4+M1ZMgQtW/fXk2aNFHDhg2LJeSRCHoAAAAAAHehgIAAtWnTRm3atJF0NRCYNGmSvvnmG/3rX//SsmXL7AKD4OBgh5kmkmwbK5vNZlvZnDlz9Omnn6pMmTJq2rSpKlSoYJvt8dVXXxV4qldAQIBDWf4SJmd7+nh6etqWPhXW9doo6Nq1IcrFixclSYcPH9bhw4cLbCczM1OS5OXlpUmTJmnKlCmKj4/Xp59+KkkqU6aMevXqpUGDBtnaf/fddzVjxgz98MMPmjx5sq1PXbt21fPPPy9fX19J0qpVq/T666/L399fTZs2VXh4uO1a/kbO+TIyMiTJYQPsfM7K899x/vz5Bb5f/jsGBwerTZs28vb21pdffqkFCxZo/vz5MplMaty4sf785z/f1F5QRYGgBwAAAABw1wsMDNTo0aO1bt06JScn6+DBg7rvvvts19PS0mSxWBzCnvPnz9vul65u2jxt2jSFhIToiy++sAsSrFarZs+e7Ya3uX3yw6BHH31U//73vwt1T3BwsEaPHq1XXnlFR48e1datWzVv3jzFxcXJy8tLAwcOlCT5+vrqueee03PPPaeTJ08qISFBCxYs0FdffaUrV67Y9teZMmWKfHx8NHPmTFWqVMmurT8uqcrvb/7v0x85K8+/Z+7cuapevXqh3rFFixZq0aKFMjIytGvXLsXHx+u7777Tiy++qPnz59stPbvd2KMHAAAAAABJJpPJNjPkj/Ly8mxHs19r+/btkqQ//elPkq4GQmazWfXq1XOYLbJ3715duXKliHvtXlWqVFFAQID27t1707OJTCaTqlatql69emnChAmSrp5u5UxERIQee+wxxcbGyt/f367eyZMnVbVqVYeQJzU1VSdPnrQrCwwMVFhYmE6cOOE01Nm1a5dDWZ06dSTJ6e/3jQQEBKhZs2b6xz/+oc6dO+v8+fN2S9/cgaAHAAAAAHDXWLBggfbs2eP02urVq3X06FEFBQU5nckxadIku2VXKSkp+vrrr+Xj46O2bdtKuroUqFSpUtq3b5+ysrJsdS9evKgPP/ywiN/G/by8vPTEE08oOTlZn376qdOw59ChQ7ZQ5dSpUzp16pRDnXPnzkn63wbGFy5c0KFDhxzqXbp0SdnZ2fLx8bGVVaxYUUlJSbZnSFf3zfn3v//ttD8dOnRQTk6O4uLi7MoTEhK0ceNGh/pdu3ZVQECAJk+e7LRPWVlZdiHQtm3b7Ja35btw4YLdO7oLS7cAAAAAAHeN9evX69///reioqJUv359lS9fXpmZmdq/f7927NghDw8P/e1vf7MLFqSrmxJnZmaqb9++evjhh5WZmalVq1YpPT1dL7/8su1odQ8PD/Xs2VNz5sxRv3799NBDDykjI0MbNmxQxYoVVb58+eJ47SL17LPPav/+/fr666+1bt06NWzYUGXKlNHZs2d18OBBHThwQNOmTVPZsmX1+++/69VXX1WdOnVsGyKfOXNGv/zyizw8PPTUU09JurpB9dNPP617771XNWrUUIUKFZSenq5ff/1Vubm5evrpp23t9+7dWx9++KH69++vRx99VHl5edq8ebOsVqvuvfdeh5PIBgwYYNsc+9ChQ2rYsKFSUlL0008/6eGHH9aaNWvsTg8rU6aMxo8fr7///e96+umnFRMToypVqig7O1vJycnavn276tWrp//85z+SpI8++kipqalq0KCBwsLCZDKZtHPnTiUmJqpu3bpq0KCBG35X/oegBwAAAABw1xg1apQaNGigzZs3a/v27bZZIeXLl1fnzp3Vu3dvu7158nl5eWnChAmaOHGili9fLrPZrMqVK+uVV15R+/bt7eq+8MILKl26tJYuXapvv/1WZcuWVbt27TRs2DBbsFGS+fj46JNPPtF3332n5cuX6+eff1ZOTo7Kli2rqlWrqkePHqpRo4Yk6b777tOAAQOUkJCgdevW6dKlSypXrpyio6P19NNP204yCwsL07Bhw7R161Zt2bJF6enpCg4O1p/+9Cc9+eSTatasma39Xr16ycvLS/PmzdPixYttJ3a98MILtn18rhUQEKDPPvtMEydO1K+//qp9+/apatWqGj9+vE6dOqU1a9Y4bET90EMP6YsvvtDs2bO1ZcsWbd68WX5+fqpQoYK6dOmijh072uoOHDhQ8fHx2rdvnzZu3CgvLy+FhYVp5MiR6tmzp22zaXcxpaWlWd3aIkqcrKwsJSUlKSoqqsD1qrdDYqKHpkzxuXHFIjJsWLbq1LG4rT3cPYprDAFGwRgCXMMYgjNnz5697swSU0qKTDd5bLe7WENCZA0NdVt7FotF3bt3lyR99913bmsX7jFmzBh9//33+vrrr1W1atVi68eNxuTNYEYPAAAAAMCONTTUrWEKcLulpqYqJCTErmzbtm368ccfVbly5WINeYoaQQ8AAAAAADC0P//5zypVqpRq1qwpPz8/HTlyRBs2bJCHh4deeeWV4u5ekSLoAQAAAAAAhta5c2d9//33+vHHH5WRkaGgoCA99NBDGjhwoOrWrVvc3StSBD0AAAAAAFzHvHnzHE7hQsny1FNPGWIj7MLwKO4OAAAAAAAAoGgQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAANyFrFZrcXcBgIp+LBL0AAAAAMBdxtfXV1lZWcXdDQCSsrKy5OvrW2TPI+gBAAAAgLtMQECAzGazMjMzmdkDFBOr1arMzEyZzWYFBAQU2XO9iuxJAAAAAIASwcPDQ+XKlVNGRoZSU1OLuzt3NIvFYptx4eHBXAkULV9fX5UrV65Iv7cIegAAAADgLuTh4aGgoCAFBQUVd1fuaFlZWbp48aJCQ0OLdHkNcLsQRwIAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBeBV3BwpjxYoV2rFjh/bu3atDhw4pJydHY8aMUZcuXRzqxsXFaerUqQU+a9GiRQoPD3co37Bhgz7//HPt379fJpNJtWrV0qBBgxQdHe30OceOHVNsbKy2bt2qrKwsRUVFqUePHnriiSdkMplu/WUBAAAAAABuUYkIemJjY5WcnKzg4GCFhIQoOTn5hvd07txZYWFhDuVBQUEOZStWrNDYsWNVpkwZde7cWZK0atUqjRo1Su+8845at25tV//w4cMaOnSorly5ojZt2igkJETr1q3T+++/ryNHjmj06NG3+KYAAAAAAAC3rkQEPa+//rqioqIUFhammTNnauLEiTe8p0uXLmrcuPEN6128eFEffvihgoODNWvWLIWGhkqSBgwYoP79++u9995TTEyMAgICbPe89957MpvN+uSTT/Tggw9KkkaMGKEXXnhB8+fPV/v27VW/fv1bfFsAAAAAAIBbUyL26ImOjnY6O6co/PTTT7p06ZJ69+5tC3kkKTQ0VL169VJaWppWr15tKz927Ji2b9+uxo0b20IeSfL29taIESMkSYsXL74tfQUAAAAAALieEhH03Irt27dr5syZmj17tlavXq3Lly87rZeQkCBJatq0qcO1mJgYSdK2bdtsZfn/nn/tWg0aNJCfn59dfQAAAAAAAHcpEUu3bkVcXJzd10FBQfrrX/9q24MnX1JSkiQpKirK4Rn5Zfl1blTf09NT4eHhOnLkiHJzc+Xldf2PNysrqxBvUvyys7PtfnWXvDwf5eV5urG9XGVlufcdcXcorjEEGAVjCHANYwhwDWMIdwJfX99C1zVc0HPvvffqzTffVKNGjRQSEqJz585p7dq1iouL01tvvaWgoCA98sgjtvpms1mSFBgY6PCs/LL8OjeqL0kBAQGyWCy6fPmySpcufd2+njp1Snl5eTf3gsUoJSXFre2ZzRVlNue6sb0rSko67bb2cPdx9xgCjIYxBLiGMQS4hjGE4uLp6alq1aoVur7hgp5WrVrZfR0eHq7evXurSpUqGjVqlGJjY+2CnuLk7Jj3O1F2drZSUlIUGhoqHx8ft7WbmemjwMBSbmsvMNDL6UwtwFXFNYYAo2AMAa5hDAGuYQyhpDFc0FOQ6OhoRUZG6uDBgzKbzbYZOdfO2gkODra7x9nsHWezfK6VkZEhk8kkf3//G/bpZqZe3Ql8fHzc2mdPTw95erpv6Zanp5d8fQ27bRXuAO4eQ4DRMIYA1zCGANcwhlBS3FU/1d5zzz2S7PfGcbYPTz5n+/Fcr35eXp5OnTql8PDwG+7PAwAAAAAAUNTumqAnMzNTR44ckZ+fn93MnUaNGkmSNm3a5HDPxo0b7epc++/51661c+dOZWZm2tUHAAAAAABwF0MFPRkZGTp27JhDeVZWlt5++21lZGSoTZs2drNt2rRpo8DAQM2bN89uc62UlBTNnz9fwcHBatmypa28cuXKatiwoRISErR+/XpbeU5OjmJjYyVJ3bp1uw1vBwAAAAAAcH0lYn3RokWLtHPnTknSoUOHJEmLFy9WQkKCJKlBgwbq3r270tPT1bt3b9WuXVtVqlRRuXLldP78eW3evFlnzpxRjRo1NGrUKLtnly5dWqNHj9bYsWM1YMAAtWnTRpK0atUqpaen6+2331ZAQIDdPX/72980bNgwjR49Wm3atFFISIjWrVunw4cPq1evXqpfv/7t/kgAAAAAAAAclIigZ+fOnVq2bJlDWX74I0ndu3dX6dKl1bNnTyUmJmr9+vW6ePGifH19VaVKFfXp00e9evVyunlWx44dFRwcrBkzZmjp0qUymUyqVauWBg8erOjoaIf61atX14wZMzR58mStW7dOWVlZqlSpkv72t7/piSeeKPoPAAAAAAAAoBBMaWlp1uLuBO5sWVlZSkpKUlRUlFt3mU9M9NCUKe47vnDYsGzVqWNxW3u4exTXGAKMgjEEuIYxBLiGMYSSxlB79AAAAAAAANzNCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMwqu4O1AYK1as0I4dO7R3714dOnRIOTk5GjNmjLp06WJXLzc3V7/++qt+/fVX7dmzRykpKTKZTKpatao6d+6sxx9/XJ6ennb3nDp1St27dy+w7aFDh+rZZ591KE9NTdXkyZO1fv16Xbp0SRUrVlSnTp00YMAAeXmViI8VAAAAAAAYTIlIJGJjY5WcnKzg4GCFhIQoOTnZab0TJ07otddek7+/v5o0aaKHH35YZrNZa9eu1fvvv6/169fro48+kslkcrj33nvvVYsWLRzKGzdu7FCWmpqqQYMG6cyZM2rZsqWioqK0bds2xcbGas+ePfrggw+ctgEAAAAAAHA7lYig5/XXX1dUVJTCwsI0c+ZMTZw40Wk9f39//e1vf1Pnzp3l5+dnK8/MzNSIESO0du1a/fTTT2rTpo3DvTVr1nQ6c8eZCRMmKCUlRa+++qqeeOIJSZLVatWbb76plStXauXKlWrfvv0tvCkAAAAAAMCtKxF79ERHRyssLOyG9SpUqKCePXvahTyS5Ofnp759+0qStm/f7lJfMjIytGrVKkVERKhHjx62cpPJpBdeeEGStGjRIpfaAAAAAAAAuBUlYkZPUcjfN+ePe/TkS01N1fz582U2m1W2bFk1btxYkZGRDvV+++03ZWdnKzo62mF5VlhYmCpXrqxdu3YpLy+vwLYAAAAAAABuh7sm6FmyZIkkqWnTpk6vb9q0SZs2bbJ9bTKZ1KFDB7322mt2M4SSkpIkSZUqVXL6nKioKB07dkynT59WRETEdfuUlZV1U+9QXLKzs+1+dZe8PB/l5bkvLMvLy1VWlnvfEXeH4hpDgFEwhgDXMIYA1zCGcCfw9fUtdN27IuhZuHCh1q9fryZNmqh58+Z213x9fTVkyBC1aNFCkZGRslgs2rdvn2JjY7VixQplZWXpvffes9U3m82SpICAAKdt5Zfn17ueU6dOKS8v71Zfy+1SUlLc2p7ZXFFmc64b27uipKTTbmsPdx93jyHAaBhDgGsYQ4BrGEMoLp6enqpWrVqh6xs+6FmzZo0++OADhYWF6a233nK4XrZsWQ0fPtyuLDo6WvXr11f//v0VHx+vffv2qVatWkXet/Dw8CJ/5u2QnZ2tlJQUhYaGysfHx23tZmb6KDCwlNvaCwz0UlRUlNvaw92juMYQYBSMIcA1jCHANYwhlDSGDnrWrVunv//97ypbtqwmTZqkkJCQQt/r6+urjh07KjY2Vjt37rQFPYGBgZKubsrsTH55fr0btVGS+Pj4uLXPnp4ebt3nyNPTS76+JWJ/cpRQ7h5DgNEwhgDXMIYA1zCGUFIY9qfatWvX6tVXX1VwcLAmT558w/1ynAkODpZkv5dO/oyP48ePO70nKSlJ3t7eCg0NvflOAwAAAAAAuMCQQc/atWv12muvqXTp0po0adItL8dJTEyUJLuj3evWrStvb29t3rxZVqvVrn5ycrKOHTum+vXr2075AgAAAAAAcBfDBT3r16/Xa6+9pqCgIE2ePLnA07Hy7d+/3yGwkaT4+HgtW7ZMpUuX1oMPPmgrDwwMVNu2bXXy5EktWLDAVm61WjVx4kRJUvfu3YvmZQAAAAAAAG5CiZh2smjRIu3cuVOSdOjQIUnS4sWLlZCQIElq0KCBunfvrqNHj+pvf/ubsrOz1bhxY/3www8OzwoPD1eXLl1sX3/88cc6ceKE6tWrpwoVKthO3dq5c6d8fHw0ZswYh/12Ro4cqYSEBL3//vvasmWLIiMjtW3bNu3evVsPP/yw2rVrd7s+CgAAAAAAgAKViKBn586dWrZsmUNZfvgjXZ1Fc+7cOWVnZ0uSVq5c6fRZjRo1sgt6OnTooPj4eO3evVtpaWmyWq0qX768unXrpn79+qlKlSoOzwgJCdGMGTM0efJkrVu3TmvWrFHFihU1fPhwDRgwQCaTqQjeGgAAAAAA4OaY0tLSHNctAdfIyspSUlKSoqKi3LrLfGKih6ZMcd/xhcOGZatOHYvb2sPdo7jGEGAUjCHANYwhwDWMIZQ0htujBwAAAAAA4G5F0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQLgc9ubm5RdEPAAAAAAAAuMjloKdTp0765JNPdOTIkaLoDwAAAAAAAG6Ry0FPenq6vvrqKz311FMaNmyYli5dqqysrKLoGwAAAAAAAG6Cy0HPxIkT1aZNG3l7e2vXrl3617/+pU6dOun999/X/v37i6KPAAAAAAAAKAQvVx/QpEkTNWnSRBcvXtTy5cu1ePFiHT58WN9++60WLFigmjVrqnv37mrfvr0CAgKKos8AAAAAAABwoshO3SpdurSefPJJzZ07V9OmTVPXrl3l5+en/fv36/3331fnzp01fvx47dq1q6iaBAAAAAAAwDVcntHjTN26dVW3bl29/PLLWrlypRYvXqzExEQtW7ZMy5YtU9WqVdW9e3d17txZgYGBt6MLAAAAAAAAd50im9HjjJ+fn8LDwxUeHi5PT09ZrVZZrVYdPnxYH3/8sbp166Yvv/zydnYBAAAAAADgrnFbZvSkpqZqyZIlWrJkiU6dOiWr1Spvb289+uij6tatm86fP68FCxZox44d+s9//iNJ6tu37+3oCgAAAAAAwF2jyIIei8WiNWvWaPHixdq4caMsFousVqsiIyPVvXt3de3aVcHBwbb67du31+rVq/Xaa6/p22+/JegBAAAAAABwkctBT1JSkr777jstW7ZM58+fl9VqlZeXl1q1aqXHH39c0dHRBd7bsmVL1axZUwcPHnS1GwAAAAAAAHc9l4OeXr16SZKsVqvCw8Nts3fKli1bqPv9/f1lsVhc7QYAAAAAAMBdz+Wgx2Qy6ZFHHtHjjz+umJiYm77/7bffVnZ2tqvdAAAAAAAAuOu5HPQsWbJEISEht3x/uXLlXO0CAAAAAAAAVATHq7sS8gAAAAAAAKDouBz0nDhxQlOnTtXatWuvW2/NmjWaOnWqTp065WqTAAAAAAAAcMLloGfx4sWaOnXqDTdUtlqtmjp1qpYsWeJqkwAAAAAAAHDC5aBn06ZN8vHx0UMPPXTdes2bN5e3t7fWr1/vapMAAAAAAABwwuXNmE+fPq2wsDB5eFw/M/L09FR4eLhSUlJuuo0VK1Zox44d2rt3rw4dOqScnByNGTNGXbp0cVrfbDZrypQpio+P17lz5xQSEqLWrVtr6NCh8vf3d6hvsVg0f/58LVq0SCdOnJCfn5+io6P13HPPKSIiwmkbGzZs0Oeff679+/fLZDKpVq1aGjRokKKjo2/6/QAAAAAAAIqCyzN6MjMz5efnV6i6fn5+MpvNN91GbGysFi5cqNOnT99w8+fMzEyNGDFCc+fOVeXKlfXUU0+pUqVK+uKLL/T888/rypUrDve8++67+uijj2S1WtW7d281a9ZM8fHxeuaZZ3T8+HGH+itWrNBLL72ko0ePqnPnzurUqZMOHz6sUaNG6aeffrrp9wMAAAAAACgKLs/oKVOmjI4fP668vDx5enoWWC8vL0/Hjx9XcHDwTbfx+uuvKyoqSmFhYZo5c6YmTpxYYN3Zs2fr999/14ABAzRy5Ehb+YQJEzRr1izNnTtXAwcOtJVv3bpVixcvVsOGDTVhwgR5e3tLktq3b68///nP+uCDD/Tf//7XVv/ixYv68MMPFRwcrFmzZik0NFSSNGDAAPXv31/vvfeeYmJiFBAQcNPvCQAAAAAA4AqXZ/TUq1dPly9fvuEmy0uWLFFGRobq1q17021ER0crLCzshvWsVqsWL14sf39/DRkyxO7akCFD5O/vr8WLF9uVL1q0SJI0fPhwW8gjSQ8++KAaN26sTZs26fTp07byn376SZcuXVLv3r1tIY8khYaGqlevXkpLS9Pq1atv+h0BAAAAAABc5XLQ06NHD1mtVn300UdatmyZ0zrLly/XRx99JJPJpCeeeMLVJgt0/PhxnT17VvXr13dYTubn56f69evr5MmTdvsEbdu2TX5+fmrQoIHD82JiYmx18iUkJEiSmjZtWqj6AAAAAAAA7uLy0q3GjRvr8ccf18KFCzV+/HjFxsaqbt26CgoK0qVLl7R7926dPXtWVqtVjz/+uB544IGi6LdTSUlJkqSoqCin16OiorRx40YdP35coaGhyszMVGpqqqpXr+502Vn+c/Kfe6M2nNW/nqysrELVK27Z2dl2v7pLXp6P8vIKXg5Y9O3lKivLve+Iu0NxjSHAKBhDgGsYQ4BrGEO4E/j6+ha6rstBjyS9+uqrKlu2rGbPnq0zZ87o559/trteqlQpPfPMMxo8eHBRNFeg/I2eAwMDnV7P3zcnv97N1r/RPfllhd1w+tSpU8rLyytU3TvBrZyY5gqzuaLM5lw3tndFSUmnb1wRuEXuHkOA0TCGANcwhgDXMIZQXDw9PVWtWrVC1y+SoMdkMunZZ59Vr169tG7dOh06dEgZGRkKCAhQjRo11Lx581vahNnowsPDi7sLhZKdna2UlBSFhobKx8fHbe1mZvooMLCU29oLDPQqcDYY4IriGkOAUTCGANcwhgDXMIZQ0hRJ0JOvTJky6tKlS1E+8qbcaEZNRkaGXb2brf/He/4YXt1ohtAf3czUqzuBj4+PW/vs6elx3ZPcir49L/n6urxtFVAgd48hwGgYQ4BrGEOAaxhDKCkM9VPtjfbIyS+vVKmSpKsbNIeEhBS4hMrZfjzXa+NGewQBAAAAAADcTkUe9Fy6dEkpKSk6ffp0gf/cLpUqVVL58uW1a9cuZWZm2l3LzMzUrl27FB4ebncseqNGjZSZmamdO3c6PG/jxo2SpIYNG9rVl6RNmzYVWD+/DgAAAAAAgDsVydKtlJQUffbZZ1qzZo0uXbp03bomk0kbNmwoimadPrtbt26aOnWqpk2bppEjR9quTZs2TZcvX9bAgQPt7unevbtWrlypzz77TBMmTJC3t7ckaf369UpISFDTpk0VFhZmq9+mTRtNmDBB8+bNU9euXW2hUUpKiubPn6/g4GC1bNnytrwfAAAAAADA9bgc9CQlJWno0KFKT0+X1Wq9Yf3C1PmjRYsW2WbcHDp0SJK0ePFiJSQkSJIaNGig7t27S5L69++vX375RbNmzdL+/ftVq1Yt7du3T5s2bVLt2rX15JNP2j27SZMm6tatmxYvXqz+/furefPmSk1N1apVq1S6dGm98sordvVLly6t0aNHa+zYsRowYIDatGkjSVq1apXS09P19ttv207rAgAAAAAAcCeXg57Y2FilpaWpcuXKeu6551SvXj2VK1dOJpOpKPonSdq5c6eWLVvmUHbtcqv8oMfPz0+fffaZ4uLiFB8fr4SEBIWEhKhfv34aOnSo082z/v73v6tGjRpauHChvv76a/n5+ally5Z67rnnFBkZ6VC/Y8eOCg4O1owZM7R06VKZTCbVqlVLgwcPVnR0dJG9NwAAAAAAwM0wpaWl3fwUm2u0b99eZrNZ33zzjd0SJxhHVlaWkpKSFBUV5dZd5hMTPTRlivuOLxw2LFt16ljc1h7uHsU1hgCjYAwBrmEMAa5hDKGkcXkz5suXL6ty5cqEPAAAAAAAAMXM5aCnYsWKsliYBQEAAAAAAFDcXA562rZtq2PHjunkyZNF0R8AAAAAAADcIpeDnmeeeUbVq1fX66+/rlOnThVFnwAAAAAAAHALXD51a/bs2WrSpInmz5+vPn36KCYmRpUqVZKfn1+B9wwdOtTVZgEAAAAAAPAHLgc9U6ZMkclkktV69fCuX3/9tcCj1a1Wq0wmE0EPAAAAAADAbeBy0NOpU6cCgx0AAAAAAAC4j8tBz9ixY4uiHwAAAAAAAHCRy5sxAwAAAAAA4M5A0AMAAAAAAGAQLi/dypeUlKSvvvpKW7ZsUUpKirKzs7Vhwwbb9cWLF+vs2bPq27ev/P39i6pZAAAAAAAA/H9FEvT8+OOPGj9+vLKzs22nb/1xg+aLFy9q6tSpqlKlitq0aVMUzQIAAAAAAOAaLi/dOnDggP75z38qJydHvXr10uTJk1WrVi2Heo8++qisVqt+/fVXV5sEAAAAAACAEy7P6Jk9e7by8vL0l7/8RX369JEk+fj4ONSLiIhQmTJllJiY6GqTAAAAAAAAcMLlGT3btm2Tv7+/LeS5ngoVKig1NdXVJgEAAAAAAOCEy0HPhQsXFBkZWai6np6eysvLc7VJAAAAAAAAOOFy0BMQEKDz588Xqm5ycrKCg4NdbRIAAAAAAABOuBz01KhRQ6mpqTpy5Mh16+3cuVMXLlxQ7dq1XW0SAAAAAAAATrgc9HTs2FFWq1XvvfeeMjIynNa5cOGC3nnnHZlMJnXs2NHVJgEAAAAAAOCEy6dude7cWUuWLNH27dvVr18/tWvXThcuXJAkLV26VAcPHtTy5cuVnp6u6OhotWrVyuVOAwAAAAAAwJHLQY+Hh4c++ugjvfnmm9qwYYNmzZplu/avf/1LkmS1WtW0aVO98847rjYHAAAAAACAArgc9EhSUFCQPvnkE23evFmrVq3SgQMHdPHiRfn7+6t69epq06aNHnrooaJoCgAAAAAAAAUokqAnX3R0tKKjo4vykQAAAAAAACgklzdjBgAAAAAAwJ2BoAcAAAAAAMAgXF669dxzz91UfZPJpEmTJrnaLAAAAAAAAP7A5aBn27ZtN6xjMpkkXT19K//fAQAAAAAAULRcDnrefPPNAq9lZWXp+PHjWrlypcxms4YOHaqQkBBXmwQAAAAAAIATLgc9Xbp0uWGdZ599Vm+88YYWLlyo2bNnu9okAAAAAAAAnHDLZsyBgYF64403dPbsWU2dOtUdTQIAAAAAANx13HbqVkhIiKpVq6ZffvnFXU0CAAAAAADcVdx6vHp2drbOnTvnziYBAAAAAADuGm4Leg4cOKCkpCQFBwe7q0kAAAAAAIC7isubMZ8+fbrAa1arVefPn9dvv/2mL774QlarVc2bN3e1SQAAAAAAADjhctDTvXv3QtWzWq2KiIjQ8OHDXW0SAAAAAAAATrgc9Fit1ute9/PzU1RUlB5++GH17dtXgYGBrjYJAAAAAAAAJ1wOejZt2lQU/QAAAAAAAICL3HrqFgAAAAAAAG4fgh4AAAAAAACDIOgBAAAAAAAwCLedunU9JpNJCxcudPk5AAAAAAAAdzOXg57k5GSXO2EymVx+BgAAAAAAwN3O5aBn8uTJ+u233xQXFyc/Pz9169ZN1apVU9myZXX+/HkdPnxY3333nTIzM/Xss8+qbt26RdFvAAAAAAAA/IHLQc8999yj6dOnq1mzZho/frz8/Pwc6gwdOlRvvvmmpk2bpunTp6t69equNgsAAAAAAIA/cDnomTJliqxWq/75z386DXkkyc/PT2PHjlXnzp01depUvfvuu642CxQ5i8WqxET37E8eEmJVaKjVLW0BAAAAAO4eLgc9O3bsULVq1RQYGHjdekFBQapWrZq2b9/uapPAbZGebtI33/i4pa1hw7IJegAAAAAARc7l6QsZGRm6ePFioepevHhRGRkZrjYJAAAAAAAAJ1wOeiIiInTq1Clt3rz5uvU2b96skydPKjw83NUmAQAAAAAA4ITLQU+nTp1ktVr12muvad68ecrKyrK7fuXKFc2fP1//+Mc/ZDKZ1LlzZ1ebBAAAAAAAgBMu79HTt29fbd68WVu2bNH//d//aeLEiQoLC7Mdr56cnKwrV67IarWqSZMm6tu3b1H0GwAAAAAAAH/gctDj5eWljz/+WNOmTdO8efOUkZGhI0eO6MiRI7Y6/v7+6t27t4YMGSIvL5ebBAAAAAAAgBNFkrp4e3trxIgRGjhwoHbs2KFjx44pIyNDAQEBqly5su6//375+voWRVMAAAAAAAAoQJFOr/H19VVMTIxiYmKK8rEAAAAAAAAoBJc3YwYAAAAAAMCdochm9CQlJemrr77Sli1blJKSouzsbG3YsMF2ffHixTp79qz69u0rf3//omoWAAAAAAAA/1+RBD0//vijxo8fr+zsbFmtVkmSyWSyq3Px4kVNnTpVVapUUZs2bYqiWQAAAAAAAFzD5aVbBw4c0D//+U/l5OSoV69emjx5smrVquVQ79FHH5XVatWvv/7qapMAAAAAAABwwuUZPbNnz1ZeXp7+8pe/qE+fPpIkHx8fh3oREREqU6aMEhMTXW0SAAAAAAAATrg8o2fbtm3y9/e3hTzXU6FCBaWmprraJAAAAAAAAJxwOei5cOGCIiMjC1XX09NTeXl5rjYJAAAAAAAAJ1wOegICAnT+/PlC1U1OTlZwcLCrTQIAAAAAAMAJl4OeGjVqKDU1VUeOHLluvZ07d+rChQuqXbu2q00CAAAAAADACZeDno4dO8pqteq9995TRkaG0zoXLlzQO++8I5PJpI4dO7raJAAAAAAAAJxw+dStzp07a8mSJdq+fbv69eundu3a6cKFC5KkpUuX6uDBg1q+fLnS09MVHR2tVq1audxpAAAAAAAAOHI56PHw8NBHH32kN998Uxs2bNCsWbNs1/71r39JkqxWq5o2bap33nnH1eYAAAAAAABQAJeDHkkKCgrSJ598os2bN2vVqlU6cOCALl68KH9/f1WvXl1t2rTRQw89VBRNAQAAAAAAoAAuBz3btm2TJNWvX1/R0dGKjo52uVMAAAAAAAC4eS4HPc8995wqVKigJUuWFEV/AAAAAAAAcItcPnUrKChI5cuXL4q+AAAAAAAAwAUuBz01atTQ6dOni6IvAAAAAAAAcIHLQc/jjz+uc+fOaeXKlUXRHwAAAAAAANwil/foad++vfbs2aPx48crOTlZ3bp1U3BwcBF0DQAAAAAAADfD5aCne/fukqTc3FxNnjxZkydPVnBwsHx9fZ3WN5lMWrhwoavNAgAAAAAA4A9cDnqSk5Mdyi5cuFBgfZPJ5GqTAAAAAAAAcOKmgx6z2SwvLy/bjJ3JkycXeadctXTpUr311lvXrdOkSRNNmjRJkhQXF6epU6cWWHfRokUKDw93KN+wYYM+//xz7d+/XyaTSbVq1dKgQYMUHR3t2gsAAAAAAADcgpsOelq3bq37779fn332mSSpUaNGtmu//vqr7rnnHjVo0KDoengLatasqaFDhzq99vPPP+vw4cOKiYlxuNa5c2eFhYU5lAcFBTmUrVixQmPHjlWZMmXUuXNnSdKqVas0atQovfPOO2rdurWLbwEAAAAAAHBzbmnpltVqdVo+evRouxCouNSsWVM1a9Z0KM/JydH8+fPl6elpC2eu1aVLFzVu3PiGz7948aI+/PBDBQcHa9asWQoNDZUkDRgwQP3799d7772nmJgYBQQEuP4yAAAAAAAAheTy8ep/VFAIdCdYvXq10tPT9dBDD6lcuXK3/JyffvpJly5dUu/evW0hjySFhoaqV69eSktL0+rVq4ugxwAAAAAAAIVX5EHPney7776TJHXr1s3p9e3bt2vmzJmaPXu2Vq9ercuXLzutl5CQIElq2rSpw7X8JWHbtm0rii4DAAAAAAAUmsunbpUUycnJ2rJliypUqKBmzZo5rRMXF2f3dVBQkP761786LPNKSkqSJEVFRTk8I78sv871ZGVlFarvxS07O9vuV3fJy/NRXp6n29qzWKzKy8tzS1t5ebnKynLv54niU1xjCDAKxhDgGsYQ4BrGEO4E+QdiFcZdE/QsWbJEFotFXbp0kaenfXhw77336s0331SjRo0UEhKic+fOae3atYqLi9Nbb72loKAgPfLII7b6ZrNZkhQYGOjQTn5Zfp3rOXXqlNuChaKQkpLi1vbM5ooym3Pd1l5urqfM5ituactsvqKkpNNuaQt3DnePIcBoGEOAaxhDgGsYQygunp6eqlatWqHr3xVBj8Vi0dKlS2UymdS1a1eH661atbL7Ojw8XL1791aVKlU0atQoxcbG2gU9RcXZke13ouzsbKWkpCg0NFQ+Pj5uazcz00eBgaXc1p6XV64CAwufkroiMNDL6YwwGFNxjSHAKBhDgGsYQ4BrGEMoaW4p6ElJSdHUqVNv+lq+go4+v102b96s06dP64EHHlBERESh74uOjlZkZKQOHjwos9lsm61z7ayd4OBgu3uuN9vnj25m6tWdwMfHR+npfkpNNbmlvexsOcy+up08PPLc1p6np5d8fe+qLbKgq2OopI174E7CGAJcwxgCXMMYQklR5EHP6dOn77igZ/HixZIK3oT5eu655x4lJSUpKyvLFt5ERUVp7969SkpKcgh6rrd/jxGkppo0ZYp7UuyePVkDCwAAAADAzbjpoKdhw4Yymdwzo6MopKWl6ddff1Xp0qXVsmXLm7o3MzNTR44ckZ+fn12g06hRI61cuVKbNm1SvXr17O7ZuHGjrQ4AAAAAAIA73XTQExsbezv6cdusWLFCOTk56tGjh9P1lBkZGUpNTVXlypXtyrOysvT2228rIyNDXbt2lZfX/z6qNm3aaMKECZo3b566du2q0NBQSVdnOs2fP1/BwcE3HSoBAAAAAAC4yvCbMX/33XeSCl62lZ6ert69e6t27dqqUqWKypUrp/Pnz2vz5s06c+aMatSooVGjRtndU7p0aY0ePVpjx47VgAED1KZNG0nSqlWrlJ6errffflsBAQG398UAAAAAAAD+wNBBT2Jiog4dOqQ6deqoRo0aTuuULl1aPXv2VGJiotavX6+LFy/K19dXVapUUZ8+fdSrVy+nG2517NhRwcHBmjFjhu1Er1q1amnw4MGKjo6+3a8GAAAAAADgwNBBT506dbR58+br1gkMDNTo0aNv6fnNmjVTs2bNbuleAAAAAACAosb5zgAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEAQ9AAAAAAAABkHQAwAAAAAAYBAEPQAAAAAAAAZB0AMAAAAAAGAQXsXdgdulW7duSk5OdnqtUaNGio2NtSvLzs7WrFmztGLFCqWkpKh06dJ66KGHNGLECJUtW9bpc77//nt99dVXOnz4sLy9vVW/fn0NHz5ctWrVKvL3AQAAAAAAuBHDBj2SFBgYqCeffNKhPDw83O5ri8WiV155RRs3blTdunXVqlUrJSUlafHixdqyZYumT5+uMmXK2N0zffp0xcbGKiwsTD169NDly5f1448/aujQoZo4caIaNGhwW98NAAAAAADgjwwd9AQFBenZZ5+9Yb1ly5Zp48aNateuncaPHy+TySRJ+vbbb/Xee+8pNjZWf//73231jx8/rilTpqhSpUr6/PPPFRgYKEnq2bOnBg8erHfeeUdz586Vhwcr4wAAAAAAgPuQREhatGiRJOmFF16whTyS1KNHD0VEROj7779XVlaWrXzp0qXKy8vToEGDbCGPJNWsWVPt2rXTkSNHtGPHDnd1HwAAAAAAQJLBg57s7GwtXbpUM2bM0Lx587R7926HOleuXFFiYqIqV66ssLAwu2smk0nR0dHKzMzU3r17beUJCQmSpKZNmzo8LyYmRpK0ffv2onwVAAAAAACAGzL00q1z587prbfesiurXbu2/vWvfykyMlKSdOLECVksFkVFRTl9RqVKlSRJSUlJatiwoe3f/f39FRIS4lA//znHjx8vsvcAAAAAAAAoDMMGPV26dNH999+v6tWry9/fX8ePH9ecOXO0YsUKvfDCC/ryyy8VEBCgjIwMSbJbgnWtgIAASZLZbLaVmc3mAk/iyq+f/9zruXY52J0sOzvb9mteno/y8jzd0q7FYlVeXp5b2nJ3e3l5ucrKynZLWyh+144hADePMQS4hjEEuIYxhDuBr69voesaNugZNmyY3dc1a9bUuHHjJEkrVqzQokWL1K9fv+Loms2pU6fcGmS4KiUlRWazSWZzrlvay831lNl8xS1tubs9s/mKkpJOu6Ut3DlSUlKKuwtAicYYAlzDGAJcwxhCcfH09FS1atUKXd+wQU9BevTooRUrVmjXrl3q16+f0xk713I24ycwMPCG9fOfez1/POb9TpWdna2UlBSFhoYqMzNAgYGl3NKul1euAgMLn1qWpPYCA70KXC4I47l2DPn4+BR3d4AShzEEuIYxBLiGMYSS5q4LeoKDgyVJmZmZkqSIiAh5eHgoKSnJaf38vXau/aE8KipKv/32m1JTUx326cl/Tv7ePtdzM1Ov7gQ+Pj7y9PSSp6d7lm55eOS5rS13t2cyeerQIX+3tBUSYlVoqNUtbeH6fHx8Sty4B+4kjCHANYwhwDWMIZQUd13Qk3/yVv4JW76+vqpdu7Z2796t5ORku5O3rFarNm/eLD8/P91333228kaNGum3337Tpk2b1LlzZ7vnb9y4UZJsGzcDzqSnm/TNN+7524Bhw7IJegAAAADgLmHI49WPHj3qdKPjo0ePasKECZKk9u3b28off/xxSdLEiRNltf7vB+IFCxbo5MmT6tChg11y26VLF3l6emrGjBl2S7h+//13rVy5UlWrVtX9999f1K8FAAAAAABwXYac0bNy5UrNnTtX999/v8LCwuTr66vjx49r/fr1ys3N1cCBA9WoUSNb/c6dO+vHH3/UypUrderUKTVq1EgnTpxQfHy8wsPDNWLECLvnV65cWcOGDVNsbKz69eunVq1a6fLly/rxxx8lSf/4xz/k4WHIDA0AAAAAANzBDBn0NGnSREePHtX+/fu1Y8cOZWVlKTg4WA8++KB69uypmJgYu/oeHh768MMPNXPmTK1YsUJz585V6dKl9dhjj2nEiBEqU6aMQxuDBw9WeHi45s6dq2+//Vbe3t66//77NXz4cNWqVctdrwoAAAAAAGBjyKCnUaNGdjN2CsPHx0fDhg1zOJb9ejp06KAOHTrcbPcAAAAAAABuC9YXAQAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABiEV3F3AMDdw5SSIlNqapE/1xoSImtoaJE/FwAAAABKGoIeAG5jSk2Vz5QpRf7c7GHDCHoAAAAAQCzdAgAAAAAAMAyCHgAAAAAAAIMg6AEAAAAAADAIgh4AAAAAAACDIOgBAAAAAAAwCIIeAAAAAAAAgyDoAQAAAAAAMAiCHgAAAAAAAIMg6AEAAAAAADAIgh4AAAAAAACDIOgBAAAAAAAwCIIeAAAAAAAAgyDoAQAAAAAAMAiCHgAAAAAAAIMg6AEAAAAAADAIgh4AAAAAAACD8CruDgAwjpQUk1JTTQVej7hgUqkLBV+/Gb6+Vvn5FcmjAAAAAMAwCHoAFJnUVJOmTPEp8HqfOp7y3Vs0Ewlr3WeRn5+1SJ4FAAAAAEbB0i0AAAAAAACDIOgBAAAAAAAwCIIeAAAAAAAAgyDoAQAAAAAAMAiCHgAAAAAAAIPg1C3A4CwWqxIT3ZPpZma6pRkAAAAAQAEIegCDS0836ZtvCj7yvCj17JntlnYAAAAAAM6xdAsAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADIKgBwAAAAAAwCAIegAAAAAAAAyCoAcAAAAAAMAgCHoAAAAAAAAMwqu4O1DUzpw5o59++knr16/X0aNHde7cOZUuXVoNGjRQ//79VbduXbv6cXFxmjp1aoHPW7RokcLDwx3KN2zYoM8//1z79++XyWRSrVq1NGjQIEVHRxf5OwEAAAAAABSG4YKeefPmadasWYqMjFTTpk1VpkwZJSUl6ZdfftEvv/yi8ePHq23btg73de7cWWFhYQ7lQUFBDmUrVqzQ2LFjVaZMGXXu3FmStGrVKo0aNUrvvPOOWrduXfQvBgAAAAAAcAOGC3rq1Kmj2NhYNWrUyK58+/bteuGFF/Tee++pRYsW8vHxsbvepUsXNW7c+IbPv3jxoj788EMFBwdr1qxZCg0NlSQNGDBA/fv313vvvaeYmBgFBAQU3UsBAAAAAAAUguH26GnVqpVDyCNJDRs2VOPGjXXx4kUdPHjwlp//008/6dKlS+rdu7ct5JGk0NBQ9erVS2lpaVq9evUtPx8AAAAAAOBWGS7ouR4vLy+7X6+1fft2zZw5U7Nnz9bq1at1+fJlp89ISEiQJDVt2tThWkxMjCRp27ZtRdVlAAAAAACAQjPc0q2CnD59Wlu2bFFISIiqV6/ucD0uLs7u66CgIP31r3+17cGTLykpSZIUFRXl8Iz8svw6AAAAAAAA7nRXBD25ubkaO3assrOzNXLkSHl6etqu3XvvvXrzzTfVqFEjhYSE6Ny5c1q7dq3i4uL01ltvKSgoSI888oitvtlsliQFBgY6tJNfll/nRrKyslx5LbfJzs62/ZqX56O8PM8b3FE0LBar8vLy3NKWu9u7W9uyWiyyWK1F0pbVYlFenkWSlJuXp+w7eDxdO4YA3DzGEOAaxhDgGsYQ7gS+vr6Frmv4oMdisWjcuHHavn27unfvrk6dOtldb9Wqld3X4eHh6t27t6pUqaJRo0YpNjbWLugpSqdOnXJrkOGqlJQUmc0mmc25bmkvN9dTZvMVt7Tl7vbu1rZycnPlmZNTJG3l5FqUYb66xPKK2azTJWAmXUpKSnF3ASjRGEOAaxhDgGsYQygunp6eqlatWqHrGzrosVgsGj9+vH744Qd17NhRr732WqHvjY6OVmRkpA4ePCiz2WybrXPtrJ3g4GC7e64328eZ8PDwQvenOGVnZyslJUWhoaHKzAxQYGApt7Tr5ZWrwMDCp5Ylqb27tS1vLy95eXsXSVveXnkK+P9jzSsw0OlyyjvFtWPojyf+AbgxxhDgGsYQ4BrGEEoawwY9FotFb731lpYvX6527dppzJgx8vC4ub2n77nnHiUlJSkrK8sW3kRFRWnv3r1KSkpyCHqut3+PMzcz9epO4OPjI09PL7ulb7eTh0ee29pyd3t3a1smDw95mExF0pbJw0Oenlef5eXpKY8SMJ58fHxK3LgH7iSMIcA1jCHANYwhlBSGPHXr2pCnbdu2Gjdu3E3/oJuZmakjR47Iz8/PLtDJP7p906ZNDvds3LjRrg4AAAAAAIA7GS7oyV+utXz5crVu3fq6IU9GRoaOHTvmUJ6VlaW3335bGRkZatOmjd1x7G3atFFgYKDmzZtnt0YzJSVF8+fPV3BwsFq2bFnk7wUAAAAAAHAjhlu6NXXqVC1btkz+/v6qVKmSpk+f7lCnZcuWqlmzptLT09W7d2/Vrl1bVapUUbly5XT+/Hlt3rxZZ86cUY0aNTRq1Ci7e0uXLq3Ro0dr7NixGjBggNq0aSNJWrVqldLT0/X2228rICDALe8KAAAAAABwLcMFPcnJyZKky5cva8aMGU7rhIeHq2bNmipdurR69uypxMRErV+/XhcvXpSvr6+qVKmiPn36qFevXk7XYHbs2FHBwcGaMWOGli5dKpPJpFq1amnw4MGKjo6+re8HAAAAAABQEMMFPWPHjtXYsWMLVTcwMFCjR4++pXaaNWumZs2a3dK9AAAAAAAAt4Ph9ugBAAAAAAC4WxH0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGIRXcXcAAG6F1WrVhQsmSdKVCyadTLx9uXVIiFWhodbb9nwAAAAAKCoEPQBKpOxsk44cvhruZO311NeJPretrWHDsgl6AAAAAJQILN0CAAAAAAAwCIIeAAAAAAAAgyDoAQAAAAAAMAiCHgAAAAAAAIMg6AEAAAAAADAIgh4AAAAAAACDIOgBAAAAAAAwCK/i7gCAO0/b+qdUNu/sTd9X7Uqe+tTxLPB6VLkM3fxTAQAAAACFRdADwEHZvLPynRJ30/eVqmaR7+GCJwqW+lsPV7oFAAAAALgBlm4BAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQRD0AAAAAAAAGARBDwAAAAAAgEEQ9AAAAAAAABiEV3F3AABcFRmWqz7aWeTPPe9ZXj/uCi/y5wIAAADA7ULQA6DE881Kk++UBUX+3LLDnpVE0AMAAACg5GDpFgAAAAAAgEEQ9AAAAAAAABgEQQ8AAAAAAIBBEPQAAAAAAAAYBEEPAAAAAACAQXDqFlCCta1/SmXzzl63TrUreepTx/OmnhtVLkPXfyoAAAAA4E5E0AOUYGXzzsp3Stx165SqZpHv4ZubvFfqbz1c6RYAAAAAoJiwdAsAAAAAAMAgCHoAAAAAAAAMgqAHAAAAAADAIAh6AAAAAAAADILNmAHgBiwWqxITbz0Xz8vzkdlcUZmZPvL0vP5zQkKsCg213nJbAAAAAO5uBD0AcAPp6SZ9843PLd+fl+cpszlXgYGl5Ol5/aPuhw3LJugBAAAAcMtYugUAAAAAAGAQBD0AAAAAAAAGQdADAAAAAABgEOzRA1yjbf1TKpt3tsifWzbCR+dPZtu+rnYlT33qXH+vlsKIKpehou8tAAAAAKCkIugBrlE276x8p8QV/XP/1kOXpyywfV2qmkW+h12fUFfqbz1cfgbgLikpJqWmmtzSFqeXAQAA4G5F0AMAcIvUVJOmTLn108tuBqeXAQAA4G5F0OOCPXv2KC4uTrt27VJubq5q1Kihp556Sm3bti3urhne7VhiVe1KnnLKZbEUCsXKYrEqMdE926f5+lqVleWeGTaSlJnptqYAAACAuxZBzy3aunWrXnzxRZUqVUpt27aVv7+/4uPj9frrr+vMmTPq169fcXfR0G7HEqtS1Szy6NmzSJ8J3Kz0dJO++cY9s1569sx2W1v57QEAAAC4vQh6bkFubq7eeecdeXh46LPPPlPNmjUlSUOHDtXAgQM1adIkPfroowoLCyvmngLA3cmoM6PYewgAAAA3QtBzC7Zu3aoTJ06oa9eutpBHkgIDAzVw4EC99dZbWrZsmYYOHVqMvSxanp6e//9XqwICCv9DRss6pxWcd+6W2qxsyVO/xs5Ppoooc0XnggNu6bkFMQVZZPLxklcRP1eSw3NNQRZ5Bbv+Q2hh+nsrbd3q53Cjtory8722rdv1++bl66mAAKu8vG7u+/6PLBarTCYpIMAqD4/rP8fVtm6GO9tyd3sZGdKSJd5uaatr12y3tdW/f7ZCQ93SlM6ckc6fd0+AVbasVRUqXL9O/v+HSpI77TPE3a0kjiHgTsIYQkliSktL468Gb9KkSZP0+eef61//+pfatWtndy01NVWdOnVSkyZNNGnSpGLqIQAAAAAAuBu5Z167wRw/flySFBUV5XAtJCRE/v7+SkpKcne3AAAAAADAXY6g5xZkZGRIurpUy5mAgACZzWZ3dgkAAAAAAICgBwAAAAAAwCgIem5BQMDVTV8LmrWTkZFR4GwfAAAAAACA24Wg5xZUqlRJkpzuw5OamqrLly873b8HAAAAAADgdiLouQUNGzaUJG3atMnh2saNGyVJjRo1cmufAAAAAAAACHpuwQMPPKCIiAj98MMP+v33323lZrNZn3/+uby9vdWpU6di7CEAAAAAALgbmdLS0qzF3YmSaOvWrXrxxRdVqlQptW3bVv7+/oqPj1dycrJeeukl9evXr7i76LI9e/YoLi5Ou3btUm5urmrUqKGnnnpKbdu2Le6uAW515swZ/fTTT1q/fr2OHj2qc+fOqXTp0mrQoIH69++vunXrOtxjNps1ZcoUxcfH69y5cwoJCVHr1q01dOhQ+fv7O9S3WCyaP3++Fi1apBMnTsjPz0/R0dF67rnnFBER4Y7XBNxu5syZmjhxoiRp2rRpqlevnt11xhHgKD4+Xt9++6327dunrKwslStXTvXq1dOoUaMUGhpqq8f4AexZrVatXr1a8+bN09GjR2U2mxUaGqrGjRtrwIABDt/njCGUZAQ9LkhMTHQIQvr27WuIIORuCLKAwpowYYJmzZqlyMhINWrUSGXKlFFSUpJ++eUXWa1WjR8/3m7cZ2ZmatiwYfr999/VtGlT/elPf9L+/fu1adMm1a5dW5999plKlSpl18bbb7+txYsXq1q1amrevLlSU1O1atUq+fn5afr06ba9wQCjOHTokJ555hl5enoqMzPTIehhHAH2rFar/v3vf2vhwoWKjIxUTEyM/P39dfbsWW3fvl1vvfWW7r//fkmMH8CZTz75RF9++aVCQkL0yCOPKCAgQAcOHNCmTZvk7++vqVOnqnr16pIYQyj5CHrgIDc3V71799aZM2c0ffp01axZU9LVVHvgwIFKTk7WN998o7CwsGLuKeAe8fHxuueeexz23tq+fbteeOEF+fv7a/ny5fLx8ZEkxcXFaerUqRowYIBGjhxpq58fGD3//PMaOHCgrXzr1q16/vnn1bBhQ02YMEHe3t6SpPXr1+vPf/6zmjZtqv/+97+3/0UBN8nNzdXgwYPl5eWlqKgorVixwiHoYRwB9r766iv93//9n3r27KmXX35Znp6edtdzc3Pl5eUlifED/FFqaqq6dOmi0NBQzZkzx+6E5C+//FKffPKJunbtqjfffFMSYwglH3v0wMHWrVt14sQJtW/f3hbySFJgYKAGDhyonJwcLVu2rBh7CLhXq1atnG6w3rBhQzVu3FgXL17UwYMHJV39G9fFixfL399fQ4YMsas/ZMgQ+fv7a/HixXblixYtkiQNHz7c9gcDSXrwwQfVuHFjbdq0SadPny7itwKKz/Tp03X48GG98cYb8vBw/KMI4wiwl5WVpalTpyoiIkJ//etfHUIeSbaQh/EDOEpOTpbFYlH9+vXtQh5JeuihhyRJFy5ckMQYgjEQ9MDBtm3bJElNmzZ1uBYTE2NXB7jb5f/BOv/X48eP6+zZs6pfv778/Pzs6vr5+al+/fo6efKkUlJSbOXbtm2Tn5+fGjRo4PB8xhyMZt++fZoxY4aGDh2qatWqOa3DOALsbdq0SRcvXlSLFi1ksVgUHx+vmTNn6ttvv1VSUpJdXcYP4CgqKkre3t7atWuXzGaz3bW1a9dKunrgjsQYgjF4FXcHcOc5fvy4pKv/QfyjkJAQ+fv7O/yhArgbnT59Wlu2bFFISIhtTXf+2HA2fvLLN27cqOPHjys0NFSZmZlKTU1V9erVnf4Nbf5zGHMwguzsbP3zn/9UzZo11b9//wLrMY4Ae/v27ZMkeXh4qG/fvrY/q+WXPfXUU3rppZckMX4AZ4KDg/XCCy/o008/Ve/eve326Nm6dat69uypXr16SWIMwRgIeuAgIyNDkhymNeYLCAhwSMKBu01ubq7Gjh2r7OxsjRw50vY/9vyxcb3xc229m60PlGSfffaZkpKSNGvWLKd/GM7HOALsnT9/XpI0d+5c/elPf9Lnn3+uKlWqaP/+/Xr33Xc1Z84cRUREqGfPnowfoAB9+/ZV+fLl9c4772jBggW28gYNGqh9+/a22dmMIRgBS7cA4CZZLBaNGzdO27dvV/fu3dWpU6fi7hJwx9u1a5fmzJmjwYMH22bAASgcq/Xq2SleXl764IMPVLt2bfn7+6thw4Z699135eHhoS+//LKYewnc2aZOnaqxY8dq4MCBWrJkiVavXq24uDhlZ2drxIgR+vXXX4u7i0CRIeiBgxulzhkZGQUm1oDRWSwWjR8/Xj/88IM6duyo1157ze56/ti43vi5tt7N1gdKotzcXI0bN041atTQM888c8P6jCPAXv737n333afy5cvbXatevboiIiJ04sQJXbp0ifEDOLF582bFxcWpV69eeuaZZxQaGip/f3/df//9+uijj+Tl5aVPP/1UEv8PgjGwdAsOKlWqJOnqOtL77rvP7lpqaqouX76s2rVrF0fXgGJlsVj01ltvafny5WrXrp3GjBnjcGLQjdZh55fnjzM/Pz+FhITo1KlTysvLc1jOcqN14kBJkJmZaftefvDBB53WyT/Z5P3331fVqlUlMY6AfPnf60FBQU6v5/8AeeXKFf4/BDixfv16SVKTJk0croWEhNiWQl6+fJkxBENgRg8cNGzYUNLVEx7+aOPGjZLk9KhpwMiuDXnatm2rcePGOd1jpFKlSipfvrx27dqlzMxMu2uZmZnatWuXwsPDFRoaaitv1KiRMjMztXPnTofn5Y+5/HEJlETe3t567LHHnP6T/wffRx55RI899pjCw8MZR8Af5P9wevToUYdrubm5OnHihPz8/BQcHMz4AZzIycmR9L8j1P/owoUL8vDwkJeXF2MIhkDQAwcPPPCAIiIi9MMPP+j333+3lZvNZn3++efy9vZmTxLcVfKXay1fvlytW7cuMOSRJJPJpG7duuny5cuaNm2a3bVp06bp8uXL6t69u115/tefffaZ7Q8i0tW/fUpISFDTpk0VFhZWpO8EuJOvr6/eeOMNp//Ur19fkvTMM8/ojTfeUM2aNRlHwB9ERkaqadOmSkpK0qJFi+yuzZw5U5cuXVLLli3l5eXF+AGcyD/2/Msvv3RYYvXtt9/qzJkzqlevnnx8fBhDMARTWlqatbg7gTvP1q1b9eKLL6pUqVJq27at/P39FR8fr+TkZL300kvq169fcXcRcJu4uDhNnTpV/v7+6tOnj9OQp2XLlqpZs6akq3/bM3ToUB04cEBNmzZVrVq1tG/fPm3atEm1a9dWbGysfH197e5/++23tXjxYlWrVk3NmzdXamqqVq1aJT8/P02bNk2VK1d2y7sC7jZu3DgtW7ZM06ZNU7169WzljCPA3okTJzR06FCdP39ezZs3ty012bp1q8LCwjRt2jSFhIRIYvwAf5SXl6fnn39e27dvV9myZfXwww8rKChI+/bt09atW1WqVCnFxsaqTp06khhDKPkIelCgxMRExcXFadeuXcrNzVWNGjXUt29ftW3btri7BrhV/g+i1zNmzBh16dLF9rXZbFZcXJzi4+N17tw5hYSEqHXr1ho69P+1d/dRWdf3H8efcKncXTgFpyYgUiwT50oIpjVutjRRUVqErtQzzybe7EYb5bJzmiRSZ9qNOzGyUtdQYRviIeNmbiJkpQkKSEQe8wZREIQGIaLIhOv3B4drXnFd4l2h/F6PP7/fz+f7/Xw/xDnx8vN5fxaYC55fqaOjg23btpGRkWFegh8UFMSSJUvw9PS85d8kcruwFfSAfo9Evu7s2bO8/fbbfPLJJzQ1NeHu7k5ISAi//OUvcXNzs2ir3x8RS21tbfztb38jNzeXyspK/vvf/+Lu7k5AQADz588314frot8huZMp6BERERERERER6SNUo0dEREREREREpI9Q0CMiIiIiIiIi0kco6BERERERERER6SMU9IiIiIiIiIiI9BEKekRERERERERE+ggFPSIiIiIiIiIifYSCHhERERERERGRPkJBj4iIiIiIiIhIH6GgR0RERET6nKysLIKCgli8eHFvD0VERORb1a+3ByAiIiK3r8WLF1NcXAyAl5cX27dvv2r7RYsWUVJSAoC3tzfbtm37xsco3QUFBQGwfv16AgICenk0t15RURFFRUXce++9hIWF9fZwREREbita0SMiIiLX5PTp0xw6dMjm/aqqqqveF7lVioqK2LhxI3v27OntoYiIiNx2FPSIiIhIj3x8fADIzMy02SYrKwuTyWRuKyIiIiLfPgU9IiIi0qMf//jHODs7k5eXR2tra7f7HR0dZGdnYzAYmDp1ai+MUERERERANXpERETkGjg5OfHII4+QmZnJ7t27mT59usX9AwcOcPbsWR5++GGGDBnS4/MOHjxIeno6ZWVlNDY24uzszJgxY3jiiScIDQ212ufAgQOkpaVRXl5OY2Mjjo6ODBo0CF9fX4KDg5k5c6ZF+9OnT7NlyxYOHjxIXV0ddnZ2DBo0CE9PTwIDA3nyySdxcnIyt6+srOSDDz5g//79VFdX09DQwIABA/Dx8eHRRx/l8ccfp3///lbH1tbWRmpqKv/85z+prq7GaDQyfvx4FixYwFdffcWSJUu466672LFjh9X++fn5vP/++xw+fJhz584xcOBAvv/97/PUU0/h7+/f43zeKqdPnyY1NZXCwkLq6uowGAx4e3sTHh5OVFQUAwYMsGh/5swZHnvsMQAKCws5dOgQycnJlJWVcenSJby8vHjssceIjo7Gzs7O6jsLCwvZvHkz5eXl5hVhUVFRREREEBkZSU1NjUWtoa76QwDZ2dlkZ2d3e541WVlZbN++nRMnTmBvb899993H/Pnz+eEPf3ij0yUiInJbUtAjIiIi1yQiIoLMzEyysrK6BT1dW7pmzJjBhQsXbD7DZDLx2muvkZaWBsDAgQO55557qK+vp6CggIKCAqKjo1m+fLlFv/fee4+XX34ZAFdXV+6++25MJhN1dXXs2bOHw4cPWwQ9R44cYfHixbS0tODg4ICHhwcODg7U19dTUlJCUVERkydPxsvLy9xn/fr15OXl4ezsjLu7O76+vjQ2NlJWVkZZWRn5+fkkJiZ2C3taW1tZtmyZuQi1p6cnRqORjz/+mL1797JgwQKb89HW1sbKlSvJy8sDYPDgwdxzzz3U1NTw4Ycf8tFHH/Hb3/6WuXPn2nzGrbJz504SEhJoa2vDwcEBT09PWltbOXLkCIcPHyY3N5c33ngDFxcXq/2zsrJISEjAaDTi4eFBbW0tx44d49VXX6WmpoZly5Z165Oens7atWuBzp+rl5cXdXV1xMfHc/z4cavvuf/++6mtreXs2bO4ublZ/AxtWb16NZmZmQwbNgxvb28qKyspKiqipKSENWvW2AwXRURE7kQKekREROSajB8/Hi8vL4qLi6mursbDwwOA5uZm9uzZw3e+8x2Cg4P517/+ZfMZW7ZsIS0tjaFDh/Lcc88RHBxsvvfJJ5+watUqtm3bxtixY5k2bRoA7e3tJCUlAfDMM88QFRVFv37/+1+YkydPUlBQYPGeDRs20NLSwtSpU1m+fDlGo9F8r7Gxkdzc3G6BxdSpU5k3bx5+fn4Wq09OnjxJfHw8xcXFpKam8vOf/9yi3zvvvENJSQmurq6sWbOGBx98EIDz58+zevVq3nrrLZvzsW7dOvLy8rj77rt5/vnnuf/++833du7cycsvv0xiYiJ+fn7f6Mqe0tJSVq1ahb29PbGxsURFRZkDrcrKSuLi4igrK+P111/nD3/4g9Vn/PGPf2TZsmXMmjULg8EAwF//+lfefPNNUlNTiYqKwtPT09z+6NGjvPbaawDMnTuXJUuWmN+ZlZXFSy+9ZHUV0IYNG3jnnXfYuHEjEydOJC4u7qrf9umnn3L8+HESExPNq3cuXrxIXFwcH3zwAa+//johISE2VxyJiIjcaVSjR0RERK7Z9OnTMZlMFttl/v3vf3Pp0iXCw8Ntbm0COHfuHH/5y18wGAysXbvWIuQBmDhxIs899xwAycnJ5uuNjY00NTXh6urK7NmzLUIegFGjRjF79myLa5WVlQDMmTPHIuSBzlUz0dHRuLm5WVwPDQ1l7Nix3f7gHzVqFKtWrQLotk2opaXFfOT8s88+aw55AIxGI/Hx8Xz3u9+1Oh+VlZVkZGTg4uLCunXrLEIegPDwcBYtWoTJZGLz5s1Wn3Gr/PnPf6a9vZ1f//rX/OxnP7P4OXp7e7NmzRqcnJzIycmhvr7e6jPCw8N58sknzSEPwPz58/H19cVkMrF3716L9ikpKbS3txMYGMjSpUst3hkREcGcOXO4fPnyTX/b5cuXiY2Ntdii5eTkxO9//3v69etHTU0Nx44du+n3iIiI3C4U9IiIiMg1mz59Ovb29mRnZ2MymYDO1RfQuW3ravbt28eFCxcYM2YMfn5+VtsEBwfTr18/Kioq+PLLLwFwc3PDwcGB8+fPdwsLbBk+fDgAu3btoqOj45r6ADQ0NPCPf/yDlStX8pvf/IaYmBhiYmKIj48HOsOZK4tRl5aWcvHiRVxcXJg8eXK35zk4OJhXJn3d7t276ejo4KGHHuKuu+6y2uYnP/kJ0HmceHt7+zV/x/Woq6ujtLQUg8FAZGSk1TbDhg3Dz8+P9vZ2iouLrbaJjo62en3cuHFAZ/2fK+3fvx/A5jt/+tOfXtP4e2I0GgkPD+92fciQIYwYMQKAqqqqW/IuERGR24G2bomIiMg1GzZsGIGBgRQUFHDw4EHc3d0pLy9n9OjR3HvvvVft+8UXXwCdBXxjYmJstutaUXP27FmGDBmCvb09c+fOZdOmTfzud7/D19eXBx98kHHjxjF+/HirxZ/nzZvHgQMHSE5OJicnhwkTJjBu3DgeeOABRo0aZfW9u3fvZvXq1T3WGDp37hyOjo7A/1YO+fj4dFtp1MXWvBw9ehSAsrIym/PRFaZdunSJpqambquQboWucRgMBp5++mmb7U6dOgV0/lysGTlypNXrXWO+cl6bm5tpaGgAbM+Ph4cHLi4utLS0XP0DeuDl5WVzW5abmxunTp266XeIiIjcThT0iIiIyHWJiIigoKCArKws3N3dzdd60tzcDHSumun6I/9qrlw5s3DhQoYPH05aWhpHjx7l2LFj/P3vf8fOzs689efKwCAoKIg333yTd999l6KiIjIzM80Fo318fFi0aJF5tQx0hk9xcXG0tbUxadIkZs+ejbe3N0ajkX79+tHR0cGECRMALLYTdYUXzs7ONr/DVvHirvmora2ltrb2uubjVuoaR1tbG6WlpTc8jitPMLtSV8jSFVqBZejT09zdbAhja1wA9vZa3C4iIn2Pgh4RERG5LmFhYbi6upKfn4+zszP9+/dnypQpPfbr+oN72rRpvPjii9f1Tjs7OyIjI4mMjKSxsZHS0lKKi4vJzc2lsLCQX/3qV6SmpjJ06FBzH39/f/z9/WltbaWsrIzS0lLy8vI4duwYK1as4E9/+hMPPfQQ0LnFq62tjbFjx5KQkNAtAGhqarI6rq6Q4mqrgGwFFV3zsWDBAhYuXHjtk3GLdY1j+PDhvP/++9/KO68Md25k7kRERMQ2/TOGiIiIXBcHBwcmTZpEa2srDQ0NBAcHM2jQoB77+fr6Atx04dvBgwcTFhZGbGws27ZtY8SIEZw7d45du3ZZbe/o6EhgYCALFiwgJSXFvJInPT3d3ObMmTNA59Hd1lZ5fPbZZ1af7e3tDUBFRYXNwsFdW6O+rms+bN3/tnSNo66uzmagdau5urqat3TZ+v7q6mqbQY9OyBIREbFNQY+IiIhct8cff5zAwEACAwNtFuH9uh/96Ec4ODjwxRdfdDsO/Ua5uLhYBBU9sbOzM59udeXpUQ4ODgDmAtBXMplMbN261erzHnjgAZycnGhpaSE3N7fb/ba2NnJycqz2feSRR7Czs2Pv3r2cOHGix7F/Uzw8PLjvvvvo6OggJSXlW3tv11Y4W6uIduzYYbNvV42kb2o7m4iIyJ1MQY+IiIhct9GjR5OUlERSUhIBAQHX1MfNzY1f/OIXADz//PNkZ2d3WwXT1NREdnY2b7zxhvnaiRMnWL16NYcOHep2glZXUWjA4iSvFStWkJ+f3y0IqKqq4r333uvW3t/fH+gsyPzxxx+br7e0tJCQkMDnn39u9ZucnZ154oknAHj11VctTqQ6f/48cXFxNgMoX19fIiMjuXz5MkuXLuWjjz6yqGMDnWFUenq6xXHz34Snn34ag8FAcnIy69evN9ft6XLp0iX27dvHihUrbtk758yZg8FgoKCggKSkJIv/FnJycti6davNAteenp4AlJeXX3Xrl4iIyP9HqtEjIiIi35r58+fT3NzM1q1bWbVqFa+88gojR47EYDDQ0NBAbW0tJpPJHLxAZ/HjrmLKjo6OeHp6MmDAAOrr682rckJCQiyONy8sLCQvLw+DwYCnpyeurq40NTVRVVWFyWTCy8vLoi5OSEgIAQEBFBUVERsby4gRIxg4cCAnT56kra2NlStX2qwrFBMTw2effUZJSQmLFy/Gy8sLo9FIRUUFJpOJRYsWkZSUZHVL2PLly2ltbWXnzp0888wzDBw4EA8PD6BzdVHX902fPv2G5vvZZ5+1GZYAPProoyxfvhx/f3/i4+NJSEjg3XffZfPmzXh7e+Ps7ExzczPV1dU2t6bdqO9973vExsbyyiuvkJycTEZGBp6ennz55ZfU1dXx1FNPkZ+fT01NTbe5mzBhAm5ubtTW1jJjxgy8vb0ZMGAAAG+99dYtHaeIiMidRkGPiIiIfGvs7OxYunQpkyZNYvv27ZSUlFBRUUF7ezuDBw9mwoQJPPzww4SGhpr7jBw5khdeeIEDBw5w5MgR6urqaGlpwdXVlaCgIKZNm0Z4eLhFGPDiiy9SUFDAp59+Sn19PVVVVTg6OjJmzBhCQ0OZNWuWxWlY9vb2rFu3jk2bNrFr1y7q6uq4ePEiAQEBzJs3D39/f5tBj6OjI4mJiaSkpJCTk0NNTQ3nz59n4sSJxMTEUFNTA1g/fat///7Ex8cTERHBjh07KCsr4/jx4wC4u7sTGhpKcHAwISEhNzTfPRUzvnI1zOTJk/nBD35AWloa+/fv58yZM7S2tmI0GvHz8yMoKIiwsLAbGoct0dHRjBw5ks2bN/P5559TUVHBqFGjWLhwITNnziQ7OxvoPndOTk4kJSWxYcMGSktLOXz4MO3t7bd0bCIiIncqu6+++srUczMRERERuRFbtmwhMTGRsLAw1q5d29vDuWM0NjYyZcoU7Ozs2L17N0ajsbeHJCIickdQjR4RERGRb8jly5fNq1LGjx/fy6O5s3QVY/b19VXIIyIich0U9IiIiIjcpLfffpvKykqLa//5z3944YUXOHHiBK6urkydOrWXRnf72rlzJ/v27bPYdtXe3k5GRgYbN24EYNasWb01PBERkTuSavSIiIiI3KT09HQ2bdrE0KFDGTp0KC0tLZw6dYr29nYGDBhAXFwcgwYN6u1h3naOHDlCSkoKzs7OeHl5YW9vz6lTp8y1haZMmcLMmTN7eZQiIiJ3FtXoEREREblJ27dv58MPP+T48eM0NTUBMGTIEAICApgzZw4+Pj69PMLbU3l5ORkZGZSWltLQ0MCFCxdwdXVl9OjRzJgxg8mTJ2NnZ9fbwxQREbmjKOgREREREREREekjVKNHRERERERERKSPUNAjIiIiIiIiItJHKOgREREREREREekjFPSIiIiIiIiIiPQRCnpERERERERERPoIBT0iIiIiIiIiIn2Egh4RERERERERkT5CQY+IiIiIiIiISB+hoEdEREREREREpI/4P6tkolVXjOuRAAAAAElFTkSuQmCC"},"metadata":{}}]},{"cell_type":"markdown","source":"> Very interesting! Through just basic EDA we've been able to discover a trend that spam messages tend to have more characters.","metadata":{}},{"cell_type":"code","source":"sms[sms.label=='ham'].describe()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:15:47.436078Z","iopub.execute_input":"2023-05-14T15:15:47.436406Z","iopub.status.idle":"2023-05-14T15:15:47.457549Z","shell.execute_reply.started":"2023-05-14T15:15:47.436366Z","shell.execute_reply":"2023-05-14T15:15:47.456136Z"},"trusted":true},"execution_count":26,"outputs":[{"execution_count":26,"output_type":"execute_result","data":{"text/plain":" label_num message_len\ncount 4825.0 4825.000000\nmean 0.0 71.023627\nstd 0.0 58.016023\nmin 0.0 2.000000\n25% 0.0 33.000000\n50% 0.0 52.000000\n75% 0.0 92.000000\nmax 0.0 910.000000","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
label_nummessage_len
count4825.04825.000000
mean0.071.023627
std0.058.016023
min0.02.000000
25%0.033.000000
50%0.052.000000
75%0.092.000000
max0.0910.000000
\n
"},"metadata":{}}]},{"cell_type":"code","source":"sms[sms.label=='spam'].describe()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:15:54.083529Z","iopub.execute_input":"2023-05-14T15:15:54.083862Z","iopub.status.idle":"2023-05-14T15:15:54.102463Z","shell.execute_reply.started":"2023-05-14T15:15:54.083841Z","shell.execute_reply":"2023-05-14T15:15:54.101565Z"},"trusted":true},"execution_count":27,"outputs":[{"execution_count":27,"output_type":"execute_result","data":{"text/plain":" label_num message_len\ncount 747.0 747.000000\nmean 1.0 138.866131\nstd 0.0 29.183082\nmin 1.0 13.000000\n25% 1.0 132.500000\n50% 1.0 149.000000\n75% 1.0 157.000000\nmax 1.0 224.000000","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
label_nummessage_len
count747.0747.000000
mean1.0138.866131
std0.029.183082
min1.013.000000
25%1.0132.500000
50%1.0149.000000
75%1.0157.000000
max1.0224.000000
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"> Woah! 910 characters, let's use masking to find this message:","metadata":{}},{"cell_type":"code","source":"sms[sms.message_len == 910].message.iloc[0]","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:16:05.915788Z","iopub.execute_input":"2023-05-14T15:16:05.916132Z","iopub.status.idle":"2023-05-14T15:16:05.926099Z","shell.execute_reply.started":"2023-05-14T15:16:05.916109Z","shell.execute_reply":"2023-05-14T15:16:05.924312Z"},"trusted":true},"execution_count":28,"outputs":[{"execution_count":28,"output_type":"execute_result","data":{"text/plain":"\"For me the love should start with attraction.i should feel that I need her every time around me.she should be the first thing which comes in my thoughts.I would start the day and end it with her.she should be there every time I dream.love will be then when my every breath has her name.my life should happen around her.my life will be named to her.I would cry for her.will give all my happiness and take all her sorrows.I will be ready to fight with anyone for her.I will be in love when I will be doing the craziest things for her.love will be when I don't have to proove anyone that my girl is the most beautiful lady on the whole planet.I will always be singing praises for her.love will be when I start up making chicken curry and end up makiing sambar.life will be the most beautiful then.will get every morning and thank god for the day because she is with me.I would like to say a lot..will tell later..\""},"metadata":{}}]},{"cell_type":"markdown","source":"# ๐Ÿ“‘ Text Pre-processing\n\n> Our main issue with our data is that it is all in text format (strings). The classification algorithms that we usally use need some sort of numerical feature vector in order to perform the classification task. There are actually many methods to convert a corpus to a vector format. The simplest is the `bag-of-words` approach, where each unique word in a text will be represented by one number.\n\n\n> In this section we'll convert the raw messages (sequence of characters) into vectors (sequences of numbers).\n\n> As a first step, let's write a function that will split a message into its individual words and return a list. We'll also remove very common words, ('the', 'a', etc..). To do this we will take advantage of the `NLTK` library. It's pretty much the standard library in Python for processing text and has a lot of useful features. We'll only use some of the basic ones here.\n\n> Let's create a function that will process the string in the message column, then we can just use **apply()** in pandas do process all the text in the DataFrame.\n\n>First removing punctuation. We can just take advantage of Python's built-in **string** library to get a quick list of all the possible punctuation:","metadata":{}},{"cell_type":"code","source":"import string\nfrom nltk.corpus import stopwords\n\ndef text_process(mess):\n \"\"\"\n Takes in a string of text, then performs the following:\n 1. Remove all punctuation\n 2. Remove all stopwords\n 3. Returns a list of the cleaned text\n \"\"\"\n STOPWORDS = stopwords.words('english') + ['u', 'รผ', 'ur', '4', '2', 'im', 'dont', 'doin', 'ure']\n # Check characters to see if they are in punctuation\n nopunc = [char for char in mess if char not in string.punctuation]\n\n # Join the characters again to form the string.\n nopunc = ''.join(nopunc)\n \n # Now just remove any stopwords\n return ' '.join([word for word in nopunc.split() if word.lower() not in STOPWORDS])","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:16:17.292118Z","iopub.execute_input":"2023-05-14T15:16:17.292472Z","iopub.status.idle":"2023-05-14T15:16:18.22433Z","shell.execute_reply.started":"2023-05-14T15:16:17.29245Z","shell.execute_reply":"2023-05-14T15:16:18.223056Z"},"trusted":true},"execution_count":29,"outputs":[]},{"cell_type":"code","source":"sms.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:16:19.819426Z","iopub.execute_input":"2023-05-14T15:16:19.820532Z","iopub.status.idle":"2023-05-14T15:16:19.830144Z","shell.execute_reply.started":"2023-05-14T15:16:19.820478Z","shell.execute_reply":"2023-05-14T15:16:19.829032Z"},"trusted":true},"execution_count":30,"outputs":[{"execution_count":30,"output_type":"execute_result","data":{"text/plain":" label message label_num \\\n0 ham Go until jurong point, crazy.. Available only ... 0 \n1 ham Ok lar... Joking wif u oni... 0 \n2 spam Free entry in 2 a wkly comp to win FA Cup fina... 1 \n3 ham U dun say so early hor... U c already then say... 0 \n4 ham Nah I don't think he goes to usf, he lives aro... 0 \n\n message_len \n0 111 \n1 29 \n2 155 \n3 49 \n4 61 ","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
labelmessagelabel_nummessage_len
0hamGo until jurong point, crazy.. Available only ...0111
1hamOk lar... Joking wif u oni...029
2spamFree entry in 2 a wkly comp to win FA Cup fina...1155
3hamU dun say so early hor... U c already then say...049
4hamNah I don't think he goes to usf, he lives aro...061
\n
"},"metadata":{}}]},{"cell_type":"markdown","source":"> Now let's \"tokenize\" these messages. Tokenization is just the term used to describe the process of converting the normal text strings in to a list of tokens (words that we actually want).","metadata":{}},{"cell_type":"code","source":"sms['clean_msg'] = sms.message.apply(text_process)\n\nsms.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:16:26.968634Z","iopub.execute_input":"2023-05-14T15:16:26.968987Z","iopub.status.idle":"2023-05-14T15:16:27.788059Z","shell.execute_reply.started":"2023-05-14T15:16:26.968963Z","shell.execute_reply":"2023-05-14T15:16:27.786732Z"},"trusted":true},"execution_count":31,"outputs":[{"execution_count":31,"output_type":"execute_result","data":{"text/plain":" label message label_num \\\n0 ham Go until jurong point, crazy.. Available only ... 0 \n1 ham Ok lar... Joking wif u oni... 0 \n2 spam Free entry in 2 a wkly comp to win FA Cup fina... 1 \n3 ham U dun say so early hor... U c already then say... 0 \n4 ham Nah I don't think he goes to usf, he lives aro... 0 \n\n message_len clean_msg \n0 111 Go jurong point crazy Available bugis n great ... \n1 29 Ok lar Joking wif oni \n2 155 Free entry wkly comp win FA Cup final tkts 21s... \n3 49 dun say early hor c already say \n4 61 Nah think goes usf lives around though ","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
labelmessagelabel_nummessage_lenclean_msg
0hamGo until jurong point, crazy.. Available only ...0111Go jurong point crazy Available bugis n great ...
1hamOk lar... Joking wif u oni...029Ok lar Joking wif oni
2spamFree entry in 2 a wkly comp to win FA Cup fina...1155Free entry wkly comp win FA Cup final tkts 21s...
3hamU dun say so early hor... U c already then say...049dun say early hor c already say
4hamNah I don't think he goes to usf, he lives aro...061Nah think goes usf lives around though
\n
"},"metadata":{}}]},{"cell_type":"code","source":"type(stopwords.words('english'))","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:16:31.050921Z","iopub.execute_input":"2023-05-14T15:16:31.051292Z","iopub.status.idle":"2023-05-14T15:16:31.059099Z","shell.execute_reply.started":"2023-05-14T15:16:31.051269Z","shell.execute_reply":"2023-05-14T15:16:31.057929Z"},"trusted":true},"execution_count":32,"outputs":[{"execution_count":32,"output_type":"execute_result","data":{"text/plain":"list"},"metadata":{}}]},{"cell_type":"code","source":"from collections import Counter\n\nwords = sms[sms.label=='ham'].clean_msg.apply(lambda x: [word.lower() for word in x.split()])\nham_words = Counter()\n\nfor msg in words:\n ham_words.update(msg)\n \nprint(ham_words.most_common(50))","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:16:31.719527Z","iopub.execute_input":"2023-05-14T15:16:31.719854Z","iopub.status.idle":"2023-05-14T15:16:31.751447Z","shell.execute_reply.started":"2023-05-14T15:16:31.719832Z","shell.execute_reply":"2023-05-14T15:16:31.749763Z"},"trusted":true},"execution_count":33,"outputs":[{"name":"stdout","text":"[('get', 303), ('ltgt', 276), ('ok', 272), ('go', 247), ('ill', 236), ('know', 232), ('got', 231), ('like', 229), ('call', 229), ('come', 224), ('good', 222), ('time', 189), ('day', 187), ('love', 185), ('going', 167), ('want', 163), ('one', 162), ('home', 160), ('lor', 160), ('need', 156), ('sorry', 153), ('still', 146), ('see', 137), ('n', 134), ('later', 134), ('da', 131), ('r', 131), ('back', 129), ('think', 128), ('well', 126), ('today', 125), ('send', 123), ('tell', 121), ('cant', 118), ('รฌ', 117), ('hi', 117), ('take', 112), ('much', 112), ('oh', 111), ('night', 107), ('hey', 106), ('happy', 105), ('great', 100), ('way', 100), ('hope', 99), ('pls', 98), ('work', 96), ('wat', 95), ('thats', 94), ('dear', 94)]\n","output_type":"stream"}]},{"cell_type":"code","source":"words = sms[sms.label=='spam'].clean_msg.apply(lambda x: [word.lower() for word in x.split()])\nspam_words = Counter()\n\nfor msg in words:\n spam_words.update(msg)\n \nprint(spam_words.most_common(50))","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:16:43.850858Z","iopub.execute_input":"2023-05-14T15:16:43.851171Z","iopub.status.idle":"2023-05-14T15:16:43.867876Z","shell.execute_reply.started":"2023-05-14T15:16:43.85115Z","shell.execute_reply":"2023-05-14T15:16:43.866461Z"},"trusted":true},"execution_count":34,"outputs":[{"name":"stdout","text":"[('call', 347), ('free', 216), ('txt', 150), ('mobile', 123), ('text', 120), ('claim', 113), ('stop', 113), ('reply', 101), ('prize', 92), ('get', 83), ('new', 69), ('send', 67), ('nokia', 65), ('urgent', 63), ('cash', 62), ('win', 60), ('contact', 56), ('service', 55), ('please', 52), ('guaranteed', 50), ('customer', 49), ('16', 49), ('week', 49), ('tone', 48), ('per', 46), ('phone', 45), ('18', 43), ('chat', 42), ('awarded', 38), ('draw', 38), ('latest', 36), ('รฅยฃ1000', 35), ('line', 35), ('150ppm', 34), ('mins', 34), ('receive', 33), ('camera', 33), ('1', 33), ('every', 33), ('message', 32), ('holiday', 32), ('landline', 32), ('shows', 31), ('รฅยฃ2000', 31), ('go', 31), ('box', 30), ('number', 30), ('apply', 29), ('code', 29), ('live', 29)]\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# ๐Ÿงฎ Vectorization\n\n> Currently, we have the messages as lists of tokens (also known as [lemmas](http://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html)) and now we need to convert each of those messages into a vector the SciKit Learn's algorithm models can work with.\n\n> Now we'll convert each message, represented as a list of tokens (lemmas) above, into a vector that machine learning models can understand.\n\n> We'll do that in three steps using the bag-of-words model:\n\n> 1. Count how many times does a word occur in each message (Known as term frequency)\n> 2. Weigh the counts, so that frequent tokens get lower weight (inverse document frequency)\n> 3. Normalize the vectors to unit length, to abstract from the original text length (L2 norm)\n\n> Let's begin the first step:\n\n> Each vector will have as many dimensions as there are unique words in the SMS corpus. We will first use SciKit Learn's **CountVectorizer**. This model will convert a collection of text documents to a matrix of token counts.\n\n> We can imagine this as a 2-Dimensional matrix. Where the 1-dimension is the entire vocabulary (1 row per word) and the other dimension are the actual documents, in this case a column per text message. \n\n> For example:\n\n\n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n
Message 1 Message 2 ... Message N
Word 1 Count01...0
Word 2 Count00...0
... 12...0
Word N Count 01...1
\n\n\n> Since there are so many messages, we can expect a lot of zero counts for the presence of that word in that document. Because of this, SciKit Learn will output a [Sparse Matrix](https://en.wikipedia.org/wiki/Sparse_matrix).","metadata":{}},{"cell_type":"code","source":"# split X and y into training and testing sets \nfrom sklearn.model_selection import train_test_split\n\n# how to define X and y (from the SMS data) for use with COUNTVECTORIZER\nX = sms.clean_msg\ny = sms.label_num\nprint(X.shape)\nprint(y.shape)\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n\nprint(X_train.shape)\nprint(X_test.shape)\nprint(y_train.shape)\nprint(y_test.shape)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:30.512367Z","iopub.execute_input":"2023-05-14T15:17:30.512683Z","iopub.status.idle":"2023-05-14T15:17:30.52231Z","shell.execute_reply.started":"2023-05-14T15:17:30.512662Z","shell.execute_reply":"2023-05-14T15:17:30.521093Z"},"trusted":true},"execution_count":36,"outputs":[{"name":"stdout","text":"(5572,)\n(5572,)\n(4179,)\n(1393,)\n(4179,)\n(1393,)\n","output_type":"stream"}]},{"cell_type":"markdown","source":"> There are a lot of arguments and parameters that can be passed to the CountVectorizer. In this case we will just specify the **analyzer** to be our own previously defined function:","metadata":{}},{"cell_type":"code","source":"from sklearn.feature_extraction.text import CountVectorizer\n\n# instantiate the vectorizer\nvect = CountVectorizer()\nvect.fit(X_train)\n\n# learn training data vocabulary, then use it to create a document-term matrix\nX_train_dtm = vect.transform(X_train)\n\n# equivalently: combine fit and transform into a single step\nX_train_dtm = vect.fit_transform(X_train)\n\n\n# examine the document-term matrix\nprint(type(X_train_dtm), X_train_dtm.shape)\n\n# transform testing data (using fitted vocabulary) into a document-term matrix\nX_test_dtm = vect.transform(X_test)\nprint(type(X_test_dtm), X_test_dtm.shape)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:32.283212Z","iopub.execute_input":"2023-05-14T15:17:32.283528Z","iopub.status.idle":"2023-05-14T15:17:32.44579Z","shell.execute_reply.started":"2023-05-14T15:17:32.283507Z","shell.execute_reply":"2023-05-14T15:17:32.444737Z"},"trusted":true},"execution_count":37,"outputs":[{"name":"stdout","text":" (4179, 7996)\n (1393, 7996)\n","output_type":"stream"}]},{"cell_type":"code","source":"from sklearn.feature_extraction.text import TfidfTransformer\n\ntfidf_transformer = TfidfTransformer()\ntfidf_transformer.fit(X_train_dtm)\ntfidf_transformer.transform(X_train_dtm)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:36.756487Z","iopub.execute_input":"2023-05-14T15:17:36.756805Z","iopub.status.idle":"2023-05-14T15:17:36.779232Z","shell.execute_reply.started":"2023-05-14T15:17:36.756784Z","shell.execute_reply":"2023-05-14T15:17:36.778341Z"},"trusted":true},"execution_count":38,"outputs":[{"execution_count":38,"output_type":"execute_result","data":{"text/plain":"<4179x7996 sparse matrix of type ''\n\twith 34796 stored elements in Compressed Sparse Row format>"},"metadata":{}}]},{"cell_type":"markdown","source":"# ๐Ÿค– Building and evaluating a model\n\n> We will use [multinomial Naive Bayes](http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html):\n\n> The multinomial Naive Bayes classifier is suitable for classification with **discrete features** (e.g., word counts for text classification). The multinomial distribution normally requires integer feature counts. However, in practice, fractional counts such as tf-idf may also work.","metadata":{}},{"cell_type":"code","source":"# import and instantiate a Multinomial Naive Bayes model\nfrom sklearn.naive_bayes import MultinomialNB\nnb = MultinomialNB()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:39.922716Z","iopub.execute_input":"2023-05-14T15:17:39.923064Z","iopub.status.idle":"2023-05-14T15:17:39.937795Z","shell.execute_reply.started":"2023-05-14T15:17:39.923041Z","shell.execute_reply":"2023-05-14T15:17:39.936539Z"},"trusted":true},"execution_count":39,"outputs":[]},{"cell_type":"code","source":"# train the model using X_train_dtm (timing it with an IPython \"magic command\")\n%time nb.fit(X_train_dtm, y_train)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:41.876091Z","iopub.execute_input":"2023-05-14T15:17:41.876455Z","iopub.status.idle":"2023-05-14T15:17:41.894127Z","shell.execute_reply.started":"2023-05-14T15:17:41.87643Z","shell.execute_reply":"2023-05-14T15:17:41.893239Z"},"trusted":true},"execution_count":40,"outputs":[{"name":"stdout","text":"CPU times: user 6.97 ms, sys: 202 ยตs, total: 7.17 ms\nWall time: 6.14 ms\n","output_type":"stream"},{"execution_count":40,"output_type":"execute_result","data":{"text/plain":"MultinomialNB()","text/html":"
MultinomialNB()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
"},"metadata":{}}]},{"cell_type":"code","source":"from sklearn import metrics\n\n# make class predictions for X_test_dtm\ny_pred_class = nb.predict(X_test_dtm)\n\n# calculate accuracy of class predictions\nprint(\"=======Accuracy Score===========\")\nprint(metrics.accuracy_score(y_test, y_pred_class))\n\n# print the confusion matrix\nprint(\"=======Confision Matrix===========\")\nmetrics.confusion_matrix(y_test, y_pred_class)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:43.570499Z","iopub.execute_input":"2023-05-14T15:17:43.570965Z","iopub.status.idle":"2023-05-14T15:17:43.579965Z","shell.execute_reply.started":"2023-05-14T15:17:43.570942Z","shell.execute_reply":"2023-05-14T15:17:43.579296Z"},"trusted":true},"execution_count":41,"outputs":[{"name":"stdout","text":"=======Accuracy Score===========\n0.9827709978463748\n=======Confision Matrix===========\n","output_type":"stream"},{"execution_count":41,"output_type":"execute_result","data":{"text/plain":"array([[1205, 8],\n [ 16, 164]])"},"metadata":{}}]},{"cell_type":"code","source":"# print message text for false positives (ham incorrectly classifier)\n# X_test[(y_pred_class==1) & (y_test==0)]\nX_test[y_pred_class > y_test]","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:47.403492Z","iopub.execute_input":"2023-05-14T15:17:47.403988Z","iopub.status.idle":"2023-05-14T15:17:47.411061Z","shell.execute_reply.started":"2023-05-14T15:17:47.403965Z","shell.execute_reply":"2023-05-14T15:17:47.410105Z"},"trusted":true},"execution_count":42,"outputs":[{"execution_count":42,"output_type":"execute_result","data":{"text/plain":"2418 Madamregret disturbancemight receive reference...\n4598 laid airtel line rest\n386 Customer place call\n1289 HeyGreat dealFarm tour 9am 5pm 95pax 50 deposi...\n5094 Hi ShanilRakhesh herethanksi exchanged uncut d...\n494 free nowcan call\n759 Call youcarlos isare phones vibrate acting mig...\n3140 Customer place call\nName: clean_msg, dtype: object"},"metadata":{}}]},{"cell_type":"code","source":"# print message text for false negatives (spam incorrectly classifier)\nX_test[y_pred_class < y_test]","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:17:52.372245Z","iopub.execute_input":"2023-05-14T15:17:52.372599Z","iopub.status.idle":"2023-05-14T15:17:52.382998Z","shell.execute_reply.started":"2023-05-14T15:17:52.372577Z","shell.execute_reply":"2023-05-14T15:17:52.38161Z"},"trusted":true},"execution_count":43,"outputs":[{"execution_count":43,"output_type":"execute_result","data":{"text/plain":"4674 Hi babe Chloe r smashed saturday night great w...\n3528 Xmas New Years Eve tickets sale club day 10am ...\n3417 LIFE never much fun great came made truly spec...\n2773 come takes little time child afraid dark becom...\n1960 Guess Somebody know secretly fancies Wanna fin...\n5 FreeMsg Hey darling 3 weeks word back Id like ...\n2078 85233 FREERingtoneReply REAL\n1457 CLAIRE havin borin time alone wanna cum 2nite ...\n190 unique enough Find 30th August wwwareyouunique...\n2429 Guess IThis first time created web page WWWASJ...\n3057 unsubscribed services Get tons sexy babes hunk...\n1021 Guess Somebody know secretly fancies Wanna fin...\n4067 TBSPERSOLVO chasing us since Sept forรฅยฃ38 defi...\n3358 Sorry missed call lets talk time 07090201529\n2821 ROMCAPspam Everyone around responding well pre...\n2247 Back work 2morro half term C 2nite sexy passio...\nName: clean_msg, dtype: object"},"metadata":{}}]},{"cell_type":"code","source":"# example of false negative \nX_test[4949]","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:18:01.075994Z","iopub.execute_input":"2023-05-14T15:18:01.076338Z","iopub.status.idle":"2023-05-14T15:18:01.085104Z","shell.execute_reply.started":"2023-05-14T15:18:01.076314Z","shell.execute_reply":"2023-05-14T15:18:01.083478Z"},"trusted":true},"execution_count":44,"outputs":[{"execution_count":44,"output_type":"execute_result","data":{"text/plain":"'Hi probably much fun get message thought id txt cos bored james farting night'"},"metadata":{}}]},{"cell_type":"code","source":"# calculate predicted probabilities for X_test_dtm (poorly calibrated)\ny_pred_prob = nb.predict_proba(X_test_dtm)[:, 1]\ny_pred_prob","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:18:06.790742Z","iopub.execute_input":"2023-05-14T15:18:06.791083Z","iopub.status.idle":"2023-05-14T15:18:06.799886Z","shell.execute_reply.started":"2023-05-14T15:18:06.791061Z","shell.execute_reply":"2023-05-14T15:18:06.798691Z"},"trusted":true},"execution_count":45,"outputs":[{"execution_count":45,"output_type":"execute_result","data":{"text/plain":"array([2.11903975e-02, 3.97831612e-04, 1.06470895e-03, ...,\n 1.31939653e-02, 9.99821127e-05, 6.04083365e-06])"},"metadata":{}}]},{"cell_type":"code","source":"# calculate AUC\nmetrics.roc_auc_score(y_test, y_pred_prob)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:18:10.548817Z","iopub.execute_input":"2023-05-14T15:18:10.549145Z","iopub.status.idle":"2023-05-14T15:18:10.559368Z","shell.execute_reply.started":"2023-05-14T15:18:10.549122Z","shell.execute_reply":"2023-05-14T15:18:10.558216Z"},"trusted":true},"execution_count":46,"outputs":[{"execution_count":46,"output_type":"execute_result","data":{"text/plain":"0.9774342768159751"},"metadata":{}}]},{"cell_type":"code","source":"from sklearn.feature_extraction.text import TfidfTransformer\nfrom sklearn.pipeline import Pipeline\n\npipe = Pipeline([('bow', CountVectorizer()), \n ('tfid', TfidfTransformer()), \n ('model', MultinomialNB())])\n\npipe.fit(X_train, y_train)\ny_pred = pipe.predict(X_test)\n\n# calculate accuracy of class predictions\nprint(\"=======Accuracy Score===========\")\nprint(metrics.accuracy_score(y_test, y_pred))\n\n# print the confusion matrix\nprint(\"=======Confision Matrix===========\")\nmetrics.confusion_matrix(y_test, y_pred)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:18:11.485621Z","iopub.execute_input":"2023-05-14T15:18:11.485965Z","iopub.status.idle":"2023-05-14T15:18:11.579696Z","shell.execute_reply.started":"2023-05-14T15:18:11.485943Z","shell.execute_reply":"2023-05-14T15:18:11.578789Z"},"trusted":true},"execution_count":47,"outputs":[{"name":"stdout","text":"=======Accuracy Score===========\n0.9669777458722182\n=======Confision Matrix===========\n","output_type":"stream"},{"execution_count":47,"output_type":"execute_result","data":{"text/plain":"array([[1213, 0],\n [ 46, 134]])"},"metadata":{}}]},{"cell_type":"markdown","source":"# ๐Ÿ“Š Comparing models\n\nWe will compare multinomial Naive Bayes with [logistic regression](http://scikit-learn.org/stable/modules/linear_model.html#logistic-regression):\n\n> Logistic regression, despite its name, is a **linear model for classification** rather than regression. Logistic regression is also known in the literature as logit regression, maximum-entropy classification (MaxEnt) or the log-linear classifier. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function.","metadata":{}},{"cell_type":"code","source":"# import an instantiate a logistic regression model\nfrom sklearn.linear_model import LogisticRegression\n\nlogreg = LogisticRegression(solver='liblinear')\n\n# train the model using X_train_dtm\n%time logreg.fit(X_train_dtm, y_train)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.632679Z","iopub.status.idle":"2023-05-14T15:07:02.633183Z","shell.execute_reply.started":"2023-05-14T15:07:02.633036Z","shell.execute_reply":"2023-05-14T15:07:02.633051Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# make class predictions for X_test_dtm\ny_pred_class = logreg.predict(X_test_dtm)\n\n# calculate predicted probabilities for X_test_dtm (well calibrated)\ny_pred_prob = logreg.predict_proba(X_test_dtm)[:, 1]\ny_pred_prob","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.634123Z","iopub.status.idle":"2023-05-14T15:07:02.63465Z","shell.execute_reply.started":"2023-05-14T15:07:02.634502Z","shell.execute_reply":"2023-05-14T15:07:02.634518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# calculate accuracy of class predictions\nprint(\"=======Accuracy Score===========\")\nprint(metrics.accuracy_score(y_test, y_pred_class))\n\n# print the confusion matrix\nprint(\"=======Confision Matrix===========\")\nprint(metrics.confusion_matrix(y_test, y_pred_class))\n\n# calculate AUC\nprint(\"=======ROC AUC Score===========\")\nprint(metrics.roc_auc_score(y_test, y_pred_prob))","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.635574Z","iopub.status.idle":"2023-05-14T15:07:02.636071Z","shell.execute_reply.started":"2023-05-14T15:07:02.635922Z","shell.execute_reply":"2023-05-14T15:07:02.635937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿงฎ Tuning the vectorizer\n\nThus far, we have been using the default parameters of [CountVectorizer:](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html)","metadata":{}},{"cell_type":"code","source":"# show default parameters for CountVectorizer\nvect","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.637008Z","iopub.status.idle":"2023-05-14T15:07:02.637587Z","shell.execute_reply.started":"2023-05-14T15:07:02.637428Z","shell.execute_reply":"2023-05-14T15:07:02.637444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ๐Ÿ“Œ However, the vectorizer is worth tuning, just like a model is worth tuning! Here are a few parameters that you might want to tune:\n\n> - ๐Ÿ“Œ **stop_words**: string {'english'}, list, or None (default)\n - If 'english', a built-in stop word list for English is used.\n - If a list, that list is assumed to contain stop words, all of which will be removed from the resulting tokens.\n - If None, no stop words will be used.","metadata":{}},{"cell_type":"code","source":"# remove English stop words\nvect = CountVectorizer(stop_words='english')","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.638534Z","iopub.status.idle":"2023-05-14T15:07:02.639041Z","shell.execute_reply.started":"2023-05-14T15:07:02.638891Z","shell.execute_reply":"2023-05-14T15:07:02.638907Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> - ๐Ÿ“Œ **ngram_range**: tuple (min_n, max_n), default=(1, 1)\n - The lower and upper boundary of the range of n-values for different n-grams to be extracted.\n - All values of n such that min_n <= n <= max_n will be used.","metadata":{}},{"cell_type":"code","source":"# include 1-grams and 2-grams\nvect = CountVectorizer(ngram_range=(1, 2))","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.640006Z","iopub.status.idle":"2023-05-14T15:07:02.640575Z","shell.execute_reply.started":"2023-05-14T15:07:02.640367Z","shell.execute_reply":"2023-05-14T15:07:02.640383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> - ๐Ÿ“Œ **max_df**: float in range [0.0, 1.0] or int, default=1.0\n - When building the vocabulary, ignore terms that have a document frequency strictly higher than the given threshold (corpus-specific stop words).\n - If float, the parameter represents a proportion of documents.\n - If integer, the parameter represents an absolute count.","metadata":{}},{"cell_type":"code","source":"# ignore terms that appear in more than 50% of the documents\nvect = CountVectorizer(max_df=0.5)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.641553Z","iopub.status.idle":"2023-05-14T15:07:02.642059Z","shell.execute_reply.started":"2023-05-14T15:07:02.641909Z","shell.execute_reply":"2023-05-14T15:07:02.641924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> - ๐Ÿ“Œ **min_df**: float in range [0.0, 1.0] or int, default=1\n - When building the vocabulary, ignore terms that have a document frequency strictly lower than the given threshold. (This value is also called \"cut-off\" in the literature.)\n - If float, the parameter represents a proportion of documents.\n - If integer, the parameter represents an absolute count.","metadata":{}},{"cell_type":"code","source":"# only keep terms that appear in at least 2 documents\nvect = CountVectorizer(min_df=2)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T15:07:02.642993Z","iopub.status.idle":"2023-05-14T15:07:02.643531Z","shell.execute_reply.started":"2023-05-14T15:07:02.643348Z","shell.execute_reply":"2023-05-14T15:07:02.643364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> - ๐Ÿ“Œ **Guidelines for tuning CountVectorizer**:\n - Use your knowledge of the problem and the text, and your understanding of the tuning parameters, to help you decide what parameters to tune and how to tune them.\n - Experiment, and let the data tell you the best approach!","metadata":{}}]}