{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Title: Parkinson's Freezing of Gait Prediction\n\n#### Cameron Presley\n#### start date: 2023-05-17\n#### contact info: cameron@cameron-presley.com\n#### Kaggel Competition - Final Submission Deadline - 2023 June 08","metadata":{"id":"5JrV28POAYVL"}},{"cell_type":"markdown","source":"### references:\n\n* @misc{tlvmc-parkinsons-freezing-gait-prediction,\n author = {Addison Howard, amit salomon, eran gazit, HCL-Jevster, Jeff Hausdorff, Leslie Kirsch, Maggie, Pieter Ginis, Ryan Holbrook, Yasir F Karim},\n title = {Parkinson's Freezing of Gait Prediction},\n publisher = {Kaggle},\n year = {2023},\n url = {https://kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction}\n}\n\n* Goki Fujiya : I leveraged several data preprocessing code snippets from here: https://www.kaggle.com/embed/gokifujiya/pd-fog-prediction-baseline-by-random-","metadata":{"id":"NCFnR9NtCuK6"}},{"cell_type":"markdown","source":"#### Background\n\nThe goal is to detect freezing of gait (FOG), a debilitating symptom that afflicts many people with Parkinson’s disease, by developing machine learning model that has been trained on data collected from a wearable 3D lower back sensor. The value of this model is to help researchers better understand when and why FOG episodes occur. This will improve the ability of medical professionals to optimally evaluate, monitor, and ultimately, prevent FOG events.\n\n#### Context\n\nAn estimated 7 to 10 million people around the world have Parkinson’s disease, many of whom suffer from freezing of gait (FOG). During a FOG episode, a patient's feet are “glued” to the ground, preventing them from moving forward despite their attempts. FOG has a profound negative impact on health-related quality of life—people who suffer from FOG are often depressed, have an increased risk of falling, are likelier to be confined to wheelchair use, and have restricted independence.\n\nWhile researchers have multiple theories to explain when, why, and in whom FOG occurs, there is still no clear understanding of its causes. The ability to objectively and accurately quantify FOG is one of the keys to advancing its understanding and treatment. Collection and analysis of FOG events, such as with your data science skills, could lead to potential treatments.\n\nThere are many methods of evaluating FOG, though most involve FOG-provoking protocols. People with FOG are filmed while performing certain tasks that are likely to increase its occurrence. Experts then review the video to score each frame, indicating when FOG occurred. While scoring in this manner is relatively reliable and sensitive, it is extremely time-consuming and requires specific expertise. Another method involves augmenting FOG-provoking testing with wearable devices. With more sensors, the detection of FOG becomes easier, however, compliance and usability may be reduced. Therefore, a combination of these two methods may be the best approach. When combined with machine learning methods, the accuracy of detecting FOG from a lower back accelerometer is relatively high. However, the datasets used to train and test these algorithms have been relatively small and generalizability is limited to date. Furthermore, the emphasis has been on achieving high levels of accuracy, while precision, for example, has largely been ignored.\n\nCompetition hosts:\n\nCenter for the Study of Movement, Cognition, and Mobility (CMCM) Neurological Institute\nTel Aviv Sourasky Medical Center\n\nTheir collective goal is to improve the personalized treatment of age-related movement, cognition, and mobility disorders and to alleviate the associated burden. They leverage a combination of clinical, engineering, and neuroscience expertise to:\n1) Gain new understandings into the physiologic and pathophysiologic mechanisms that contribute to cognitive and motor function, the factors that influence these functions, and their changes with aging and disease (e.g., Parkinson’s disease, Alzheimer’s)\n2) Develop new methods and tools for the early detection and tracking of cognitive and motor decline. A major focus is on using leveraging wearable devices and digital technologies\n3) Develop and evaluate novel methods for the prevention and treatment of gait, falls, and cognitive function.\n\n### IMPACT\n\nThe work in this competition will help advance the evaluation, understanding and treatment of FOG, improving the lives of the many people who suffer from this debilitating Parkinson’s disease symptom.\n\n","metadata":{"id":"2xvvIxeOAYVO"}},{"cell_type":"markdown","source":"#### Datasets\n\nThe data series include three datasets, collected under distinct circumstances:\n\nThe tDCS FOG (tdcsfog) dataset, comprising data series collected in the lab, as subjects completed a FOG-provoking protocol.\n\nThe DeFOG (defog) dataset, comprising data series collected in the subject's home, as subjects completed a FOG-provoking protocol\n\nThe Daily Living (daily) dataset, comprising one week of continuous 24/7 recordings from sixty-five subjects. Forty-five subjects exhibit FOG symptoms and also have series in the defog dataset, while the other twenty subjects do not exhibit FOG symptoms and do not have series elsewhere in the data.\n\n* Metadata:\n\n#### Approach\n\n1. Detect FOG epsidoes from the tdcsfog and defog datasets.\n2. Use the daily dataset to support detection modeling from tdcsfog and defog datasets - will require use of unsupervised or semi-supervised approaches.\n\n","metadata":{"id":"Fa35Tou0AYVQ"}},{"cell_type":"code","source":"#### Build header & import anticipated libraries for data operations, visualization, evaluation\nauthor = 'Cameron Presley'\nemail = 'cameron@cameron-presley.com'\ntitle = 'KAGGEL: Parkinson''s Freezing of Gait Prediction'\n\nprint ('Title :', title)\nprint ('')\nprint ('Author :', author)\nprint ('')\nprint ('email :', email)\nprint ('')\n\n\n#Import the neccesary libraries and avoid warnings\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\nimport numpy as np\nimport pandas as pd\nimport math\n\n#library needed to read .xlsx files\n#import xlrd\n\n#data visualization librariers\n\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n#statistical analysis libraries\n\n#import scipy.stats as stats\n#import statsmodels.api as sm\n#from statsmodels.stats.proportion import proportions_ztest\n#from statsmodels.formula.api import ols # for ANOVA\n#from statsmodels.stats.anova import anova_lm # for ANOVA\n#from scipy.stats import chi2_contingency # for CHI SQUARE\n#from scipy.stats import ttest_rel #paired T-test\n#from scipy.stats import levene #Levene's test\n\n#data cleansing tools\n#from sklearn.impute import SimpleImputer\n#from sklearn.preprocessing import OneHotEncoder\n#from sklearn.impute import KNNImputer\n#import missingno as mi\n\n#model building libraries and tools\n\nfrom sklearn import metrics\nfrom sklearn import tree\nfrom sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score\nfrom sklearn.metrics import confusion_matrix, classification_report\nfrom sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.model_selection import GridSearchCV, RandomizedSearchCV\n\nfrom sklearn.pipeline import Pipeline, make_pipeline\n\n# --> CLASSIFIER MODELS\n\n#from sklearn.tree import DecisionTreeClassifier\n#from sklearn.ensemble import GradientBoostingClassifier, AdaBoostClassifier, StackingClassifier\n#from sklearn.ensemble import BaggingClassifier\nfrom sklearn.ensemble import RandomForestClassifier\n#from xgboost import XGBClassifier\n#from sklearn.linear_model import LogisticRegression\n\n\n#use seaborn styling\nsns.set_style(\"whitegrid\")\n\n\n\n","metadata":{"id":"J-mO6A0yAYVS","outputId":"35f9f6df-88cf-40a9-9d38-34717178e090","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load Data Sets\n\nFOG Detection Data Sets\n- these datasets consist of numerous .csv files with sensor data associated with subjects under different settings\n- rather than load all fo them now, we will sample data from \"tdscfog\" and \"defog\"","metadata":{"id":"HsHw2SaBAYVU"}},{"cell_type":"markdown","source":"## EDA ","metadata":{"id":"S5_T3BoKAYVU"}},{"cell_type":"markdown","source":"### tdcsfog sample and aggregate files","metadata":{"id":"1Q35Mz2QAYVU"}},{"cell_type":"code","source":"#from google.colab import drive\n#drive.mount('/content/drive')","metadata":{"id":"65FO61-QArew","outputId":"69935eb1-9c90-453e-e4f7-15112d67d3a0","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os \n\nos.getcwd()\nos.chdir('/kaggle/')\n \nos.getcwd() \n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.listdir('/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir('/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# read in data\n\nfilename_1 = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog/003f117e14.csv'\ndata_1 = pd.read_csv (filename_1)\ndf1 = data_1.copy()\ndf1.info()\n","metadata":{"id":"KBbaqlSwAYVV","outputId":"526e823e-b0fc-452a-cdf0-20b96738292f","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Explore 0330ea6680.csv (df1) : initial data review\n","metadata":{"id":"GMAUl6YLAYVV"}},{"cell_type":"code","source":"#open up view of columns and manage the number of rows being viewed\npd.set_option('display.max_columns',None)\npd.set_option('display.max_rows',30)\n\n#set random seed(99) and sample(15)\nnp.random.seed(99)\ndf1.sample(n=15)","metadata":{"id":"UN2LGQL6AYVV","outputId":"dffcbd4b-9fab-490b-f228-71b61144dbfd","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df1.describe().T","metadata":{"id":"R2g_DcRAAYVV","outputId":"3ba9566d-f148-4065-bef1-f962e13c5e8e","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Observations\n* Data set is numeric without any categorical data.\n* Acceleration is measured in m/s^2\n* AccV = lower back sensor acceleration - Vertical\n* AccML = lower back sensor acceleration - Mediolateral\n* AccAP = lower back sensor acceleration - Anteroposterior\n* EVENT TYPES (3):\n - StartHestitation\n - Turn\n - Walking\n* Acceleration is directional (+) and (-)\n* This particular subject has no recorded activity for the 3 event types\n* Time is measured in time steps @ 128Hz (128 time steps per second)\n* 4,533 records, no missing data","metadata":{"id":"Hthq-IBJAYVW"}},{"cell_type":"code","source":"df1.isnull().sum()","metadata":{"id":"l4U_0mCPAYVW","outputId":"0f8c5739-73cc-49de-fe8a-8f7140951d76","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df1.isna().sum()","metadata":{"id":"mIQXzwbLAYVW","outputId":"76efd75a-7beb-4761-8193-490e68a3ade2","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#look at data distribution - quick view\ncolumns = list(df1)[0:] #showing all columns\ndf1[columns].hist(stacked = False, bins=100, figsize =(10,40), layout = (15,2), color ='lime');","metadata":{"id":"-p2-_TibAYVW","outputId":"7c4e4dec-bc95-4ecc-ffc0-68d9f1fea7a1","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Combine individual tdcsfog training files into a single dataframe\n\nThe 3 event types are reporting zero. At this point, we need to integrate the individual files to explore the population of training data for this protocol.","metadata":{"id":"ZzXIBqCZAYVb"}},{"cell_type":"code","source":"import os","metadata":{"id":"UXCNgHZlAYVc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set the directory path to the folder containing the CSV files.\ntdcsfog_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog'\n# Initialize an empty list to store the dataframes.\ntdcsfog_list = []\n\n# Loop through each file in the directory and read it into a dataframe.\nfor file_name in os.listdir(tdcsfog_path):\n if file_name.endswith('.csv'):\n file_path = os.path.join(tdcsfog_path, file_name)\n file = pd.read_csv(file_path)\n # file.Time = file.Time / (len(file) - 1)\n tdcsfog_list.append(file)\n\n# Concatenate the dataframes vertically using pd.concat().\ntdcsfog = pd.concat(tdcsfog_list, axis = 0)\n\n# Show the concatenated dataframe.\ntdcsfog","metadata":{"id":"vIcqiJxZAYVc","outputId":"2bb510fa-bbcf-4869-844a-7c184c4b8758","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog.info()","metadata":{"id":"9w6AgHEbAYVc","outputId":"9ca2ed69-7b97-4b0e-90b5-d045ced10a13","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog.describe().T","metadata":{"id":"GWhfG2D3AYVc","outputId":"aa15b727-1b28-4089-d2df-b36409616148","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Correlation Matrix","metadata":{"id":"jCws-sBFAYVc"}},{"cell_type":"code","source":"sns.set(rc={\"figure.figsize\": (10, 5)})\nsns.heatmap(\n tdcsfog.corr(),\n annot=True,\n linewidths=0.5,\n center=0,\n cbar=False,\n cmap=\"rainbow\",\n fmt=\"0.2f\",\n)\nplt.show()","metadata":{"id":"lwoPueMBAYVc","outputId":"87c198ec-b453-4ced-d2b8-d733d5449c02","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Reduce Memory Usage","metadata":{"id":"RvxDQVQ4AYVd"}},{"cell_type":"code","source":"def reduce_memory_usage(df):\n\n start_mem = df.memory_usage().sum() / 1024 ** 2\n print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n\n for col in df.columns:\n col_type = df[col].dtype.name\n if ((col_type != 'datetime64[ns]') & (col_type != 'category')):\n if (col_type != 'object'):\n c_min = df[col].min()\n c_max = df[col].max()\n\n if str(col_type)[:3] == 'int':\n if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n df[col] = df[col].astype(np.int8)\n elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n df[col] = df[col].astype(np.int16)\n elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n df[col] = df[col].astype(np.int32)\n elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n df[col] = df[col].astype(np.int64)\n\n else:\n if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n df[col] = df[col].astype(np.float16)\n elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n df[col] = df[col].astype(np.float32)\n else:\n pass\n else:\n df[col] = df[col].astype('category')\n mem_usg = df.memory_usage().sum() / 1024 ** 2\n print(\"Memory usage became: \",mem_usg,\" MB\")\n\n return df","metadata":{"id":"l50a-ew6AYVd","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog = reduce_memory_usage(tdcsfog)","metadata":{"id":"db4kKFhlAYVd","outputId":"4f99a2f0-51b5-4422-e9c6-c4a3e49e4732","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print ('StartHesitation Mean =', tdcsfog['StartHesitation'].mean())\nprint ('Turn Mean =', tdcsfog['Turn'].mean())\nprint ('Walking Mean =', tdcsfog['Walking'].mean())","metadata":{"id":"PDrzxehIAYVd","outputId":"9bb6fb1a-13da-42bf-dcff-595667945399","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.pairplot(tdcsfog[['AccV', 'AccML', 'AccAP']], corner = False, palette ='icefire')\nplt.show()","metadata":{"id":"yUJ8rSriAYW_","outputId":"0a91f83c-b7dd-439a-a40e-0f46c88dfc46","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set the figure size.\nplt.figure(figsize = (20,6))\n\n# Plot StartHesitation values over time.\nplt.plot(tdcsfog['Time'], tdcsfog['StartHesitation'], label = 'StartHesitation', color = 'purple')\n\n# Plot the Turn values over time.\nplt.plot(tdcsfog['Time'], tdcsfog['Turn'], label = 'Turn', color = 'lime')\n\n# Plot the Walking values over time.\nplt.plot(tdcsfog['Time'], tdcsfog['Walking'], label = 'Walking', color = 'orange')\n\n# Add a legend to the plot.\nplt.legend()\n\n# Set the x-label of the plot.\nplt.xlabel('Time')\n\n# Set the y-label of the plot.\nplt.ylabel('Value')\n\n# Set the title of the plot.\nplt.title('StartHesitation, Turn, and Walking over Time')\n\n#cmap = plt.get_cmap ('coolwarm')\n#plt.set_cmap (cmap)\n\n# Display the plot.\nplt.show()","metadata":{"id":"TCmdD4EK3A_g","outputId":"8aaad9c2-f3ed-4f56-a5a6-2d1fa43bb0b5","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set the figure size.\nplt.figure(figsize = (10,6))\n\n# Calculates the mean values of all other columns for each unique value of Time.\n# Reset the index of the resulting dataframe.\ntdcsfog_means = tdcsfog.groupby('Time').mean().reset_index()\n\n# Plot the mean StartHesitation values over time.\nplt.plot(tdcsfog_means['Time'], tdcsfog_means['StartHesitation'], label = 'StartHesitation', color = 'red')\n\n# Plot the mean Turn values over time.\nplt.plot(tdcsfog_means['Time'], tdcsfog_means['Turn'], label = 'Turn', color = 'yellow')\n\n# Plot the mean Walking values over time.\nplt.plot(tdcsfog_means['Time'], tdcsfog_means['Walking'], label = 'Walking', color = 'green')\n\n# Add a legend to the plot.\nplt.legend()\n\n# Set the x-label of the plot.\nplt.xlabel('Time')\n\n# Set the y-label of the plot.\nplt.ylabel('Mean Value')\n\n# Set the title of the plot.\nplt.title('Mean Values of StartHesitation, Turn, and Walking over Time')\n\n#cmap = plt.get_cmap ('coolwarm')\n#plt.set_cmap (cmap)\n\n# Display the plot.\nplt.show()","metadata":{"id":"YIqi1u5JAYW_","outputId":"466df785-d923-4653-9032-8725bd54aa6e","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Explore Outliers","metadata":{"id":"iZOj4gVMAYW_"}},{"cell_type":"code","source":"#Cite reference: from Life Expectancy Case Stdy\n# outlier detection aggregated in one spot\nnumeric_columns = tdcsfog.select_dtypes(include=np.number).columns.tolist()\n\nplt.figure(figsize=(10,20))\n\nfor i, variable in enumerate(numeric_columns):\n plt.subplot(5,4,i+1)\n plt.boxplot(tdcsfog[variable],whis=1.5)\n plt.tight_layout()\n plt.title(variable)\n\nplt.show()","metadata":{"id":"LasDtWpkAYXA","outputId":"3b4e5a48-995b-4505-cb39-ee69439bc247","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations\n\n* Outliers\n\n This data set has a high level of outliers as shown. Need to use a ML Model that is \"not\" sensitive to Outliers or we will need to treat the outliers. Will start with former approach.\n\n\n* Imbalanced Predictors\n\n Distribution of predictor values also indicate a high concentration of zeros. We will need to balance Predictor variables (StartHesitation, Turn, and Walking) given this concentration of zeros. Wait until we have a combined dataset.\n\n* Influence of Time\n\n It is dubious whether Time has an influence on Predictor values.","metadata":{"id":"fsROJwGZHA0C"}},{"cell_type":"markdown","source":"### EDA defog training data","metadata":{"id":"-fHke6IDAYXB"}},{"cell_type":"code","source":"# read in data\n\nfilename_2 = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/defog/13a4fe5159.csv'\n\ndata_2= pd.read_csv (filename_2)\ndf2 = data_2.copy()\ndf2.info()","metadata":{"id":"NKoiJnBfAYXB","outputId":"e6906607-ca17-4e64-d6e3-7f6304d531cb","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"n = len(pd.unique(df2['Task']))\n\nprint(\"No.of.unique values :\",\n n)","metadata":{"id":"qLnZGUT8AYXB","outputId":"4a776fcb-f94c-4a12-e0be-3b78f5de7c39","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.unique(df2['Task'])","metadata":{"id":"UIAPhIYAAYXC","outputId":"293020a3-14c7-4c19-e7c2-7c582f941ee7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2.head(5)","metadata":{"id":"0nOtriSmAYXC","outputId":"08692c56-bb58-4571-f7b8-ae33e3fc8893","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2.tail (5)","metadata":{"id":"3A8LjNtTAYXC","outputId":"707a0400-6036-4b6c-ca65-4447b76c2736","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2.isnull().sum()","metadata":{"id":"JKg4BkMhAYXC","outputId":"762c821c-5b3b-4487-bc80-11a04c882a39","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2.isna().sum()","metadata":{"id":"GRACaMwmAYXC","outputId":"154cdda5-0fcd-42c7-b482-6dfc573c6fa3","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2.describe().T","metadata":{"id":"MboVjEYWAYXD","outputId":"b6da56a8-a96b-43f7-bc5e-69cc72e89fe8","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#look at data distribution - quick view\ncolumns = list(df2)[0:] #showing all columns\ndf2[columns].hist(stacked = False, bins=100, figsize =(10,40), layout = (15,2), color ='brown');","metadata":{"id":"Xzw2VTewAYXD","outputId":"eea88a9d-f246-486c-dade-160d7297e394","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set the directory path to the folder containing the CSV files.\ndefog_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/defog'\n\n# Initialize an empty list to store the dataframes.\ndefog_list = []\n\n# Loop through each file in the directory and read it into a dataframe.\nfor file_name in os.listdir(defog_path):\n if file_name.endswith('.csv'):\n file_path = os.path.join(defog_path, file_name)\n file = pd.read_csv(file_path)\n #file.Time = file.Time / (len(file) - 1)\n defog_list.append(file)\n\n# Concatenate the dataframes vertically using pd.concat().\ndefog = pd.concat(defog_list, axis = 0)\n\n# Show the concatenated dataframe.\ndefog\n\nt = defog['Task'].unique()\nv = defog['Valid'].unique()\n\nprint ('Unique Task Values:',t)\nprint ('Unique Valid Values:',v)","metadata":{"id":"PnrXWjT-AYXD","outputId":"fb0d729a-6708-46de-f9d8-bd5b28bb55a6","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog.info()","metadata":{"id":"ktin2XkdAYXD","outputId":"174a9e42-aed3-4536-d4b5-4d589478cca4","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog.describe (include ='bool').T","metadata":{"id":"D0y9QVIo-KrJ","outputId":"c5a532d4-992b-42b4-d1fd-3975afab861c","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### observations on Valid and Task\n\n False values represent about 70% of the data. We will be dropping this amount of data from defog","metadata":{"id":"KmbFWtO6-i1a"}},{"cell_type":"code","source":"defog = reduce_memory_usage(defog)","metadata":{"id":"InjNn4QYYgE4","outputId":"c28db7ac-64c9-4d4b-b096-be294d3c7825","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# only consider records that are valid & annotated tasks\ndefog = defog[(defog['Task'] == 1) & (defog['Valid'] == 1)]\n","metadata":{"id":"-5DjpPSUAYXE","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog.info()","metadata":{"id":"HUfbOSpnbbAk","outputId":"d1340450-5003-4cc3-d90e-a4c48123b2af","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"drop_cols = ['Valid','Task']\ndefog.drop(drop_cols, axis =1, inplace= True)\n\ndefog.info()","metadata":{"id":"zBUrvw5kc90W","outputId":"f491fdf9-5456-4515-cf26-b7e4561798db","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print ('The unique values of StartHestitation are:', defog['StartHesitation'].unique())\nprint ('The unique values of Turn are:', defog['Turn'].unique())\nprint ('The unique values of Walking are:', defog['Walking'].unique())","metadata":{"id":"rpksqY-RdE__","outputId":"05c87911-fbaa-46f0-d56b-8e8f0580adbe","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#look at data distribution - quick view\ncolumns = list(defog)[0:] #showing all columns\ndefog[columns].hist(stacked = False, bins=100, figsize =(10,40), layout = (15,2), color ='brown');","metadata":{"id":"G5Q9U1UyeYs_","outputId":"a219c3fb-bc3e-48ad-b83a-94f3cf2f9eb4","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog.info()","metadata":{"id":"fxyrqftIe7Gi","outputId":"7bc34d15-9cbd-472e-f9a1-67f0ae3bba22","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog.info()","metadata":{"id":"7bVMgKTefOlV","outputId":"abaca5da-3331-4f80-fad6-9ec30b47c559","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Build a combined training data set for model building\n\ncombined = pd.concat([tdcsfog, defog], axis = 0)\ncombined.info()","metadata":{"id":"zsRDNjrFfRs_","outputId":"17e23a7e-6b03-4c4c-9806-47fe0ada8d88","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### row count totals check out after tdcsfog and defog are combined\n","metadata":{"id":"f8YZIL_aRoIG"}},{"cell_type":"markdown","source":"### Explore Observed and Not Observed Balance for Predictor Variables (ZEROs and ONEs) balancing","metadata":{"id":"eoPUwYjm_kjQ"}},{"cell_type":"code","source":"print(\"StartHesitation Events Observed - Training : {0} ({1:0.2f}%)\".format(len(combined.loc[combined['StartHesitation'] == 1]), (len(combined.loc[combined['StartHesitation'] == 1])/len(combined.index)) * 100))\nprint(\"StartHesitation Events Not-Observed - Training: {0} ({1:0.2f}%)\".format(len(combined.loc[combined['StartHesitation'] == 0]), (len(combined.loc[combined['StartHesitation'] == 0])/len(combined.index)) * 100))\nprint(\"\")\n\nprint(\"Turn Events Observed - Training : {0} ({1:0.2f}%)\".format(len(combined.loc[combined['Turn'] == 1]), (len(combined.loc[combined['Turn'] == 1])/len(combined.index)) * 100))\nprint(\"Turn Events Not-Observed - Training: {0} ({1:0.2f}%)\".format(len(combined.loc[combined['Turn'] == 0]), (len(combined.loc[combined['Turn'] == 0])/len(combined.index)) * 100))\nprint(\"\")\n\nprint(\"Walking Events Observed - Training : {0} ({1:0.2f}%)\".format(len(combined.loc[combined['Walking'] == 1]), (len(combined.loc[combined['Walking'] == 1])/len(combined.index)) * 100))\nprint(\"Walking Events Not-Observed - Training: {0} ({1:0.2f}%)\".format(len(combined.loc[combined['Walking'] == 0]), (len(combined.loc[combined['Walking'] == 0])/len(combined.index)) * 100))\nprint(\"\")\n","metadata":{"id":"_HiQAytQ_hAR","outputId":"47493443-bf60-4f9a-9b92-06456e4a894e","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Severely imbalanced data set. ","metadata":{"id":"yMT7W3YcR7A5"}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\npredictors = ['StartHesitation','Turn','Walking']\nX = combined.drop(predictors,axis=1) # Drop target variables, leaving feature columns\ny1 = combined ['StartHesitation'] # Target Variable\ny2 = combined ['Turn'] # Target Variable\ny3 = combined ['Walking'] # Target Variable\n\nX1_train, X1_test, y1_train, y1_test = train_test_split(X, y1, test_size=0.3, random_state=99)\nX2_train, X2_test, y2_train, y2_test = train_test_split(X, y2, test_size=0.3, random_state=1)\nX3_train, X3_test, y3_train, y3_test = train_test_split(X, y3, test_size=0.3, random_state=50)\n\n\n# using 99 as the random seed\n\nprint('Independent variables for Start Hesitation Predictor - train and test split:', X1_train.shape, X1_test.shape)\nprint('Independent variables for Walking Predictor - train and test split:', X2_train.shape, X2_test.shape)\nprint('Independent variables for Turn Predictor - train and test split:', X2_train.shape, X2_test.shape)\nprint ('*'*120)\n\nprint('Start Hesitation predictor - train and test split:',y1_train.shape, y1_test.shape)\nprint('Turn predictor - train and test split:', y2_train.shape, y2_test.shape)\nprint('Walking predictor - train and test split:',y3_train.shape, y3_test.shape)\n\n","metadata":{"id":"06UDO-jtfa25","outputId":"78965a0f-26a2-4e3e-b2b3-d065dced9952","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"{0:0.2f}% data is in training set - Features\".format((len(X1_train)/len(combined.index)) * 100))\nprint(\"{0:0.2f}% data is in test set - Features\".format((len(X1_test)/len(combined.index)) * 100))","metadata":{"id":"zt0fCdy-lFQI","outputId":"f4c37506-3947-4326-ef3d-8d54fedabe42","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"{0:0.2f}% data is in training set - Features\".format((len(X2_train)/len(combined.index)) * 100))\nprint(\"{0:0.2f}% data is in test set - Features\".format((len(X2_test)/len(combined.index)) * 100))","metadata":{"id":"bnCKNyVw9PvP","outputId":"0901be0a-0df9-4352-bbea-9ffc240e65db","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"{0:0.2f}% data is in training set - Features\".format((len(X3_train)/len(combined.index)) * 100))\nprint(\"{0:0.2f}% data is in test set - Features\".format((len(X3_test)/len(combined.index)) * 100))","metadata":{"id":"zKvXT8Qc9QBK","outputId":"e4b0f7b4-437b-41e6-e49a-012b4ea10616","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"{0:0.2f}% data is in training set - y1\".format((len(y1_train)/len(combined.index)) * 100))\nprint(\"{0:0.2f}% data is in test set - y1\".format((len(y1_test)/len(combined.index)) * 100))","metadata":{"id":"cyH9VnGVlQun","outputId":"8dc490eb-4a92-41e3-fe55-7ff673776a5c","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"{0:0.2f}% data is in training set - y2\".format((len(y2_train)/len(combined.index)) * 100))\nprint(\"{0:0.2f}% data is in test set - y2\".format((len(y2_test)/len(combined.index)) * 100))","metadata":{"id":"PaL6lz2hlrMy","outputId":"33a85c36-3f0e-43e0-e257-85fc9e578899","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"{0:0.2f}% data is in training set - y3\".format((len(y3_train)/len(combined.index)) * 100))\nprint(\"{0:0.2f}% data is in test set - y3\".format((len(y3_test)/len(combined.index)) * 100))","metadata":{"id":"0MBA-HlXlvki","outputId":"329df86f-a456-4c32-933a-77cf94a3afc8","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Address True-False Balancing in Predictor Variables (Zeros and Ones)","metadata":{"id":"KQ1xKO4zjDO5"}},{"cell_type":"markdown","source":"### Predictor variables are severely imbalanced with 0's, need to use under/over sampling techniques using SMOTE & manual balancing approaches\n\n","metadata":{"id":"F9B0voHIki2w"}},{"cell_type":"code","source":"from imblearn.over_sampling import SMOTE\n","metadata":{"id":"ykzlTJqfky4H","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### oversampling: create oversampled training sets","metadata":{"id":"IKla1FnZk9cB"}},{"cell_type":"markdown","source":"#### Started Hesitation Predictor : Up Sampling","metadata":{"id":"UuZj7d5Om_Cv"}},{"cell_type":"code","source":"print(\"Before UpSampling, counts of POSITIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train==1)))\nprint(\"Before UpSampling, counts of NEGATIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train==0)))\nprint('*'*120)\n\nsm = SMOTE(sampling_strategy = 1 ,k_neighbors = 5, random_state=99) #Synthetic Minority Over Sampling Technique\nX1_train_over, y1_train_over = sm.fit_resample(X1_train, y1_train)\n\nprint(\"After UpSampling, counts of POSITIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train_over==1)))\nprint(\"After UpSampling, counts of NEGATIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train_over==0)))\nprint('*'*120)\n\nprint('After UpSampling, the shape of train_X: {}'.format(X1_train_over.shape))\nprint('After UpSampling, the shape of train_y: {} \\n'.format(y1_train_over.shape))\nprint('*'*120)","metadata":{"id":"ufkshcpblJ-s","outputId":"f0476a7c-0a42-4505-abaa-7bfdedc5ea18","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Turn Predictor : Up Sampling","metadata":{"id":"HTNuQlJfnKRQ"}},{"cell_type":"code","source":"print(\"Before UpSampling, counts of POSITIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train==1)))\nprint(\"Before UpSampling, counts of NEGATIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train==0)))\nprint('*'*120)\n\nsm = SMOTE(sampling_strategy = 1 ,k_neighbors = 5, random_state=1) #Synthetic Minority Over Sampling Technique\nX2_train_over, y2_train_over = sm.fit_resample(X2_train, y2_train)\n\nprint(\"After UpSampling, counts of POSITIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train_over==1)))\nprint(\"After UpSampling, counts of NEGATIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train_over==0)))\nprint('*'*120)\n\nprint('After UpSampling, the shape of train_X: {}'.format(X2_train_over.shape))\nprint('After UpSampling, the shape of train_y: {} \\n'.format(y2_train_over.shape))\nprint('*'*120)","metadata":{"id":"Cnq6OWtvnVPh","outputId":"c43c6827-817f-40dd-eae8-b7697156244b","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Walking Predictor: Up Sampling","metadata":{"id":"IS2URK5WuicP"}},{"cell_type":"code","source":"print(\"Before UpSampling, counts of POSITIVE FOG event observations of FOG events at Walking: {}\".format(sum(y3_train==1)))\nprint(\"Before UpSampling, counts of NEGATIVE FOG event observations of FOG events at Walking: {}\".format(sum(y3_train==0)))\nprint('*'*120)\n\nsm = SMOTE(sampling_strategy = 1 ,k_neighbors = 5, random_state=50) #Synthetic Minority Over Sampling Technique\nX3_train_over, y3_train_over = sm.fit_resample(X3_train, y3_train)\n\nprint(\"After UpSampling, counts of POSITIVE FOG event observations of FOG events at Turn: {}\".format(sum(y3_train_over==1)))\nprint(\"After UpSampling, counts of NEGATIVE FOG event observations of FOG events at Turn: {}\".format(sum(y3_train_over==0)))\nprint('*'*120)\n\nprint('After UpSampling, the shape of train_X: {}'.format(X3_train_over.shape))\nprint('After UpSampling, the shape of train_y: {} \\n'.format(y3_train_over.shape))\nprint('*'*120)","metadata":{"id":"Pons-GmpuncF","outputId":"f383cf40-247b-4639-b5df-9f991c2b43d1","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Under Sampling: Create Under Sampling Data Sets","metadata":{"id":"RPt-dQSevwHU"}},{"cell_type":"code","source":"from imblearn.under_sampling import RandomUnderSampler\nrus = RandomUnderSampler(random_state = 99)","metadata":{"id":"i70fm9mHv84f","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Started Hesitation Predictor : Under Sampling","metadata":{"id":"l2LFHSfN4C7q"}},{"cell_type":"code","source":"print(\"Before Under Sampling, counts of POSITIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train==1)))\nprint(\"Before Under Sampling, counts of NEGATIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train==0)))\n\nprint ('*'*120)\n\nX1_train_under, y1_train_under = rus.fit_resample(X1_train, y1_train)\n\nprint(\"Before Under Sampling, counts of POSITIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train_under==1)))\nprint(\"Before Under Sampling, counts of NEGATIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train_under==0)))\nprint ('*'*120)\n\nprint('After Under Sampling, the shape of train_X: {}'.format(X1_train_under.shape))\nprint('After Under Sampling, the shape of train_y: {} \\n'.format(y1_train_under.shape))\nprint ('*'*120)\n\n\n\n","metadata":{"id":"eXKoXNb3wmvL","outputId":"96b06848-5aef-4099-9e2b-e2fb12b0cf63","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Turn Predictor: Under Sampling","metadata":{"id":"ttNwfZw74X2a"}},{"cell_type":"code","source":"print(\"Before Under Sampling, counts of POSITIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train==1)))\nprint(\"Before Under Sampling, counts of NEGATIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train==0)))\n\nprint ('*'*120)\n\nX2_train_under, y2_train_under = rus.fit_resample(X2_train, y2_train)\n\nprint(\"Before Under Sampling, counts of POSITIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train_under==1)))\nprint(\"Before Under Sampling, counts of NEGATIVE FOG event observations of FOG events at Turn: {}\".format(sum(y2_train_under==0)))\nprint ('*'*120)\n\nprint('After Under Sampling, the shape of train_X: {}'.format(X2_train_under.shape))\nprint('After Under Sampling, the shape of train_y: {} \\n'.format(y2_train_under.shape))\nprint ('*'*120)","metadata":{"id":"B7DHgSUi4caE","outputId":"2836767a-98bc-4e1c-d78b-d4f5b13d2233","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Walking Predictor: Under Sampling","metadata":{"id":"_ZOz9WPF4zcP"}},{"cell_type":"code","source":"print(\"Before Under Sampling, counts of POSITIVE FOG event observations of FOG events at Walking: {}\".format(sum(y3_train==1)))\nprint(\"Before Under Sampling, counts of NEGATIVE FOG event observations of FOG events at Walking: {}\".format(sum(y3_train==0)))\n\nprint ('*'*120)\n\nX3_train_under, y3_train_under = rus.fit_resample(X3_train, y3_train)\n\nprint(\"Before Under Sampling, counts of POSITIVE FOG event observations of FOG events at Walking: {}\".format(sum(y3_train_under==1)))\nprint(\"Before Under Sampling, counts of NEGATIVE FOG event observations of FOG events at Walking: {}\".format(sum(y3_train_under==0)))\nprint ('*'*120)\n\nprint('After Under Sampling, the shape of train_X: {}'.format(X3_train_under.shape))\nprint('After Under Sampling, the shape of train_y: {} \\n'.format(y3_train_under.shape))\nprint ('*'*120)","metadata":{"id":"1sI9uDhd434O","outputId":"205d6e62-44b8-4ced-9e47-17fe709b22da","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Manually build a balanced data set for consideration as well","metadata":{"id":"50-S3u_fV3xy"}},{"cell_type":"code","source":"# Find the positions of y1 where it equals 0.\ny1_zeros = np.where(y1 == 0)[0]\ny1_ones = np.where(y1 == 1)[0]\n\n# Choose the same number of samples with y1 == 1 as there are with y1 == 0.\nnum1_ones = (y1 == 1).sum()\nnp.random.seed(42)\ny1_zeros = np.random.choice(np.where(y1 == 0)[0], size = num1_ones, replace = False)\n\n# Combine the positions of y1 == 0 and y1 == 1.\ny1_balanced_idxs = np.sort(np.concatenate([y1_zeros, y1_ones]))\n\n# Use the balanced indices to get the corresponding rows of X and y1.\nX1_balanced = X.iloc[y1_balanced_idxs, :]\ny1_balanced = y1.iloc[y1_balanced_idxs]","metadata":{"id":"6Kph3nJ1V2P8","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Find the positions of y2 where it equals 0.\ny2_zeros = np.where(y2 == 0)[0]\ny2_ones = np.where(y2 == 1)[0]\n\n# Choose the same number of samples with y2 == 1 as there are with y2 == 0.\nnum2_ones = (y2 == 1).sum()\nnp.random.seed(42)\ny2_zeros = np.random.choice(np.where(y2 == 0)[0], size = num2_ones, replace = False)\n\n# Combine the positions of y2 == 0 and y2 == 1.\ny2_balanced_idxs = np.sort(np.concatenate([y2_zeros, y2_ones]))\n\n# Use the balanced indices to get the corresponding rows of X and y1.\nX2_balanced = X.iloc[y2_balanced_idxs, :]\ny2_balanced = y2.iloc[y2_balanced_idxs]","metadata":{"id":"-tywieAzXBxW","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Find the positions of y3 where it equals 0.\ny3_zeros = np.where(y3 == 0)[0]\ny3_ones = np.where(y3 == 1)[0]\n\n# Choose the same number of samples with y3 == 1 as there are with y3 == 0.\nnum3_ones = (y3 == 1).sum()\nnp.random.seed(42)\ny3_zeros = np.random.choice(np.where(y3 == 0)[0], size = num3_ones, replace = False)\n\n# Combine the positions of y3 == 0 and y3 == 1.\ny3_balanced_idxs = np.sort(np.concatenate([y3_zeros, y3_ones]))\n\n# Use the balanced indices to get the corresponding rows of X and y3.\nX3_balanced = X.iloc[y3_balanced_idxs, :]\ny3_balanced = y3.iloc[y3_balanced_idxs]","metadata":{"id":"aV5LNihSXE5W","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X1_train_b, X1_test_b, y1_train_b, y1_test_b = train_test_split(X1_balanced, y1_balanced, test_size = 0.3, random_state = 99)\nX2_train_b, X2_test_b, y2_train_b, y2_test_b = train_test_split(X2_balanced, y2_balanced, test_size = 0.3, random_state = 99)\nX3_train_b, X3_test_b, y3_train_b, y3_test_b = train_test_split(X3_balanced, y3_balanced, test_size = 0.3, random_state = 99)","metadata":{"id":"8uKuGDePXShN","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Before Balancing, counts of POSITIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train==1)))\nprint(\"Before Balancing, counts of NEGATIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train==0)))\nprint('*'*120)\n\nprint(\"After Balancing, counts of POSITIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train_b==1)))\nprint(\"After Balancing, counts of NEGATIVE FOG event observations of FOG events at Started Hesitation: {}\".format(sum(y1_train_b==0)))\nprint('*'*120)\n\nprint('After Balancing, the shape of train_X: {}'.format(X1_train_b.shape))\nprint('After Balancing, the shape of train_y: {} \\n'.format(y1_train_b.shape))\nprint('*'*120)","metadata":{"id":"TOVERS1DXqxn","outputId":"b21ec36c-7efc-40da-df46-1e09713376c7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model evaluation criteria\n\n1 = \"Yes\" or Freezing of Gait (FOG) event observed\n0 = \"No\" or Freezing of Gait (FOG) event \"not\" observed\n\n\n* True Positive (TP) is: \n * Predict that a subject will experience a FOG event and a FOG event does occur. \n * Predict = Y, Actual = Y\n\n* False Positive (FP) is:\n * Predict that a subject will experience a FOG event and a FOG event does \"not\" occur.\n * Predict = Y, Actual = N\n\n* True Negative (TN) is:\n * Predict a subject will \"not\" experience a FOG event and they do \"not\" experience a FOG event.\n * Predict = N, Actual = N\n\n* False Negative (FN) is: \n * Predict that a subject will \"not\" experience a FOG event, however they \"do\" experience a FOG event. \n * Predict = N, Actual = Y\n\n\nThe model will be evaluated on Mean Average Precision, or take the average of the precision for each predictor, and then the average of those precision calculations for an overall score.\n\nGiven that we want to maximize Precision, we need to minimize False Positives (FPs).\n\n\n\n![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAvwAAACGCAIAAAAuHMlWAAAKN2lDQ1BJQ0MgUHJvZmlsZQAASImVlgdUU9kWhs+96Y2W0FuoofcSQHpv0quohARCjRCqiIrI4AiMKCIioAzoUBUclSI2RBQLg4ACimVABgFlHCxgQeVdZMYpb7233tt37bu/7Nz1n31y7lr5ASCdYSUmxsNCACTwUvi+znb04JBQOm4KwEAAkIEqQLHYyYm23t4eAIk/6t9jcQRAK/WO9orWv3//X0OEE5nMBgAKQziJk8xOQHgAYRd2Ij8FAFgCYaX0lMQVXoMwjY8MiLD3CnNXmbPCEauc8eUZf197hAsBwJNZLD4XAGIZ0qensbmIDrEDYT0eJ4aH8AzCVuxoFqJBUkNYKyFh0wr7I6yGPJ+IcBbCzIi/aHL/ph/xVZ/F4n7l1X19CXEHRw8PureBkb6ZC92eFR8TwWelRHL+z1/qf4iE+NQ/1lw5EXIkL8BvZU9IygAH4Ag8kIsOvIEBMAL6wAy4IJ/sAQvEgxgQAfgIpYBIwEmJzEhZEbHflLiZH8ONTqHbIicbSXflsXW06AZ6BvoArLwnq0u9vvdlNUgM/2ePMwWA4cpZqv3Zi0XesnOLAAi3/tljTAAgdBiALm92Kj9ttYdeuWEAEQgCGpAEckAJqAFtZGYTYAFskF24AS/gD0LABsAG0SABmTsdZIEdIA8UgL3gACgHVeAoqAcnwCnQDs6Dy+AauAUGwDB4AMbBFHgO5sEiWIIgCAdRICokCclDKpAmZAAxISvIEfKAfKEQKBziQjwoFcqCdkIFUDFUDlVDDdCP0FnoMnQDGoTuQxPQLPQK+gCjYDJMg2VhVVgXZsK2sDvsD6+HuXASnAnnwnvgMrgGPg63wZfhW/AwPA4/hxdQAEVCiaEUUNooJsoe5YUKRUWh+KhtqHxUKaoG1YzqRPWi7qDGUXOo92gsmoqmo7XRFmgXdACajU5Cb0MXosvR9eg2dA/6DnoCPY/+jKFgZDCaGHOMKyYYw8WkY/IwpZhaTCvmKmYYM4VZxGKxYlgG1hTrgg3BxmK3YAuxh7Et2C7sIHYSu4DD4SRxmjhLnBeOhUvB5eEO4Y7jLuGGcFO4d3gSXh5vgHfCh+J5+Bx8Kb4RfxE/hJ/GLxGECCoEc4IXgUPYTCgiHCN0Em4TpghLRGEig2hJ9CfGEncQy4jNxKvEh8TXJBJJkWRG8iHFkLJJZaSTpOukCdJ7sghZg2xPDiOnkveQ68hd5Pvk1xQKRZViQwmlpFD2UBooVyiPKe8EqAI6Aq4CHIHtAhUCbQJDAi8ECYIqgraCGwQzBUsFTwveFpwTIgipCtkLsYS2CVUInRUaFVoQpgrrC3sJJwgXCjcK3xCeEcGJqIo4inBEckWOilwRmaSiqEpUeyqbupN6jHqVOkXD0hg0V1osrYB2gtZPmxcVETUSDRTNEK0QvSA6LoYSUxVzFYsXKxI7JTYi9kFcVtxWPFJ8t3iz+JD4WwlpCRuJSIl8iRaJYYkPknRJR8k4yX2S7ZKPpNBSGlI+UulSR6SuSs1J06QtpNnS+dKnpMdkYBkNGV+ZLTJHZfpkFmTlZJ1lE2UPyV6RnZMTk7ORi5UrkbsoNytPlbeSj5Evkb8k/4wuSrelx9PL6D30eQUZBReFVIVqhX6FJUWGYoBijmKL4iMlohJTKUqpRKlbaV5ZXtlTOUu5SXlMhaDCVIlWOajSq/JWlaEapLpLtV11hiHBcGVkMpoYD9UoatZqSWo1anfVsepM9Tj1w+oDGrCGsUa0RoXGbU1Y00QzRvOw5qAWRstMi6dVozWqTda21U7TbtKe0BHT8dDJ0WnXeaGrrBuqu0+3V/eznrFevN4xvQf6Ivpu+jn6nfqvDDQM2AYVBncNKYZOhtsNOwxfGmkaRRodMbpnTDX2NN5l3G38ycTUhG/SbDJrqmwablppOsqkMb2ZhczrZhgzO7PtZufN3pubmKeYnzL/zULbIs6i0WJmDWNN5JpjayYtFS1ZltWW41Z0q3Cr763GrRWsWdY11k9slGw4NrU207bqtrG2x21f2OnZ8e1a7d7am9tvte9yQDk4O+Q79DuKOAY4ljs+dlJ04jo1Oc07Gztvce5ywbi4u+xzGXWVdWW7NrjOu5m6bXXrcSe7+7mXuz/x0PDge3R6wp5unvs9H65VWctb2+4FvFy99ns98mZ4J3mf88H6ePtU+Dz11ffN8u31o/pt9Gv0W/S38y/yfxCgFpAa0B0oGBgW2BD4NsghqDhoPFg3eGvwrRCpkJiQjlBcaGBobejCOsd1B9ZNhRmH5YWNrGesz1h/Y4PUhvgNFzYKbmRtPB2OCQ8Kbwz/yPJi1bAWIlwjKiPm2fbsg+znHBtOCWc20jKyOHI6yjKqOGqGa8ndz52Nto4ujZ6LsY8pj3kZ6xJbFfs2ziuuLm45Pii+JQGfEJ5wlifCi+P1bJLblLFpMFEzMS9xPMk86UDSPN+dX5sMJa9P7kihIX/Ifalqqd+kTqRZpVWkvUsPTD+dIZzBy+jbrLF59+bpTKfMH7agt7C3dGcpZO3Imthqu7V6G7QtYlv3dqXtudunsp2z63cQd8Tt+ClHL6c4583OoJ2dubK52bmT3zh/05QnkMfPG91lsavqW/S3Md/27zbcfWj353xO/s0CvYLSgo+F7MKb3+l/V/bd8p6oPf1FJkVH9mL38vaO7LPeV18sXJxZPLnfc39bCb0kv+TNgY0HbpQalVYdJB5MPThe5lHWcUj50N5DH8ujy4cr7CpaKmUqd1e+Pcw5PHTE5khzlWxVQdWH72O+v1ftXN1Wo1pTehR7NO3o02OBx3p/YP7QUCtVW1D7qY5XN17vW9/TYNrQ0CjTWNQEN6U2zR4POz5wwuFER7N2c3WLWEvBSXAy9eSzH8N/HDnlfqr7NPN08xmVM5Wt1Nb8Nqhtc9t8e3T7eEdIx+BZt7PdnRadred0ztWdVzhfcUH0QtFF4sXci8uXMi8tdCV2zV3mXp7s3tj94Erwlbs9Pj39V92vXr/mdO1Kr23vpeuW18/fML9x9ibzZvstk1ttfcZ9rT8Z/9Tab9Lfdtv0dseA2UDn4JrBi0PWQ5fvONy5dtf17q3htcODIwEj90bDRsfvce7N3I+//3IsbWzpQfZDzMP8R0KPSh/LPK75Wf3nlnGT8QsTDhN9T/yePJhkTz7/JfmXj1O5TylPS6flpxtmDGbOzzrNDjxb92zqeeLzpbm8X4V/rXyh9uLMbza/9c0Hz0+95L9cflX4WvJ13RujN90L3guPFxMWl97mv5N8V/+e+b73Q9CH6aX0j7iPZZ/UP3V+dv/8cDlheTmRxWd9sQIoJOGoKABe1QFACQGAinhD4rpVH/e7z4H+4nj+A696vS9hAgAiBQKR9LUBoBJJBpKC2QB4I9XfBsCGhl/z90iOMjRY1SK1I9akdHn5dRAAOHUAPo0uLy+1Ly9/qkWGHUN8zOKqf1wJoeMADIzpGxn7jY02Z//Tq616y7/s8Z8VfJ3gb/VfaYfLxDIfUDIAAAA4ZVhJZk1NACoAAAAIAAGHaQAEAAAAAQAAABoAAAAAAAKgAgAEAAAAAQAAAvygAwAEAAAAAQAAAIYAAAAAEH4eTAAAQABJREFUeAHtXQd8zdcX/5tVtLVX7VGrNjFjix2iSNQeIcvelKI1asSKVXvPEjsRxMoQkpgttfcsNaoU7f/7ct47ufm9l/deEiSSk4++nt+9597f/X1/6/zOTPbff//9T/4EAUFAEBAEBAFBQBBI7AgkT+wHKMcnCAgCgoAgIAgIAoKADgEReuQ6EAQEAUFAEBAEBIEkgYAIPUniNMtBCgKCgCAgCAgCgoAIPXINCAKCgCAgCAgCgkCSQECEniRxmuUgBQFBQBAQBAQBQUCEHrkGBAFBQBAQBAQBQSBJICBCT5I4zXKQgoAgIAgIAoKAICBCj1wDgoAgIAgIAoKAIJAkEBChJ0mcZjlIQUAQEAQEAUFAEBChR64BQUAQEAQEAUFAEEgSCIjQkyROsxykICAICAKCgCAgCIjQI9eAICAICAKCgCAgCCQJBEToSRKnWQ5SEBAEBAFBQBAQBETokWtAEBAEBAFBQBAQBJIEAiL0JInTLAcpCAgCgoAgIAgIAiL0yDUgCAgCgoAgIAgIAkkCARF6ksRploMUBAQBQUAQEAQEARF65BoQBAQBQUAQEAQEgSSBgAg9SeI0y0EKAoKAICAICAKCgAg9cg0IAoKAICAICAKCQJJAQISeJHGa5SAFAUFAEBAEBAFBQIQeuQYEAUFAEBAEBAFBIEkgIEJPkjjNcpCCgCAgCAgCgoAgIEKPXAOCgCAgCAgCgoAgkCQQEKEnSZxmOUhBQBAQBAQBQUAQEKFHrgFBQBAQBAQBQUAQSBIIpEwSRykHKQgIAoLA+0TgxfPn9x7cf/Pmzb///vvff//9D//+9z/8lwz/kuv+kiVPljJ5yhQpU6RO/Un27Nnf51pkbkFAEIgWARF6ooVGOgQBQSAWCPz++3nHtq1iMZCGfJom7YCBg1q3cYz1DPEycOq0yVu2/GLlrgsXLuLUrn2TJs0+/fRTK4e8J7bXr1/779+XKVPGChVtkiWDhBb7v6tXrpw4EVa7Tt0MGTLGfhYZKQi8ZwTEvPWeAZbpBYEkhsCmjevjcsR/v3zxSepP4jJDvIzNmCFjsaLFCuQvoNk7WiDiFC3yFYgvc31JvRcvXvjxhzH16tTcsnmThv9DbkI71aRR/aFDBzo7d9u3d090u378+PG1a1ej66X2FcuXOjg0Gzt2dLcuHc1zSq8gEL8IiKYnfvGXvQsCiQqBt2/f7vH1pUPKmyevq3vvUiVLpfssfeqUqd7+91+Hb9tev3Gdeqd7zqpavQbsPq9evcRr9URY2KxZ0x/+8RC9efPnJ56P6Ld33/74hwWPGzuaVT5Tpk6vX99OPYo//3y8bav3ksWLnjz9E+LduHHfZ82WvUYNW5Xng9G+e3wIcOzx2rVrJvfr67t72NBB6Bo3dnzzFi1N8qBx5Ypl1HX37r1//vknderU0XFKuyAQvwiI0BO/+MveBYFEhcDx0BC8znFI/fsP6tCxM2QaPrwXL16wxAMbVs3adag3VapU6dN/lidPXnCO/n4kfgsUKMijPjri9OlTvObKlasyTQRMP506dwU9ffpUapk3d3Z8CT3wMOLltfqmDdMqsWzJItq8c/eO2q6hU6VMRS2OTk4i8WjAkc0EhUDkRZ+gliWLEQQEgY8RgX1+flh2jeq29GpXDwEOH7xZrVo1VR6i9qLFi4P44vMMn3/+OXN+XMSzZ89gutIfTpGvPvvsM5Prb9ioCQs9v/56FuqfePGDadi4yYuXL54+efpN67YZM5pwxHny5M9z58/RIRQrXszksVCj58zZu3buqFylSrVqNcywSZcgEO8IiNAT76dAFiAIJBIEELhEtq0+/QYYH1LosWPcWMnGhmkmYBoD/VXRr7jloyNOngjnNVeqXIVpDfHixV9qy6tX/6ibH4yGSsbR8VszuzsWcpR7y5WryLQxUQwyUTGdzCp/gkACRyBS+ZzAFyrLEwQEgQSOwIkT4bBtfV2yVJEiJgSX4KNBvP6KFUwIPTdv3ABDoUKFme2jI44fD+E1mxTsqPfiBb02iDYTrGYrNPQ4rRCO2NFprfh4hRAEPgoEROj5KE6TLFIQ+AgQ8N+/F6ts29bJeK1w6IEdh9phwCpY2IRkc/PmRy/0HDsWKfSYUY34++9jiGARivfAdV6Mhjh+XK+cq1y1mqZLNgWBjxQBMW99pCdOli0IJDgE0qT5BK/w+nYNjVcWFhbKjTaVTaeEyZI5S9ky5cqULcec165enTfP686d2zY2NnXqNihRoiS6Qo8f8/befPv2rXz58jdp0rR8hUqqexBsZKtXrQgMOPL6zeuSJUt17+H8xRcZeEIinjx5cuiAP2YrVKSIpos3X716BRnu1KlT58//hpSD5cqVr2RTuWrV6uq+mJkIVbAzoxp5/vzZ7t07eWz79tHGeN+4cT0o8EhYaOgff/wB/gIFCzVq3AQrMZNQ59Ah/xNh4Rcu/P7o8SP4hter18C2Zq00adLw7kDAU8fHZ9fePXtSf5Ia3ldtnb5NkSIFMcBAuXP7NoSVYRM0+yc9ePBg/brV2C9czps0bc6zhYeF7tix7ffz5wsWKtSxc5fChUzgeebUSYSJXbx4saZtzQYNG2XJkpWH37t3b9nSRUjs1L/fwK9Ll+F2DQGGoMCAM2dO46TnyvWljU3lmjVrZ8+RQ8Mmm4KANQgk0yUPlT9BQBAQBN4nAjOmT12+fCntYfjwkW3NupIQ2/3791vaN6UXMLVMnuwJVcrGjevUlbq4uvXq5U4tkFR6dOt85uxpZkDY/IZN3p988gm3QOJp3LA+poXC6cChAG5nAo/EXbu2z/CcxuHc3OXq6tGzlytvaoiAgMMe7i7U2KFDp4GDhmoYaFONaW/n1H7IsBHGbHBtnjF92tatW4y7GjZsPG7c+NTKEREPjmv8j2P8/PZohlS2qTJvwSKWk+7dvduhvaN6aC1aOIwZ+yONOnvmVIcO7TQzqJs5s+fc5avT5+FvwXyv+fPnEU2/W7bsyF+ggNoy12vWwkUL1BaVZ9CAvvsitIPRnQusdtZMz12KjEhTgX+z9/ZMmTKpMwstCFiDgJi3rEFJeAQBQSBOCBwNjnToqWQTrYevug/PaZMhmnTr2mPmLC9qHzJkACQeKIS85swfMWIUNW7fupVHTRg/DhJPxQqV1q7dWL58BbQjSF71xkWLj89OEqTSp0/HA5m4dePGt06tvxs5HGIBkgqOHj12t8/ew0eCBw0cAh6onSAWMLOGUD21jYPVwYxkgKO+G85ZfCC+DBxsQjCCKqtJIzuSeMCzfMWa46Gn9u0/XKtWHUyCxDnffTdMs2sIav36uEPiQS4AN7feECzA7+Kik8+OhgTv2KaH6OXLlz2du+LQevRwPnAwkCbBjv7++2+iv8iQqU0bp4h/jsCZGkEYGp369OtPjUirCIkH6RbXrf+lS5duhqk2E0G/27d6Q+KBgLJkyYpOnbroG7d5E4E1k8SDTbiCQa1F7fy7ZPHCRo3qkcTTrl2HhQuXHA0J37BxC84L+MeN0V8AzC+EIGANAmLesgYl4REEBIHYI4BAbo58xivQmjQ8Dx8+wNsdr9teru6XL13kfWP4qjUbYEdjnQp3/fbbr9u2eUMVMX3mbFhhrhuy7T14cJ95QLD4VaFiJbUdNHIk9urV/dbtW6Bhp5vmOTNt2rTE822HTgsX/ox37dVrV0t+XVozkDYhXnD7vfv39u7dA/3Kv2//ffv2zdOnT44dOxZw5AjJWxBNxoz9wa5hI+ZnAha93h5uxObu3qeHcy/qglZj3A8TGtnVQxeEGzhIkbGPer23/HLipC5wbMjQYS0dvqHGA/7+RHA+5R3bt0IKrFe3vrtHPyBMvfgFTXmScufOPWKkXpgIDw8jhVCz5vZ9+w1kZhCwIc6dMxvE1GkzihYtxjm479+7x2zgQbZJbC5dvhJnnHke//mYeAAO3L82bNDr7WAc5LEgkO1w9uwZ1DJjxuxatesSDR/51m3aTpo4nk1v6iihBQGLCIjQYxEiYRAEBIE4IRAerg8Cwixw6LFmLj9fH7BBLEBYNaQZHvLj+Imo1nnekDwG7XXq1qPejRGvTwhJkHjOnfuNLTiFCkc6msBPJThIL5qULx8lBhsvaQ/3XiTxQLSaMtWTJR7Mf/vmTSQaBpEz55e0O82v6tCDLlSZ0DDQJsS4OnXru/RyzZRFr0dR2eBt4+7WkySeZs3sWeIhHgR5FSpUiIx3fnt8VKHn4MEDxAM/JyJwpCxoIn0OGqFZgSQBYvCQ4fjdv0+XUYn+cuTIaSD1/8dKWKqoUEErHfr5+QJeB4dvihUvgR357dHb1PIrWSV9fXaBx9a2JiQeYOtvkMBIuqLdDBg4BM5etKMcio/Ogf37PD2nEI+HRx+WeKjl8qVLIHLmzEWb8isIxAgBEXpiBJcwCwKCQIwRCDl6lMfYWGfbOhYR+10/ooYDIuFpOOwaNWxrgoZ2oUnjpniPNm3WjLMg+uzeja4GEW7UJDNhE0LG11+XouH4hUssiRSgy1fQ2b/4b99eP44vc3Vzg+TEXUdDgvr36YOBkDPIasZdTKie2ijCZWtbi7qSp4DrMxyFU8CaVq58xa++KspDjInJkyaQ1IUuCATGDBky6J2yb1yPYgy6eOF3YvacNqVf/4Gly5RFnmsPj7779+8FUCgmil7IFlDzYP3kArx1q97M1LhxUzBr9sVxW2gvX668pnfvHl+0NI3waA4MDIACjBhU3RUUXWh0dPoWv3BDZtjrN7AjZvzC1wp2qx9++B5SJvubQzgjFREYcPo6dOzC/DDD/TjuezJ4denWnduFEASsR0CEHuuxEk5BQBCIDQLHlRx3FSNewBZnce7p2tKhFUVywceF+GvXrs0Dx0+czDQR0zxnIEwJ6hm8NXfv0odH4TWsxlvxuxxvWVXlgBlIC0JT3b93f/48r+fPn9+8efP3c+fu3LuDdgRkeU7X2XRM/h0/FinYtWnbrtU3rU2ymWmEcoU9dqHmMZki+datmzTDA8U4hZasWbORtAQ9UI8eXdBSrVp1CCWr12wgfvzieFHvjJQxFy9dYAmvhX1L5mEi9LheOQcBLm369NxORHfnXvYtW5aL8JratlXvx1O6dBnWM4GNziDi3UDv2rWDBkJq1MCeKSITdMWKkVq3oKCAK1evEH/mTJlQyvTVy5c4XtgryYSHrqFDh1evbks88isIxAgBEXpiBJcwCwKCQMwQePr06XmDHgKihia6J7q5ihcvgX/oRQwXKz/Ma4mqVtO9X/F36tRJElNAo+BDRJv+57ghj47Gyga/HzIbEd+SpYt4FNYMAaJ58xZ16zUwU1VKzdBTsZLWHsSzmSG2b9/Gvc2UsHBufP78GUsDWZXAbzBUr2HLAgHxQwGDf4cOHZw4aQqFbkEirG0wBbJrM1QpJjNHHzPIcDam8krz2UHIGMeL2bdw4KWCYB6oZzhEH37ZKg/oCxFVOypVrsztULkxjSuHLx40olI9IvAdHFpbeRXxPEIIAoyACD0MhRCCgCDw7hEIM+hpMHWVCOeSGO0jLFSv5sGo0mXKWTN2r5/O+II/ODWXKhXpdAwNEBvaKlSIVC2A8/69uxEjdD+tWrWGiPNJmjTp06eHGw2bXZjBmICnNitOIEbkzZvPmMdiy1bvyNCnijaRQgAPhDDHtKpTQWO37s7nfvuVg6GYDc7gtWvXadS4KbeAgIeN95Yt1GJv31LVhFFjFIceRQejTkI0QuG40c7OhF82eo8cPsQ89Rs0YJqIAwf2g1CFIdUbetGiZalSp06bLu1n6T7LlDmzsRlOM5tsCgIWERChxyJEwiAICAKxRyDEoFzBFBUrWeXFrO6MNShwRlHT7ag8GjrYEB4PxQbnpwEPnFrYs6RsuSgOPQ/uP+BJELdV1siLhXtNElE8tU3JKyZHaRrZcRgh95wtUOU54K+TD+ivnuIZgxYILlOmzQiDUSo05ET4ieBgfTg6unx279IIPQEBh9gLp7kp2xYbATHcTF5p9Hpv/gW/+IPUEl2dCpiriAf2wVy5chNNv0jDA2ERCRLVeqsP7utDwKDXMY6wU4cLLQjEAoHksRgjQwQBQUAQsBIBNpSA30qHHnVmNkhVNIowV9mYRioalh40rsqsNELEOFUHm/zTxO9HjcDYlIonb+48eXg2K4mjwfqIMPAjcbOVo1Q2xEDxZtFixZhmArFjVMwVLZAeOHQLLi8tmjdu07ol8txARECexnnzF/ofOAIPG/3YZMl4EiK2eetdmDEJ2YkmTvhhQP/ezMYOPWpeacTz93Z3Wb5sCbNdunCBA8Sa27dAO3RRri7OiL1iHhCnT+szG5VWcm0Twy+/rAfRtLm9yp/CcC5ipzBTpxJaEDBGQIQeY0ykRRAQBN4NAqqhxHqHHt73Q7ivGnLWWSn0PHv2lIeXKmV48Uc0HQsJoa4KFSpANQIn5bVrV12+fBmNuXNHCjopU5rWf0Ocmj1z+tUreh9b3guIEMVT20ZxT1F5zNNYD/Ahnk8/TWPMvPDnuaye+W7UGGI4efLE9OlTARHkvD0RQf7UDsVJu3btiUbZCiLoF144bAVr2bIVGhH/j2Q5NyKqvRIPy6kVFM3c8mWLjwQcRkUOnm3bdr3whJWTwzLSCapKJuL88/FjIjSRa/D1WbVyFQTQ2oYcPMTGns7RnQiwQZmnil80UH4FAWsQEKHHGpSERxAQBGKDgJoNWeM7bM104WFhzFaqdFmmzRCqPetvJd/d7ds3AwP1Rp/yEQ49ByOqflKanzx587LMcfRopNqGd4T8fh07OMHBefXqFdxIxKNHj1i3BC8ijQVHw2xms3YdXcJl/D15Eim3UQtqaS1atJBoZ+eeXKZKzbWj8e09dVLvAKQGRmEGtQ68XYRb8ZrVK9Heu09fmh+/fDgEFFqQNQBVRABR+w6dmA2O0kS3dHAgIfLgQX9Ee9WqU5d5QGQxZCTSCDGoLwFrY9du3TWlwcqX10tpOF+qAozmhGPW4kULXF16zJgxTZN2Ut2p0IJAdAiI0BMdMtIuCAgCMUYA7yS8qOAqi5ILiIteu3Y1T5E2bbpHDx9CX2L8JmMeDYGkwNSCQqSaV6OGkzczZszE4svZX09TO3xH+vbxYIce+Bq/fv2acgE3jHC/hQ8N3r7EfNB/P46CJwSBlDPtHNtAFEAlr14ubtyFSaDKWrliKbfkzZcPLTh8zQzMYIZoacikvGP79rt3dUHy9Icaom4uzkSjHISLa6QdCh6+1A7n67p16+sH/O9/qP+wdp0O+Tp16mryR7OqBj5SX3zxRUDAYaSxhhdRzZp6kQujoH2hqTJHVKIAeoMH9kML0j2rZ+GfV6+IrUYNW1Q9G//jWGwOGjxMlTvRwgqeAwf8iR+/qGKxbt0aVLHo3LkrNxJh36IVLQDniy2S1AU1Vd8+bl5es7CJwiAI1NeMlU1BwCICUnDUIkTCIAgIAtYi0LG9kxr7bWbYgYMBFgOj2n/blqKi1KqiZuakLrUQJqQEWIs2btiANyhkHeQIBg98WTJnznz48CEnp2+HDhtJo/Da7tfXg6wz8GWuU6fO48ePzpw+c/r0abIroV5V9x4u9NZHIH1Du0gpwXhJyKO4cZPe+mPcG10LkgMtWDAPvZDbOnbqBHwQ+nTw0AHiHzJ4WLuoJdmvX7/m1LY1Dg3SWNmy5ZB7+umTJ2d/PUtHAaXL3PmLNPl+YJ5zcGiGCaGUQn0Jqga6Zu1GShBAO0KORJKZatWsnS9//hUrlqHdw6Nv9x49iYF+hw0dhOgw0G3aOMJGBoce8M+YNUflAc17BN24cdNixYoHBQVihZBs1qzdoFFQ0Vgcde/erqDB06lzZ5TgOHfu3K9nTlP4OjyNRowaE+m0RGPkVxCwDoF4EHpgp//zyZ8oSYOPIXwV0jrxYYXvA+hI8ZssebKUyVOmSpky/eefWXwsWneYwiUICAIfAoFqVSqRQoW1BbxXaqdNvHF37N6D+517TRIs9Hh778Tb1ySPcSOeKzCdrF+3jveIxQwfPhIOsyOGD6H3NEZB1TF9hheC0nkGqC5Q25ze99yIsVWqVkHxKTVKHPmXu3fTGXo0h8l7RDTTpJ+m8iRWElj5zu1bp06dwu47NBDF2Dt36UaZlDVTwfA0bcpPGkET4p1Dq1Y9ergYF2PH8LlzZqKOGM2D9U+c9JOmzgNw8PBw4bg58PTt18/RSe8kxAuAU1TP7l04JRIkngkTfjLOZAh+pLT+acJ4TjKEFqjuRo7+vnChIjybhkBGykmTtAW2EM/l6NQOuR8tXjma2WRTEGAE4kHoadbEjrON8TqiI5AWDHnKq1Wv8RFd5fiyOXEirHadumocZnQHaGU7dOb79+0tV66CyQ8jKycRNkHg40Lg3r17v509U6BQIVXgsPIQIEBcunwR+ZRzfZm7RPES/PpHFNKpk+EpUqbCs8XkUwX3GlLioWJo5sxZvypcxGSRLCvXEDs2BGrduHYN0VhIL501c5Y8+fKZSYpIu4A57OLFiy/++gtphVCbDHCZPDReD55R165dSZ36kzKly5gUU8AJj5nQ0ON4iJUs+XV04egwVp4MD4NLMhI9QyrVGLZ4dyBg07xy6RIO6tXrfwoXhLQTrbjDo3AG4cyEITDJFSxYKH/+ApKnh8ERItYIxIPQg5jGx388/PvlS47LwOrxMZEjR/YUKVOmSJbsn9evX/79kj8g0IvvwinTPDXG6Vgf83sdiAhSxFNgF/go2eytT74exz2Gh4V2i/isxDx7/PzFkh1HPGW4ICAICAKCQNJEwHRw5nvFAmkkaH5V5bNjly8Mt+p+8S2yadP61RGRBRCA3N1c12/4xaSCVx0V7zRX8Ll79x6+2Cx+olmz4E0bddks6O/27dsi9BjAkP8LAoKAICAICAIxQMCCTT0GM8WQFTpkNnLB4U4j8WAy2HEQCGAbUVQZmzByr1+3JoY7iQf2VClT0V4dnZzeicSD2TjUEw6YZcpYFbgbD0cuuxQEBAFBQBAQBBI2AvGg6SFATp4MZ2TMJKevWbM2giyIE1Va+vQbwKMSJuE5c/aunTsqI/yjWo13tcLeffsj+0e2bDmaNWv+ruaUeQQBQUAQEAQEgaSGQLwJPZzpHIhXsom2Ig982fiUwFrEdIIlEJCJf+92eVmyZHVx9Xi3c8psgoAgIAgIAoJAUkMg3sxbVta0u3DhAp+SLzJkYFoIQUAQEAQEAUFAEBAEYoRA/Ag9L168oJxjWKta006zdKh5/Pfv5cZ69RowLYQgIAgIAoKAICAICAIxQiB+zFvh4aG8ykqVqzCtIY4fO0oZVNGOmPaWLRyY4ciRw2vXrHz75i08nevWb5AzZy6kfvfx2enn6/vs+TNUb2nYuGl0ma+Qf+LgAX/U/r186VK6dOlgXKtW3VZNSMp7IQKpuoKPBqKQ8pkzp5MnS16gYMFixYo5tGqjJo1AVPmOHdt+P3++YKFCHTt3iW7XPDNSy58IC0cWikePH6HAHuQ525q11BTv4ES61R3btiJ7KRJvIEV9DYNPN0+iIZDQIijwSFho6B9//IGuAgULNWrcBOUG1eQZyJaxauVy5GfLlevLypWrfNO6LfWiYkBQQMDp0ydv3br15Zdforij5gA1+5JNQUAQEAQEAUHgo0MgHvL0AKNZMzyXLltMYM2c5aWWfWEE4cHj1LYVJ/GcMnV6/fp21It6MR7uLswJeWjegoXIwYo0qdwIYtGiZRUqVlJbkEQLtYIp0bvaDnrhwiUVK1XWNGITZXc8p0xWkwYRDzKKzpkzn/J6qWnvqXfLlh3RZRFE+ZjxP47x89uj2VdlmyrzFixiASUoMMDNLUrS9x/HT2za1F4zijb//PMxMslu3brFuBeZYceNG0+Z2SDx9HLudjz0GLMhSf+w4SOXLV08e/YMbiQCIXXLV6zhlG6aXtkUBAQBQUAQEAQ+OgTiR+jh7PLA6/CR4PTpP9MAByXH8GGD2QQ2cNDQDobqvtDoNLKrCw2Q15z50L6sXLmMx0IQGTp85NrVK7dt34pGFOEbNXos90I59N2IYZTfHRnTm9m3qFy5KnKYQlratXsnit3s9t376aefMj92NHTwgH0R9jXIVf0HDKxWtXrGzJndXXueiAg9IykElfPGjfselfOmTZ/ls3vnsmVLMEOXLt2Qt56nYgIGu25dOmI4JkSBwwYNGiKD6sYNa+bP1xXcGTd2fPMWLUH8/vt5x7atQEyZ4pkmzadUhgbx6qvXbOCpmEC+9t4ebpT/HiLOt+07IoMqRKtxY0ej6DHYGjSwmzxlOogN69dMnDgeiWgBS+NG9WkGJFGEZIn0j9179rK3b3np0sXe7i6kYBswYHDHTl2IzeIv9qhWSbTIb5IhW7bsmlJBJtmkURAQBAQBQUAQiAUC8WDeUh16IGoEBQYmT6FzLYKt6s2b1w8ePIAih8u+oG7fDz9OVOOhELiOVzJe5NWr2169fJmPGZV0fl64FL0k8aBdjfxChZreHnrlEAoHurn3JZ0K0qsjIBxCD4Qh1OpjoQcSD8rpkcSDYn6YmfIi+vjsIokH81/4/fe3jd7OnTMb9NRpM4oWLcZZBO/fu8cLUwnvLb/QcNQr5qLKB/z1xYevXbtKzEsW6SrjDBs2on6DhtgjNZqc89rVqyzxuLv36eHci5iR92jcDxMa2dWDMAStEsRHncy0aiV6h48YnSNHTiBP8h8knkqVbObM/ZmsdYAahrYtW34B59mzZ2g2a34njh/nu8fHGk4zPCgiCPWSGQbpEgQEAUFAEBAEYo1APAg9qkMP3rtDhphOvQNxp1GjJp06d1VdZ3CcO3dsw2+TxrpCwawKguLkp8nTUqRIce/OHcaidp06RN++fdPNRW8qghXJ3aMf84A4feoUbWbImJHbUcMPhi3anDDxJ84EffbMaeaxta3l5+cLCczB4ZtixUvAeOS3Rz8ElWiYTSUOHjxAm1xLCKPOnT9HjcjuA+Le3buQHqA6auv4LTb37/OjXvgSEcG/SPDo7taTdDzNmtmzxEMMqKRYqFAhqkTot8cn9SepUfcDok/u3LkfPnxAEg84dSU+pk5XQc7whT5K7q/nz3lfFgn7lg5p06WzyAZZk6oCGQx5yTAkWbL/oTHZ/5KZ8ayyOLMwCAKCgCAgCAgC5hGIB6Hn+LEQXlO9uvVRSY42U6TEiw/Jh1PCwFG1anWWM5iZiOCgYBBVq+tS/4WHhVFjS4dWSGYDukHDRpAYbt+65dTu26pV9ekBly1dwtWPBw4eSkPwC4Fj3ZpVU6b+BLprl+6ffPIJdV25cpkrLaNdrflVp279XzZtAlvrNm3LV6g4eKBOfmratDl+AwMDWJKwa9iIptL8XrzwO7V4TpvSr//A0mXKQtrw8Oi7f//eJo2bVqioy1fk768LWGvW3B5iwdOnT9n7p1GjpprZJk+awFmtBwwcounFZgZDkP+N69dDI2Anvyg4OzPz9FlemlL2v537jXozZ8nCbBYJJGN8h/kYLe5OGAQBQUAQEAQEgZgiEA9CT0jIUV4lTEus8+BG88Q0zxmIcoKAggrM7F9cs1ZtGoW6VBr7CNQhGw21q6AQOnTQ38/X508Uwbh1MzwsnIQhCBwubpHZ/2bO8OQ1ODrp1C38ByPa4YBgSEukGunu3Mu+Zcty5SuAYdvWzcQGG010B4XlkZgCBUyPHl3ADw8byEyqs06dOvVzZM9Ztnx59PoabFug7Ro2xC//4bhglaNNqHlMusLgGInhwcMHtWvX+zJ33goVKqKFkyR9XbIUrHI8JwhdqFpwILUk/JIX5cqWVBcvtCAgCAgCgkC8IBB+4my87DemO/3QQs/z58/YJpUlc5bohAMzh1G1WnXqDQs9zmxly5ZjWkMgDIpbIOJ4ec3iTVh26tarB+OUGuT1+vVr8v8FG8QXY4UTjGj4o0lgjiGLDNx4WSVjr4TW876IqF7Dll2CqAX6Ifw7dOjgxElTyOKDPfJOvb31ghQ8lDXu3tu368x89NcsQtVk2NL/H1Bz7FvWLFnVaY8axBobG23AWli4XnmGWSrZRJtNQLMv2RQEBAFBQBAQBBI+Ah9a6AkLi5RUjN+4McLr+HG9mQxBW5oMN+o8UAjx5ogRo4p8VTRtWkSap8uYMTO7LTMDiIsXI3NAV6tWTe0yQyNFEPfa2Zm2bYGhW3fnc7/9Sv7RzA8CiXNq167TqHEUAxZy57CAaB8R1aUO2WqQh9BY0Uh2QeOpUyeZXxUuHz18COce6ipfUaf4Uf/2+O6mTUR1wftH7UqA9MfybZEAoZMlCQKCgCCQBBH40ELPsZBIhx4zdUatORNHg4OIzaayVl2hDr9/P1LoQTCUSTOQys9yBhrLRtit1N7oaO/NunAn/EElg4gwoo1/4bU0ZdoM6KiOh4acCD/BhiRw+uzepRF6tnl70wxQiaGAqWY2Fs4qVqjEmieV54D/ft6s18CO6WNKnp5y5XSGOf6Dlstnt17oaWJKe8ScxgRwO6UUkTVmsKalQKFClW2qWsMpPIKAICAICAKCQEwR+OBCz1GdGzL92ZhKBmjotPB/ZFVmH15EXJvh5rgkOPRYlHgwz5+PH/Fs+fMWYNoMcenCBY7Aam7fApzQssyb6+XY1ql23Xo0cMXypb9s2oBcf9M8Z8KaRgY1JBXs28ddr5JBCJPyh5j5bVv1Qk/z5i0gLcFcNXzo4K9Lfd3LxQNORcxbtFgUpxxqR2rHPb6+RKPQB4K2mB95fYhGI5Re3A4iMOAwe3yTBAbm5cuXdO3ag/yWVGYNjQOEvkrTGNNNqJc2e++I6SjhFwQEAUFAEBAErEHggwo98L09bwhfgj/Nl3nyWLNEkzyhikNPqdJlTfJQY+7ckXtB5h5DpHSUEYhpX79uba+errB7wbeX+7Jlz860Srx8+XLhgnnN7VtS2uVt2/XSCZLfIO4MnEsWL4QWB0IPjTp58sT06VOJ3uPrw7HlGTJkbNeuPQk9qBeh7gKuSBwLRhkLt3lvORJwOE/efGCDDMSJdj79NI06kOiFP8/l4d+NGqMyRGrIjIxi/gblEEAg29akSeOhUurcuZs6g0na1dXDtmZNk13WNxYooA/ls36IcAoCgoAgIAgIAlYi8EGFnpCjkXFbqHhl5RJNsnHQNRx6ONTcJCeFQaELOozz58+peQ6J/8jhQ0MGD0Rv8RIlkRmoVKnSPM/jx4+R5Y83iUDe4X693SC9PX32dOR336MRnsjU1dLBAeLIzZs34QqNMg616tSlds61g01NeYpTJ/WeNxWjutfwnNDHFChQEKqdNWtWYXiP7s40J7IQUd2JJ0+eUgv/oqTXokULadPZuefXpctwF46IHXoqGQk9IcF6PRyF3AcHB0HiQW5Gi2oezJ8vf3784x0JIQgIAoKAICAIJDQEkn+ABUG/gkBo5NzbuHEd7y5Hzpwvnj9XzTTcZQ3BGQ7N27YwFWp/Iiyc5lRrtqMFJqS5c2aiyAMkHiSwQVEINELZ09DgiTx/nhcNpF/YjPb6+Tq1bQ2JB6JALxc3ffurV0TUqGH76tWr8T+OxeagwcNYq5Q2nd6KhMoYdevWJ2b8bt/qTQmB6tSpq2YDQheSUxMb5gQxf64XzHkuLq6ZDLlzOKHzju3b1foPKGXq5qIXjFBay8W1N81Dv6ojucahB6eJUwBAHsIpmzN7Jkb17tMPkpw6idCCgCAgCAgCgsDHiMB7r70FKWHw4AFmoGnXrsOQocPNMBh3oW5otar6sKPFS1aUt+RuDNVLz+5d6I0OsQNB5tevXz979jQVKIWdaPDQoU2aNGcZBYJLT+euZHWqUd0WCQnhKRwUFHDo4EFyeenWFRYqVw4ZQ8EKcmdp08YRyiQMRG2vGbPm8MpRSgyiEsZCVEJ0faHCRVDy4uyvZ8mRGTqhufMXafyNUNFi/PhxmAEppLNmy4YS7linpjoYZDIqnoqujp06IccgtFYHDx2g/Q4ZPKxd+468BiK41CvcnxcuXqbpde7e5XiEmzNOCvDBgeDwZ3nNY2Q0/LIpCAgCgoAgIAh8RAi8d6Fn8aIFlBoHfsSMC4kOtDly5OjWbRy5yxoCeYpr1awKTsgQ8Hs1GbukmQe6kAnjxx0+fEhth18RHI1d3TyM463gfvTD2O8DAwPVpeIQqteo4ebeG/YmdR5VqEI7JJ4JE36iAuzMhuJfKG1BRSG4EWFZDq1a9ejhYlzMHIFUKPx5NERvb8KRzpw1V2Mag25m5/atU6dOYfcdmrmdU/vOXbpxsh/eHQgWekzWbIdHdm8PV9b3tG7ddtjw76yBV92F0B8MAagqkWv79ZvX0JjiYkC1Oewa/8ElHsU+IKqiql3K5CmTJU8GEzASY8qp/GCn5gPvCBfAn48iIzBitPcMmTLFXZWL/BrQ5SMy4/FjZH599NfzF916OFOW/Bgt5p0wo8wOFvHvW91N8Z8h5kN3X0TUwMEv7gjcF6lSpkz/+WeafPTvZAEySUJG4L0LPe/p4M+d++3+/btlypSL0SULh+XLl688efI4T558hQsW0sglxkvFLQOnFgS9P3/+PE+evPAHiu7pANfmk+FhUEGh6hZcW3BbGc+GFsheFy9eRGl3FFfPnj070udENyGYsXfoWnADZ8+arXjJr6N7Y8HoduPatRs3rqdJmzZr5ix58uVLnTq1yb2jESsMOHIoS9ZsUDiZ5IGWC3VGHz9+VOyrYnHxNDc5+ftrhDGuaRM7Kg4fi71AnEWayh/HT4rF2HgcsnPntu9GWqslhTrQ0cnpm9aO2bJli8c1064RFfjo0aNatWobi/sxWhs+Tvbv2wtDreZ7IEaTJALmLZs3jRun8y+Mxd/8eQsrV9VmxIjRPEjNWruWdoYd233i6wHSrIkdx/ZaPBA4P0CxXa16DTOPYouTvBOGq1eunDgRVrtOXQS4xHHCd3V/xXEZCXP4xyr0JEw0ZVXxhQBi4oYOHRiXvds3bzH2hwlxmeHDj4XteOnSxZB6OWkTrQHVaj9NmzZFsmRvI9zpHjx4qCosYYSFAi8eH/F9eruSzrVtW6fhI0aZxA3SOeT4woWLmFlneFhot26daPgeP3+oskxOlRQaoVBfuWKFRuNLBw6Bnr6CcJ2gRb0SiGHVqrUah8KYIoYEsMOGDIQAyingIWEfOKQP74jpbHHnd3VxfvwHrvmXHLSBOYEDct2nSJkS98U/r1+//Pslq7TRq6u7PM0zjjjEZeVI+UERvmbSdkCfd+XSpVy5c5tMq8t7t+b+YuYkSHzQ6K0kiK8c8odBYNcufXYfPNpcXN2q1aiROVMWFK/FA27qtMlbtuhTR3bq1MXF1QPtb968gZH0woXfF8ybQzbHvPnyf5ilvsO9INkm/mHCXTu3jxw5jGZ2cvp26LCR6l7wttu312/JkoUkG6EU3WfpP0PZO5Xng9Hwb2Mr8/Xr103u99q1q+0c2+D1bF4Shd8bD799+3ZSFnq69+iFf0CD352gTaIHkyj8DtWsWjlz5GIYY0dAab10uS62FK6QxyIKG2tiUWM3baxHzZuvj11VVT47dvlqQnGhWdm0af3q1SuxIwhA7m6u6zf8YtIrINYrsX7gyhXLiPnu3Xu4YY219Wjs1N4RMTSQz3bs3hPdx4A195f1q0qUnMkT5VHJQSUpBKAVoHJpSIft67e/U+euCNmDYzhctWDBxCOe0UCaaXwkIV8lfvGkRmTc5Kme1Fuo4EecIujM6dN8jFWNaqfgAdq4SdNxih5rydJFkPl4yIckkidPwbtr30Gvp+EWIjb/spEUEnfu3tF0qZsQXmkTaR0SfnFcWipkDjgX7tvnpx7IO6Rh9+fZatauwzQTsJIDLlT6q1VL35sxc2bujQuBQyOJB5NQYeO4zBb3sdA8sZELwSIaiQfzwySKGFtbW312MejJ1q9bE/f9xm6GVClT0UDYoI0lHnSFHA2mLHeQz+D5EN1erLm/ohubRNpF05NETnRiPsxDhw7g8KBRHz1qjMZPC/IQ1xWBEuirr4pqgMiZMxfa8YqFM5am6yPaDItSWkQf2KhZP4IWARFbQEKOBpGWSMP2vjeR9HK656zwE2H2zVsWKlLE5O44SVWxYiVMMlAjlFU5c+XKli1Hs2bNzbAlqK6//voLmrb79+7Vq9fgfSzsWEhkLrSKFStFtws4HdasWRufCrCERueAGN3Y6NrhC8hd5aPfNfO8b+KkUhXHTMkj4MCqxwMH9vfpZy7W+P2t2XPm7F07d1RGvaFqNUzuJfKmKFosOv9ODLTm/jI5f9JpFKEn6ZzrRHuk+/buwbG5uOoSamsOkvM5oR3lY00+30mpkCdvXs3Yj2UT9UnoExALRskR41BEkwcCbbnJ9g/QiJBJLs9ivDvE/7CLUsVKpgU4GoXgIBgrjWdIyC1mvtHjvmx4QbEvP64E80Ee+fLlwx7VhPVxXADXt8FXRJEiX8VxtrgPDz0eWdzaTC5c9YzE401RrFhx/DNz1MFBeh8pMwIcDTd/f5nZRRLpEvNWEjnRifYwETfn56cTepo0tTc+yONKgVuTDz68YjEKzoNmPp6Mp01QLSfCw3k9lSpXYVpD4EhZzYOuz7/IoGFIIJuhVmitEshSY7WMaA0TsZotyqDjEf401GR8JcDcg1BQHgA/X9C5vvySW+JIHD8eQjOgAnR0Hidx3EWMhh83FBnEqHLlopWeL1y4wNN+kSGB3hTIBcBO4iafY3wIQlhEQDQ9FiEShgSNQEhIENYHn83PP//ceKGc6AhdFUwVpr11Q+dLa2z2Mp4qwbaEKXXozDwQ/ffv40PAt7jFVObM/IEJrjADPwwrtVYfeIVx2R0SKsVluPmxUYSeqHV+oM9o0bwpNBmBwfp6w8jcgdly5Xo3Qg8cesLD9MK3RVWE+aN4J70vXrxgu7YZ9SdgUdP0vyebY9yP6ERYGE9iRoBjHiHMICBCjxlwpOsjQOCff14jJbfTtx2M14o6J/zggztLoYKFjXlSp0mD8m01a9XmLsSFrluzat/+vZkzZ65pWwu64vTpP0Mmkq3emwMDjoANVc/sGjTieiA0EK6pu3fuQOxuwYIFO3XqauytgrcCPAbSfpq2qqEoCu9RJY4GBYaGHTt79uyff/5ZtGgxG5vKWIOx2S7KEEMGSzSaeSBu36Yviwu2Nm3bIl2hOgnT0Aeg2G1QUOD9+/f/+ecVQqLqN7DDGswk1Dn326/wirhw4fytW7cyZMiA7Dv169lp8EGyzYMH9iPIDkiWLl2mS9fuqvElIODw7Vu3kEgOyzh00J8WkzxFivXrVpNFsrm9A3zPqR3xKTu2bcUK4YqOYiw1DI6ofAggkC90j+9u+H6WKl2qUcOm6ulAJqo1q1fu3bvn2/YdmprSDtI8Dx7cP3jA//TpU5cvXUqXLh2kyWrVbeEXpe4lFrRqTInFcPNDQhSHnrJlK6jM4eFh0PPhTuHGXDm/bNy4qW3NWtyiIXAZQIr6/ffzCPOuWrWqQ6s2cLCF/3v69OmNFTm/nT1DZmJMYsaXiHaBUjknwsIvXPj90eNHyH8GUQPL4AT3mmVgE2vAYs6cOX379i1IabqbomZt82FWql3bWOnFuzh+7CgbBPEl0LKFA3epBNKAHTuOqyn40qULUC3jLsYTo4FdQzMJdSBf7tyxDSu/eOHCi79f4JqvX98ORQw10CHtArLt/37+fMFChTp27oIIDN4vxp48EU43xeFDh6gdi9y5YytNUrlKNWR6o3bz9xfzBAUd2ePr+/Lvv+vbNYQnO99TYDh75tTixQtTp/6k/4DBuLNoiPFvLB5QxpPEb4vk6Ylf/GXv7xEBFOXo3duVdtCggd3kKdOt2dnsmdMR3MScKDjff+DgoYMH8MMRXRChvLft4Efe5J8mrl27ioeA2LJlhyZdXvdunajsiXEXDTx58sTUyZM0ObvRhTokc+cv1DwreV/4oq1eTe+vii/adRu3cJdKqDHtJUqUXLp0pbEQA7Fs1crlM2ZMUwcSjXw5Xl7zjV8zEBAXLVwwL2qJOgwBPlu372SxBk/kPr3dqOgKTQh74tr1v5DghUlqVKvMr0zjvaNl124/uJyDwMvPza2nymOcWzww8Ii7Wy+V54dx45vZt6QWCFKTJukTMnl77zSukgvn9+XLFlOBF3US0AsXLqlYqbKmMUabjx4+rFe/lqZMTYxmiI4ZsmAL+ybUa3wlIL37kYDDzs493dz7RjcDt1+5cvmnieNVLSm68G3wxRdfoMqNyTz4y5ctoSsHb+UjgUeju1wh8o7/cQzZo3l3IHCRz1uwyNjlDlmeZ8303LV7p8oMGhfYZu/txgFZzMap59Eyc5ZXzZomAtkglzi1bcVmoylTp0Mu4RmYOLB/H25wNakPdeFIZ832Mnk9ID/18KGD2TWNp3J374MCRry5YL7X/PnzeBOE+nCgU6b2auhxY8c3b6G7qs3fXzQKPO5uPTm8Do3qPYjeBvVqk/kbMbCTfpqq2Rc2Y/eAMp4n3lvEpyfeT4Es4H0hcOxYZCRLJeveVUjdAYkHj7PdPnsh7mBlkEIgr0DiadPGcf2GzXjioxFPh+CgQFr3jm3ekHiQPANBSXipUKO3tz4zEG3iIUgSDzaRNpAa+Re6B+R269K5PfaFXffp03/d+l+OhoTP/1kne+Hds3rVCmbWEOoXrcm8ulAtICkLZ/HB69Bz+mxjiQfeHm2+aUHvLfB4zZkfEHgs+GgYkh5hj3h8d+3cAevU7H2u1yySeFDSbsmSFYFBx+lxCXw8p05m5jGjR0LiQerbffsPQ+RCO9406vO31TffoDguEKZeGhjRgkYn1NkliQffviTxTJniOXu2/m2xZnUUcROlVAYN0KUgGjZ8JPOsXx9Z6vjQoYO8MLX+LjUeOXK4ccP6JPFANMGL8NDhYB+ffU0aNwXDoIEDIBLx8ARFsB8xVlXi61KQhp89e4a01/g0x9sOEg/ay1ewsbhmeEN37dyRJB5gGBQcunOXL0yNJ06GU10/ZPx7ZQRCiEHdaMahB5divz7ukHhwkbu59cYLHtcDTi6WhN1Be6dZ25LFCxs1qkcSD5ImQ+LETbFh4xaI4LjAxo0xndaSJlEltvLlTTj0QEbs2qUDSzwDBw01lnjwGTBkcP/+A/pA4sGaITojB2Zo2Gnc6diEmO7s3A3XpGbZkGs7tf8Wtwwi41Bkab//ERwpriWwzZkzC/slfiTRhsQDHtzsXbp0o8atWzcTgd8atrZ0U6AcNTfWq1uf74s69epRu8X7C2w/jvsedxzOo+8ef74H2TJ+/rdfSeIBp3oh0fxxeUDRDAnqN2WCWo0sRhB4hwhAH82zWelnsGH9Wgzp179/jhw5Yd7i4d27OXv06YcPPs7x+vbft+jFxyJqn4GYOn0mHiUwitGQB/fv81gQx4L1K4FslD2q6hh6jqFDBlKeIfTie5d1D5VtquJBDEMMvrzV2VQ69JjeRQONeB+DGZ/LqDf05u3bVy9fnTlz6oD/ftZR9ezp0rOXm7HLNjT2ffu40wsASm8kcWG9d69e7sGBQXjh4bmPLCbIgcR7x6oWL1mIzaZNm48aPZbaz57Rxy1fvXaVWsCG9xYObcrUGWnTpn3+7Bm1P7h/jwhoBZAuhWggTBZJgDlipPattmTRz2AbNmwEgu19fHbREIR/E0G/C36ei7cRXjao6If3CjWirArzODh8w9G/f794we0gUCCvt4cLtfTo4QylCOke4FqE8HgcBV4MKBXM4Khj451WhUhk4+SEnOrCLGYzgnHTtVcPev+NHj3WoVVrDM+VK/f0mV6NG9WnqfCe1thbVYeeyjbRasK8t/yCCwmTDBk6DHZJmu2Av96aiYyU1EK/SNY3e/YMomfMmA0zM9GIC2vdpu2kieON9Sg8XHXogU4oKDAQRejQ+/bN2zdvXj948ADmVIYLItQPP04sZipyavJPE0gphU8dr7kLYImjXcDk3aEjhDDd1eg1e8Ysg/xNvdOmTSa1Je5lGoKiRFwH+s6tW3nz5gNic+fMBv/UaTNgxeY0m+rF7OjUniaE6dzffz/R48aN14Bv8f7CQHx0bdu+FSdu6fLVMCPyXmBDp2khJQMHgvSvv6LcFP/G7QFF8yeoXxF6EtTpkMW8MwTwmXvOkJYQDz5NjViTu8Ht7e2tMw81bNQEv2cNeUfwAnbz6IMWuBTwwBo1aoL28/PBG6JZM3vw4EG2b69e6ClUONI2D7ajBmeLCpW0qVM8p00miQdsEyZNZokHmxBiaI/Zc2THpsk/9YsWCWDwz5gNX6UolNutu7NJlxR8f383YiiJGngsTp7imTp1anUSOILQu8rXd7cq9ARFeDiBkx0LQJ879yuNhcGCCDjQgHDv0wcSDx6+LDXmz1+AGNRfVs4Z+1nD0uG7xwcrbOv4LYbsN+T3K1CwIM9w585tUiTYRxiz/P33UZeKKgQmyD0kE2TLkYPHojCfm4teUYfFu3v04y4Qp0+dos0MGTOq7QmHDgzQhzRHtyRoLs2La/BcgRWSEvrhema5BBPiGwDWEBKLKxjl4Dn321m2TpoMF6AlHTx4gAi+YHDH8U2KFDXUi19YlDw9dd8S+PPw6MMSD7XAywoEKf+oRfOrqj9xew4ZYjr1Dl7zjRo1wSWdKlUqzQzYXLt65YYNegUhij2zxEOcEc5zOqEH3my4T1Vg/Q1iXPbs+qvryuXIj5bSEUUP/fx88SmC67BY8RIAwW/PHprWZLawUEPsPU6KRuLBKGvurzWrdPdgW8d2kHh+++1X/grKkzs37RcfHlOnzmjnpEuDjmId1Ei/cXxAqVMlEFqEngRyImQZ7xgBNV9fFeV5amY3+NDBPY/XLZxRUBqWHw0QF8hHoX37Dhcv/A77lLOzCzms+O/TvVbt7VvgF4olfvTXrav/LEY7HmohR4/SfiFAEEG/cHymLPjYhOK6bLny3It38IC+vSkBj2PEa567mNB4ardt60hdKCKtq6+eImWqVKlLlChRrnxFY+0OT4Lvb1ZQDRsxSiPxgI3jeK9cvsKjQFy8dJE2V69amTdfPiRVg0akQ8dOf7/4u2y5ct926Ei9O3fsAFG3bgP87ty+jRqzZM5SxqjkLdQM/PlesaINcfKvv79OoGzW3B7aF7jTsl9Io0Y6wxP94WUJwsmpHex34OGkcw0aNDSw6P7fuUs3EnrUdDLLli7h0wcnLubH6YNj+5SpP6Gla5fu0TmAM3+8ENeuXiX1DPYO6WSztw5z/MEwARdmV5ceoG2i18FE8P5v8eKfUeGY6JHffU9aLtqERpMtQRUraAX344a3MsRrFVIay7+4d4j2nDalX/+BpcuUhbTh4dF3//69sB5WMJxxSOGzZk0nTlwnHTp24RkgXsBMQwavLt26c7uGUKPYcFsVNCRbT5ES9zHyeKdEuvaqVasb+6jxPDBzT54yiTZhe9L456GdbwrQN2/e4KPGHc1X0fgfxyLnOLpwpHCUefToj/btO5J4tHePLwZCRYpf6B353Nk1bIQWzZ+ZLwFwWnN/+ezeDc4WLVvh19egIoXyFQoe3hc+DBCisXv3Tshh3BjHBxTPk6AIEXoS1OmQxbwzBFh9jRmttG1B7QxrPX1ssbUbwzlHaiWbKjt26b/JaKHdnXvZt2xJz2sfg7slfGLUpySs/vwc1LgXbFDS3v/7378LFszR1UG8c+fSpYv0+sdbxGvufPaY1qATFh4ZyFqtenVSR2l4LG6SRQ9seAhWr17DmB+RUNSIo4DDI38W44VE7XhkDxs6CDS+ROvWrbd4yXLVZ2ia5wx8X+JZD+lhy2a9y0KLFg4kR6q7UxOrQGxSu0DXqVM/R/acZcvr5EJ+cIO2axgp0NSt1wAKAOLZG5G9iSapF9VBNWvEyoEtzjgxQN5iJRnaEUHm5+vzJ6ymiVUAABWSSURBVOoY3LqJSGw6fXgxu7hZzoUIt5W7d27TtMa/KHWJxpCQEKc2Dsa93NK1R0+8JnnTInFcSW5UuUpV5oeIVqVKVVyQkJ7LV6zI7cYE5AmULKV2WHNwKlWeXw1aTzRWMMoYyUJGdPk/aSpEApIaCb5rPXp0QSPcvPDiX71mAzHQb1BQAAtYmTNlgkfaq5cvHzx8cP3aNdI4gm3o0OHVq9uqo1RajWKDXZIVSyqPeXqzwTAKttbftDVmvmW4KdAFJ57/GZIxZlQy/Wzb5o1/UDNXqVq1Q/uOX5cuw/PQcwPBXGjZZvDjQYSX8VLNfwlguDX3F/HADx334M4d22kZDRs31tyDkAXRVbp0aWLAbxwfUDxPgiJE6ElQp0MW884QQIQpz2WsNuAulcC7GdZ6auEXMIJWVN21yg8aBiOyGcE04OfnR712EdYx5uSp8PhT7SxggMGI2WC2Z8s9GmGMwGdfc/sW0Uk84OF0cKCN7UE8sxkCpZrYvtCkaTP1455HnT51gmhIAyzxoAWx9wsXLWA2ELCR4Z+Pz+6fFy6lByixEQ8C/vmLtrkhlkodzmp8uFsiTYDaBRrf5fxp7u2tF54gGaicKs/OnXqtUo3qtpp8P+cjnE9h8uPjRVAY7w4ijpfXLN6ELFi3Xj1YIozNOsyjEng9c4JstV2lsQvzPAjgV/kt0lGqTxjlo7p+Qye2lisXRcuomRMJBUi2Q7uNUYrLUINQBTTg4qOOxXs0NDSUWioaWW9Vzuo1bFlqoXYoOfAPruVwI+NzgeK4PAooqUBBiYXgdgeH1upHBTMT8fz5M7LVYhNyubEYoeE33oRkv22rN7VDEPkyTx5jntOnT3EjMoMzDXEftlHV6IxrHrc5/sEXjd10+LmBcDbWWdqbCpjnpwd2YfwlgEZOgWHm/mIeRMizAruBnVardCFCFVdR0QjG8QHFsCQoQoSeBHU6ZDHvBgE8SvhZCVHDzCMyuv2xE7SVwkTw0UB+Z9hF/UZnQ5tx6WkuiIgP60GDhqZOnSpdOlRAT585cxZ+B0S3QrQfDQ7iXitfycxPBD/csWky4Q0UACwVFSpUSB2O79QBAwZ7GnwvuAs6Knh3smszt2/dqvOXwh/kOY3wR+2RanyDPxC1a37hGMRvNfuIkF0NAzYRtcThcnZGD3cq1ta0STMeCDU+0yNGjCryVVF4IKVPny5jxsxmRF4eohIavYXaBfo9hawHB0VeCZo0OTiDuDKhU8ERaRajbp46qTdsodH4WmIDlnH2y99+jczQwyYqdWamYSZGSie2pXI73qy1a9dpFBEfh0Z2swW9aNGyVKlTp02X9rN0n2XKnFmVuXm4hlAj8ixa9DRjaROXB0vntram8xhBeuCxuQyeMdQyYdKUTh2c+NZmNiRKqF6jFmpjcQsIH5/IaHzjCxUM5r8E1Kks3l9g9vX1oSE621ZUZR6cIKEdL1y4iJoriI8idg8odXkJhxahJ+GcC1nJO0MgXElSbKVDj7pvJKbjux0huGpXdHRwUDB14amhPtfgoMBdGocePGV4tpIlS5Gum1ssEvBZYXEEX7RsqbE4UGW4di3STadEya/VLqJJPiAa2dg0DB07dYGoFBBw5NTJ8JCjIfyq2L1rl0bogZYekWU0vEVLE5adKGp8IwOKut9t3vqvcBx1lSrV1C6mVXHQuNi43x5faK2qK1kN7xtCyTADPJ1ZTcUTJmQCDiiMPC4/To9Ea4bQFnLshEVxAf6tfIwasQke+pxjyVhpymoeQIq98yTGBIwpU6bNgOH4eGjIifATPCc4fXbvYqGHw/qg1zEWv4yn1bQcUyrPWGnX1swAOxq3lFPc7LgRzwdY6GgTekSNWIzsQZu3bD985NBJRAOGhTIn+P3378Utw/OA8N78C21CZ6nRR1K7lV8CFu8vmi0wInMBaA53p3b8ImMkfps317kn0l8cH1CGaRLc/3WBfPInCCQyBEKUIkSxePCFKjJTqdJlrQGHSzprXhiXL15kDVC5Cjqniq3eW1xdnFEJCw6VPHMeUyp07jVJhCmlhVQ3DpPM0TW+MISnwu3D2IUZo3bvivwSZZsUno/tv23brIkdHIcRFtehQyckftznf6htW6fodsQOT2AgTdjmXzYBh1s3btAQVY2v5pUeN3b0iOFDeFq8gFk7hQc03qMwZyCNG/K8MQ+I0waHXLinwJVB7ToaEgSJtmmzZqocwDTe3B+XxIND4/ciaJO6DT46FQcN/c/rf6gFiKn2GjQiFTIz0zUMPxvk/oEKCe1sWatStQrUk9Cx4bSqZhHwgL9F88ZtWrdEEiDIMciDMG/+Qv8DR2A80s+MqG7DXwpDLFXs5PhjSqIKG+uycxn2rP+/msigaLFiml5ssr4EdBtHR2KAdhkJFxrUq4WrETms69VrMGDg4JWr161duxHKZuJJljzKCxc5pfi7BYZs8MCRHOiRPz42o/sSQGQZ9oJgC5oWvxbvL5qNv+VKl9E+1jZt0HlWUeAqTRvHBxRNkgB/o5yDBLg+WZIgEAsE1NdALD4WOT0XHHqsjNZ5Ysh4UapUpBsgVn7suN61CG9TJORAC0w/+MbFtPhA5KdhqlSRApDmeNevXwN/C00jNlXBjuPDjdnMt2TNpndHwGqMOYODgzicvn//QSQNwHtp2JAhMDDhAbpOccRGgJiLq97Pt1z5cprZvLfovXCQGhtV0hANNGvmdODAX8msxof4xZ+8yKmNMCsqCksTwvmGtRqUjnab9xZk3nvyJFJtBs6nz54Sf/GoOnw0LluyBL9NFNsWNtVi41DO0VjNL+LppntOQcScpj3eN1W/XZNCjzUrzJ49O7GVNYqqY/UAVGvQYuLczZ83F/7duIajOPRERHWhWgtOq+qTdPLkienTpyJVAeyeewzmFewLzmrt2ulT0agKFQ4OV1+6mkPAlYkc0JpGbEJKYLs2LDgm3XGMR2lasmbLxi2qxxg1Qqqb6zWbaFvbmpzrGbl2kKT04R8PcTWyzA22YsVL1Ktfn/jLlY0iamzb7k3teA4gmgw08jGqCjD1S6BMmfLEjOKjiCyDgk1V6Vm8vzD2+dPIe6ToV1GEOQhbcLeCKZ8vA/DH8QFFq02AvyL0JMCTIkuKEwJ4KnHkM54m1mTo0eyPhZ5KNpU0XdFtckw4EqMxDzwiodehTcgBUEvApwGPRRhlyMHCprIN9QYbshfyWBCYavCg/sjDhhBftV0/JCjS99Y4oMaY32RLrdp1qP3PiGrzKg8yFo77fhS1wDDXoWNnovGKZd2VmiMHvexOYRPVKQcfwfxFa2+vs23t8dkF2aVjxy5coosNbeUNEdF4gY2NyLrbt29/2jV+4fdKNJwMcGbx0l2zZhVaenR3Zh4QmTJlpk3NixMmNrxU8HDXGBMp4AtDcGjnDemd1AkhfrVu5bBixbJDRw6p7fFOQ0QLOHKEl4H0BEzHiGBR4+atm+pAlGT6+ef51EK5DYODA4ASXuS4npGEky8GUgL5RsQwVqkaaXbkjEqYRONdx45Eqrtb+YgAPTAHBgbi/KqLAY3jXbxoAYLwkT0cZiZNL+eGQLux+5GGObpNtR6ZLjIr6t/kSePpkPEZM+b7H7hzr58uCp3+8kXNQUWJAMBfrHiUmDi+mFs66IIZESaJbwx48dcypGC+adCDwm5ICksc/uhRI7AXN4/e/MFgzf2FIRkz628K0KkUNTO0pxN+GING/mgBTX9xeUAZ5khw/xehJ8GdEllQLBDAswC3LvTt+BxfZHhGYx5oMqAERrvx0zO6veBdyxGzVhavwFScqINtAUiRMuq74fyyx1cy2NZGvKGbNtGnlulsSHCMhzXMNOqSoPpu5/gNXtK6Z+vYH7kLhwlOKMB5kWBAmUB8fwMEZrOSwHcqaZvwIY7ErzwKzsK9enajekOwQUzznImHMvWmTZeOCDyIXVzdeQicQn4YOwab+MJuE9XO9TYiTps4YYnDw33uHC8sW61DxBq1zFl0j2adPmnoYAiIDhEJ3GgsfpFRl+gaNXQRy/PnekHhhFIGLDxRbxGDcwmce1gMhXZq9HcjwTB8hF6YI2b8wnkTrr60CccLbgcBwOfOmdm7tytedfXr2zWImvJH5fzANBYGrwsY+1jswKl88eIvSKvoiunFYN+8Ja0fDuAAn2nnHt35uLJl02mD1q9bi99GjZvg97PPP+feL774HE4sULTAQKamwYQbMvGgVomav2r7Vu+161ajC2UWSn4dqR+1b9EK1wbacVwcBEAz4O3et48bxdYNGjgEMfDUjoPFmpG+cuPGddSC3xw5c0ItZ/2NHzkwR072FVsUkQScuvBkGPP9d8iQiU1AvXjpcvWqY50QMlmzhRR7nzB+HH2D9e7dhz+NaELGGRczHhfjfxyLdmQn5yCG1AZLX2aDED9/nheyT8HbydGxHU2CX2vuL7DhQwv3Jo1iDS5WCAsyzhoyMuDbhuckIhYPKM0MCXBTCo4mwJMiS4oZApoSoWYGz1Dy2UfHFhBw2MPdhXqPHg1TU85ENwTtKNeMQjzEgIf4V18VhSYfcglkHby50Y6nJELQ8VCGrIAaXixDoHwVBX7jVYHKCXh849185vQpsr7jLTto0FAO1UZyF9bem1wM/CTMhLibHHIsJLhnT/2LrV27DkWKFEEcO+eitW/eYuSoMaq7D57UXbt2xCJxRPhAL1a8BPwxoB2hyFs0ennNVVOS0E5r16xOZqnevftB8gMmiPxSnTohcg0a2A/MeKA3a96CeGA+mD7DS31VwIgwfvw4sMGiBzMEKlRjj7t99/JXL+0Oi3Ry/IbkQghtcLA4f+43SmqH6mAmU+BAFOvZvQvJeXg947V9/fr1s2dPUxQY9jJ46NAmTZrzC4l2FNPfdxK9hRdVs8Z2tNToFoAFHzgUEF2vyXa8/ChnI16Nrds6XrlyGfCCk2Ow0V7d1nbTpg2AdOmyVXQNQzQnyR55yVGQDmd28mRP1ecd1aac2raGBIMrHLYzJCtHKY+zv54lOw4UG3PnL2IpgRbGpYIh/XTq3BmuweeQ6vvMabr4YQAdMWoM1kDMULEMHjzA5BFRI67qIUOHm2Ew7sJpatWqBV2xuJ2r16j58OH9tavXUAvWPGPWXNUShBl4GQhOLFGyZL68+W7euhV2PITWjPtozLjxmosH2a3I+alNG0fcQVAIaSrR4hMO+kVABxx6ubjCcQdQ4ywsW7kmm2KDw94t3l90jDt3bvtupB4K6FnTpU+7188PMhn0posWL9fcRDQkRg8oGpLAf0XoSeAnSJZnGYEB/XtThhv6QMQA+vbFJhE8BaIqLFq7WOiBjmH097r3q5V/qAY1bcpPJOLQkE6duqCu8pYtm7isN2Sg2XPmF4ta6GfjhnVzvGbT85T3hWe6i4s7J9hAO17ktWvVoCcgs4HgY8Tkvn7+LE6pPOZpeF1M/HGsRpzC89e5Zy/1E5wngQERL0j+WOR2PNndPPpqXgbUC9/nvn30Hj9o6dXLtZeLu+YdACveypXLeDa8CQYhjP+TT7gFBCyGcOHkPCh4j6JEgMZoQvxwkZ4w8Qe2IKARb4uhw0doahqok0MpiO9yzuNMXRiF7E2ubh7saaQOiSn9ToQeODk1bWzHFznWoF4V1A4THrIlxXR5Py+YN29epEs44P1u1PfIyclvPkyIywwvSE46gNyb/Xq7swTWr99AJLzW7BcaINwaahwTzePQqlWPHi6aU0xjYWKeNElbYEun4XBq16ZtO/Uih7WLdD98+2MGvilAUyE2mtb6X+iNpk79CapWdQjkrR49XZADU10AM6DW2+JFP7OzMLVjSPsOncn/jDmJUOVstOCOmzDhJ02hCUhFY78fzYeD0zp23PicOXNpprLm/qIh69etnjljBk+IRty2AwcPg6edZk7etPIBxfwJnBChJ4GfIFneh0YAuhaobd68eQsPaFXDYeU68Kw8ffokSlWUKlWGnyOwOJw8Ff782XPYy7hRnRACDb7hbt++9WmaNDC15PzyS5NPVXXIu6WhPEApRKhGoFb5/PPPCuQvqHn4Gu8OtpWzZ888e/oUH4hZsmZFFjiTX4o8EO/702dOoTZG0aJF2TDBvUTAEBlyNOi/f/+DAzLrtzQ8OEH4Jn748EH2rNmKl/xa1QNpOLGJ98r161dRcDRPnnyQNa05ofi8vnz5ypMnjzGkcMFCFnEw3qmZlnci9JiZP+5dECthqbx7506unDlVjR1SJODCfv3qH5sqVTUpf2CagW7y7r07JUuUMimA0qogUyLR+Iu//vr8iy8gGeOCMX+R40QjXR4Cvt68eVOwYKH8+QtYE4YWdwTUGXC+cFO8fPUyzSdpIHKp9iyVTaWhs7l3784/r/7JlDlLTt2fVkBRmXVPhvAwXPZIBA85UvMZQJwwVuKzBA5MhQvi2VBEHa7S1txfxI/dXb504cbNm3jaFCtWIrobTZ083h9Q6mLiSIvQE0cAZbggIAgIAtYikPCFHmuPRPgEgY8TgeQf57Jl1YKAICAIfHwIQG8EJ1kUY/r4li4rFgQSBQKi6UkUp1EOQhAQBAQBQUAQEAQsISCaHksISb8gIAgIAoKAICAIJAoEROhJFKdRDkIQEAQEAUFAEBAELCEgQo8lhKRfEBAEBAFBQBAQBBIFAiL0JIrTKAchCAgCgoAgIAgIApYQEKHHEkLSLwgIAoKAICAICAKJAgERehLFaZSDEAQEAUFAEBAEBAFLCIjQYwkh6RcEBAFBQBAQBASBRIGACD2J4jTKQQgCgoAgIAgIAoKAJQRE6LGEkPQLAoKAICAICAKCQKJAQISeRHEa5SAEAUFAEBAEBAFBwBICIvRYQkj6BQFBQBAQBAQBQSBRICBCT6I4jXIQgoAgIAgIAoKAIGAJARF6LCEk/YKAICAICAKCgCCQKBAQoSdRnEY5CEFAEBAEBAFBQBCwhIAIPZYQkn5BQBAQBAQBQUAQSBQIiNCTKE6jHIQgIAgIAoKAICAIWEJAhB5LCEm/ICAICAKCgCAgCCQKBEToSRSnUQ5CEBAEBAFBQBAQBCwhIEKPJYSkXxAQBAQBQUAQEAQSBQIi9CSK0ygHIQgIAoKAICAICAKWEBChxxJC0i8ICAKCgCAgCAgCiQIBEXoSxWmUgxAEBAFBQBAQBAQBSwiI0GMJIekXBAQBQUAQEAQEgUSBgAg9ieI0ykEIAoKAICAICAKCgCUEROixhJD0CwKCgCAgCAgCgkCiQECEnkRxGuUgBAFBQBAQBAQBQcASAiL0WEJI+gUBQUAQEAQEAUEgUSAgQk+iOI1yEIKAICAICAKCgCBgCQEReiwhJP2CgCAgCAgCgoAgkCgQEKEnUZxGOQhBQBAQBAQBQUAQsITA/wE+jO2rvW84UgAAAABJRU5ErkJggg==)\n","metadata":{"id":"pme2Qo8FAddC"}},{"cell_type":"markdown","source":"### Build confusion matrix to assess Precision","metadata":{"id":"LxIoVKpAmnOs"}},{"cell_type":"code","source":"## Function to create confusion matrices\ndef make_confusion_matrix_1 (model, y_actual, labels=[1, 0]):\n \"\"\"\n model : classifier to predict values of X\n y_actual : ground truth\n\n \"\"\"\n y_predict = model.predict(X1_test)\n cm = metrics.confusion_matrix(y_actual, y_predict, labels=[0, 1])\n df_cm = pd.DataFrame(\n cm,\n index=[i for i in [\"Actual - No\", \"Actual - Yes\"]],\n columns=[i for i in [\"Predicted - No\", \"Predicted - Yes\"]],\n )\n group_counts = [\"{0:0.0f}\".format(value) for value in cm.flatten()]\n group_percentages = [\"{0:.2%}\".format(value) for value in cm.flatten() / np.sum(cm)]\n labels = [f\"{v1}\\n{v2}\" for v1, v2 in zip(group_counts, group_percentages)]\n labels = np.asarray(labels).reshape(2, 2)\n plt.figure(figsize=(10, 7))\n sns.heatmap(df_cm, annot=labels, fmt=\"\")\n plt.ylabel(\"True label\")\n plt.xlabel(\"Predicted label\")\n\n\n","metadata":{"id":"f5mvr6lB2GYc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def make_confusion_matrix_2 (model, y_actual, labels=[1, 0]):\n \"\"\"\n model : classifier to predict values of X\n y_actual : ground truth\n\n \"\"\"\n y_predict = model.predict(X2_test)\n cm = metrics.confusion_matrix(y_actual, y_predict, labels=[0, 1])\n df_cm = pd.DataFrame(\n cm,\n index=[i for i in [\"Actual - No\", \"Actual - Yes\"]],\n columns=[i for i in [\"Predicted - No\", \"Predicted - Yes\"]],\n )\n group_counts = [\"{0:0.0f}\".format(value) for value in cm.flatten()]\n group_percentages = [\"{0:.2%}\".format(value) for value in cm.flatten() / np.sum(cm)]\n labels = [f\"{v1}\\n{v2}\" for v1, v2 in zip(group_counts, group_percentages)]\n labels = np.asarray(labels).reshape(2, 2)\n plt.figure(figsize=(10, 7))\n sns.heatmap(df_cm, annot=labels, fmt=\"\")\n plt.ylabel(\"True label\")\n plt.xlabel(\"Predicted label\")","metadata":{"id":"OFf9ZnV8kn7N","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def make_confusion_matrix_3 (model, y_actual, labels=[1, 0]):\n \"\"\"\n model : classifier to predict values of X\n y_actual : ground truth\n\n \"\"\"\n y_predict = model.predict(X3_test)\n cm = metrics.confusion_matrix(y_actual, y_predict, labels=[0, 1])\n df_cm = pd.DataFrame(\n cm,\n index=[i for i in [\"Actual - No\", \"Actual - Yes\"]],\n columns=[i for i in [\"Predicted - No\", \"Predicted - Yes\"]],\n )\n group_counts = [\"{0:0.0f}\".format(value) for value in cm.flatten()]\n group_percentages = [\"{0:.2%}\".format(value) for value in cm.flatten() / np.sum(cm)]\n labels = [f\"{v1}\\n{v2}\" for v1, v2 in zip(group_counts, group_percentages)]\n labels = np.asarray(labels).reshape(2, 2)\n plt.figure(figsize=(10, 7))\n sns.heatmap(df_cm, annot=labels, fmt=\"\")\n plt.ylabel(\"True label\")\n plt.xlabel(\"Predicted label\")","metadata":{"id":"-cO7LauWkz3w","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Random Forest Modeling","metadata":{"id":"lcF0ehmMjpm9"}},{"cell_type":"markdown","source":"### Under Sampling and Balanced Model Training","metadata":{"id":"hDuIaBJlqHBh"}},{"cell_type":"code","source":"#StandardScaler(),\n\nRF = make_pipeline (RandomForestClassifier(\n random_state=99,),\n)\n\n\n# Fit the model on training data - Under Sampling\n#RF1_under = RF.fit (X1_train_under, y1_train_under)\n#RF2_under = RF.fit (X2_train_under, y2_train_under)\n#RF3_under = RF.fit (X3_train_under, y3_train_under)\n\n#Fit the model on training data - Over Sampling\n\n#RF1_over = RF.fit (X1_train_over, y1_train_over)\n#RF2_over = RF.fit (X2_train_over, y2_train_over)\n#RF3_over = RF.fit (X3_train_over, y3_train_over)\n\n#Fit the model on training data - Balanced\n\nRF1_b = RF.fit (X1_train_b, y1_train_b)\nRF2_b = RF.fit (X2_train_b, y2_train_b)\nRF3_b = RF.fit (X3_train_b, y3_train_b)\n\n","metadata":{"id":"oSZ22IZsiHiU","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#pred_train_1_under = RF1_under.predict(X1_train_under)\n#pred_test_1_under = RF1_under.predict(X1_test)\n\n#train_acc_1_under = RF1_under.score(X1_train_under, y1_train_under)\n#test_acc_1_under = RF1_under.score(X1_test, y1_test)\n\n#train_recall_1_under = metrics.recall_score(y1_train_under, pred_train_1_under)\n#test_recall_1_under = metrics.recall_score(y1_test, pred_test_1_under)\n\n#train_precision_1_under = metrics.precision_score(y1_train_under, pred_train_1_under)\n#test_precision_1_under = metrics.precision_score(y1_test, pred_test_1_under)\n\n\n#pred_train_2_under = RF2_under.predict(X2_train_under)\n#pred_test_2_under = RF2_under.predict(X2_test)\n\n#train_acc_2_under = RF2_under.score(X2_train_under, y2_train_under)\n#test_acc_2_under = RF2_under.score(X2_test, y2_test)\n\n#train_recall_2_under= metrics.recall_score(y2_train_under, pred_train_2_under)\n#test_recall_2_under = metrics.recall_score(y2_test, pred_test_2_under)\n\n#train_precision_2_under = metrics.precision_score(y2_train_under, pred_train_2_under)\n#test_precision_2_under = metrics.precision_score(y2_test, pred_test_2_under)\n\n#pred_train_3_under = RF3_under.predict(X3_train_under)\n#pred_test_3_under = RF3_under.predict(X3_test)\n\n#train_acc_3_under = RF3_under.score(X3_train_under, y3_train_under)\n#test_acc_3_under = RF3_under.score(X3_test, y3_test)\n\n#train_recall_3_under = metrics.recall_score(y3_train_under, pred_train_3_under)\n#test_recall_3_under = metrics.recall_score(y3_test, pred_test_3_under)\n\n#train_precision_3_under = metrics.precision_score(y3_train_under, pred_train_3_under)\n#test_precision_3_under = metrics.precision_score(y3_test, pred_test_3_under)","metadata":{"id":"kDKrM3I5i0Ln","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"##pred_train_1_over = RF1_over.predict(X1_train_over)\n##pred_test_1_over = RF1_over.predict(X1_test)\n\n##train_acc_1_over = RF1_over.score(X1_train_over, y1_train_over)\n##test_acc_1_over = RF1_over.score(X1_test, y1_test)\n\n##train_recall_1_over = metrics.recall_score(y1_train_over, pred_train_1_over)\n##test_recall_1_over = metrics.recall_score(y1_test, pred_test_1_over)\n\n##train_precision_1_over = metrics.precision_score(y1_train_over, pred_train_1_over)\n##test_precision_1_over = metrics.precision_score(y1_test, pred_test_1_over)\n\n\n##pred_train_2_over = RF2_over.predict(X2_train_over)\n##pred_test_2_over = RF2_over.predict(X2_test)\n\n##train_acc_2_over = RF2_over.score(X2_train_over, y2_train_over)\n##test_acc_2_over = RF2_over.score(X2_test, y2_test)\n\n##train_recall_2_over= metrics.recall_score(y2_train_over, pred_train_2_over)\n##test_recall_2_over = metrics.recall_score(y2_test, pred_test_2_over)\n\n##train_precision_2_over = metrics.precision_score(y2_train_over, pred_train_2_over)\n##test_precision_2_over = metrics.precision_score(y2_test, pred_test_2_over)\n\n##pred_train_3_over = RF3_over.predict(X3_train_over)\n##pred_test_3_over = RF3_over.predict(X3_test)\n\n##train_acc_3_over = RF3_over.score(X3_train_over, y3_train_over)\n##test_acc_3_over = RF3_over.score(X3_test, y3_test)\n\n##train_recall_3_over = metrics.recall_score(y3_train_over, pred_train_3_over)\n##test_recall_3_over = metrics.recall_score(y3_test, pred_test_3_over)\n\n##train_precision_3_over = metrics.precision_score(y3_train_over, pred_train_3_over)\n##test_precision_3_over = metrics.precision_score(y3_test, pred_test_3_over)","metadata":{"id":"PsGDTpotz0Qa","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_train_1_b = RF1_b.predict(X1_train_b)\npred_test_1_b = RF1_b.predict(X1_test_b)\n\ntrain_acc_1_b = RF1_b.score(X1_train_b, y1_train_b)\ntest_acc_1_b = RF1_b.score(X1_test_b, y1_test_b)\n\ntrain_recall_1_b = metrics.recall_score(y1_train_b, pred_train_1_b)\ntest_recall_1_b = metrics.recall_score(y1_test_b, pred_test_1_b)\n\ntrain_precision_1_b = metrics.precision_score(y1_train_b, pred_train_1_b)\ntest_precision_1_b = metrics.precision_score(y1_test_b, pred_test_1_b)\n\n\npred_train_2_b = RF2_b.predict(X2_train_b)\npred_test_2_b = RF2_b.predict(X2_test_b)\n\ntrain_acc_2_b = RF2_b.score(X2_train_b, y2_train_b)\ntest_acc_2_b = RF2_b.score(X2_test_b, y2_test_b)\n\ntrain_recall_2_b= metrics.recall_score(y2_train_b, pred_train_2_b)\ntest_recall_2_b = metrics.recall_score(y2_test_b, pred_test_2_b)\n\ntrain_precision_2_b = metrics.precision_score(y2_train_b, pred_train_2_b)\ntest_precision_2_b = metrics.precision_score(y2_test_b, pred_test_2_b)\n\npred_train_3_b = RF3_b.predict(X3_train_b)\npred_test_3_b = RF3_b.predict(X3_test_b)\n\ntrain_acc_3_b = RF3_b.score(X3_train_b, y3_train_b)\ntest_acc_3_b = RF3_b.score(X3_test_b, y3_test_b)\n\ntrain_recall_3_b = metrics.recall_score(y3_train_b, pred_train_3_b)\ntest_recall_3_b = metrics.recall_score(y3_test_b, pred_test_3_b)\n\ntrain_precision_3_b = metrics.precision_score(y3_train_b, pred_train_3_b)\ntest_precision_3_b = metrics.precision_score(y3_test_b, pred_test_3_b)","metadata":{"id":"KscnmwCqbDYq","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### RF1 Results - UNDER\n","metadata":{"id":"2WM6NvH0knjZ"}},{"cell_type":"code","source":" #print(\"Accuracy on training set : \", RF1_under.score(X1_train_under,y1_train_under))\n #print(\"Accuracy on test set : \", RF1_under.score(X1_test, y1_test))\n #print(\"Recall on training set : \", metrics.recall_score(y1_train_under, pred_train_1_under))\n #print(\"Recall on test set : \", metrics.recall_score(y1_test, pred_test_1_under))\n #print(\"Precision on training set : \", metrics.precision_score(y1_train_under, pred_train_1_under))\n #print(\"Precision on test set : \", metrics.precision_score(y1_test, pred_test_1_under))","metadata":{"id":"cuPa-D1Vj_AH","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### RF2 Results - UNDER","metadata":{"id":"vRxAvOzak2Dx"}},{"cell_type":"code","source":"#print(\"Accuracy on training set : \", RF2_under.score(X2_train_under,y2_train_under))\n#print(\"Accuracy on test set : \", RF2_under.score(X2_test, y2_test))\n#print(\"Recall on training set : \", metrics.recall_score(y2_train_under, pred_train_2_under))\n#print(\"Recall on test set : \", metrics.recall_score(y2_test, pred_test_2_under))\n#print(\"Precision on training set : \", metrics.precision_score(y2_train_under, pred_train_2_under))\n#print(\"Precision on test set : \", metrics.precision_score(y2_test, pred_test_2_under))","metadata":{"id":"rkIdgtAuku1d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### RF3 Results - UNDER","metadata":{"id":"co3YoE9MlPsS"}},{"cell_type":"code","source":"#print(\"Accuracy on training set : \", RF3_under.score(X3_train_under,y3_train_under))\n#print(\"Accuracy on test set : \", RF3_under.score(X3_test, y3_test))\n#print(\"Recall on training set : \", metrics.recall_score(y3_train_under, pred_train_3_under))\n#print(\"Recall on test set : \", metrics.recall_score(y3_test, pred_test_3_under))\n#print(\"Precision on training set : \", metrics.precision_score(y3_train_under, pred_train_3_under))\n#print(\"Precision on test set : \", metrics.precision_score(y3_test, pred_test_3_under))","metadata":{"id":"QcE_C-7vlBJ9","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### RF1 Results - Balanced","metadata":{"id":"nPsGXA7YdiXC"}},{"cell_type":"code","source":" print(\"Accuracy on training set : \", RF1_b.score(X1_train_b,y1_train_b))\n print(\"Accuracy on test set : \", RF1_b.score(X1_test_b, y1_test_b))\n print(\"Recall on training set : \", metrics.recall_score(y1_train_b, pred_train_1_b))\n print(\"Recall on test set : \", metrics.recall_score(y1_test_b, pred_test_1_b))\n print(\"Precision on training set : \", metrics.precision_score(y1_train_b, pred_train_1_b))\n print(\"Precision on test set : \", metrics.precision_score(y1_test_b, pred_test_1_b))","metadata":{"id":"e4-4o4N0dcwg","outputId":"852cda9c-05b7-4e62-fd1c-d3cf10942fe9","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### RF2 Results - Balanced","metadata":{"id":"jfsyoHpkdo_A"}},{"cell_type":"code","source":"print(\"Accuracy on training set : \", RF2_b.score(X2_train_b,y2_train_b))\nprint(\"Accuracy on test set : \", RF2_b.score(X2_test_b, y2_test_b))\nprint(\"Recall on training set : \", metrics.recall_score(y2_train_b, pred_train_2_b))\nprint(\"Recall on test set : \", metrics.recall_score(y2_test_b, pred_test_2_b))\nprint(\"Precision on training set : \", metrics.precision_score(y2_train_b, pred_train_2_b))\nprint(\"Precision on test set : \", metrics.precision_score(y2_test_b, pred_test_2_b))","metadata":{"id":"SE4LfT-Iddgb","outputId":"49f462b5-ae79-496f-9878-bc1468e9f5c1","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### RF3 Results - Balanced","metadata":{"id":"nTVcPl3QdtFD"}},{"cell_type":"code","source":"print(\"Accuracy on training set : \", RF3_b.score(X3_train_b,y3_train_b))\nprint(\"Accuracy on test set : \", RF3_b.score(X3_test_b, y3_test_b))\nprint(\"Recall on training set : \", metrics.recall_score(y3_train_b, pred_train_3_b))\nprint(\"Recall on test set : \", metrics.recall_score(y3_test_b, pred_test_3_b))\nprint(\"Precision on training set : \", metrics.precision_score(y3_train_b, pred_train_3_b))\nprint(\"Precision on test set : \", metrics.precision_score(y3_test_b, pred_test_3_b))","metadata":{"id":"3moQqaxZddq3","outputId":"4f618f98-377a-4e46-a6b2-285d1ffac7cb","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### The precision scores for the balanced data set are superior to the Under/Over Sampled using SMOTE. Use the Balanced Data set for the remainder of the model build.","metadata":{"id":"3Ebbt5zsBZTs"}},{"cell_type":"code","source":"# Creating new pipeline with best parameters\n# (\"scaler\", StandardScaler()),\n\n\n#rf_tuned = Pipeline(\n# steps=[\n# \n# (\n# \"RF\",\n# RandomForestClassifier(\n# \n# n_estimators=10,\n# min_samples_leaf = 6,\n# max_features = 'log2',\n# max_samples = None,\n# random_state=99\n# \n# ),\n# ),\n# ]\n#)\n\n","metadata":{"id":"RLVX2IWNDth_","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fit the model on training data\n#RF1 = rf_tuned.fit(X1_train_b, y1_train_b)\n","metadata":{"id":"JYox0F6toZag","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"make_confusion_matrix_1 (RF1_b,y1_test)","metadata":{"id":"HhGWViEpoqU6","outputId":"d42b0a47-d1ca-4776-cb84-df82e2ebeb07","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fit the model on training data\n#RF2 = rf_tuned.fit(X2_train_b, y2_train_b)","metadata":{"id":"WNcJBIrfpTh-","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"make_confusion_matrix_2 (RF2_b, y2_test)","metadata":{"id":"zT9_ctsMpuZ5","outputId":"4dafebac-b614-4433-9dc3-7eb05abdc541","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fit the model on training dataR\n#RF3 = rf_tuned.fit(X3_train_b, y3_train_b)","metadata":{"id":"TKNxTmBTp0jj","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"make_confusion_matrix_3 (RF3_b, y3_test)","metadata":{"id":"OA_GnVxdp86Y","outputId":"ba11b257-4a5e-4c5c-a9dd-d0a97c3c7e2c","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Build Test Data Set","metadata":{"id":"VQdmVJSQrqwG"}},{"cell_type":"code","source":"# Set the directory path to the folder containing the CSV files.\ntdcsfog_test_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog'\n\n# Initialize an empty list to store the dataframes.\ntdcsfog_test_list = []\n\n# Loop through each file in the directory and read it into a dataframe.\nfor file_name in os.listdir(tdcsfog_test_path):\n if file_name.endswith('.csv'):\n file_path = os.path.join(tdcsfog_test_path, file_name)\n file = pd.read_csv(file_path)\n file['Id'] = file_name[:-4] + '_' + file['Time'].apply(str)\n # file.Time = file.Time / (len(file) - 1)\n tdcsfog_test_list.append(file)\n\n# Concatenate the dataframes vertically using pd.concat().\ntdcsfog_test = pd.concat(tdcsfog_test_list, axis = 0)\n\n# Show the concatenated dataframe.\ntdcsfog_test","metadata":{"id":"6mKZq1djrUUC","outputId":"b35f1d01-479b-4c14-8212-0aade9dc283a","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog_test = reduce_memory_usage(tdcsfog_test)","metadata":{"id":"gHCqQstwsXmA","outputId":"5cc3edfd-6f87-443e-e1a5-6194fe349566","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set the directory path to the folder containing the CSV files.\ndefog_test_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/defog'\n\n# Initialize an empty list to store the dataframes.\ndefog_test_list = []\n\n# Loop through each file in the directory and read it into a dataframe.\nfor file_name in os.listdir(defog_test_path):\n if file_name.endswith('.csv'):\n file_path = os.path.join(defog_test_path, file_name)\n file = pd.read_csv(file_path)\n file['Id'] = file_name[:-4] + '_' + file['Time'].apply(str)\n # file.Time = file.Time / (len(file) - 1)\n defog_test_list.append(file)\n\n# Concatenate the dataframes vertically using pd.concat().\ndefog_test = pd.concat(defog_test_list, axis = 0)\n\n# Show the concatenated dataframe.\ndefog_test","metadata":{"id":"7j34UF_Dsd5_","outputId":"c37f8d1f-7f84-47d8-d49a-28d1a16a18a4","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog_test = reduce_memory_usage(defog_test)","metadata":{"id":"ozgQP7Lcsnw-","outputId":"e3d0ccd4-dcea-4eaf-a7ae-15018b921b49","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test = pd.concat([tdcsfog_test, defog_test], axis = 0).reset_index(drop = True)\ntest.info()\ntest.head(10)","metadata":{"id":"p1UUFzTPssa8","outputId":"a076c0df-29fd-4282-b006-e3fe01665608","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Inference","metadata":{"id":"3MNsbzkss3JA"}},{"cell_type":"code","source":"\n\n# Separate the dataset for the independent variables.\ntest_X = test.iloc[:, 0:4]\n\n# Standardize the independent variables by a new scaler.\n#scaler = StandardScaler()\n#scaler.fit(test_X)\n#test_X = scaler.transform(test_X)\n\n# Get the predictions for the three models on the test data.\npred_y1 = RF1_b.predict(test_X)\npred_y2 = RF2_b.predict(test_X)\npred_y3 = RF3_b.predict(test_X)\n\ntest['StartHesitation'] = pred_y1 # target variable for StartHesitation\ntest['Turn'] = pred_y2 # target variable for Turn\ntest['Walking'] = pred_y3 # target variable for Walking\n\n# Get the probability predictions for the three models on the test data.\npred_proba_y1 = RF1_b.predict_proba(test_X)[:, 1]\npred_proba_y2 = RF2_b.predict_proba(test_X)[:, 1]\npred_proba_y3 = RF3_b.predict_proba(test_X)[:, 1]\n\n# Update the values in the test dataframe.\ntest['StartHesitation'] = pred_proba_y1\ntest['Turn'] = pred_proba_y2\ntest['Walking'] = pred_proba_y3\n\ntest","metadata":{"id":"4UX27Yxos0po","outputId":"804bf0e1-680f-4912-9d60-1fceafdf2ea4","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### SUBMISSION","metadata":{"id":"eacfC31wtXDx"}},{"cell_type":"code","source":"submission = test.iloc[:, 4:].fillna(0.0)\nsubmission","metadata":{"id":"bfgcuFAftMhW","outputId":"7bf50295-6c00-4619-af2d-1a93232a02f0","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.head(20)","metadata":{"id":"VpDM5lnltaEC","outputId":"5c5b80bd-bdf3-4eb7-c21a-5fd93343b895","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.tail(20)","metadata":{"id":"m_xF3yMstlTB","outputId":"0e452503-41b7-4ec3-93a9-9de5f84f7810","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv('/kaggle/working/submission.csv', index = False)","metadata":{"id":"YcpP7uGhtoK7","trusted":true},"execution_count":null,"outputs":[]}]}