{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Parkinson's Freezing of Gait Prediction**","metadata":{}},{"cell_type":"markdown","source":"🧠Parkinson's Disease: is a brain disorder that causes unintended or uncontrollable movements (shaking, stiffness, and difficulty with balance and coordination).\n\n🧠Freezing Of Gate: is defined as a brief, episodic absence or marked reduction of forward progression of the feet despite the intention to walk. It is one of the most debilitating motor symptoms in patients with Parkinson's disease as it may lead to falls and a loss of independence.\n\n🧠Detecting FOG: A way to detect FOG is by using sensors while doing FOG-provoking testing. Another is by documenting the moment FOG appared within a patient by watching a video, frame by frame. While the ML trained on this data has good accuracy, the datasets are farily small and the focus has been on recall, while precission has been ignored.*","metadata":{}},{"cell_type":"markdown","source":"Dataset Description\nThis competition dataset comprises lower-back 3D accelerometer data from subjects exhibiting freezing of gait episodes, a disabling symptom that is common among people with Parkinson's disease. Freezing of gait (FOG) negatively impacts walking abilities and impinges locomotion and independence.\n\nOur objective is to detect the start and stop of each freezing episode and the occurrence in these series of three types of freezing of gait events: Start Hesitation, Turn, and Walking.","metadata":{}},{"cell_type":"markdown","source":"
\n\n

💡 Notes:

\n\n* tdcsfog dataset are collected from the lab. Therefore, data quality is good comparing with defog dataset.\n* defog dataset are collected from subjects' home, therefore there are two additioanl columns (valid & test) to check the quality of data.\n* On evaluation precess, it only uses validly annotated data. Therefore, it's reasonable to remove invalid data from defog dataset.\n* However, as the document mentioned, we can use it for developing semi or unsupervised model (with notype dataset)\n* As the size of dataset is quite huge, we should convert data type to reduce memory usage.\n* Need to build two different model. one for tdcsfog and one for defog. Because, via EDA, you could find that both datasets' distribution is different.\n* This is just an initial plan for modeling. It could be changed as we dig more into the dataset.","metadata":{}},{"cell_type":"code","source":"# import library\nimport os\nimport random\nimport cv2\nimport pandas as pd\nimport numpy as np\nfrom sklearn.metrics import precision_recall_curve, plot_precision_recall_curve, plot_roc_curve, roc_auc_score, roc_curve, average_precision_score\nfrom sklearn.metrics import accuracy_score, f1_score, recall_score, precision_score,roc_auc_score, make_scorer\nfrom sklearn.metrics import classification_report, confusion_matrix, plot_confusion_matrix, average_precision_score","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:12:58.328626Z","iopub.execute_input":"2023-07-07T04:12:58.328977Z","iopub.status.idle":"2023-07-07T04:12:58.572812Z","shell.execute_reply.started":"2023-07-07T04:12:58.328946Z","shell.execute_reply":"2023-07-07T04:12:58.571402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Reduce Memory Usage\n# reference : https://www.kaggle.com/code/arjanso/reducing-dataframe-memory-size-by-65 @ARJANGROEN\n\ndef reduce_memory_usage(df):\n \n start_mem = df.memory_usage().sum() / 1024**2\n print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n \n for col in df.columns:\n col_type = df[col].dtype.name\n if ((col_type != 'datetime64[ns]') & (col_type != 'category')):\n if (col_type != 'object'):\n c_min = df[col].min()\n c_max = df[col].max()\n\n if str(col_type)[:3] == 'int':\n if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n df[col] = df[col].astype(np.int8)\n elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n df[col] = df[col].astype(np.int16)\n elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n df[col] = df[col].astype(np.int32)\n elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n df[col] = df[col].astype(np.int64)\n\n else:\n if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n df[col] = df[col].astype(np.float16)\n elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n df[col] = df[col].astype(np.float32)\n else:\n pass\n else:\n df[col] = df[col].astype('category')\n mem_usg = df.memory_usage().sum() / 1024**2 \n print(\"Memory usage became: \",mem_usg,\" MB\")\n \n return df","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:13:05.070319Z","iopub.execute_input":"2023-07-07T04:13:05.071546Z","iopub.status.idle":"2023-07-07T04:13:05.089039Z","shell.execute_reply.started":"2023-07-07T04:13:05.071499Z","shell.execute_reply":"2023-07-07T04:13:05.087632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#reference: https://www.kaggle.com/code/ghrangel/read-data-and-merge\n\nDATA_ROOT_DEFOG = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/defog/'\ndefog = pd.DataFrame()\nfor root, dirs, files in os.walk(DATA_ROOT_DEFOG):\n for name in files: \n f = os.path.join(root, name)\n df_list= pd.read_csv(f)\n words = name.split('.')[0]\n df_list['file']= name.split('.')[0]\n defog = pd.concat([defog, df_list], axis=0)\n\ndefog\n ","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:13:24.221106Z","iopub.execute_input":"2023-07-07T04:13:24.221860Z","iopub.status.idle":"2023-07-07T04:14:21.027547Z","shell.execute_reply.started":"2023-07-07T04:13:24.221814Z","shell.execute_reply":"2023-07-07T04:14:21.026274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog.shape","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:16:22.902734Z","iopub.execute_input":"2023-07-07T04:16:22.903214Z","iopub.status.idle":"2023-07-07T04:16:22.916599Z","shell.execute_reply.started":"2023-07-07T04:16:22.903172Z","shell.execute_reply":"2023-07-07T04:16:22.915183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog.info()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:20:08.079953Z","iopub.execute_input":"2023-07-07T04:20:08.080289Z","iopub.status.idle":"2023-07-07T04:20:08.092655Z","shell.execute_reply.started":"2023-07-07T04:20:08.080249Z","shell.execute_reply":"2023-07-07T04:20:08.091395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog = reduce_memory_usage(defog)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:20:20.532812Z","iopub.execute_input":"2023-07-07T04:20:20.533255Z","iopub.status.idle":"2023-07-07T04:20:22.715022Z","shell.execute_reply.started":"2023-07-07T04:20:20.533197Z","shell.execute_reply":"2023-07-07T04:20:22.713587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 we reduced memory usage from 954MB to 335MB","metadata":{}},{"cell_type":"code","source":"defog = defog[(defog['Task']==1)&(defog['Valid']==1)]","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:20:29.352151Z","iopub.execute_input":"2023-07-07T04:20:29.352802Z","iopub.status.idle":"2023-07-07T04:20:29.739228Z","shell.execute_reply.started":"2023-07-07T04:20:29.352765Z","shell.execute_reply":"2023-07-07T04:20:29.738026Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 As I mentioned above, We are going to use valid data only.","metadata":{}},{"cell_type":"code","source":"print('the shape of defog dataset is {}'.format(defog.shape))","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:20:36.736591Z","iopub.execute_input":"2023-07-07T04:20:36.737250Z","iopub.status.idle":"2023-07-07T04:20:36.743170Z","shell.execute_reply.started":"2023-07-07T04:20:36.737197Z","shell.execute_reply":"2023-07-07T04:20:36.741998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog.info()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:23:09.896682Z","iopub.execute_input":"2023-07-07T04:23:09.897457Z","iopub.status.idle":"2023-07-07T04:23:09.914167Z","shell.execute_reply.started":"2023-07-07T04:23:09.897388Z","shell.execute_reply":"2023-07-07T04:23:09.912930Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 Now it's time to combine it with metadata.","metadata":{}},{"cell_type":"code","source":"defog_metadata = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/defog_metadata.csv\")\ndefog_metadata","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:23:31.402778Z","iopub.execute_input":"2023-07-07T04:23:31.403994Z","iopub.status.idle":"2023-07-07T04:23:31.427453Z","shell.execute_reply.started":"2023-07-07T04:23:31.403947Z","shell.execute_reply":"2023-07-07T04:23:31.426269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog_m= defog_metadata.merge(defog, how = 'inner', left_on = 'Id', right_on = 'file')\ndefog_m.drop(['file','Valid','Task'], axis = 1, inplace = True)\ndefog_m","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:24:08.522534Z","iopub.execute_input":"2023-07-07T04:24:08.522935Z","iopub.status.idle":"2023-07-07T04:24:10.170515Z","shell.execute_reply.started":"2023-07-07T04:24:08.522901Z","shell.execute_reply":"2023-07-07T04:24:10.168255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# summary table function\ndef summary(df):\n print(f'data shape: {df.shape}')\n summ = pd.DataFrame(df.dtypes, columns=['data type'])\n summ['#missing'] = df.isnull().sum().values * 100\n summ['%missing'] = df.isnull().sum().values / len(df)\n summ['#unique'] = df.nunique().values\n desc = pd.DataFrame(df.describe(include='all').transpose())\n summ['min'] = desc['min'].values\n summ['max'] = desc['max'].values\n summ['first value'] = df.loc[0].values\n summ['second value'] = df.loc[1].values\n summ['third value'] = df.loc[2].values\n \n return summ","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:24:37.096797Z","iopub.execute_input":"2023-07-07T04:24:37.097891Z","iopub.status.idle":"2023-07-07T04:24:37.108470Z","shell.execute_reply.started":"2023-07-07T04:24:37.097831Z","shell.execute_reply":"2023-07-07T04:24:37.107125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 Let's look at the summary table for defog dataset (data from subjects' home)","metadata":{}},{"cell_type":"code","source":"summary(defog_m)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:24:47.949088Z","iopub.execute_input":"2023-07-07T04:24:47.950139Z","iopub.status.idle":"2023-07-07T04:24:55.728135Z","shell.execute_reply.started":"2023-07-07T04:24:47.950097Z","shell.execute_reply":"2023-07-07T04:24:55.726779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# garbage collection for memory\nimport gc\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:25:09.061733Z","iopub.execute_input":"2023-07-07T04:25:09.062152Z","iopub.status.idle":"2023-07-07T04:25:09.939034Z","shell.execute_reply.started":"2023-07-07T04:25:09.062117Z","shell.execute_reply":"2023-07-07T04:25:09.937664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 prepare tdcsfog dataset for modeling (data collected from the lab🥼)","metadata":{}},{"cell_type":"code","source":"DATA_ROOT_TDCSFOG = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog/'\ntdcsfog = pd.DataFrame()\nfor root, dirs, files in os.walk(DATA_ROOT_TDCSFOG):\n for name in files: \n f = os.path.join(root, name)\n df_list= pd.read_csv(f)\n words = name.split('.')[0]\n df_list['file']= name.split('.')[0]\n tdcsfog = pd.concat([tdcsfog, df_list], axis=0)\ntdcsfog","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:26:37.121964Z","iopub.execute_input":"2023-07-07T04:26:37.122845Z","iopub.status.idle":"2023-07-07T04:28:16.915607Z","shell.execute_reply.started":"2023-07-07T04:26:37.122804Z","shell.execute_reply":"2023-07-07T04:28:16.914295Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog = reduce_memory_usage(tdcsfog)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:30:21.565322Z","iopub.execute_input":"2023-07-07T04:30:21.566520Z","iopub.status.idle":"2023-07-07T04:30:22.612930Z","shell.execute_reply.started":"2023-07-07T04:30:21.566467Z","shell.execute_reply":"2023-07-07T04:30:22.611646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 we reduced memory usage from 484MB to 154MB","metadata":{}},{"cell_type":"code","source":"tdcsfog_metadata = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/tdcsfog_metadata.csv\")\ntdcsfog_metadata","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:30:57.321725Z","iopub.execute_input":"2023-07-07T04:30:57.322187Z","iopub.status.idle":"2023-07-07T04:30:57.354714Z","shell.execute_reply.started":"2023-07-07T04:30:57.322147Z","shell.execute_reply":"2023-07-07T04:30:57.353680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog_m= tdcsfog_metadata.merge(tdcsfog, how = 'inner', left_on = 'Id', right_on = 'file')\ntdcsfog_m.drop(['file'], axis = 1, inplace = True)\ntdcsfog_m","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:31:03.932871Z","iopub.execute_input":"2023-07-07T04:31:03.934022Z","iopub.status.idle":"2023-07-07T04:31:07.138169Z","shell.execute_reply.started":"2023-07-07T04:31:03.933981Z","shell.execute_reply":"2023-07-07T04:31:07.135262Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# garbage collection for memory\nimport gc\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:31:59.453646Z","iopub.execute_input":"2023-07-07T04:31:59.454109Z","iopub.status.idle":"2023-07-07T04:31:59.646521Z","shell.execute_reply.started":"2023-07-07T04:31:59.454069Z","shell.execute_reply":"2023-07-07T04:31:59.645377Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

💡 Notes:

\n\n* Simple multi-classification model with LGBM developed.\n\n* This is time series data, a time variable can be created later to see the change over time.","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import KFold, StratifiedKFold, train_test_split, GridSearchCV\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:43:55.852180Z","iopub.execute_input":"2023-07-07T04:43:55.852612Z","iopub.status.idle":"2023-07-07T04:43:55.860532Z","shell.execute_reply.started":"2023-07-07T04:43:55.852573Z","shell.execute_reply":"2023-07-07T04:43:55.859323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conditions = [\n (defog_m['StartHesitation'] == 1),\n (defog_m['Turn'] == 1),\n (defog_m['Walking'] == 1)]\nchoices = ['StartHesitation', 'Turn', 'Walking']\ndefog_m['event'] = np.select(conditions, choices, default='Normal')","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:44:04.439715Z","iopub.execute_input":"2023-07-07T04:44:04.440725Z","iopub.status.idle":"2023-07-07T04:44:05.454440Z","shell.execute_reply.started":"2023-07-07T04:44:04.440668Z","shell.execute_reply":"2023-07-07T04:44:05.452914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog_m.sample()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:46:01.986424Z","iopub.execute_input":"2023-07-07T04:46:01.987201Z","iopub.status.idle":"2023-07-07T04:46:02.124454Z","shell.execute_reply.started":"2023-07-07T04:46:01.987146Z","shell.execute_reply":"2023-07-07T04:46:02.123252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog_m.sample()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:46:55.672220Z","iopub.execute_input":"2023-07-07T04:46:55.673460Z","iopub.status.idle":"2023-07-07T04:46:55.796902Z","shell.execute_reply.started":"2023-07-07T04:46:55.673411Z","shell.execute_reply":"2023-07-07T04:46:55.795703Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog_m.sample()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:47:35.196731Z","iopub.execute_input":"2023-07-07T04:47:35.197733Z","iopub.status.idle":"2023-07-07T04:47:35.322290Z","shell.execute_reply.started":"2023-07-07T04:47:35.197678Z","shell.execute_reply":"2023-07-07T04:47:35.321067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog_m['event'].value_counts().to_frame().style.background_gradient()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:36:06.732100Z","iopub.execute_input":"2023-07-07T04:36:06.732607Z","iopub.status.idle":"2023-07-07T04:36:07.543135Z","shell.execute_reply.started":"2023-07-07T04:36:06.732565Z","shell.execute_reply":"2023-07-07T04:36:07.541743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 Turn is the most frequently occured event while StartHesitation rarely occurs...","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"train_df = defog_m[['AccV','AccML','AccAP','event']]","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:49:59.217041Z","iopub.execute_input":"2023-07-07T04:49:59.217456Z","iopub.status.idle":"2023-07-07T04:49:59.298142Z","shell.execute_reply.started":"2023-07-07T04:49:59.217421Z","shell.execute_reply":"2023-07-07T04:49:59.296957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬just three sensor data used as inputs of the model. This model does not consider time-related effect.","metadata":{}},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nle = LabelEncoder()\n\ntrain_df['target'] = le.fit_transform(train_df['event'])","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:51:39.416460Z","iopub.execute_input":"2023-07-07T04:51:39.416859Z","iopub.status.idle":"2023-07-07T04:51:40.881190Z","shell.execute_reply.started":"2023-07-07T04:51:39.416825Z","shell.execute_reply":"2023-07-07T04:51:40.880096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X = train_df.drop(['event','target'], axis=1)\ny = train_df['target']","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:52:10.465298Z","iopub.execute_input":"2023-07-07T04:52:10.466406Z","iopub.status.idle":"2023-07-07T04:52:10.487345Z","shell.execute_reply.started":"2023-07-07T04:52:10.466344Z","shell.execute_reply":"2023-07-07T04:52:10.486284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 train simple LGBM model without hyper-parameter tuning. the size of dataset is quite huge, it might take a lot of time for traing a decent model.","metadata":{}},{"cell_type":"code","source":"import lightgbm as lgb\n\n\n# split dataset into training and test set\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1004)\n\n#Converting the dataset in proper LGB format\nd_train=lgb.Dataset(X_train, label=y_train)\n#setting up the parameters\nparams={}\nparams['learning_rate']=0.03\nparams['boosting_type']='gbdt' #GradientBoostingDecisionTree\nparams['objective']='multiclass' #Multi-class target feature\nparams['metric']='multi_logloss' #metric for multi-class\nparams['max_depth']=7\nparams['num_class']=4 #no.of unique values in the target class not inclusive of the end value\nparams['verbose']=-1\n#training the model\nclf=lgb.train(params,d_train,1000) #training the model on 1,000 epocs\n#prediction on the test dataset\ny_pred_1=clf.predict(X_test)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T04:52:15.028698Z","iopub.execute_input":"2023-07-07T04:52:15.029738Z","iopub.status.idle":"2023-07-07T05:05:31.393373Z","shell.execute_reply.started":"2023-07-07T04:52:15.029681Z","shell.execute_reply":"2023-07-07T05:05:31.392311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 Let's look at what it predicts","metadata":{}},{"cell_type":"code","source":"y_pred_1[:1]","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:09:01.936903Z","iopub.execute_input":"2023-07-07T05:09:01.938011Z","iopub.status.idle":"2023-07-07T05:09:01.947072Z","shell.execute_reply.started":"2023-07-07T05:09:01.937972Z","shell.execute_reply":"2023-07-07T05:09:01.945567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 it shows the probability of beloing each class (event) ; we can take the highest probability by using numpy argmax function as below, and check average precision.","metadata":{}},{"cell_type":"markdown","source":"According to the Evaluation notice, it says \"Submissions are evaluated by the Mean Average Precision of predictions for each event class. We compute the average precision on predicted confidence scores separately for each of the three event classes (see the Data Description for more details) and take the average of these three scores to get the overall score.\"\n![image.png](attachment:0b18ab7f-dfb2-423c-bf9a-d4be68dc93ff.png)","metadata":{},"attachments":{"0b18ab7f-dfb2-423c-bf9a-d4be68dc93ff.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAQUAAAB+CAYAAAAtBAQnAAABW2lDQ1BJQ0MgUHJvZmlsZQAAKJF1kEtLQmEQhh+zCNRFhIuCFkJQBCbeCFqaRQUtTlZ02R2Pt+hoh6Mh7voHbcJN++gfCBFUi7YRBN1W7WofuCk5zdFKLZqP4X14mZlvGOjxqIah9wL5QslMzM/4Nja3fP0vOPHgYpQxVSsaMUVZkhK+tTvqdzhsvZ20Zz0GTh/KV4WDcWXnenjqtfK3vitcqXRRE/2QjGqGWQJHUFgplwyb94W9piwlfGhztsUnNidbfNasWU3EhW+EB7ScmhJ+FvYnO/xsB+f1Pe1rB3t7T7qwtiI6JDnCLHMsyfOhECZKiGkWiP/TE232xNnFoILJNllylKQ7Jo6BTlp4kQIaAfzCYYKSEfvWv2/Y9pJliCzLV9W2p7/BpRcGQ21vXAf3EZxnDNVUfy7rqPcWM5Fwi9016Kta1ts69E9A496y3muW1TgG5xNc1D8BgNBjuQtcyBQAAABWZVhJZk1NACoAAAAIAAGHaQAEAAAAAQAAABoAAAAAAAOShgAHAAAAEgAAAESgAgAEAAAAAQAAAQWgAwAEAAAAAQAAAH4AAAAAQVNDSUkAAABTY3JlZW5zaG90QNugoQAAAdZpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IlhNUCBDb3JlIDYuMC4wIj4KICAgPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4KICAgICAgPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIKICAgICAgICAgICAgeG1sbnM6ZXhpZj0iaHR0cDovL25zLmFkb2JlLmNvbS9leGlmLzEuMC8iPgogICAgICAgICA8ZXhpZjpQaXhlbFlEaW1lbnNpb24+MTI2PC9leGlmOlBpeGVsWURpbWVuc2lvbj4KICAgICAgICAgPGV4aWY6UGl4ZWxYRGltZW5zaW9uPjI2MTwvZXhpZjpQaXhlbFhEaW1lbnNpb24+CiAgICAgICAgIDxleGlmOlVzZXJDb21tZW50PlNjcmVlbnNob3Q8L2V4aWY6VXNlckNvbW1lbnQ+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgo8ZHPKAABAAElEQVR4AexdBYBWVdp+p+hOkW7pkJBOO1AMxM4VE9dV19bfXhW7sF0UDAQVgxCku7uHHJoZmM77P89z7v1mQBAMhtH9Dsx37z31njdPnxORmprmmeG/7yIi8OJFWA78+Bq4CETJiYyAn4vuIY7i4psxIyOQAn5yzA5xLQcvSMjwvHnRLwJxGQ0R8Ofe9KXPMPww/cPyl1dn8lP/IgOFpOrS8deT0vpMcZov3Y3wEBroL4xABOMhPtU7ZBCYidK7J0NdHHnjB185ACsnaP6by1g+YfggYZj+Uoqw/DlVATHyS/8iAyWPNF9RoZXUfSovK34vB8qPT73rDUEM9t+dKjtbQX85pqVQBwxF3BwEqCHBzKH0FHmJvZ9BGH6Y/pIdyENY/kCJ46h/kdJbFYDa6TQ00ld4coeKHSizU+OQ6sNgiI36cb78JVddGrU+8KkWhSL4psQ3DPQKw3dEYOssTP+w/EEIUD07bTpu+qcWAFsD4gdrK6gyugauXCie/PnjGhXs68pIMCai01A4R0TYqsATiaTwjEijQuPB8QU/rVoQfiZh+KBKmP5h+ZMaFQz9i9BAI2vuSPxBQ6nO6rv4yi2Fl55TzVX3S+Gl9rQYSMMQGgs6vjurwTz57nu6YIb6MYNgpAjDD9M/LH/Sm4Kgf2AFtFWaTA1mxe63EtREoCbjj/+huPpSe58xfX88PbUQkJYR6NhSUJ7+J7LU4KX7RFgQzrRIFIYPyoTpT/EIy9/x1z+0FFKkldRnKjWVme/UVLYbPDb/4aHOA8NcoB+Dyo5XpnMP+ef9oT97D3zSEUaolaFEzJCQXFgYvsgpmoTpH5a/46F/MAqpTq+plUFzH6+Bc3qLQF9bGdkpuEKCaHgyAPGk3fjk03+EOhjIg1EOdsqJ/mH4IboFNHK0AXHC9BdtwvJ37PWPPTm/5nZiSCHkn2sSqHEvgZQffqjgfPdDXFzGpxVgZnRsGlCOwUHXKPbT0Y/hgWXwk7iWAwNcuOKoSRKGL5LSmPrECdM/LH/HWv8wJUmxk+hJmUNfEZh5gDD6If5T6it74RQ3lJIiG3IRNDWIkOMrP1Xb5ePnx8VLrPn4PwwfdPOpzLGX4CtM/7D8HSf9i0jBMmepKqWR1XqeAQAqPs2Ae/qC6wut0rAGU2o/zEV0afK0BpyoMzBwjO++Xf74plcYfpj+YfkLVMPXu/zXP+xQgDZSP32FdjU3lTeo3/HkjASVlg5GwFXySOnrstPv4INqjj/EC6VxEZBY0FQDuDAXV8Fh+KJXQaf/rp27bOWKFRa7PjbM/0DkKbsUaPznz19d/v1JBWIjzDTbkGsl/BEBtgZcMIIc8kTdqTi//fRsOfCPpEE8TmPSMZ5L76eijZFlQf6I4vIJwyetRBdRkPQoePSfNn2q9e3b194e/I7PtzD//27yrxWVOdJYp8DsPoSU3Zm+kIiqiUDJlXMKrthUcDiuXKQd8DdHInog1G5lpB+LSSDuvjDhPQyflPlr0L9QocK2YcMGa9y4EdjIckNawvyn+P9t5D+ayJC1cmAu6nS84o8KzVfU+FRabmlSI4FazzB+0OFbr8iEik5f7ppWqEvgtxL87jICglYH4TJeGD6IQFeA6e/lZFlUZBS6DetZUOvcpbPNX7jAxo0dZzVq1LBL+18KBme7CgExsgPGqrXIrfWRlpKWZkM+/q9t3bbV7r/vfpswfrxNnTrNevbqaT179hT+WgJPqQjLn68c+a9/MgqUR7oIbVKgiuJP1T39yF36uBqBCh1s68WSfTUe1GZAfO6FID9dCvdOU+GbA8V1BkCZuHihyMwXGRJ2GH7Boz+UlC2C6TNnirE1YQhGjPjaXn31VRv59ddi8cX9LrFNGzchPK8jP3NsLIzHj6N/tOzsLHtn8LsWGxtrF154oU2dNtW+GvGVrVy5EjVttkVEUSSRJix/IsPx0D9nFMg3KbPrOPitQVejs5XAGkzK62mLBFkebI2mt1oIvkFg2IGOJiMSYsEWBWM7YHz1l7sHXowhQxGG7yioFlVBoT94k5aaaj+NG2ctW7Swb7791n7+eYINH/6VNWvaFPzNsQcffMiyM7McP0NCgJouO8eKFitmp/bubbNnzbaEhHhr1Kix9ezWwx55+BHwHJwn09ESkYRQROAKFP4sGEqnhT18gwBLL/D+d5N/qrKoL5zBePIj6Bm4J5svjMQBA0UFQRjPWQHXVUAqlwFT4w8fYqx+wFyqO9+p9CSm/5ZH+13yMHxRyadlQaM/Zxz2J+63+vUb2AP3PWAZGenWvn1b4ziDByFJTUmxFBiOlJRkPPGewvdUS0M8GoYKFSraytWrxP/z+5xn6ZmZtnHjRjv9tDMkPhyDKsj4s5A0VH93+UdLwVdHcoPVN7DW3n5KpEw11Z9xXC3uXt3YAVXdJWc4/KDkTpDhr54A/ZES+TJrDjZ4OQTkO8Hw44Thg4SgRQGm/4qVK8Tvy6+43LZs3WKjx4x1RVa3z7MxY8fCMKSCucQDD5/VmTAKzZo0Q2uhKFoaE6xMmXJWr159dR3S09OtR4/uMhQFHX9JqvD6e8s/jILPOWkt3qXVUF58u3obDUMpu7TWRRd18jT//TzUHaBxoDz42TIP2ge2KHKQZyQiMWdZ2wAmwiUQzCcMn9TLH/qr3+4zKnj/FfqPhdKTRW3btrUzzzzTZs6YYc8++6ydeGJVu/rqq+z6a6+1bAw0if/kKRz5n4WxglKlSlp8fLzNmT3DWp/cxqJjCllCfILizJo1y3bu3GEDBtwc5v+v0F+Wluw6xvqnMQUpKLQYHQUxUQtoUKPz6VoDIdV24z8slJSbPMUHlZvp6U+95o9KjvQMYgsBL+pwqPXBOIzsXBg+aZ3/9M/RSDF5EeV4dAT+d+ncxRo0bGhlSpe1fpf2M2yms+rVq1nPHj1V/mrVqmNYgFzO5b9am6wRwP+c7Gy7/Y6B1qp1a/G/ffv2dv/991utWnWsa9cukAhXEYXlD7pxPPWPuySlmlRmvAUfavJDUAMf9+Q3nWOfWgZuVCLwFrNpWpwBgLeSMFf4EACaDSFj41K53zD8fKc/DTVdVlaWjRo1yubMmWNPPfUUWnMwEghz6w9cnBzU9gn79oudpUuXshwIbfzePRZVKNrKlS2Lb7QQ0NqIhlE4HP8zszIsYW+CFS9e1IoULWaZGFPYv3+fRUfHWCnkyRGFiEhnVA6GH5Y/8iF/9E8nL7lam0KA0YOgRgd8FiOSNTuFhw8YCTGLIXxHuJtFdBae3QQ6+vPHxWUtmDvEkIuaIjBTRKVvGH5+0Z/Hv2Vj4C8GCr1k8VJ77rn/2IL582xPQoJt3rTZYqIwCwD2kCsB/7NhFCKjo8XTHDCdIhEFAxDwn5E1jOTLxZH4T/jZWPsQE10YXUvwnnlC/uhUoRwEn/mF5S9/9C8iLS2VBh40J9X1IsboR9/wh0MIY+R54UceR+VmFkHMUFqmhKOUcQ+FcvFzIzhIQBg+qULikSCilvsJ0TAP2X3S5Ynlxz16+lMB9+zZa8/95zmbMnUyBg3jbPeu3VayRAnbtn0blD3qF/Cz0PTn4iUVT0VFRUCGB+438p+oSVT4g9ZBfuIfoibJHZa/X+hftJteAXVo8n0rT35JOsl8vpH3epEvXp0HGUkj4OL4b84y5DEQLiuGapMUkrp09IcvVjyppRCGL0IHpBal8XEs6M/VhWwRsqXwzDP/sXfeGWwjR460qGhf6cGag+FHQnFpTNjNSMvIkASwrBnoAviigS+8if+BZDAfOOLBID9iTEyMvDMzM6xfv/4Y0YAL81+0yw/+H0n/QlOSmh4A06Tw5KAGDPzksBcaKyLvxE7HXQ5MUgjUxFQI+pNI60wFIgYYIroMC9NSQJDMQ/6RMAjOpMDfBxCG72vQMaZ/6bKl7aZ/DLAaNWvYd9+NcrwCGxz9yQ8wjbU/+Uo2SyYibOKkSVidOIYRrE6dWtaEC5eyOOPAlAfd7BXwH7HzOi5g2rV3FxqOETAKWB5NF+Y/iS8qyoIeRP/81D/NPpAnOhcBjNdCI5+ZXHasfh5rdEbiTxCGD7YyAm8XlKPBInnTw9kM4ajEiO5aJhAfhPMQFtUezBaJWHuF4YMmAY2PIf0LRRey6rVqOLYAYLDVXZw+BPwc9PGiYMSvvPJK++yzzzCdGG87d+ywBx540MqWLUdpkAGRGfHTU16IC+2JGK33CC1Wuu76a23nrp3oObhxhDD/C478O6NAhaRlIu/0K1lxHBVnEUZNZyAUnU8pvh+XsRkkgaAfhYGPkHT7YUikmofGgPH8vJhZGP5xpj+tM5givjgW4kNcxJPrS6C8+N+8WXN7/vnn7YYbb7DEpET7+OOP7Y033kBF77oXbvbA5UMuk9/EzOXkfuvXq2dXXH65vfjSS6oItDwuLH/UiMPSPz/1LzDTMuS5hXJrClRK/lCZyVlaAvdfiq/wQNEZTA/9OObzQ8l8b/fFSGwl4Id/dDQSShd4heE7xjjyHGv6q3V2BPpz5EflQDv2/D59bMBNN6lwI74aaR9+8L4Yx67j0fCfA5m9Tz9N6bMxgBnmf8GSf8geVNUXCHKJqxfJYXq5N76zJg9U3PE9SEI7EUwlMb0SMU+88B9j85eOU090yjfIgHFD72H4+Ul/LV4C/fk/cEeCzy5e4SJF7M5/3oXlyT2wF2KfvTDoJZs+fbprTYjZR+Z/08aN7e67787tXvgFOBJ8RmNxgyKH5Q+04BRK4I6S/kElTEoerH84uBVXuyIjspEugvOD/rsjOJaisCbn+EAeJWcaOpkM5ep7KI4L0x4KfisIMZFhwEy+8D0MP//pT2PADUriaIj/jmdu/sEJyqH4HwnGR6ErUQsDlI899phVrFjRNm6ItXvuvge7H7lsOZAD8FeSd2j+lytb3m6+5RYtXArLX8HSv0jqs1NUx0wanWALlAIkGU5g+ErTzvhc0RYIgBZHaw0C4/kZssVBucO3s0S5UILBROYQhn886M9WW4Rti4uzZcuW2azZs8koS05O1rqFuK3btMrxUPznmAH/yP+TsYfhiSeeULQFC+djevMZctQhdAT+R2P688QqVRA/qCgoDfgKy99x17+ohx568DEygy0FtQR8XecUlBankLlw/kPvzgP9fgS5UMU4IMzlRUPg2hfKlp9MBIFy+dHDub8rfI7aUwG5ilD9cuCfkwU/PnmoiE9Y4s9VZKIDwo4l/SNwxD6nk4d8MsTmz59vaTgRiVOT9erWw6alBFu+YrnVrVMHexxKhxh/KP5zRWOdOnVt165dtnDhQlu7dq3VrFnTGjfxj2rzWwriPdiszz/If9flQY7IjFkxb+fcm5ZbAzfSnHFIf9Y8/P4z4AfQDgdf4QAkqRdAvamcfxX4EdjqSgyEBAWSjr8iMX4oqBJqvqOpQLvOMMWRH1O4b/ZslAMjsLsBD9dKYCqXjnGd8/18YC5npXZ5MxLz/6vCx/iZTsEmxYAWlwZH+ysFNfUa+GGVIPEk/owjRwE+hvhTcdhtW7J0iWWmZ4QWLUWifDwjIRs0b3TSSVYSOxuPhv7r1q2z62+4wWbjVKaTGjWyDz/60Fo0a3FM+I8Cybhm4ym9p5DBSXKAl/PEAx7ciCej4AjsaKrY/Pmby98f0L8InKYDXacIgorSYAqkI5gjtST2gDD6iwn8kfSSyHxlOgq2W5QU2u1G48AIvlGhUsjM4yHIf0P43A786qsvA79Ie/KpJ6xmrVp6J4VYMz+JjUf79++3Qljdx7MLs7IyrXKlStayZSs75+xzrFDhQiQY/v58+nMDVFSMW7f2Z9F/ypQpdvlll9uu3busc+fOOGJtBLZLl3Ctwj+R/088/oStxkEt9erX16lNVPzArVy+0l565UUrWbKUZQLHLKyYJP2rVjvRunTtap07doHNgOz+D8ifKmFaRrbKfyP9ox58+KHHVBtQ+EBACaLMrPvmq9/ukojyU1EZE9Gp/s6DJkGjCyE5po8sN6OQEcySXsELPvl6vOFnZ6EZz+JFRKlJ71ZqorHPFZeoffjNVpKm3GDA3IYgkF1NYXYBSLdc/NNR29522232w/c/oCm+0i668CKrXq2acCW+bHmNHz/BPvrwQ1u8dLH1veACq1rlRCw1/tomTZpoO3btsO4Y2Sd1jwX91bQWr/88+nPbdLGihXXQyqbNm9Qq6tGtu2gmfv8R/kvGIjHeMdXuw1bredi8xUHNGwf840D5A7FeeGGQTZoy2RJhcC/se6FSDn57sM2bOxfGoarVrVdXnHLM+PPwz8t/8hj/HRxWeAVc/g/WP0g9WYb6S9JH+wIBh3XRH74QHKAnMRKyRNKhrPhKjzeaBAUxPV5cWrxIZ5SR/NmWkCVjNgUAPhfeqIwsNAxDdiYwIQ6gCU2ClB5CHQUGEwuODchPzVWO1xPvXPxHffOd8dKUUlhKTEyzsBsQCYit8I9AM71EyeI6tqxY0eJ25RVX2hVXXaEadtmyFfb+ux9Y4r5ERT4W9GfL8M+mPwnTr/9ldtVVV4mnH7z/gY364TuH9h/lP8qbif0Wjzz0kMY6KKqJiUkECTzEDeFTDse9IQgGIcnatW1nF15ykV133bVWvWZ1mwOj8Omnn4r+BU3+JD8FSP+wjoyKAEo6rQATQTJ++4UUFRFF/WB4u64G49OT/wN18JMwCGkDHdAnstSMAz/gNMUpmC718YafsC/B3nzzbQjQdTZ06BD1safPnGb33nuP3XHH7Tb+p/Eoc4QOK73tjjvsnnvuUROWfmo1yBw6/Hm60HPP/wenCN2E3YZuwWhmWsYv8OcAH137U9pb2XJl0eQtbVWrVlUmu/fsdpmRwI7Ifyr9PY+tG0L/8+jPlmBZHL7CtQft2rXVEuYnn3gSdFqt6ew/yv8ff/zRduzcaZdc6vZKpKen+aTJlb+EvfE685EBHTqcgpZLMTvhhBOseJFioKfZvn37iHSBk7/jLf8Hw492dTt+wVTaAVfriXSKq1oF/lp4REFCPIoqneRKfkyFIHkoyMVRfq7RobXtTAM/7ozkC/PWRhukPi7wATcbffkHH3wQzcp6NmzYMJs/b55lpGdaUlISDhGJty+GD7eFixar+UnlXbp4sS1duhR9/sL2NMYFAiIE+HPZL41Fn/PPt4F33qngFAhwXvzT07NsxvQZolG3bl1FMDbh6Ef6tD75ZCuKFoR4gbxAckd/xOSeg9dff92KYAERhd8vgOAwnnsxjUkMGDDAChcqpEZfAN/D+gSetERjxlOSjkT/bPIbf9HBUD+/AYZO7PYR59oFurp169qjjz6qFsOihYvsMbwPfvddK1WypI5qi0Kr7Lfyn4b22WeftptvHmD1MCtC+Bn+Tk2Hr5M/zn4kJrKFZdayVSuRZ/u2bbZk+TK9d8W4QoGSv+Mt/4eBT+0UXUksNhYChvON75oagwDxHY884c6cBB6aWwiMQvBEGiTxIdAIID3DBIwZ4u94wid4wO+CgbHzzjlXX6tQsy1ctEir9Xr26oXR+XQZCs7N9+51qjXH8eacz1+zZq0rvm8JiT/PIhg8eDCMwUBsEiorwSWKKckp+M3Ff8vWzbZ3717BK1G8hPJ/adCL6DNPgVLVU7+ZS4F/QX94cKsyFSI4KTmFJyjjL5knJ6e5dz5p1OgOpj85uX3nNtWaPAUpIWGf7UP/fB+mItli4pRkPL55DPteLkZiFwlTpyzLr/Ffo1mUHxiNrjB0bE3RfY8a3t0FgRxgWH4P/0fAMKempVu/Sy6VYWG+SUnJlg46qGD4Jv0XLFrAICtWorjWWUycONHugGGmce/Vs6f1vehCy8S9EzloKckRqeMpfwUUfrTKBcIEihroM5lLb6fsznCQoXyjOVBfWyQN+IL4LjMniT65NdobkN5lCcFgXGXuhISAfEOBN7n8gs9puIsv6QclXy18iUNf1PK8y2DOnNlCqXr16nZZ//5WDU/uDGQZOa8vB6Xh8AM3DA1+azBOKi5j3bp0syT0eYMMOSgmCvr4L1q0UEmLFS2qvu6SJUuhjHut/2X9YXh6yyCpBke+Af054EnDxBr3mquv9pU9gOCKEvoFHLpCUYVUVn4G8JnPP++6E7MehZE5DZVCFD8UmV8+/99//wOsOsSJS/TCD2Mfif/REdEaJ3kRy59LYgaiYsUKSsW2hMsHJTpK/m/bvsNex4arqzFWUax4MRjYZOXBqVMaQ7aEHLqeTZ44mSW3cujGfDbsMxlOTgOzJXgG9lrUqlFTA8es/Hh+xEoMAjdt3szOZYVwnOSPZBBVChD8aE0b+kJESXOFZDnRtERB0ap1jGTZJaIQCfqJu3niK9z9RHCkAhnRcFAQ8OWHUpzwjsUzCsD78YbPVnEkBhd5PiGLVhWzBO07dBD+XM9P165dOxmEeNSeM2fMZDTcd9Be8Vn7kRRcGfjfIf/VuMCgF58X/ozAsH04h9BRwOE/ZfI0EZDTan36QCBBq1KlSlmVE6pY9RrVkYIO+eahPwnOxU48/Sg1Nc3xCYxgV+Vgx3Skf6aXZYUsGrAddOZJI3j6qWc6+ouRCAuYzmih7Bx8wqWBdmNJxOco+B/l2Xfff2ep6an2nwefsUqVKx+A/2/h/9toeW3atAmtgEX28MMP2YbYjSFsOLUbYMbWE1tadOecexZmb7pjbCjaTsSYQmWsnKxQrryQo2ElvoloSb351pv24EMPwN/P5X9Q/g+lfzh5iaO3FAzQhhoMwjjBcH4UAsqJEyzGpIAwCsKpEPhwYX46eOcwTwim0uFbcf1cEAseudQvKPDZn2cRu3TqbEXRX2dN9POEn4VUd0yt0a1duw6DXTv03h6DaaIXWqJUzEGDBlmzZs103kAGbkki/sM+G4am+T60GvYrDbTLstB8nYFBTL5zQK5Xr9NQe+H8w2AG5DD0F/kAcF8iBkXffgs1ZAwBH0h/B4WcYZA9/fTT5qmWlw98YaShFNdei5YG4PPdV/ND8F/kQHOdyfhDd2T+M8958xdik9QLGFe40s7Djsrfy/+16KJ9jLsnH8A0ZPUaNXCuJI6KL10G902MVml44Qylk/K3detWHBO/U/5nnHmOnX76qXp3ezwkhPgmHeAgf7y4Zvfu3dbm5Paa3kyDoS1SrIiVKVXGIc64Sva/If8ija//bqCR2ktHIrCWAaH54ZMQRgBvmqKjN4iElgCNAVsRcoqOD1oLP/UBxkL5MaZjoFZISvgdpFAwsjge8KmoE36ewKJj1BotALhdOAAkDoNUdO1PaaewVbjvkK5mrZpWFXcdzMIJQpz6mj13tk4v+vTToXbqqb0Rg4IUocU1NArs/9NREXnDElcA0rG1IYVhc4XuCPjT+BQtXARdjF5aB+CTMFTrhxQcudJF+7MfLI9zPv1pzGHMAl6Fgg8BX7yisCBM8Y7A/0Tg98jDD2PDVC27c+CdGDDFyH/IHZn/nBlhK4fLwAe9OMhOqFzJrrv+OiuLC2QIf+qUaTpoliKZjtWYLBb952PtQuBatmgeMpauJXUQ/qAjW3x0jRo11KU0nwz5xK644gp1M7iik/CVN+L83eX/YP6zbekrouO8molSbpKMxEQEfNNLpCU39J9iT7FkFHgwlDWXvpkERgXEJVNIXm6J1aAlv+Cvviz8aFmcITh+8ONw29EmnGJM1wwDicRlIUbOHcae1apVW/gznI4Dh08+9TT6yuXVx+cIO+fNy6GJKiEEOdauX6saiPFnzpiOGn6fFcPU2FfDRygu897PkXLEVYtWNHd0yQVGapLYufQvjkG0C7DYyZVNZsHFQX6O/oyvJKIzAknxX9BfQJn9kegPPkJFVSRGBxNd/vA9mP+8COalF1/CCdGL7cuvhqsrxTSHgn84/hOjLLS0hg//UhfYtmrVEkfClxB8rlVYsBDKryJE4GKZWVqOvWP7dvvkk08d/jS8mHqsfAK6LAfIH4vuzCbzn4QVmBw0ZpyRuOC2ChaPceyIsy1uMZqQxY8j5t9Z/g/WP40guavmgTwduOIi+d/wkjAwjEosA8APR2D+imCknS+jjEIV14i0iOq6KMxH7Q+8sOGqrPB+vOFnZGbb9aiNOKDWsH5DCTF38dGvGC5GLVGc04NoRXTsiJuQrtby5BI4+bhTx062efMWq123rjYGqeYiHfAXu249zh/spzxjorCUOSlFx5klJe23m3BACfMrjDsYOTXIlZNUHLm/MP1HffudvfPeu3b3v+6x9mhBUTbIcS59JX7uC994ORz/sZEcF9km6saoizBbUAlLv9nUL4QBxaRkN3hL+pFaVG5mthPdgPoYn6lRoxa6Zxm2fcdOa9CgwS/kLxis5TFw2+O2YiFUbbsHa1FKosvwz3/eiTUNVXRhDRtYklMxRMV1b39T+f+F/oUugyHaQJqDShJQ9nFJedb40HrWCqrzKfEMUw2CKIxDB0b7lQhfldS9KBNF8bML1TqheAplHscHPqcYudOPg3DVqlXjUAm+d2PqEcJYOAa1SFWg4vDnXDgvUeWAZHmsW9iIq9e1cw9pSsCAVKhYSWjHbtggmrD5zU05PHeAg1zbcJx6NMYDhDtq1hrYVeiauMcP/4D8f4T+CzGjcuvNN1tTHNfGq+RKY4elbouSgBDbkDgdkf9pGCuI27Zdg6oUqkqVKmMhUhG10LahSxcDYyq5BP1q1qwl3iXjUluugWBrhTMSVU48UfR3AurgaxwFlB/70zjrc955Gqt486237FIsiHoJR8MxAbuSvNBGMu4L7N9d/sUc/gT6l9co0Bg4e44Ivi4HfhJiJmIMhOFVU9h8Mo2m0Ng8YLoDHFP6TQi8OsPhvzBenvgBLCUvoPB5KQpx4LZhyJ/o8L+E/6H4vw/7DK65Ci0otIK4z6AOFxjBGHCtBSiEv4LF/8f+73F79j/P2JIlS+z22263DTDgP/30k+3BZq5m2t2JMhdQ+TsU/f9s+aMaO77hwcylo+Qj6SLaQOFlDOiJARhE0BuY7gyCawqi1ecIiceBjrmysQgXalYANeZDWaFjhnB/Bfjc6aDBt784/uvXx9qwT4bauHHjULtma0HUT+Mn2KdDh9r8BViCTd7zDy+H479udQLf3n7zTZszbw5q3gdwm3Q9xecshHMFj/+rVq2E8je32jVq2flYk8JFWx999BEWPPlWnvJ4FPj/XeXfLc4H92WBKAAUAygslZhPvrk+BwjGeCQYZiKcoLjDLNi8ylV4RUBin6rMGRlxNsLlhneF0d+3JMwX/wi9oMNX34L4O8r4BoK4iljwZYgvXMK54OH//XffY9XmQps6dTq2cadqIG/uvLm6Mo53OtSpXdvegKKfyCb4r/A/AlMYE8b/ZG+hdTDw9oHWvXt34c8xqYLKfxrASzHWo4tv0JI566yzdCRc6TKlrX6D+mAjZJqH0ICN/7Pyn5qaQg46WaZcB9QgRSTULgghEI8DHb+pAs5R+KEsviIosh8olac/p3l8fVFuDJcy4UXv/OY7cg7Dd7QASUgauj+L/sOGDcUAahHUjh/bmDFjsFail/W9sC8WT51gfbHdmG7cT2Otc6cuej8Afh6mc9zk+uuu0yzDK6+8bGXLY/blN/B/xYoVVqFCeYy3VMoX/muxvqbWiQSrITwhf9q4BnmjbOZk8xsYq75ymB+AvyjifvKQwuX3N5F/oO6jHNR0VEb4uU+GcTqLv348/0GysCvhHD3RrNa3SB3KlnWqhuiYBzKNjKRVYBx6MLWfYRi+T8pjT//evXvZueedrwU/BMql2f0x2MYzFwMXE8NDXsibg/hPZYHj4OwTj/2f9ljcd9/9mo79rfx//PHHbfHiJS7DfOA/AUVhYb8aqPjhgjHesC0H+GzBRtIg4OmKcwj8fXFlmr+r/IvFav6r3++0nM0/WnzW8BIMvQezD6CKanP3kGbTxMKF+BooOtJrNoNERriML9Piw/koGeKQ0pqkkkcY/rGlP6f5OKjG3Z50119/o3ZlLl6yWN/ly5fDbkQeRnJ4/n/wIc5LwFLm++77tzVp3Ejpfgv/Cf/rr7/Wrsb84j/PwyAs/uMME42AusGSP25Ac4YB3v/T8i+j4PqAqMVBLDpHOOqu+6ZwUDxc/U6NRkOMD8kM4ojCiEBiKpafksRmFjIaCNG7iwPfkAvDd8TMT/oHBoHrLZrgoFUyc+qUqeJJ8+YtrUL5ChiARCsBUyzkKaf6NPOC92nTp9grr7xqt9x8C1oc5/0u/vN8BDpOXYb5n//8F/H9n4Ppj4FGv0UALefAn5r4VGyYCxoBXR4rC0ClRlzGQ2a0plJ4NhLYssA/vro3PGUPGNdZY+2qREIZBiVkBOdBoaOVCcPPP/rP4DJfsKBx4yYwABwL8HSMGVhoPbp1sywsN05CF6FkMZ6zCDmgbIDpmzZutKee5PmS+9BlKGsff/jxAfxnemSrHycr5L+GXsVublLiUWlffPGl4pQpU9bFD/O/wMi/dkmC52AMrABeyEB86kdMJYd9P30z0I+keAxHBLYu2GDwTYbyCmYXwG8IBALRwqDx8CFIUriPIgyfFMxf+k/ljkLwpWOnDjiBjk3nCIvbvJXMtAwcPPPeu+9Z2zZtrFXr1molcOlvGlYWPgmD8DPOKWALgzsYuTtRDvx1YiEBQd5keh5e+/xnHz4Jd1DyHAe6ohjwVL2Sz/gLOACH5e+X+oddklRTMNCfFshlI5XYqW+It85LAsQQlzJoOfDpC0LogRc65YP4Egx+CiJekCJY4huGD3qQMo7mItoxor9bHozLYnGWQIdTsE2cCgzenXnWmZhiHK/7G7gjsTgUn2XiykRG2bN3DwYGF1lbLGHOy392K7ilOyg5x4fIYZ/7QsVVDIgiGaqiMG4FL40lxsQ5LH+0UPnD/yPpX0RoRWNIGimJYCdHG6jEZB/+0zAEn+ShuOvPx4emocBedRNcBBclkIwgfz6BP2+ZjoRBCIlOEE4RCcM/pvRPT023H0b/ABZHWrv27fwl2BG2DlvD586dhyXFRXGmRHtMUbp1CgH/d+HsyKmTp1oRnNrMAnJOX3crYH81Fyv5rHVKThkI8ZQfeRxFil1HyEFv7CqNiYnJEzfM/+Mt/yGjIOsBZfQrc3HQdR9kHXI56jNaDyo8X/BD5eav8vHjBMJEHVeHEv5qmQiID4thDGbqMPx8ob+HuXj0BqDXoDron4kj3kh/KTZaBWIpeSJ+HMj/HTh5aiqOWs/GvQoXXNAX+xCg0OTd0fJfC79wNiZOuF6My2jq162LrkhJwQ/zv2DoHzlOs+0sN6QhEAjqKgeW8EvpYCwXCOtOp1ZDEFkCwWAKBwP9fJQe38oXP+oq8JvGAw8/rzD8/KW/5uKh8AH9oyKjLTIa7YY8BoF8PBT/2fXg/ZHvffC+z2RE/I38X7tujb2D8Yhnn3rGtuO4tTD/85f/R9I/ZxSopHn0nkx2AWQ4HCyAwlXD+7Lgx1eYMwXuV/5UeYmV/5v3SzlqQZQsAz/D8POd/hgECCk1jQR3GIoV+j08/0uVLmWrV63S5bBs9iclJlsyBg5VyzPtEfi/YOEC7bf4+JMh9u2ob90Bs2H+5zv/uSDxcPrndo5T6RkJzh2G4njrvNxUI5dvkfG56u7ydC0GCBTC5JTINSZdyyGYcWBrlXnQF86Pzhf5OWBh+GAEDTDJ40hSsOjPexxYtk6du+Ccg1Qb9NKLOIegst2IuyRjY2NDp1RTAjhmgbFtYcKdk+XLl9VKSB5m0qxJE1u0YCH2HWDmAxmG5Q9kgisI+ocpyWCk2IlhhOYH3bsUHkylGqupKc7535BYMRNsd0JMEXbp3BO/CoC/gtBEQnAQgy/Mn/Pf7HTwHyOG4YMOpBupQroVMPrPnIm7KVCqNie3tpmzZtqPP3xvl112mZYLr0ILYvnyFVB0tjoQixGBCs8oKFG8pI6979Cho84rCBZKaWkx4oX5X3DkH1OSYJwcOQg+kp/o+weLjRxnFSQhpYGQcnPASElYF3CNAlc5+lLAPLlowTcwrnXgACkGhJ750IXhiwz4cfQo6PSfNdsdez99+iybPXum3X//fTg9uQ9WPmbqHAVeUhMTjcHHPPznQbbl0EoIbrtmZcHpSLpMhOVKRlj+CgL/1X0gc2ippfD8oNJCobUunJpPzcWDo9EhByPAcMdRpHbVmgtmenyrpaBclVxx3aWsygxxc/MLwy/49E/D6UZz58zVqdWTJv+Msydn4Lq9N3XHZjYqCZ5exZ2Pjq+sXVzXgUJRt3Zti8TSaYkJtybTM48L87/g8F8Ht1LhXf8PrHL/fYOAT6x516Ei1H9wlE19vMK534C5fKqRoSDEolFBbeHsiL9+gUl8A8NcmAb2Q1mF4ftrNkAPkQQGlwpUkOgfG7tJ4waPPvKI7sH4btQoLGZarHMUa2PMYDsOUJVRIJ9hD1jrBfzv2KkTTsH2PcB0XuRCl5mRqWeY/wWH/xhTAOMogBRFcRBdASms+3atBfANYeS1a/ZTbOGVWxEoTNpNaeCiJGk780CKYI+pb1Q4YEkLwihh+CTRX4P+a9asspJYU8BTkHlQKhc3DRkyxHr37m21a9fGuYd97LzzcLkNmKuanwxmNxO85vmXlAXujuTlurynoTIuiZmOk655lRsvhJWMIXJY/nLl4XjoX9SDDz/0GGt1MiJU1wfaSl/w1f2Q0eAb/nwLQN0X+50HzYUbhGR4yIBI65kQKZ0nHv6Ln1cY/l+D/rzpqmSpkloOXRNXsEVi5oB8ZSuAR6RzljMa4wk81YjXtfHeCW6A4hJotjbJ9t1YKr1g/gIdrNKpYwccaIJEEI1GDU9CVlxARfmirFBGwvJ3PPQPKxrTYNTBBNbmYArbA27sgEyEH51eySS1J6TSajcwXLU/W4suLmPRj3HBUz38BAyR83P1gxE3DL/g058T2+A3t1CT7Wwkcn2DB/8odBN5UjL3QNAY/Br/3cnXiI98YvyNWOQ/Nmar5RkJQxKWP2qIo7cURq/4ySf94+oDwXcajFfOPRK+DAILhz/+B+P0JQvOovr+eLr5Bz8Jg5A2FI2f7FHQj2Fw3L/twpkW+SiAFoRhYfgFkf5UevKfOyp5VD35xxYeeS/+49u1Nhkp4C85is88/Nf1eEjDG5hoDAL+R+JSWjaV3egTZYLQ/D88aET0FRIsfOm/KwNFSEkEUDEls/rMA1/fofLlws9P+adJdWjgcmJtBGSpCo78o6XgzmgkGUlUMZtlBHvUs4EHeIciM1B8VKiLgW9ywwWJaaFA/4XB4AmTyhGGzIOAwYuZCxJ+mQ9huJj/s/CpgNioJirw6nmuHOSdEn8u/X3jK2I7HpNHufQXOPD3aPhPjlJaful+E/99QaGihLZkAz5vyMZuCY1LSFx8MMz7cPLHXZ7840BtCi+ixZPHznHl5oHy54AKg98sfzAqfiH0ULlIBX45x7e88s9p+kTcJ8JzINPA4yLFimIDWjHRrqDIv3/ICkpOXGCscokuGw5FRQD+tA4e2JEIDmUKAcMc8iFC85sRfH9HEEca5aHooUSKrPzo9Qv4yNWHvyE21lauXouaKhIHinbCwaPFHXwAyMhMs0mTpgIuMsF/ngfA16LY7XdClROsfr0GWjDDMZADHGGisIeHzyyZ58H4M2GAv1IDLJ4+fGWovAOBYBzUdNQ4OQYyX344+ATDLhxPzqYw8x6CDTjQJAMCzROHG2MFYDCw69TvD8IXXgQfoRuYJk+eIt43bNrMalfn7shslIWDfoYxgPm2e88elTcbhchB+XjYaaVKFXHLUl3cmF3SKQfRIx7BE6+s3zl0/Uv8/bg+/tIc4L8LMxiz5i3EydIblFWlChVwmGxf8B08YMsEeTuKBfgjH8R09Hd56gt4cfPWqO9GYRk2lBBjHbffcas2fblC+vBR3oArv5Q/hB2S/4RJ+BA+CTjfWTBkBuMD6xUCoWBBcPxPz0i170eNtF174nXT2JVXXmk1scJTpfiF/B8JPuCR3nAh+Pw+gP78pMfB8qdEiiz8mc6Hj2Ms6XyE3Jt8NEMAbwfMF1xG0z/XaGRm/HN54Ne9OEIpDLFBKHorB7wovpQHb0xCD/rSz73hF2mkJXii7ZmVmW6v4rLRsZOmqNYcievJarDmhGMrNAtG4asvh9nsOQuUVeeu3SwGTdzly5ZblapV7KJLLrUzTz8Ncu6abdy2LcAAfmj4CCZ8FZFlOwh/KgbKyyJrdFglYZ56OTr8ITwZGWk2bvwkKw4D17l7N1waT0eK59ia5cvsIVxaQtejZ089Ue8hDhvpgM94gu9SCLhfBDbX6cQrRGKxDqA/fYAWMhF9J06aZPfeey8/7O5HnrA61fsI/2zMCkRgdeKSRQvsldfeUD5NYJxq4gboxTjfkeMHHXGV3i233I7aLjjo1eXrSkWo5KGkSEUkWM1MkfAsL5EgH/AvJSke18O/bsOGfW2tT26J+xj26gaoiy6+GPngMlnlwzyjHP54cyndQijzoIzwycH9DeRLWkqavfH6G7Z+fay16dDD7vrnQIQSnBD34cOD2sDCwDGc7lf5j7iO/oAvNJiWeOBBeiN75sPS5ZV/ilQOaDpjyjh756MvLKZwUeuP1aAuN/KV/5w7KviCEZTch8/kR+K/T3MVWQDxQz84fmJQmB/OQ0ISfKGZlSvwQQzHXF9f5OmnRKpcpylI5E7FoSPD3RtZCIfFK6Imy/IL+IhLACQQKM6U8+ZMt5E/jtHhHytXrLTETDAdMbADWPgXK17c2rVqbitWrrBNW7bZLQNvszsG3oE1+RXss2HD7MFHHrZ4NNmYSiXJZmq4Q8L3cT2W+GcDf1izFStW21133Wlv42p5lYjTdyhhFO4CP7VXN5GoTJnyuI+gAeptJIG0MZ5bL8IXvpNW/HOO8nBE+rPJRGnGX0ICrrZ/7TVL2hdv3L24BwuQlKP6+Mw5yzq1P9mWY1ES1yD0Pu10+9e/7rbTUT6es/gwbphew1u0mZ/v3BQ0DSdKDW+WUcUUdoiUl//iIJXYbNrEKfbscy9KR19/c7A9//zz9g/cG0nH+Rn+EgpbDK4GZ94MpS8B8R1vsA05OE6uXv1a1pB3SiKg1xlnMQRviBTwXz74CRLSVuBLOnU0/Fd+PlwftmCw0oH3wfJP7lFWO3fsjPAIq1G3Ie6/rIZ3mBgiAnIxG2V1NPARUXGJQh53RP7npf8v9A8zSLknHyFX0l1SRQgOYmC9REyfqDTyJC8FWysXg6LBn69sdlLZ9YlvxXVfzFhEYO1A9wv4kn6EkUj4z6bfcy+8YldedimeL8LLMzbBFAj4jMbm6Q7cPUjXsFFLrJ6rZzEgNAX3sy+G20qcUrx77z4rVYKtCyYg9wEDjPkF/KPCn3gjH+Iv7PCO/0GFcyT8U9OTcWXZUnv//fdx5uEWq1G9pg3/7HPr2LmjVT8RQoLm5xysHGSenbr0sFSclThj+nR0kzKtJdYIVENzk/CJAqf60tLSodDrtLCIR69XqnyCtWzZ3MqULevTh4UjvfmEI/5QLC4g+OKLERDGLGvTvKl9N2GSpWNREWORDdHULridO3frGVO4mJ18SnsrU66kXdr3NLv7vgfF/w2xa61Fs8YhEMI/EtfrIadsGMBNmzbaOly4uxtrFHgmY2Oc/ly9eg3kSW5GYgFTmi3DUe+vDn5XcKpUqWbjxnxvZ599Fs5aKAUYiEmrgXYSyyab5t5s4wYsqNoQq64Cj4jjnZ2t2rVHizLT0hN347YrtB7h2ndoK2jSAVwzkJ6WCZqtRStiIxZSJaGLWd8qVq5oy5cutx44+KUQp0phpFPRfVu7epWtW78ecpdhZTEucWLVqta0SWOID+KgQOzmrQP9d+OiWy7z5n2ijRufpL0eTj5ZarZvVHqbMWcO0nk49eoUdMMK4YbzOcBjq6WBDl06dUH+eQ63QTrHOZcHKSbRZV7HQv8gGqiTkDmh0vEVVPPVmZ9yauJpSopxUEhYQhoDypWcSo0PltZPfYCxCAEgNIgBlVHK5yCFgglfdEY+EVwTH2GTcBnomnWb7AVcWkqjQJeCa92ljIjPYaPsrFSb9PMkhbVHraahJNQUq1eulh935VUsWcKlgY9Wz7HswJUlltTxSdwOwp/9++VoyvucIHqIr//OfvIbZeahI7yKjPPuzPPX8E+HoE2aMN7++9//IiaYUKSwjRg5Aiv+alp1CByJMxknKzPr2vXq2uixo23oJ8NsGcpx5dXX2dNPPw4Y7jLVJFxnz2vfx2MMIgLGJDFhj22J22HX33Cj/ePmAX6XRGDwcyD947ZtxdXxg+z1N9+wzz8cLNwysjJkEPjBJjKxWbgA17/DnVijttWHASON1m+IU3wvIgaXwQRCTMwdTR3+OTiQZRqulB+hJdDsjmzdshnrGjrbPXffi4tgyqFIvNQ13ebMmAIcxot2RYpEgh5fYSdmR9xeVRKXNYASkD/mzHwjolDn4nsqFj59iQNgeUwcaTp79hxr166dDf3iC7F0S6wzFkWLlbTGJ9UDXlRMg+FLt6+/+RbjDd/CIKRa4SKFwL8cq127NjZ5zbJuPbqDp9FqiX7z9UiU/yugBYqDtzydisb7lZdeFv/XrV2L27Ret/UwCsWxsCseZWGer776qgwgwAknFF4vWZkpODV7Br1x6lUbhIHXE0bb08+9LiPyCbaUB0aBSejyVf/AW5lepwiQbJRcFhkEcM4xmQShl3yRiAiKQSILk/kYg8GKw3AqF5t58oOA8Yl4jOEGjPAGP1qWEHyEsxYQAZH3nt3x9tzLr9vAgXei9oMA5XHKCYpP7u/dEWeLaACQXx3UQnswwLRy9TLcS/CjCH3zgJtUu0XIErG4KAOtLMB7OkqMzTcA5rc8hQWgRWDDTrqNHTtORXK+Pq6MLPz59LSxp0GDkyDERfn5q/iXKVPSunTGCj7AiokpYi9AMaufUBX9clxNz7xwFfusBbyDAYNl2+OsacPadsopbWAoJtvHn35uTzxFoyBK2vCvvrJbb7nV2rdvj27IYIvITEJN2dlefuNNuxF4s5jEOwcGg08aE47EZ4HW7+Bw1ooVKlrPXj1hFN4BPLP9MLhKxF+1FDIBdwqDrFGDepaYmmQrV22yV94eIr9u3Xv5TXQCYpnghL/ZmtXr7cEHH7QNsevtzcFvW0+cEn3LjTdAYV6z5q1PscsuvVA3MhUvXgz0QJOaSSNj7PkXBmHcojqOfSsmHlH4WJlkI38dMYv8V65aY/f9+98Wu36DvfraK9aje0+75NJ+VqliFZWB4wYzZzlj1qBBI6taCfJD+YO8/Dxhgt11990aKH0dZWl9cnPQr7N99913dtHlV6GvXwQlybZkdK0eeOABLd+ejtOv69WtY3feeacWZZE2nD7//PPPdKnugJtvw23bT9kEGOfnnnvOeO5ENuARgRg8A/mP37nTlqxci/wjcNZlK8tMT7eMpGSrXbuOXXPN1daiZUuF4cd/OmLmi/6RyigvJoepl07AVA7U4E5JURjfidF8RwJnAPjhKzgyUIFZdtX+Lgp1W9U+4lHTyVbmQ+byhbU5acb3XPjwgMCS2Iz3BS47TURNeMkll1hE9n7FZfLUtAzlFMmmCkw/WxIJIKxZIduzc7uN/PobmzjxZytVoZKdjh18N9wA5UBU2rN9qEnXrN+iQbJmzZujDPBEIYizKwx+8a5yMkdca06BZaiPAN9UdtgV+BN//CICTypmdvz7dfxj0IrBxiFErFangdWpUQtX3hcCKiAg0satX4Ur7jcKTteO7eyaq6+2Tz6G0gLeCZXdVfck9s64zfb4408AWAxaD09bRRzVPv5H18Jo0KAxWmQshfpjKB/LCKzYdQCc9atX2uDB70Goh8of47JyaThlWbyheUJZMtISce+kuzSmMJTl+9FjbAFqUzbZzzn7bPs/wC9eErU53AH8h1C889ZgnPk41y7qd7mde/a5gJNldWHg6KZMnWn9+vVVzZ3jxcCIr5J/xQonWgO0jgoVjkHJUSiUAWiTKfrzcjD4CUAv4xyHeXPnWf8rrrcLLrwQLQfPnn/uBdwjUVItWAr2+J8nK8+Onbqin0ysMZiZuNeee/JJS0CN/ihuqOrUhcYox0oVcQPXXTp3leEh8mkpiZaM4+gJf/6C5TgPopw99NBDyhOFgD82ge3YpgLujk+wJYuXWlcMGJctV95qVquBfNCaIwLEg3oFgVmxwuFZFIPLMSjzF19+bhNnLgYf/0/LxZmnM4FM55zw5+sx1z/AQDk14J0XqFMGcsIxhMgHF8xS8KVEDOMHnYQNT2TihAlPvCvUJZDRYFSKo7x8gHwwnv+JF0cO+m7HGMHLr7xmbTt2snGw7JnJu11axN+bgM00SCSZR67LFuHqMXxXrHyilS0Sg4M8krAG/zxrhFZDCzTpudgmC+kYZ8+O7fYAaq/qGEHnbclaiMMwujzwA/zZZGzdpi2SspZF+agpPv4sNz/l8BFNo4CPI+OfY9NmTmNxrEvXLlCAwk4QUfsw/XxdpRZh5StVtn6o/SjMs+fOEv5t0adHHQUgOegyjUYtBqGMLGZbtm6xpeibjhzxNWrN7nYPBgOjaACgQK4/DgyQjMqTgeb6G6+9jD0HOZhq3GvffjPStm7bKfzjk1LFRxUO5dm4bpVtiEMYXD0oazymDBs0aKC7J1uiVuPMibQQ8+55+Y+VsjbiK9ztAHf2RReL0Tkozw50behiWBa9uef8RcsF/5ROJ8tAkg4Oz0D+0JUBLpHoVu7fu9e+QfeClVHfK69wogjetcI4CtORXulobc3FJbp0rTryOjyZb1uxaLHNXLRIl9Ccgb0anMdJTtht65dBWZFH23atICsY7AWPy1auahf2Ocu+GfOz3XbbrXbrgGvtxptuRssIF9FCkCmt551zuk2bMcuGf/6J7d62yW7F1fann3Yaeh9QLXSzoyAgLBOxZLnmootD1+CkFjYaA7UPYaD2hdfewQG2p2qBVmCcA/kjcvmtf24WTMVEsQNTT3axFibjfKmXQlBpfD88NCULL6CKUMQPmv6OCIigFxgQkIOZMS7T+Rm74FBk+vofiP7Wa29a8dJlLQuDO599OhTNfGdzmdpD39TBR57oX07FIByTdujS3u644w7GcDDwm4NyMV+Wn2nKlSttkydNtov794dkot+Yhfl47NTjPHYU77lULMT08c/BWMHXI78OZRnkLPj4cPjjiffzzj8PC20KBVi4NFCVg/HPxBTqzJlzRMtOXTtD+OFAWNakFIa589l18Kx7z94YHC2BKdckmzh+ivLtjkEwhhGneQtdDd6wYR378suvrHbtmnZR/yusZ8+eVqt2bUQL8MET03WyZbBYi+cttc9Hfme9unfF9N9QlXD9pq3idTq6SwE/wVSbN8+VpVjx0vZPNJ3LlcPYDMsKAvDqdvEUxpD0ysv/7Tt2WtzOPcCnkJ3cspnKngVaz5vLgb8Ia4omu0iNLzAAMw8TBb95244KJ78jYGhy5Q86BiLzLpE1OP2JLcOomKLW5CQoKDPiVBSlGajSOG3DOoc16zYrr1bNMQiqN7PZ86mUEdakaROrVqk8lDbTpk2dYglemgYp63IQF/+ydc5Dpj3x5BNWoer7kgHexB2fmm3vvo6xLZQtMyPbep52hj2EnQJvv/G6TZw82ZYsXWbjcUx+w4YNQXNfWigceI8ArLkcZET+Z2CKfPv2rSyWzZg13W664UrIAeMAFxLVl79c/Kl7/EMw80I6NKp9+UOJD6K/Mib4Q8gf4SMT8cT9KDa8nLw4o0AILDciCw1+00v+FAD4CwCttR/GgtEfn6ohKPt+mIsR/DLX4NAWRnDAmB/zEowD4EfY5g2r7bmXXrOPPv7YdR0QLx0j9mXKVRCMHHQfXI2NWg/Mm01CI48mrdo6oAEYfHE5Lf+xjHSbN2KADO9dOSqPggAANSJJREFUUUNnYB579I8/WOkyJaxb154iMPMhT+hYtkwI8ueff+7zlwUOYYAYAf6URc9OO+N0GQUlDv38Ev/9e+NtNQSWYJo2rIfZgzTbjKnU+vVqwAAk2+wZrjZp074jcnVN1NWxmy0a4w+n4MSjjRs2YAaiCubjHU59L7zYHnzoXouCYOzatRv97ZVWFS2hQrI2LKU/rw+eJyWn2KAXX7Ievc+2zz5+z7JYmwHhewfebK8N/gh0TleLggLK2nD+Qta2Edawyckal1GXDQUn/TmoyhIciv/JGptgYFGryHEWpFm7cqUtxjRsJeyu7NixvUWjPGy9JMbvthkLFyGyWcvWJysuT5Y+QP6oJMiEHdF0nOtA+SmMMYcK6LpE4mM/xjq2b9uGfn99GL8IW7WWBsEwplTGGmPxGmca6tWtZQnY1EUmsgyEv3N3gg37/AvF7dCxi0Wj1bYNrS62gMZigLdPn/NwAc4T2PBVyf519724pfsHKOMLHKG3qdOmWCVsBLugTx9r1fwkDIx2tz179tgSTJs3gFHQtnfJH6gJA5uRvNfmzCOennXp3sEKpyfa62+8jYHGsRgozbAUlI0rHEtiUJzuAPzF6vzQP7/7QOi0G0598EQBxAK/IEHzRfFADHLSGQT/rAWWnlySUwTwj358d8IlC+i+lJbxgxuk8sLn4OIrL72unHr26CVlo9AlxaOZrEKZbUMTNgLVEm+wXoZm54atrnlbrQpG7il9RECg88JXljYTJwexaK1x+9EiCDzn2S9C87Znj9Mk6ODhAfgXRl9/GKYLWUu5vPEQ/iwOa0o0Cync2Ayk+wsEGB6/gn9mulvCTIpvwfjGCjRn4xNTcNz5DZawe4fNmO8UpH6jkwjMNm12063RUUVt9dJFNn/OPKwV+CfmuRsIzJKFc23ezNkYa0m1adOm28qVy3WfA6pSIa2SozycuvwOx6d9883X9v6wEajZnbnIzMiyvf4A49Z1aywLTf8odMMS0LWYMhmtMLj6dWryXDXLwQ5Ijvwfif+VK5SxWtWrgTfxGH/YYCV377R3cYJz6TKl7bob/2GN6tdDZch8PE3PEkY0FvM0bVSfKIt+IjmpzHg+82moqtasq7sud+Meip/GjsU4S0XksVhLo+vd0gCJPUxRsxaO0PbsiWiFLAGNB95xm27HJms2b92GFuZMW716BWYvOB4F+4W8R2OwkVLDacdrrrlOtGrcpJmV5nQ2CNmqRVO0YGJQ7ky79193WuMWre2RRx7DdKVZNZxVaZGFrAKmVLUUAoaMlZLramBqFnTYsmuPeFa7fgMrhdYJp8l3xW2z0aPH4Yau9ZiGPVutQ/GMPyH8QRTQQSp1FPQXEEX+bfpHOkc99NCDjxE0CSXHjPgujugF1pleQQR9uKh5/ZAZmzeheMJK0eCHD36TQCqon5ceDEAwmi6sfX784Uf78L+fWHlMV5177gVWsRxHjbNs9Pff29IVa3CsV3mLxl0DPXt0I0R79533Nf/NQaDqEMIuHXENGpE5AL60FvDNXnv5VdQi66171+72DW497tfvEowoD1QZ3CYdpGW58uAvQqGJzHz1BzwEA01Zh69jPiTgqPCPhFDxHIEiRYvYlk2b0ESfZ5df3s+qQRDXrVlnEzEdWfmESnbbHQOtLIQmC+syps2YjWXb0VictVWj4B1Q01atVh2DW0ts8+bNWng0dfo027J5C45Iu9+qnlgVZSTnfFSA/GpMn72KcRrWwq3RqmrdqilCPduwbrUNx7RbDPYYcONRr9690JQubz+jGTz6p4kYYCuP8Y0KoFkX1GTFyUY5GrVcdyD/i5YobUXQDt2DVsC+BCyMWrPSps9ZaGefda79+1//wgh/YfGDbchRI4fZ2PFTrQZq9Pvv+qdaDzzez2eEk7+Ap4BZGgN5e9Ha4l2XS2EMVmOQciFOie5/WX+rdmI1ycV+DPwtWrZMd1psQteoV4/uWARWz4oVKW4LMAa1b+8uTI9ux9hCCZwxeakuwcE+IMyUbLALL7rIdu7aYWvWrEUTfwfGJ1I1k1UV/PnXXXdZHUwd50Amh2KNx6aNm8CXIpgOnWWbMQ52AVptV16CgU/QPpqE4n+/wpw5ay5aXkuwqO4EuxNjFKXQFYtFVygFrYT1G2MRLxuDt+dqjCmv/OWH/pG8ctDP0GUwB2gRMFEtKHZAWakMvtWRHDg9Vrnl7XsyXwbl0UeEuAEWIkY2q18IP7cqz88I/llZPIEnwr7CFBsdFb5B/YbWqlULy0ItPPyLrzATgJoPQDJgljt37qJNQqPHjoE/BAzxk5OS7LLLL9ce/gBHwYfBiYRlz8jMxuKf1prqKlGyFOb096PlMMPlq+lN15sikGON/+bNW232rJlWGMrRoWM3K122OJqkuKVp3XqN2BfDNN3p6K9Gs50K4s2dvcBiY9erL9ykcRMpLwfD9kLhpk+focHV6tWqYZ4eA5EYK+FUK5vRnIFg14aCOWXaNDWNOX5Chb4Iewropk38ybZggRLPQsjCGEo1GJsOHdti0HIUQpVaYy9t2pxsdevWVRr+0G4ejv+c84iC4ixbttKWQjnZ2uOAbd06tZCQo/Kc+mIGOVCivjb8u7F27U2325svP+fPl3BgNwq5/FL+OHzgoZxTMDa0EwuiWN42bcBX0FKtNuSbgW7QDBwXFx8fj1ZTe63/yEZ5ojEWsGQJy7Qc8lUP8tUKspiFbsE4LLTK0bLtsmXLWWLSfhgyLghbZ+lYNFYKi6g6osIphUt1IxCPxnMzDMKqNauxhyER5fSwoKkJBre5qAkoSgApfZB/yR8GGecvhNFfgzMpStupZ54J/DPQ3Ug0tmTYzenSBbMkGKCkQTnW8new/rH8lHs6GQVFwLcrCNDgOzRYT0SiAFG0KJ2y30yvAQE8mZnyUwZ+vsycngzDOy0mhDLwZaq87tfga/8+ajaSKS/8YPQ3BAOF1boIVWMBJB++OJSJ5tlmq4upOtak3DH3b8xzL1++HLVFGXxjWg3M0OnSLG4+4o8xNThuJybN+ePgc7EPHRcl6YBkn/5UeO7fYPcrmhwhjQNmuRQSROWJGQTSn5uYSBXOtohm+GIFRj9219XMRV4SY3hyMU8wZsDwKFgAD8tjJQVMdAT+c1g4AtOHkT588R8AM1EOtlRUDOATj5ZER9xNuQ3KMfzrUdarWwekISxfDlXaI8kfCkjHh0NSxo3HxwfyR/w5eBkVQ/z5x4jAHcqdDSujuIBL/HUdHuIEJM0r/1mIT2NI/LkIT05wHT15cjUVm4VRqHjDCKQtfQmbDkxHF5i01ZoH+sA4Eb7ono/yd7D+qYSaowembPzSOSsSQgu+rLf5TUSJBYSSD3qSMFREJtWDsfIShAkpSkjJOEzHnzwugC+GMAoiuqgQBkiPn8zPlxkwPzyUIUJ9+EH5DobPAR8KwqLFHEnHYFaL5ta6dSu9Dx/+pc1Cjc3tyuIQfPPCZ6Rjjz8EBpxw6wooHI5CEai5KWA8xITFY/eKIbARMgjUSz8hCynnyurHo0KKeDQ46P5gYDAv/SXTSkcJDPgP6uEzCnQnHeTgQVAyxGT6UfCfahHJml7wZfU0OxGNMrAYdFu3xuE4t6G2DysMTzvzHOvdpYOMjlt6zqI6+A4nfju8DoDPOOS//ydFBp1iqJgsP7NA+Yk/W1CahRF090P5ionhsCriMDLRQ7mVTL/8JmQGob8NOKQ/jXdopoCw8R92DK3UKLV0VPYQfKbmYCMQJ6kVgxCQF/NhzqArl5WzPPkvf0Q64D/5hiKpeCCkpjxUVLxTfhCRoZIDxgLFGZdOAkXmUt8orMiXqLu6hmkZhz58Am0/Av1FMOWnUOXJ5m0W4rFuFJGOFr4ydPBZCx4SPvLi6rKEfUk6Oqxxk6YwDM20RoDNRu7d5wg8FYwl/k3w/wT8qYCEH9CfzWp4QWDy0J/lgxVgZ4yOvQopV1748NPKOaKCwMPTn8QFpn6/nfBJfwcfLQouMWcUwYOKSHDBeyoE8qZjcSg9v8p/hqPESJWH/6QuE0fb3HlzbNS3o+yqq66yt197hQgry99EfzKMqEAOHHWIN2nJMjKQT37z3fk7+IygUD1C9AdNIrCR6/fIv5M+LhMWKrnwQV+Whz8c3HUxEIvERNlIS0qeC1Mpfxd8YnhI+Q/wRwSW41D45/If8NPSUtnqBmEYWy+klHP6hj8cAeot9CLv3B9AI3JAzcUMpeU3HEvjC2EoNwTJePKJf25ZKG2q70J5hFLkeQkiBXEPD59NxBw0EdPS9qMxEGWluAIPZUlLxThDVgamgLAkFSPrPEuQ5Qm5Pwm+8jsK/I9Mf1gAkTdvIf3S/mb6Ix3wo8I4QSLzyAhXq4sLfyr+zJsWjFJEJCItidODQKV4CVxHT/oIHuMxDtyfCh95HgCf8OAF+AfKvzO6Kthvhs8WFXGDBIfS8hvuT+E/88GfTx5mG3K/mf+Hwx/Z64xGQvKRIDyHhntTGZgeL+SVq/BdGBWZRFAclc6PxAL6/vIO0vsRXWqG0IMCjmcYvgicSxv3JpIF9AOdRH9ZLtr7P0h/9hPYxchLf/CBMJ119HkbwMfzd/MfmudX1ig1IErxCyD//wj+oiex87sEpCNdQD8xU5+OxvotePKPEkkEgAeR4RcxICIOA/lCdlh0OhfbhbEfxugkAeOpmagIgWApiRJJBlwk6b/rmvgEYbQCBN81nQog/iBirjLRIPxB+sMgcFBNjs1aKARbDWLqn81/8tf/o4z9GfznmQmUNBUVZf+98ke8XUPlD+JPeop+f235D3QdhAXToAdCh3KB92DZI/H0dV3yQiGiSDpKujCKFkmqDJmezsmsiwDCk3t8MDXhBYdQyKcAwRcNWE4ojJj8N8afXauA/jSGOkgUjNNMjgwGkC+g+LN4rszYQUnF/l3yR5kEs4O8SA/Kr8SVtCm4+B8r/XPrFKiQHMqmIviOr6SNHE0x49DDp1NAOD+Gi4bfIAtGpULReAR5BVaUkRUeCsBXvsL3S5YXvt+Xjo/fiwM7dkM5UG7UzA1POgmDeoj4t8I/l/40ArxOPg7r8GUIiHODhlCyHO0kJa9ozAsU/igSx4L27N5j+/ZhXAKuHBa5Va5U6XfIX5bFbdlhSanJuq2qNNYQVMMiuEA0lHlBw5/K4xRMxQt+8nq599+nfxwoFdOFN1/xRzV2vSIGwkEhXDhC8aI4pBodwzCAQ195iZqIgTlX5sZpmFBUxVIqTd+oJeIHcgyIcf80+LJeyBPCrakgFpUlgbJzdH1/UhoGGtOwQq+YlSxe3IebY8sXzLVncdLTLizxrYyVZyO++YYFy8Uf00m5zVRkyr4yCp+jpqPIgx9iQetJmO7PvSMuqze4HI746w0/pCmLxlf8eWgWi/4cO1B8fCkcoUeiv2o6xnOZHxI+8uX0GfPkFNjylcvsmSeftO07d2HJcBWc0fCFBl1zC5QH/0PADwbw1BUREgTuao8Q/30++MVy8F0RHU7CL/Dw5Y+4C3HCp3yx0FmWloxDdXB4y5gxYywhfp9t2bLRLul3KbbI3+ATMYDvPlkkOvJf9Af+XIbM2QK6MaNH2JChI3Vy0qWXXoYzFP6NOkrSAnjkwa/jf0j5Z97EmXCZB5+ErzdQh6j478RRaOKbfown+H663w1feTE/544WPjGmnoQcp7RYNHq5N8cMTdH42QtRPwXoFUJaXkzEPJGaI9tYMaJ86MspNhJK+QbNNPgJvNKRWAqVn3v7A/AJSQR3zHeLYgjfs59++Fan4yxcttTH38Xp2K2zlcBKIa5pqIdlt9yXT1yIv1gsA0gMiSYNDh5QLKZWDiw0QoU/4euNfj7+LBL//GY76XMg/REREZgvyQUqKgfRH3HpCFvw+US8QOgYxi3VOrWKxu9w8BFNrFBG2da+XRsrX6KQ7oWsUach5u0LIQL5dxB8H2oIPl4IPysQXsF3mbuxDwSy9FLuYAEa83TwGZ3hwl8I4fAXAkWm9KOycW6E1BT9RbNsmzZlvJ1//vk60en2gbdbFSxtdqc/ITtmrhTE3odPLziuo6SXfH14jHvlFVfqstw1WG3YoGkrxPTpz7goSMB/vtHx173hiTgH0J+ZIzTEf8ofJQM0kPwjlFFIf+cQVxm6OCH5J2ykOwA++U+ZQ8Ig+RHhqyQO0q/C90tD+FhZqzoJSQkGZNT8jHsnQIq6yEuJJ8OCb7yKcQSK6ERdRXUY4p1fmAZMz8Qa9XitOS+G3XJsrnItPKcAS+FADG44otjzgFAekvKH4QflIHwWjCYZSitBAw5ccZYQv99eHvSKzca+gU6dOtruXTutYqUySIFWDZCOw+k4dF17nWnJ2H2nI7uwL6BoqVKqwZktj6RjfUYiJO5PwjHpOIcAawNK4EiuIjhejRQUYQJ6It8srKiLT9inZnkZrKJkHObNFkthpCnBFgsZDzpnZmLnX2I8ih6Bsxax/wNlp5FNTNyvZd7RWIjDlZikHdMnJSZpL0WJEsWRawbWZCQL1xLFS8g/C0u8mZYrRMuWLo1Vc1pkDEEFztlptmUHNxCZde99KsqU7HBGmbhjT/zHKkrxH4qalYkuB86syMCWc147XxKbnIKTrLi5itfLcel5cZw+xJPU9icl4rCWTBxaWtSKomVGSSHylK/Q4h3gnYY4ydgSnY2VkEWxA7JEKeKSK38c/+BS9rdxqhRF8RKc0l0VuxQ/+vADlA28kHPS5N6pRHgDDVmuJJSDPOAp1EWLFLWIotgujxbe3r37FY3Lv5vWr4V3p8hSdsBJJT1wViiAYEMXDnEBLTIyM4xL5VXJQQY4msZzPMhPlln4gxekT9CSSk1N134NHZGP6e9CpC+2xnOfB2Ey/2zQNCkZ9CJtsayf9Ofycy6tJh7MOxXnaKZgnwaXZTP/othZWQT4aCWm6MC8SF+fyEwGYlM1HRR+yyuX/n4I9Q8HtzI5HVPil7SFWWYxmWfgzzcykQKizKFsshGIR7YJMUUgNEQGAju377I5c2fbNKzNb4UDOXjwCQ+4/HH0DxaPc+9PPf0M64b13hgmQl6uIHnhc097Jtada1UjgnOQJ1mu/4AVwOcy4MLYJ+xW4LlwEVAK6cqrdCj/pk2bbdRXn9ocGAQK3sKF3FCzBucwDCCK2KW43eYsWibhrVmjqo0b+5PNmj0bO/5qWN8rLrcK2KDEbC0nE0KcbauwTXnmrDnY+pqMk5H3ay39qVCsQti7wK3MYjbKmsLNO0sX2A8//oSTecrYNdfdgFOAEnFfxWT4L9H++4txlHmxYpG2f18qDlWZj3MfJqGMRe3q62+wyrj/gItqRmIT16qVq7AHooVd1r8f1s7vshkzp6Omm4nDT07CBq9+2BS13n4Y/ZNtj9uOxVodcER8L5yzuMAmY78/L5m5oO/52O3XiiZNtE/Yu8MW4yBZ0qNKtSo2btx44DxLR7kzPy4JB+FVW+/EeMsynBmwHCcIpaEfzr0S55xzrjVu1hjrDpJx4OwcnbVYtXpVu/rKq7GVe7t9j5O447ZutbMQr1OHUySs5CSdkz/P4rbt0J6P2NhYbN/O0HFmvXBydO3atbQYiMq3H/sRpoz5xr77aTJSRkDJ0+2zz4fZDTf+A/zHyk9sNCNG5A9F0d1f4Gnl5Cps2161CmXGNvV0HJDaFScstcf9IeyqLZyPQ3LhKlatbSedxPEUp385MKQbsL9hHu692IaNaJS/Fs2byQAnJWdYv/44PAb7cjgHsga7Ledg9+quXbs0NlGkWBHr0a2HtcDqWYpLPDZwTZ06FbK2Glvgo0CrJKtRsyY2Y10OpYfCY23D3r0JOg90+bIVMh4ZGenY4t9V507GoMLh9uvN2Nw1a+YsdJnisETb6UcbLBM/tVcPGXxeMiOANCAkBPTKrZBkKcTGQ9A/CEE4hAJjCi4yhddXOWVEi+FWghEIM6e3i+tyhymgVsrLAZZ/8A0r9sGH72NP+lidTMyDTb//7nt7+93BOH1nm82C8Az+YIht3RgLorC0hO9nx3fA37oFpwkth4LSqgdhAsJ4qKcBn4NjVU480Ro1auSH4MH0kApZSmGl4gsPnrz707hJKnabNh10mhDzp6Xnv0ULlildnVq17MuRI2wxjpefCmVKg6AWq1zNrrjgbODt7mz48cdxdvPNt2h77iOPPGJDhz6Pc/qetZ9/noCTenkWArYag99ceTh7xmT7v0cfsLWbd+rsyYii5W3nRpwvAOM06eeftSegQaPGdkq75jZh7Pf2+NODbFvcVovHcWBZkUXs4XvvROsqzV4c9KKE+9kXX0U5PPsUB9B889UQnNa0VgLPcZAh//0YtNuqQ0jfe7+UffDBBzbohRdsKwziJmwppvD+gC3C4jkMzZrVsbYvKQeGqY5NwQG4U3/+0RbDiOzCQF4RGiWsOGRttxubpp5+6hmdP9D34kusbYvGNvDOf9nXOO1pOgzTuHHj7KmnnoISbsEAYCI2+ZTXASZjR4/GyVBbrU6TVtapIzgZDGiIC6Yjzf5937/Bl5/sJhzpXr5CWbsf+1KaffYlDMooKwn+0Mjv2bbR3e0BFsfguLZu3dqrZcYlzaKzv+qN+zS4WA1WGSdR77H777vPxuCczRuuvdaKlSxug54fZMm3pVgHtBJZhElTp0h2OnTEuZngFfdsAF1s5lpi/xhwq85puP32281DBXX2WWcr7q33PGqXAQ67SXFxW6x//8tlfF544Xnt2v3PM/+Bcd+HMxdboCWWY6/jGH3KRj9s2DsbleF/hjwDQ7hN54VEo7u6F7s6X0Ocl19+2c7FGQ5nIc6tt95qL738ki3CaVFVcEN3BloHt992u02YMMEefexRnEdZEdu2H7XWqLR69+qpcunnsPIPdGUswAPEEfJ+KskCdEXqi2avh9rDQxNUT76n4I9P7M/3UlJSEOa/+3H0DT+uhlQ8vDON809GumQPW1s93Gfg7U/Y7VWoUFElwDXl3qZNm7zpE8fqO7pQYW/jxjjkkYy0+w/Ii/kO/2q416dPH++y/v09bIv1Lu1/mdcf7/rmE34XXXSR99Zbb7kyJ+eWm3jlli/NxynRS0pK8tq2bSv4Tz/3ssd4LHuSj9sLTz2sMDTNvSGfDPFSkpO8JnVr0mJ5L7/5gcqYlpLsTfvpR8qDhyPFPWyq8uITErxrrrleaT8ZNgx0I/xED81dpEnzPvzvO17chsXezf+4SnEaNG7pffPttx6a/V7F0iXZ7vE+GfE94iZ6n370jodZEO+i805T3LtxMTh5sTV2ub7BUm/SzJnIP9nDqcFeckKch0M9WDl6bTp09pavWO4tmztT5SPnzz3vfG/f/v3eB2+96mA3bS580QUA/vu8F556RPCLlyjhvTfsS9AjxXv4wbuE8+WXXy6+Jibt8wbcfIvSX3PjLaLDts0rvVKlSnlMtyce/AYf4vfu9c4443TFa9b6FPH72+Gfe5jF8abNma10TmbAE9AoOXG/N+CmqwV/wC0DEZ4C/3ivRYsWgv/duHG+/O1DuZK8QU8/KjwbtzjZS/LlD8fae9hG7csfZSDZ279vP/Le6cHIoCwR3tX/cGWm/FFmxo+f4CWC78mJe7z2rZoL/rODXpMspKQmeHt2x3lQaKV94/XXRf+dW1cLL9J52LdjvOS0FOEzdOhQ+bcD7WGMETfFG3jHQG/69GmgbzJosseLimJHyrxRP0/2cO6oN3rUCO/Rx8BX6FdqcqL39JOPK/zUU3urXDt37vBwOpQXHVNI8kX+L5y/UHEqVqjgrY2NBZ2SJftDhkBOgQtao46+pMsh5T9XXymT0nPEIz/y6j+agPBMg0EQgR2SuRHc98FhgVEIDEpgGJQuhUqQIsRoHJbOni7hjC5UyJsyZSqAp3gfvfumGN6oWQsvmfDhlwoF0vOAAv42+KnJNGxM4yNMpvnESRF+Sd6WTetEWCr56NFjPCo4hStRhEz0LjirlwTk+htvV14JuzZ5FcuUVBrGZ95Mc97p3eV39z33eGgSeiO+GuEVLVbBq16jurdo7QZvv+gKWoBRyYifmpYAA7HX69CKgmbePffei7xSvY2xa/QdGRPtTZ49S4wmjRITE7yGtasqbNhnw1SWHyFINB64e9CLi9vqC0EyjE1gLKK8T7/8VmX8AXEJp2iRYt7idRsAP8X758ABUqgLL7va53cicNnn9T33VPGjz8WXwlhQQJK8++65XemvuuZaxV22ZLG+CX/C+PEezjHwHr73Lg9dC++yq64Vv8n//ft2yMgR9hsffyrFp3HEFmaVS7ICRaAS833r+iXKl4M0U2Q0Er2UhG1ei+bN5f/hhx/6ZaVcJXqXnX+OcBhw54NeIo1KoAgqt8//FMf/NUumKY8YyN4yGErxDvGkiOJ3qrdt0wofvgk+DQUrqK8//1T+pcpU9eK2uYprzWLIMvBHyxI8jkU+lNlkD5fieJjKVNg9997jrVq13EtEZZBI+UP50pJ2eic3a4z8Irx2HTt7Y8eM8dDNEP5pMECUsdKlOCdv3pAhn3i42cx7+YVnPIwneB179PKNX5IXu2KxV7lSZeGPc0C8ObPneLjMR/m4Svnw8u90GLKbF3/h+kv9jxQEtL3UmmDbCQ0InfgLc8i2lPNnedGnwa+GRBSGmGxquQYHQujJLgXacKhCuaWXo86r1ixX76NajfrWtCnuQzQcGT5tFhJHWFf06RAVXQDGJewD4Sdi/nwbdtJt28a/HWpubcV7HA4r3YamVxxO0t2Ob95yxEEXwWc52HxiiYgc3pl/NroZ+LDYtasVVrhQUTupUVMWAwP2GLdAcGZmKo7Ymg0/3Ch0LpuJ2HuP8YVdaAoXw4DdSY2b6VDPjLTd9u2YiQiOtLXrN9h9aJ5CGOy03h1s2NBh1qBaZVFL3S1NVQIvwN+3Z7uOHeOU5lkXuLMMFmNTEF2lE2tgkKuBWtYept02b95iq2LjMJBUSIOhpM1SXN/GZt4pHTqiaV7OstF/5I67xTgfgK5o8bK4zQmHlKLtuxxxSYP2p3SyOlUqY4Y4xSageU58e3XvJfozTQYOOJ0/dzHJZP2uuBY9W/bJ02zlcp7/GGEnVK+Dp2ezcPQYXRGMlfCeieswzjF05Lc4zeh23MnxlGBhGNc2rlpru3C+AHd3dkbznMeye7gQhmcdMJ8ssgT/tIMSPnNnu/588RK4xKUpzm1EHolJCRa3aQPeIzSQyHTEPwOHzUxHfOLQqBmOOwNZuWvROT4hR/hHZlL2Zs3mrljkUaMB+u+18E7ZdvD9F9w6tgH+2H+Bw2NaCj7jRNqYiePgG2F9+5wLWpdG0gybMmMec7DmLdtZ3aonOPlHl7pLly4Y17jBTsSRbc8/9zwO273M1sfGohgsC/CPLG6vvfKy1a5Vw1YsXmCnn366fYauUbCScjFOy96/PweD7kUx7jPDbrr5H/ba2+9ho9g19tY774knXk60Va5Vx5565mngUh1nXYywnqf20pkcLL+b4/PxP0D+AV+qQRr6+EvpUDb64OH0j19O/zAELTLigUT6D0PgK5AUSmldHL4yaxeVfiQRZy/IDr67eE6yERcEmTYHTEeEVjwEA7fnZKftRZ95jHLqcVYfnIGIEVQIsUZ6QWDl58OfNGkijsP6BiO5nKFw0JDQfwMCKAj3/fPC2ctxqq9GlkFCwWd5hCNKrKgsZYRO0yUOzZq1woxDOQ2UMSQqxrN1uEtgF3ZSRkZGW+NGDeGbgRN1ZuJp1r3XaVa5Ig6SxdjI1q3r4INUUSXtGsBFcw5XlNXGttnCGhRlB1enIvrwOaZgXrRTXgCPKVbOalWtjhxwIjMOPqHrhENjYkAfLxuHzWBaca6UkEfAN8QpSJUsFaPuE0ZPUNyePbrKCEZGYjcjjM5ynD5E16ZNS9xOhFuhslJB4wlkrbWGwEZBSeJxivWi5WuFWxecmJyJQ20wNmcb12+2DTC4nM1o3KSB6L8nLs5+xgBoDPxO7Yp+Nkq6fvM2wejSraddCIN2w3U3Wu26tXS2YBnMZuBYE2ATDUWEkQPg2hDgutWgOOQp6wkWhg74U0Ydpw0X17iZnlM6t3HCj99lS2FYMGiLrok1xcAe4fPfVhx6sgGDp3xv1bgh2QpSYyQILwfIH8cTIFPzcHQdITVu1giVj1MOHpvH1Y+c1YnBLMScWbMRBzNNGBxmXB5GG42W/oplsfpudjLOjIR/OmYuvhk5Eu+gc6duqkT+v71zAfK6quL43V1EhALloVmIPJaS1URBkJciaE5MTROmoVgh6ExEmDP5SBSJR2aoURm6+EiaUFcehYqo+ByQSERRwYCEQMspSQtFJXks9Pl+7/39H2jNOC1sY787u7/f/d3Xuffcc8499/E/d7cWm7frh3U7MPA6OQw+uW/4+nkXsKj7Qqi9bVb4ydRJVKw+bMYE22d79cX8/oJw3TXXhlosi112+YRw9tlnsSZyUHiR7W+hp6amezjrq0NDm0NGYRPycPfJgezWaMCrZ9Fxy1tbWVw+J/Tu3i2MGT02LH12ZZg05WrWYk4xPpNIAKZwXU7/BqDKJ/zLY7IEsFOX8D8LtLCyQmMS/OqyWKhLUBQZfRSUYM0F4oIjpfuPh8pQHnmVFQB61SNdH2MlW+GycKOwLa//FSLE5iAfNUcexnVhq0LXrtVs7/DLRZVTAl/Xi+miEO+bp7IFv3CwxDkCpse4AIR4fjFPCC7BVxa1S0VWevTYgzWd9YZ9ZMeOrP5vxQLP2tC5cycumznUC6LK2w4LRtUQtOSv7i1Q/bsf28Nbldr6a8H2ZFMWh3Yggnsc3xMt4kBvRT04f0HoRJ1P6tWd5kXNSu0U/N3ARwV3jWRdWAKJlauwAjuBcj17nehLR/ZwUu+I9p9kNyEKi+PRrpgfc4HJE2HJcxhRpf1d0SjWrl3DCb7WoS1m4Bc/uTiWwYq6MLB1y9/CsudSuT17A78+rISAqAqHtdpggLWNdzx6HHcMC5RiHAyctmsfOrRrDZNvC3Pn3cfCY70vK+k7oD/13ONtVqVrzZYc83139WZMlc2ePSdceOG32TVhMRA8acFRnn79B7ku/nUCjCsu8shJbNxpUm32gLtmvENo3rQFDFnvn7HPqrvT1rN0OcrHsUgkKSI6XbNutdt/UIt2GLzt6u1OG5sh7n30R/Ft27Rx2dvY5mTtwvXbsGGDd38ktBSwyoZpZWOjV3idrWgZnK3u0ik0w1KXHEoD+H83LFuyLDz6JBou+D+WBVbtook2V4E/LbBOufqHDByfD+ePPDdcN+3GsJ3dBbkd294MYzALP4URvgZjrpdMuCrUzrqDmG3sBG3jF6LN6A/sP9KCps2bQmfHQKtVPqn56zvne4epbds2YdP6tWHc5OvDzbfeGKq7HYtQuSQsPXM4C9Fvaznb7RcPuJPL6J+gRH/Cv5NkCT0Iu0lktIgw//Gz/FiSmF5phVyFRB9oAwlEoQU5kgB1T3TqVleiIG1SBC+N/FveeCus+v16EFsZPt1NUweUw51NMW3VynuxczG9dv+CBeGee+4FOTIdXg5f15x37lId4Ss3AFWXzJms/JBAFvkVBYOTxSwWnNmJuFYtDw7NmDq8gsXoutmzw1RWhBctethMuvyp37kOA/v0hqCFiaqwq0kLE+i6F1eFieOvDP0HDQkjzvliGIB9w+WMvJdfcRmXxZzENtFTmPZ+PNxVdxe1kJl3U7LRg4+OqQzLMQ+m6g449bRI0DR4F8ZWteX6h+efDhcvfjwMH3YmpsPaew9bKvcfN66zVeutTJGGDv0yaudsLlJZGmbcckuYPGkiBNUq3PvgEy73hJ5MHXAbsGT85tv11rCOrukaFTcMiuqMgK5Xm1F7e3hm+WLw/htU4hU+IrydsyTLljzpMwYTf3gth4Hah9raGbalsAfhPnhgP9+oNJ8+O45LdGRGTrsa1dXVEFKcZu5i5FzO7o3c8dyTqA7RNFIHYkSYUaATGVU4vrGAffpA2yxcyuUwK59dHTZuXBvm37sodO/Rj2nZOBCHcEX4apN88TIEDgQwYGCf0JRdEeHSjw+iP6IGnYytybY3h6VMBWbePpMRt577LqZzTV5tkFAwbXOUQVOdNQxOE8dfxVRgQKiG7o46ultYvHxlmDe3zlaY//Tyy+GMM4ayw1QXVix5lBu1ZnIRzpVcAfcYd4LeHvqcPDB8psMRYeXTz4KnQ8PpXxji9q9fv8m7G9NvnRkuvOA87F4+Hg5ueUj4HLsYrUmncyl9ep/iHawX2L6/4Wc3eDetru5u7ueoRJsYBhqr2IVbw67WQ5glnBP6ntgrPPDIIvfb8GFnmVZpRsSF3rgPpn+SEBFvQ4OP1S9mKPF4kf8x3Dp+opGrUiQs9nIxjkgKECTwHzsjPYvJCxFZgrBp0yucSXiIC1Q7hPGXfTear2abShJS6wAtOfxRO2OGz6y7Ff8FfJ1VqNQerVx6yatmib3l28P2WzemDbrw9DUsQh/IiD99+nRGhi5E7wxz754T3uII7QhGKI2Gcq1QYV9if3sHdzWMPG9U+MoZXwKRFWHwoFPCOxjclI3/1atX277eTbU3hS4qy/D1kICVFJd3R5g585dhJ6rd2NEXoJ10JrCKvfEOYQ02DFXHMWO+GYYMGaKahk7VR/lSUx1UObqmJlx66SVcbtqa0TLeLDVu0uTQC+3rZUy/67ozHWS6+KIxoTVrDU+hFbAIxUjfK4z6xrmYnW+C0Ds0/JmtYN272bW6Y5g2bZo1ME3PRo0cyQjV3XccPMN0b8SIEWHq1B/57ITrDhI/cfgR7OF341DUW2HhwoU+XPMt7qq8YtyVlHOA2/zGG//gHok6TK83C2MvGhs+xTaalFRpTUKChYJxwWdESjjoY62Z/g3gANlfws+n/xRcvxRGf+ficM2U8WgJ2EMkfbzKZ1f4MfP1V5jGDB3+tXCqNJgyR0qXqQx4qPPh4LYjWuBrTI8efeRhzLofFq5na1a2FjPXiqnZurVrORD2T+x7Due49DCiIv7ZqWD7/NVwFCO8Lm2RMdznsQotN378Fcbv5s1/53xEVfgVfTtn3jwPfhO+P4EtxdOcbuu77zFn3xNeWLE83PKL2xB6G8NorvP7Hv3ZXFuttK4lPNG/X38L5AcWPoCl6VfD+aNGhanX/CC0kgYNw77OcW7RyH33Lwiz7riTqUVzNLSxXA04Wk3FgeWs/dmb0HL6TxEl8crnTyVM/FehVcuMaRQrSSKXTRcyTCtjBjySeowphjtbfCgh0l3lSohI2iuPgi2pUCUdwUtGWZVOdul4NRx8wUoFRuUS+K4TI08JfNcPRq2oRF2s4FCz647mQUcyk/PJP4pSSYX27+ZUZAWE4KYVwpUqOakGOMPn9uUIfydvHZoGfhTXFCh/TOe60au7UaE1ovkknDtJUMrhC5c7IJRK1i40/42WfMClpD1hBJhQBV+nA5GX+GI6t0PwabP6SEe0ld5Mqrqo/bi9+38n1qlYxSdGpVoP4q0vpQUmHtk6FIOoPDmPQrSpgH+F8R/bj490WtRWCglIRRbgg/9dqf1NWHzc8c6W0KodV7FxEvOJ3y4NJyC0VZbL413mMvipwL3h7wInVawbRfgqo1iKfMK/2i/TahFCrJdpxalpPzC0PqHTkQlcsQoECCXCSxXTzL3hl7e/BD7AC+0HvqsVH2U1LMW/NORK4bzUpQq5XRT4YeFzzFktSCWqIcKKnBiZODW+iLJUvMJSYCF9zJWyigG5YpwKQ5VIIJm7VqerOSwOcoRXc0wSGYYsHtk1KHwt46XFlEIDmQ6IIFj00zk0wTc+DR8KTPDr6WwRnxhMi5c6CluPmqcjuTZwqqqrbbzNBOBChGQ/DakQo4KYcvgQIZh0eSJIEzz4pT5Cuo7wyiaj0KpOVs11b6LwXyA86rAbZjbDmhEZQ33BgJVH0lMGuNYRbLWBrCymccQXZo2ETwNdMko9WpPajxTyQR9NYZxBFbBjBwmCzvo/El4sR3N/2yLkrQprB0ACQUZahQfRjRZrY/tjn2dlqv9ZYCm03wfgqJZxTj+Intw0cH1Aor9tWMh6mJOgcjXH9AwncJpTLdaJxX9Pf+X4V70qEfqCX1URNRs12PQvQYqzUEMQCG+if/WJ2p/1v9I3qcTitfpeeCJOfrU7E6ql7deOlyhb/aX2ZZkskFL/Z/zn+pFadKny9CvdUvyrTjLk24RTm8K/1h1UhxhepD2Suf/fT3+OIT05yvDPtwqRS/SPpvAeeE0RqqjwowanV0ZOLiwGuwznARNxrp5KTYhyHgkbOb/0SIUWAuN3LJ94Jdmf8MUQjFLqNjOMCEPaSoO3XwWWtl+NLH7v+/ZnDaLH8ELr+xl+OTx9fdj2z+dOijGc7tMulKZrR2odAxzKJH7D0R/CWYJRDPP/QP+pkR9Ef9GeQmJmdZeQrYSx42KIdRqpmw7mwUgkno/kpnA+xGSFfGKzWJJKKLoYprmPCNT5lIc/84mC9hd8aTAWXADPRsl9Ad8NpWC71H7ojkEfuGr4Pm5/URIAFPi0+X8C/x+i/xcuvJ9tv+1h8KmDva4iCtVFs7vRRuO5mAagv6QteOuylCDdPyL2rJ9ELx8B+v8P+K/QiT//4i+tCEd8iFhLmbz4Zdr2Q2lccjGtJLfy6Y9oLbAIyQrzCrSKFAAEiFc+XQRpQLJXRJVRYdYJldgfyqQYf/nDD2dOoSk2hw+OwIX+QM9HEv8FQpBHLqMR3nn/N0j/W1Mo4Fk45gPc8iqGZmiPEVlnZN0RR5+oLCAA8Ehp8GiUOswqHuGFckq6MoF0nPw5/Bz/Of01Lv95hilmtaM3PMKYMxVFDH9aH1Y1vZqcTRMUpH+HR39mxCEKBIVJgEgriIJEeoWKjqEpu5IpXC6Hn+MfYvCA5MUPPvjL6U8csv/4L64pmCOFf3WHKoAT57pPYpirJaZVCuIkzTUN01t54vZTyqf8BaecaZqBV3kNIwkKPgouh5/jP6c/8QuuEfnPmkJRDogtcaoX/1Fr0JRA4QpkgYcE9qExRIHAVEG5VJIz8y5zyukUxGcJIH6VY9WBxCoQp5ROoW/+c/gRJzn+c/rbn/wXhQLcF9lVi4LwY+Ld+I5nFTzaE25G5eFTauQqrB2kPJHD+TCj+0EeNcklkwM/XkHUYqNdDh9syOX4N5UkWsrpTzTRCPwn+wPmYPWGOkNcr97Qv/y4+IzRDkgPJc/iFOsdBedR3mKkBYDCkSDeIXMmEuhdukahPDn8iPsc/5EWimRkkuKz4EpIjLCc/hqK/9AUEltnqr3FM8wrjEeujSdis3QpuWI1lYhOgegO/pYu4E9HaezTQpGnGhRaaWvDSkOe0oQ5fOMrqmk5/nP6y5hI2/lilcR46aXYfcV/nj7oSKQWBRI7w8BUgTCN8BYM9pu1ERaqIf/pZc5OB3QKfJ01QCUSGOfEgkBq51d2SwRCUlgOP8d/oomc/hqZ/2QSy5wpPseXfWg3Ic75M+bVm0R2caSXpPKZpxSqzGJ2iY8oAIhwFuWVUsLbOxYxf5bN7xx+jn/RUyKGnP4aj//EqebbrBPUKWJsMXu2QChedirmAJlY8BqhOF87CNIGUodGfUJ5yaNAnDWFlCAqCipFCRzrVw4/EkGOf5FFTn+Nyn+yeBx/CQeHSv83o4pZcf6OAZFYFcZ/aRqlk4PbvW6QJSjkVYYYH/cglTkVwktnVHL4Qil4yfFfTlsFGipQTInHVFV85PTXYPyHZbM0+ntboLRPIFK43CydCYEYREfEgExviGmQ7p4bxH6yJhFzO7lPNUpNcO5UINpDDj/Hv6khpz/zSeIMc0lj8Z8mCWZUHzjAx0QgSWOFJ4JliqCZglxMHeMyISATKmqMDyk5gQSEA/R0JguJmMgDYpyaqNRYYg4/kkOO/5z+zBKeejcO/2W8zppA7Ayzs+gTXpWZ7Ex8JV73J4EIAEIKedLSApqCC4z0HRcVlViZrQpKM1AAjSXv7rTm4JBCWSTI4ef4Fxnk9GdeET+YbfRIvLUv+S/+9kEM6a0EsWd0JfDVO9SKNKpQkhMZ42bp9Va0K1/wS9JFrSHGFVPYlwHJ4ef4z+mvyDyJj8QjdvuZ/6KmANBs0I4V4ZBRViG9YVrHJ8HgNBn3Ky6JAgf5YZZXRscoKP47UiX6QIaliD5y+Dn+E2nk9CeGaFz++xcON7s1QvJWJwAAAABJRU5ErkJggg=="}}},{"cell_type":"code","source":"# 'macro' option is to calculate metrics for each label, and find their unweighted mean. \n# This does not take label imbalance into account.\nfrom sklearn.metrics import precision_score\nprecision_score(y_test, np.argmax(y_pred_1, axis=-1), average='macro')","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:09:28.367381Z","iopub.execute_input":"2023-07-07T05:09:28.368214Z","iopub.status.idle":"2023-07-07T05:09:28.577164Z","shell.execute_reply.started":"2023-07-07T05:09:28.368175Z","shell.execute_reply":"2023-07-07T05:09:28.575951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# weighted mean\nprecision_score(y_test, np.argmax(y_pred_1, axis=-1), average= 'weighted')","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:59:57.485071Z","iopub.execute_input":"2023-07-07T05:59:57.485858Z","iopub.status.idle":"2023-07-07T05:59:57.689396Z","shell.execute_reply.started":"2023-07-07T05:59:57.485819Z","shell.execute_reply":"2023-07-07T05:59:57.687923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import precision_recall_curve, plot_precision_recall_curve, plot_roc_curve, roc_auc_score, roc_curve, average_precision_score\nfrom sklearn.metrics import accuracy_score, f1_score, recall_score, precision_score,roc_auc_score, make_scorer\nfrom sklearn.metrics import classification_report, confusion_matrix, plot_confusion_matrix, average_precision_score","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:37:22.839727Z","iopub.execute_input":"2023-07-07T05:37:22.840220Z","iopub.status.idle":"2023-07-07T05:37:22.846525Z","shell.execute_reply.started":"2023-07-07T05:37:22.840183Z","shell.execute_reply":"2023-07-07T05:37:22.845273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"accuracy_score(y_test, np.argmax(y_pred_1, axis=-1))","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:34:57.236554Z","iopub.execute_input":"2023-07-07T05:34:57.237358Z","iopub.status.idle":"2023-07-07T05:34:57.279483Z","shell.execute_reply.started":"2023-07-07T05:34:57.237317Z","shell.execute_reply":"2023-07-07T05:34:57.278416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clf = lgb.LGBMClassifier()\nclf.fit(X_train, y_train)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:44:16.287825Z","iopub.execute_input":"2023-07-07T05:44:16.288262Z","iopub.status.idle":"2023-07-07T05:45:20.784820Z","shell.execute_reply.started":"2023-07-07T05:44:16.288207Z","shell.execute_reply":"2023-07-07T05:45:20.783858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# predict the results\ny_pred=clf.predict(X_test)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:46:06.996290Z","iopub.execute_input":"2023-07-07T05:46:06.997057Z","iopub.status.idle":"2023-07-07T05:46:18.261075Z","shell.execute_reply.started":"2023-07-07T05:46:06.997015Z","shell.execute_reply":"2023-07-07T05:46:18.259904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# view accuracy\nfrom sklearn.metrics import accuracy_score\naccuracy=accuracy_score(y_pred, y_test)\nprint('LightGBM Model accuracy score: {0:0.4f}'.format(accuracy_score(y_test, y_pred)))","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:46:34.036822Z","iopub.execute_input":"2023-07-07T05:46:34.037567Z","iopub.status.idle":"2023-07-07T05:46:34.097528Z","shell.execute_reply.started":"2023-07-07T05:46:34.037524Z","shell.execute_reply":"2023-07-07T05:46:34.096176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_train = clf.predict(X_train)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:47:21.587154Z","iopub.execute_input":"2023-07-07T05:47:21.587681Z","iopub.status.idle":"2023-07-07T05:48:09.473013Z","shell.execute_reply.started":"2023-07-07T05:47:21.587638Z","shell.execute_reply":"2023-07-07T05:48:09.471824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Training-set accuracy score: {0:0.4f}'. format(accuracy_score(y_train, y_pred_train)))","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:48:37.458890Z","iopub.execute_input":"2023-07-07T05:48:37.459608Z","iopub.status.idle":"2023-07-07T05:48:37.575192Z","shell.execute_reply.started":"2023-07-07T05:48:37.459567Z","shell.execute_reply":"2023-07-07T05:48:37.574033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print the scores on training and test set\n\nprint('Training set score: {:.4f}'.format(clf.score(X_train, y_train)))\n\nprint('Test set score: {:.4f}'.format(clf.score(X_test, y_test)))","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:48:50.018189Z","iopub.execute_input":"2023-07-07T05:48:50.018920Z","iopub.status.idle":"2023-07-07T05:49:47.851489Z","shell.execute_reply.started":"2023-07-07T05:48:50.018881Z","shell.execute_reply":"2023-07-07T05:49:47.850088Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# view confusion-matrix\n# Print the Confusion Matrix and slice it into four pieces\n\nfrom sklearn.metrics import confusion_matrix\ncm = confusion_matrix(y_test, y_pred)\nprint('Confusion matrix\\n\\n', cm)\nprint('\\nTrue Positives(TP) = ', cm[0,0])\nprint('\\nTrue Negatives(TN) = ', cm[1,1])\nprint('\\nFalse Positives(FP) = ', cm[0,1])\nprint('\\nFalse Negatives(FN) = ', cm[1,0])","metadata":{"execution":{"iopub.status.busy":"2023-07-07T05:51:06.298361Z","iopub.execute_input":"2023-07-07T05:51:06.299133Z","iopub.status.idle":"2023-07-07T05:51:06.394273Z","shell.execute_reply.started":"2023-07-07T05:51:06.299095Z","shell.execute_reply":"2023-07-07T05:51:06.393133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#from sklearn.metrics import classification_report\n#print(classification_report(y_test, y_pred))","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:10:06.996782Z","iopub.execute_input":"2023-07-07T06:10:06.997799Z","iopub.status.idle":"2023-07-07T06:10:07.002946Z","shell.execute_reply.started":"2023-07-07T06:10:06.997744Z","shell.execute_reply":"2023-07-07T06:10:07.001886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"> #### 💬 Creat inference table (test dataset) and make prediction","metadata":{}},{"cell_type":"code","source":"test_defog_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/defog/02ab235146.csv'\ntest_defog = pd.read_csv(test_defog_path)\nname = os.path.basename(test_defog_path)\nid_value = name.split('.')[0]\ntest_defog['Id_value'] = id_value\ntest_defog['Id'] = test_defog['Id_value'].astype(str) + '_' + test_defog['Time'].astype(str)\ntest_defog = test_defog[['Id','AccV','AccML','AccAP']]\ntest_defog.set_index('Id',inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:12:42.191200Z","iopub.execute_input":"2023-07-07T06:12:42.192247Z","iopub.status.idle":"2023-07-07T06:12:42.774684Z","shell.execute_reply.started":"2023-07-07T06:12:42.192186Z","shell.execute_reply":"2023-07-07T06:12:42.773554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# predict event probability\ntest_defog_pred=clf.predict(test_defog)\ntest_defog['event'] = np.argmax(test_defog_pred, axis=-1)\n\n","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:12:46.328432Z","iopub.execute_input":"2023-07-07T06:12:46.329742Z","iopub.status.idle":"2023-07-07T06:12:50.140779Z","shell.execute_reply.started":"2023-07-07T06:12:46.329693Z","shell.execute_reply":"2023-07-07T06:12:50.139513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# expand event column it to three columns\ntest_defog['StartHesitation'] = np.where(test_defog['event']==1, 1, 0)\ntest_defog['Turn'] = np.where(test_defog['event']==2, 1, 0)\ntest_defog['Walking'] = np.where(test_defog['event']==3, 1, 0)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:02:45.803110Z","iopub.execute_input":"2023-07-07T06:02:45.803891Z","iopub.status.idle":"2023-07-07T06:02:45.817018Z","shell.execute_reply.started":"2023-07-07T06:02:45.803851Z","shell.execute_reply":"2023-07-07T06:02:45.815655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_defog.head()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:11:07.183471Z","iopub.execute_input":"2023-07-07T06:11:07.184664Z","iopub.status.idle":"2023-07-07T06:11:07.202494Z","shell.execute_reply.started":"2023-07-07T06:11:07.184618Z","shell.execute_reply":"2023-07-07T06:11:07.201406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 apply the same process for tdcsfog dataset, but I am not going to train another model for tdcsfog. Instead, I will just use the same model trained from defog dataset. After all, we can develop two different model because the data distribution is quite different.","metadata":{}},{"cell_type":"code","source":"test_tdcsfog_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog/003f117e14.csv'\ntest_tdcsfog = pd.read_csv(test_tdcsfog_path)\nname = os.path.basename(test_tdcsfog_path)\nid_value = name.split('.')[0]\ntest_tdcsfog['Id_value'] = id_value\ntest_tdcsfog['Id'] = test_tdcsfog['Id_value'].astype(str) + '_' + test_tdcsfog['Time'].astype(str)\ntest_tdcsfog = test_tdcsfog[['Id','AccV','AccML','AccAP']]\ntest_tdcsfog.set_index('Id',inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:14:37.250471Z","iopub.execute_input":"2023-07-07T06:14:37.250851Z","iopub.status.idle":"2023-07-07T06:14:37.275383Z","shell.execute_reply.started":"2023-07-07T06:14:37.250817Z","shell.execute_reply":"2023-07-07T06:14:37.274398Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_tdcsfog_pred=clf.predict(test_tdcsfog)\ntest_tdcsfog['event'] = np.argmax(test_tdcsfog_pred, axis=-1)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:14:40.882240Z","iopub.execute_input":"2023-07-07T06:14:40.882999Z","iopub.status.idle":"2023-07-07T06:14:40.938444Z","shell.execute_reply.started":"2023-07-07T06:14:40.882959Z","shell.execute_reply":"2023-07-07T06:14:40.937547Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_tdcsfog['StartHesitation'] = np.where(test_tdcsfog['event']==1, 1, 0)\ntest_tdcsfog['Turn'] = np.where(test_tdcsfog['event']==2, 1, 0)\ntest_tdcsfog['Walking'] = np.where(test_tdcsfog['event']==3, 1, 0)\ntest_tdcsfog.reset_index('Id', inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:14:48.091576Z","iopub.execute_input":"2023-07-07T06:14:48.092637Z","iopub.status.idle":"2023-07-07T06:14:48.104781Z","shell.execute_reply.started":"2023-07-07T06:14:48.092582Z","shell.execute_reply":"2023-07-07T06:14:48.103562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_tdcsfog.head(10)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_tdcsfog.sample()","metadata":{"execution":{"iopub.status.busy":"2023-07-07T06:03:14.978663Z","iopub.execute_input":"2023-07-07T06:03:14.979001Z","iopub.status.idle":"2023-07-07T06:03:14.994861Z","shell.execute_reply.started":"2023-07-07T06:03:14.978969Z","shell.execute_reply":"2023-07-07T06:03:14.993621Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#submit = pd.concat([test_tdcsfog,test_defog])\n#submit = submit[['Id', 'StartHesitation', 'Turn','Walking']]","metadata":{"execution":{"iopub.status.busy":"2023-03-12T05:45:44.456721Z","iopub.execute_input":"2023-03-12T05:45:44.45761Z","iopub.status.idle":"2023-03-12T05:45:44.484039Z","shell.execute_reply.started":"2023-03-12T05:45:44.457558Z","shell.execute_reply":"2023-03-12T05:45:44.483017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#submit.head(10)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T05:45:55.848461Z","iopub.execute_input":"2023-03-12T05:45:55.849047Z","iopub.status.idle":"2023-03-12T05:45:55.861012Z","shell.execute_reply.started":"2023-03-12T05:45:55.849009Z","shell.execute_reply":"2023-03-12T05:45:55.859936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> #### 💬 Let's compare it with sample submission data.","metadata":{}},{"cell_type":"code","source":"#sample = pd.read_csv('/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2023-03-12T05:46:17.405928Z","iopub.execute_input":"2023-03-12T05:46:17.406302Z","iopub.status.idle":"2023-03-12T05:46:17.700211Z","shell.execute_reply.started":"2023-03-12T05:46:17.406268Z","shell.execute_reply":"2023-03-12T05:46:17.699171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#sample.head(10)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T05:46:58.502528Z","iopub.execute_input":"2023-03-12T05:46:58.502895Z","iopub.status.idle":"2023-03-12T05:46:58.513582Z","shell.execute_reply.started":"2023-03-12T05:46:58.502854Z","shell.execute_reply":"2023-03-12T05:46:58.512474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

💡 Notes:

\n\nWe can make hyperparameter tunning for lgbm and also, we can develop other machine learning models ...\n \n\n","metadata":{}}]}