{"cells":[{"metadata":{},"cell_type":"markdown","source":"Forked from Basic Feature Benchmark, courtesy inversion (https://www.kaggle.com/inversion/basic-feature-benchmark)"},{"metadata":{},"cell_type":"markdown","source":"**What can we learn from Medical warning systems to help improve this Earthquake warning system?**\n\nAs a potential lifesaving warning device, LANL Earthquake Prediction has much in common with other lifesaving warning devices, most notably medical devices. There is additional commonality, in that many data scientists and researchers work on several machine learning areas, so I expect participants analyzing geophysical data in this competition may also be interested in biomedical signal analysis. Because of this, I wanted to share the very latest \"Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML) – Based Software as a Medical Device (SaMD)\" published earier this month by the United States Food and Drug Administration (FDA).\n\nThe FDA Document helps us categorize the system, describe ongoing changes to the system, and discusses good machine learning practices.\n\nI will break down the components of the proposed regulatory framework, and use this example Kernel to demonstrate how it might impact your future work as a researcher, programmer, and data scientist."},{"metadata":{},"cell_type":"markdown","source":"I will first give a **summary of the medical device document**, its pertinent details, and invitation for feedback. I will then **use this Kernel to demonstrate** the similarities between medical warning devices with LANL Earthquake Prediciton, and what learnings from the former might inform our creation of the latter. \n\n**WHAT IS IN THE *MEDICAL DEVICE* DOCUMENT?**\n\nOn April 2, 2019, the US FDA posted the **\"Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML) – Based Software as a Medical Device (SaMD)\"** document, found here (https://www.regulations.gov/document?D=FDA-2019-N-1185-0001)\n\n**EXECUTIVE SUMMARY**\n\nThe framework first categorizes the “risk” of the AI/ML-based SaMD as being levels, I, II, III, or IV, from lowest to highest, based on a combination of how critical the healthcare situation is, and also how significant the SaMD classification will be to the final healthcare decision. The framework goes on from there to categorize the three (3) “types of modifications” which can help developers decide when a formal (510[k]) review is required. The three types are i) Performance, ii) Inputs, and iii) Intended use.\n\n**PERTINENT DETAILS**\n\nThe proposal has not yet been made into regulation, and so may change substantially or be withdrawn, but some aspects of the current proposal are pertinent to those working in the field. These details include:\n\n* SaMD’s are subdivided into three broad classes based on how significant the SaMD classification will be to the final healthcare decision. These three are “Inform clinical management,” “Drive clinical management,” and “Treat or diagnose.” These classes drive the clinical risk (from I to IV), further modified by the state of the healthcare situation or condition (Critical, Serious, or Non-serious). \n* Modifications are divided into three broad categories as follows:\n* Performance: Modifications related to performance, with no change to the intended use or new input type. This may include re-training with new data sets within the intended use population from the same type of input signal. As a “litmus test” this type of change will NOT change any of the explicit use claims about the product.\n* Inputs: Modifications to inputs, with no change in intended use. These changes may also involve changes to the algorithm for use with new types of signals, but do not change the product use claims. As examples, the document cites “modification to support compatibility with CT scanners from additional manufacturers,” or an atrial fibrillation diagnoses system that now will “include oximetry data in addition to heart rate data.”\n\n* Intended Use: These types of modifications include those that result in a change in the significance of information provided by the SaMD and/or result in a change to the healthcare situation or condition explicitly claimed by the manufacturer. Examples include an expanded patient population, such as the inclusion of pediatric population when the original SaMD was initially intended for adults, or an expanded number of diseases or conditions, such as for lesion detection from one type of cancer to another.\n* Finally, the proposal also examines the total product lifecycle, with an understanding that AI/ML-based technologies have the potential to transform healthcare by deriving new and important insights from the vast amount of data generated during the delivery of healthcare every day.\n\n**INVITATION FOR FEEDBACK BY THE US FDA**\n\nAlthough the entire document is open so as to initiate discussion and for the whole world to provide feedback, there are two major areas that seem to stand out as needing expert advice. The first is the “GMLP” or “Good Machine Learning Practices” which, if followed, can insure that the SaMD is accurate, reliable, precise, and achieves the intended purpose. The second is the establishment of the SPS and ACP filings. The “SPS”, or SaMD Pre-Specification describes the anticipated modifications. The “ACP”, or Algorithm Change Protocol, covers the steps used to control risks of violating GMLP."},{"metadata":{},"cell_type":"markdown","source":"**HOW CAN WE TAKE LEARNINGS FROM THE ABOVE AN APPLY THEM TO LANL EARTHQUAKE PREDICTION?**\n\nThere are three main areas where I thought it might be useful to implement \"medical style\" methods to this competition - but I definitely welcome your feedback and further brainstorming.\n\n1. Being clear about how we categorize the device and its intended use.\n1. Making plans for future modifications.\n1. Using good machine learning practices learned from medicine."},{"metadata":{},"cell_type":"markdown","source":"First, some Kernel Basics..."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":1,"outputs":[{"output_type":"stream","text":"['test', 'train.csv', 'sample_submission.csv']\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom tqdm import tqdm\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.svm import NuSVR\nfrom sklearn.metrics import mean_absolute_error\n\n# For future use - to use CNN to extract features instead pf simple min/max/etc..\nfrom keras.layers import * \nfrom keras.models import Model, Sequential, load_model\nfrom keras import backend as K \nfrom keras import optimizers \nfrom keras.callbacks import * \nfrom keras.backend import clear_session","execution_count":2,"outputs":[{"output_type":"stream","text":"Using TensorFlow backend.\n","name":"stderr"}]},{"metadata":{},"cell_type":"markdown","source":"**Being clear about how we categorize the device and its intended use.**\n\nFrom a medical perspective, the FDA thinks of these deivces ase being used for one of three (3) major purposes.\n\nThe major purposes are “Inform clinical management,” “Drive clinical management,” and “Treat or diagnose.” \n\n**How can we relate this to LANL Earthquake prediction?**\n\n1. If this Kernel is to merely \"inform\" then perhaps data will be recorded, or indicators may be used.\n1. If this Kernel must \"drive\" then perhaps an earthquake alarm will sound, and people who have practiced earthquake drills will take action\n1. If this Kernel will \"treat\" or take action, then perhaps the result will be an automatic shutdown of an assembly line, or other costly event that will be worth the cost if there is indeed an earthquake.\n\n**What does that mean to the software?***\n\nThe current contest accounts for major purpose 1 above, with a range of approximately 0 to 16 time to failure. If major purpose 2 or 3 are needed, then perhaps we have to be more certain of not having too many false alarms, and we may want to have a different target \"y\" that is not a numerical \"time till earthquake\" but rather a binary choice. Using the data provided in this contest, perhaps we only \"pull the alarm\" if the earthquake is at some median value."},{"metadata":{"trusted":true},"cell_type":"code","source":"rows = 150_000\nsegment = 0\nfor chunk in tqdm(pd.read_csv('../input/train.csv', dtype={'acoustic_data': np.int16, 'time_to_failure': np.float16}, chunksize=rows * 100)) :\n    if (segment == 0) :\n        X_train = np.int16(chunk.acoustic_data.values)\n        y_raw = np.float16(chunk.time_to_failure.values)\n    else: \n        X_train = np.concatenate((X_train, np.int16(chunk.acoustic_data.values)))\n        y_raw = np.concatenate((y_raw, np.float16(chunk.time_to_failure.values)))\n    segment += 1\nsegments = (segment - 1) * 100","execution_count":3,"outputs":[{"output_type":"stream","text":"42it [03:00,  5.04s/it]\n","name":"stderr"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plot the head of the seismic acoustic data\nplt.plot(X_train[0:150000])","execution_count":4,"outputs":[{"output_type":"execute_result","execution_count":4,"data":{"text/plain":"[<matplotlib.lines.Line2D at 0x7f84ac993d68>]"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAYIAAAD8CAYAAAB6paOMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd4FOXaBvD7IQm9Q+hEOhg6hCDSBCK9HLGBx4Z6OKjYj34Ido/KsR+76LFXVOwFBZEiAiYonUDAUEJJaKGEksD7/bGzm9k+W2Znk7l/18XF7szszLOT2XnmLTOvKKVARET2VcHqAIiIyFpMBERENsdEQERkc0wEREQ2x0RARGRzTARERDbHREBEZHNMBERENsdEQERkc4lWB2BE/fr1VYsWLawOg4ioTMnKytqnlEoOtlyZSAQtWrRAZmam1WEQEZUpIrLNyHKsGiIisjkmAiIim2MiICKyOSYCIiKbYyIgIrI5JgIiIptjIiAisjlbJoLM3API3nPE6jCIiOJCVBKBiLwhIvkislY3ra6I/CQim7X/62jTRUSeE5EcEVktIj2iEUMoLnrlNwx7dlGsN0tEFJeiVSJ4C8Bwj2nTAMxXSrUFMF97DwAjALTV/k0G8HKUYiAiojBEJREopRYBOOAxeRyAt7XXbwP4m276O8phGYDaItI4GnEQEVHozGwjaKiU2q293gOgofa6KYAduuV2atOIiMgCMWksVkopACqUz4jIZBHJFJHMgoICkyIjIiIzE8FeZ5WP9n++Nj0PQHPdcs20aW6UUrOUUmlKqbTk5KBPUSUiojCZmQi+AnCV9voqAF/qpl+p9R46B0ChrgqJiIhiLCrjEYjIhwDOA1BfRHYCuB/ATACzReRaANsAXKIt/h2AkQByABQBmBSNGIiIKDxRSQRKqYl+Zg3xsawCcGM0tktERJGz5Z3FRERUiomAiMjmmAiIiGyOiYCIyOaYCIiIbI6JgIjI5pgIiIhsjomAiMjmmAiIiGyOiYCIyOaYCIiIbI6JgIjI5pgIiIhsjomAiMjmmAiIiGyOiYCIyOaiMjCNPyLSHsDHukmtANwHoDaAfwBwjko/XSn1nZmxEBGRb6YmAqVUNoBuACAiCXAMUv85HMNTPqOUetLM7RMRUXCxrBoaAmCLUmpbDLdJRERBxDIRTADwoe79VBFZLSJviEidGMZBREQ6MUkEIlIRwFgAn2iTXgbQGo5qo90AnvLxmckikikimQUFBZ6ziYgoSmJVIhgBYKVSai8AKKX2KqVOK6XOAHgNQLrnB5RSs5RSaUqptOTk5BiFSURkP7FKBBOhqxYSkca6eRcAWBujOIiIyIOpvYYAQESqATgfwD91kx8XkW4AFIBcj3lERBRDpicCpdQxAPU8pl1h9naJiMgY3llMRGRzTARERDbHREBEZHO2TgSHik5ZHQIRkeVsnQhG/nex1SEQEVnO1olgV+EJq0MgIrKcrRMBERExERAR2R4TARGRzTEREBHZHBMBEZHNMREQEdkcEwERkc0xERAR2RwTARGRzcViYJpcAEcAnAZQopRKE5G6AD4G0AKOgWkuUUodNDsWIiLyFqsSwSClVDelVJr2fhqA+UqptgDma++JTDU7cwc27T1idRgheX/5NmwtOGp1GFTOWVU1NA7A29rrtwH8zaI4yEbu+nQ1hj6zyOowDMvddwwzPl+LsS/8anUoVM7FIhEoAD+KSJaITNamNVRK7dZe7wHQMAZxkI0ppawOIWQTZi0DABw9WWJxJFTemd5GAKCfUipPRBoA+ElENupnKqWUiHj9SrWkMRkAUlJSYhAmlWeb9pa96hUmAIoV00sESqk87f98AJ8DSAewV0QaA4D2f76Pz81SSqUppdKSk5PNDpPKuTNlsERAFCumJgIRqSYiNZyvAQwFsBbAVwCu0ha7CsCXZsZBVBaJ1QGQbZhdNdQQwOci4tzWB0qpH0TkdwCzReRaANsAXGJyHGRzUhbPqmUxZiqTTE0ESqmtALr6mL4fwBAzt23UnsITaFSrstVhkAVOFJ/GXZ+uxv+N6ICmtatYHQ6RZWx/Z/HM7zdYHQJZ5JfsfHy1ahce+nqd1aH4xAIBxYrtE8GSnH3457uZOHOGjYnlmfC0SuSX7RPBvqOnMHfdXhw6Xmx1KBRzjuQQqEPRht2HMf6lX1F0KvZdOaVMNmxQWWT7RED24Ouc6pwWqCz4yLcbsHL7IWRti/2jsAp5cUIxwkRAUXH3nDV4Yu7G4AvGITvdYvBZ1k5c8upvVodBcSYWdxaTDXy4YjsA4M5hHSyOxLjSQoJ9MsEdn6yyOgSKQywRaA6Xw2L43XPW4KYP/7A6jLjgq7bdWQc/b4PXje1EtsJEoFmdVxjxOo6dLEG7e77Hzxv3RiGiyH24Yju+XrXL6jDKtCU5+wAAJafLV6mhLD6Ej8zDRBBFWwqO4lTJGTzz02arQyEPPhuLQ/h8eWu4ZR4gPSYCTTQ66pWVH1f+4RNoMe1bLNm8z+pQLBVK70xVztoRyte3oUgxEWgWby6I2rrivfv3yu2HAADv/JZryfaz9xxBi2nfYm0UquMiEe9/JzPxaaykx0SgmZ25M+J18KdlzE/r9wAAvl+7O8iS0eR91g/lbuPydt4sb9+HIsNEEEXOBriydqH5yLfr0WLat1aHEdfK24mzvFV1UWRskQh2HixCi2nfosW0b2PSW2LVzkJs319k+nYi5dwTry3+y9I4YsFnNVBZy9hRVN4SG0XGFolg/a7Drtdm/gD0q17jUf894/M1cXPV7Xq0ggn7wsjD+z7O3AEAyD98MvoB+JGT7z1UZSh5oCyeN8+cUX4vfA4cOxXjaLwdO1mCFtO+xSsLt1gdiu3ZIhHoxeoHfaL4tNv795c77ryNhx/gQS2GeRuif79Dx/vnBl1mx4HjAIBtMSw1/aE1kAPAoSLH9y/vD3VrNf07XPXm7z7n7S48HuNovDl/C+8t22ZxJGRaIhCR5iKyQETWi8g6EblFm/6AiOSJyJ/av5FmxaCLxexNAHC/wi7ySARO+tKJVXYVnjBt3cf9fG9fYllPrT8E9molkSIbDA6/aFP0esNR+WVmiaAEwB1KqVQA5wC4UURStXnPKKW6af++MzEGAMDK7bF6cmTpic1f6rn8f8uRd8j9amzXoeMoOX3GxLji06kY3q2r/3s4E9CRE8YTAe/EpUgdPlEct+OemJYIlFK7lVIrtddHAGwA0NSs7QWyUvcIYTN/0D+uN1bVkrvvmOv1mp2FOHfmz0g1UKUSrn1H3eviIy0fFZ8+g8LjxSgsKo4oga3acSj4QiZwHgIVKpTuid9zD0S83qJTJTh+yniJyFrxUy12Ok5PjtFUcOQkujzwI57/OcfqUHyKSRuBiLQA0B3Acm3SVBFZLSJviEgds7cfq8NMf6duoNoofS7asNtRVXSqxLwSwf6j7u0Ske6PKe9moeuDP6LrQz/izk9XR7i22DtU5HhchD5Bzvw+8CO0jeyz1PvmottDP0YSWsxUr2T9g4edv4PdJlZVxouxLywBADwzb1NcXiyYnghEpDqAzwDcqpQ6DOBlAK0BdAOwG8BTfj43WUQyRSSzoCB69Zzfr90TtXV5CufKxqx68vzDpT8uo0nG6Chc8zeWPq3z8z/yQgssDkx8bRm+X7Pb7eQfdOAZg3+mkyYm9GiqXjkOEkGAnXrkhKPEWZYd1n0HfbIb9uyigJ87eOwUjsW4/crURCAiSXAkgfeVUnMAQCm1Vyl1Wil1BsBrANJ9fVYpNUsplaaUSktOTo4sEN3xFs5IU0WnStxOrEYYvWt1kQnP+zlZchrpj853vR+jXY0EM/K/i93eK6Wwbf8xP0tHz4ni0zHvxRLqcXCynLXh3PbRn1aHEJCzxBkKfZVrPOjygO/vsP1A4N5y3R/+CT0e/smssHwys9eQAPgfgA1Kqad10xvrFrsAwFqzYoiGnPyjSL1vrtuJ1R/981sOFvnvJqq/EjKjF1FxkEZYfykq16M751tLczHwiV+weqe5dfnXv5eFPo/9jKMxvAoKtRvvvV8YP0zNuJlw+/6iqO6fFR5tIkop5OQfCXk9+YdPhP1k1kDHfqiF6/kb9uK8J3/BN6vj77Hrvvbr3HWBayZiXbI0s0TQF8AVAAZ7dBV9XETWiMhqAIMA3GZiDADcT7xbQ7hqyMw9gIynF/qdn73niFt1kP6Z9U/MzTYWmwmN18EG2TG6xUztqjnU/v4nS0KrA12Q7aj662Rig7mnOSZWaeUf8V163LjncNi9RgY8sQCjn1scfEHNLl3PNCNVlh//vgMZTy/C0pzQSqjpj85H///8HNJnnLYFuTIOxZyVjr/n1A/ibyCmjKe9q4KWb428c0I0mdlraIlSSpRSXfRdRZVSVyilOmvTxyqlTH/ymP53EKhf9fwNe92uUjyvkPXW7SrEsGcX4YWfc1BYVIxt+48ZTjL6c7/RLoxr8woNt0GcO9P4D/OQR8lFP5BNuI1aMz6P60KeX/uOnvTq2hsOX3+lP3ccwvBnF+PVRVuDf14pn6WwQMejJ32J57n5vsfHWLipADu0k7HzTvgtBd53YAdzOIRuuHrRvAbK3ht6aSZSa/MKcfRkic+71oN549fgj3WJZZdyW9xZbPSRu9e+nYmRzy3G8q37AQTuYLdHa/z5c8dBDH12IQY+8UtYse03UEWxNq8Qo59f4vpBr9pxCMURHCT679XtIfe6SP0J6GetUVifgHYeDH4yWqbtv7Im7d/z0DdAEt1/1NgjMZb7+P7OE66Re1q+WrULY1/41dC2jFjlp2rvqjdWoP/jCwDoHjviscyJ4tNRe1z4zoNFrt8NYOx3abQtJxY9cfT7YsnmfRj9/BJ0un+uW61B3qHjbt8xEs/Oi90AV7ZIBKG6dNYyAIG7gDrn7S484bpT1Sgjdb1KKdcJxVnMX7erEBv3HMa4F3/FjM/XhLRNo1USvuom9YnASN163qHjrhNfMLFqJA7l5vIdB4p8lgyKDJ5sNuz2vjp1Nrr/ZOBek2icePXfN1jcSilX5wbPc/O0z1Zj9PNLUHDEcYyv+OuAz5Lpgo3u4z4v37rfq9qz338W4JzHgre16V348lL8YKCnn/6O9pz8o24Jt/B4cVTa4v71ySqMfn4JFm0qwOR3M93mOccz6Tvz55C/o57+uFu3K3bjddgiEYRbBDVy8ti4x3+RdJOf4uoN768Mut53l23DpbOWYe66PW73KzuvNjzHT8jadjBgl7N9x066fgyBdsfyrQfw4YrtbtP0yxvJJ0oB/R9fgB/W7sbK7QexcvtBv8mvz2PGq7HyD5/Am7/+5ToR5R067lUsLzpVgswIbw7r//gCnyWDwPeGlO6Yb9d413YeOFbabhPsRrpwnwa7Zmeh6zlS+l5rK/46gHnr9/ptj/pwxQ7dgwhLlyk6VYIv/nRUFebkH8WvOftwyau/4dVF3g+Jm/RW6TON5q7bg0tnLcM7vwV+hpB+W877aXz5cd0eVyLytDRnH06fUW4XKJPfzcT4l5a6ehH1m/kzRobQvuLPb1scF2ZXvrHCK7le8b8VEa8fcG/fS6gQu5v+yn0i2FpwFNkBTtb+7D96Ektz3Iv4o59fjKxtB/Dln3muB6cFMvSZRViwMR+/GmyA+yRzB7K2HcCrC7dg0SbHZ37bst+VyOZt2IsHvlrnWt7Z3lF4vBgXvrwUV7+5wme1BACkPzIfI59bHLQxMHvvEdw9Z43bnbYHjpX+CD3bFAKZ8t5KjH9pKca/tBTXv5eFYydL8P7ywCeHFxfkoOT0GWzbfwxrdhZi6ZZ92HXoODbsPoz0R+fjwa/Xuxri+878GRlPL8SvOftcD/m7Y/YqXPTKb14NtqEMQuPPsq0HUFhUjKxt3okm2MXG8eLSROjvBqqS02fwS3a+z3me9h4+4VVyGPPCEgx+6hes2el9JXndO5l4+qdNPtf1ysItrirKA8dOIWvbQSzIzsedn5TeLDjxtWWuq9Ut+cfw175jXu0JWdsOYHfhcezWllub53gc+2aPC6Li02ewcFOB2z4LVJ0y54889Hpkntf0Bdn5uOz15XhpgfvdulsLHAnggHasHtEuQtbsLMSRE8VY8Vfp3y/v0HG89Iv/u31X7zzkOpaMVOOGouhUCWZ+vxGnzyjX31N/sTFvg7FjIRqsv6vEZIOf8t/rJ5Ce//Y+8NbmHcaFL/8W0nr0V0p69325Fg+M6eg2zdddum8tzUVyjUqu9/oGwyvfWIHPru+DSokJAIDfcw+6qrX8uez15eh5VvCbuS9+pfR7PvrdRkwe0BoAcLWPp1k++PU6r2meVm47iNtn/4m56wJXjTwxNztoj6tXFm7BlIGtXO///vpyTExvjrtHnu26YXDBxnxc2isFgCNRvrAg9Fv79ScMwFE14PT2NekY2C4Z2XuOYP/Rk6iU5H5NdexkCbK2HcSAdo57YE6VlJ71pryXhex/D3f93Zzu/2qd6ym1/nzxRx5W5B7AB9pyuTNHASitTjpYVIwxLyzBg2M7en3W3+MNth8ocvVtf+7nHDzn7zEI2ldYt6sQg578xWu252/jk6yd+CRrp1ucAHD77FX4etUu9GlVzzVt0lu/Y8ujIwNeBR89WYI/th9EtUqJaFq7Cu7U/h7ObXi678u1+Oam/q73+4+dxMwfNuDXnP1YdOcgNKtTxVXyq1u1Iro2r429h0/gvPYNsGBjPvq0roexL/yKWlWSsOr+oX7j8sVIW1DqfY5ecj+s3e36Xc+9dYDbMmvzCtGpaa2Qth0OKQsP00pLS1OZmZnBF/RQdKrEtbPjUdWKCYbrnf25d3QqHv5mfZQi8u/7W/rj7MY142ZMBU+9WtRB5aQELNbdoLfx4eHYvPeo4RvqQtW0dpWgvYxa1q+GV6/oiRcX5ODLP937uL96RU8oBaS3rIuNuw/jsteX+1mLf5n3ZGBhdgHu0CWpeDS4QwNX5wN/hnRogCcu7oodB4ow7kXjjeU1Kif67X0364qemPxuFgDgqYu7uu2nNg2q++zxc2lac3ycuQOX9U5xS7jBjn0jywQzpmsTt557APDetb3Rr239sNYnIllKqbSgy5XnRJB/+IShG8GIiAKZmJ7i1Xbm6c1JvTDJz/gPkdKXqEJhNBGU6zaCE8Xl67EARGSNYEkAgGlJIBbKdSIo5wNQERFFRblOBEREFBwTARGRzZXrRMCqISKi4Mp1IigDHaKIiCxXrhMBEREFx0RARGRzliUCERkuItkikiMi06yKg4jI7ixJBCKSAOBFACMApAKYKCKpVsRCRGR3VpUI0gHkKKW2KqVOAfgIwDiLYiEisjWrEkFTADt073dq06KKvYaIiIKL28ZiEZksIpkikllQ4H+cYSIiioxViSAPQHPd+2baNBel1CylVJpSKi05OTmmwRER2YlVieB3AG1FpKWIVAQwAcBXFsVCRGRrliQCpVQJgKkA5gLYAGC2Uir4MFehbifg6LwUip/vGGh1CCHZ8ujIsJ/hHsjmR0Z4TZsysHXI67l5cBs8eXFX1/uNDw/H9JEdwoppxsizTfmuwZyf2jDq60ypWzXq6yxLruvX0mvaJWnNTN+uZW0ESqnvlFLtlFKtlVKPmLGNCiY8bCiWP7inL+mKge2SA54gFt81KKR1jujUKKxYWiVXx+x/9gnrs89cWnrCu/yclJA/36dVPay893yf8xb86zz8o39LbH5kBF65vKdrunPIww/+0Tvk7TlVreg+lOT0kR2QlFD6k3nv2t7InTkKo7s0Dmm9T13cFbcPbY+LejZD7sxRyJ05CpWTEjCpb0tMHtAq+Ao8eA7vOOeGc/H5DeeGtA79cf3etb1x/XmtA+67OTeci9eudB/vZNndQ9z+BnqdmtYMGsPrV6Zh0V2DsOXRkT7nP3pBZ+TOHIWpg9oEXZc//xrazuf0f/Rvid9nZABwjHj29dR++Ed/x0m5dXI1zLrC9/fy5daMtphjYP//76o0TBnYGrWrJmFgu2TMu30gLuvt/ft4cGwnw9sOV9w2FkdDszpVkHF2eFcto3z8uGtUdgzx/OWNfSOKy2njw8O9pm15dCSmDmqDlfeej/E9muHta9Jd4wV7enNSLzSvWxX/ndDNNa1+9YoBt/nkxV3R2cAYqM9e2s1rWnrLuvjh1v5e02eO7+xzHbcMaYsV04fggu6lVzT//pvvZQOZPvJs1K3m/b1uzWiLlvWrYcaoVCQlVMDwTo1wQfemuPrcFq5lzm1dH3895vvEEsz6h9z/PqO6NAEAPDa+M764sa9r+MCOTWpiUt8Wnh/H11P74eKezfCfCztj5vjOGNguGSumD8GFPX1f4SUlVMD0kWcbjq9xrcqYMrA1/q4l1/vHpGLurQPQI6UOKiYa/2m/eXUvAMDLf++Bt69JR7+29fF/wzu4jSns3B4ANKpZGT1SvMe9TkwQDO/UyK2E1KtFHTx6QWec3cg9EeTOHIWJ6aUnvaQEQYZWwvA3bvFF2n6bOrgNpgxsjVbJ1QJ+r/ev641fpw12ve/arBZ6aON1D01t6JYUzk9thOQalTB9ZAd8eWNfdG5WC9NHno07h7XHW5PSMbRj8AuorHsykDtzFG7NaIfuzWsHXb5Ls9qYNqID/rxvKN6+Jh1tGlRHy/ru3+l/V6WhiscFiRnKdSIQEbx+VRpSGwe/GvGU3qKu17SFdzquvrs2r40ZBn+wb1ydhrcm9fKavur+oaic5P4Hnv3PPkioIPjXsPY+T3yAexIa1L4BAGBct9Ket5n3+L5ydqpWKRFjuga+gs2dOQp/6166zq66g9pzwHUAmJDu+yr/tvPboUFNx8njkQs6eQ3M7cv0kR1wy5C2yLwnw+8yE9NT8PqVabg1w/vq7plLu+EBj4HbRQQ3D2kbdNvX9muJr6b6T/KJ2glqYnoKuun2iYjg/jHu2/xCO5k8cXFXXNorBRPSU/D2Nemu/RGuaSMcpcNaVZIw+599MG1EB9ffZFLflmjfqIbfz7Zv6HveoA6O42hE58YY2K60Y4aIYGJ6aZ+OxXcNws2D22Cen2rC+tUrAQD+b3h717RHLuiMy3qn4J7RpfeLfqEdwxf1LD3GAg3Q/viFXfDNTf1cya1yUgKmjeiAz6aci9sy2mHroyPR38eYviKOcaWdalROwjkt6+H289vhsfGdMXVw6THRq4UjQUwe0Bqtkqu7vv+Ng9qguVZdNbZrE78xAkA97fs7PxtMco1KXtP0n7t5cBsMCfNCNlSJMdmKxa7scxamzVkTdLk5N5yL8S8tBeD4sb+4IAf5R0665utPzr7+iJ6W3T0EjWr5/uH7uuhJb+mdfDx1bV4b/53QDa21g1Uf+97CE0E/H45muh+TERPTU3D8lPtg4n/vfZbPZQe0S8aiTaXdg/Wln35t6mNJzj4kJjh21vvX9cYj327Av//Wye9VYyTuHNbeKznrNTR4Eq9VJcktUYTqvWt74/L/+R7I/pq+LXGq5AwmD2gVMFaB9/45p1VdZO89ElIsNSonuV4nJlTA7UPbB1ha27YIlk8fgq9X7ULbBo7jtFaV0vV0beY86ZfGWFs3X29Q+2Rc0qu5z3l1qlXELRltXdv05Ll/XrisOypUcL8o+OamfthScNTQifuZS7vhK4+B5c10i48LHbOU6xKBk75et02D6n6X0xd3KyZWwE0BriL1y04d1MZn3Z6/JDCwXbLbDyxU47o19bqC6pFSByM6G6ur1p8kluqKzp6cV1Pje4R2r98N57XGsxO6G1r2ch/7zem/E7rh/jGp6KBd5fZtUx/f3dLflCQAeJ84wuVZdx6qvm3q4b7Rvp+4UjGxAm4e0jZorJWSSo/5G85zJNfqlUO/7rstox16nlXHUJ23XsOalXFd/1ZuJ9g3J/XCp1P6uKbpz71PXeJdFQkAj43vYmh7+iPij3vPx72jU13VM5W1feGruqxT01puJepAzDru4mF7tigR6A84FeR24zcn9UI97cq/eiX/PzbnVSoA3DG0HUQEHywPPsA1ALfitlGzrujpKqKGKrVxTazffdj1Xt+bqonH1b7+aqltw+rIO3Tcbf8lJQQ/OEOJM+PshkisICg54/13qVe9Eib19e5FYQZ9tcn71/WOKCk0rBm8tBiIiOCafi3x0Dfrw16HvsR4x1BHVePl55yFFxdsCWk9VSom4LPrgyeBe0YFryp1VmU66Y8kf1Wh/i6mPOmP0TrVKuJaH71vfJWSwrXx4eFYsnkfrnsnEwDQpZn/qq3nJnbHzR/+EbVtm8EWJQK9+tUD/0gHtW+ALs0cVxLjuvq/UmhSuwqevLgrMu/J8FmsdF6F+Va6fLDGXaehHRvhbINtHZ5VTCLAD7f2x5taW0Xxaf/J8ObBpT0yGmjVX/qTYrM67if5GpUiu5aoUEHcSmyxcPcI715YV55bWnXVt0199DzLuzG0rEqoILiuv3dV0qtX9Ixat+BwqsKMVMcYXlfU1mRM5aQEZKQ2xNJpg9G/bX1Xg7svwdoW4oHtEoFno14gFSoIHr/Qf9H0op7N/CYWoyftmhFUEfkzuIP7lVelxAro0Kim1xWZL4m6k/IDYzviPxd29uo9otfPRyNdqJrVCa0NIhz6E8XgDg3Q2yNZ/s1g9UAgzpJAKD12rDSsYyNXw2i4XrisO244r3VYiTOaJ+9qAS5IovnMsW9u6ufWjbpJ7Sp499rebg3FRoyJs+RQNo7YCOkvPConhfaVe7cK3oDra91VolTfHA7PA9+zeG20FFK1YiIu7ZXideVWT1eM91WlY5TzZPyyn77nZnrl8p5uPU2icc/JD7cMwIuX9UDjWuYntkhFq/p5dJcmuGt4h7Cu7qN5m4+zRNLVR8nE2SUzGtvr1LSWoU4dwTylu5kwHtiijUAv3NOWkTseW9Wv7qqLj6d7mj3rRvu2cZwA6/mplw2mb5v6rt4TpyNIBM6ujs4eWGkmVsfooxRx1CNf0L0pFm/e55oWqTrVKvq8/yQa/m94h5BvXAvkvevCv9EuWgLV2c+9dQCOF58OeZ09fdzf8N51vbF656GodQYIhb+q03grNdouEZjp3tGpmPjasqDL6U86DWpWwtZ9x6IaR5PagRvYnCWGcH+OO2hGAAAPlklEQVQYj43vjOZ1q+DFBVvc+mmHyhlHrSpJeHNSL0M34USH4w+g77llwk3oUXV9wDan0AVrK4sF5z5v4KMrdqD7IUJVv3olDO4Qm/74eh9NPgdn1Qu9g8fCO8/D/mOnTIjIPyaCIJxVPO0aBq9L1Wf5YL2TnG44rw2WbV0RXnB+jOvWFFnbDuKd37YFXM7z5Gf0cRXVKiXizmEd0COljqt0ESkj7ReR0H9V5/fWJ8xo9igpCwLVqcdaqPXrvjjv8Wgag/Ymo87Rta3VqpKEwuPFAII/buOsetVwVr3Ad01HW/wcDTHi+fyYYBrUrIwPruuNLiFerdYzWA8/oF0ybhrcBlf08X3DVbgeGtfJbyI4oyUpz3rxULunRnrXo68rQbPov6rzpb63UrA6cyPdI8uSSEpy0eL8mxi9aApkdJfGqFoxwfQLinAtnz4EM7/fiBGdGkW1tBMttkgEzqu9JrUqh9WQd67hq97SA7rnWcYblO4wcLdmJFp73ETnrNY3ozrkw3+cE3SZjk1qYt2uwxjQzppxJpwNm/q7XRODdGGNdRdXO4jmQyFFJGaPYwhH5aQEr0efxBNbHd29otDaHw2xroTQ3xsAlF6BmfF0ViP9yZ09O2pXjX7XWSOc39roIyPIHM7D7wzHlLWcrRKB2cdboPV30BUHY9290PNq1/moAedjMoy0fxhlJLfcPyYVX9zYN6b1oPq/jT5Guz//3krOkjrzgPXsUTVk4OTk6+mFoXIez76udGdP6YOtBceQk38UnQPcjh4LDWpUxnc393c9xveTKeeiQPdwPbNVSkyI6KFskdI3lH49tR/2H4vdd6dSzt9JWovycxd3WWVKIhCRJwCMAXAKwBYAk5RSh0SkBRwjkmVriy5TSk0xI4ZQPXVJ9G7waOPjbs2alR1PpLTyBKiX2qS050KtKklu9eXlkf5iQN91slbVJNSyqIrK7hrWrIwfbxuAFjHuIUPezKoa+glAJ6VUFwCbANytm7dFKdVN+xfTJGB2CdTZCyaUu5HLm3jtj8/qh/jUrmGNuLu5yo5MKREopX7UvV0G4CIzthNVUThRnFWvGhbeeZ7Xg9nsJCFeM0GEaoTxCGeisiIWR/c1AD7WvW8pIn8AOAzgHqXUYl8fEpHJACYDQEpK6OPcWiXWN4IEsur+oTF/1kWwbphlTf+29bF4874yW3XGxnAyIuxEICLzAPgayHOGUupLbZkZAEoAvK/N2w0gRSm1X0R6AvhCRDoqpQ57rkQpNQvALABIS0uL6HTmHIzm3Nb+n6JZHmsOyurJK54kxngwkmj6877zfQ4tSuQp7ESglPI/qCwAEbkawGgAQ5TWcV0pdRLASe11lohsAdAOQGa4cRjRsUkt/D4jw/BTN4k8lcUar9pVebyTMaaU40VkOIC7AIxVShXppieLSIL2uhWAtgC2mhGDp+QalQI+Krd6HD17heJHRqrjbtWW9aN3rwVRvDHr7PcCgEoAftJOvs5uogMAPCQixQDOAJiilDpgUgyGrXtwWFw9hIui7+zG4T3f5bL0FIzt2iSiMaaJ4p1ZvYba+Jn+GYDPzNhmuGpUSmQSsIE2DcK7ohcRJgEq98pXF48wvHZVmtUhEBFZyvaJgOyiDLb2muDafi2tDoHikO0TgXM8UyI7uHd0KnJnjrI6DIoztk8EdcMct5coVvgIBjKb7VtJ+Qya6OjctBZvYCMqo2yfCCg6vr6pn9UhBKQfn7ismTa8g9UhUDnHMifZQtWKZe+aZ/rIDqhWMQHXsIGXTFb2fh1ENjF5QGtMHtDa6jDIBmxfIijLDxUjIooG2yeCCkwERGRztk8ERER2x0RARGRzTARERDbHREBEZHNMBERENmdaIhCRB0QkT0T+1P6N1M27W0RyRCRbRIaZFQMREQVn9g1lzyilntRPEJFUABMAdATQBMA8EWmnlDptcixEROSDFVVD4wB8pJQ6qZT6C0AOgHQL4iAiIpifCKaKyGoReUNE6mjTmgLYoVtmpzaNiIgsEFEiEJF5IrLWx79xAF4G0BpANwC7ATwV4roni0imiGQWFBREEiYREQUQURuBUirDyHIi8hqAb7S3eQCa62Y306Z5rnsWgFkAkJaWxlEDiIhMYmavoca6txcAWKu9/grABBGpJCItAbQFsMKsOIiIKDAzew09LiLdACgAuQD+CQBKqXUiMhvAegAlAG5kjyEiIuuYlgiUUlcEmPcIgEfM2jYRERnHO4uJiGyOI5SRbbz89x5o16iG1WEQxR0mArKNEZ0bB1+IyIZsXTWUcXYDq0MgIrKcrRPBC5f1sDoEIiLL2ToRVE5KsDoEIiLL2ToREBEREwERke0xERAR2RwTARGRzTEREBHZHBMBEZHNMREQEdkcEwERkc0xERAR2RwTARGRzZny9FER+RhAe+1tbQCHlFLdRKQFgA0AsrV5y5RSU8yIgYiIjDElESilLnW+FpGnABTqZm9RSnUzY7tERBQ6U8cjEBEBcAmAwWZuh4iIwmd2G0F/AHuVUpt101qKyB8islBE+vv7oIhMFpFMEcksKCgwOUwiIvsKu0QgIvMANPIxa4ZS6kvt9UQAH+rm7QaQopTaLyI9AXwhIh2VUoc9V6KUmgVgFgCkpaWpcOMkIqLAwk4ESqmMQPNFJBHAeAA9dZ85CeCk9jpLRLYAaAcgM9w4iIgoMmZWDWUA2KiU2umcICLJIpKgvW4FoC2ArSbGQEREQZjZWDwB7tVCADAAwEMiUgzgDIApSqkDJsZARERBmJYIlFJX+5j2GYDPzNomERGFjncWExHZHBMBEZHNMREQEdkcEwERkc0xERAR2RwTARGRzTEREBHZHBMBEZHNMREQEdkcEwERkc0xERAR2RwTARGRzTEREBHZHBMBEZHNMREQEdlcRIlARC4WkXUickZE0jzm3S0iOSKSLSLDdNOHa9NyRGRaJNsnIqLIRVoiWAvHuMSL9BNFJBWOEco6AhgO4CURSdCGqXwRwAgAqQAmassSEZFFIhqhTCm1AQBExHPWOAAfaYPV/yUiOQDStXk5Sqmt2uc+0pZdH0kcoXp+YnfUqpIUy00SEcUts4aqbApgme79Tm0aAOzwmN7bpBj8GtO1Saw3SUQUt4ImAhGZB6CRj1kzlFJfRj8k13YnA5gMACkpKWZthojI9oImAqVURhjrzQPQXPe+mTYNAaZ7bncWgFkAkJaWpsKIgYiIDDCr++hXACaISCURaQmgLYAVAH4H0FZEWopIRTgalL8yKQYiIjIgojYCEbkAwPMAkgF8KyJ/KqWGKaXWichsOBqBSwDcqJQ6rX1mKoC5ABIAvKGUWhfRNyAiooiIUvFf65KWlqYyMzOtDoOIqEwRkSylVFqw5XhnMRGRzTEREBHZHBMBEZHNlYk2AhEpALAtglXUB7AvSuGYId7jA+I/xniPD2CM0RDv8QHxFeNZSqnkYAuViUQQKRHJNNJgYpV4jw+I/xjjPT6AMUZDvMcHlI0YPbFqiIjI5pgIiIhszi6JYJbVAQQR7/EB8R9jvMcHMMZoiPf4gLIRoxtbtBEQEZF/dikREBGRH+U6EcRyWEwRaS4iC0RkvTZ85y3a9Loi8pOIbNb+r6NNFxF5TotttYj00K3rKm35zSJylW56TxFZo33mOfExIpDBWBNE5A8R+UZ731JElmvr/Vh7ICC0hwZ+rE1fLiItdOswZShSEaktIp+KyEYR2SAifeJtH4rIbdrfeK2IfCgila3ehyLyhojki8ha3TTT95u/bRiM7wnt77xaRD4Xkdrh7ptw9r+RGHXz7hARJSL1rdqHplJKlct/cDzUbguAVgAqAlgFINXE7TUG0EN7XQPAJjiG43wcwDRt+jQA/9FejwTwPQABcA6A5dr0ugC2av/X0V7X0eat0JYV7bMjwoz1dgAfAPhGez8bwATt9SsArtde3wDgFe31BAAfa69Ttf1ZCUBLbT8nRGOfA3gbwHXa64oAasfTPoRjgKW/AFTR7burrd6HAAYA6AFgrW6a6fvN3zYMxjcUQKL2+j+6+ELeN6Huf6MxatObw/GgzG0A6lu1D838F/MTdMy+GNAHwFzd+7sB3B3D7X8J4HwA2QAaa9MaA8jWXr8KYKJu+Wxt/kQAr+qmv6pNawxgo26623IhxNUMwHwAgwF8ox2U+3Q/SNd+0w7+PtrrRG058dyXzuUi3ecAasFxkhWP6XGzD+FIBDu0H3qitg+HxcM+BNAC7ida0/ebv20Yic9j3gUA3vf1nYPtm3CO4VBiBPApgK4AclGaCCzZh2b9K89VQ84frJN+uExTacXP7gCWA2iolNqtzdoDoGGQ+AJN3+ljeqieBXAXgDPa+3oADimlSnys1xWLNr9QWz7U2I1qCaAAwJviqLp6XUSqIY72oVIqD8CTALYD2A3HPslC/OxDvVjsN3/bCNU1cFwlhxNfOMewISIyDkCeUmqVx6x43IdhK8+JwBIiUh3AZwBuVUod1s9TjpRvWTctERkNIF8plWVVDEEkwlE0f1kp1R3AMTiKyi5xsA/rABgHR9JqAqAagOFWxWNULPZbuNsQkRlwjFvyftSDioCIVAUwHcB9sdqmVcd3eU4EgYbLNIWIJMGRBN5XSs3RJu8Vkcba/MYA8oPEF2h6Mx/TQ9EXwFgRyQXwERzVQ/8FUFtEnIMU6dfrikWbXwvA/jBiN2ongJ1KqeXa+0/hSAzxtA8zAPyllCpQShUDmAPHfo2XfagXi/3mbxuGiMjVAEYD+Lt2Egwnvv0Iff8b0RqOhL9K+800A7BSRBqFEaNp+zAqYl0XFat/cFxdboXjD+lsWOpo4vYEwDsAnvWY/gTcG4Ie116Pgntj0wptel046snraP/+AlBXm+fZ2DQygnjPQ2lj8Sdwb2i7QXt9I9wb2mZrrzvCvTFvKxwNeRHvcwCLAbTXXj+g7b+42YcAegNYB6Cqto63AdwUD/sQ3m0Epu83f9swGN9wOEYxTPZYLuR9E+r+Nxqjx7xclLYRWLIPzfoX043F/Ms5WvY3wdHTYIbJ2+oHR5FuNYA/tX8j4aiPnA9gM4B5uoNCALyoxbYGQJpuXdcAyNH+TdJNTwOwVvvMCwjQ6GUg3vNQmghaaQdpjvaDqqRNr6y9z9Hmt9J9foYWRzZ0PW8i3ecAugHI1PbjF9qPKa72IYAHAWzU1vMuHCcsS/chgA/haLMohqNkdW0s9pu/bRiMLweO+nTn7+WVcPdNOPvfSIwe83NRmghivg/N/Mc7i4mIbK48txEQEZEBTARERDbHREBEZHNMBERENsdEQERkc0wEREQ2x0RARGRzTARERDb3/xaEzKiJK8FvAAAAAElFTkSuQmCC\n"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train = np.reshape(X_train[0:(segments * rows)], (segments, rows,1))\nX_train.shape","execution_count":5,"outputs":[{"output_type":"execute_result","execution_count":5,"data":{"text/plain":"(4100, 150000, 1)"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Sanity check - plot the head of the seismic acoustic data\n# plt.plot(X_train[0,:])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train = np.zeros(segments)\nfor segment in tqdm(range(segments)):\n    seg = y_raw[segment*rows:segment*rows+rows]\n    y_train[segment] = seg[-1]\nmedian = np.median(y_train)\noverall_max = y_train.max()\nprint (\"time to failure median\", median, \"and maximum\", overall_max)","execution_count":6,"outputs":[{"output_type":"stream","text":"100%|██████████| 4100/4100 [00:00<00:00, 590300.92it/s]","name":"stderr"},{"output_type":"stream","text":"time to failure median 5.349609375 and maximum 16.109375\n","name":"stdout"},{"output_type":"stream","text":"\n","name":"stderr"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# add the \"pull alarm\" category\ny_pull_alarm = np.zeros(segments)\nfor segment in tqdm(range(segments)):\n    y = y_train[segment]\n    if (y <= median) :\n        y_pull_alarm[segment] = 1\n    else :\n        y_pull_alarm[segment] = 0\nprint(\"Only pull the alarm if the time to failure is less than or equal to\", median)","execution_count":7,"outputs":[{"output_type":"stream","text":"100%|██████████| 4100/4100 [00:00<00:00, 975087.68it/s]","name":"stderr"},{"output_type":"stream","text":"Only pull the alarm if the time to failure is less than or equal to 5.349609375\n","name":"stdout"},{"output_type":"stream","text":"\n","name":"stderr"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"The original target data in BLUE is plotted alongside the newly added pull_alarm decision data in ORANGE.\")\nplt.plot(y_train)\nplt.plot(y_pull_alarm)","execution_count":8,"outputs":[{"output_type":"stream","text":"The original target data in BLUE is plotted alongside the newly added pull_alarm decision data in ORANGE.\n","name":"stdout"},{"output_type":"execute_result","execution_count":8,"data":{"text/plain":"[<matplotlib.lines.Line2D at 0x7f84ac939f98>]"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXmUJEd957+/qurqI7N7+qis0dw9lTpAyBzawYCxwSCBAYOwd9l9YGQDxqtd4zVg/J4f2G/tPfy8uzZgs4t2sRYJ8C6XQfbCYgyWQFhmjYVHHDrQNfcpTR8z0zPdPX1V7B+ZUVVdnUdkRGRWVlV83tNTTx2RkVGRv4j4xTd+P2KMwWAwGAzdT6HTFTAYDAaDHoxBNxgMhh7BGHSDwWDoEYxBNxgMhh7BGHSDwWDoEYxBNxgMhh7BGHSDwWDoEYxBNxgMhh7BGHSDwWDoEUpZXqxSqbDp6eksL2kwGAxdz4MPPjjLGHPiPpepQZ+ensbBgwezvKTBYDB0PUR0XORzxuViMBgMPYIx6AaDwdAjGINuMBgMPYIx6AaDwdAjGINuMBgMPYIx6AaDwdAjxBp0IrqLiM4R0SNtr/86ET1ORI8S0R+mV0WDwWAwiCAyQ/8kgNe0vkBErwDwRgDPY4w9B8AH9VfN0G988/FncOr8UqerYTB0LbEGnTF2P4D5tpd/FcB/Zoyt+J85l0LdDH3GL3/yIH76j77V6WoYDF2LrA/9WgA/RUQPENHfEtELdVbK0L+s103ScoNBFtmj/yUAkwBeDOCFAP6ciGqMsS1PIxHdBuA2ANi7d69sPQ0Gg8EQg+wM/RSAv2Ae3wVQB1AJ+iBj7A7G2AHG2AHHiY0tYzBgcWW901UwGLoSWYP+fwC8AgCI6FoAZQCzuipl6G+Ozi52ugoGQ1ciIlv8LIDvALiOiE4R0TsB3AWg5ksZPwfgbUHuFoNBhsMzlztdBYOhK4n1oTPG3hLy1q2a62LocwoE1Blw+Jwx6AaDDOakqCE3lEtedzw8Y1wuBoMMxqAbcodxuRgMchiDbsgdR2cXsWH06AZDYoxBN+QGxoDxkQGsrNdx5sJyp6tjMHQdxqAbcoXr2ACAQ8btYjAkxhh0Q65wHQsAcMRsjBoMiTEG3ZArJqwyJkYGzMaowSCBMeiG3FFzbKNFNxgkMAbdkDtcxzJadINBAmPQDbnDdWzMXl7BxeW1TlfFkGMePnURx+fMwN+KMeiG3MGVLkeMH90QwRs++m283CRE2YQx6IbcUfOVLsbtYjAkwxh0Q+7YMzmCgSIZpYtBiJX1jU5XITcYg27IHQPFAvZNWcblYhDixJxJLM4xBt2QG1qjtxili0EUs5JrIpLg4i4iOucns2h/7zeJiBFRYPo5gyEpBALgbYwen1vE2ka9wzUy5JXtY4MAzF5LKyIz9E8CeE37i0S0B8CrAZzQXKeO8MGvP4HPPNATt9IT1BwbaxsMJ+fNctoQjDXo5ecxM/QmsQadMXY/gPmAt/4YwG9h80q5a/nofYfw23/5cKerYfBxjdLFIIjpI02kfOhE9EYApxljP9RcH4MBgDdDB4wW3RDPkXOXYVIaeyQ26EQ0AuC3Afyu4OdvI6KDRHRwZmYm6eUy59IVczoxD2wbHoAzOmiW04ZwfBt+aWUdM5dWOluXnCAzQ3cB7AfwQyI6BmA3gO8R0VVBH2aM3cEYO8AYO+A4jnxNM8KEbc0PtYpRuhiimRgZAGDcLpzEBp0x9jBjrMoYm2aMTQM4BeBGxtjT2mvXAcyMMD+4Vdv8HoZIeJgI0088RGSLnwXwHQDXEdEpInpn+tXKnl3jwwBMx8gTrmPjwtIa5hdXO10VQ07Zvm0II+WieW59SnEfYIy9Jeb9aW216SCloqd/PnzOLN3yQlPpchmT1mSHa2PIIwUi7DeuuQbmpGgbR2bNSN8x2oQKjeW0SXZhiMB1bKOG8jEGvY1js0tYN6cTOwZR8++d48MYLBXMctoQievYOH1hGcurJkiXMeg+XMa6ulHHqfPLna2MAQBQLHjLaaM8MkThVi0wBhydNf3EGPQW9k6OADAbo3nCKF0McdQqRunCMQa9BZ5YwcwI84NbsXBifsnEvDaEsr9igcg8t4Ax6JuYGCljyiqbkT5HuFUbdQYcNzGvDW3wPfThchG7xofNcwtj0LfgOmaJnydMflFDFHwP3Ty3Hsagt1EziRVyxf6KibpoiKfmeJvn9Xp/B+kyBr0N17Exv7iK8+Z0Yi6wBkvYsW3IaNENkbiOjeW1DZxduNLpqnQUY9DbcKv+xqg5YJQbzHLaEIdxzXkYg95G83Rivpf484urPXeQgoXkSuH5RU3Ma0MYfCLW7ys5Y9B9uDHZPTGCcjH/pxNv/I/34A0f/Xanq6EdCnjNrdq4bGJeGyJw7EGMDpb6fq/FGPQWCN7pxOnKSFd0jEN9MhvhB0cO5XyQNXQOIkLNHEIzBj0IE+wnXzSW010wyBo6h+uYMBHGoAfgOjaOzy9hdT3/Qbr6QY1z1Zgf87pPViQGMdr3VFzHxtMLV3B5Zb1DNeo8xqAHUHMsbNQZTsznf7TvBzUOEXmrJhN8ydBGa3ROo3QRy1h0FxGdI6JHWl77IyJ6nIgeIqK/JKLxdKuZLc20Vvk3IHlX4+ii5lhmhm6IpDUhSr8iMkP/JIDXtL12D4AbGGPPBfAkgA9orldHqXVBx5iyygDyXUedmJjXhjj2To2gWKC+9qPHGnTG2P0A5tte+xvGGHdU/QOA3SnUrWOMDg1g+9hgrme/hYKfMq+PDDrQHy4mgxyDpSL2To70zTMRhA4f+i8D+GsN5eSKWiXfEii+H9Qvs5HGCd4+uV+DHK5j5XoiljZKBp2IfgfAOoBPR3zmNiI6SEQHZ2ZmVC6XKu2HEN2qhSMzl3N/OrFb1DiqTE95Ma/zPMgaOk/NsXF0bhEbfRqkS9qgE9HbAbwewFtZhNVjjN3BGDvAGDvgOI7s5bKhbcd84co6Zi/nVxY4PFDsGjWOCFFj59BAEbsnhrtio9rQOVzHwup6Haf7NI2klEEnotcA+C0AtzDGejLzQFPpkt8ZIXdDHOqhJSYFnf33cR3bKF0MkXTDc5smIrLFzwL4DoDriOgUEb0TwEcBjAK4h4h+QEQfS7memdMNShd+JL5fNgpdx8bRWRPz2hBOvxv0UtwHGGNvCXj5zhTqIsTJ+SUMl4uo2IOpXmfntmEMDRRyvQlnD5Vyr8bRSWvM613jw52ujqHDBA3rE1YZEyMDfWvQu+6k6E/94X048Pv3pn6dQoFyr3QB8q/G0Ulj1WTcLgafwOicjt23ey1dZ9CzxO2C6G1u1cLhLlDj6KDXl9Mf/7sj+MwDJzpdja6nn4Prda1BX9tIX6pXq1g4dX4ZV9byeDrRM+CuY+NSztU4uqjYZYwNlXLtBlPh9//qMfz2Xz7c6Wp0PW7VwuzlVVxY6v1nop2uNegn5tMX17hVG4wBx+byaUAIvT9rbYWIumLVpEo/rLbShIsF+tHt0rUGXfcsLegZagT7yfGmYzeocXTSD3sG50xmJiXcav9GXew6g14d9dQtaTzU1LbF0hzp89sxuBonz4OOTtyqhWcWVnDpylqnq5IaZtNXjT0TwxgokpmhdwOjQ57SMotOP1wuYtf4cK4NOlfj9IIWXcTR0Ix53XsPa8XWH0FzbaOO6ff/Ff7Xd45pKzPvlIoFTE9ZuX5u06LrDDonqx+r1gVprXrJr9y+Smqnl6MuVkeHAOj1/a74cX7+7Zce1VZmN+BJF3uvj8TRdQadz+IOzyxmsnnEO0aeN6pcJ89qHL3snfRiXveii4mHPdBpiKKHx+4m6pGsORZOzC1loobLE11n0DkXl9cwn0E+TbdqY2l1A08vXEn9WrLUHE+Nc7QPUrSVSwXs69GY1/0WElkHFBL8x3VsrNdZJmq4PNGVBr3hR8+g47uVfCpdWmcnXI3TL4ag1uPL6dMXlrG0qj/RcT8lT+ZKl37bYO5Kg56l9rohgcqxz7Yb1Dg6casWjs0u9XTM6zQG56N9MuADrXLe/rlnoEsN+q7xYQyWCpnoTKujg7AHS7kc6flqsxvUODpxKzZWN+o4db73ltN7Jr2gY0dScJ/1S/8AgLGhATijg311z0CXGnQivuxOf/QlIi+tVc5H+prTPzItHge+F++3kZlJ0wSidQ3Ti+0VhetYfXe4qCsNOpCOAQtLrtANPlsvIFE2yp9O03Ax5WxfQweDpSL2TKSz6dsveywcHnWxH54JTvcZdP+3cR0bJ+eXsLKevlTPdSycvXgFizneVOoGNY4uJqwypqxyrvc1VEhrRZj3SYluXMfGxeU1zGWghssLIhmL7iKic0T0SMtrk0R0DxE95f9/It1qbqkTXMdCnQHH5zII0uVvwuZZFtgNcWd0UuvB7O58HllzbBydvaw1M1OBPL98L28kt9OP8fNFZuifBPCattfeD+AbjLFrAHzD/3emNJQuGfxYDQlUjmc4vRB1kTEWmVO0lV49CUjk3duVtTrOXNSX6Ljm2H2XPLl5qri3Bv4oYg06Y+x+APNtL78RwKf8vz8F4Oc01yuWLKMM7psaQUHjRpUO2udZXI3TL5tArmNjbrE3Y167KUjuGmX2kJuKxUT/4Wq4PD23aSPrQ9/OGDvr//00gO1hHySi24joIBEdnJmZkbzcVkbKJezcNpSJ+mSwVMSeyREczvFI3y1qHF00lS69d79pHIrJckWbJVELukKBukLQoBPlTVHmbSGHDpWMsTsYYwcYYwccx1G93CbcanapplzHzt3D0B7IqlfdEEH08mGqKcvLzKTj3rjCY2KkjEmr3JMDYBS1PprkAPIG/Rki2gEA/v/P6auSOFnKklzHwtGcbyrVukCNo4vdE8MoFws9adB5ZiadMkMiL6ViL7ZXFK5j49T5pb4IXAfIG/QvA3ib//fbAHxJT3WSUXMsXF5Z15LhJW5QqDk2VtbrOHMhv5tK3aDG0UWpWMB0ZaTnlC6cNFZb/KxCP5GlGi4PiMgWPwvgOwCuI6JTRPROAP8ZwKuI6CkAN/v/zgSGpt9Mt18wyh/XDSqSblDj6CQP2d3rdYY/+vrjmNE8qXAdG+curWBBY2YmL3nyCi4u9W62p3a64bnViYjK5S2MsR2MsQHG2G7G2J2MsTnG2E2MsWsYYzczxtpVMJnQ+LEymJGmoTzQTR7VOGniOjZOzHc25vVjTy/g9vsO412fflBrubUUImg2n5f+6B9A/2nRu++kaAvbxwZhlYuZ/FiTVhnjIwO5HukbapwcDzo6qTkW1uuso8vpkbIXyvnRMwtaymtffepcgdR6VOkSBVfD6dCi33H/YfyPbx3WUKv06GqDTpSdLImIvE2lnDwMYT7/flK65Gk5vbSqd9Nt39QISgXSem/9mjxZV4rGP/jq4/gvX3tcQ43So6sNOsAjqmXTQV3Hzv2ps25Q44SRtMZZHi7LmoFiAXun1Dd9W9uUJ0/u9L6DLkTFbVxyrEsNl+dnqwcMuo3TF5axrHmGFHitqo2ZSyu4uJyPTaWgY/JuF6hxokiSA3N0aADbxwZzo9zQvdmYltKlpwZAgQ7jOhYWVzfwzIL6xjWAXMfh736DnmFGoTT8mrrhftJDOa6jTmqV/Bgo3ZuNNcfC8bklrGvc9OVl9lPy5Jrm5zYvE4ggut6gZ5lqqhvSWvVbflG3amldTiel9bq691dch2dm0rfa6sfkybr2WvZX8u/i6zqD3h6RT1eGFxFzsHfS26jK8wy9G9Q4OnEdGwtX1jF7ufNBunQP9Gls+jZWtH0y4AMtajjFex4fGQBgDHqqDA14GV50bFbGhW4dKBawb0o9m8zDpy6mdprTC9KVv7gzaZEnN5iOOrT2QTeFTd9e3kgOg4dSUL1nvhjL8+nkrjfogJ/hJSMDxuPHqPCGj34br/jgt/RUKAAvZkd+O51O8uQG020kx0f8zEwa762RPFnheVnfqGP6/X+Fj9z7lLZ6pY1OyXGeM2X1hEGvOTaOaM7wEnWt43OLWjeqdONWbe+Id07UOGmyc9swhgY6H6Rr1/hwKpuNqqqUoK0FVzEf77r/nP3xvU9Kl5E1rmPjzMUrWFpVD1w3ezm/cfh7wqCnkeEl/FoW1jYYTmrYqFKJABc1dOXJDZE2hQLlQuniVtPZbHSr+ldb/Zg8WffeQR5WhEH0iEHPTtmhM/nAsTm1+oa5/Lsh7oxOdIeaTQI3iWn1QdexMb+4innFRMfU4pznyZNVy1Rho87w377xlNbgY1Ho2mDeNzWipZy06A2DnmGUQVdjYoW0Nlf2TOo/Np5nahULJzsc8zqtMATNIF1pbIyq9z9Z18MDR+bwoXuexAfuflj62kkWGPumRjw1nOI98zj8eVUJdaVBb5+Z6szwEse2kQFU7LLStXaNDwNIbwDiapxuc7kwhnipUQBu1QZj6iseFUaHSsqbjcH+bv0yQ52Dj6yBHBv2JIAPn76odP32rF1hNNRwKvsRAIoFPw5/Tp+trjPoQYNyQ5aUkZyoppgoYLhcBJCuj1uHGqdbaLiYOiwnU91sBLYaqN0TI9ozM6kmT24deGTrVSx495nlAScdOXcJ+Tqd3I6SQSei3yCiR4noESL6LBEN6apYUtJQA6R1LU6aBtetemqcfjjizfOLdnpFksZmY7FA2K85dRxPnqzj7EZeDVsQPCGKqhrOrVo4kdPwCdIGnYh2AXg3gAOMsRsAFAG8WVfFksIzvFxS2GQRXb65joXzS+qbSodn0juyXqv4apw+OOI9XC5i1/hwx41LWpuNSkqXkO5V07CaAPSsilbWs9n7cKte4LrTioHr8hw+QdXlUgIwTEQlACMAzqhXSY40MryEocsHubS6gacXruio0hZUZVp//fBZfO2Rp3VWKVU6ld29dTxO65BTreJlZlpd1zcjdB0bJ+fVN5JlD9m0ttuJjBKU1PxYLKorE92pL3UibdAZY6cBfBDACQBnAVxkjP2NroolJctkBzp03hN+XAhZgxs3sVdV4/zqp7+Hf/2/9aZVSxPuBuuUtpqHXAD090G3amGjznBiXt4Qta89dSRPLhVIi+shq5WVLslxnk4nt6PicpkA8EYA+wHsBGAR0a0Bn7uNiA4S0cGZmRn5msaQRoaXMHZNDKNcKkj/oIwxLQ8/RShCPDXOoHJ7ZHH6Vgdu1ZZe8TDG8OUfnlE+/cs3G3X78nlfOaRx01fHpIS7HlRTAGZlGKesMrYNKwSu8ycLo0MDqI6qP1tpoOJyuRnAUcbYDGNsDcBfAPiJ9g8xxu5gjB1gjB1wHEfhck2CDBnP8JKFy6VYUE9Ht31sCPZgKdVlmw43xNmUXEK6cSvyLrdvPTGDd3/2+/jQPWpH2flmo/RAH+LwrqUw81cJ0sXr6Vb1BPrKynXhraLU9g646eEbrHlDxaCfAPBiIhohz8LeBOAxPdUKJ2pFnaWcSHlTifTIqKLQ0eny6CcMQuVw2WDJewweODKnXA9dm42t2IMl7ZmZePJklf7XVBep1etwhmkdVSXHHL5RnbfwCSo+9AcAfBHA9wA87Jd1h6Z6SeFWLRyb1ZvhJfRajo2T55eVduhrKY/yKmqcvZP5PuLcTnV0UHrFM8TPBWgwLHyzUbZfhHnRUklHpxhSdnSopOx62D42iCMKCUqSnkPjajjVkAO1iqdomutg+IQglFQujLHfY4w9izF2A2PsFxljepL2SZJGhpeoa23UmdIOvetYOHPxChZX1CPABZavMGut2GXp73aC5nJa3ihfkMoJutkQ6dhsDEJ20zfMjcPLPKI4y5QdaBpuG8fGpZV1zFzKxnToirmjM6aTTrrupGgUjY0eCSlVVMePupaKweNlpJXsoqF0Ueh0WcesSH7wv4mOFY/sJjCvd1qSNtexcOnKOmYu6zN8rmPh8so6zikY05qjlgKwseHbZUqXxsCQobtIhB4z6NkdAd+vIF1qROhLOagYV+OodLpumaED8iueVlukei4grbyTzUTHcr9lkGuipmHw4SkAZV0PWUcGbaSRlJr0NWnE4Tcz9PTgGV5kH6Yk/jh7sISrxobkrwVPalmQjAAnMiPSocZ5ZkHt9G2W6FjxqK5IrEH1zcYg0hj8ZVeZvOsRqc94r9o2hJFyMTPFCFfDyU76uIkoFAj7cxjTpacMOpDO5lHotRSTDwyWitgzmW7kNh3tkddQoe3oMHo6fgsvPrvEDDBijN4xNoThgaLW1aeO5MmqM2wiyvyUry4bkbZKTYauNOhRE2m3amVmgFzHVtqh52WkrUU/IaG6YAB2bPNireU5h2IrKisejo4Hned01Slp8zTueiWROpIny6YA3BQyoZJtUnPXsXFMQxpJ17FxqsNx+NvpOoMet3lZq9iYW1zF+QzkRLWKpbxD7zoWjs4uYiOlE5muY6PO5OJl7JsaQbFAiWeFaxt1PPXMpcTXU0XHikfFMHHcqi292Rjl9ktj9VmrqE2AVF0PBO++Tl9YxvJqNoax5qeRVFXDuVU7FUWTCl1n0OPgp9eymFXyJb7KDn3N8SLAnVGMABeGihpnoFjAXgkD+d++eQiv+uP7O2LU5VY8nlWu2HoO76SZvej0heVEM8LYmD++MVVJnuw6aoMCf2bTUnttuZ6m36eW0ga4Cr1n0Bs799lFXUzcmVsesrSDiqkGEpI5Kn3Czxz0/ZMXpK6pAl/xyMgPXcfC2YtXcFniXEDrzLr5m+rPL8qYXsMnE5WzvWW9Q3byrgf5zVlZqaQeQ9x4tnKkdOk5g97I8JLBDP2qMW+HXqZj8Hg0acu2LK7Gkex0rmPj2OxSIpdQGrFHRHEd+ZjX3LgdVfwtGpuN2rXo8u0anlCcn91Ifs88f0DNsZRSAO6vWH6+T333FcX4SNlLI5lw0sfY5jhSI+USdo0P50qL3nMGvVggL+efxI+VlEIjm4z8DzpplTE+ohABTgC3aiWOl8Hbo3n6VtxPOOWfMk0069O0gagymOhaLXnKDf3+7obh07j6bCRPVtSiA/L1GhrgCUqyjemiZQM8hdg9KvScQQeyjYSmqlIhktOKJzF/smocTwUhvzyVW7kk/somVFY8+yY9lYyOviPjV477dYbLRezcNqx1f4gnT1YxSs3kMmqDQpbRC13H0ha7R+WkrG6606DHPPSuY+O4RIYXGWPiOjbOXFTboXc15XeMKl9WjVOryM++OqFfl1nx8GexXOKbwPL+ZE5ayg1VmWFgmYp6au56UHM9enFlsoq/7zo25hdXldMFuo6FxdUNPLPQ0TBWDbrToMfQzPCSvpzIrVrKG1Vu1cbMpRVcXE52IlN0AOIzKBk1zoRVxqRVlp4VZn3KlGcOklk1EemTBjaO6idst7i8tq5j4fA5vYbPdWwcnVVLnqx6OMitWlhe28gs/r6OBB86y9FF1xl0kZVNTTH9WhJkrsWweZGRdqeQVuM0vm9J+0c7MUuvVeSX027VljoX0G6Im64q/UqX5TXxzEwid1FzbFxZq+PMRbGN5CD3AneZyLoeahoCySVB135J2vGYktJ1Bl0ElWwsSVHZoeeknaNQRo3T+liqzFo7onSRXPEA3mCg41zA9JTXL3QP0mkkQ5fdd9gs1VRzPTTOj2TUXxqB6xK519iW9VN1VD18gk560qCPDg1g+9hgJlr04bK3Q6/ygDUiwKXUmZvHxhM+sP7/a46FucVVXFgS8ze2TtI6JV0E5IyDjsNiQOtmo94+eLXkzDIq/6yOkLKqM17HHsToUCkzw1gsEPZPqStUdIRP0ImSQSeicSL6IhE9TkSPEdFLdFVMFW+jMUOli8IPOlAsYN9UukG6VOJlyB6UGR0qZTKotpN0xtm+GgH0LP15rHCdOKODGB0sae0rPHmyyvOSVC7aiNjo/7ux9yG5ipRBNbheoxxNae10oDpD/wiArzHGngXgecggp6goqoH3k15LdYfeVUguLFq+rBpHdvZVy3BQbWWP5IqHQJi0ypgYGRD2wUd1Lz6pEO0XIn2ViFDTPCNsRDxUGHx1HKaS0XSryFxdx8YJCTXc1nIs5fAJupA26ES0DcDLANwJAIyxVcZY9me9Q+CB92cvpx+ki29UqezQ1xwbx5NEgEs4dqiocXZPDHunbxM+bK6TXY7XVlRXPDVNETBdf7NRt3JDZZM6vEy1QYK7HlTkt65j45mFFanQCzLUHK6GU2tL1eQjOlGZoe8HMAPgE0T0fSL6OBFZ7R8iotuI6CARHZyZmVG4XEuZAgd+k84qVebxSX22jLEtMwvXjwB3MqV8qCo+zhI3kIJGpJGRKcMcr+2orHhkdNnBGYHSifXhOjaeXpCLORNVpmjy5CjtvQ4/fJaHAgHgkOLgmHY8piSoGPQSgBsB/A/G2AsALAJ4f/uHGGN3MMYOMMYOOI6jcLlkyAQdks1o2ZCoqXRmiY0pkYGNk1iN0zboyOxJJOnoTZ+q4lFRfu1qwhVP63cdG7OX5VQy7eUACR90gdvnewQiMWdEXY46kifXKmpJz69WOJUsg4zfP2jg5nH4u32GfgrAKcbYA/6/vwjPwKeKqEt8x9iQVOB9GRybb1QpLDdT1s7zeBnSWvSqhRNzS1hLYCCvruqX2IlSq4iveNr7lK6ZYsUuY2xI7wYmkM6MsKbhnhvBzQTcLkF5DfZOWlLx92VRTSPJGRpIP/OYKNIGnTH2NICTRHSd/9JNAH6kpVYaKBTIU3Zk0Mg6Nqq2jQx4EeBymo7OdWys11miYP7bhtVyvKogteLxZ1+6zgU0/MqaB7S9PPFIgnaN2zzcN+VtJKv8VjIDTWu9mqEXsusvNYmYO4HlKAbp04WqyuXXAXyaiB4C8HwAf6BeJX1kqQ9VDfIP+KqQlJUusmoc2SiGWeZ43XRdhRXPnskRDBTFjJtIBi3d9z9YKmLPhNrZh3ZUkycDelIA6niOkl3P+31U1XA6wifoQMmgM8Z+4PvHn8sY+znG2HldFdOB61g4dT5Zhhf5a4lvVEVuKqWpRXfk42UkOqHY8nDo0vomxVvxDEpnato3pUdD7lYtPLOwoj2mTRp9RbXMoYEiditGbvQMY3opGbdez8KlK+uYuawWXMutJgufkBY9eVKUU/MzvMgG3k9Cc1NJrDOoFL0gAAAgAElEQVQHrYBdx8L5pTXlCHBhJPENtz9OY0MDqI4mM5BE3gx1PqMcr+2oLKeThleNTSAhsoEpfDU0JII6DV/NsXB8Ll5m2tjADvDjuIqHqWTi76vQdM3pUrp01u3SlQZd9DBB48RghunosvJByjzGSdU47c0sM4PLMsfrlmsL1jdouc3PBSTZBA6rA6D//l3Hwup6Hac1SkJ1yEz5DFvW9ZAkDpOOM4NJfh/vesHGJy/p6LrSoIuSZdRFvlGllCw3ZR2uarwMmdO3WeZ43XrtZCue9giYaxsMJxVDMO+dHEmk3BAVbTb2NGIMUdJEKIDipKQqlgKwOcsProPoc6Qqc71qbAjDA0Xl/snDJ3Ra6dLTBr0ZOCv90X6wVFTeoecR4EQNbtJjz6qp0fjp27kYA9nalI0cr51Uukilo9OjdCmXCtiXgnJDZ8yZZpnqOvBaRa0MHn8/q/7SDFynIUhXxhu6QfS0QQeSBd5XTX/mpZKT/0GLBbl0dElQ6XQyUsBGjtcOdHSudJFZ8YjqsoXi86egXuIxZ4T7tsBnePJkpVVmVd2XnEZog+jr6dlg7pSiq5WeN+i6ZElC16raODoXv1HVnj28lbSTzoqqcYKaK+msld+hSL7INH6dpCueVrYNy6tk2nGrVirKjTQMiJDMMuI2GpEblWb52RpGni5QVQ1XSxA+IS1636BXbSytimd4UbqWho0q17Fx8vwyVtbTkVom8dO3Dzo7tw1Lnb5NkuNVdZXUiuqKJ0lMl6h6u5V0lBtphG1NIjMNumXuelDzwyeLv69KzVFPIwnoCZ+gSu8b9AwbWZfSZaPOcCLBicxk5cv7OAsFwv5K8uzszRyvndgYjZ/tNSac7Rt0mg6muaIxShJO4GuO5cWcWdI3I9SRPLkmEBgt6lazlgAmeW4jB+5quqIGEfrAoGeYXzRj6aIMqmockVlrWGyUTvjRa46FE/NLUise17FxQcO5gFolmXJDFFdQ6SJTplJMF8dLASjieghSqWQdvbARuC7Gbx833vLMY530o3elQU+yKq+ODsIeLGWiD21uVKkF+QfijZ/snoCqGsdzCS0J+Ru5y2a/ovJBBdexUWeQWvHoyk07YYnHtIlKFdeOyCZ10m6iI2ep6qp498SwcOgFHXA1nOr1dIRPUKUrDXoSmj697JZvKteyNEWAi0JERRAWo8StJj99m2WO13ZUZnuN/J0aDKZqRqAg9jQMn75ydchMVXOUlooFTE/Fq7Hi4ugkQVd2rU4rXbrOoMvMTLNsZDFFx9bs4ZvKENyYkt0/rDliapyg8rnOOOnsq1MdXSVy4s7xYQyW9Gjo07j/kh9zRsg9IjjzLxYI+yvRm5rckIYVqcP1INpeujbR+SRHNbiW69hC4RPSousMugw1x8LZ2MD7ekZ7b6NKbYfedWwcSTEfqooaR+SIc1C9+UCXhXy0FWuwhB3bhoRm2e3+XG7cdPi+XcdORbmhqigJLFMxoFrD9aCodEkaf18FnkZSVQ3nOlbHsnQBfWLQ+bI7TpakY7DXsQHoOjYuraxj5pJaBLio8gE5N8RIuSTlb6w5VmY5XoOufVhSkiY+s47uPU2li/6N0eOaDV+top48OU5SyZqjaGgdksbfV0H0mYizETqEESr0h0FXOAIuey2lgxWajp2HoaoiED192x4bReWaKqiseFwFlUwrInGFpAKu+YlHTijGnNlUpgaZqevYOCaZAtCrQ7b9RWQjV6T/6AifoIKyQSeiop8k+is6KpQGjcD7GShddGxUpW38JhTVODLuk6wf0E3XVljxuFV5lUwruyeGU4lpo0OV0o6O5Mk1xaTnadxXFM4oTyOp9vvoCJ+ggo4Z+nsAPKahHGGSboQMlvycf4onwUTgG1UqHeOqsSGMlIsZpKOLmo2Et7PrWFhc3cAzC+IGcocf1S6so/PBQeNB0QYNAyUT0yVmZi2qtCgVC5iujGh/0OOW+DJKEK3nKSQnUTLx91XQkUaSoxIATxUlg05EuwH8LICP66lOeriOnVms4riNKsYQabmaEeAiDK5C/QAxNU7Ud4EoI7KVgoB6Ii3iZntRqg2d7i+RGCVJB7RtwwNwRge19m17sITtY4MRg6/3/0illqMeBz9rZZSuoGBZyqTbUZ2h/wmA3wLQGY1OAlwnnQBJwdeylXfoaxJH7JPgVuXVOKLuky2xrjPM8dqKyopHRCUjSlrKDRGlS9KBQtWYctdDmIEUeQpl4u+rkCSNZFw5ncrSJW3Qiej1AM4xxh6M+dxtRHSQiA7OzMzIXq6B7E/rOl7g/TMxgfd1oGOjikeAW14N34xT0eA2XQnJZxLV0UFY5WJiI5dljtdWRFY8UbiOHeuuE/kteL/Qrdzg7jOdhk9HlFLVFYlo/H1d8FXF0Yh+Ivo7A53J0qUyQ38pgFuI6BiAzwF4JRH97/YPMcbu8BNJH3AcR+FyamQpJ9KRjsqt6okAF16+vBqHiBo5LRNdM8Mcr0HXlv09ao6l5VxAWhmpao6Ni8t6c9HqSJ7s6dnTO3Gqe+KuS4zQySxd0gadMfYBxthuxtg0gDcD+CZj7FZtNdNMXCxvnZ2jOXjkV+kSp8aJyp8IRBvIsLZsDnQdCNJVsXHmYvSKJwxd5wLi/PGyA4au7EqtNBN8qPVhlaTnIvelM9wyD1yn+sw14/B31wy9q5i0yhiPkerp6hx8oypqJhaXC5FHgEtL/sTjZchLFy2ciT19u5ksc7y2E7niibGjUSqZJDZ4NCXlRhqDvw6ZqeqKRDb+viw8cJ3qM1csEPYrKt1k0WLQGWPfYoy9XkdZIsgkhvWCdOVD6SJiBIYG9ESAi0IlOxKfwUW5hNp/J11R7WQQMXphvYqf8tQVAkD3/e/yY84EGk7JleeOiOTJvMi4yJCq0Spl4++rUNOkxHKrnckv2jczdMD7sZL6faWvpWGjKm3ZlooaR3ZW6Fb1Z9kRoRHzWqI9dZ4L4A96WL+QWSU2JaH62lVH8uRm5MaAQYFLH2NuOGsJIN8bUlXDJcnSpZO+Muhu1Qu8f3E5/Zx/rr9RpbJDz+NhBEWA0+HzV1HjNE7fJo66aCmrJ2RorniSGweieJWMqB2uVdT7RRBxklCZgUL1gEwzuJma20Y0/r4O4gLXiXoHak5nsnR1nUFXsQNpqQyCr6W+TK85lpYIcGFEqQjimnlowD99G+RXjvhezQnO8dpcxsdcWAHVw1R6tOhqJyhDy3VsnJzXa/hcx1JOnqwauZHn+8xKGRWVBSqJ7dERPkGGrjPoKqShBgi/Vvrp6JJktwkiTnURV3xsAuaA77sdVLqErXhEnlPXkVfJbC6Hn6DUHXXRQp1Bq8ady0xVpLM8cqNscDNXg9omCQ2FmuKA21AJZaxF7yuDvscPvJ/FDJ1vVIV1DKEDClV1PXsUY0PxapwoXMfG0dlkSQGu7lBHB7z2XF7bwNmQFU/UAKkrM3xDuZHCDB3Qu/qMn1AIlMEjN0oONDrOdCShmUZS7Xe2eeYxM0NPj4FiAfsUA++LoiN2iWMPYnSolOqKQiVBglv1Tt+eTnD6thHVLqMHtJWGbFLi2rqkgVy5kdRVFUcaeVsbZbYZpURRNhXPZETF309rF0aXGEF1U1mGvjLoQHiUwTQ6h8xpyla8zTg9uQ7DUDk2HmbkosoS2WBMi6b8MHl7qqhkttQjhfu3BkvYuW1oS7kq/ZrLTFX6X/gJbfHomtH9Rf+mS1xyjiTlZC0A6EqDruI6dqs2jocE3pfRt0deS8NGla4IcGHwY+PtqguRTigbibBT+UVVVjxDA0XsnhgOfdCT7Ge4jo1TIcoNlR6YRvAz1TJ55EbVvaQs0xd6aSRXcHFJTQ3HwydkmaWrKw26CrWKWuD9JOjYqNIVAS68/HA1TpxxmbLK2DY8EDrjDY2nXrUFcrzqp3G4bMuKQuz7IsGmROBJM9II0hWlcZcrUz15clzsfZE6JI2/r0KY0oWBJZpMdiKpS98Z9LRkY4HX0qh0aTeaMokLosqXqaNnIJP7CRtR7TI65NVKlF8z7mGNOheQqA4p+LsB794ur6zjXEDMGdnVZ01D8mTVGXbW6Qt12YhO5BftOoOuasjcDOOJ8E2lLcaYMeHHS4eePYo4NU4cMrOvTibSdR0bzyysSK14glQyMjYqLeWGapag4DK39r/GmYEEZahEbsx6pqsjjSQQHT4hLbrOoKuybWQAFTub1FZhG1VJ0BUBLgxVNU7N8U7fLlwR9zdmmeO1HRV5ny5pIFdu6Nei6zd8V2sos6ntbhkUGkf/478vG39fFh64TvV35uETspTo9p1BB/z41hkeVFB5GHgEuKAydG3hqqhxZHzwjXua3fqAp83VVXl3h86ZdZDrR7UNto/5hk9j39aRPFl1hi0bf18FXZLDrAUAfWnQs2xkV0MarbSVLmFqHLGDI3LL/LCj9KqnX+PYO2l5K57W2aKgGy9KJSOV4k1zejUuc23t26rF60ieHJcgXIQsI6Xy6x3XkC6wlnGWrq406KrPvOtYSoH3E12ramNxdSNwo0qUmmPj6Fx6+VBV1Dh7/dO3SR/4WoY5XlsplwpezOuAZXBct+IqGR1L6LSUG24Kq0+3ojah0BG5USb+vgphgeuS2p6ss3R1pUFXJWgJmJbGVcdGVVwEOFVUfK9Bp29FmjLLHK9bry1voLyZop5DJ8BWf7zqCoXnol1a1Wf43Kq6dFbV9RgUfz9NN10zRaPa9bJOR6eSJHoPEd1HRD8iokeJ6D06K5YmXOmSRUyXoCD/DMlG+rRlW2FqHFFqEifrOqHRbVzbj0EjszqoOdYm4yarukpL6cPbtV0SqnQYry15csOwJTpMpRa5MewZSMtDp5qcg6PzhLEIKjP0dQC/yRi7HsCLAfwaEV2vp1rp0sz5p6/Th9FMjqA3v6jO2YmqGsd1bBwLOH0bNdtUjfGhguvYWN2o49T55C4mXUqXNDYwAfnTu1HoOk/RGtysKX0Ue+hk4+/LwgPXqfrth8tF7NyWXZYulSTRZxlj3/P/vgTgMQC7dFUs/LrqZRQLFB/6VRNhpxOTMKEpAlwU7Ue8GRN/2Fwn+elbkRyvaVFrU+Yk6VNXa0pHx5Ubuu9/esqfEWrs23sbxjS9UNBxRMXfTwuVwHWbyskwS5cWHzoRTQN4AYAHAt67jYgOEtHBmZkZHZfTQtZKF9UfNO361irRqdGi0K10SRuV5XtDJaNJ0rb5wI76bGVooIg9E03Dp6NMHcmTwyI3JiGrSRhHJXDd5nKyy9KlbNCJyAZwN4D3MsYW2t9njN3BGDvAGDvgOI7q5bRRcyycPL8sHXg/2bXUN6oCs+1odBG5VTv02Hjsd/mexGwyI+I62eV4bWXCKmPSKksZZa6S0eHLrVUs7RuYQDrRHFUnFDoShMvE31eBB65rVcPJhFBwQ7J0pYGSQSeiAXjG/NOMsb/QU6VscB1bKfB+0msBm5fpSTuGW7Uwe3lVOQJcaPkKahzv9G15y+wr7g5dJ7scr1uvraJ00XMuIEhJoQPP8F3WaviCkicn1t5X1SSfMvH3VWjPcCbbms0DaelPXlRULgTgTgCPMcY+rK9KQldXLiHLgD+NONyzyX22nEZyhpSOEau2h4wsLcscr+2oRE50W84F6Mhx21oPHYuummPjylodZy7qM3y1SlM6K+vGaY3cmOTof/P7GQfp0nS963eM4T++8TkNw54mKjP0lwL4RQCvJKIf+P+9TlO9UicNNUAYOjaq0o4S2a66SBwqVMKgt/4GuqJHiuJWLcwtruLC0mriK9c0nQvgyg39M/StfVt1oGj0P8VkFyqRG7PMCQw0A9epTjjGR8r4xZdMY+f4sKaahVOS/SJj7NtII11IRlg8519j8yg9eHIElZFeVwS4MIKOjSeh9fSt6Kx1z+QIBorZ5HhtJ1g2Karqac7cJq2ydB28fqFfudE6+F+73dZTZotL7jk7xyTLaGq7B0vFxN+f3BJ/P91JQDNwXfb7PLJ0xUnR7504j7sfPKW9XLea3Y+lGuSfR4BLc7mposaRcZ94p0yzz7sIqC2ndS7909jAbCQe0eie05E8+eqAvaQkBMXfT3tGmYa0NE26wqB/+Qdn8HtffhSM6V2Yu46NIy0BktLsHDo2qlqVLqnkQFVQ4wT6g0WCe3Uov+juxoon+cPaVMm0uDQkO09rv9ClamvkbT23qPUAmurxfS2RGxUnRjLXOzm/1FTD5dwn0RUG3Y3IxKJWro1LK+uY0Vxu2LVaN6qkZG6OpSUCXHj5wcfGRQg7fStyzeMpBh4LoxnzWs441BRiyLfiVvVvYALpnFtoX8El7cPNBOFqe0lJ4++r0Bq4Lstkz7J0hUGvOelsCPJNuUOZxnSR3wBsjwCnPal1taWOLNkDWywQ9kskBXAd2ztlOt+JIF1qSpcjM5eVV0rNdHT6pYvnLsllZooqc/byChYUZKY8uJnsqrhWSTeDVztpZIFKk64w6O3LeV0xV4L04WmhJepiykoXVTUO35NIYuRaN8qyxq1aODG3hLV1b8WTLAGwfy5AUUO/5TfV1rfVAq4Fl+nV9ZCCnppHblyUPEyVZU5gQF+QrqyQVrlkyfbyMvaXL+Lc6WOosHmAVcW/vLYMLJ8PfOsqMEyXL+Lc6aNw2DzAdqhVNOJaFcZw9dACZs4c9e8h5NTsyiXvvwDcoXVsxzyeOX0UDs4DbDp5HddXgKW5wLeGALxg2zJmzxzFVH0RxEa2fujSMwALPl37vLEl/ODRUxhYHMAIrjRXECuXgZUth4g33dPTJ4+ggovJ7yeOxVlgIzju/fXWIqbqc5g5cxRjCDBSS/PAerDE7nrrsl/vw9iGmId97QqwPB/41hRjuKbRL+YANhFdVitXLgKrwcb1muFFr36njmISC+KD1ZUFYDX4fpplHsEkgn9PANHtZvN2O4qxsHZbXfTuLYC9pTp2Fc7jmdNH/Odoe+TtCBHxTIwAeO7YImbPHEOlvgBiEaqh9VVgaTb8/eFJYGBIra4xUJZ+oQMHDrCDBw8m/+Jf/Sbwjx9v/PPe7e/Ezb8qeJbpI88Hzh8V+ui9O/4Vbv5Xf5i8fpzbXwTMPC52reo7cPO7/mTzi2vLwAevDTV+7dyz73141Tt+L1kd73w1cHJLyJ1Avjn+T/HK936i+cLBTwBfea/Qd+eZjeEPHMFwCcAHrwl9QNu597p/h5vf8htCn43lya8Dn/kXQh9dZwU8/s+/hRtueJ73wsnvAne+Sui7dUZ48Gfuxgt/4qbgD3zsp4CnHxIq6xtTb8VNv/7f4z+4OAt8+HpgQ2z/5/4X3o6X/eyt0R9amgc+/OxQY9zOt278r/jpW962+cWzDwF/+jKIbNtvMMJDb/gqXnDgJ5ov1uvAh64FFsXiPt171a/g5n/9IaHPhvLxm4FT/yj00W+Mvwk3vffO4Dc/dQtw9G/Dv/zWu4FrbpaoIEBEDzLGDsR9ritm6Pixf47PnxzH0dlF/Nr6pzC2dk78uwtnAPcm4PpbAt/+7HdP4vjcIt69fhfG1hSDhy2cAfa/DLjhnwW+/ecHT+Gpc5fw7o0/w7age+Az2RveBOz/qcAy7rj/CNbrDP9y4XaMriZoh9Y67v5x4AVvDXz7Kw+dxQNH5/Ce4hcxvj679bsA8IaPBH731PllfPS+Q3ij/SO8ZOXvsby+AtTXPWP+nJ8Haj8d+L0/vf8INtbX8a7F22GvSNxTGAunvf+/6j8AQ9u2vL28toF//39/hJePz+C1i1/CwJWW35/f68vfD4xtXblt1Bl+90uP4MD4Zfz85c9hcDmi7yycAfa9FHhu8ODyhYOn8OS5S/i1jU9j27rg/S/Necb8xl8Cdv2TwI986G+exFRxCW9f+gSGrwiUu3zeM+YvuBXY/cLAj3z4nicxUbyCdyzeiZGg3+rS0wAY8JO/AUxMb3l7baOO3/3So3jJ+Hncsng3yu3tVl/zjPmzXg9cEzyg/tl3jmP28gpuu3IXxmSegXYWzgB7XgQ8/xcC3/7yD8/gwePn8e7CFzCxHvM773g+cOAdwe9Xn6Ve1xi6w6DvfTHOXTOFjx17Em8f/DwKSKDyYBvAjucB/+TtgW+fO/8U/vTEk/iV8mdAIa4EYeobwFXPDb3W7KVD+J9Hn8AvD34BFHQP/Pr7fiK0jOMnHsZXHjqLt+FPUWASapf6BuBcG1r+wvoJ/K8nHsatg19Hob092AZAxdDvjq+s43P3fh1TKyt4Cf7e+zyv497oe/q/3z+JdxVuV/8NWqn7ZT3vLYC91U03DOCb37gXl678EK/Fl7zZIYfX4zk/H/ggFgF89+/+FieWnsDP43OgqN+CbQDV6yP6xWH8z6OP49ah/yP+m/J7c1/p1TGAxx89iKcOH8Lb6RPR9Wsvs/YK4MfeFPiRJ350EI8fPoJ34M7gMnm7PfsWYNeNW94eAPCdv70P5648iltwN9D+HPA67D4Q2l5nzj2OO799BLcOfCaZLQijvgE414Ve78LqMXzqyUdx6+BfR1+PbQBTV4eWkwVdsSkKNDdD1lEQ65yc+gZQCD+V5lYtMAZsgNQ7B9sAKLxJ+abSOisEP7i8M0fV148At4ECCBLGjxvl0PItv46EQvuyOaYtbf/0bWOfsF5vuafodrm06rWHlgeUw9s44n5rjoX5Za+OmwYwbtxjfovZRX9zL2ogqtdjyvHbvE7BA30Q/HqRv6WNSyv+hq/IQNkoM/y3qjk2Ll7hZcr34fml9eB6Cd2XF39/jSW0BWHEPhP8uaXo68U8H1nQPQbdb9Q6S2B4GQPAhH6sDRTUjUnc4BF3LYHOzHfd6wgZFBTrWGutY9gMPQK3amHD71aEjQT3RNiIe2CSIjiY1Jn/PguYoUcat5Z7jZuhR95/a5snnKFH/paC9UtQpuvYqDfKDBgkBH/v0HqJ1KHKbUEBBZlJTTuqzy2H1WOfj7TpGoPOAxkFGpowBDoHz/lXTzrzDyLmwd07OYJigbzZddA9CD5QAPwZukR9Y+pYscsYGyp5A8aW5XD0TBPwohjyhxX1DaF7unrTA6PR5SI4i93w1TibfhPB32Ijyri1lhUxqOybGkHJ7xfC9y94b5HGV6rMmEEiYbttWdkIrKp4/H0tkzBeh4jr8cB1sQNuzO+cBV1j0Hkgow0UxQ0vi+9cQwNe4P0NlmCgCLwW8zpjxLXKpQL2TY4EG0tAqDPzCHD1JANbKzFGmadG2wga4Fj8ktJ1rIYR8Xzo8UZip39PiX5bEQRne/Ug49LoO+HbTK3fjTSYMQZjoFjA3qmRZKsuIZdQ0/gKlSs0648ZxEQM8qZ2Sz5D5/H3A/uoDALPBL/vyAFXYAWbNl1j0IFmBxUelevxxsQr15af8XIEOjLQfCBkZzeFAkWXEVtPAbdJo/MGPGyxLpe22ZfAPRX9qHahKxdZRJb/lZAZp0DfaXUdbNnca0XAt5q4Dwq4hMZHyhi3PN2zULkCfXjb8AAmLC8MbOBzqLqyEbgvwHuOpCc17cTsfQGCtsf40JPhJv0RBWbom8tVMOgC/lrA8zGHz9BFO7Ml754QWBbWHF7HgIdNqC19F0a9LjzQ8Vmb3hl6/Cx21/gwikVvFk5BPvSI744NDTQNZlifZExoEG20uUYfOgDsq4xG1y+wzOj+Me1ElCnQhyetMqwhL+ww1ZPP0AFNk7DWawraCJW9kixQTUH3GiJ6gogOEdH7dVUqDD77EzZkdV+BIPAwJfLNK1zLrXg+28Br8TJEOheTnM3W18Vm6EEuKIHvXjU21DCQqK+LD3QVDb9BOwK/SaFA2DHh+WSJtRxHF1zd7Znyv1sPM+jxgwrQuirS50MHgP1VbnxFXC5ifXh/dcwvU74P75rkbR7QxwTq4DYGQF0zdIGVJyugGPX7COwxpY1KCroigNsBvBbA9QDeQkTX66pYEG5SV4PADI2XW5eVAXJEVwPVCMMl7CLiZUi6XISNS/JN0UKBMDU63LyWoOFp+O3T2BSNGUx2TfqpwYJcLjHf3TMVMwNO6PbTPUPf72yLrl8rwn14FHVG0Rv7Mfe7O8ygJ1hVa5EaA0IzdD7p6+UZ+o8DOMQYO8IYWwXwOQBv1FOtYBIvSwVdGG7Vwro2l0v0WS1PBVKMPlgUU0biga1RPt+4jS5/39QI6lTc2h6CHbYy5hvITSqX+HvS73LZiL0uAOyc9I1y6yxbcCDaW+Gz1ZB6CxsoCxusKD6gifaV6qi4HFS0DzsRz4tgvfb4rqAtKpfGJEzkGSiq77lwaXPM9aanLNQp/z50lZOiuwCcbPn3KQAvUqtONFNWGaeLRexbesiLmxLHhn/CJaaRHXsQs8USpi9/X6zcIARnJhNWGaVSCfuWHtl6rbUlofrWHAunUEDt4t8nq28jM290+QPFAkYGy9i5+qPN5S+cAUYmYy+zc8IGzgLlL/xC85oC97SAAq6ZuVf+N2jn8jmhAYi7TXZ99/eBR2/3XuTBmuJ81L4/efrR24ETn9/6AUGXy/hIGcVSEXuWnxC7/xUenTG63OuuGsUGCrj6xOeB2yPijADNQF9xZW4fRR0FuKfuBm7/f5vf5MHpYn373kC49wcfBg59svnGuh+bJmYStmtiGLNUwPSioC0IQ/CZGBooYqhcxq6Vx8Ovt77c1QZdCCK6DcBtALB3717VsrDr1e8BHfsaUBQMH7frRu94dEy5e1/7PhSPfA0oKMQv3fG80PgTrUy/9j0oHP5q8LX2/WRobA7OSLmEheffhrXF72JwMOFPuP05wLN+NvZjz7nl3Sg8+ueb6+hc58WqieHlr7oFF9cfwLaS7w/d91IvfkwEI+US1l7xPpSe+Ud9WWGc64Crfiz2Yy994QE88cS/wLX2yuZrT+wHBgIiTrbw48928cM9t+IGayG871z1Y8C1r4mtx1U3/Ro2jn8FGBA0CoOvAKrPjvzIzvFhzL/4fRhfeFKsXb+CSC0AAAUtSURBVGsvB666IbbMmRe9D1OXHg8uc2w3MDIVWcaLn/ccPPnwL+Dq4cWtZex5ETD9k5HfLxYI9OO3YWXuWygPKZoxwWfi2tf9OvDE3eG2p/ps4PpUnRSxSEdbJKKXAPh3jLGf8f/9AQBgjP2nsO9IR1s0GAyGPkY02qKKD/0fAVxDRPuJqAzgzQC+rFCewWAwGBSQXqswxtaJ6N8A+Dq84HN3McYe1VYzg8FgMCRCyfnEGPsqgK9qqovBYDAYFOiqk6IGg8FgCMcYdIPBYOgRjEE3GAyGHsEYdIPBYOgRjEE3GAyGHkH6YJHUxYhmAByX/HoFwGzspwymncQxbSWGaScx0mynfYwxJ+5DmRp0FYjooMhJqX7HtJM4pq3EMO0kRh7aybhcDAaDoUcwBt1gMBh6hG4y6Hd0ugJdgmkncUxbiWHaSYyOt1PX+NANBoPBEE03zdANBoPBEEFXGPSsk1HnDSK6i4jOEdEjLa9NEtE9RPSU//8J/3Uiov/qt9VDRHRjy3fe5n/+KSJ6WyfuJU2IaA8R3UdEPyKiR4noPf7rpq1aIKIhIvouEf3Qb6d/77++n4ge8Nvj835YbBDRoP/vQ/770y1lfcB//Qki+pnO3FG6EFGRiL5PRF/x/53fdmKM5fo/eKF5DwOoASgD+CGA6ztdr4zb4GUAbgTwSMtrfwjg/f7f7wfwX/y/Xwfgr+HlgXkxgAf81ycBHPH/P+H/PdHpe9PcTjsA3Oj/PQrgSXgJzE1bbW4nAmD7fw8AeMC//z8H8Gb/9Y8B+FX/73cB+Jj/95sBfN7/+3r/eRwEsN9/Toudvr8U2ut9AD4D4Cv+v3PbTt0wQ888GXXeYIzdD2C+7eU3AviU//enAPxcy+t/xjz+AcA4Ee0A8DMA7mGMzTPGzgO4B0B8XrQugjF2ljH2Pf/vSwAeg5f71rRVC/79+klJMeD/xwC8EsAX/dfb24m33xcB3ERE5L/+OcbYCmPsKIBD8J7XnoGIdgP4WQAf9/9NyHE7dYNBD0pGvatDdckT2xljZ/2/nwaw3f87rL36qh395e4L4M0+TVu14bsRfgDgHLwB6zCAC4wxPxHspntutIf//kUAU+iDdgLwJwB+C4Cf7RtTyHE7dYNBN8TAvHWdkSv5EJEN4G4A72WMLbS+Z9rKgzG2wRh7PoDd8GaLz+pwlXIHEb0ewDnG2IOdroso3WDQTwPY0/Lv3f5r/c4zvnsA/v/P+a+HtVdftCMRDcAz5p9mjP2F/7JpqxAYYxcA3AfgJfBcTjyLWes9N9rDf38bgDn0fju9FMAtRHQMnqv3lQA+ghy3UzcYdJOMOpgvA+Dqi7cB+FLL67/kKzheDOCi7274OoBXE9GEr/J4tf9az+D7K+8E8Bhj7MMtb5m2aoGIHCIa9/8eBvAqePsN9wF4k/+x9nbi7fcmAN/0VzpfBvBmX92xH8A1AL6bzV2kD2PsA4yx3YyxaXh255uMsbciz+3U6R1kwV3m18FTLBwG8Dudrk8H7v+zAM4CWIPnf3snPN/cNwA8BeBeAJP+ZwnA7X5bPQzgQEs5vwxvQ+YQgHd0+r5SaKefhOdOeQjAD/z/Xmfaaks7PRfA9/12egTA7/qv1+AZmkMAvgBg0H99yP/3If/9WktZv+O33xMAXtvpe0uxzX4aTZVLbtvJnBQ1GAyGHqEbXC4Gg8FgEMAYdIPBYOgRjEE3GAyGHsEYdIPBYOgRjEE3GAyGHsEYdIPBYOgRjEE3GAyGHsEYdIPBYOgR/j9jyutVn1cXtQAAAABJRU5ErkJggg==\n"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Clean up unused RAM \ndel y_raw\ndel chunk","execution_count":9,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Making plans for future modifications.**\n\nAs per the FDA document relating to Medical Devices, modifications are divided into three broad categories as follows:\n* Performance - Example: re-training with new data sets within the intended use population from the same type of input signal. \n* Inputs - Example: modifications to inputs and/or algorithms with no change in intended use.\n* Intended Use - Example: Change from a laboratory prediction to a real-world use and/or change from informing to driving or taking action.\n\nIt is clear that if this Kernel is to ever be put into use serving the population living in earthquake areas, there will be a change of intended use. In addition, a change to improve performance as well as inputs will likely also occur as part of this.\n\n**Additional Details**\n\nWhen it comes to all changes, we want to make sure that the device is accurate, reliable, precise, and achieves the intended purpose. Because of this, we need to understand the meaning of these terms and relate them to our application.\n\nAccuracy and Precision: Accuracy refers to the closeness of a measured value to a standard or known value. This competition uses Mean Absolute Error to measure accuracy, and this is a valid measure because it is of the same scale as the Time_to_Failure. Precision, on the other hand, refers to the closeness of two or more measurements to each other. We may want to add training samples that are include additive gaussian white noise (AGWN) to see that small changes to the measurements do not yield large changes to the output (especially with regard ot pulling the alarm).\n\nSometimes precision is equated to \"reliability\", although the latter is a measure of how dependably an observation is exactly the same when repeated. It could be over several earthquakes, or over several sensors measuring the same earthquake.\n\nSometimes accuract is equated to \"achieving the intended purpose\", although the latter assumes that indeed we have chosen the correct training set that allows us to generalize to the real world population of earthquakes.\n\n**What does that mean to the software?***\n\nI'm still trying figure out what \"planning for future modifications\" might mean. Below I am trying out one idea.\n\nIn all of the above situations, it means that we must use \"a priori\" methods, such as the \"scaler parameters\" calculations below, on the original training data, and then AFTERWARDS make additions to our training set (adding noise, additional sensor measurements, etc.) otherwise we will not really be training to improve accuracy and precision.\n\nSo for this reason, we create and scale the training data set, and then afterwards add some AGWN for additional sanity check to see if our precision and accuracy are greatly affected.\n\nOther ideas?"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Make a CNN Model to learn features\ndef make_model(linear = True) :\n    \n    scale = 1\n    droput = 0.1\n\n    # use a simple sequential convolutional neural network model\n    model2 = Sequential()\n    model2.add(Conv1D(int(12 * scale), 3, strides = 1, activation='relu', input_shape=(rows,1)))\n    model2.add(MaxPooling1D(pool_size = 10))\n    model2.add(BatchNormalization())\n    model2.add(Dropout(droput))\n\n    model2.add(Conv1D(int(100 * scale), 3, strides = 1, activation='relu'))\n    model2.add(Dropout(droput))\n    model2.add(MaxPooling1D(pool_size = 2))\n\n    model2.add(Conv1D(int(200 * scale), 3, strides = 1, activation='relu'))\n    model2.add(Dropout(droput))\n    model2.add(MaxPooling1D(pool_size = 2))\n\n    model2.add(Conv1D(int(300 * scale), 3, strides = 1, activation='relu'))\n    model2.add(Dropout(droput))\n    model2.add(MaxPooling1D(pool_size = 2))\n\n    model2.add(Conv1D(int(200 * scale), 3, strides = 1, activation='relu'))\n    model2.add(Dropout(droput))\n    model2.add(MaxPooling1D(pool_size = 2))\n\n    model2.add(Conv1D(int(100 * scale), 3, strides = 1, activation='relu'))\n    model2.add(Dropout(droput))\n    model2.add(MaxPooling1D(pool_size = 2))\n\n    # Include a couple of dense layers in case the classes are not linearly seperable by this point\n    model2.add(Flatten())\n    model2.add(Dropout(droput))\n\n    model2.add(Dense(int(100 * scale), activation='relu'))\n    model2.add(Dropout(droput))\n\n    model2.add(Dense(int(100 * scale), activation='relu'))\n    model2.add(Dropout(droput))\n\n    if (linear == True) : \n        model2.add(Dense(1, activation = 'linear'))\n    else :\n        model2.add(Dense(1, activation = 'sigmoid'))\n       \n    \n    # binary_crossentropy is used as the error function when we have only two choices, like \"pull_alarm\"\n    if (linear == True) : \n        model2.compile(loss='mse', optimizer='adam')\n    else :\n        # by all rights, should probably use binary_crossentropy below, but seems to get stuck if I do (mse works). Will investigate further at some other time.\n        model2.compile(loss='mse', optimizer='adam')\n        \n    return model2","execution_count":33,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# we will be slowing down the learning using \"Dropouts\" (see above) so the patience needed to exit local minima can be large\npatience = 15\n# Probably will never reach this many epochs, but want to use a number larger than what we expect\nepochs = 300\n# Divide the data into 15 different versions of training/validation\n# n_fold = 15\n# Using KFold instead of StratifiedKFold becuase there is a low degree of confidence that the test classification distributions \n# or more importantly, the real world classification probabilities, are equal to those found in the training set\n# folds = KFold(n_splits=n_fold, shuffle=False, random_state=1234)\n\nsam = X_train.shape[0]\ncol = X_train.shape[1]\n\n# sam_test = X_test.shape[0]\n# col_test = X_test.shape[1]\n\n# prediction = np.zeros((sam_test, y_count))\n\nmodel = make_model()\n\ncheckpointer = ModelCheckpoint('LANL_value', verbose=1, save_best_only=True)\n\nearlystopper = EarlyStopping(patience = patience, verbose=1) \n\nresults = model.fit(X_train, y_train, epochs = epochs, batch_size = 32,\n                    callbacks=[earlystopper, checkpointer], validation_split=0.2)\n\nmodel = load_model('LANL_value')","execution_count":37,"outputs":[{"output_type":"stream","text":"Train on 3280 samples, validate on 820 samples\nEpoch 1/300\n1184/3280 [=========>....................] - ETA: 19s - loss: 27.4795","name":"stdout"},{"output_type":"error","ename":"KeyboardInterrupt","evalue":"","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)","\u001b[0;32m<ipython-input-37-3b1bff743a7c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m results = model.fit(X_train, y_train, epochs = epochs, batch_size = 32,\n\u001b[0;32m---> 26\u001b[0;31m                     callbacks=[earlystopper, checkpointer], validation_split=0.2)\n\u001b[0m\u001b[1;32m     27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     28\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mload_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'LANL_value'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)\u001b[0m\n\u001b[1;32m   1037\u001b[0m                                         \u001b[0minitial_epoch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minitial_epoch\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1038\u001b[0m                                         \u001b[0msteps_per_epoch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msteps_per_epoch\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1039\u001b[0;31m                                         validation_steps=validation_steps)\n\u001b[0m\u001b[1;32m   1040\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1041\u001b[0m     def evaluate(self, x=None, y=None,\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/engine/training_arrays.py\u001b[0m in \u001b[0;36mfit_loop\u001b[0;34m(model, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)\u001b[0m\n\u001b[1;32m    197\u001b[0m                     \u001b[0mins_batch\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mins_batch\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtoarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    198\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 199\u001b[0;31m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mins_batch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    200\u001b[0m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mouts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    201\u001b[0m                 \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mout_labels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mouts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2713\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_legacy_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2714\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2715\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2716\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2717\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mpy_any\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mis_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2673\u001b[0m             \u001b[0mfetched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_callable_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0marray_vals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_metadata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2674\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2675\u001b[0;31m             \u001b[0mfetched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_callable_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0marray_vals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2676\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mfetched\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2677\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1437\u001b[0m           ret = tf_session.TF_SessionRunCallable(\n\u001b[1;32m   1438\u001b[0m               \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_handle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatus\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1439\u001b[0;31m               run_metadata_ptr)\n\u001b[0m\u001b[1;32m   1440\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1441\u001b[0m           \u001b[0mproto_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mKeyboardInterrupt\u001b[0m: "]}]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(model.summary())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# SCORE FOR SCALED TRAINING SET\ny_pred = model.predict(X_train)\nplt.figure(figsize=(6, 6))\nplt.scatter(y_train, y_pred)\nplt.xlim(0, 20)\nplt.ylim(0, 20)\nplt.xlabel('actual', fontsize=12)\nplt.ylabel('predicted', fontsize=12)\nplt.plot([(0, 0), (20, 20)], [(0, 0), (20, 20)])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"score = mean_absolute_error(y_train, y_pred)\nprint(f'Score: {score:0.3f}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**As noted above...***\n\nI'm still trying figure out what \"planning for future modifications\" might mean. Below I am trying out one idea.\n\nIn all of the above situations, it means that we must use \"a priori\" methods, such as the \"scaler parameters\" calculations below, on the original training data, and then AFTERWARDS make additions to our training set (adding noise, additional sensor measurements, etc.) otherwise we will not really be training to improve accuracy and precision.\n\nSo for this reason, we create and scale the training data set, and then afterwards add some additive gaussian white noise (AGWN) for additional sanity check to see if our precision and accuracy are greatly affected.\n\nOther ideas?"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a version of the scaled training set that has 20% additive gaussian white noise slapped on top.\nX_train_random = X_train + (np.random.rand(X_train.shape[0],X_train.shape[1],1) * 0.2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# SCORE FOR RANDOMIZED TRAINING SET\ny_pred = model.predict(X_train_random)\nplt.figure(figsize=(6, 6))\nplt.scatter(y_train, y_pred)\nplt.xlim(0, 20)\nplt.ylim(0, 20)\nplt.xlabel('actual', fontsize=12)\nplt.ylabel('predicted', fontsize=12)\nplt.plot([(0, 0), (20, 20)], [(0, 0), (20, 20)])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"score = mean_absolute_error(y_train, y_pred)\nprint(f'Score: {score:0.3f}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del X_train_random","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"** Now Back to an alternative Purpose **\n\nWhat if instead of training to find out how much time there is, we instear train to find out if we should pull the alarm or not..."},{"metadata":{"trusted":true},"cell_type":"code","source":"# we will be slowing down the learning using \"Dropouts\" (see above) so the patience needed to exit local minima can be large\npatience = 15\n# Probably will never reach this many epochs, but want to use a number larger than what we expect\nepochs = 300\n\nmodel_pull_alarm = make_model(linear = False)\n\ncheckpointer = ModelCheckpoint('LANL_value_pull_alarm', verbose=1, save_best_only=True)\n\nearlystopper = EarlyStopping(patience = patience, verbose=1) \n\nresults = model_pull_alarm.fit(X_train, y_pull_alarm, epochs = epochs, batch_size = 32,\n                    callbacks=[earlystopper, checkpointer], validation_split=0.2)\n\nmodel_pull_alarm = load_model('LANL_value_pull_alarm')","execution_count":38,"outputs":[{"output_type":"stream","text":"Train on 3280 samples, validate on 820 samples\nEpoch 1/300\n 736/3280 [=====>........................] - ETA: 30s - loss: 0.2405","name":"stdout"},{"output_type":"error","ename":"KeyboardInterrupt","evalue":"","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)","\u001b[0;32m<ipython-input-38-3e369bf5362b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m results = model_pull_alarm.fit(X_train, y_pull_alarm, epochs = epochs, batch_size = 32,\n\u001b[0;32m---> 13\u001b[0;31m                     callbacks=[earlystopper, checkpointer], validation_split=0.2)\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0mmodel_pull_alarm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mload_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'LANL_value_pull_alarm'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)\u001b[0m\n\u001b[1;32m   1037\u001b[0m                                         \u001b[0minitial_epoch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minitial_epoch\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1038\u001b[0m                                         \u001b[0msteps_per_epoch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msteps_per_epoch\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1039\u001b[0;31m                                         validation_steps=validation_steps)\n\u001b[0m\u001b[1;32m   1040\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1041\u001b[0m     def evaluate(self, x=None, y=None,\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/engine/training_arrays.py\u001b[0m in \u001b[0;36mfit_loop\u001b[0;34m(model, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)\u001b[0m\n\u001b[1;32m    197\u001b[0m                     \u001b[0mins_batch\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mins_batch\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtoarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    198\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 199\u001b[0;31m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mins_batch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    200\u001b[0m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mouts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    201\u001b[0m                 \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mout_labels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mouts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2713\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_legacy_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2714\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2715\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2716\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2717\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mpy_any\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mis_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2673\u001b[0m             \u001b[0mfetched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_callable_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0marray_vals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_metadata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2674\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2675\u001b[0;31m             \u001b[0mfetched\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_callable_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0marray_vals\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2676\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mfetched\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2677\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1437\u001b[0m           ret = tf_session.TF_SessionRunCallable(\n\u001b[1;32m   1438\u001b[0m               \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_handle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatus\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1439\u001b[0;31m               run_metadata_ptr)\n\u001b[0m\u001b[1;32m   1440\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1441\u001b[0m           \u001b[0mproto_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mKeyboardInterrupt\u001b[0m: "]}]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(model_pull_alarm.summary())","execution_count":35,"outputs":[{"output_type":"stream","text":"_________________________________________________________________\nLayer (type)                 Output Shape              Param #   \n=================================================================\nconv1d_61 (Conv1D)           (None, 149998, 12)        48        \n_________________________________________________________________\nmax_pooling1d_61 (MaxPooling (None, 14999, 12)         0         \n_________________________________________________________________\nbatch_normalization_11 (Batc (None, 14999, 12)         48        \n_________________________________________________________________\ndropout_91 (Dropout)         (None, 14999, 12)         0         \n_________________________________________________________________\nconv1d_62 (Conv1D)           (None, 14997, 100)        3700      \n_________________________________________________________________\ndropout_92 (Dropout)         (None, 14997, 100)        0         \n_________________________________________________________________\nmax_pooling1d_62 (MaxPooling (None, 7498, 100)         0         \n_________________________________________________________________\nconv1d_63 (Conv1D)           (None, 7496, 200)         60200     \n_________________________________________________________________\ndropout_93 (Dropout)         (None, 7496, 200)         0         \n_________________________________________________________________\nmax_pooling1d_63 (MaxPooling (None, 3748, 200)         0         \n_________________________________________________________________\nconv1d_64 (Conv1D)           (None, 3746, 300)         180300    \n_________________________________________________________________\ndropout_94 (Dropout)         (None, 3746, 300)         0         \n_________________________________________________________________\nmax_pooling1d_64 (MaxPooling (None, 1873, 300)         0         \n_________________________________________________________________\nconv1d_65 (Conv1D)           (None, 1871, 200)         180200    \n_________________________________________________________________\ndropout_95 (Dropout)         (None, 1871, 200)         0         \n_________________________________________________________________\nmax_pooling1d_65 (MaxPooling (None, 935, 200)          0         \n_________________________________________________________________\nconv1d_66 (Conv1D)           (None, 933, 100)          60100     \n_________________________________________________________________\ndropout_96 (Dropout)         (None, 933, 100)          0         \n_________________________________________________________________\nmax_pooling1d_66 (MaxPooling (None, 466, 100)          0         \n_________________________________________________________________\nflatten_11 (Flatten)         (None, 46600)             0         \n_________________________________________________________________\ndropout_97 (Dropout)         (None, 46600)             0         \n_________________________________________________________________\ndense_31 (Dense)             (None, 100)               4660100   \n_________________________________________________________________\ndropout_98 (Dropout)         (None, 100)               0         \n_________________________________________________________________\ndense_32 (Dense)             (None, 100)               10100     \n_________________________________________________________________\ndropout_99 (Dropout)         (None, 100)               0         \n_________________________________________________________________\ndense_33 (Dense)             (None, 1)                 101       \n=================================================================\nTotal params: 5,154,897\nTrainable params: 5,154,873\nNon-trainable params: 24\n_________________________________________________________________\nNone\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# SCORE \"PULL_ALARM\" VALUE FOR SCALED TRAINING SET\ny_pred = model_pull_alarm.predict(X_train)\nplt.figure(figsize=(6, 6))\nplt.scatter(y_pull_alarm, y_pred)\nplt.xlim(-.1, 1.1)\nplt.ylim(-2, 2)\nplt.xlabel('actual', fontsize=12)\nplt.ylabel('predicted', fontsize=12)\nplt.plot([(0, 0), (20, 20)], [(0, 0), (20, 20)])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"score = mean_absolute_error(y_pull_alarm, y_pred)\nprint(f'Score: {score:0.3f}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# spread out the results to the desired range of 0 to 1\nymin = y_pred.min()\nymax = y_pred.max()\ny_pred = ((y_pred - ymin) / (ymax - ymin))\n\n# Categorize the time remaining as being large (if we are not pulling the alarm) or small (if we are)\ny_pred = overall_max * (1 - y_pred)\n\nplt.figure(figsize=(6, 6))\nplt.scatter(y_train, y_pred)\nplt.xlim(0, 20)\nplt.ylim(0, 20)\nplt.xlabel('actual', fontsize=12)\nplt.ylabel('predicted', fontsize=12)\nplt.plot([(0, 0), (20, 20)], [(0, 0), (20, 20)])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"score = mean_absolute_error(y_train, y_pred)\nprint(f'Score: {score:0.3f}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y_pred_pull_alarm = y_pred\n\ny_pred = model.predict(X_train)\n# Now combine it with our original time to fault estimates\ny_pred = (y_pred_pull_alarm + y_pred) / 2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(6, 6))\nplt.scatter(y_train, y_pred)\nplt.xlim(0, 20)\nplt.ylim(0, 20)\nplt.xlabel('actual', fontsize=12)\nplt.ylabel('predicted', fontsize=12)\nplt.plot([(0, 0), (20, 20)], [(0, 0), (20, 20)])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"score = mean_absolute_error(y_train, y_pred)\nprint(f'Score: {score:0.3f}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Using good machine learning practices learned from medicine.**\n\nIn the aforementioned FDA document, a number of steps and documentation and procedural aspects are discussed. These are definitely worth reading in detail - and if you are familiar with Kaggle competitions, you'll find much of this familiar, but below is a biref summary..\n\n**For the initially created SaMD, the steps include:**\n* Data selection and management\n* Model training and tuning\n* Performance and Clinical model validation\n* Premarket testing\n\n**For modfications (which should be planned for) the steps include:**\n* Data for re-training\n* All of the usual steps abve, with the addition of SaMD Pre-specifications and Algorithm Change Protocol\n\n**Documentation should include:**\n1. Establish clear expectations on quality systems and good ML practices (GMLP);\n2. Conduct premarket review for those SaMD that require premarket submission to demonstrate reasonable assurance of safety and effectiveness and establish clear expectations for manufacturers of AI/ML-based SaMD to continually manage patient risks throughout the lifecycle;\n3. Expect manufacturers to monitor the AI/ML device and incorporate a risk management approach and other approaches utlined in “Deciding When to Submit a 510(k) for a Software Change to an Existing Device” Guidance18 in development, validation, and execution of he algorithm changes (SaMD Pre-Specifications and Algorithm Change Protocol); and\n4. Enable increased transparency to users and FDA using postmarket real-world performance reporting for maintaining continued assurance of safety and effectiveness.\n\n**What does this mean for the software?**\n\nAs a Kaggle competition entrant, mostly just the documentation of the above will be sufficient - while adherence to Good Machine Learning Practices (GMLP) should be something you are doing anyway. In the case of this competition, the rules for submissions have already been set - whereas the rules for submission of medical trial documentation will have its own (lengthly) rules which must be adhered to."},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.read_csv('../input/sample_submission.csv', index_col='seg_id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del X_train\ndel y_train\ndel y_pred","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"rows = 150_000\nsegment = 0\nfor seg_id in tqdm(submission.index):\n    seg = pd.read_csv('../input/test/' + seg_id + '.csv')\n    if (segment == 0) :\n        X_test = np.int16(seg.acoustic_data.values)\n    else: \n        X_test = np.concatenate((X_test, np.int16(seg.acoustic_data.values)))\n    segment += 1\nsegments = segment","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plot the head of the seismic acoustic data\nplt.plot(X_test[0:150000])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_test = np.reshape(X_test, (segments, rows,1))\nX_test.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Sanity check - plot the head of the seismic acoustic data\nplt.plot(X_test[0,:])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"saving PULL_ALARM estimates, scaled to time\")\ny_pred = model_pull_alarm.predict(X_test)\n# spread out the results to the desired range of 0 to 1\nymin = y_pred.min()\nymax = y_pred.max()\ny_pred = ((y_pred - ymin) / (ymax - ymin))\n# Categorize the time remaining as being large (if we are not pulling the alarm) or small (if we are)\ny_pred = overall_max * (1 - y_pred)\nsubmission['time_to_failure'] = y_pred\nsubmission.to_csv('submission_pull_alarm.csv')\n\nprint(\"saving TIME_TO_FAILURE estimates\")\ny_pred_pull_alarm = y_pred\ny_pred = model.predict(X_test)\nsubmission['time_to_failure'] = y_pred\nsubmission.to_csv('submission_time_to_failure.csv')\n\nprint(\"saving average of the above estimates\")\n# Now combine it with our original time to fault estimates\ny_pred = (y_pred_pull_alarm + y_pred) / 2\nsubmission['time_to_failure'] = y_pred\nsubmission.to_csv('submission.csv')\n\nprint(\"done!\")","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}