{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Overview\n\nAccelerometer data can be pretty fun and unique to work with. Firstly, it is important to understand that this data is a time series. We are looking for Turning, StartHesitation, and Walking events in these time series. In my notebook, I demonstrate an example of each of these types of events in a signal.\n\nI also take a look at a few useful feature engineering ideas. One idea to to search for relative extrema (maximums and minimums). This could be useful as a feature or useful in detecting outliers. This information could then be used to help us determine the time of the freezing episode.\n\nAnother piece of information that could be useful is the change in signal from one time step to the next. This information could be useful in determining a sudden and quick change in movement.\n\nI believe an effective approach for this competition could be the following:\n- take a window of values around a given time step\n- feature engineer those values\n- feed the values and the features the the sub signal to a model\n- post process the results with interpolation and other techniques","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport itertools\nimport matplotlib.pyplot as plt\nfrom scipy.signal import find_peaks","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-03-13T01:10:27.434984Z","iopub.execute_input":"2023-03-13T01:10:27.436416Z","iopub.status.idle":"2023-03-13T01:10:27.442764Z","shell.execute_reply.started":"2023-03-13T01:10:27.436366Z","shell.execute_reply":"2023-03-13T01:10:27.441297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"events = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/events.csv\")\n\nturn_id = list(events[events.Type==\"Turn\"].Id)[0]\nhes_id = list(events[events.Type==\"StartHesitation\"].Id)[0]\nwalk_id = list(events[events.Type==\"Walking\"].Id)[0]\n\nturn_df = pd.read_csv(f\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog/{turn_id}.csv\")\nhes_df = pd.read_csv(f\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog/{hes_id}.csv\")\nwalk_df = pd.read_csv(f\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog/{walk_id}.csv\")\ndefog_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/defog/02ea782681.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-13T00:57:23.704480Z","iopub.execute_input":"2023-03-13T00:57:23.705040Z","iopub.status.idle":"2023-03-13T00:57:24.143414Z","shell.execute_reply.started":"2023-03-13T00:57:23.705002Z","shell.execute_reply":"2023-03-13T00:57:24.142456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plotting Full Time Series","metadata":{}},{"cell_type":"code","source":"def plot_xyz(V, ML, AP):\n    plt.subplots(3, 1, sharex='col', sharey='row', figsize=(16,10))\n\n    plt.subplot(3, 1, 1)\n    plt.plot(V, color='#69cf83', label='V')\n    plt.xlabel('Time')\n    plt.ylabel('Sensor Value')\n    plt.legend()\n\n    plt.subplot(3, 1, 2)\n    plt.plot(ML, color='#d6b258', label='ML')\n    plt.xlabel('Time')\n    plt.ylabel('Sensor Value')\n    plt.legend()\n\n    plt.subplot(3, 1, 3)\n    plt.plot(AP, color='#96bcfa', label='AP')\n    plt.xlabel('Time')\n    plt.ylabel('Sensor Value')\n    plt.legend()\n\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T00:57:24.147444Z","iopub.execute_input":"2023-03-13T00:57:24.148371Z","iopub.status.idle":"2023-03-13T00:57:24.156803Z","shell.execute_reply.started":"2023-03-13T00:57:24.148330Z","shell.execute_reply":"2023-03-13T00:57:24.155350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_xyz(defog_df.AccV, defog_df.AccML, defog_df.AccAP)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T00:57:24.159513Z","iopub.execute_input":"2023-03-13T00:57:24.160788Z","iopub.status.idle":"2023-03-13T00:57:25.104877Z","shell.execute_reply.started":"2023-03-13T00:57:24.160716Z","shell.execute_reply":"2023-03-13T00:57:25.103540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plotting Events\n\nThe shaded section of eah graph shows the time the vent happends.","metadata":{}},{"cell_type":"code","source":"def to_ranges(l_set):\n    rb_set = sorted(l_set - {i +1 for i in l_set})\n    re_set = sorted(l_set - {i -1 for i in l_set})\n    return [(rb_set[i], re_set[i]+1) for i in range(len(rb_set))]\n\ndef plot_event(V, ML, AP, start, stop, padding):\n    plt.plot(V[start-padding:stop+padding], color='#69cf83', label='V')\n    plt.plot(ML[start-padding:stop+padding], color='#d6b258', label='ML')\n    plt.plot(AP[start-padding:stop+padding], color='#96bcfa', label='AP')\n    plt.axvspan(start, stop, color='brown', alpha=0.2)\n    plt.xlabel('Time')\n    plt.ylabel('Sensor Value')\n    plt.legend()\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T00:58:49.674925Z","iopub.execute_input":"2023-03-13T00:58:49.675304Z","iopub.status.idle":"2023-03-13T00:58:49.683190Z","shell.execute_reply.started":"2023-03-13T00:58:49.675272Z","shell.execute_reply":"2023-03-13T00:58:49.682211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Turn Event Example","metadata":{}},{"cell_type":"code","source":"vals = list(turn_df[turn_df.Turn!=0].Time)\nrng = to_ranges(set(vals))[0]\nplot_event(turn_df.AccV, turn_df.AccML, turn_df.AccAP, rng[0], rng[1], 1000)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T00:58:50.854161Z","iopub.execute_input":"2023-03-13T00:58:50.854547Z","iopub.status.idle":"2023-03-13T00:58:51.055011Z","shell.execute_reply.started":"2023-03-13T00:58:50.854511Z","shell.execute_reply":"2023-03-13T00:58:51.053494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Start Hesitation Event Example","metadata":{}},{"cell_type":"code","source":"vals = list(hes_df[hes_df.StartHesitation!=0].Time)\nrng = to_ranges(set(vals))[0]\nplot_event(hes_df.AccV, hes_df.AccML, hes_df.AccAP, rng[0], rng[1], 1000)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T00:59:16.130911Z","iopub.execute_input":"2023-03-13T00:59:16.131497Z","iopub.status.idle":"2023-03-13T00:59:16.455599Z","shell.execute_reply.started":"2023-03-13T00:59:16.131462Z","shell.execute_reply":"2023-03-13T00:59:16.453633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Walking Event Example","metadata":{}},{"cell_type":"code","source":"vals = list(walk_df[walk_df.Walking!=0].Time)\nrng = to_ranges(set(vals))[0]\nplot_event(walk_df.AccV, walk_df.AccML, walk_df.AccAP, rng[0], rng[1], 100)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T00:59:00.893332Z","iopub.execute_input":"2023-03-13T00:59:00.894807Z","iopub.status.idle":"2023-03-13T00:59:01.086386Z","shell.execute_reply.started":"2023-03-13T00:59:00.894723Z","shell.execute_reply":"2023-03-13T00:59:01.084770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Finding Relative Extrema\n\nThe example event I am using is a walking event. The red dots show the relative maximums and minimums.","metadata":{}},{"cell_type":"code","source":"vals = list(walk_df[walk_df.Walking!=0].Time)\nrng = to_ranges(set(vals))[0]\nV = walk_df.AccV[rng[0]-100:rng[1]+100].reset_index(drop=True)\nML = walk_df.AccML[rng[0]-100:rng[1]+100].reset_index(drop=True)\nAP = walk_df.AccAP[rng[0]-100:rng[1]+100].reset_index(drop=True)\n\nplt.subplots(3, 1, sharex='col', sharey='row', figsize=(16,10))\n\nplt.subplot(3, 1, 1)\npeaks = find_peaks(V)\nvalleys = find_peaks(-V)\nplt.plot(V, color='#69cf83', label='V')\nplt.scatter(peaks[0], [V[p] for p in peaks[0]], color='r')\nplt.scatter(valleys[0], [V[p] for p in valleys[0]], color='r')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\n\nplt.subplot(3, 1, 2)\npeaks = find_peaks(ML)\nvalleys = find_peaks(-ML)\nplt.plot(ML, color='#d6b258', label='ML')\nplt.scatter(peaks[0], [ML[p] for p in peaks[0]], color='r')\nplt.scatter(valleys[0], [ML[p] for p in valleys[0]], color='r')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\n\nplt.subplot(3, 1, 3)\npeaks = find_peaks(AP)\nvalleys = find_peaks(-AP)\nplt.plot(AP, color='#96bcfa', label='AP')\nplt.scatter(peaks[0], [AP[p] for p in peaks[0]], color='r')\nplt.scatter(valleys[0], [AP[p] for p in valleys[0]], color='r')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\n    \nplt.plot()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:13:47.727804Z","iopub.execute_input":"2023-03-13T01:13:47.728240Z","iopub.status.idle":"2023-03-13T01:13:48.332524Z","shell.execute_reply.started":"2023-03-13T01:13:47.728201Z","shell.execute_reply":"2023-03-13T01:13:48.331124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plotting the Delta","metadata":{}},{"cell_type":"code","source":"plt.subplots(3, 1, sharex='col', sharey='row', figsize=(16,10))\n\nplt.subplot(3, 1, 1)\nplt.plot(V, color='#69cf83', label='V')\nplt.plot(V.diff(), color='#2d8e46', linestyle='dashed', label='ΔV')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\n\nplt.subplot(3, 1, 2)\nplt.plot(ML, color='#d6b258', label='ML')\nplt.plot(ML.diff(), color='#917224', linestyle='dashed', label='ΔML')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\n\nplt.subplot(3, 1, 3)\nplt.plot(AP, color='#96bcfa', label='AP')\nplt.plot(AP.diff(), color='#0b5ee5', linestyle='dashed', label='ΔAP')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:33:22.705460Z","iopub.execute_input":"2023-03-13T01:33:22.705862Z","iopub.status.idle":"2023-03-13T01:33:23.205538Z","shell.execute_reply.started":"2023-03-13T01:33:22.705826Z","shell.execute_reply":"2023-03-13T01:33:23.203644Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plotting the Trend","metadata":{}},{"cell_type":"code","source":"plt.subplots(3, 1, sharex='col', sharey='row', figsize=(16,10))\n\nplt.subplot(3, 1, 1)\nplt.plot(V, color='#69cf83', label='V')\na, b = np.polyfit(V.index, V, 1)\nplt.plot(V.index, a*V.index+b, color='red')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\n\nplt.subplot(3, 1, 2)\nplt.plot(ML, color='#d6b258', label='ML')\na, b = np.polyfit(ML.index, ML, 1)\nplt.plot(ML.index, a*ML.index+b, color='red')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\n\nplt.subplot(3, 1, 3)\nplt.plot(AP, color='#96bcfa', label='AP')\na, b = np.polyfit(AP.index, AP, 1)\nplt.plot(AP.index, a*AP.index+b, color='red')\nplt.xlabel('Time')\nplt.ylabel('Sensor Value')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:33:48.662115Z","iopub.execute_input":"2023-03-13T01:33:48.662523Z","iopub.status.idle":"2023-03-13T01:33:49.143906Z","shell.execute_reply.started":"2023-03-13T01:33:48.662486Z","shell.execute_reply":"2023-03-13T01:33:49.142330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}