{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#
🧠 PFOGP 🧠 - Data and problem investigation
\n

Event detection from wearable sensor data

\n\n***","metadata":{}},{"cell_type":"markdown","source":"#
(ಠಿ⁠_⁠ಠ) Overview
\n\n

⚪ In this competition, we aim to detect freezing of gait (FOG) using data collected from a wearable 3D lower back sensor.

\n\n\n

⚪ FOG is a debilitating symptom that affects many people with Parkinson's disease, leading to restricted independence and increased risk of falls.

\n\n

⚪ By developing a machine learning model trained on this dataset, we can improve the ability of medical professionals to evaluate, monitor, and prevent FOG events.

\n\n

⚪ There are multiple methods of evaluating FOG, but most involve FOG-provoking protocols, which are time-consuming and require specific expertise.

\n\n

⚪ Wearable devices can help detect FOG more easily, but compliance and usability may be reduced.

\n\n

⚪ The datasets used to train and test machine learning algorithms for detecting FOG have been relatively small, and generalizability is limited to date.

\n\n

⚪ The competition host, the Center for the Study of Movement, Cognition, and Mobility (CMCM), Neurological Institute, Tel Aviv Sourasky Medical Center, aims to improve the personalized treatment of age-related movement, cognition, and mobility disorders and to alleviate the associated burden.

\n\n

🔴 Submissions are evaluated by the Mean Average Precision of predictions for each event class. Average precision is computed on predicted confidence scores separately for each of the three event classes.

","metadata":{}},{"cell_type":"markdown","source":"#### \n#
Table of contents
\n\n
\n\n* [0. Import all dependencies](#0)\n* [1. Overview availiable directories](#1)\n * [1.1 Overview train/ directory](#1.1)\n * [1.2 Overview test/ directory](#1.2)\n * [1.3 Overview unlabeled/ directory](#1.3)\n* [2. Overview availiable csv files](#2)\n * [2.1 Overview tdcsfog_metadata.csv file](#2.1)\n * [2.2 Overview defog_metadata.csv file](#2.2)\n * [2.3 Overview daily_metadata.csv file](#2.3)\n * [2.4 Overview subjects.csv file](#2.4)\n * [2.5 Overview events.csv file](#2.5)\n * [2.6 Overview tasks.csv file](#2.6)","metadata":{}},{"cell_type":"markdown","source":"\n#
0. Import all dependencies
","metadata":{}},{"cell_type":"code","source":"import os\nimport random\nimport cv2\nimport pandas as pd\nimport numpy as np\nimport plotly.express as px\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:34.880831Z","iopub.execute_input":"2023-03-12T12:22:34.881237Z","iopub.status.idle":"2023-03-12T12:22:38.432887Z","shell.execute_reply.started":"2023-03-12T12:22:34.881203Z","shell.execute_reply":"2023-03-12T12:22:38.431545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class color:\n PURPLE = '\\033[95m'\n CYAN = '\\033[96m'\n DARKCYAN = '\\033[36m'\n BLUE = '\\033[94m'\n GREEN = '\\033[92m'\n YELLOW = '\\033[93m'\n RED = '\\033[91m'\n BOLD = '\\033[1m'\n UNDERLINE = '\\033[4m'\n END = '\\033[0m'","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:38.435579Z","iopub.execute_input":"2023-03-12T12:22:38.436388Z","iopub.status.idle":"2023-03-12T12:22:38.443519Z","shell.execute_reply.started":"2023-03-12T12:22:38.436339Z","shell.execute_reply":"2023-03-12T12:22:38.441936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#
1. Overview availiable directories
","metadata":{}},{"cell_type":"markdown","source":"

⚪ The data series include three datasets, collected under distinct circumstances:

\n\n*

The tDCS FOG (tdcsfog) dataset, comprising data series collected in the lab, as subjects completed a FOG-provoking protocol.

\n*

The DeFOG (defog) dataset, comprising data series collected in the subject's home, as subjects completed a FOG-provoking protocol.

\n*

The Daily Living (daily) dataset, comprising one week of continuous 24/7 recordings from sixty-five subjects. Forty-five subjects exhibit FOG symptoms and also have series in the defog dataset, while the other twenty subjects do not exhibit FOG symptoms and do not have series elsewhere in the data.

\n\n\n

⚪ Trials from the tdcsfog and defog datasets were videotaped and annotated by expert reviewers documented the freezing of gait episodes. That is, the start, end and type of each episode were marked by the experts. Series in the daily dataset are unannotated.

\n

⚪ You will be detecting FOG episodes for the tdcsfog and defog series. You may wish to apply unsupervised or semi-supervised methods to the series in the daily dataset to support your detection modelling.

\n

🔴 See this page for more on these datasets as well as video examples of freezing of gait events: Additional Data Documentation.

","metadata":{}},{"cell_type":"markdown","source":"\n##
1.1 Overview train/ directory
","metadata":{}},{"cell_type":"markdown","source":"

⚪ train/ Folder containing the data series in the training set within three subfolders: tdcsfog/, defog/, and notype/. Series in the notype folder are from the defog dataset but lack event-type annotations. The fields present in these series vary by folder.

\n\n*

Time An integer timestep. Series from the tdcsfog dataset are recorded at 128Hz (128 timesteps per second), while series from the defog and daily series are recorded at 100Hz (100 timesteps per second).

\n*

AccV, AccML, and AccAP Acceleration in units of g, from a lower-back sensor on three axes: V - vertical, ML - mediolateral, AP - anteroposterior.

\n*

StartHesitation, Turn, Walking Indicator variables for the occurrence of each of the event types.

\n*

Event Indicator variable for the occurrence of any FOG-type event. Present only in the notype series, which lack type-level annotations.

\n*

Valid There were cases during the video annotation that were hard for the annotator to decide if there was an Akinetic (i.e., essentially no movement) FoG or the subject stopped voluntarily. Only event annotations where the series is marked true should be considered as unambiguous.

\n*

Task Series were only annotated where this value is true. Portions marked false should be considered unannotated.

","metadata":{}},{"cell_type":"markdown","source":"

❔ let's check what directories are in the train folder.

","metadata":{}},{"cell_type":"code","source":"os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:38.445209Z","iopub.execute_input":"2023-03-12T12:22:38.445619Z","iopub.status.idle":"2023-03-12T12:22:38.475639Z","shell.execute_reply.started":"2023-03-12T12:22:38.445584Z","shell.execute_reply":"2023-03-12T12:22:38.474333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How many files are in folder tdcsfog/.

","metadata":{}},{"cell_type":"code","source":"temp = len(os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog\"))\nprint(\n f\"Number of files in folder tdcsfog/: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:38.478369Z","iopub.execute_input":"2023-03-12T12:22:38.479402Z","iopub.status.idle":"2023-03-12T12:22:38.542458Z","shell.execute_reply.started":"2023-03-12T12:22:38.479349Z","shell.execute_reply":"2023-03-12T12:22:38.541248Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"train_tdcsfog_example_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog/003f117e14.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:34.878578Z","iopub.execute_input":"2023-03-12T13:01:34.879122Z","iopub.status.idle":"2023-03-12T13:01:34.896515Z","shell.execute_reply.started":"2023-03-12T13:01:34.879071Z","shell.execute_reply":"2023-03-12T13:01:34.894833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = len(train_tdcsfog_example_df)\nprint(\n f\"Length of dataframe: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:35.166102Z","iopub.execute_input":"2023-03-12T13:01:35.167197Z","iopub.status.idle":"2023-03-12T13:01:35.176025Z","shell.execute_reply.started":"2023-03-12T13:01:35.167136Z","shell.execute_reply":"2023-03-12T13:01:35.174247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_tdcsfog_example_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:37.470695Z","iopub.execute_input":"2023-03-12T13:01:37.471181Z","iopub.status.idle":"2023-03-12T13:01:37.488066Z","shell.execute_reply.started":"2023-03-12T13:01:37.471131Z","shell.execute_reply":"2023-03-12T13:01:37.486652Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_tdcsfog_example_df.describe()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:37.832995Z","iopub.execute_input":"2023-03-12T13:01:37.834636Z","iopub.status.idle":"2023-03-12T13:01:37.877049Z","shell.execute_reply.started":"2023-03-12T13:01:37.834570Z","shell.execute_reply":"2023-03-12T13:01:37.875663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ Is there any NaN values in the dataframe.

","metadata":{}},{"cell_type":"code","source":"train_tdcsfog_example_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:39.207248Z","iopub.execute_input":"2023-03-12T13:01:39.208617Z","iopub.status.idle":"2023-03-12T13:01:39.220921Z","shell.execute_reply.started":"2023-03-12T13:01:39.208549Z","shell.execute_reply":"2023-03-12T13:01:39.219711Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How many files are in folder defog/.

","metadata":{}},{"cell_type":"code","source":"temp = len(os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/defog\"))\nprint(\n f\"Number of files in folder defog/: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:40.539826Z","iopub.execute_input":"2023-03-12T13:01:40.540302Z","iopub.status.idle":"2023-03-12T13:01:40.548243Z","shell.execute_reply.started":"2023-03-12T13:01:40.540264Z","shell.execute_reply":"2023-03-12T13:01:40.547114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How many files are in folder notype/.

","metadata":{}},{"cell_type":"code","source":"temp = len(os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/notype\"))\nprint(\n f\"Number of files in folder notype/: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:41.300816Z","iopub.execute_input":"2023-03-12T13:01:41.301291Z","iopub.status.idle":"2023-03-12T13:01:41.309585Z","shell.execute_reply.started":"2023-03-12T13:01:41.301253Z","shell.execute_reply":"2023-03-12T13:01:41.307929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How is the time sirieses for features 'AccV','AccML','AccAP' looks like.

","metadata":{}},{"cell_type":"code","source":"for column in ['AccV','AccML','AccAP']:\n fig = px.line(train_tdcsfog_example_df, x=\"Time\", y=column, color_discrete_sequence=['darkslateblue'])\n fig.update_layout(\n title={\n 'text': f\"{column} Time Siries\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n )\n fig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:11:10.107228Z","iopub.execute_input":"2023-03-12T13:11:10.107768Z","iopub.status.idle":"2023-03-12T13:11:10.312859Z","shell.execute_reply.started":"2023-03-12T13:11:10.107724Z","shell.execute_reply":"2023-03-12T13:11:10.310992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How is the correlation matrix for the data looks like.

","metadata":{}},{"cell_type":"code","source":"corr_mat = np.round(train_tdcsfog_example_df.drop(\n [\"StartHesitation\", \"Walking\"], axis=1\n).corr(), 3)\n\nfig = px.imshow(\n corr_mat,\n x=corr_mat.columns,\n y=corr_mat.columns, \n text_auto=True\n )\nfig.update_xaxes(side=\"bottom\")\nfig.update_layout(\n title={\n 'text': \"Correlation Matrix of tdcsfog train instance\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:01:43.400834Z","iopub.execute_input":"2023-03-12T13:01:43.402309Z","iopub.status.idle":"2023-03-12T13:01:43.469459Z","shell.execute_reply.started":"2023-03-12T13:01:43.402247Z","shell.execute_reply":"2023-03-12T13:01:43.467881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n##
1.2 Overview test/ directory
","metadata":{}},{"cell_type":"markdown","source":"

⚪ test/ Only the Time, AccV, AccML, and AccAP fields are provided for the test series.

\n","metadata":{}},{"cell_type":"code","source":"os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.731185Z","iopub.execute_input":"2023-03-12T12:22:40.732032Z","iopub.status.idle":"2023-03-12T12:22:40.745000Z","shell.execute_reply.started":"2023-03-12T12:22:40.731981Z","shell.execute_reply":"2023-03-12T12:22:40.742522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How many files are in folder tdcsfog/.

","metadata":{}},{"cell_type":"code","source":"temp = len(os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog\"))\nprint(\n f\"Number of files in folder tdcsfog/: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.747000Z","iopub.execute_input":"2023-03-12T12:22:40.747567Z","iopub.status.idle":"2023-03-12T12:22:40.763700Z","shell.execute_reply.started":"2023-03-12T12:22:40.747508Z","shell.execute_reply":"2023-03-12T12:22:40.762156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How many files are in folder defog/.

","metadata":{}},{"cell_type":"code","source":"temp = len(os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/defog\"))\nprint(\n f\"Number of files in folder defog/: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.765317Z","iopub.execute_input":"2023-03-12T12:22:40.766112Z","iopub.status.idle":"2023-03-12T12:22:40.780358Z","shell.execute_reply.started":"2023-03-12T12:22:40.766068Z","shell.execute_reply":"2023-03-12T12:22:40.777963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the test data looks like.

","metadata":{}},{"cell_type":"code","source":"test_tdcsfog_example_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog/003f117e14.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.782118Z","iopub.execute_input":"2023-03-12T12:22:40.782513Z","iopub.status.idle":"2023-03-12T12:22:40.805029Z","shell.execute_reply.started":"2023-03-12T12:22:40.782472Z","shell.execute_reply":"2023-03-12T12:22:40.804009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = len(test_tdcsfog_example_df)\nprint(\n f\"Length of dataframe: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.806408Z","iopub.execute_input":"2023-03-12T12:22:40.807454Z","iopub.status.idle":"2023-03-12T12:22:40.814138Z","shell.execute_reply.started":"2023-03-12T12:22:40.807387Z","shell.execute_reply":"2023-03-12T12:22:40.812847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_tdcsfog_example_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.816047Z","iopub.execute_input":"2023-03-12T12:22:40.816888Z","iopub.status.idle":"2023-03-12T12:22:40.836788Z","shell.execute_reply.started":"2023-03-12T12:22:40.816848Z","shell.execute_reply":"2023-03-12T12:22:40.835017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_tdcsfog_example_df.describe()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.838691Z","iopub.execute_input":"2023-03-12T12:22:40.840392Z","iopub.status.idle":"2023-03-12T12:22:40.874230Z","shell.execute_reply.started":"2023-03-12T12:22:40.840342Z","shell.execute_reply":"2023-03-12T12:22:40.872518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How is the correlation matrix for the data looks like.

","metadata":{}},{"cell_type":"code","source":"corr_mat = np.round(test_tdcsfog_example_df.corr(), 3)\n\nfig = px.imshow(\n corr_mat,\n x=corr_mat.columns,\n y=corr_mat.columns, \n text_auto=True\n )\nfig.update_xaxes(side=\"bottom\")\nfig.update_layout(\n title={\n 'text': \"Correlation Matrix of tdcsfog test instance\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.876599Z","iopub.execute_input":"2023-03-12T12:22:40.877152Z","iopub.status.idle":"2023-03-12T12:22:40.943741Z","shell.execute_reply.started":"2023-03-12T12:22:40.877098Z","shell.execute_reply":"2023-03-12T12:22:40.942332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n##
1.3 Overview unlabeled/ directory
","metadata":{}},{"cell_type":"markdown","source":"

⚪ unlabeled/ Folder containing the unannotated data series from the daily dataset, one series per subject. Forty-five of the subjects also have series in the defog dataset, some in the training split and some in the test split.

\n","metadata":{}},{"cell_type":"markdown","source":"

❔ How many files are in folder unlabeled/.

","metadata":{}},{"cell_type":"code","source":"temp = len(os.listdir(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/unlabeled\"))\nprint(\n f\"Number of files in folder unlabeled/: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.945407Z","iopub.execute_input":"2023-03-12T12:22:40.947049Z","iopub.status.idle":"2023-03-12T12:22:40.971770Z","shell.execute_reply.started":"2023-03-12T12:22:40.947004Z","shell.execute_reply":"2023-03-12T12:22:40.970718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"unlabeled_example_df = pd.read_parquet(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/unlabeled/00c4c9313d.parquet\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:40.973142Z","iopub.execute_input":"2023-03-12T12:22:40.974038Z","iopub.status.idle":"2023-03-12T12:22:50.387926Z","shell.execute_reply.started":"2023-03-12T12:22:40.974003Z","shell.execute_reply":"2023-03-12T12:22:50.386848Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unlabeled_example_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:50.389544Z","iopub.execute_input":"2023-03-12T12:22:50.390070Z","iopub.status.idle":"2023-03-12T12:22:50.402720Z","shell.execute_reply.started":"2023-03-12T12:22:50.390018Z","shell.execute_reply":"2023-03-12T12:22:50.401518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unlabeled_example_df.describe()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:22:50.404306Z","iopub.execute_input":"2023-03-12T12:22:50.405735Z","iopub.status.idle":"2023-03-12T12:23:01.028171Z","shell.execute_reply.started":"2023-03-12T12:22:50.405689Z","shell.execute_reply":"2023-03-12T12:23:01.026846Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How is the correlation matrix for the data looks like.

","metadata":{}},{"cell_type":"code","source":"corr_mat = np.round(unlabeled_example_df.corr(), 3)\n\nfig = px.imshow(\n corr_mat,\n x=corr_mat.columns,\n y=corr_mat.columns, \n text_auto=True\n )\nfig.update_xaxes(side=\"bottom\")\nfig.update_layout(\n title={\n 'text': \"Correlation Matrix of unlabeled instance\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:01.029795Z","iopub.execute_input":"2023-03-12T12:23:01.030399Z","iopub.status.idle":"2023-03-12T12:23:06.234124Z","shell.execute_reply.started":"2023-03-12T12:23:01.030363Z","shell.execute_reply":"2023-03-12T12:23:06.232725Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#
2. Overview availiable csv files
","metadata":{}},{"cell_type":"markdown","source":"\n##
2.1 Overview tdcsfog_metadata.csv file
","metadata":{}},{"cell_type":"markdown","source":"

⚪ tdcsfog_metadata.csv Identifies each series in the tdcsfog dataset by a unique Subject, Visit, Test, Medication condition.

\n\n*

Visit Lab visits consist of a baseline assessment, two post-treatment assessments for different treatment stages, and one follow-up assessment.

\n*

Test Which of three test types was performed, with 3 the most challenging.

\n*

Medication Subjects may have been either off or on anti-parkinsonian medication during the recording.

","metadata":{}},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"tdcsfog_metadata_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/tdcsfog_metadata.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.236359Z","iopub.execute_input":"2023-03-12T12:23:06.236926Z","iopub.status.idle":"2023-03-12T12:23:06.250948Z","shell.execute_reply.started":"2023-03-12T12:23:06.236875Z","shell.execute_reply":"2023-03-12T12:23:06.249626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog_metadata_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.262820Z","iopub.execute_input":"2023-03-12T12:23:06.264300Z","iopub.status.idle":"2023-03-12T12:23:06.279075Z","shell.execute_reply.started":"2023-03-12T12:23:06.264242Z","shell.execute_reply":"2023-03-12T12:23:06.277841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdcsfog_metadata_df.describe()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.281005Z","iopub.execute_input":"2023-03-12T12:23:06.282115Z","iopub.status.idle":"2023-03-12T12:23:06.306217Z","shell.execute_reply.started":"2023-03-12T12:23:06.282060Z","shell.execute_reply":"2023-03-12T12:23:06.304694Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is length of the dataframe.

","metadata":{}},{"cell_type":"code","source":"temp = len(tdcsfog_metadata_df)\nprint(\n f\"Length of the tdcsfog_metadata.csv file is: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.308783Z","iopub.execute_input":"2023-03-12T12:23:06.309345Z","iopub.status.idle":"2023-03-12T12:23:06.317741Z","shell.execute_reply.started":"2023-03-12T12:23:06.309293Z","shell.execute_reply":"2023-03-12T12:23:06.315938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How many unique subjects the dataframe has.

","metadata":{}},{"cell_type":"code","source":"temp = len(tdcsfog_metadata_df.Subject.unique())\nprint(\n f\"Number of unique subjects: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.319551Z","iopub.execute_input":"2023-03-12T12:23:06.319944Z","iopub.status.idle":"2023-03-12T12:23:06.335604Z","shell.execute_reply.started":"2023-03-12T12:23:06.319909Z","shell.execute_reply":"2023-03-12T12:23:06.334312Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the data for an unique subject looks like.

","metadata":{}},{"cell_type":"code","source":"unique_subject_id = \"13abfd\"\ntdcsfog_metadata_df[tdcsfog_metadata_df.Subject == unique_subject_id]","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.338152Z","iopub.execute_input":"2023-03-12T12:23:06.338642Z","iopub.status.idle":"2023-03-12T12:23:06.359168Z","shell.execute_reply.started":"2023-03-12T12:23:06.338602Z","shell.execute_reply":"2023-03-12T12:23:06.357815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ Is there any missing data in the dataframe.

","metadata":{}},{"cell_type":"code","source":"tdcsfog_metadata_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.360981Z","iopub.execute_input":"2023-03-12T12:23:06.361449Z","iopub.status.idle":"2023-03-12T12:23:06.371829Z","shell.execute_reply.started":"2023-03-12T12:23:06.361391Z","shell.execute_reply":"2023-03-12T12:23:06.370658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Visit.

","metadata":{}},{"cell_type":"code","source":"tdcsfog_visit_counts = tdcsfog_metadata_df.Visit.value_counts()\n\nfig = px.bar(x=tdcsfog_visit_counts.index, y=tdcsfog_visit_counts.values, color_discrete_sequence=['darkgreen'])\nfig.update_layout(xaxis_title=\"Visit\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:11:51.839998Z","iopub.execute_input":"2023-03-12T13:11:51.841760Z","iopub.status.idle":"2023-03-12T13:11:51.915633Z","shell.execute_reply.started":"2023-03-12T13:11:51.841678Z","shell.execute_reply":"2023-03-12T13:11:51.914044Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Test.

","metadata":{}},{"cell_type":"code","source":"tdcsfog_test_counts = tdcsfog_metadata_df.Test.value_counts()\n\nfig = px.bar(x=tdcsfog_test_counts.index, y=tdcsfog_test_counts.values, color_discrete_sequence=['darkgreen'])\nfig.update_layout(xaxis_title=\"Test\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:11:58.367804Z","iopub.execute_input":"2023-03-12T13:11:58.368269Z","iopub.status.idle":"2023-03-12T13:11:58.436694Z","shell.execute_reply.started":"2023-03-12T13:11:58.368231Z","shell.execute_reply":"2023-03-12T13:11:58.435133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Medication.

","metadata":{}},{"cell_type":"code","source":"tdcsfog_medication_counts = tdcsfog_metadata_df.Medication.value_counts()\n\nfig = px.bar(x=tdcsfog_medication_counts.index, y=tdcsfog_medication_counts.values, color_discrete_sequence=['darkgreen'])\nfig.update_layout(xaxis_title=\"Medication\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:12:02.085291Z","iopub.execute_input":"2023-03-12T13:12:02.086599Z","iopub.status.idle":"2023-03-12T13:12:02.158996Z","shell.execute_reply.started":"2023-03-12T13:12:02.086542Z","shell.execute_reply":"2023-03-12T13:12:02.157304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n##
2.2 Overview defog_metadata.csv file
","metadata":{}},{"cell_type":"markdown","source":"

⚪ defog_metadata.csv Identifies each series in the defog dataset by a unique Subject, Visit, Medication condition.

","metadata":{}},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"defog_metadata_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/defog_metadata.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.872585Z","iopub.execute_input":"2023-03-12T12:23:06.873288Z","iopub.status.idle":"2023-03-12T12:23:06.884977Z","shell.execute_reply.started":"2023-03-12T12:23:06.873250Z","shell.execute_reply":"2023-03-12T12:23:06.883400Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"defog_metadata_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.886849Z","iopub.execute_input":"2023-03-12T12:23:06.888228Z","iopub.status.idle":"2023-03-12T12:23:06.901258Z","shell.execute_reply.started":"2023-03-12T12:23:06.888184Z","shell.execute_reply":"2023-03-12T12:23:06.899897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ Is there any missing data.

","metadata":{}},{"cell_type":"code","source":"defog_metadata_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.903739Z","iopub.execute_input":"2023-03-12T12:23:06.904306Z","iopub.status.idle":"2023-03-12T12:23:06.914880Z","shell.execute_reply.started":"2023-03-12T12:23:06.904255Z","shell.execute_reply":"2023-03-12T12:23:06.913361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is length of the dataframe.

","metadata":{}},{"cell_type":"code","source":"temp = len(defog_metadata_df)\nprint(\n f\"Length of the defog_metadata.csv file is: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.917118Z","iopub.execute_input":"2023-03-12T12:23:06.917632Z","iopub.status.idle":"2023-03-12T12:23:06.927060Z","shell.execute_reply.started":"2023-03-12T12:23:06.917582Z","shell.execute_reply":"2023-03-12T12:23:06.925217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = len(defog_metadata_df.Subject.unique())\nprint(\n f\"Number of unique subjects: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.929021Z","iopub.execute_input":"2023-03-12T12:23:06.929394Z","iopub.status.idle":"2023-03-12T12:23:06.950776Z","shell.execute_reply.started":"2023-03-12T12:23:06.929357Z","shell.execute_reply":"2023-03-12T12:23:06.949511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the data for an unique subject looks like.

","metadata":{}},{"cell_type":"code","source":"unique_subject_id = \"bf608b\"\ndefog_metadata_df[defog_metadata_df.Subject == unique_subject_id]","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:06.952130Z","iopub.execute_input":"2023-03-12T12:23:06.952487Z","iopub.status.idle":"2023-03-12T12:23:06.970155Z","shell.execute_reply.started":"2023-03-12T12:23:06.952415Z","shell.execute_reply":"2023-03-12T12:23:06.968459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Visit.

","metadata":{}},{"cell_type":"code","source":"defog_visit_counts = defog_metadata_df.Visit.value_counts()\n\nfig = px.bar(x=defog_visit_counts.index, y=defog_visit_counts.values, color_discrete_sequence=['darkslateblue'])\nfig.update_layout(xaxis_title=\"Visit\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:12:23.788453Z","iopub.execute_input":"2023-03-12T13:12:23.788936Z","iopub.status.idle":"2023-03-12T13:12:23.860478Z","shell.execute_reply.started":"2023-03-12T13:12:23.788897Z","shell.execute_reply":"2023-03-12T13:12:23.858842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Medication.

","metadata":{}},{"cell_type":"code","source":"defog_medication_counts = defog_metadata_df.Medication.value_counts()\n\nfig = px.bar(x=defog_medication_counts.index, y=defog_medication_counts.values, color_discrete_sequence=['darkslateblue'])\nfig.update_layout(xaxis_title=\"Medication\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:12:28.952762Z","iopub.execute_input":"2023-03-12T13:12:28.953195Z","iopub.status.idle":"2023-03-12T13:12:29.026823Z","shell.execute_reply.started":"2023-03-12T13:12:28.953160Z","shell.execute_reply":"2023-03-12T13:12:29.025511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n##
2.3 Overview daily_metadata.csv file
","metadata":{}},{"cell_type":"markdown","source":"

⚪ daily_metadata.csv Each series in the daily dataset is identified by the Subject id. This file also contains the time of day the recording began.

","metadata":{}},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"daily_metadata_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/daily_metadata.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.111973Z","iopub.execute_input":"2023-03-12T12:23:07.112826Z","iopub.status.idle":"2023-03-12T12:23:07.124896Z","shell.execute_reply.started":"2023-03-12T12:23:07.112781Z","shell.execute_reply":"2023-03-12T12:23:07.123339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"daily_metadata_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.126500Z","iopub.execute_input":"2023-03-12T12:23:07.126896Z","iopub.status.idle":"2023-03-12T12:23:07.141120Z","shell.execute_reply.started":"2023-03-12T12:23:07.126815Z","shell.execute_reply":"2023-03-12T12:23:07.139747Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ Is there any missing data.

","metadata":{}},{"cell_type":"code","source":"daily_metadata_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.142301Z","iopub.execute_input":"2023-03-12T12:23:07.142655Z","iopub.status.idle":"2023-03-12T12:23:07.152992Z","shell.execute_reply.started":"2023-03-12T12:23:07.142623Z","shell.execute_reply":"2023-03-12T12:23:07.151478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is length of the dataframe.

","metadata":{}},{"cell_type":"code","source":"temp = len(daily_metadata_df)\nprint(\n f\"Length of the daily_metadata.csv file is: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.155311Z","iopub.execute_input":"2023-03-12T12:23:07.155796Z","iopub.status.idle":"2023-03-12T12:23:07.163864Z","shell.execute_reply.started":"2023-03-12T12:23:07.155748Z","shell.execute_reply":"2023-03-12T12:23:07.162247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = len(daily_metadata_df.Subject.unique())\nprint(\n f\"Number of unique subjects: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.166147Z","iopub.execute_input":"2023-03-12T12:23:07.166635Z","iopub.status.idle":"2023-03-12T12:23:07.177601Z","shell.execute_reply.started":"2023-03-12T12:23:07.166592Z","shell.execute_reply":"2023-03-12T12:23:07.175976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Visit.

","metadata":{}},{"cell_type":"code","source":"daily_visit_counts = daily_metadata_df.Visit.value_counts()\n\nfig = px.bar(x=daily_visit_counts.index, y=daily_visit_counts.values, color_discrete_sequence=['cornflowerblue'])\nfig.update_layout(xaxis_title=\"Visit\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:12:48.710211Z","iopub.execute_input":"2023-03-12T13:12:48.710714Z","iopub.status.idle":"2023-03-12T13:12:48.777653Z","shell.execute_reply.started":"2023-03-12T13:12:48.710664Z","shell.execute_reply":"2023-03-12T13:12:48.776450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for the field Beginning of recording.

","metadata":{}},{"cell_type":"code","source":"daily_bor_counts = daily_metadata_df[\"Beginning of recording [00:00-23:59]\"].value_counts()\n\nfig = px.bar(x=daily_bor_counts.index, y=daily_bor_counts.values, color_discrete_sequence=['cornflowerblue'])\nfig.update_layout(xaxis_title=\"Beginning of recording\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:12:56.647860Z","iopub.execute_input":"2023-03-12T13:12:56.649012Z","iopub.status.idle":"2023-03-12T13:12:56.717876Z","shell.execute_reply.started":"2023-03-12T13:12:56.648941Z","shell.execute_reply":"2023-03-12T13:12:56.716465Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n##
2.4 Overview subjects.csv file
","metadata":{}},{"cell_type":"markdown","source":"

⚪ subjects.csv Metadata for each Subject in the study, including their Age and Sex as well as:

\n\n*

Visit Only available for subjects in the daily and defog datasets.

\n*

YearsSinceDx Years since Parkinson's diagnosis.

\n*

UPDRSIIIOn/UPDRSIIIOff Unified Parkinson's Disease Rating Scale score during on/off medication respectively.

\n*

NFOGQ Self-report FoG questionnaire score.

","metadata":{}},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"subjects_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/subjects.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.318277Z","iopub.execute_input":"2023-03-12T12:23:07.319329Z","iopub.status.idle":"2023-03-12T12:23:07.331168Z","shell.execute_reply.started":"2023-03-12T12:23:07.319291Z","shell.execute_reply":"2023-03-12T12:23:07.329678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"subjects_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.333108Z","iopub.execute_input":"2023-03-12T12:23:07.333674Z","iopub.status.idle":"2023-03-12T12:23:07.354435Z","shell.execute_reply.started":"2023-03-12T12:23:07.333620Z","shell.execute_reply":"2023-03-12T12:23:07.352282Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ Is there any missing data.

","metadata":{}},{"cell_type":"code","source":"subjects_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.356198Z","iopub.execute_input":"2023-03-12T12:23:07.356620Z","iopub.status.idle":"2023-03-12T12:23:07.370575Z","shell.execute_reply.started":"2023-03-12T12:23:07.356581Z","shell.execute_reply":"2023-03-12T12:23:07.368912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is length of the dataframe.

","metadata":{}},{"cell_type":"code","source":"temp = len(subjects_df)\nprint(\n f\"Length of the subjects.csv file is: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.372175Z","iopub.execute_input":"2023-03-12T12:23:07.372630Z","iopub.status.idle":"2023-03-12T12:23:07.381270Z","shell.execute_reply.started":"2023-03-12T12:23:07.372568Z","shell.execute_reply":"2023-03-12T12:23:07.379997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = len(subjects_df.Subject.unique())\nprint(\n f\"Number of unique subjects: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.384459Z","iopub.execute_input":"2023-03-12T12:23:07.385321Z","iopub.status.idle":"2023-03-12T12:23:07.393898Z","shell.execute_reply.started":"2023-03-12T12:23:07.385276Z","shell.execute_reply":"2023-03-12T12:23:07.392008Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Visit.

","metadata":{}},{"cell_type":"code","source":"subjects_visit_counts = subjects_df.Visit.value_counts()\n\nfig = px.bar(x=subjects_visit_counts.index, y=subjects_visit_counts.values, color_discrete_sequence=['goldenrod'])\nfig.update_layout(xaxis_title=\"Visit\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:13:17.609643Z","iopub.execute_input":"2023-03-12T13:13:17.610042Z","iopub.status.idle":"2023-03-12T13:13:17.679297Z","shell.execute_reply.started":"2023-03-12T13:13:17.610010Z","shell.execute_reply":"2023-03-12T13:13:17.677823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is the distibution of the field Age.

","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(subjects_df, x=\"Age\", nbins=30, color_discrete_sequence=['goldenrod'])\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:13:24.083035Z","iopub.execute_input":"2023-03-12T13:13:24.083633Z","iopub.status.idle":"2023-03-12T13:13:24.156767Z","shell.execute_reply.started":"2023-03-12T13:13:24.083584Z","shell.execute_reply":"2023-03-12T13:13:24.155298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Sex.

","metadata":{}},{"cell_type":"code","source":"subjects_sex_counts = subjects_df.Sex.value_counts()\n\nfig = px.bar(x=subjects_sex_counts.index, y=subjects_sex_counts.values, color_discrete_sequence=['goldenrod'])\nfig.update_layout(xaxis_title=\"Sex\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:13:26.912750Z","iopub.execute_input":"2023-03-12T13:13:26.913342Z","iopub.status.idle":"2023-03-12T13:13:27.003321Z","shell.execute_reply.started":"2023-03-12T13:13:26.913291Z","shell.execute_reply":"2023-03-12T13:13:27.001602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is the distibution of the field YearsSinceDx.

","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(subjects_df, x=\"YearsSinceDx\", nbins=30, color_discrete_sequence=['goldenrod'])\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:13:29.343135Z","iopub.execute_input":"2023-03-12T13:13:29.343547Z","iopub.status.idle":"2023-03-12T13:13:29.408201Z","shell.execute_reply.started":"2023-03-12T13:13:29.343512Z","shell.execute_reply":"2023-03-12T13:13:29.407249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is the distibution of the field UPDRSIII_On.

","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(subjects_df, x=\"UPDRSIII_On\", nbins=30, color_discrete_sequence=['goldenrod'])\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:13:31.969230Z","iopub.execute_input":"2023-03-12T13:13:31.969955Z","iopub.status.idle":"2023-03-12T13:13:32.036698Z","shell.execute_reply.started":"2023-03-12T13:13:31.969912Z","shell.execute_reply":"2023-03-12T13:13:32.035392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is the distibution of the field NFOGQ.

","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(subjects_df, x=\"NFOGQ\", nbins=20, color_discrete_sequence=['goldenrod'])\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:13:34.808255Z","iopub.execute_input":"2023-03-12T13:13:34.808764Z","iopub.status.idle":"2023-03-12T13:13:34.874136Z","shell.execute_reply.started":"2023-03-12T13:13:34.808725Z","shell.execute_reply":"2023-03-12T13:13:34.872705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How is the correlation matrix for the data looks like.

","metadata":{}},{"cell_type":"code","source":"corr_mat = np.round(subjects_df.corr(), 3)\n\nfig = px.imshow(\n corr_mat,\n x=corr_mat.columns,\n y=corr_mat.columns, \n text_auto=True\n )\nfig.update_xaxes(side=\"bottom\")\nfig.update_layout(\n title={\n 'text': \"Correlation Matrix of sybjects features\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.850293Z","iopub.execute_input":"2023-03-12T12:23:07.850706Z","iopub.status.idle":"2023-03-12T12:23:07.912168Z","shell.execute_reply.started":"2023-03-12T12:23:07.850668Z","shell.execute_reply":"2023-03-12T12:23:07.910414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n##
2.5 Overview events.csv file
","metadata":{}},{"cell_type":"markdown","source":"

⚪ events.csv Metadata for each FoG event in all data series. The event times agree with the labels in the data series.

\n\n*

Visit The data series the event occured in.

\n*

Init Time (s) the event began.

\n*

Completion Time (s) the event ended.

\n*

Type Whether StartHesitation, Turn, or Walking.

\n*

Kinetic Whether the event was kinetic (1) and involved movement, or akinetic (0) and static.

","metadata":{}},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"events_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/events.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.913955Z","iopub.execute_input":"2023-03-12T12:23:07.914459Z","iopub.status.idle":"2023-03-12T12:23:07.931763Z","shell.execute_reply.started":"2023-03-12T12:23:07.914403Z","shell.execute_reply":"2023-03-12T12:23:07.929900Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"events_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.933563Z","iopub.execute_input":"2023-03-12T12:23:07.933966Z","iopub.status.idle":"2023-03-12T12:23:07.952135Z","shell.execute_reply.started":"2023-03-12T12:23:07.933929Z","shell.execute_reply":"2023-03-12T12:23:07.950210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is length of the dataframe.

","metadata":{}},{"cell_type":"code","source":"temp = len(events_df)\nprint(\n f\"Length of the events.csv file is: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.953720Z","iopub.execute_input":"2023-03-12T12:23:07.954128Z","iopub.status.idle":"2023-03-12T12:23:07.962992Z","shell.execute_reply.started":"2023-03-12T12:23:07.954064Z","shell.execute_reply":"2023-03-12T12:23:07.961238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ Is there any missing data.

","metadata":{}},{"cell_type":"code","source":"events_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.965057Z","iopub.execute_input":"2023-03-12T12:23:07.965694Z","iopub.status.idle":"2023-03-12T12:23:07.980136Z","shell.execute_reply.started":"2023-03-12T12:23:07.965652Z","shell.execute_reply":"2023-03-12T12:23:07.978516Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is the distibution of the field Init.

","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(events_df, x=\"Init\", nbins=50)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:07.981856Z","iopub.execute_input":"2023-03-12T12:23:07.982243Z","iopub.status.idle":"2023-03-12T12:23:08.048941Z","shell.execute_reply.started":"2023-03-12T12:23:07.982211Z","shell.execute_reply":"2023-03-12T12:23:08.047456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is the distibution of the difference if fields Init and Completion.

","metadata":{}},{"cell_type":"code","source":"event_time = events_df.Completion - events_df.Init\nfig = px.histogram(x=event_time, nbins=30)\nfig.update_layout(\n xaxis_title=\"Event duration\",\n title={\n 'text': \"Distribution of event durations\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.050587Z","iopub.execute_input":"2023-03-12T12:23:08.050960Z","iopub.status.idle":"2023-03-12T12:23:08.124011Z","shell.execute_reply.started":"2023-03-12T12:23:08.050926Z","shell.execute_reply":"2023-03-12T12:23:08.122831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Type.

","metadata":{}},{"cell_type":"code","source":"events_type_counts = events_df.Type.value_counts()\n\nfig = px.bar(x=events_type_counts.index, y=events_type_counts.values)\nfig.update_layout(xaxis_title=\"Type\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.125293Z","iopub.execute_input":"2023-03-12T12:23:08.125717Z","iopub.status.idle":"2023-03-12T12:23:08.195013Z","shell.execute_reply.started":"2023-03-12T12:23:08.125678Z","shell.execute_reply":"2023-03-12T12:23:08.194035Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Kinetic.

","metadata":{}},{"cell_type":"code","source":"events_kinetic_counts = events_df.Kinetic.value_counts()\n\nfig = px.bar(x=events_kinetic_counts.index, y=events_kinetic_counts.values)\nfig.update_layout(xaxis_title=\"Kinetic\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.196482Z","iopub.execute_input":"2023-03-12T12:23:08.197043Z","iopub.status.idle":"2023-03-12T12:23:08.263055Z","shell.execute_reply.started":"2023-03-12T12:23:08.197006Z","shell.execute_reply":"2023-03-12T12:23:08.261884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How is the correlation matrix for the data looks like.

","metadata":{}},{"cell_type":"code","source":"corr_mat = np.round(events_df.corr(), 3)\n\nfig = px.imshow(\n corr_mat,\n x=corr_mat.columns,\n y=corr_mat.columns, \n text_auto=True\n )\nfig.update_xaxes(side=\"bottom\")\nfig.update_layout(\n title={\n 'text': \"Correlation Matrix of events features\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.264829Z","iopub.execute_input":"2023-03-12T12:23:08.265481Z","iopub.status.idle":"2023-03-12T12:23:08.332115Z","shell.execute_reply.started":"2023-03-12T12:23:08.265440Z","shell.execute_reply":"2023-03-12T12:23:08.330466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n##
2.6 Overview tasks.csv file
","metadata":{}},{"cell_type":"markdown","source":"

⚪ tasks.csv Task metadata for series in the defog dataset. (Not relevant for the series in the fog or daily datasets.)

\n\n*

Id The data series where the task was measured.

\n*

Begin Time (s) the task began.

\n*

End Time (s) the task ended.

\n*

Task One of seven tasks types in the DeFOG protocol, described on this page.

\n*

Description Description of the task.

","metadata":{}},{"cell_type":"markdown","source":"

❔ How the data looks like.

","metadata":{}},{"cell_type":"code","source":"tasks_df = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/tasks.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.334547Z","iopub.execute_input":"2023-03-12T12:23:08.334983Z","iopub.status.idle":"2023-03-12T12:23:08.351899Z","shell.execute_reply.started":"2023-03-12T12:23:08.334946Z","shell.execute_reply":"2023-03-12T12:23:08.350234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tasks_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.354128Z","iopub.execute_input":"2023-03-12T12:23:08.354569Z","iopub.status.idle":"2023-03-12T12:23:08.369180Z","shell.execute_reply.started":"2023-03-12T12:23:08.354530Z","shell.execute_reply":"2023-03-12T12:23:08.367625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is length of the dataframe.

","metadata":{}},{"cell_type":"code","source":"temp = len(tasks_df)\nprint(\n f\"Length of the subjects.csv file is: {color.BLUE}{temp}{color.END}\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.370942Z","iopub.execute_input":"2023-03-12T12:23:08.371311Z","iopub.status.idle":"2023-03-12T12:23:08.382380Z","shell.execute_reply.started":"2023-03-12T12:23:08.371270Z","shell.execute_reply":"2023-03-12T12:23:08.381309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ Is there any missing data.

","metadata":{}},{"cell_type":"code","source":"tasks_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.383558Z","iopub.execute_input":"2023-03-12T12:23:08.383973Z","iopub.status.idle":"2023-03-12T12:23:08.404595Z","shell.execute_reply.started":"2023-03-12T12:23:08.383916Z","shell.execute_reply":"2023-03-12T12:23:08.403031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ What is the distibution of the difference between fields Begin and End.

","metadata":{}},{"cell_type":"code","source":"tasks_df[\"Duration\"] = tasks_df.End - tasks_df.Begin\nfig = px.histogram(x=tasks_df[\"Duration\"], nbins=30, color_discrete_sequence=['lightslategrey'])\nfig.update_layout(\n xaxis_title=\"Task duration\",\n title={\n 'text': \"Distribution of task durations\",\n 'y':0.95,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n }\n)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:14:03.584055Z","iopub.execute_input":"2023-03-12T13:14:03.584604Z","iopub.status.idle":"2023-03-12T13:14:03.672743Z","shell.execute_reply.started":"2023-03-12T13:14:03.584559Z","shell.execute_reply":"2023-03-12T13:14:03.670998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the bar chart looks like for categorical field Task.

","metadata":{}},{"cell_type":"code","source":"tasks_task_counts = tasks_df.Task.value_counts()\n\nfig = px.bar(x=tasks_task_counts.index, y=tasks_task_counts.values, color_discrete_sequence=['lightslategrey'])\nfig.update_layout(xaxis_title=\"Task\", yaxis_title=\"Count\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:14:12.058772Z","iopub.execute_input":"2023-03-12T13:14:12.059416Z","iopub.status.idle":"2023-03-12T13:14:12.141054Z","shell.execute_reply.started":"2023-03-12T13:14:12.059366Z","shell.execute_reply":"2023-03-12T13:14:12.139564Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

❔ How the boxplot looks like for each task on duration.

","metadata":{}},{"cell_type":"code","source":"fig = px.box(\n tasks_df, \n x='Duration', y='Task',\n orientation='h', height=1000, \n color_discrete_sequence=['lightslategrey']\n)\n\nfig.update_layout(\n title={\n 'text': \"Task Duration Boxplot\",\n 'y':0.98,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'\n },\n margin=dict(l=50, r=0, t=50, b=50)\n)\nfig.show(autosize=True)","metadata":{"execution":{"iopub.status.busy":"2023-03-12T13:14:39.136590Z","iopub.execute_input":"2023-03-12T13:14:39.137018Z","iopub.status.idle":"2023-03-12T13:14:39.218023Z","shell.execute_reply.started":"2023-03-12T13:14:39.136983Z","shell.execute_reply":"2023-03-12T13:14:39.216455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# (⁠ ⁠ꈍ⁠ᴗ⁠ꈍ⁠) WORK STILL IN PROGRESS","metadata":{"execution":{"iopub.status.busy":"2023-03-12T12:23:08.666609Z","iopub.execute_input":"2023-03-12T12:23:08.667025Z","iopub.status.idle":"2023-03-12T12:23:08.673696Z","shell.execute_reply.started":"2023-03-12T12:23:08.666988Z","shell.execute_reply":"2023-03-12T12:23:08.671786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#
༼⁠ ⁠つ⁠ ⁠◕⁠‿⁠◕⁠ ⁠༽⁠つ Thank You!
\n\n

💌 Thank you for taking the time to read through my notebook. I hope you found it interesting and informative. If you have any feedback or suggestions for improvement, please don't hesitate to let me know in the comments.

🚀 If you liked this notebook, please consider upvoting it so that others can discover it too. Your support means a lot to me, and it helps to motivate me to create more content in the future.

❤️ Once again, thank you for your support, and I hope to see you again soon!

","metadata":{}}]}