{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.14","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":81933,"databundleVersionId":9643020,"sourceType":"competition"}],"dockerImageVersionId":30775,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Whats this about:

\n
\n This notebook explores working with actigraphy time series recordings to get an idea of the data at hand.\n \n
\n Tabular features EDA notebook is here\n
\n
","metadata":{}},{"cell_type":"markdown","source":"
\n \n Under some section (no motion periods, circadian rhythms and others) I added ideas for feature engineering, with some code just to demonstrate the principle (will add more). But to generate features you need to clean the data first, e.g. for the participant I examine here - it is worth removing the data after day 36.\n
","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib\nimport matplotlib.pyplot as plt\nimport os","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:49.919723Z","iopub.execute_input":"2024-09-28T18:07:49.920199Z","iopub.status.idle":"2024-09-28T18:07:50.378735Z","shell.execute_reply.started":"2024-09-28T18:07:49.920154Z","shell.execute_reply":"2024-09-28T18:07:50.377527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Function to plot ENMO, Angle Z, Light and any other column on below each other.","metadata":{}},{"cell_type":"code","source":"def plot_series_data(df, col='non-wear_flag',\n label='Worn (0 = Worn, 1 = Not Worn)',\n title='Non-Wear Flag',\n x_col='day_time', x_label='Day Relative to PCIAT + Time'):\n plt.figure(figsize=(18, 12))\n \n # ENMO\n plt.subplot(4, 1, 1)\n plt.scatter(df[x_col], df['enmo'], label='ENMO', color='green', s=1)\n plt.title('ENMO (Euclidean Norm Minus One)')\n plt.ylabel('Movement Intensity')\n\n # Angle Z\n plt.subplot(4, 1, 2)\n plt.scatter(df[x_col], df['anglez'], label='Angle Z', color='blue', s=1)\n plt.title('Angle Z')\n plt.ylabel('Angle (degrees)')\n\n # Light\n plt.subplot(4, 1, 3)\n plt.scatter(df[x_col], df['light'], label='Light', color='orange', s=1)\n plt.title('Ambient Light')\n plt.ylabel('Light (lux)')\n\n # Any other column\n plt.subplot(4, 1, 4)\n plt.scatter(df[x_col], df[col], label=col, color='red', s=1)\n plt.title(f'{title}')\n plt.ylabel(f'{label}')\n plt.xlabel(f'{x_label}')\n\n plt.tight_layout()\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:50.381029Z","iopub.execute_input":"2024-09-28T18:07:50.381641Z","iopub.status.idle":"2024-09-28T18:07:50.394977Z","shell.execute_reply.started":"2024-09-28T18:07:50.381599Z","shell.execute_reply":"2024-09-28T18:07:50.393746Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# One participant data preview","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/child-mind-institute-problematic-internet-use/train.csv')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:50.507950Z","iopub.execute_input":"2024-09-28T18:07:50.508385Z","iopub.status.idle":"2024-09-28T18:07:50.594551Z","shell.execute_reply.started":"2024-09-28T18:07:50.508313Z","shell.execute_reply":"2024-09-28T18:07:50.593275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path = '/kaggle/input/child-mind-institute-problematic-internet-use/series_train.parquet/id=0417c91e/part-0.parquet'\nseries_train = pd.read_parquet(path)\nseries_train","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-09-28T18:07:51.466613Z","iopub.execute_input":"2024-09-28T18:07:51.467896Z","iopub.status.idle":"2024-09-28T18:07:51.642005Z","shell.execute_reply.started":"2024-09-28T18:07:51.467845Z","shell.execute_reply":"2024-09-28T18:07:51.640573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The description of the actigraphy data columns and my comments below each one:","metadata":{}},{"cell_type":"markdown","source":"**series_{train|test}.parquet/id={id}** - Series to be used as training data, partitioned by id. Each series is a continuous recording of accelerometer data for a single subject spanning many days.\n\n- id - The patient identifier corresponding to the id field in train/test.csv.\n- step - An integer timestep for each observation within a series. \n\n> Corresponds to sequential data collection points.\n\n- X, Y, Z - Measure of acceleration, in g, experienced by the wrist-worn watch along each standard axis.\n\n> The three-dimensional accelerometer readings, describe the physical movement of the person wearing the device\n\n- enmo - As calculated and described by the wristpy package, ENMO is the Euclidean Norm Minus One of all accelerometer signals (along each of the x-, y-, and z-axis, measured in g-force) with negative values rounded to zero. Zero values are indicative of periods of no motion. While no standard measure of acceleration exists in this space, this is one of the several commonly computed features.\n\n> A scalar value for the level of motion, where higher values indicate more activity, and zeros - of no motion\n\n- anglez - As calculated and described by the wristpy package, Angle-Z is a metric derived from individual accelerometer components and refers to the angle of the arm relative to the horizontal plane.\n\n> Provide information about posture or how the arm is positioned during the activity\n\n- non-wear_flag - A flag (0: watch is being worn, 1: the watch is not worn) to help determine periods when the watch has been removed, based on the GGIR definition, which uses the standard deviation and range of the accelerometer data.\n\n> Would be great to know how it was calculated (I can see the `detect_nonwear` function in the wristpy package, but was it used? what were the parameters at least?)\n\n- light - Measure of ambient light in lux. See [here](https://actigraphcorp.my.site.com/support/s/article/Lux-Measurements) for details.\n\n> May help distinguish between different times of day (day vs. night) or types of activities (e.g., being indoors or outdoors)\n\n- battery_voltage - A measure of the battery voltage in mV.\n\n> Unlikely to be important for modeling, but may help in identifying issues related to the data collection process (low battery voltage might align with gaps, inconsistencies, or interruptions in the data)\n\n- time_of_day - Time of day representing the start of a 5s window that the data has been sampled over, with format %H:%M:%S.%9f.\n\n> Again as @AmbrosM have been already said, this description is not true. Time of the day was measured in nanoseconds since midnight.\n\n- weekday - The day of the week, coded as an integer with 1 being Monday and 7 being Sunday.\n\n> This could provide insights into behavior patterns during weekdays versus weekends\n\n- quarter - The quarter of the year, an integer from 1 to 4.\n\n> Could be relevant if there are seasonal patterns in activity (e.g., more outdoor activity in summer)\n\n- relative_date_PCIAT - The number of days (integer) since the PCIAT test was administered (negative days indicate that the actigraphy data has been collected before the test was administered).\n\n> Seems clear","metadata":{}},{"cell_type":"markdown","source":"The mentioned Wristpy package is here: [Wrist-Worn Accelerometer Data Processing](https://github.com/childmindresearch/wristpy)","metadata":{}},{"cell_type":"markdown","source":"Participant's info:","metadata":{}},{"cell_type":"code","source":"participant_id = path.split('/')[-2].split('=')[-1]\nparticipant_id","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:51.656979Z","iopub.execute_input":"2024-09-28T18:07:51.657457Z","iopub.status.idle":"2024-09-28T18:07:51.665617Z","shell.execute_reply.started":"2024-09-28T18:07:51.657404Z","shell.execute_reply":"2024-09-28T18:07:51.664292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train[train['id'] == participant_id]","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:51.869212Z","iopub.execute_input":"2024-09-28T18:07:51.870356Z","iopub.status.idle":"2024-09-28T18:07:51.904899Z","shell.execute_reply.started":"2024-09-28T18:07:51.870273Z","shell.execute_reply":"2024-09-28T18:07:51.903425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Plot data by steps:","metadata":{}},{"cell_type":"markdown","source":"So we'll look at the data collected from a 6-year-old girl, with no data on sleep or activity based on Physical Activity Questionnaire, who spends no time on the Internet and has an SII of 0. \n\nHer physical data includes a BMI of 15.48, height of 45.5 inches, and weight of 45.6 pounds, normal blood pressure readings (Systolic: 115, Diastolic: 73), and a heart rate of 86 bpm\n\nThe fitness endurance test shows a max stage of 6, with a duration of 9 minutes and 5 seconds.\n\nThe girl completed a small number of curl-ups, no push-ups, and had moderate sit-and-reach flexibility on both sides. There is missing data for grip strength tests, and the participant's trunk lift performance is in zone 1.0\n\n**A CGAS score of 40 suggests that the child has significant impairment in daily functioning**","metadata":{}},{"cell_type":"code","source":"plot_series_data(series_train, x_col='step', x_label='Step')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:52.075430Z","iopub.execute_input":"2024-09-28T18:07:52.075883Z","iopub.status.idle":"2024-09-28T18:07:53.929059Z","shell.execute_reply.started":"2024-09-28T18:07:52.075837Z","shell.execute_reply":"2024-09-28T18:07:53.927900Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: Looks ok, and the wear flag looks correct (always indicating that the device was worn). But when using the step column as the x-axis, the time points are assumed to be equidistant, meaning that each step represents a uniform interval (e.g., every 5 seconds). But in reality, there may be irregular time gaps between data points.\n
","metadata":{}},{"cell_type":"markdown","source":"Create a continuous time scale in days by transforming the `time_of_day` column (which is in nanoseconds) to hours and then combining it with `relative_date_PCIAT`:","metadata":{}},{"cell_type":"code","source":"series_train['time_of_day_hours'] = (\n series_train['time_of_day'] / 1e9 / 3600 #nanoseconds to hours\n)\nseries_train['day_time'] = series_train['relative_date_PCIAT'] + (\n series_train['time_of_day_hours'] / 24\n)\nseries_train['day_time']","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:53.931576Z","iopub.execute_input":"2024-09-28T18:07:53.932542Z","iopub.status.idle":"2024-09-28T18:07:53.950382Z","shell.execute_reply.started":"2024-09-28T18:07:53.932489Z","shell.execute_reply":"2024-09-28T18:07:53.949360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_series_data(series_train, x_col='day_time', x_label='Day Relative to PCIAT + Time')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:53.951636Z","iopub.execute_input":"2024-09-28T18:07:53.951958Z","iopub.status.idle":"2024-09-28T18:07:55.669387Z","shell.execute_reply.started":"2024-09-28T18:07:53.951923Z","shell.execute_reply":"2024-09-28T18:07:55.668323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(series_train[series_train['relative_date_PCIAT'] > 36])","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:55.672638Z","iopub.execute_input":"2024-09-28T18:07:55.673553Z","iopub.status.idle":"2024-09-28T18:07:55.683459Z","shell.execute_reply.started":"2024-09-28T18:07:55.673470Z","shell.execute_reply":"2024-09-28T18:07:55.682184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: Here, the data is plotted with real time on the x-axis, revealing gaps (especially on the right end) - the data for a few last weeks (after day 36) is only 1,830 measurements out of 287,179. These gaps suggest that some data was either not collected at all or was intentionally cut. Let's zoom in.\n
","metadata":{}},{"cell_type":"markdown","source":"Plot data for specific day(s):","metadata":{}},{"cell_type":"code","source":"start_day = 2 # second day of wearing the device\nshow_days = 1\n\nfirst_day = min(series_train['relative_date_PCIAT']) + start_day - 1\nfiltered_data = series_train[\n (series_train['relative_date_PCIAT'] >= first_day) &\n (series_train['relative_date_PCIAT'] <= first_day + show_days - 1)\n].copy()\n\nplot_series_data(filtered_data)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:55.684923Z","iopub.execute_input":"2024-09-28T18:07:55.685624Z","iopub.status.idle":"2024-09-28T18:07:56.909493Z","shell.execute_reply.started":"2024-09-28T18:07:55.685581Z","shell.execute_reply":"2024-09-28T18:07:56.908195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"Plot data for specific hour(s):","metadata":{}},{"cell_type":"code","source":"show_day = 2 # show second day\nhour_from = 13 # starting from 1 pm\nhour_to = 14 # to 2 pm\n\nshow_day = min(series_train['relative_date_PCIAT']) + show_day - 1\nfiltered_data = series_train[\n (series_train['relative_date_PCIAT'] == show_day) &\n (series_train['time_of_day_hours'] >= hour_from) & \n (series_train['time_of_day_hours'] < hour_to)\n].copy()\n\nplot_series_data(filtered_data, x_col='time_of_day_hours', x_label='Time, hours')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:56.911296Z","iopub.execute_input":"2024-09-28T18:07:56.911686Z","iopub.status.idle":"2024-09-28T18:07:58.140398Z","shell.execute_reply.started":"2024-09-28T18:07:56.911647Z","shell.execute_reply":"2024-09-28T18:07:58.139389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"Maybe the data has been truncated for periods of low battery? Let's check the battery voltage over time.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(18, 5))\n\nplt.scatter(series_train['day_time'], series_train['battery_voltage'], \n color='purple', label='Battery Voltage (mV)', s=1)\n\nplt.xlabel('Day Relative to PCIAT + Time')\nplt.ylabel('Battery Voltage (mV)')\nplt.title('Battery Voltage and Irregular Time Intervals')\nplt.legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:07:58.141778Z","iopub.execute_input":"2024-09-28T18:07:58.142134Z","iopub.status.idle":"2024-09-28T18:08:01.160193Z","shell.execute_reply.started":"2024-09-28T18:07:58.142095Z","shell.execute_reply":"2024-09-28T18:08:01.159019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: The battery did not seem to run out completely at the end. I guess, the smooth decline suggests that the device was likely functioning properly during this time.\n
","metadata":{}},{"cell_type":"markdown","source":"# - Day/nigth periods","metadata":{}},{"cell_type":"markdown","source":"Use the time data to categorize the day and night. Daytime is set between 8 AM and 9 PM.","metadata":{}},{"cell_type":"code","source":"day_start_hour = 8\nday_end_hour = 21\n\nseries_train['day_period'] = np.where(\n (series_train['time_of_day_hours'] >= day_start_hour) &\n (series_train['time_of_day_hours'] < day_end_hour),\n 'day', 'night'\n)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:08:01.161833Z","iopub.execute_input":"2024-09-28T18:08:01.162307Z","iopub.status.idle":"2024-09-28T18:08:01.208831Z","shell.execute_reply.started":"2024-09-28T18:08:01.162253Z","shell.execute_reply":"2024-09-28T18:08:01.207712Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(18, 5))\n\nplt.scatter(series_train['day_time'], series_train['light'], label='Light (Lux)', \n color='orange', s=5)\n\nplt.fill_between(series_train['day_time'],\n 0, series_train['light'].max(),\n where=(series_train['day_period'] == 'day'),\n color='blue', alpha=0.1, label='Day Period')\n\nplt.title('Light Levels (Lux) and Day/Night Categorization')\nplt.ylabel('Light (Lux)')\nplt.xlabel('Day Relative to PCIAT + Time')\nplt.legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:08:01.210277Z","iopub.execute_input":"2024-09-28T18:08:01.210767Z","iopub.status.idle":"2024-09-28T18:08:04.586369Z","shell.execute_reply.started":"2024-09-28T18:08:01.210714Z","shell.execute_reply":"2024-09-28T18:08:04.585152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: The purple areas (representing the day period) toward the end of the timeline appear stretched and irregular, but we already know that a number of records have been cut from the data.\n
","metadata":{}},{"cell_type":"markdown","source":"# - Time difference between steps","metadata":{}},{"cell_type":"markdown","source":"From the description, `time_of_day` should represent the start of a 5s window over which the data was sampled. But in the plots above, we saw that time periods can be irregular. Let's check this out.","metadata":{}},{"cell_type":"code","source":"expected_diff = 5\n\nseries_train['time_diff'] = (series_train['day_time'].diff() * 86400).round(0) # seconds in a day\nseries_train['measurement_after_gap'] = series_train['time_diff'] > expected_diff\nseries_train['measurement_after_gap'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:08:04.590592Z","iopub.execute_input":"2024-09-28T18:08:04.590983Z","iopub.status.idle":"2024-09-28T18:08:04.611424Z","shell.execute_reply.started":"2024-09-28T18:08:04.590942Z","shell.execute_reply":"2024-09-28T18:08:04.610320Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"5417 readings for this participant are off by more than 5 seconds.","metadata":{}},{"cell_type":"code","source":"series_train['time_diff'].describe()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:08:04.612661Z","iopub.execute_input":"2024-09-28T18:08:04.613002Z","iopub.status.idle":"2024-09-28T18:08:04.635885Z","shell.execute_reply.started":"2024-09-28T18:08:04.612966Z","shell.execute_reply":"2024-09-28T18:08:04.634655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"156445 / 60 / 60 # sec to hours","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:08:04.637218Z","iopub.execute_input":"2024-09-28T18:08:04.638198Z","iopub.status.idle":"2024-09-28T18:08:04.644822Z","shell.execute_reply.started":"2024-09-28T18:08:04.638155Z","shell.execute_reply":"2024-09-28T18:08:04.643614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: There are many time gaps >5s and the maximum time gap between measurements is 43 hours.\n
","metadata":{}},{"cell_type":"markdown","source":"
\n \"Flower\"\n \"Flower\"\n \"Flower\"\n
","metadata":{}},{"cell_type":"markdown","source":"# Main Statistics for all Participants","metadata":{}},{"cell_type":"markdown","source":"Here I calculate the percentage of rows with wear flag = 1 (device not worn) and the main statistics for key parameters for periods when the device was worn, ignoring time gaps, to get an idea of the values distributions.","metadata":{}},{"cell_type":"code","source":"DIR = '/kaggle/input/child-mind-institute-problematic-internet-use/series_train.parquet'\n\ndef process_file(file_path, participant_id):\n data = pd.read_parquet(file_path)\n non_wear_percentage = (data['non-wear_flag'].sum() / len(data)) * 100\n worn_data = data[data['non-wear_flag'] == 0]\n\n return {\n 'id': participant_id,\n 'non_wear_percentage': non_wear_percentage,\n 'enmo_stats': worn_data['enmo'].describe(),\n 'anglez_stats': worn_data['anglez'].describe(),\n 'light_stats': worn_data['light'].describe(),\n 'battery_voltage_stats': worn_data['battery_voltage'].describe(),\n 'unique_days': worn_data['relative_date_PCIAT'].nunique()\n }\n\nresults = []\nfor participant_id in os.listdir(DIR):\n file_path = os.path.join(DIR, participant_id, 'part-0.parquet')\n result = process_file(file_path, participant_id)\n results.append(result)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:08:04.646969Z","iopub.execute_input":"2024-09-28T18:08:04.647540Z","iopub.status.idle":"2024-09-28T18:09:32.645196Z","shell.execute_reply.started":"2024-09-28T18:08:04.647485Z","shell.execute_reply":"2024-09-28T18:09:32.644192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_results = []\nfor result in results:\n flat_row = {\n 'id': result['id'].replace('id=', ''),\n 'non_wear_percentage': result['non_wear_percentage'],\n 'unique_days': result['unique_days']\n }\n \n for key, stats in result.items():\n if isinstance(stats, pd.Series):\n for stat_name, stat_value in stats.items():\n flat_row[f'{key.replace(\"_stats\", \"\")}_{stat_name}'] = stat_value\n \n final_results.append(flat_row)\n \nstats_df = pd.DataFrame(final_results)\nstats_df","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:32.646617Z","iopub.execute_input":"2024-09-28T18:09:32.647044Z","iopub.status.idle":"2024-09-28T18:09:32.782855Z","shell.execute_reply.started":"2024-09-28T18:09:32.647005Z","shell.execute_reply":"2024-09-28T18:09:32.781642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# non_wear_percentage\nplt.figure(figsize=(12, 5))\nplt.subplot(1, 2, 1)\nplt.hist(stats_df['non_wear_percentage'], bins=20, edgecolor='black')\nplt.title('Non-Wear Percentage')\nplt.xlabel('Non-Wear Percentage')\nplt.ylabel('Frequency')\n\n# unique_days\nplt.subplot(1, 2, 2)\nplt.hist(stats_df['unique_days'], bins=20, edgecolor='black')\nplt.title('Number of Days')\nplt.xlabel('Unique Days')\nplt.ylabel('Frequency')\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:32.784196Z","iopub.execute_input":"2024-09-28T18:09:32.784578Z","iopub.status.idle":"2024-09-28T18:09:33.443851Z","shell.execute_reply.started":"2024-09-28T18:09:32.784538Z","shell.execute_reply":"2024-09-28T18:09:33.442637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"stats_df[['non_wear_percentage', 'unique_days']].describe()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:33.445144Z","iopub.execute_input":"2024-09-28T18:09:33.445535Z","iopub.status.idle":"2024-09-28T18:09:33.466266Z","shell.execute_reply.started":"2024-09-28T18:09:33.445494Z","shell.execute_reply":"2024-09-28T18:09:33.465163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"Let's make simple plots to get a general overview of the variability in min, max, median, and std across the parameters (enmo, anglez, light, and battery_voltage)","metadata":{}},{"cell_type":"code","source":"def plot_parameter_statistics(stats_df, parameter):\n stats_to_plot = ['min', 'max', '50%', 'std']\n stat_labels = ['Min', 'Max', 'Median', 'Std']\n\n plt.figure(figsize=(14, 5))\n\n for j, stat in enumerate(stats_to_plot):\n plt.subplot(1, 4, j + 1)\n \n data = stats_df[f'{parameter}_{stat}']\n plt.hist(data, bins=20, alpha=0.7, edgecolor='black')\n \n plt.title(f'{parameter.capitalize()} - {stat_labels[j]}')\n plt.xlabel('Values')\n plt.ylabel('Frequency')\n plt.grid(True)\n\n plt.tight_layout()\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:33.468183Z","iopub.execute_input":"2024-09-28T18:09:33.468694Z","iopub.status.idle":"2024-09-28T18:09:33.476922Z","shell.execute_reply.started":"2024-09-28T18:09:33.468639Z","shell.execute_reply":"2024-09-28T18:09:33.475795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### ENMO","metadata":{}},{"cell_type":"code","source":"plot_parameter_statistics(stats_df, 'enmo')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:33.478502Z","iopub.execute_input":"2024-09-28T18:09:33.478897Z","iopub.status.idle":"2024-09-28T18:09:34.626724Z","shell.execute_reply.started":"2024-09-28T18:09:33.478858Z","shell.execute_reply":"2024-09-28T18:09:34.625401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"code","source":"plot_parameter_statistics(stats_df, 'anglez')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:34.628036Z","iopub.execute_input":"2024-09-28T18:09:34.628452Z","iopub.status.idle":"2024-09-28T18:09:35.777407Z","shell.execute_reply.started":"2024-09-28T18:09:34.628403Z","shell.execute_reply":"2024-09-28T18:09:35.776298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"code","source":"plot_parameter_statistics(stats_df, 'light')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:35.778739Z","iopub.execute_input":"2024-09-28T18:09:35.779098Z","iopub.status.idle":"2024-09-28T18:09:37.058614Z","shell.execute_reply.started":"2024-09-28T18:09:35.779060Z","shell.execute_reply":"2024-09-28T18:09:37.057365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"code","source":"plot_parameter_statistics(stats_df, 'battery_voltage')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:37.060495Z","iopub.execute_input":"2024-09-28T18:09:37.061514Z","iopub.status.idle":"2024-09-28T18:09:38.196466Z","shell.execute_reply.started":"2024-09-28T18:09:37.061450Z","shell.execute_reply":"2024-09-28T18:09:38.195286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"### Save the aggregated features","metadata":{}},{"cell_type":"code","source":"stats_df['n_records'] = stats_df['enmo_count']\nstats_df = stats_df.loc[:, ~stats_df.columns.str.endswith('_count')]\nstats_df.to_csv('stats.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.198001Z","iopub.execute_input":"2024-09-28T18:09:38.198397Z","iopub.status.idle":"2024-09-28T18:09:38.271747Z","shell.execute_reply.started":"2024-09-28T18:09:38.198327Z","shell.execute_reply":"2024-09-28T18:09:38.270752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \"Flower\"\n \"Flower\"\n \"Flower\"\n
","metadata":{}},{"cell_type":"markdown","source":"Below, I explore several key aspects of actigraphy data to derive meaningful insights for feature engineering using only measurements when the device was worn according to the `non-wear_flag` column.\n\nSome of the articles I referred to:\n- [Accelerometer assessed moderate-to-vigorous physical activity and successful ageing: results from the Whitehall II study](https://www.nature.com/articles/srep45772)\n- [Intensity Thresholds on Raw Acceleration Data: Euclidean Norm Minus One (ENMO) and Mean Amplitude Deviation (MAD) Approaches](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0164045)","metadata":{}},{"cell_type":"code","source":"worn_data = series_train[series_train['non-wear_flag'] == 0]\n\n# recalculate time difference between rows and measurement_after_gap flag\nworn_data['time_diff'] = (worn_data['day_time'].diff() * 86400).round(0)\nworn_data['measurement_after_gap'] = worn_data['time_diff'] > expected_diff\nworn_data['measurement_after_gap'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.273006Z","iopub.execute_input":"2024-09-28T18:09:38.273379Z","iopub.status.idle":"2024-09-28T18:09:38.303206Z","shell.execute_reply.started":"2024-09-28T18:09:38.273307Z","shell.execute_reply":"2024-09-28T18:09:38.302104Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# No motion periods","metadata":{}},{"cell_type":"markdown","source":"Zero values are indicative of periods of no motion in the `enmo` column. Let's see if there are any for our participant and how long they lasted.","metadata":{}},{"cell_type":"code","source":"worn_data[worn_data['enmo'] == 0]","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.304587Z","iopub.execute_input":"2024-09-28T18:09:38.304923Z","iopub.status.idle":"2024-09-28T18:09:38.340140Z","shell.execute_reply.started":"2024-09-28T18:09:38.304887Z","shell.execute_reply":"2024-09-28T18:09:38.339001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To calculate the duration of periods of no motion, we need to account for time gaps between records, e.g., treat such gaps as breaks in continuous data collection.\nNext, group consecutive periods of no motion by groupping rows with consecutive zeros in `enmo`","metadata":{}},{"cell_type":"code","source":"no_motion = worn_data['enmo'] == 0\n\nmotion_group = (\n (no_motion != no_motion.shift()) |\n (worn_data['measurement_after_gap'])\n).cumsum()\n\nno_motion_periods = worn_data[no_motion].groupby(\n motion_group\n)['day_time'].agg(['min', 'max'])\n\nno_motion_periods['duration_sec'] = (\n (no_motion_periods['max'] - no_motion_periods['min']) * 86400\n).round(0).astype(int)\n\nprint(f\"Min duration in seconds: {no_motion_periods['duration_sec'].min()}\")\nprint(f\"Max duration in seconds: {no_motion_periods['duration_sec'].max()}\")","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.341567Z","iopub.execute_input":"2024-09-28T18:09:38.341936Z","iopub.status.idle":"2024-09-28T18:09:38.388997Z","shell.execute_reply.started":"2024-09-28T18:09:38.341897Z","shell.execute_reply":"2024-09-28T18:09:38.387807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A duration of 0 seconds results from single records surrounded by periods of motion. Given that the time column in the data represents the start of a 5 second window over which the data is sampled, it is reasonable to assume that such single period records have a duration of 5 seconds and add 5 seconds to all durations.","metadata":{}},{"cell_type":"code","source":"no_motion_periods['duration_sec'] += 5\n\nprint(\"Total duration in hours:\", no_motion_periods['duration_sec'].sum() / 3600)\nno_motion_periods.sort_values(by='duration_sec')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.390370Z","iopub.execute_input":"2024-09-28T18:09:38.390727Z","iopub.status.idle":"2024-09-28T18:09:38.408299Z","shell.execute_reply.started":"2024-09-28T18:09:38.390682Z","shell.execute_reply":"2024-09-28T18:09:38.407210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are 3034 periods of no movement, ranging from 0 (a single record surrounded by periods of movement) to 110 seconds, or 7.6 hours of no movement during the entire time the device was worn.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10, 5))\n\nplt.hist(no_motion_periods.loc[:, 'duration_sec'],\n bins=50, edgecolor='black')\nplt.xlabel('Duration, sec')\nplt.ylabel('Frequency')\nplt.title('Duration of No-Motion Periods')\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.409665Z","iopub.execute_input":"2024-09-28T18:09:38.410030Z","iopub.status.idle":"2024-09-28T18:09:38.878130Z","shell.execute_reply.started":"2024-09-28T18:09:38.409990Z","shell.execute_reply":"2024-09-28T18:09:38.877015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"bins = [0, 15, 30, 60, 120]\nno_motion_periods['duration_bin'] = pd.cut(no_motion_periods['duration_sec'], bins)\nperiod_counts = no_motion_periods['duration_bin'].value_counts().sort_index()\nperiod_counts","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.885669Z","iopub.execute_input":"2024-09-28T18:09:38.886073Z","iopub.status.idle":"2024-09-28T18:09:38.906209Z","shell.execute_reply.started":"2024-09-28T18:09:38.886035Z","shell.execute_reply":"2024-09-28T18:09:38.905056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The majority of the no-motion periods lasted less than 30 sec. Now let's plot the frequency of no motion by time of the day.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10, 5))\nplt.hist(worn_data[no_motion]['time_of_day_hours'],\n bins=24, range=(0, 24), edgecolor='black')\nplt.title('Frequency of No Motion by Time of Day')\nplt.xlabel('Time of Day (Hours)')\nplt.ylabel('Frequency')\nplt.xticks(range(0, 25))\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:38.907738Z","iopub.execute_input":"2024-09-28T18:09:38.908101Z","iopub.status.idle":"2024-09-28T18:09:39.338093Z","shell.execute_reply.started":"2024-09-28T18:09:38.908064Z","shell.execute_reply":"2024-09-28T18:09:39.336870Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: This distribution appears consistent with a typical day cycle where activity is lower at night and early morning (sleep) - more no motion periods, increases during the day (wakefulness) - less no motion periods, and then decreases again in the evening and night - a bit more no motion periods after 5 pm. I think no motion data (e.g. mean, max, or sdt) can be used as a feature.\n
\n ","metadata":{}},{"cell_type":"markdown","source":"Let's visually assess the stability of no-motion periodds for the participant","metadata":{}},{"cell_type":"code","source":"no_motion_periods['day'] = no_motion_periods['min'].astype(int)\n\ndaily_stats = no_motion_periods.groupby(no_motion_periods['day']) \\\n .agg(total_duration=('duration_sec', 'sum'),\n count_periods=('duration_sec', 'size'))\n\n\nfig, ax1 = plt.subplots(figsize=(10, 5))\n\ncolor = 'tab:blue'\nax1.set_xlabel('Day')\nax1.set_ylabel('Total Duration (sec)', color=color)\nax1.plot(daily_stats.index, daily_stats['total_duration'], color=color)\nax1.tick_params(axis='y', labelcolor=color)\n\nax2 = ax1.twinx()\ncolor = 'tab:orange'\nax2.set_ylabel('Count of Periods', color=color)\nax2.plot(daily_stats.index, daily_stats['count_periods'], color=color)\nax2.tick_params(axis='y', labelcolor=color)\n\nplt.title('Daily No-Motion Periods: Total Duration and Count')\nfig.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:39.339533Z","iopub.execute_input":"2024-09-28T18:09:39.339897Z","iopub.status.idle":"2024-09-28T18:09:39.802171Z","shell.execute_reply.started":"2024-09-28T18:09:39.339858Z","shell.execute_reply":"2024-09-28T18:09:39.800995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"# - Ideas for feature engeneering","metadata":{}},{"cell_type":"markdown","source":"To account for the variation in wearing durations across participants, we must calculate relative values by normalizing against the total duration of wearing the device (e.g., days). \n\nAggregated (mean, median, max, std) across days:\n- Total duration of no motion per day\n- Count of no-motion periods per day\n\nExample of aggregated features calculation:","metadata":{}},{"cell_type":"code","source":"features = daily_stats[['total_duration', 'count_periods']].agg(\n ['median', 'max', 'std']\n).T.unstack().to_frame().T\n\nfeatures.columns = ['_'.join(col) for col in features.columns]\n\nfeatures","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:39.803938Z","iopub.execute_input":"2024-09-28T18:09:39.804436Z","iopub.status.idle":"2024-09-28T18:09:39.831817Z","shell.execute_reply.started":"2024-09-28T18:09:39.804378Z","shell.execute_reply":"2024-09-28T18:09:39.830645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Circadian Rhythm Analysis","metadata":{}},{"cell_type":"markdown","source":"Group the activity data by hour of the day and plot the average activity (ENMO) across the 24-hour day.","metadata":{}},{"cell_type":"code","source":"hourly_activity = worn_data.groupby(\n worn_data['time_of_day_hours'].astype(int)\n)['enmo'].agg(['mean', 'max', 'std'])\n\nfig, axes = plt.subplots(1, 3, figsize=(18, 5))\n\n# average ENMO\naxes[0].plot(hourly_activity.index, hourly_activity['mean'], marker='o', color='blue')\naxes[0].set_xticks(hourly_activity.index)\naxes[0].set_xlabel('Hour of Day')\naxes[0].set_ylabel('Average ENMO')\naxes[0].set_title('Average ENMO by Hour of Day')\naxes[0].grid(True)\n\n# maximum ENMO\naxes[1].plot(hourly_activity.index, hourly_activity['max'], marker='o', color='green')\naxes[1].set_xticks(hourly_activity.index)\naxes[1].set_xlabel('Hour of Day')\naxes[1].set_ylabel('Maximum ENMO')\naxes[1].set_title('Maximum ENMO by Hour of Day')\naxes[1].grid(True)\n\n# standard deviation of ENMO\naxes[2].plot(hourly_activity.index, hourly_activity['std'], marker='o', color='red')\naxes[2].set_xticks(hourly_activity.index)\naxes[2].set_xlabel('Hour of Day')\naxes[2].set_ylabel('ENMO Std Dev')\naxes[2].set_title('Standard Deviation of ENMO by Hour of Day')\naxes[2].grid(True)\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:39.833260Z","iopub.execute_input":"2024-09-28T18:09:39.833649Z","iopub.status.idle":"2024-09-28T18:09:41.046654Z","shell.execute_reply.started":"2024-09-28T18:09:39.833607Z","shell.execute_reply":"2024-09-28T18:09:41.045392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"# - Ideas for feature engeneering","metadata":{}},{"cell_type":"markdown","source":"Here we can make features capturing the variation in activity across the 24-hour cycle, separately for day and night times (or wakefulness and sleep periods). It can be useful to calculate features from both average and maximum hourly activity, as they provide different insights: the overall behaviour during each hour and the peak or most extreme activity during each hour. Here i go with mean enmo values.\n\nAggregated (mean, median, max, std):\n- Standard deviation across hourly means per day (captures how different each hour is from the rest)\n- The hour with the highest activity\n- Entropy of the distribution across hours (how evenly the activity is distributed throughout the day)","metadata":{}},{"cell_type":"code","source":"hourly_activity = worn_data.groupby(\n [worn_data['relative_date_PCIAT'].astype(int),\n worn_data['time_of_day_hours'].astype(int),\n worn_data['day_period']]\n)['enmo'].agg(['mean', 'max'])\n\nfeatures = hourly_activity['mean'].groupby(\n ['relative_date_PCIAT', 'day_period']\n).agg(\n std_across_hours='std',\n peak_hour=lambda x: x.idxmax()[1],\n entropy=lambda x: -(x / x.sum() * np.log(x / x.sum() + 1e-9)).sum()\n)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:41.048197Z","iopub.execute_input":"2024-09-28T18:09:41.048566Z","iopub.status.idle":"2024-09-28T18:09:41.186241Z","shell.execute_reply.started":"2024-09-28T18:09:41.048528Z","shell.execute_reply":"2024-09-28T18:09:41.184956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(3, 1, figsize=(12, 12), sharex=True)\n\n# Plot std_across_hours\nfor period in ['day', 'night']:\n subset = features.xs(period, level='day_period')\n axes[0].plot(subset.index, subset['std_across_hours'], label=period)\n\naxes[0].set_title('Std Across Hours Over Days')\naxes[0].set_ylabel('Std Across Hours')\naxes[0].legend()\n\n# Plot peak_hour\nfor period in ['day', 'night']:\n subset = features.xs(period, level='day_period')\n axes[1].plot(subset.index, subset['peak_hour'], label=period)\n\naxes[1].set_title('Peak Hour Over Days')\naxes[1].set_ylabel('Peak Hour')\naxes[1].legend()\n\n# Plot entropy\nfor period in ['day', 'night']:\n subset = features.xs(period, level='day_period')\n axes[2].plot(subset.index, subset['entropy'], label=period)\n\naxes[2].set_title('Entropy Over Days')\naxes[2].set_xlabel('Relative Date PCIAT')\naxes[2].set_ylabel('Entropy')\naxes[2].legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:41.187667Z","iopub.execute_input":"2024-09-28T18:09:41.188046Z","iopub.status.idle":"2024-09-28T18:09:42.457506Z","shell.execute_reply.started":"2024-09-28T18:09:41.188006Z","shell.execute_reply":"2024-09-28T18:09:42.456302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"code","source":"day_features = features.xs('day', level='day_period').agg(\n ['median', 'max', 'std']\n).stack().to_frame().T\nday_features.columns = [\n f'{stat}_{feature}_day' for stat, feature in day_features.columns\n]\n\nnight_features = features.xs('night', level='day_period').agg(\n ['median', 'max', 'std']\n).stack().to_frame().T\nnight_features.columns = [\n f'{stat}_{feature}_night' for stat, feature in night_features.columns\n]\n\nfeatures = pd.concat([day_features, night_features], axis=1)\nfeatures","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:42.458913Z","iopub.execute_input":"2024-09-28T18:09:42.459293Z","iopub.status.idle":"2024-09-28T18:09:42.504260Z","shell.execute_reply.started":"2024-09-28T18:09:42.459253Z","shell.execute_reply":"2024-09-28T18:09:42.503169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Physical Activity Analysis","metadata":{}},{"cell_type":"markdown","source":"Analyze the Moderate to Vigorous Physical Activity (MVPA) based on a threshold of ENMO values, and calculate the duration of the detected MVPA activity bouts. To avoid over-segmentation of MVPA bouts, where brief interruptions or noise may break up continuous periods of activity, we can introduce a smoothing mechanism. This ensures that short interruptions in MVPA do not split the bouts into separate periods.","metadata":{}},{"cell_type":"code","source":"# Example smoothing method\n\nwindow_size = 12 # Rolling window size in rows (equal to 60 s)\n\ndef rolling_average_per_segment(df, window_size):\n return df['enmo'].rolling(window=window_size, min_periods=1).mean()\n\nsegment_group = worn_data['measurement_after_gap'].cumsum()\n\nworn_data['smoothed_enmo'] = worn_data.groupby(segment_group)['enmo'].transform(\n lambda x: x.rolling(window=window_size, min_periods=1).mean()\n)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:42.505584Z","iopub.execute_input":"2024-09-28T18:09:42.505955Z","iopub.status.idle":"2024-09-28T18:09:44.135268Z","shell.execute_reply.started":"2024-09-28T18:09:42.505916Z","shell.execute_reply":"2024-09-28T18:09:44.134236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"show_day = 2 # show second day\nhour_from = 15\nhour_to = 17\n\nshow_day = min(worn_data['relative_date_PCIAT']) + show_day - 1\nfiltered_data = worn_data[\n (worn_data['relative_date_PCIAT'] == show_day) &\n (worn_data['time_of_day_hours'] >= hour_from) & \n (worn_data['time_of_day_hours'] < hour_to)\n].copy()\n\nplt.figure(figsize=(18, 5))\nplt.plot(\n filtered_data['time_of_day_hours'],\n filtered_data['enmo'],\n label='ENMO', alpha=0.7)\nplt.plot(\n filtered_data['time_of_day_hours'],\n filtered_data['smoothed_enmo'],\n label='Smoothed ENMO', alpha=0.9\n)\nplt.xlabel('Time, hours')\nplt.ylabel('ENMO Value')\nplt.title('ENMO and Smoothed ENMO Over Time')\nplt.legend()\nplt.grid(True)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:44.136615Z","iopub.execute_input":"2024-09-28T18:09:44.136945Z","iopub.status.idle":"2024-09-28T18:09:44.588968Z","shell.execute_reply.started":"2024-09-28T18:09:44.136909Z","shell.execute_reply":"2024-09-28T18:09:44.587898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"worn_data.drop('smoothed_enmo', axis=1, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:44.590582Z","iopub.execute_input":"2024-09-28T18:09:44.591072Z","iopub.status.idle":"2024-09-28T18:09:44.613635Z","shell.execute_reply.started":"2024-09-28T18:09:44.591016Z","shell.execute_reply":"2024-09-28T18:09:44.612389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As in the aforementioned article, in order to classify physical activity as MVPA, I only retained activities that lasted at least 1 minute and met the criteria for the 100 mg (= 0.1g) threshold. But first I merge MVPA episodes that are less than 60 seconds apart. Example with dummy data:","metadata":{}},{"cell_type":"code","source":"mvpa_threshold = 0.1\nmerge_gap = 60","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:44.614928Z","iopub.execute_input":"2024-09-28T18:09:44.615286Z","iopub.status.idle":"2024-09-28T18:09:44.620681Z","shell.execute_reply.started":"2024-09-28T18:09:44.615247Z","shell.execute_reply":"2024-09-28T18:09:44.619508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = {\n 'day_time': [\n 1.00011574, 1.00023148, 1.00034722, 1.00046296, 1.00113426, \n 1.00125000, 1.00136574, 1.00148148, 1.00159722, 2.00023148\n ],\n 'enmo': [0.05, 0.12, 0.11, 0.08, 0.15, 0.12, 0.09, 0.08, 0.15, 0.2]\n}\ndata = pd.DataFrame(data)\ndata['time_diff'] = (data['day_time'].diff() * 86400).round(0)\ndata['is_mvpa'] = data['enmo'] > mvpa_threshold\n\ndata['mvpa_group'] = (\n (data['is_mvpa'] != data['is_mvpa'].shift()) |\n (data['time_diff'] >= merge_gap)\n).cumsum()\n\ndata['is_mvpa_start'] = (\n (data['mvpa_group'] != data['mvpa_group'].shift()) &\n data['is_mvpa']\n)\n\ndata['last_mvpa_time'] = data['day_time'].where(data['is_mvpa']).ffill().shift()\ndata['mvpa_time_diff'] = ((data['day_time'] - data['last_mvpa_time']) * 86400).round(0)\ndata['group_increment'] = data['is_mvpa_start'] & (\n (data['mvpa_time_diff'] >= merge_gap) | data['last_mvpa_time'].isnull()\n)\ndata['group_increment'] = data['group_increment'].astype(int)\ndata['merged_group'] = data['group_increment'].cumsum()\ndata.loc[~data['is_mvpa'], 'merged_group'] = np.nan\ndata","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:44.622426Z","iopub.execute_input":"2024-09-28T18:09:44.622914Z","iopub.status.idle":"2024-09-28T18:09:44.666683Z","shell.execute_reply.started":"2024-09-28T18:09:44.622858Z","shell.execute_reply":"2024-09-28T18:09:44.665317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The first MVPA activity group is 2, the second is 4 and is 68 s apart, so they are not merged (the merged group numbers remain different). Groups 4 and 6 are separated by 30 s of non-MVPA activity, so they are merged into one group (merged group 2). Groups 6 and 7 are consecutive but are not merged as there is a huge time gap between them.","metadata":{}},{"cell_type":"markdown","source":"Let's apply to our data:","metadata":{}},{"cell_type":"code","source":"def merge_mvpa_groups(df, allowed_gap=60, merge_gap=60):\n last_mvpa_time = df['day_time'].where(df['is_mvpa']).ffill().shift()\n \n mvpa_time_diff = (\n (df['day_time'] - last_mvpa_time) * 86400\n ).round(0)\n \n mvpa_group = (\n (df['is_mvpa'] != df['is_mvpa'].shift()) |\n (df['time_diff'] >= allowed_gap)\n ).cumsum()\n \n is_mvpa_start = (\n (mvpa_group != mvpa_group.shift()) &\n df['is_mvpa']\n )\n \n group_increment = is_mvpa_start & (\n (mvpa_time_diff >= merge_gap) | last_mvpa_time.isnull()\n )\n \n merged_group = group_increment.cumsum()\n merged_group.loc[~df['is_mvpa']] = np.nan\n \n return merged_group\n\nworn_data['is_mvpa'] = worn_data['enmo'] > mvpa_threshold\nworn_data['mvpa_merged_group'] = merge_mvpa_groups(worn_data)\n\nmvpa_periods = worn_data[\n worn_data['is_mvpa']\n].groupby('mvpa_merged_group')['day_time'].agg(['min', 'max'])\n\nmvpa_periods['duration_sec'] = (\n mvpa_periods['max'] - mvpa_periods['min']\n) * 86400 # days to seconds\n\nmvpa_periods = mvpa_periods[mvpa_periods['duration_sec'] >= 60]\nmvpa_periods['duration_min'] = mvpa_periods['duration_sec'] / 60\n\nmvpa_periods.sort_values(by='duration_sec')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:44.668245Z","iopub.execute_input":"2024-09-28T18:09:44.668736Z","iopub.status.idle":"2024-09-28T18:09:44.755423Z","shell.execute_reply.started":"2024-09-28T18:09:44.668684Z","shell.execute_reply":"2024-09-28T18:09:44.754243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 5))\nplt.hist(mvpa_periods['duration_min'], bins=30, edgecolor='black')\nplt.xlabel('Duration (min)')\nplt.ylabel('Frequency')\nplt.title('Distribution of MVPA Bout Durations')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:44.756911Z","iopub.execute_input":"2024-09-28T18:09:44.757250Z","iopub.status.idle":"2024-09-28T18:09:45.121629Z","shell.execute_reply.started":"2024-09-28T18:09:44.757215Z","shell.execute_reply":"2024-09-28T18:09:45.120347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"summary_stats = mvpa_periods['duration_min'].describe()\n\nshort_mvpa_count = mvpa_periods[mvpa_periods['duration_min'] < 10].shape[0]\nlong_mvpa_count = mvpa_periods[mvpa_periods['duration_min'] >= 10].shape[0]\n\nprint(f\"Summary Statistics:\\n{summary_stats}\\n\")\nprint(f\"Short MVPA periods (< 10 min): {short_mvpa_count}\")\nprint(f\"Long MVPA periods (>= 10 min): {long_mvpa_count}\")","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.123155Z","iopub.execute_input":"2024-09-28T18:09:45.123580Z","iopub.status.idle":"2024-09-28T18:09:45.138291Z","shell.execute_reply.started":"2024-09-28T18:09:45.123538Z","shell.execute_reply":"2024-09-28T18:09:45.136941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: I got the distribution where the majority of MVPA bouts are very short, between 0 and 10 minutes. Perhaps the thresholds need to be adjusted, and if it looks realistic, number of MVPA activity boots and their duration can also be used as a feature for modeling SII prediction.\n
","metadata":{}},{"cell_type":"code","source":"worn_data.drop(['is_mvpa', 'mvpa_merged_group'], axis=1, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.139599Z","iopub.execute_input":"2024-09-28T18:09:45.140110Z","iopub.status.idle":"2024-09-28T18:09:45.162234Z","shell.execute_reply.started":"2024-09-28T18:09:45.140053Z","shell.execute_reply":"2024-09-28T18:09:45.161239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's visually assess the stability of MVPA periodds for the participant","metadata":{}},{"cell_type":"code","source":"mvpa_periods['day'] = mvpa_periods['min'].astype(int)\n\ndaily_stats = mvpa_periods.groupby(mvpa_periods['day']) \\\n .agg(total_duration=('duration_sec', 'sum'),\n count_periods=('duration_sec', 'size'))\n\n\nfig, ax1 = plt.subplots(figsize=(10, 5))\n\ncolor = 'tab:blue'\nax1.set_xlabel('Day')\nax1.set_ylabel('Total Duration (sec)', color=color)\nax1.plot(daily_stats.index, daily_stats['total_duration'], color=color)\nax1.tick_params(axis='y', labelcolor=color)\n\nax2 = ax1.twinx()\ncolor = 'tab:orange'\nax2.set_ylabel('Count of Periods', color=color)\nax2.plot(daily_stats.index, daily_stats['count_periods'], color=color)\nax2.tick_params(axis='y', labelcolor=color)\n\nplt.title('Daily MVPA Periods: Total Duration and Count')\nfig.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.163683Z","iopub.execute_input":"2024-09-28T18:09:45.164059Z","iopub.status.idle":"2024-09-28T18:09:45.753586Z","shell.execute_reply.started":"2024-09-28T18:09:45.164020Z","shell.execute_reply":"2024-09-28T18:09:45.752321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"# - Ideas for feature engeneering","metadata":{}},{"cell_type":"markdown","source":"Here we can make the same features as for no_motion periods:\n\nAggregated (mean, median, max, std):\n- Total duration of MVPA per day\n- Number of MVPA bouts per day","metadata":{}},{"cell_type":"code","source":"features = daily_stats[['total_duration', 'count_periods']].agg(\n ['median', 'max', 'std']\n).T.unstack().to_frame().T\n\nfeatures.columns = ['_'.join(col) for col in features.columns]\n\nfeatures","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.755119Z","iopub.execute_input":"2024-09-28T18:09:45.755528Z","iopub.status.idle":"2024-09-28T18:09:45.780402Z","shell.execute_reply.started":"2024-09-28T18:09:45.755485Z","shell.execute_reply":"2024-09-28T18:09:45.779449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Activity Transition Analysis","metadata":{}},{"cell_type":"markdown","source":"Now I extend the analysis to look at transitions between low, moderate and vigorous activity. To smooth out sudden, short bursts of different activities, I filter out segments with a duration below a 1 minute threshold.","metadata":{}},{"cell_type":"code","source":"mvpa_threshold = 0.1\nvig_threshold = 0.5\n\nworn_data['activity_type'] = pd.cut(\n worn_data['enmo'],\n bins=[-np.inf, mvpa_threshold, vig_threshold, np.inf],\n labels=['low', 'moderate', 'vigorous']\n)\nactivity_group = (\n (worn_data['activity_type'] != worn_data['activity_type'].shift()) |\n (worn_data['measurement_after_gap'])\n).cumsum()\n\nactivity_periods = worn_data.groupby(activity_group).agg(\n min=('day_time', 'min'),\n max=('day_time', 'max'),\n activity_type=('activity_type', 'first')\n)\nactivity_periods['duration_sec'] = (\n activity_periods['max'] - activity_periods['min']\n) * 86400 + 5 \n\nactivity_periods = activity_periods[activity_periods['duration_sec'] >= 60]\nactivity_periods['duration_min'] = activity_periods['duration_sec'] / 60\n\nactivity_periods['day'] = activity_periods['min'].astype(int)\nactivity_periods['transition_num'] = (\n activity_periods.groupby('day')['activity_type']\n .apply(lambda x: (x != x.shift()).cumsum())\n .reset_index(level=0, drop=True)\n)\n\nactivity_periods.sort_values(by='duration_sec')","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.782024Z","iopub.execute_input":"2024-09-28T18:09:45.782453Z","iopub.status.idle":"2024-09-28T18:09:45.860992Z","shell.execute_reply.started":"2024-09-28T18:09:45.782411Z","shell.execute_reply":"2024-09-28T18:09:45.859827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"max(activity_periods['transition_num'])","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.862316Z","iopub.execute_input":"2024-09-28T18:09:45.862708Z","iopub.status.idle":"2024-09-28T18:09:45.871405Z","shell.execute_reply.started":"2024-09-28T18:09:45.862668Z","shell.execute_reply":"2024-09-28T18:09:45.870187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"activity_periods[activity_periods['activity_type'] == 'vigorous']","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.872768Z","iopub.execute_input":"2024-09-28T18:09:45.873114Z","iopub.status.idle":"2024-09-28T18:09:45.894167Z","shell.execute_reply.started":"2024-09-28T18:09:45.873077Z","shell.execute_reply":"2024-09-28T18:09:45.892846Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"summary_table = pd.DataFrame({\n 'Number of Periods': activity_periods['activity_type'].value_counts(),\n 'Total Duration (min)': activity_periods.groupby(\n 'activity_type', observed=False\n )['duration_min'].sum()\n}).reset_index()\n\nsummary_table.rename(columns={'index': 'Activity Type'}, inplace=True)\nsummary_table","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.895636Z","iopub.execute_input":"2024-09-28T18:09:45.896050Z","iopub.status.idle":"2024-09-28T18:09:45.914825Z","shell.execute_reply.started":"2024-09-28T18:09:45.896009Z","shell.execute_reply":"2024-09-28T18:09:45.913558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: Most of the recorded activity was classified as low intensity, with very few periods classified as moderate, and one 1-min spike as vigorous\n
","metadata":{}},{"cell_type":"markdown","source":"Visualize changes across days:","metadata":{}},{"cell_type":"code","source":"for activity_type in ['low', 'moderate']:\n activity_data = activity_periods[activity_periods['activity_type'] == activity_type]\n\n daily_stats = activity_data.groupby('day').agg(\n total_duration=('duration_sec', 'sum'),\n count_periods=('duration_sec', 'size')\n )\n \n fig, ax1 = plt.subplots(figsize=(10, 5))\n ax1.set_xlabel('Day')\n ax1.set_ylabel('Total Duration (sec)', color='tab:blue')\n ax1.plot(daily_stats.index, daily_stats['total_duration'], \n color='tab:blue', marker='o')\n ax1.tick_params(axis='y', labelcolor='tab:blue')\n \n ax2 = ax1.twinx()\n ax2.set_ylabel('Count of Periods', color='tab:orange')\n ax2.plot(daily_stats.index, daily_stats['count_periods'], \n color='tab:orange', marker='s')\n ax2.tick_params(axis='y', labelcolor='tab:orange')\n \n plt.title(f'Daily {activity_type.capitalize()} Activity')\n fig.tight_layout()\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:45.918741Z","iopub.execute_input":"2024-09-28T18:09:45.919369Z","iopub.status.idle":"2024-09-28T18:09:47.018771Z","shell.execute_reply.started":"2024-09-28T18:09:45.919298Z","shell.execute_reply":"2024-09-28T18:09:47.017568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transitions_stats = activity_periods.groupby('day').agg(\n count_transitions=('transition_num', 'max')\n)\n\nfig, ax = plt.subplots(figsize=(10, 5))\nax.set_xlabel('Day')\nax.set_ylabel('Count of Transitions', color='tab:green')\nax.plot(transitions_stats.index, transitions_stats['count_transitions'], \n color='tab:green', marker='^')\nax.tick_params(axis='y', labelcolor='tab:green')\n\nplt.title('Daily Activity Transitions (All Types)')\nfig.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:47.020324Z","iopub.execute_input":"2024-09-28T18:09:47.020752Z","iopub.status.idle":"2024-09-28T18:09:47.314703Z","shell.execute_reply.started":"2024-09-28T18:09:47.020705Z","shell.execute_reply":"2024-09-28T18:09:47.313429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# - Ideas for feature engeneering","metadata":{}},{"cell_type":"markdown","source":"Again, we are dealing with similar measures - periods of activity, so we can make similar features:\n\nAggregated (mean, median, max, std):\n- Total duration of different of activity per day\n- Number of different activity boots per day","metadata":{}},{"cell_type":"code","source":"activity_summary = {}\n\nfor act_type in ['low', 'moderate']:\n activity_data = activity_periods[activity_periods['activity_type'] == act_type]\n \n stats = activity_data.groupby('day').agg(\n total_duration=('duration_sec', 'sum'),\n count_periods=('duration_sec', 'size')\n ).agg(['median', 'max', 'std'])\n \n for stat in ['median', 'max', 'std']:\n activity_summary[f'{act_type}_duration_{stat}'] = stats.loc[stat, 'total_duration']\n activity_summary[f'{act_type}_count_periods_{stat}'] = stats.loc[stat, 'count_periods']\n\ndaily_transitions = activity_periods.groupby('day')['transition_num'].max()\n\ntrans_stats = daily_transitions.agg(['median', 'max', 'std'])\nfor stat in ['median', 'max', 'std']:\n activity_summary[f'transitions_{stat}'] = trans_stats[stat]\n\nfeatures = pd.DataFrame([activity_summary])\nfeatures","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:47.316110Z","iopub.execute_input":"2024-09-28T18:09:47.316528Z","iopub.status.idle":"2024-09-28T18:09:47.363361Z","shell.execute_reply.started":"2024-09-28T18:09:47.316486Z","shell.execute_reply":"2024-09-28T18:09:47.362277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Activity and Light Exposure","metadata":{}},{"cell_type":"markdown","source":"Investigate the correlation between ambient light levels and activity levels by looking at how ENMO varies with light exposure.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10, 5))\nplt.scatter(worn_data['light'], worn_data['enmo'], alpha=0.5)\nplt.xlabel('Light (Lux)')\nplt.ylabel('ENMO (Activity)')\nplt.title('ENMO vs Light Exposure')\nplt.show()\n\ncorrelation_light_enmo = worn_data[['light', 'enmo']].corr().iloc[0, 1]\nprint(f\"Correlation between Light and ENMO: {correlation_light_enmo}\")","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:47.364814Z","iopub.execute_input":"2024-09-28T18:09:47.365245Z","iopub.status.idle":"2024-09-28T18:09:48.408601Z","shell.execute_reply.started":"2024-09-28T18:09:47.365205Z","shell.execute_reply":"2024-09-28T18:09:48.407324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"# Seasonal & Week-based Trends","metadata":{}},{"cell_type":"markdown","source":"Look at how the activity levels change over time. We start with changes by weekday:","metadata":{}},{"cell_type":"code","source":"daily_activity = worn_data.groupby('weekday')['enmo'].agg(['mean', 'max', 'std', 'count'])\n\nplt.figure(figsize=(16, 5))\n\nplt.subplot(1, 3, 1)\nplt.plot(daily_activity.index, daily_activity['mean'], marker='o')\nplt.title('Mean ENMO')\nplt.xlabel('Week Day')\nplt.ylabel('Mean ENMO')\nplt.grid(True)\n\nplt.subplot(1, 3, 2)\nplt.plot(daily_activity.index, daily_activity['max'], marker='o', color='orange')\nplt.title('Max ENMO')\nplt.xlabel('Week Day')\nplt.ylabel('Max ENMO')\nplt.grid(True)\n\nplt.subplot(1, 3, 3)\nplt.plot(daily_activity.index, daily_activity['std'], marker='o', color='green')\nplt.title('Std ENMO')\nplt.xlabel('Week Day')\nplt.ylabel('Std Dev ENMO')\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:48.410030Z","iopub.execute_input":"2024-09-28T18:09:48.410425Z","iopub.status.idle":"2024-09-28T18:09:49.195650Z","shell.execute_reply.started":"2024-09-28T18:09:48.410382Z","shell.execute_reply":"2024-09-28T18:09:49.194500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"daily_activity = daily_activity.reset_index()\ndaily_activity.columns = ['Weekday', 'Mean ENMO', 'Max ENMO', 'Std ENMO', 'Count']\ndaily_activity","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:49.197423Z","iopub.execute_input":"2024-09-28T18:09:49.198463Z","iopub.status.idle":"2024-09-28T18:09:49.214832Z","shell.execute_reply.started":"2024-09-28T18:09:49.198404Z","shell.execute_reply":"2024-09-28T18:09:49.213653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The mean ENMO by week:","metadata":{}},{"cell_type":"code","source":"worn_data['week'] = worn_data['relative_date_PCIAT'] // 7\nweekly_activity = worn_data.groupby('week')['enmo'].agg(['mean', 'max', 'std', 'count'])\n\nplt.figure(figsize=(16, 5))\n\nplt.subplot(1, 3, 1)\nplt.plot(weekly_activity.index, weekly_activity['mean'], marker='o')\nplt.title('Mean ENMO')\nplt.xlabel('Week')\nplt.ylabel('Mean ENMO')\nplt.grid(True)\n\nplt.subplot(1, 3, 2)\nplt.plot(weekly_activity.index, weekly_activity['max'], marker='o', color='orange')\nplt.title('Max ENMO')\nplt.xlabel('Week')\nplt.ylabel('Max ENMO')\nplt.grid(True)\n\nplt.subplot(1, 3, 3)\nplt.plot(weekly_activity.index, weekly_activity['std'], marker='o', color='green')\nplt.title('Std ENMO')\nplt.xlabel('Week')\nplt.ylabel('Std Dev ENMO')\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:49.216507Z","iopub.execute_input":"2024-09-28T18:09:49.216874Z","iopub.status.idle":"2024-09-28T18:09:50.301107Z","shell.execute_reply.started":"2024-09-28T18:09:49.216836Z","shell.execute_reply":"2024-09-28T18:09:50.299932Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"weekly_activity = weekly_activity.reset_index()\nweekly_activity.columns = ['Week', 'Mean ENMO', 'Max ENMO', 'Std ENMO', 'Count']\nweekly_activity","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:50.302721Z","iopub.execute_input":"2024-09-28T18:09:50.303179Z","iopub.status.idle":"2024-09-28T18:09:50.320580Z","shell.execute_reply.started":"2024-09-28T18:09:50.303127Z","shell.execute_reply":"2024-09-28T18:09:50.319251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"By season:","metadata":{}},{"cell_type":"code","source":"season_activity = worn_data.groupby('quarter')['enmo'].agg(['mean', 'max', 'std', 'count'])\nseason_activity = season_activity.reset_index()\nseason_activity.columns = ['Season', 'Mean ENMO', 'Max ENMO', 'Std ENMO', 'Count']\nseason_activity","metadata":{"execution":{"iopub.status.busy":"2024-09-28T18:09:50.322008Z","iopub.execute_input":"2024-09-28T18:09:50.322397Z","iopub.status.idle":"2024-09-28T18:09:50.350706Z","shell.execute_reply.started":"2024-09-28T18:09:50.322359Z","shell.execute_reply":"2024-09-28T18:09:50.349686Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
\n\n\n","metadata":{}},{"cell_type":"markdown","source":"
\n \"Flower\"\n \"Flower\"\n \"Flower\"\n
","metadata":{}}]}