I see in the public part of the test dataset that time-series are separated in two folders (defog and tdcsfog).
\nI also see that those two type of time-series have different sampling (128 vs 100 Hz) and units (g vs m/s2).
Would it be best to downsample, convert units and have a single model or keep the two time-series types separated with separated models?
", "messages": [ { "id": "2193526", "postDate": "03/23/2023 10:58:28", "content": "I see in the public part of the test dataset that time-series are separated in two folders (defog and tdcsfog).
\nI also see that those two type of time-series have different sampling (128 vs 100 Hz) and units (g vs m/s2).
Would it be best to downsample, convert units and have a single model or keep the two time-series types separated with separated models?
", "rawMarkdown": "I see in the public part of the test dataset that time-series are separated in two folders (defog and tdcsfog).\nI also see that those two type of time-series have different sampling (128 vs 100 Hz) and units (g vs m/s2).\n\nWould it be best to downsample, convert units and have a single model or keep the two time-series types separated with separated models?", "votes": null }, { "id": "2193787", "postDate": "03/23/2023 13:35:32", "content": "I would suggest building two separate models since the distribution of the data in both these samples are quite different. EDA analysis will confirm this. The only hassle is training! It will take some time to properly train two models but for performance and as a good practice you should ideally build two separate models.
", "rawMarkdown": "I would suggest building two separate models since the distribution of the data in both these samples are quite different. EDA analysis will confirm this. The only hassle is training! It will take some time to properly train two models but for performance and as a good practice you should ideally build two separate models.", "votes": null }, { "id": "2194907", "postDate": "03/24/2023 08:56:39", "content": "I think resampling 100 Hz data to 128 Hz is better. Converting accelerometer data to m/s2 is also a good option. If you want to calculate displacements as a feature you have to use m/s2 as unit.
", "rawMarkdown": "I think resampling 100 Hz data to 128 Hz is better. Converting accelerometer data to m/s2 is also a good option. If you want to calculate displacements as a feature you have to use m/s2 as unit.", "votes": null }, { "id": "2201668", "postDate": "03/29/2023 13:29:01", "content": "They've come from two different types of accelerometers so they'll have some similarity but will be different, I'd reccomend training on one and then finetuning a new model for the other. Less training that way hopefully!
", "rawMarkdown": "They've come from two different types of accelerometers so they'll have some similarity but will be different, I'd reccomend training on one and then finetuning a new model for the other. Less training that way hopefully!", "votes": null }, { "id": "2202863", "postDate": "03/30/2023 11:27:02", "content": "why would you prefer upsampling to 128Hz than downsampling to 100Hz?
", "rawMarkdown": "why would you prefer upsampling to 128Hz than downsampling to 100Hz?", "votes": null }, { "id": "2203799", "postDate": "03/31/2023 06:18:17", "content": "if you downsample from 128Hz to 100Hz, you loose 28 data points. Right? Which is not good for ML applications.
", "rawMarkdown": "if you downsample from 128Hz to 100Hz, you loose 28 data points. Right? Which is not good for ML applications.", "votes": null }, { "id": "2206797", "postDate": "04/02/2023 21:08:27", "content": "Hi @tayfungrlevik -- I'm a little late seeing this discussion but thought I'd mention a couple of possible issues with using displacement as a feature. The coordinate system we use [let's say (x,y,z) = (ML, AP, Vert)], rotates if the person turns. If we imagine a \"fixed\" reference frame (x',y',z') aligning with (x,y,z) at t=0, x and x', e.g., will not necessarily be the same over time as the person could turn.
\nSo if you integrate the acceleration data in order to find the velocity in (say) the AP direction, but then the person turns their body to the left but continues accelerating in the y' direction, the integrated accelerometer data would find that the person begins to move in the x direction once they turn, but would not have the previous accelerations contributing to that velocity. It would then return an incorrect displacement.
\nWe'd need gyroscopes to solve this problem, I think.
\nIn any event, integrating twice will likely produce some fairly significant errors.
", "rawMarkdown": "Hi @tayfungrlevik -- I'm a little late seeing this discussion but thought I'd mention a couple of possible issues with using displacement as a feature. The coordinate system we use [let's say (x,y,z) = (ML, AP, Vert)], rotates if the person turns. If we imagine a \"fixed\" reference frame (x',y',z') aligning with (x,y,z) at t=0, x and x', e.g., will not necessarily be the same over time as the person could turn. \n\nSo if you integrate the acceleration data in order to find the velocity in (say) the AP direction, but then the person turns their body to the left but continues accelerating in the y' direction, the integrated accelerometer data would find that the person begins to move in the x direction once they turn, but would not have the previous accelerations contributing to that velocity. It would then return an incorrect displacement.\n\nWe'd need gyroscopes to solve this problem, I think. \n\nIn any event, integrating twice will likely produce some fairly significant errors.", "votes": null }, { "id": "2211607", "postDate": "04/06/2023 06:17:36", "content": "Yes, you are right. To solve that problem I am trying to train another model that predicts angular velocities from acceleration data. To do that you can use you mobile phone imu and gyro. there are some apps like phyphox or matlab mobile that reads phones sensor measurments. If you can predict angular velocity in x,y,z directions then you can calculate rotation matrices and linear accelerations without gravity.
", "rawMarkdown": "Yes, you are right. To solve that problem I am trying to train another model that predicts angular velocities from acceleration data. To do that you can use you mobile phone imu and gyro. there are some apps like phyphox or matlab mobile that reads phones sensor measurments. If you can predict angular velocity in x,y,z directions then you can calculate rotation matrices and linear accelerations without gravity.", "votes": null } ], "comments": [ { "id": 2193787, "author_name": "rishabsinghh", "author_url": "", "post_date": "03/23/2023 13:35:32", "content": "I would suggest building two separate models since the distribution of the data in both these samples are quite different. EDA analysis will confirm this. The only hassle is training! It will take some time to properly train two models but for performance and as a good practice you should ideally build two separate models.
", "votes": null, "replies": [] }, { "id": 2194907, "author_name": "tayfungrlevik", "author_url": "", "post_date": "03/24/2023 08:56:39", "content": "I think resampling 100 Hz data to 128 Hz is better. Converting accelerometer data to m/s2 is also a good option. If you want to calculate displacements as a feature you have to use m/s2 as unit.
", "votes": null, "replies": [ { "id": 2202863, "author_name": "albertoannoni", "author_url": "", "post_date": "03/30/2023 11:27:02", "content": "why would you prefer upsampling to 128Hz than downsampling to 100Hz?
", "votes": null, "replies": [ { "id": 2203799, "author_name": "tayfungrlevik", "author_url": "", "post_date": "03/31/2023 06:18:17", "content": "if you downsample from 128Hz to 100Hz, you loose 28 data points. Right? Which is not good for ML applications.
", "votes": null, "replies": [] } ] }, { "id": 2206797, "author_name": "austinhinkel", "author_url": "", "post_date": "04/02/2023 21:08:27", "content": "Hi @tayfungrlevik -- I'm a little late seeing this discussion but thought I'd mention a couple of possible issues with using displacement as a feature. The coordinate system we use [let's say (x,y,z) = (ML, AP, Vert)], rotates if the person turns. If we imagine a \"fixed\" reference frame (x',y',z') aligning with (x,y,z) at t=0, x and x', e.g., will not necessarily be the same over time as the person could turn.
\nSo if you integrate the acceleration data in order to find the velocity in (say) the AP direction, but then the person turns their body to the left but continues accelerating in the y' direction, the integrated accelerometer data would find that the person begins to move in the x direction once they turn, but would not have the previous accelerations contributing to that velocity. It would then return an incorrect displacement.
\nWe'd need gyroscopes to solve this problem, I think.
\nIn any event, integrating twice will likely produce some fairly significant errors.
", "votes": null, "replies": [ { "id": 2211607, "author_name": "tayfungrlevik", "author_url": "", "post_date": "04/06/2023 06:17:36", "content": "Yes, you are right. To solve that problem I am trying to train another model that predicts angular velocities from acceleration data. To do that you can use you mobile phone imu and gyro. there are some apps like phyphox or matlab mobile that reads phones sensor measurments. If you can predict angular velocity in x,y,z directions then you can calculate rotation matrices and linear accelerations without gravity.
", "votes": null, "replies": [] } ] } ] }, { "id": 2201668, "author_name": "researn", "author_url": "", "post_date": "03/29/2023 13:29:01", "content": "They've come from two different types of accelerometers so they'll have some similarity but will be different, I'd reccomend training on one and then finetuning a new model for the other. Less training that way hopefully!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2193526": "I see in the public part of the test dataset that time-series are separated in two folders (defog and tdcsfog).\nI also see that those two type of time-series have different sampling (128 vs 100 Hz) and units (g vs m/s2).\n\nWould it be best to downsample, convert units and have a single model or keep the two time-series types separated with separated models?", "2193787": "I would suggest building two separate models since the distribution of the data in both these samples are quite different. EDA analysis will confirm this. The only hassle is training! It will take some time to properly train two models but for performance and as a good practice you should ideally build two separate models.", "2194907": "I think resampling 100 Hz data to 128 Hz is better. Converting accelerometer data to m/s2 is also a good option. If you want to calculate displacements as a feature you have to use m/s2 as unit.", "2201668": "They've come from two different types of accelerometers so they'll have some similarity but will be different, I'd reccomend training on one and then finetuning a new model for the other. Less training that way hopefully!", "2202863": "why would you prefer upsampling to 128Hz than downsampling to 100Hz?", "2203799": "if you downsample from 128Hz to 100Hz, you loose 28 data points. Right? Which is not good for ML applications.", "2206797": "Hi @tayfungrlevik -- I'm a little late seeing this discussion but thought I'd mention a couple of possible issues with using displacement as a feature. The coordinate system we use [let's say (x,y,z) = (ML, AP, Vert)], rotates if the person turns. If we imagine a \"fixed\" reference frame (x',y',z') aligning with (x,y,z) at t=0, x and x', e.g., will not necessarily be the same over time as the person could turn. \n\nSo if you integrate the acceleration data in order to find the velocity in (say) the AP direction, but then the person turns their body to the left but continues accelerating in the y' direction, the integrated accelerometer data would find that the person begins to move in the x direction once they turn, but would not have the previous accelerations contributing to that velocity. It would then return an incorrect displacement.\n\nWe'd need gyroscopes to solve this problem, I think. \n\nIn any event, integrating twice will likely produce some fairly significant errors.", "2211607": "Yes, you are right. To solve that problem I am trying to train another model that predicts angular velocities from acceleration data. To do that you can use you mobile phone imu and gyro. there are some apps like phyphox or matlab mobile that reads phones sensor measurments. If you can predict angular velocity in x,y,z directions then you can calculate rotation matrices and linear accelerations without gravity." }, "source": "meta" }