What is the sample rate for this data? I tried finding the difference between the time_to_failure of subsequent data points in the training dataset and I got approx. 1e-9 \"time units\" as the time between datapoints. There's no way these units can be seconds, right? That would mean that the training dataset of ~700M datapoints is only a second in duration. Is there any material published regarding this? Is time_to_failure somehow normalized?
", "messages": [ { "id": "455018", "postDate": "01/12/2019 18:58:21", "content": "What is the sample rate for this data? I tried finding the difference between the time_to_failure of subsequent data points in the training dataset and I got approx. 1e-9 \"time units\" as the time between datapoints. There's no way these units can be seconds, right? That would mean that the training dataset of ~700M datapoints is only a second in duration. Is there any material published regarding this? Is time_to_failure somehow normalized?
", "rawMarkdown": "What is the sample rate for this data? I tried finding the difference between the time_to_failure of subsequent data points in the training dataset and I got approx. 1e-9 \"time units\" as the time between datapoints. There's no way these units can be seconds, right? That would mean that the training dataset of ~700M datapoints is only a second in duration. Is there any material published regarding this? Is time_to_failure somehow normalized?", "votes": null }, { "id": "455117", "postDate": "01/13/2019 02:47:56", "content": "I think sampling rate is about 4MHz (250 ns between observations). The resolution of timer is less than sampling rate - therefore time_to_failure stays constant for about 4000 observations (resolution is 1e-3 sec). There are ~630M (629'145'480) observations covering about 157 seconds.
", "rawMarkdown": "I think sampling rate is about 4MHz (250 ns between observations). The resolution of timer is less than sampling rate - therefore time_to_failure stays constant for about 4000 observations (resolution is 1e-3 sec). There are ~630M (629'145'480) observations covering about 157 seconds.", "votes": null }, { "id": "455866", "postDate": "01/14/2019 18:01:37", "content": "I don't understand. Signal sampled this way can pass frequencies between 0 and ~2MHz. Aren't seismic waves between, I don't know, 0-200Hz? Does anyone have any reference?
", "rawMarkdown": "I don't understand. Signal sampled this way can pass frequencies between 0 and ~2MHz. Aren't seismic waves between, I don't know, 0-200Hz? Does anyone have any reference?", "votes": null }, { "id": "455879", "postDate": "01/14/2019 19:01:58", "content": "Edit: Sorry; I just deleted my original reply as I realized my math was all wrong (was working on the wrong array). I think magicsany had a good estimate of 4MHz, but I agree that seems a bit high for acoustic/seismic data. I'll do my best to add more insight from my side if anything magically resolves itself.\nBest of luck.
", "rawMarkdown": "Edit: Sorry; I just deleted my original reply as I realized my math was all wrong (was working on the wrong array). I think magicsany had a good estimate of 4MHz, but I agree that seems a bit high for acoustic/seismic data. I'll do my best to add more insight from my side if anything magically resolves itself.\nBest of luck.", "votes": null }, { "id": "455925", "postDate": "01/14/2019 20:42:26", "content": "I got 3.85 MHz when I looked at the data (just count number of samples in one earthquake and use the maximum time-to-failure to compute sample rate). This is indeed extremely high for seismic data. However, this is a lab experiment, carried out on a table-top scale, related by a scaling model to the real thing, and the frequencies are much higher.
", "rawMarkdown": "I got 3.85 MHz when I looked at the data (just count number of samples in one earthquake and use the maximum time-to-failure to compute sample rate). This is indeed extremely high for seismic data. However, this is a lab experiment, carried out on a table-top scale, related by a scaling model to the real thing, and the frequencies are much higher.", "votes": null }, { "id": "456219", "postDate": "01/15/2019 11:24:55", "content": "The clarification including sampling rate value is here: https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/77526
", "rawMarkdown": "The clarification including sampling rate value is here: https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/77526", "votes": null }, { "id": "456235", "postDate": "01/15/2019 11:51:58", "content": "Except it doesn't really answer the question why \"time_to_failure\" values don't follow 4MHz pattern.
\n\nMy current observations show cycles in training data of 25 chunks 4096 observations each (102400 observations) with total duration of 0.0266(seconds?) (9 * 0.001 + 16 * 0.0011). Which gives 3.849624 MHz rate for each of 6144 cycles.
", "rawMarkdown": "Except it doesn't really answer the question why \"time_to_failure\" values don't follow 4MHz pattern.\n\nMy current observations show cycles in training data of 25 chunks 4096 observations each (102400 observations) with total duration of 0.0266(seconds?) (9 * 0.001 + 16 * 0.0011). Which gives 3.849624 MHz rate for each of 6144 cycles.", "votes": null }, { "id": "456354", "postDate": "01/15/2019 16:53:25", "content": "The sampling time in the train.csv file appears to be 1.1e-9 s, which would be a sampling rate of about 909 MHz. How does this correspond to the sampling rate of 4 MHz in the test files? Are there different time scales being used? Please clarify.
", "rawMarkdown": "The sampling time in the train.csv file appears to be 1.1e-9 s, which would be a sampling rate of about 909 MHz. How does this correspond to the sampling rate of 4 MHz in the test files? Are there different time scales being used? Please clarify.", "votes": null }, { "id": "456497", "postDate": "01/15/2019 23:34:10", "content": "Sounds to me that time_to_failure can only be acurate up to ~4000 samples. There's basically a time uncertainty that you cannot get around because your measurements are physically constrained by the sampling rate of the experiment. No?
", "rawMarkdown": "Sounds to me that time_to_failure can only be acurate up to ~4000 samples. There's basically a time uncertainty that you cannot get around because your measurements are physically constrained by the sampling rate of the experiment. No?", "votes": null }, { "id": "456580", "postDate": "01/16/2019 05:17:53", "content": "My guess is the acoustic sampling rate is 4MHz, but the timeoffailure sample rate is 1000Hz. Need to be verified by the author.
No, the time to failure steps decrease by 1.1e-9 from data point to data point. They are not constant. One needs to look at the full precision of the numbers, then one can see it. This sampling rate does not fit the 4 MHz in the test data set. The question is still unanswered.
", "rawMarkdown": "No, the time to failure steps decrease by 1.1e-9 from data point to data point. They are not constant. One needs to look at the full precision of the numbers, then one can see it. This sampling rate does not fit the 4 MHz in the test data set. The question is still unanswered.", "votes": null } ], "comments": [ { "id": 455117, "author_name": "magicsany", "author_url": "", "post_date": "01/13/2019 02:47:56", "content": "I think sampling rate is about 4MHz (250 ns between observations). The resolution of timer is less than sampling rate - therefore time_to_failure stays constant for about 4000 observations (resolution is 1e-3 sec). There are ~630M (629'145'480) observations covering about 157 seconds.
", "votes": null, "replies": [] }, { "id": 455866, "author_name": "davids1992", "author_url": "", "post_date": "01/14/2019 18:01:37", "content": "I don't understand. Signal sampled this way can pass frequencies between 0 and ~2MHz. Aren't seismic waves between, I don't know, 0-200Hz? Does anyone have any reference?
", "votes": null, "replies": [ { "id": 455879, "author_name": "pgilley", "author_url": "", "post_date": "01/14/2019 19:01:58", "content": "Edit: Sorry; I just deleted my original reply as I realized my math was all wrong (was working on the wrong array). I think magicsany had a good estimate of 4MHz, but I agree that seems a bit high for acoustic/seismic data. I'll do my best to add more insight from my side if anything magically resolves itself.\nBest of luck.
", "votes": null, "replies": [] }, { "id": 455925, "author_name": "petewills", "author_url": "", "post_date": "01/14/2019 20:42:26", "content": "I got 3.85 MHz when I looked at the data (just count number of samples in one earthquake and use the maximum time-to-failure to compute sample rate). This is indeed extremely high for seismic data. However, this is a lab experiment, carried out on a table-top scale, related by a scaling model to the real thing, and the frequencies are much higher.
", "votes": null, "replies": [] }, { "id": 456219, "author_name": "davids1992", "author_url": "", "post_date": "01/15/2019 11:24:55", "content": "The clarification including sampling rate value is here: https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/77526
", "votes": null, "replies": [] }, { "id": 456235, "author_name": "elvenmonk", "author_url": "", "post_date": "01/15/2019 11:51:58", "content": "Except it doesn't really answer the question why \"time_to_failure\" values don't follow 4MHz pattern.
\n\nMy current observations show cycles in training data of 25 chunks 4096 observations each (102400 observations) with total duration of 0.0266(seconds?) (9 * 0.001 + 16 * 0.0011). Which gives 3.849624 MHz rate for each of 6144 cycles.
", "votes": null, "replies": [] }, { "id": 456497, "author_name": "horizonpicking2k18", "author_url": "", "post_date": "01/15/2019 23:34:10", "content": "Sounds to me that time_to_failure can only be acurate up to ~4000 samples. There's basically a time uncertainty that you cannot get around because your measurements are physically constrained by the sampling rate of the experiment. No?
", "votes": null, "replies": [] }, { "id": 456901, "author_name": "henningvoss", "author_url": "", "post_date": "01/16/2019 18:10:16", "content": "No, the time to failure steps decrease by 1.1e-9 from data point to data point. They are not constant. One needs to look at the full precision of the numbers, then one can see it. This sampling rate does not fit the 4 MHz in the test data set. The question is still unanswered.
", "votes": null, "replies": [] } ] }, { "id": 456354, "author_name": "henningvoss", "author_url": "", "post_date": "01/15/2019 16:53:25", "content": "The sampling time in the train.csv file appears to be 1.1e-9 s, which would be a sampling rate of about 909 MHz. How does this correspond to the sampling rate of 4 MHz in the test files? Are there different time scales being used? Please clarify.
", "votes": null, "replies": [] }, { "id": 456580, "author_name": "", "author_url": "", "post_date": "01/16/2019 05:17:53", "content": "My guess is the acoustic sampling rate is 4MHz, but the timeoffailure sample rate is 1000Hz. Need to be verified by the author.