I was checking the stability of the prediciton of my model over the whole training set, and I noticed a bheaviour that on one side let me confident on the model, but on the other side rises some doubts about the earthquake events detection.
\n\nIn the graph, the prediction of my model are showed (in orange) over the entire training every chunks of 150k samples , together the time to event (in green) and the magnitude of the signal (in blue).
\n\nAs it is possible to see, there are spikes in the signal coming from the piezoceramic sensor that do not correspond to an event as detected by the shear stress sensor.
\n\nAnyway, the model seems able to predict (with variance) these spikes better than the time to event, altought trained only with the absolute mean time error.
\n\nIndeed the prediction time reaches a minimum just before the spikes, recovering afterawards (and in line with a prediction of the next spike) also if the earthquake event is not recorded.
\n\nPlease note that each prediction have no memory of previous chunks, so it seem that the conditions detectable by the audio signal \"relax\" after spikes also if undetected by the shear stress sensor.
\n\nMaybe the time to \"spike\"condition is more predictable than the time to shear stress event detection?
\n\nThere are similar evidences on other models?
\n\nI will investigate this on a modified dataset, where time to predict will be related to next spike and not to next event.
", "messages": [ { "id": "475311", "postDate": "02/20/2019 15:51:51", "content": "I was checking the stability of the prediciton of my model over the whole training set, and I noticed a bheaviour that on one side let me confident on the model, but on the other side rises some doubts about the earthquake events detection.
\n\nIn the graph, the prediction of my model are showed (in orange) over the entire training every chunks of 150k samples , together the time to event (in green) and the magnitude of the signal (in blue).
\n\nAs it is possible to see, there are spikes in the signal coming from the piezoceramic sensor that do not correspond to an event as detected by the shear stress sensor.
\n\nAnyway, the model seems able to predict (with variance) these spikes better than the time to event, altought trained only with the absolute mean time error.
\n\nIndeed the prediction time reaches a minimum just before the spikes, recovering afterawards (and in line with a prediction of the next spike) also if the earthquake event is not recorded.
\n\nPlease note that each prediction have no memory of previous chunks, so it seem that the conditions detectable by the audio signal \"relax\" after spikes also if undetected by the shear stress sensor.
\n\nMaybe the time to \"spike\"condition is more predictable than the time to shear stress event detection?
\n\nThere are similar evidences on other models?
\n\nI will investigate this on a modified dataset, where time to predict will be related to next spike and not to next event.
", "rawMarkdown": "I was checking the stability of the prediciton of my model over the whole training set, and I noticed a bheaviour that on one side let me confident on the model, but on the other side rises some doubts about the earthquake events detection.\n\n![Graph of prediction over the entire training set][1]\n\nIn the graph, the prediction of my model are showed (in orange) over the entire training every chunks of 150k samples , together the time to event (in green) and the magnitude of the signal (in blue).\n\nAs it is possible to see, there are spikes in the signal coming from the piezoceramic sensor that do not correspond to an event as detected by the shear stress sensor.\n\nAnyway, the model seems able to predict (with variance) these spikes better than the time to event, altought trained only with the absolute mean time error.\n\nIndeed the prediction time reaches a minimum just before the spikes, recovering afterawards (and in line with a prediction of the next spike) also if the earthquake event is not recorded.\n\nPlease note that each prediction have no memory of previous chunks, so it seem that the conditions detectable by the audio signal \"*relax*\" after spikes also if undetected by the shear stress sensor.\n\nMaybe the time to \"spike\"condition is more predictable than the time to shear stress event detection?\n\nThere are similar evidences on other models?\n\nI will investigate this on a modified dataset, where time to predict will be related to next spike and not to next event.\n\n [1]: https://drive.google.com/uc?id=1UbCr2m1SBxNlnLm9Frd1zi2CRiVcaU-e", "votes": null }, { "id": "475471", "postDate": "02/20/2019 19:38:13", "content": "I also worried about this look at my genetic program kernel - you will see the exact same problem
", "rawMarkdown": "I also worried about this look at my genetic program kernel - you will see the exact same problem", "votes": null }, { "id": "476082", "postDate": "02/21/2019 15:34:56", "content": "I am also noticing a similar pattern in my predictions. One observation I noticed in both my model and yours is that with the start of each new earthquake, the predictions are always relatively the same, almost irrespective of how far away the earthquake actually is. Therefore it is interesting that after these spikes, the predictions recover to a quite good accuracy. Based on how the model behaves immediately after an earthquake, I would expect it to jump right back up to ~8 seconds after a significant spike, but instead it is in line with the true value. I don't have any real conclusions to offer, but it seems to me that the model is treating actual earthquakes a little bit differently than spikes. Maybe the data that is closer to each earthquake provides patterns which are more useful in precise prediction, whereas the beginnings of the earthquakes may be all very similar, leading to the model to make a very general guess at first.
", "rawMarkdown": "I am also noticing a similar pattern in my predictions. One observation I noticed in both my model and yours is that with the start of each new earthquake, the predictions are always relatively the same, almost irrespective of how far away the earthquake actually is. Therefore it is interesting that after these spikes, the predictions recover to a quite good accuracy. Based on how the model behaves immediately after an earthquake, I would expect it to jump right back up to ~8 seconds after a significant spike, but instead it is in line with the true value. I don't have any real conclusions to offer, but it seems to me that the model is treating actual earthquakes a little bit differently than spikes. Maybe the data that is closer to each earthquake provides patterns which are more useful in precise prediction, whereas the beginnings of the earthquakes may be all very similar, leading to the model to make a very general guess at first.", "votes": null } ], "comments": [ { "id": 475471, "author_name": "scirpus", "author_url": "", "post_date": "02/20/2019 19:38:13", "content": "I also worried about this look at my genetic program kernel - you will see the exact same problem
", "votes": null, "replies": [] }, { "id": 476082, "author_name": "terryrcarson", "author_url": "", "post_date": "02/21/2019 15:34:56", "content": "I am also noticing a similar pattern in my predictions. One observation I noticed in both my model and yours is that with the start of each new earthquake, the predictions are always relatively the same, almost irrespective of how far away the earthquake actually is. Therefore it is interesting that after these spikes, the predictions recover to a quite good accuracy. Based on how the model behaves immediately after an earthquake, I would expect it to jump right back up to ~8 seconds after a significant spike, but instead it is in line with the true value. I don't have any real conclusions to offer, but it seems to me that the model is treating actual earthquakes a little bit differently than spikes. Maybe the data that is closer to each earthquake provides patterns which are more useful in precise prediction, whereas the beginnings of the earthquakes may be all very similar, leading to the model to make a very general guess at first.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "475311": "I was checking the stability of the prediciton of my model over the whole training set, and I noticed a bheaviour that on one side let me confident on the model, but on the other side rises some doubts about the earthquake events detection.\n\n![Graph of prediction over the entire training set][1]\n\nIn the graph, the prediction of my model are showed (in orange) over the entire training every chunks of 150k samples , together the time to event (in green) and the magnitude of the signal (in blue).\n\nAs it is possible to see, there are spikes in the signal coming from the piezoceramic sensor that do not correspond to an event as detected by the shear stress sensor.\n\nAnyway, the model seems able to predict (with variance) these spikes better than the time to event, altought trained only with the absolute mean time error.\n\nIndeed the prediction time reaches a minimum just before the spikes, recovering afterawards (and in line with a prediction of the next spike) also if the earthquake event is not recorded.\n\nPlease note that each prediction have no memory of previous chunks, so it seem that the conditions detectable by the audio signal \"*relax*\" after spikes also if undetected by the shear stress sensor.\n\nMaybe the time to \"spike\"condition is more predictable than the time to shear stress event detection?\n\nThere are similar evidences on other models?\n\nI will investigate this on a modified dataset, where time to predict will be related to next spike and not to next event.\n\n [1]: https://drive.google.com/uc?id=1UbCr2m1SBxNlnLm9Frd1zi2CRiVcaU-e", "475471": "I also worried about this look at my genetic program kernel - you will see the exact same problem", "476082": "I am also noticing a similar pattern in my predictions. One observation I noticed in both my model and yours is that with the start of each new earthquake, the predictions are always relatively the same, almost irrespective of how far away the earthquake actually is. Therefore it is interesting that after these spikes, the predictions recover to a quite good accuracy. Based on how the model behaves immediately after an earthquake, I would expect it to jump right back up to ~8 seconds after a significant spike, but instead it is in line with the true value. I don't have any real conclusions to offer, but it seems to me that the model is treating actual earthquakes a little bit differently than spikes. Maybe the data that is closer to each earthquake provides patterns which are more useful in precise prediction, whereas the beginnings of the earthquakes may be all very similar, leading to the model to make a very general guess at first." }, "source": "meta" }