![\"\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/543866/13403/Screen%20Shot%202019-06-04%20at%203.32.39%20PM.jpg\")
I am just wondering maybe this is not a good machine learning problem. Everyone's private score is very close to 2.5. What is 2.5? The range of earthquake is close to 1 to 11. If it's evenly distributed, if we use 6 as prediction, the expected mae should be 2.5. If it has special distribution, like to following case. the mae can be even lower if you just use the mean 'time_to_failure' as the prediction.\n
I am just wondering maybe this is not a good machine learning problem. Everyone's private score is very close to 2.5. What is 2.5? The range of earthquake is close to 1 to 11. If it's evenly distributed, if we use 6 as prediction, the expected mae should be 2.5. If it has special distribution, like to following case. the mae can be even lower if you just use the mean 'time_to_failure' as the prediction.\n
This problem is not even learnable yet, not to mention \"machine\" learnable or not.
", "rawMarkdown": "This problem is not even learnable yet, not to mention \"machine\" learnable or not.", "votes": null }, { "id": "543941", "postDate": "06/05/2019 00:34:41", "content": "I made a similar calculation during the competition (the 2.5) but was discouraged by the LB entries getting 1.3 - it certainly seemed as they were learning something and I was thrilled to get below 2 lol. Now, not so much and your point is a good one.
\n\nI might rephrase slightly, though: while the basic problem seems hopeful for machine learning, the question might be: do the actual data contain any information that can be used to predict when the next earthquake might happen? After all the effort, I do not think that this question has been answered (at least by scores although feature correlations look compelling), no thanks to the \"complicated\" choice on data that the organizers made. It's a pity.
", "rawMarkdown": "I made a similar calculation during the competition (the 2.5) but was discouraged by the LB entries getting 1.3 - it certainly seemed as they were learning something and I was thrilled to get below 2 lol. Now, not so much and your point is a good one.\n\nI might rephrase slightly, though: while the basic problem seems hopeful for machine learning, the question might be: do the actual data contain any information that can be used to predict when the next earthquake might happen? After all the effort, I do not think that this question has been answered (at least by scores although feature correlations look compelling), no thanks to the \"complicated\" choice on data that the organizers made. It's a pity.", "votes": null }, { "id": "544049", "postDate": "06/05/2019 04:22:19", "content": "Yes, I think my question is exactly the same. Your description is more accurate. The actual data, and the test data probably don't contain enough information in order to predict the next earthquake.
", "rawMarkdown": "Yes, I think my question is exactly the same. Your description is more accurate. The actual data, and the test data probably don't contain enough information in order to predict the next earthquake.", "votes": null }, { "id": "547043", "postDate": "06/07/2019 07:33:18", "content": "I thought exactly the same as you did. My physics knowledge said that this cannot really work, as I explained in another thread: it is physically impossible to predict ttf for individual quake cycles. The signal (for example 1-3 sec after a major quake) is statistically identical for all cycles, no matter if they are 8 or 16 sec long or if there will be a minor quake or not. Otherwise, there would be a major revision necessary to the way we think about nonlinear dynamics today.
\n\nThe best one could do was to average over all cycles and improve the prediction uncertainty for an \"averaged cycle\". Those that did this best won. But maybe that's what the organizers wanted anyway. Plus the tiny chance for a revolutionary discovery :)
", "rawMarkdown": "I thought exactly the same as you did. My physics knowledge said that this cannot really work, as I explained in another thread: it is *physically impossible* to predict ttf for *individual* quake cycles. The signal (for example 1-3 sec after a major quake) is statistically identical for all cycles, no matter if they are 8 or 16 sec long or if there will be a minor quake or not. Otherwise, there would be a major revision necessary to the way we think about nonlinear dynamics today.\n\nThe best one could do was to average over all cycles and improve the prediction uncertainty for an \"averaged cycle\". Those that did this best won. But maybe that's what the organizers wanted anyway. Plus the tiny chance for a revolutionary discovery :)", "votes": null }, { "id": "547520", "postDate": "06/07/2019 20:22:10", "content": "I have the same idea. Even though we had access to the leaked data, the score was about 2.4\nIf a long test data including short and long EQ are given to us, the score will be even worse.\nThe EQ cannot be predicted 11 s before, given a small data length of 0.037 s\nHowever the probability function of TTF can be calculated. This probability function evolves in time. \n11 s before the EQ it has a flat shape. The more we approach the next EQ, the narrower the function becomes.
", "rawMarkdown": "I have the same idea. Even though we had access to the leaked data, the score was about 2.4\nIf a long test data including short and long EQ are given to us, the score will be even worse.\nThe EQ cannot be predicted 11 s before, given a small data length of 0.037 s\nHowever the probability function of TTF can be calculated. This probability function evolves in time. \n11 s before the EQ it has a flat shape. The more we approach the next EQ, the narrower the function becomes.", "votes": null }, { "id": "547557", "postDate": "06/07/2019 21:25:01", "content": "Great observation! If you haven't already, check out my recent kernel that uses one feature and no machine learning to get a score in the gold medal range.
", "rawMarkdown": "Great observation! If you haven't already, check out my recent [kernel](https://www.kaggle.com/trentb/one-feature-no-ml-gold-medal-range) that uses one feature and no machine learning to get a score in the gold medal range.", "votes": null }, { "id": "548180", "postDate": "06/08/2019 23:41:41", "content": "The reason why I posted this discussion is that I am actually doing research in seismology simulation. Based on the background knowledge, there is no significant frequency feature, let alone low frequency features (infrasound waves that are most important in earthquake). From a geophysics perspective. There is too many noises than information. The abnormal signal-to-noise ratio makes the input data pretty useless.
", "rawMarkdown": "The reason why I posted this discussion is that I am actually doing research in seismology simulation. Based on the background knowledge, there is no significant frequency feature, let alone low frequency features (infrasound waves that are most important in earthquake). From a geophysics perspective. There is too many noises than information. The abnormal signal-to-noise ratio makes the input data pretty useless.", "votes": null }, { "id": "548182", "postDate": "06/08/2019 23:44:38", "content": "Yes, your kernel confirmed my observation. Actually, I also tried to submit different solutions with the same prediction of all entries. It seems the only thing that matters is how close is the distribution of your submission to the test data, or more specifically the center of your data to the center of the test data.
", "rawMarkdown": "Yes, your kernel confirmed my observation. Actually, I also tried to submit different solutions with the same prediction of all entries. It seems the only thing that matters is how close is the distribution of your submission to the test data, or more specifically the center of your data to the center of the test data.", "votes": null }, { "id": "548844", "postDate": "06/10/2019 02:25:50", "content": "Nice kernel. I think that it demonstrates that no progress in earthquake science was made in this competition. However, whether there's no useful info in the data - I'm not quite there yet.
", "rawMarkdown": "Nice kernel. I think that it demonstrates that no progress in earthquake science was made in this competition. However, whether there's no useful info in the data - I'm not quite there yet.", "votes": null }, { "id": "550281", "postDate": "06/11/2019 13:13:07", "content": "I did correlations and models for each individual quake. Some quakes were well-behaved. I had a feature with a 0.9 correlation to TTF for about 80% of the quakes. Other quakes were random noise. No feature was greater than 0.5. The model for those quakes just predicted the average TTF.
\n\nI tried to figure out what separated well behaved from not. The \"fake quakes\" were definitely a problem, but that wasn't all of it.
", "rawMarkdown": "I did correlations and models for each individual quake. Some quakes were well-behaved. I had a feature with a 0.9 correlation to TTF for about 80% of the quakes. Other quakes were random noise. No feature was greater than 0.5. The model for those quakes just predicted the average TTF. \n\nI tried to figure out what separated well behaved from not. The \"fake quakes\" were definitely a problem, but that wasn't all of it.", "votes": null }, { "id": "551366", "postDate": "06/12/2019 16:22:12", "content": "that's exactly the key to win
", "rawMarkdown": "that's exactly the key to win", "votes": null }, { "id": "552723", "postDate": "06/14/2019 12:12:46", "content": "Thanks for your kernel!
", "rawMarkdown": "Thanks for your kernel!", "votes": null } ], "comments": [ { "id": 543906, "author_name": "scaomath", "author_url": "", "post_date": "06/04/2019 23:35:33", "content": "This problem is not even learnable yet, not to mention \"machine\" learnable or not.
", "votes": null, "replies": [] }, { "id": 543941, "author_name": "petewills", "author_url": "", "post_date": "06/05/2019 00:34:41", "content": "I made a similar calculation during the competition (the 2.5) but was discouraged by the LB entries getting 1.3 - it certainly seemed as they were learning something and I was thrilled to get below 2 lol. Now, not so much and your point is a good one.
\n\nI might rephrase slightly, though: while the basic problem seems hopeful for machine learning, the question might be: do the actual data contain any information that can be used to predict when the next earthquake might happen? After all the effort, I do not think that this question has been answered (at least by scores although feature correlations look compelling), no thanks to the \"complicated\" choice on data that the organizers made. It's a pity.
", "votes": null, "replies": [ { "id": 544049, "author_name": "yiminchern", "author_url": "", "post_date": "06/05/2019 04:22:19", "content": "Yes, I think my question is exactly the same. Your description is more accurate. The actual data, and the test data probably don't contain enough information in order to predict the next earthquake.
", "votes": null, "replies": [] }, { "id": 547043, "author_name": "friedchips", "author_url": "", "post_date": "06/07/2019 07:33:18", "content": "I thought exactly the same as you did. My physics knowledge said that this cannot really work, as I explained in another thread: it is physically impossible to predict ttf for individual quake cycles. The signal (for example 1-3 sec after a major quake) is statistically identical for all cycles, no matter if they are 8 or 16 sec long or if there will be a minor quake or not. Otherwise, there would be a major revision necessary to the way we think about nonlinear dynamics today.
\n\nThe best one could do was to average over all cycles and improve the prediction uncertainty for an \"averaged cycle\". Those that did this best won. But maybe that's what the organizers wanted anyway. Plus the tiny chance for a revolutionary discovery :)
", "votes": null, "replies": [] } ] }, { "id": 547520, "author_name": "molaee", "author_url": "", "post_date": "06/07/2019 20:22:10", "content": "I have the same idea. Even though we had access to the leaked data, the score was about 2.4\nIf a long test data including short and long EQ are given to us, the score will be even worse.\nThe EQ cannot be predicted 11 s before, given a small data length of 0.037 s\nHowever the probability function of TTF can be calculated. This probability function evolves in time. \n11 s before the EQ it has a flat shape. The more we approach the next EQ, the narrower the function becomes.
", "votes": null, "replies": [] }, { "id": 547557, "author_name": "trentb", "author_url": "", "post_date": "06/07/2019 21:25:01", "content": "Great observation! If you haven't already, check out my recent kernel that uses one feature and no machine learning to get a score in the gold medal range.
", "votes": null, "replies": [ { "id": 548182, "author_name": "yiminchern", "author_url": "", "post_date": "06/08/2019 23:44:38", "content": "Yes, your kernel confirmed my observation. Actually, I also tried to submit different solutions with the same prediction of all entries. It seems the only thing that matters is how close is the distribution of your submission to the test data, or more specifically the center of your data to the center of the test data.
", "votes": null, "replies": [] }, { "id": 548844, "author_name": "petewills", "author_url": "", "post_date": "06/10/2019 02:25:50", "content": "Nice kernel. I think that it demonstrates that no progress in earthquake science was made in this competition. However, whether there's no useful info in the data - I'm not quite there yet.
", "votes": null, "replies": [] }, { "id": 551366, "author_name": "asterisk", "author_url": "", "post_date": "06/12/2019 16:22:12", "content": "that's exactly the key to win
", "votes": null, "replies": [] }, { "id": 552723, "author_name": "", "author_url": "", "post_date": "06/14/2019 12:12:46", "content": "Thanks for your kernel!
", "votes": null, "replies": [] } ] }, { "id": 548180, "author_name": "yiminchern", "author_url": "", "post_date": "06/08/2019 23:41:41", "content": "The reason why I posted this discussion is that I am actually doing research in seismology simulation. Based on the background knowledge, there is no significant frequency feature, let alone low frequency features (infrasound waves that are most important in earthquake). From a geophysics perspective. There is too many noises than information. The abnormal signal-to-noise ratio makes the input data pretty useless.
", "votes": null, "replies": [] }, { "id": 550281, "author_name": "ericfreeman", "author_url": "", "post_date": "06/11/2019 13:13:07", "content": "I did correlations and models for each individual quake. Some quakes were well-behaved. I had a feature with a 0.9 correlation to TTF for about 80% of the quakes. Other quakes were random noise. No feature was greater than 0.5. The model for those quakes just predicted the average TTF.
\n\nI tried to figure out what separated well behaved from not. The \"fake quakes\" were definitely a problem, but that wasn't all of it.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "543866": "I am just wondering maybe this is not a good machine learning problem. Everyone's private score is very close to 2.5. What is 2.5? The range of earthquake is close to 1 to 11. If it's evenly distributed, if we use 6 as prediction, the expected mae should be 2.5. If it has special distribution, like to following case. the mae can be even lower if you just use the mean 'time_to_failure' as the prediction.\n", "543906": "This problem is not even learnable yet, not to mention \"machine\" learnable or not.", "543941": "I made a similar calculation during the competition (the 2.5) but was discouraged by the LB entries getting 1.3 - it certainly seemed as they were learning something and I was thrilled to get below 2 lol. Now, not so much and your point is a good one.\n\nI might rephrase slightly, though: while the basic problem seems hopeful for machine learning, the question might be: do the actual data contain any information that can be used to predict when the next earthquake might happen? After all the effort, I do not think that this question has been answered (at least by scores although feature correlations look compelling), no thanks to the \"complicated\" choice on data that the organizers made. It's a pity.", "544049": "Yes, I think my question is exactly the same. Your description is more accurate. The actual data, and the test data probably don't contain enough information in order to predict the next earthquake.", "547043": "I thought exactly the same as you did. My physics knowledge said that this cannot really work, as I explained in another thread: it is *physically impossible* to predict ttf for *individual* quake cycles. The signal (for example 1-3 sec after a major quake) is statistically identical for all cycles, no matter if they are 8 or 16 sec long or if there will be a minor quake or not. Otherwise, there would be a major revision necessary to the way we think about nonlinear dynamics today.\n\nThe best one could do was to average over all cycles and improve the prediction uncertainty for an \"averaged cycle\". Those that did this best won. But maybe that's what the organizers wanted anyway. Plus the tiny chance for a revolutionary discovery :)", "547520": "I have the same idea. Even though we had access to the leaked data, the score was about 2.4\nIf a long test data including short and long EQ are given to us, the score will be even worse.\nThe EQ cannot be predicted 11 s before, given a small data length of 0.037 s\nHowever the probability function of TTF can be calculated. This probability function evolves in time. \n11 s before the EQ it has a flat shape. The more we approach the next EQ, the narrower the function becomes.", "547557": "Great observation! If you haven't already, check out my recent [kernel](https://www.kaggle.com/trentb/one-feature-no-ml-gold-medal-range) that uses one feature and no machine learning to get a score in the gold medal range.", "548180": "The reason why I posted this discussion is that I am actually doing research in seismology simulation. Based on the background knowledge, there is no significant frequency feature, let alone low frequency features (infrasound waves that are most important in earthquake). From a geophysics perspective. There is too many noises than information. The abnormal signal-to-noise ratio makes the input data pretty useless.", "548182": "Yes, your kernel confirmed my observation. Actually, I also tried to submit different solutions with the same prediction of all entries. It seems the only thing that matters is how close is the distribution of your submission to the test data, or more specifically the center of your data to the center of the test data.", "548844": "Nice kernel. I think that it demonstrates that no progress in earthquake science was made in this competition. However, whether there's no useful info in the data - I'm not quite there yet.", "550281": "I did correlations and models for each individual quake. Some quakes were well-behaved. I had a feature with a 0.9 correlation to TTF for about 80% of the quakes. Other quakes were random noise. No feature was greater than 0.5. The model for those quakes just predicted the average TTF. \n\nI tried to figure out what separated well behaved from not. The \"fake quakes\" were definitely a problem, but that wasn't all of it.", "551366": "that's exactly the key to win", "552723": "Thanks for your kernel!" }, "source": "meta" }