{"cells":[{"metadata":{"_uuid":"f63f5a37e117689904ecdee908c7e6622ac754e1"},"cell_type":"markdown","source":"**How I think we should approach this problem**\n\nI think I have a little bit of domain knowledge that I am going to share with you all. I think it might be useful and/or at least gives you some ideas on how to approach this problem. Despite my will, I won't have enough time to go into the model architecture of this challenge but I enjoy stepping in and see how I would have approached this challenge if I had enough time to participate. Now, let's get started!\n\n**Point 1:**\nMy guess is that the *acoustic_data* in dataset is the recorded ground acceleration at equal time intervals. It could also be the recorded ground displacement or velocity. However, referring to the plot below, the failure (the point where the orange line approaches zero and then starts over) occurs shortly after the peak in acoustic_data. That is why I assume it must be the acceleration so that it causes displacements large enough to be considered as a failure, with a short delay. \n\n**Point 2:**\nIt took about 6 million datapoints to get a failure. Each of test csv files contains 150,000 datapoints. Thus, we should be able to get about 6000/150=40 samples (subsets similar to our test dataset for training our model) before the first failure in our training set. Their corresponding labels should be the time_to_failure at the last datapoint in each subset. \n\n**Point 3:**\nA quick view over the dataset shows that failures occur after the absolute value of ground acceleration passes a certain value (Exact value needs to be identified). A better approach to what I saw in a few other kernels [using statistical measures of the input (such as mean, std, etc.) and using a binary outcome (wheter it failed or not)] might be to try to predict the continuous variable of accelerations in the future, based on the datapoints we have in our subset; something like the stock prices. Then predict what time is the shortest time we pass the absolute threshold acceleration in the future? that would be our outcome. If you are taking this approach, my suggestion is to use Recurrent Neural Networks (if you are not familiar with it, this video https://www.youtube.com/watch?v=CznICCPa63Q should give you a brief introduction).\n\nI will try to update this kernel if I found something else that was interesting enough to share. Let me know what u think about these points I raised.\n\nBest of luck to all of you,\nMassoud."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\ntrain_data = pd.read_csv('../input/train.csv', nrows= 6_000_000, dtype={'acoustic_data': np.int16, 'time_to_failure': np.float64})\ntrain_data.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"119557842ede09843b19b2c13dbff9d1cd2b7d86"},"cell_type":"code","source":"plt.figure()\nplt.plot(train_data['acoustic_data']/train_data.acoustic_data.max())\nplt.plot(train_data['time_to_failure']/train_data.time_to_failure.max())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1c8140be741636d239691032879f5cb72228b503"},"cell_type":"code","source":"print(train_data.time_to_failure.min(), train_data.time_to_failure.max())\nprint(train_data.acoustic_data.min(), train_data.acoustic_data.max())\nminimum_time_to_failure_index = (train_data.time_to_failure <= 0.0007) # 5656573 \nprint(sum(minimum_time_to_failure_index))\nprint(train_data.loc[5656573:5656573+1, :])\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"636817da302370147ea530aa0f376e54aa6f5329"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}