{"cells":[{"metadata":{"trusted":true,"_uuid":"def804c89bbe5764429461fbd406fb6d40ecfc4f"},"cell_type":"markdown","source":"## In this kernel, I will show a way to extract the actual underlying trend in the voltage signal using the [seasonal](https://github.com/welch/seasonal) package (which uses scipy periodograms). Then, the noise in the time series can be obtained by subtracting the trend from the noisy time series."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"##### This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport pyarrow.parquet as pq\nimport matplotlib.pyplot as plt\nimport time\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"055db53b0545fd7928e14cd524a44c0a4395dcf5"},"cell_type":"markdown","source":"**Install seasonal**"},{"metadata":{"trusted":true,"_uuid":"ebd635a386af79641ed4dba50ab6612ab56c28d7"},"cell_type":"code","source":"!pip install --user seasonal\nfrom seasonal import fit_seasons, adjust_seasons","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cb9407d7075cbad38b5f1d8fb1c0c8f8ff614ebd"},"cell_type":"markdown","source":"**Load a small part of the training data**"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"praq = pq.read_pandas('../input/train.parquet', columns=[str(i) for i in range(1000)]).to_pandas()\nsignals = praq.T.values.astype(float)\nmetadata = pd.read_csv('../input/metadata_train.csv', nrows=1000)\ntargets = metadata['target']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"86f7d1a3fc3f4f8c0097528f16fde8fce1a79e92"},"cell_type":"markdown","source":"**Divide the data based on class**"},{"metadata":{"trusted":true,"_uuid":"bdc7218aab10d1e72599c6305d89769c430f1555"},"cell_type":"code","source":"pos_indices = []\nneg_indices = []\nfor i in range(len(targets)):\n    if targets[i] == 0:\n        neg_indices.append(i)\n    else:\n        pos_indices.append(i)\n\nindices = [index for index in range(signals.shape[1]) if index % 20 == 0]\nneg_signals = signals[neg_indices]\npos_signals = signals[pos_indices]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9d3be238ae5e59c484ce5c12207dae4f08abf600"},"cell_type":"markdown","source":"**Visualize the actual signals and noises for non-faulty cases**"},{"metadata":{"trusted":true,"_uuid":"9d1bbbb07c83b5705abb05e4bc41b116c17b5627"},"cell_type":"code","source":"s = time.time()\nfor i in range(50):\n    signal = neg_signals[i]\n    short_signal = signal[indices]\n    seasons, trend = fit_seasons(short_signal)\n    e = time.time()\n    print(\"SIGNAL SAMPLE {}\".format(i+1))\n    print(\"Total time : {}\".format(str(e - s) + \" s\"))\n    \n    color = 'g'\n    plt.plot(short_signal, color)\n    plt.show()\n    print(\"Trend\")\n    plt.plot(trend, color)\n    plt.show()\n    print(\"Noise\")\n    plt.plot(short_signal - trend, color)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"aaaf4861eab93be01f34fd9935e45049482a65a4"},"cell_type":"markdown","source":"**Visualize the actual signals and noises for faulty cases**"},{"metadata":{"trusted":true,"_uuid":"e28c4ff20ca8e0b354e285c1ab2356fe7b49478b"},"cell_type":"code","source":"s = time.time()\nfor i in range(50):\n    signal = pos_signals[i]\n    short_signal = signal[indices]\n    seasons, trend = fit_seasons(short_signal)\n    e = time.time()\n    print(\"SIGNAL SAMPLE {}\".format(i+1))\n    print(\"Total time : {}\".format(str(e - s) + \" s\"))\n    \n    color = 'r'\n    plt.plot(short_signal, color)\n    plt.show()\n    print(\"Trend\")\n    plt.plot(trend, color)\n    plt.show()\n    print(\"Noise\")\n    plt.plot(short_signal - trend, color)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d4b68ea0bee2c32201a0e9b71a60f620b634af38"},"cell_type":"markdown","source":"## One can clearly see the potential discharges in the noise graphs of the faulty cases. But, please feel free to share any other insights you can find from these plots."}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}