{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\n\nimport numpy as np\nimport pandas as pd\nimport pyarrow.parquet as pq\n\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nprint(os.listdir('../input'))\nprint(os.listdir('../input/phase-subs'))\n\ndata_dir = '../input'","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"361c359d8372dd484cb436fcaa556d2544e696a0"},"cell_type":"markdown","source":"## Phase Fault Distributions and Concatenating Phases Problem\n\nAs we can see below, 57 of the lines have a fault in only 1 phase, and another 57 of the lines only have a fault in 2 phases. If the test set has a similar distribution of phase faults then a model such as https://www.kaggle.com/braquino/5-fold-lstm-attention-fully-commented-0-694 that concatenates all 3 phases and always predicts a fault for all 3 phases in a measurement then it will inherently predict some false positives. Concatenating the phases does seem to be very effective but maybe we can combine predictions from a model that trains and predicts on each phase individually to further improve the score."},{"metadata":{"trusted":true,"_uuid":"2fe234dcb88e6d062ade8dc52ec879226d48b658"},"cell_type":"code","source":"metadata_train = pd.read_csv(data_dir + '/vsb-power-line-fault-detection/metadata_train.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9b65553688c419591d80e3abfc0b470205d04513"},"cell_type":"code","source":"metadata_train['target_phase_sum'] = metadata_train.groupby('id_measurement')['target'].transform(np.sum)\nmetadata_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"20450429a8b80c3059532baea754292b2cc2953b"},"cell_type":"code","source":"plt.hist(metadata_train['target_phase_sum'].values)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"aed6da7592284f572c8eeeafa17009d22edea503"},"cell_type":"code","source":"print(len(metadata_train[metadata_train['target_phase_sum'] == 1]))\nprint(len(metadata_train[metadata_train['target_phase_sum'] == 2]))\nprint(len(metadata_train[metadata_train['target_phase_sum'] == 3]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"99f38dab484c1005d441656bd7ae75a6574bddf4"},"cell_type":"code","source":"metadata_train[metadata_train['target_phase_sum'] == 1].head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6181f9b4935399fbc7a5694faea28302cf9e681b"},"cell_type":"code","source":"metadata_train[metadata_train['target_phase_sum'] == 2].head(10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1196ae200f70f2fc3dffdae5f30e20d44b9745b5"},"cell_type":"markdown","source":"## Compare Predicted Targets"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"# Submission from LSTM model trained on each phase seperately\n# Source https://www.kaggle.com/braquino/5-fold-lstm-attention-fully-commented-0-694 with modification to train and predict on each phase\nsub_phase = pd.read_csv(data_dir + '/phase-subs/lstm_5fold_phase_564_sub.csv')\nsub_phase_n_faults = sub_phase.target.sum()\nprint(sub_phase_n_faults)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bdfd7ef09af91ac68ee7a962d05e763b402c3614"},"cell_type":"code","source":"# Submission from LSTM model trained on concatenated phases\n# Source https://www.kaggle.com/braquino/5-fold-lstm-attention-fully-commented-0-694\nsub_phase_concat = pd.read_csv(data_dir + '/phase-subs/lstm_5fold_phase_concat_648_sub.csv')\nsub_phase_concat_n_faults = sub_phase_concat.target.sum()\nprint(sub_phase_concat_n_faults)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"87698386440c24410e0d6be9b5f163c6a1054bd1"},"cell_type":"code","source":"diff_faults = sub_phase_n_faults - sub_phase_concat_n_faults\nprint(diff_faults)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b0bde4124c57d0daf6d9682ffe06de26a2cbff2a"},"cell_type":"markdown","source":"The phase only model predicted 239 more faults than the phase concat model, it scores lower on the LB so this might indicate it is predicting more false positives."},{"metadata":{"trusted":true,"_uuid":"28dac7defc88727eb3b038a1c174a7d489f0b3ba"},"cell_type":"code","source":"meta_test = pd.read_csv(data_dir + '/vsb-power-line-fault-detection/metadata_test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b6d55f28c8d180578792f4f788216cac4b984d78"},"cell_type":"code","source":"# Merge meta test meta data so we can compare predictions from each submission\nsub_merge = sub_phase.copy()\nsub_merge = sub_merge.drop(columns=['target'])\nsub_merge['id_measurement'] = meta_test.id_measurement.values\nsub_merge['phase'] = meta_test.phase.values\nsub_merge['target_sub_phase'] = sub_phase['target'].values\nsub_merge['target_sub_phase_concat'] = sub_phase_concat['target'].values\nsub_merge.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f8e4a25ca1441f65ff99c3ac02a0509613e8160c"},"cell_type":"code","source":"sub_merge[sub_merge['target_sub_phase_concat'] == 1].head(50)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b7b3567d2ee13d9ce616b5e719651e4c4776031b"},"cell_type":"markdown","source":"The phase concat model contains many of the same predictions as the phase model, but notice target_sub_phase=0 for signal_id 9166, 9467 and 10059. I am thinking the phase concat model has predicted a false postive here. If the test set target distribtion is similar to the train set then the phase concat model would certainly predict some false postives since some faults only affect 1 or 2 phases."},{"metadata":{"_uuid":"588a942bcdfe52223c2a90037025718e7115f784"},"cell_type":"markdown","source":"## Blend Submissions"},{"metadata":{"_uuid":"257567ab53fdfdc361ed43c4935552484a3b14f3"},"cell_type":"markdown","source":"Let's try improve the phase concat model submission by setting the suspected false positives to zero. "},{"metadata":{"trusted":true,"_uuid":"4cd5d6836bc3a91132eadf563303ee0bcb875d6e"},"cell_type":"code","source":"sub_merge['target_sub_phase_group_sum'] = sub_merge.groupby('id_measurement')['target_sub_phase'].transform(np.sum)\nsub_merge[sub_merge['target_sub_phase_concat'] == 1].head(20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d963d62d5620983b90ef09dc5b3af2bc066017b9"},"cell_type":"code","source":"sub_merge['target'] = sub_merge.target_sub_phase_concat.values\nsub_merge.loc[(sub_merge['target_sub_phase'] == 0) & (sub_merge['target_sub_phase_concat'] == 1) & (sub_merge['target_sub_phase_group_sum'] == 2), 'target'] = 0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"51cdce3567b2a6b079c9b9de71de3b97ae1d8e1e"},"cell_type":"code","source":"sub_merge.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ed35c9a9cf06d67e97671d17a742f4ad6ec85b3c"},"cell_type":"code","source":"final_sub = sub_merge[['signal_id', 'target']]\nfinal_sub.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"abf99b809e6deb672aa681024336434044f19066"},"cell_type":"code","source":"final_sub.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e2bc88927305266b4ada2254976e9c1ab0262bf4"},"cell_type":"markdown","source":"The new submission scores 0.628 on the LB, so it's not an improvement on the phase concat model which scored 0.648. Maybe with a better scoring submissions from a model trained on each phase this method might help, or maybe every fault in the test set happens on all 3 phases. "}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}