{"cells":[{"metadata":{},"cell_type":"markdown","source":"### This is the latest version of a small baseline model. Due to unfamiliarity with signal analysis, utilization of SciPy's signal functions may be incorrect."},{"metadata":{},"cell_type":"markdown","source":"### To-Do List\n* In-Depth EDA\n* Optimization of dataset generation. Currently going from Pandas to non pandas to Pandas is not efficient.\n* Compare model output with feature selection criteria.\n* Hyperparameter Tuning. PyCaret allows for AutoTuning but i've had a few issues with this dataset, so i will be tuning the winning model from PyCaret with the usual GridSearchCV from sklearn.\n"},{"metadata":{},"cell_type":"markdown","source":"Library loading"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\nimport numpy as np # linear algebra\nimport pandas as pd\nfrom scipy import signal\nfrom scipy import signal\nfrom scipy.signal import find_peaks\n!pip install pycaret\nfrom pycaret.regression import *","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training set generation:\n\nCurrently approach is to extract a few features from each of the time-series(TS) in each of the .csv files to generate a single row in a final dataset that will correspond to the id and its matching 'time_to_eruption'. Null values are imputted as 0. This is not a correct approach and is done for the baseline procedure, will change inputting strategy once EDA is perfomed.\n\nCurrent Features:\n* Basic Stats(mean,std,median,max,etc.) of original TS, output of df.describe without 'count'.\n* Basic Stats of periodogram of original TS.\n* Amount of Stds that the Max or Min amount to\n* Number of peaks in each sensor.\n* Basic Stats(mean,std) of the prominence of the peaks. "},{"metadata":{"trusted":true},"cell_type":"code","source":"from pathlib import Path\nimport time\nstart = time.time()\nA = []\nrownames = []\npathlist = sorted(Path('../input/predict-volcanic-eruptions-ingv-oe/train/').glob('**/*.csv'))\nc=0\nfor path in pathlist:\n    path_in_str = str(path)\n    dat=pd.read_csv(path_in_str)\n    dat=dat.fillna(0) #Handling NaN in this manner is not correct. But this is a baseline.\n    dat2=dat.describe().iloc[1:,:]\n    dat2=pd.concat([dat2,pd.DataFrame(signal.periodogram(dat)[1]).describe().iloc[1:,:]],axis=1) \n    dat2=pd.DataFrame(dat2.unstack()).T\n    for (columnName, columnData) in dat.iteritems():\n        dat2[str(columnName)+ '_Mode'] = dat[str(columnName)].mode()\n        dat2[str(columnName)+ '_MaxStds'] = [dat[str(columnName)].max()/dat[str(columnName)].std() if dat[str(columnName)].std()!=0 else np.nan]\n        dat2[str(columnName)+ '_MinStds'] = [dat[str(columnName)].min()/dat[str(columnName)].std() if dat[str(columnName)].std()!=0 else np.nan]\n        dat2[str(columnName)+ '_PeakNumber']=len(find_peaks(dat[str(columnName)])[0])\n        dat2[str(columnName)+ '_MeanProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].mean()\n        dat2[str(columnName)+ '_StdProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].std()\n        #dat2[str(columnName)+ '_MaxProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].max()\n        #dat2[str(columnName)+ '_MinProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].min()\n    dat2.index=dat2.index.to_flat_index()\n    A.append(dat2.T.values.flatten())\n    rownames.append(path.stem)\n    c=c+1\n    print(path.stem,c)\nprint(time.time()-start)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Test set generation:\n\nSame as Training set description."},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"from pathlib import Path\nA_test = []\nrownames_test = []\nc=0\npathlist = sorted(Path('../input/predict-volcanic-eruptions-ingv-oe/test/').glob('**/*.csv'),reverse=True)\nfor path in pathlist:\n     # because path is object not string\n    path_in_str = str(path)\n    dat=pd.read_csv(path_in_str)\n    dat=dat.fillna(dat.mean(0))\n    dat2=dat.describe().iloc[1:,:]\n    dat2=pd.concat([dat2,pd.DataFrame(signal.periodogram(dat)[1]).describe().iloc[1:,:]],axis=1)\n    dat2=pd.DataFrame(dat2.unstack()).T\n    for (columnName, columnData) in dat.iteritems():\n        dat2[str(columnName)+ '_Mode'] = dat[str(columnName)].mode()\n        dat2[str(columnName)+ 'MaxStds'] = [dat[str(columnName)].max()/dat[str(columnName)].std() if dat[str(columnName)].std()!=0 else np.nan]\n        dat2[str(columnName)+ 'MinStds'] = [dat[str(columnName)].min()/dat[str(columnName)].std() if dat[str(columnName)].std()!=0 else np.nan]\n        dat2[str(columnName)+ '_PeakNumber']=len(find_peaks(dat[str(columnName)])[0])\n        dat2[str(columnName)+ '_MeanProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].mean()\n        dat2[str(columnName)+ '_StdProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].std()\n        #dat2[str(columnName)+ '_MaxProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].max()\n        #dat2[str(columnName)+ '_MinProminence']=signal.peak_prominences(dat[str(columnName)],signal.find_peaks(dat[str(columnName)])[0])[0].min()\n    dat2.index=dat2.index.to_flat_index()\n    A_test.append(dat2.T.values.flatten())\n    rownames_test.append(path.stem)\n    c=c+1\n    print(path.stem,c)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Tidying up the output from the loops."},{"metadata":{"trusted":true},"cell_type":"code","source":"A = pd.DataFrame(A)\nA['segment_id']=rownames\nA.index=A['segment_id']\nA=A.drop('segment_id',axis=1)\nA.index=A.index.astype('int32')\ntarget = pd.read_csv('../input/predict-volcanic-eruptions-ingv-oe/train.csv')\ntarget.index=target['segment_id']\ntarget=target.drop('segment_id',axis=1)\nB=pd.merge(A, target, left_index=True, right_index=True)\nB=B.astype('float32')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Feature Selection:\n\nBasic feature selection with sklearn to lower amount of features. \nAlthough, 190 features is not that many considering the size of the dataset, but lower features returned better results this time.\nThis should be optimized by quick checking the effect of k-features on model MAE."},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.feature_selection import SelectKBest, chi2, mutual_info_regression\nB=B.fillna(B.mean())\nSelection = SelectKBest(mutual_info_regression, k=80).fit(B.iloc[:,:-1], B['time_to_eruption'])\nX_new=Selection.transform(B.iloc[:,:-1])\nX_new1=pd.DataFrame(X_new)\nX_new1.index=B.index\nX_new1['time_to_eruption']=B['time_to_eruption']\nX_new1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Model Selection:\n\nThe AutoML library PyCaret will be employed for this approach.\nIt will perform the training over many supervised models and return a mean of a 10-Fold for each of them. \n(Hyperparameters are *not* modified in this part)."},{"metadata":{"trusted":true},"cell_type":"code","source":"comparison_setup = setup(X_new1, target = 'time_to_eruption',session_id=17,silent = True)\ncompare_models()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The best model is ExtraTrees, so i will create a new training with just this model and finalize it to perform predictions over the test dataset."},{"metadata":{"trusted":true},"cell_type":"code","source":"xt_clu = setup(X_new1, target = 'time_to_eruption',session_id=17,silent = True)\net1 = create_model('et')\nfinal_et1 = finalize_model(et1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Tidying up the test database"},{"metadata":{"trusted":true},"cell_type":"code","source":"A_test=pd.DataFrame(A_test)\nA_test['id']=rownames_test\nA_test.index=A_test['id'].astype('int32')\nA_test=A_test.drop(['id'],axis=1) \nA_test2=A_test.fillna(0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Apply the feature selection to test"},{"metadata":{"trusted":true},"cell_type":"code","source":"X_test=Selection.transform(A_test2)\nX_test=pd.DataFrame(X_test)\nX_test.index=rownames_test","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Predictions:"},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = predict_model(final_et1, data = X_test)\nsub=predictions[['Label']]\nsub['segment_id']=sub.index\nsub['time_to_eruption']=predictions['Label']\nsub=sub.drop('Label',axis=1)\nsub","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Create submission for competition:"},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.to_csv('submission.csv',index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}