{"cells":[{"metadata":{"_uuid":"e4405a547b988f8496e03d198b98bfd34397709d"},"cell_type":"markdown","source":" # LANL Earthquake Prediction\n# 1. Overview\nThe goal of this competition is to predict the time remaining before an earthquake takes place using laboratory data. A forecasting model of the time at which an earthquake will occur is imperative because of their devastating consequences"},{"metadata":{"_uuid":"233c97af77bb76c10a6a0da57a1f08b581ae108b"},"cell_type":"markdown","source":"# 2. Import Libraries"},{"metadata":{"trusted":true,"_uuid":"1f965b27a85e577cff578a90b0ff7cd5aaf46d99"},"cell_type":"code","source":"# Data Analysis\nimport pandas as pd # data processing\nimport numpy as np # linear algebra\nfrom tqdm import tqdm # Instantly make your loops show a smart progress meter \nfrom scipy import stats\n\n# Visualization\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport plotly.plotly as py\nimport plotly.graph_objs as go\nimport cufflinks as cf  #Interactive plots\ncf.go_offline()\n%matplotlib inline\n\n# Predictive model \n# Importing required ML packages\nfrom sklearn import preprocessing\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.svm import NuSVR\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report\nfrom sklearn.metrics import mean_absolute_error","execution_count":1,"outputs":[{"output_type":"display_data","data":{"text/html":"<script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script><script>requirejs.config({paths: { 'plotly': ['https://cdn.plot.ly/plotly-latest.min']},});if(!window._Plotly) {require(['plotly'],function(plotly) {window._Plotly=plotly;});}</script>","text/vnd.plotly.v1+html":"<script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script><script>requirejs.config({paths: { 'plotly': ['https://cdn.plot.ly/plotly-latest.min']},});if(!window._Plotly) {require(['plotly'],function(plotly) {window._Plotly=plotly;});}</script>"},"metadata":{}}]},{"metadata":{"_uuid":"804669ba69d8085f327232da931596b4585d590e"},"cell_type":"markdown","source":"# 3. Acquire data"},{"metadata":{"trusted":true,"_uuid":"ca826fded7b66c8b0f6093990705dc7354e2511e"},"cell_type":"code","source":"# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Results are saved as output.","execution_count":2,"outputs":[{"output_type":"stream","text":"['test', 'train.csv', 'sample_submission.csv']\n","name":"stdout"}]},{"metadata":{"trusted":true,"_uuid":"f9070be72e275ec86f12ab9c8066295f786fbbb9"},"cell_type":"code","source":"train = pd.read_csv('../input/train.csv', dtype={'acoustic_data': np.int16, 'time_to_failure': np.float64})","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6110dce483b91d7dd321408c2117c9f67f1d93bf"},"cell_type":"markdown","source":" # 4. Exploratory Data Analysis\nWe start by doing an exploratory data analysis."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"## 4.1. Data Preview\n\nThe training dataset has more thatn 600 million rows of data. \n\n### Features:\n\n** Training data\n\nThe input dataset has two continuous numerical features:\n1.  acoustic_data - the seismic signal [int16]\n2.  time_to_failure - the time (in seconds) until the next laboratory earthquake [float64]\n\n** Test data\n\n* The test data is a folder containing many small segments of test data.\n* Every segment of test data has one feature \"acoustic_data\" with **150,000** rows\n"},{"metadata":{"trusted":true,"_uuid":"78cea08b2e52b7cab0ffea80ce22518b87dbe67b"},"cell_type":"code","source":"train.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"47e97de164cdced3c325d430a9139e5df2b24560"},"cell_type":"code","source":"pd.options.display.precision = 15\ntrain.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"258a8b4376a18ac0b962affdede716500c43d18c"},"cell_type":"markdown","source":"**Observations:**\n\n* The time of failure feature requires high precision to distinguish differences between observations\n* Acoustic data ocsilates from negative to positive values."},{"metadata":{"_uuid":"3ec64ae8b0de023090794fa9984d8e76a7fd3411"},"cell_type":"markdown","source":" ## 4.2. Data Visualization"},{"metadata":{"trusted":true,"_uuid":"e9c5fab941c9822c16a02a6928407712e5a7c593"},"cell_type":"code","source":"Seg_lenght = 150000\nSeg_dataframe = train.iloc[0:Seg_lenght]\n##\n# Edit the layout\nlayout = dict(title = 'First segment of earthquake acoustic data',\n              xaxis = dict(title = 'Time to Failure'),\n              yaxis = dict(title = 'Acoustic Data'),\n              )\nSeg_dataframe.iplot(kind='scatter',x='time_to_failure', y ='acoustic_data',mode = 'lines+markers',size=4,color = 'rgb(22, 96, 167)',layout = layout)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"63d1fd34023cdf313e7c662cc8b109fc19ee202b"},"cell_type":"markdown","source":"Distribution of acustic data for the first segment"},{"metadata":{"trusted":true,"_uuid":"549397dc5e6dc9dc69b971428e2dac09a72c4111"},"cell_type":"code","source":"sns.set_style('whitegrid')\nsns.distplot(Seg_dataframe['acoustic_data'],kde=False,color='darkblue',bins=50)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"be6b239b2a9d68817073ddd03def41fbc314bb71"},"cell_type":"code","source":"Seg_dataframe['acoustic_data'].describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d4d75b5355b104d3520366dd4d6f7db04f2ca262"},"cell_type":"markdown","source":"## 4.3. Missing Data\n\nThere are not missing data."},{"metadata":{"trusted":true,"_uuid":"118a62053190a653a504f3bd7a19525398c3c66d"},"cell_type":"code","source":"train.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9d8fd6614c04ff468467ca0c4b86ffb94c1fdd3e"},"cell_type":"markdown","source":"# 5. Feature engeneering.\n\nThe input dataset has just one feature that can be used to predict the time of failure, therefore we are going to decompose several features from the acoustic_data feature to be included in the prediction model.\n\ntime_to_failure = f(feature1, feature2, feature3)\n\nWhere all the features are derived from acoustic_data feature\n\n**New training dataframe**\n*  The input training dataframe contains a single, continuous training segment of experimental data\n* The input training dataframe will be used to create a new training dataframe. Each row of the new dataframe corresponds to an experiment segment that would correspond to the segment's lenght in the test dataset.  Example:\n\n**Segment         &nbsp; | &nbsp;   Segment Average &nbsp; | &nbsp;   std &nbsp; | &nbsp; \tmax &nbsp; | &nbsp; \tmin**\n <br>segment 1\n <br>segment 2\n <br>segment 3"},{"metadata":{"trusted":true,"_uuid":"1a39cd9cbb48e8c2cec88ed4c6880dbd07c89770"},"cell_type":"code","source":"# Trend function from: https://www.kaggle.com/jsaguiar/baseline-with-abs-and-trend-features\n#\"Simple trend feature: fit a linear regression and return the coefficient\"\ndef add_trend_feature(arr, abs_values=False):\n    idx = np.array(range(len(arr)))\n    if abs_values:\n        arr = np.abs(arr)\n    lr = LinearRegression()\n    lr.fit(idx.reshape(-1, 1), arr)\n    return lr.coef_[0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"53173b6a5847e8a5e1bf84f65015e4b5bf4ff014"},"cell_type":"code","source":"# based on https://www.kaggle.com/inversion/basic-feature-benchmark\n#****************************\n    # Additional features\n    # [1] Quantiles based on https://www.kaggle.com/andrekos/basic-feature-benchmark-with-quantiles\n    # [2] Absolute values and Trend features from: https://www.kaggle.com/jsaguiar/baseline-with-abs-and-trend-features\n    # [3] Rolling Quantiles from: https://www.kaggle.com/wimwim/rolling-quantiles\n    # [4] Additional features from: https://www.kaggle.com/artgor/earthquakes-fe-more-features-and-samples\n    # [5] Skewness and kurtusis from: https://www.kaggle.com/gpreda/lanl-earthquake-eda-and-prediction\n    #****************************\n    \nrows = 150_000 # Length of test segments\nsegments = int(np.floor(train.shape[0] / rows)) #Number of segments in the train dataset\n\n# New train features dataframe \ntrain_columns = ['mean', 'std', 'max', 'min', 'kurtosis', 'skew', 'mean10', 'X_seg_sum', \n                       'q001','q01', 'q05', 'q95', 'q99', 'q999', 'iqr','trend', 'abs_max', \n                       'abs_mean', 'abs_std', 'abs_X_seg_sum', 'abs_trend', 'abs_median',\n                       'abs_q95', 'abs_q99', 'F_test', 'p_test', 'mean_change_abs', \n                       'mean_change_rate', 'mean_roll_std_10', 'std_roll_std_10',\n                       'max_roll_std_10', 'min_roll_std_10','q01_roll_std_10', \n                       'q05_roll_std_10', 'q95_roll_std_10', 'q99_roll_std_10',\n                       'mean_change_abs_roll_std_10', 'abs_max_roll_std_10', \n                       'mean_roll_mean_10','std_roll_mean_10', 'max_roll_mean_10', \n                       'min_roll_mean_10', 'q01_roll_mean_10', 'q05_roll_mean_10', \n                       'q95_roll_mean_10', 'q99_roll_mean_10', 'mean_change_abs_roll_mean_10',\n                       'mean_change_rate_roll_mean_10', 'abs_max_roll_mean_10', \n                       'mean_roll_std_100', 'std_roll_std_100','max_roll_std_100',\n                       'min_roll_std_100','q01_roll_std_100', 'q05_roll_std_100', \n                       'q95_roll_std_100', 'q99_roll_std_100','mean_change_abs_roll_std_100', 'abs_max_roll_std_100', \n                       'mean_roll_mean_100','std_roll_mean_100', 'max_roll_mean_100', \n                       'min_roll_mean_100', 'q01_roll_mean_100', 'q05_roll_mean_100', \n                       'q95_roll_mean_100', 'q99_roll_mean_100', \n                       'mean_change_abs_roll_mean_100','mean_change_rate_roll_mean_100', \n                       'abs_max_roll_mean_100',\n                       'mean_roll_std_1000', 'std_roll_std_1000','max_roll_std_1000',\n                       'min_roll_std_1000','q01_roll_std_1000', 'q05_roll_std_1000', \n                       'q95_roll_std_1000', 'q99_roll_std_1000','mean_change_abs_roll_std_1000', 'abs_max_roll_std_100', \n                       'mean_roll_mean_1000','std_roll_mean_1000', 'max_roll_mean_1000', \n                       'min_roll_mean_1000', 'q01_roll_mean_1000', 'q05_roll_mean_1000', \n                       'q95_roll_mean_1000', 'q99_roll_mean_1000', \n                       'mean_change_abs_roll_mean_1000','mean_change_rate_roll_mean_1000', \n                       'abs_max_roll_mean_1000']\n\nX_train = pd.DataFrame(index=range(segments), dtype=np.float64,\n                       columns=train_columns)\ny_train = pd.DataFrame(index=range(segments), dtype=np.float64,\n                       columns=['time_to_failure'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f7704ab3bdac91ba547c4e390bead35da7defe62"},"cell_type":"code","source":"# Function to create Features\ndef create_features(segment, DF_seg, DF_X_Output):\n    x = DF_seg ['acoustic_data']    \n    DF_X_Output.loc[segment, 'mean'] = x.mean()\n    DF_X_Output.loc[segment, 'std'] = x.std()\n    DF_X_Output.loc[segment, 'max'] = x.max()\n    DF_X_Output.loc[segment, 'min'] = x.min()\n    DF_X_Output.loc[segment, 'kurtosis'] = x.kurtosis() #[5]\n    DF_X_Output.loc[segment, 'skew'] = x.skew()#[5]\n    # Trimmed mean, which excludes the outliers, of an array, in this case excludes 10% at both ends\n    DF_X_Output.loc[segment, 'mean10'] = stats.trim_mean(x, 0.1)#[2]\n    DF_X_Output.loc[segment, 'X_seg_sum'] = x.sum() \n    \n    # Quantile\n    DF_X_Output.loc[segment, 'q001'] = np.quantile(x,0.001)#[2]\n    DF_X_Output.loc[segment, 'q01'] = np.quantile(x,0.01) #[1]\n    DF_X_Output.loc[segment, 'q05'] = np.quantile(x,0.05) #[1]\n    DF_X_Output.loc[segment, 'q95'] = np.quantile(x,0.95) #[1]\n    DF_X_Output.loc[segment, 'q99'] = np.quantile(x,0.99) #[1]\n    DF_X_Output.loc[segment, 'q999'] = np.quantile(x,0.999)#[2]\n    # Interquartile range IQR: The IQR describes the middle 50% of values when ordered from lowest to highest. \n    # IQR = q75 - q25\n    DF_X_Output.loc[segment, 'iqr'] = np.subtract(*np.percentile(x, [75, 25]))#[2]\n    \n    # Trends\n    DF_X_Output.loc[segment, 'trend'] = add_trend_feature(x)#[2]\n    \n    # Absolut Values\n    DF_X_Output.loc[segment, 'abs_max'] = np.abs(x).max()#[2]\n    DF_X_Output.loc[segment, 'abs_mean'] = np.abs(x).mean()#[2]\n    DF_X_Output.loc[segment, 'abs_std'] = np.abs(x).std()#[2]\n    DF_X_Output.loc[segment, 'abs_X_seg_sum'] = np.abs(x).sum() \n    DF_X_Output.loc[segment, 'abs_trend'] = add_trend_feature(x, abs_values=True)#[2]\n    DF_X_Output.loc[segment, 'abs_median'] = np.median(np.abs(x))#[3]\n    DF_X_Output.loc[segment, 'abs_q95'] = np.quantile(np.abs(x),0.95)#[3]\n    DF_X_Output.loc[segment, 'abs_q99'] = np.quantile(np.abs(x),0.99)#[3]\n    \n    # Change/diff in acoustic data within a segment [3]\n    # Divide the segment in groups of 30000 sample as and do a oneway anova test\n    DF_X_Output.loc[segment, 'F_test'], DF_X_Output.loc[segment, 'p_test'] = stats.f_oneway(x[:30000],x[30000:60000],x[60000:90000],x[90000:120000],x[120000:])\n    DF_X_Output.loc[segment, 'mean_change_abs'] = np.mean(np.diff(x))\n    DF_X_Output.loc[segment, 'mean_change_rate'] = np.mean(np.nonzero((np.diff(x) / x[:-1]))[0])\n    \n    # Rolling features [3], and [4] added 1000 windows\n    for windows in [10,100,1000]:\n        x_roll_std = x.rolling(windows).std().dropna().values\n        x_roll_mean = x.rolling(windows).mean().dropna().values\n        \n        DF_X_Output.loc[segment, 'mean_roll_std_' + str(windows)] = x_roll_std.mean()\n        DF_X_Output.loc[segment, 'std_roll_std_' + str(windows)] = x_roll_std.std()\n        DF_X_Output.loc[segment, 'max_roll_std_' + str(windows)] = x_roll_std.max()\n        DF_X_Output.loc[segment, 'min_roll_std_' + str(windows)] = x_roll_std.min()\n        DF_X_Output.loc[segment, 'q01_roll_std_' + str(windows)] = np.quantile(x_roll_std,0.01)\n        DF_X_Output.loc[segment, 'q05_roll_std_' + str(windows)] = np.quantile(x_roll_std,0.05)\n        DF_X_Output.loc[segment, 'q95_roll_std_' + str(windows)] = np.quantile(x_roll_std,0.95)\n        DF_X_Output.loc[segment, 'q99_roll_std_' + str(windows)] = np.quantile(x_roll_std,0.99)\n        DF_X_Output.loc[segment, 'mean_change_abs_roll_std_' + str(windows)] = np.mean(np.diff(x_roll_std))\n        DF_X_Output.loc[segment, 'mean_change_rate_roll_std_' + str(windows)] = np.mean(np.nonzero((np.diff(x_roll_std) / x_roll_std[:-1]))[0])\n        DF_X_Output.loc[segment, 'abs_max_roll_std_' + str(windows)] = np.abs(x_roll_std).max()\n        \n        DF_X_Output.loc[segment, 'mean_roll_mean_' + str(windows)] = x_roll_mean.mean()\n        DF_X_Output.loc[segment, 'std_roll_mean_' + str(windows)] = x_roll_mean.std()\n        DF_X_Output.loc[segment, 'max_roll_mean_' + str(windows)] = x_roll_mean.max()\n        DF_X_Output.loc[segment, 'min_roll_mean_' + str(windows)] = x_roll_mean.min()\n        DF_X_Output.loc[segment, 'q01_roll_mean_' + str(windows)] = np.quantile(x_roll_mean,0.01)\n        DF_X_Output.loc[segment, 'q05_roll_mean_' + str(windows)] = np.quantile(x_roll_mean,0.05)\n        DF_X_Output.loc[segment, 'q95_roll_mean_' + str(windows)] = np.quantile(x_roll_mean,0.95)\n        DF_X_Output.loc[segment, 'q99_roll_mean_' + str(windows)] = np.quantile(x_roll_mean,0.99)\n        DF_X_Output.loc[segment, 'mean_change_abs_roll_mean_' + str(windows)] = np.mean(np.diff(x_roll_mean))\n        DF_X_Output.loc[segment, 'mean_change_rate_roll_mean_' + str(windows)] = np.mean(np.nonzero((np.diff(x_roll_mean) / x_roll_mean[:-1]))[0])\n        DF_X_Output.loc[segment, 'abs_max_roll_mean_' + str(windows)] = np.abs(x_roll_mean).max()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2918e74c8d005e0d3857abe7253b999b4d7d6046"},"cell_type":"code","source":"# Create new features for the training dataset\nfor segment_id in tqdm(range(segments)):\n    seg = train.iloc[segment_id*rows:segment_id*rows+rows]\n    create_features(segment_id, seg, X_train)\n    y_train.loc[segment_id, 'time_to_failure'] = seg['time_to_failure'].values[-1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c6b622e309adc3061249e9e2163e426063e9a608"},"cell_type":"code","source":"X_train.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b610d2c5af00e0cce77a31f366bffeaeb9c74479"},"cell_type":"markdown","source":"# 6. Scaling data"},{"metadata":{"trusted":true,"_uuid":"0903ea9ed3564c765d4b2503e63a127e4ad10ece"},"cell_type":"code","source":"scaler = StandardScaler()\nscaler.fit(X_train)\nX_train_scaled = scaler.transform(X_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"70e50690df28615b843610d270d3f95de7a65d86"},"cell_type":"code","source":"y_train.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"884a5e965854bf4497312a277bdc7777683a2fd3"},"cell_type":"markdown","source":"# 7. Building a Predictive Model\nPredict whether a passenger will survive or not using Classification Algorithms.\n\n## 7.1 Linear Regresion model"},{"metadata":{"trusted":true,"_uuid":"ece496e8a873118898be13cca05ed8fad7d782c5"},"cell_type":"code","source":"#Create an instance of a LinearRegression() model named lm.\nlm = LinearRegression()\n\n# Train lm using the training data.\nlm.fit(X_train_scaled,y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"25c98590e839cc16111d3319df7c00838fc88a4f"},"cell_type":"code","source":"# Print the coefficients\nprint('Coefficients: \\n', lm.coef_)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"17df3b662c52b511339bbf74217b7a56f4c6ee97"},"cell_type":"markdown","source":"# 7.2 Nu Support Vector Regression"},{"metadata":{"trusted":true,"_uuid":"7309ffe6814a224b6f17fb808f1a1f023367ed8c"},"cell_type":"code","source":"svm = NuSVR()\nsvm.fit(X_train_scaled,y_train.values.flatten())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"173dd82f86174e3573941d0da42752d870a6fcbc"},"cell_type":"markdown","source":"# 8. Evaluate training error"},{"metadata":{"_uuid":"0440adf1a5eb37a87cc0c7c12202e9e330c9e7f8"},"cell_type":"markdown","source":"We use the entired dataset to train the model, before we do the submision, we would estimate the training error\n## 8.1 Training Error for the Linear Regresion Model "},{"metadata":{"trusted":true,"_uuid":"2e3e7d7e03b133b49c5fac607f5cb921ff207860"},"cell_type":"code","source":"Lm_y_pred = lm.predict(X_train_scaled)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a907326e9acb24c73cc91f4baf7c17f418e27620"},"cell_type":"code","source":"# Scatterplot of the real test values versus the \"predicted\" trained values.\ny_pred = Lm_y_pred\nplt.scatter(y_train,y_pred)\nplt.xlabel('Time to Failure From the Training Data')\nplt.ylabel('Predicted Time to Failure')\nplt.plot([(0, 0), (20, 20)], [(0, 0), (20, 20)]) # Perfect correlation","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2255704395384a2ed709f03db195dfb5dc488b89"},"cell_type":"code","source":"score = mean_absolute_error(y_train.values.flatten(), y_pred)\nprint(f'Score: {score:0.3f}')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f174a640c790972e688fbb778a5f69b25377dab8"},"cell_type":"markdown","source":"## 8.2 Training Error for the Nu Support Vector Regression Model "},{"metadata":{"trusted":true,"_uuid":"3a23f921cc400a261a1e965fa52a14c323ef5b8d"},"cell_type":"code","source":"NuSVR_y_pred = svm.predict(X_train_scaled)\n\n# Scatterplot of the real test values versus the \"predicted\" trained values.\ny_pred = NuSVR_y_pred\nplt.scatter(y_train,y_pred)\nplt.xlabel('Time to Failure From the Training Data')\nplt.ylabel('Predicted Time to Failure')\nplt.plot([(0, 0), (20, 20)], [(0, 0), (20, 20)]) # Perfect correlation\nscore = mean_absolute_error(y_train.values.flatten(), y_pred)\nprint(f'Score: {score:0.3f}')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c5e0eb669e59270552cfd961447b54ceab19cf2b"},"cell_type":"markdown","source":"# 9.  Create a Submision File\nLoad files"},{"metadata":{"trusted":true,"_uuid":"873e835ce30cc85089d585c4b6af5bf8acca0824"},"cell_type":"code","source":"submission = pd.read_csv('../input/sample_submission.csv', index_col='seg_id')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a216e1f93fecea951158a245b5e530d4b5f18b75"},"cell_type":"markdown","source":"Create Testing DataFrame Based on Features we Created for the Training DataFrame\n\n--> Used the Beanchmark as an starting point from: https://www.kaggle.com/inversion/basic-feature-benchmark"},{"metadata":{"trusted":true,"_uuid":"f1fc7c361a62e51eaa87621dc99ef3220d2221da"},"cell_type":"code","source":"X_test = pd.DataFrame(columns=X_train.columns, dtype=np.float64, index=submission.index)\n# Create new features for the testing dataset\nfor segment_id in X_test.index:\n    seg = pd.read_csv('../input/test/' + segment_id + '.csv')\n    create_features(segment_id, seg, X_test)\n    \nX_test_scaled = scaler.transform(X_test)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e3c4ae3943ef4307cf76b3a1ccc747a3b7eb8d29"},"cell_type":"code","source":"X_test.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bab4bda928f4b723a6bf89215a79324c442f1e34"},"cell_type":"markdown","source":"## 9.1.  Prediction Time to Failure Linear Regresion\n### 9.1.1.  Linear Regresion Model"},{"metadata":{"trusted":true,"_uuid":"54251076cdcde91aa577ff68b23aba054ed3a08f"},"cell_type":"code","source":"y_lm_submission= lm.predict(X_test_scaled)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cdb208da306e7bcfb920609aff000c1cc27203f7"},"cell_type":"markdown","source":"### 9.1.2 . Nu Support Vector Regression"},{"metadata":{"trusted":true,"_uuid":"667f2ccd9c01b366e9ea40684ad2db6f172cfca7"},"cell_type":"code","source":"y_svm_submission= svm.predict(X_test_scaled)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7ec73205eedd91b2bf3487295d0c7237c0887c7e"},"cell_type":"markdown","source":"## 9.2 Kaggle Score"},{"metadata":{"trusted":true,"_uuid":"8d71a3a66def575ea4cd113e47f3b432cd2449f6"},"cell_type":"code","source":"columns = ['Training Error','Kaggle Score']\nindex = ['Linear Regresion']\nsummary = pd.DataFrame([[2.061,1.673],[2.014,1.563]],columns=columns,index=index)\nsummary","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ce97e34d117c00d4d843f2c19f242d094dc700bf"},"cell_type":"markdown","source":"## 9.3 Submision file"},{"metadata":{"trusted":true,"_uuid":"50ca3d4c3528fd54fe30f55a0b4f18f9b28473e2"},"cell_type":"code","source":"submission['time_to_failure'] = y_svm_submission\nsubmission.to_csv('submission.csv')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}