{"cells":[{"metadata":{},"cell_type":"markdown","source":"Final Project\n* Rucha Athavale, Rishikesh Mane, Parita Shah, Shaildeep Kaur\n- Deep within the Earth it is so hot that some rocks slowly melt and become a thick flowing substance called magma. Since it is lighter than the solid rock around it, magma rises and collects in magma chambers. Eventually, some of the magma pushes through vents and fissures to the Earth&#39;s surface. Magma that has erupted is called lava. Explosive volcanic eruptions can be dangerous and deadly. They can blast out clouds of hot tephra from the side or top of a volcano. These fiery clouds race down mountainsides destroying almost everything in their path. Ash erupted into the sky falls back to Earth like powdery snow. If thick enough, blankets of ash can suffocate plants, animals, and humans. When hot volcanic materials mix with water from streams or melted snow and ice, mudflows form. Mudflows have buried entire communities located near erupting volcanoes. How do scientist study volcanoes and predict eruptions early? Scientists use a wide variety of techniques to monitor volcanoes, including seismographic detection of the earthquakes and tremor that almost always precede eruptions, precise measurements of ground deformation that often accompanies the rise of magma, changes in volcanic gas emissions, and changes in gravity and magnetic fields. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        #print(os.path.join(dirname, filename))\n        continue\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nfrom sklearn import metrics as metrics\nfrom sklearn import model_selection\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import classification_report\nfrom sklearn.metrics import mean_squared_error as mse\nfrom sklearn.feature_selection import RFE\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import roc_curve, auc\nfrom sklearn.cluster import KMeans, MeanShift, DBSCAN\nfrom sklearn.metrics import silhouette_score\n\nfrom xgboost import XGBRegressor\nimport xgboost as xgb\nimport math\n\nimport plotly.express as px\nfrom plotly.subplots import make_subplots\nimport plotly.graph_objs as go\nimport matplotlib.pyplot as plt\nimport glob\nimport math\n\nimport plotly.express as px\nfrom plotly.subplots import make_subplots\nimport plotly.graph_objs as go\n\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.metrics import mean_squared_error as mse\nfrom sklearn.feature_selection import RFE\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.tree import DecisionTreeRegressor\n\nfrom lightgbm import LGBMRegressor\nfrom xgboost import XGBRegressor\n\nimport optuna\nfrom optuna.samplers import TPESampler\n\nimport tensorflow as tf\nimport tensorflow.keras.backend as K\nfrom tensorflow.keras.layers import Input, Dense\nfrom sklearn.model_selection import KFold\nfrom tensorflow.keras.callbacks import EarlyStopping","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/predict-volcanic-eruptions-ingv-oe/train.csv\")\nsample_submission = pd.read_csv(\"../input/predict-volcanic-eruptions-ingv-oe/sample_submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.histogram(\n    train, \n    x=\"time_to_eruption\",\n    width=800,\n    height=500,\n    nbins=100,\n    title='Time to erupt distribution'\n)\n\nfig.show()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.line(\n    train, \n    y=\"time_to_eruption\",\n    width=800,\n    height=500,\n    title='Time to erupt for all volcanos'\n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train['time_to_eruption'].describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Median:', train['time_to_eruption'].median())\nprint('Skew:', train['time_to_eruption'].skew())\nprint('Std:', train['time_to_eruption'].std())\nprint('Kurtosis:', train['time_to_eruption'].kurtosis())\nprint('Mean:', train['time_to_eruption'].mean())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_sensor = glob.glob(\"../input/predict-volcanic-eruptions-ingv-oe/train/*\")\nlen(train_sensor)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_sensor = glob.glob(\"../input/predict-volcanic-eruptions-ingv-oe/test/*\")\nlen(test_sensor)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"example = pd.read_csv('../input/predict-volcanic-eruptions-ingv-oe/train/2037160701.csv')\nexample.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sensors = set()\nobservations = set()\nnan_columns = list()\nmissed_groups = list()\nfor_df = list()\n\nfor item in train_sensor:\n    name = int(item.split('.')[-2].split('/')[-1])\n    at_least_one_missed = 0\n    frag = pd.read_csv(item)\n    missed_group = list()\n    missed_percents = list()\n    for col in frag.columns:\n        missed_percents.append(frag[col].isnull().sum() / len(frag))\n        if pd.isnull(frag[col]).all() == True:\n            at_least_one_missed = 1\n            nan_columns.append(col)\n            missed_group.append(col)\n    if len(missed_group) > 0:\n        missed_groups.append(missed_group)\n    sensors.add(len(frag.columns))\n    observations.add(len(frag))\n    for_df.append([name, at_least_one_missed] + missed_percents)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Number of sensors: ', sensors)\nprint('Number of observations: ', observations)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Number of missed sensors:', len(nan_columns))\n\nabsent_sensors = dict()\n\nfor item in nan_columns:\n    if item in absent_sensors:\n        absent_sensors[item] += 1\n    else:\n        absent_sensors[item] = 0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"absent_df = pd.DataFrame(absent_sensors.items(), columns=['Sensor', 'Missed sensors'])\n\nfig = px.bar(\n    absent_df, \n    x=\"Sensor\",\n    y='Missed sensors',\n    width=800,\n    height=500,\n    title='Number of missed sensors in training dataset'\n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for_df = pd.DataFrame(\n    for_df, \n    columns=[\n        'segment_id', 'has_missed_sensors', 'missed_%_sensor1', \n        'missed_%_sensor2', 'missed_%_sensor3', 'missed_%_sensor4', \n        'missed_%_sensor5', 'missed_%_sensor6', 'missed_%_sensor7', \n        'missed_%_sensor8', 'missed_%_sensor9', 'missed_%_sensor10'\n    ]\n)\nfor_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.merge(train, for_df)\ntrain","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sensors = set()\nobservations = set()\nnan_cols = list()\nfor_test_df = list()\n\nfor item in test_sensor:\n    name = int(item.split('.')[-2].split('/')[-1])\n    at_least_one_missed = 0\n    sensor = pd.read_csv(item)\n    missed_percents = list()\n    for col in sensor.columns:\n        missed_percents.append(sensor[col].isnull().sum() / len(sensor))\n        if pd.isnull(sensor[col]).all() == True:\n            at_least_one_missed = 1\n            nan_cols.append(col)\n    sensors.add(len(sensor.columns))\n    observations.add(len(sensor))\n    for_test_df.append([name, at_least_one_missed] + missed_percents)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for_test_df = pd.DataFrame(\n    for_test_df, \n    columns=[\n        'segment_id', 'has_missed_sensors', 'missed_%_sensor1', 'missed_%_sensor2', 'missed_%_sensor3', \n        'missed_%_sensor4', 'missed_%_sensor5', 'missed_%_sensor6', 'missed_%_sensor7', \n        'missed_%_sensor8', 'missed_%_sensor9', 'missed_%_sensor10'\n    ]\n)\n\nfor_test_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Number of sensors: ', sensors)\nprint('Number of observations: ', observations)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Number of missed sensors:', len(nan_columns))\n\nabsent_sensors = dict()\n\nfor item in nan_cols:\n    if item in absent_sensors:\n        absent_sensors[item] += 1\n    else:\n        absent_sensors[item] = 0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"absent_df = pd.DataFrame(absent_sensors.items(), columns=['Sensor', 'Missed sensors'])\n\nfig = px.bar(\n    absent_df, \n    x=\"Sensor\",\n    y='Missed sensors',\n    width=800,\n    height=500,\n    title='Number of missed sensors in testing dataset'\n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = make_subplots(rows=5, cols=2)\ntraces = [\n    go.Histogram(\n        x=example[col], \n        nbinsx=100, \n        name=col\n    ) for col in example.columns\n]\n\nfor i in range(len(traces)):\n    fig.append_trace(\n        traces[i], \n        (i // 2) + 1, \n        (i % 2) + 1\n    )\n\nfig.update_layout(\n    title_text='Data from sensors distribution',\n    height=800,\n    width=1200\n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = make_subplots(rows=5, cols=2)\ntraces = [\n    go.Scatter(\n        x=[i for i in range(60002)], \n        y=example[col], \n        mode='lines', \n        name=col\n    ) for col in example.columns\n]\n\nfor i in range(len(traces)):\n    fig.append_trace(\n        traces[i], \n        (i // 2) + 1, \n        (i % 2) + 1\n    )\n\nfig.update_layout(\n    title_text='Data from sensors',\n    height=800,\n    width=1200\n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def features(feats, ts, sensor_id):\n    X = pd.DataFrame()\n    f = np.fft.fft(feats)\n    f_real = np.real(f)\n    X.loc[ts, f'{sensor_id}_sum']       = feats.sum()\n    X.loc[ts, f'{sensor_id}_mean']      = feats.mean()\n    X.loc[ts, f'{sensor_id}_std']       = feats.std()\n    X.loc[ts, f'{sensor_id}_var']       = feats.var() \n    X.loc[ts, f'{sensor_id}_max']       = feats.max()\n    X.loc[ts, f'{sensor_id}_min']       = feats.min()\n    X.loc[ts, f'{sensor_id}_skew']      = feats.skew()\n    X.loc[ts, f'{sensor_id}_mad']       = feats.mad()\n    X.loc[ts, f'{sensor_id}_kurtosis']  = feats.kurtosis()\n    return X","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_set = list()\nj = 0\nfor seg in train.segment_id:\n    files = pd.read_csv(f'/kaggle/input/predict-volcanic-eruptions-ingv-oe/train/{seg}.csv')\n    train_row = []\n    if j%500 == 0:\n        print(j)\n    for i in range(0, 10):\n        sensor_id = f'sensor_{i+1}'\n        train_row.append(features(files[sensor_id].fillna(0), seg, sensor_id))\n    train_row = pd.concat(train_row, axis=1)\n    train_set.append(train_row)\n    j+=1\n\ntrain_set = pd.concat(train_set)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_set = train_set.reset_index()\ntrain_set = train_set.rename(columns={'index': 'segment_id'})\ntrain_set = pd.merge(train_set, train, on='segment_id')\ntrain_set","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_set = list()\nj=0\nfor seg in sample_submission.segment_id:\n    files = pd.read_csv(f'/kaggle/input/predict-volcanic-eruptions-ingv-oe/test/{seg}.csv')\n    test_row = []\n    if j%500 == 0:\n        print(j)\n    for i in range(0, 10):\n        sensor_id = f'sensor_{i+1}'\n        test_row.append(features(files[sensor_id].fillna(0), seg, sensor_id))\n    test_row = pd.concat(test_row, axis=1)\n    test_set.append(test_row)\n    j+=1\ntest_set = pd.concat(test_set)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_set = test_set.reset_index()\ntest_set = test_set.rename(columns={'index': 'segment_id'})\ntest_set = pd.merge(test_set, for_test_df, on='segment_id')\ntest = test_set.drop(['segment_id'], axis=1)\ntest","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"drop_cols = list()\nfor col in train_set.columns:\n    if col == 'segment_id':\n        continue\n    if abs(train_set[col].corr(train_set['time_to_eruption'])) < 0.01:\n        drop_cols.append(col)\ndrop_cols","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"no_drop_cols = list()\n\nfor col1 in train_set.columns:\n    for col2 in train_set.columns:\n        if col1 == col2:\n            continue\n        if col1 == 'segment_id' or col2 == 'segment_id': \n            continue\n        if col1 == 'time_to_eruption' or col2 == 'time_to_eruption':\n            continue\n        if abs(train_set[col1].corr(train_set[col2])) > 0.98:\n            if col2 not in drop_cols and col1 not in no_drop_cols:\n                drop_cols.append(col2)\n                no_drop_cols.append(col1)\nno_drop_cols","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Variance check is inlcuded in the features: '{sensor_id}_variance'_\n* Correlation check is included in dropping columns"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_set.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_set.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Logistic Regression"},{"metadata":{"trusted":true},"cell_type":"code","source":"X = train_set.drop(['time_to_eruption'], axis = 1)\ny = train_set['time_to_eruption']\nX_test = test_set\n#X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3,random_state=101)\n\n#logmodel = LogisticRegression()\n#logmodel.fit(X, y)\n#pred_1 = logmodel.predict(X_test)\n#pred_1\n\n#print(classification_report(y_test, pred_1))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* print('Mean Absolute Error:', metrics.mean_absolute_error(y, pred_1))  \n* print('Mean Squared Error:', metrics.mean_squared_error(y, pred_1))  \n* print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y, pred_1)))\n* print('R-squared Error:', metrics.r2_score(y, pred_1))"},{"metadata":{"trusted":true},"cell_type":"code","source":"#X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=0.9)\n\nknn = KNeighborsClassifier(n_neighbors=1)\nknn.fit(X, y)\npred_2 = knn.predict(X_test)\npred_2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"* print('Mean Absolute Error:', metrics.mean_absolute_error(y, pred_2))  \n* print('Mean Squared Error:', metrics.mean_squared_error(y, pred_2))  \n* print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y, pred_2)))\n* print('R-squared Error:', metrics.r2_score(y, pred_2))"},{"metadata":{},"cell_type":"markdown","source":"Linear Regression"},{"metadata":{"trusted":true},"cell_type":"code","source":"sc = StandardScaler()\nX_train_sc = sc.fit_transform(X)\nX_test_sc = sc.transform(X_test)\nregressor = LinearRegression()  \nregressor.fit(X, y)\nprint(regressor.intercept_)\nprint(regressor.coef_)\npred_3 = regressor.predict(X_test)\npred_3","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"plt.scatter(X_test, y_test,  color='gray')\nplt.plot(X_test, y_pred, color='red', linewidth=2)\nplt.show()"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"* print('Mean Absolute Error:', metrics.mean_absolute_error(y, pred_3))  \n* print('Mean Squared Error:', metrics.mean_squared_error(y, pred_3))  \n* print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y, pred_3)))\n* print('R-squared Error:', metrics.r2_score(y, pred_3))"},{"metadata":{},"cell_type":"markdown","source":"K-Means Cluster"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"def kmeans(model):\n    pipeline = Pipeline([\n        ('kmeans', KMeans()),\n        (f'{model}', model)\n    ])\n    pipeline.fit(X, y)\n    preds = pipeline.predict(X_test)\n    return(mean_squared_error(y, preds))"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"Running K-means with K = 3"},{"metadata":{"trusted":true},"cell_type":"code","source":"kmeans = KMeans(n_clusters = 3, max_iter = 50, random_state = 14)\nkmeans.fit(X)\nkmeans.labels_ \npred_4 = kmeans.predict(X) \npred_4","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Density based Clustering"},{"metadata":{"trusted":true},"cell_type":"code","source":"def DBSCAN_params(max_distance,min_cars,ride_data):\n    \n    ## get coordinates from ride data\n    coords = ride_data.as_matrix(columns=['lat', 'lon'])\n    \n    ## calculate epsilon parameter using\n    ## the user defined distance\n    kms_per_radian = 6371.0088\n    epsilon = max_distance / kms_per_radian\n    \n    ## perform clustering\n    db = DBSCAN(eps=epsilon, min_samples=min_cars,\n                algorithm='', metric='').fit(np.radians(coords))\n    \n    ## group the clusters\n    cluster_labels = db.labels_\n    num_clusters = len(set(cluster_labels))\n    clusters = pd.Series([coords[cluster_labels == n] for n in range(num_clusters)])\n    \n    ## report\n    print('Number of clusters: {}'.format(num_clusters))\n\n    ## initialize lists for hot spots\n    lat = []\n    lon = []\n    num_members = []\n \n    \n    ## loop through clusters and get centroids, number of members\n    for ii in range(len(clusters)):\n \n        ## filter empty clusters\n        if clusters[ii].any():\n \n            ## get centroid and magnitude of cluster\n            lat.append(MultiPoint(clusters[ii]).centroid.x)\n            lon.append(MultiPoint(clusters[ii]).centroid.y)\n            num_members.append(len(clusters[ii]))\n            \n    params = [lon,lat,num_members]\n    return params \n  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.cluster import DBSCAN\nfrom itertools import product\n\neps_values = np.arange(8,12.75,0.25) # eps values to be investigated\nmin_samples = np.arange(3,10) # min_samples values to be investigated\nDBSCAN_params = list(product(eps_values, min_samples))\nno_of_clusters = []\nsil_score = []\n\nfor p in DBSCAN_params:\n    DBS_clustering = DBSCAN(eps=p[0], min_samples=p[1]).fit(X)\n    no_of_clusters.append(len(np.unique(DBS_clustering.labels_)))\n    #sil_score.append(silhouette_score(X, DBS_clustering.labels_))\n    #print(no_of_clusters, silhouette_score(X,DBS_clustering))\n    #print(DBS_clustering)\n    dbs = np.zeros_like(DBS_clustering.labels_, dtype=bool)\n    dbs[DBS_clustering.core_sample_indices_] = True\n    labels = DBS_clustering.labels_","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#plt.scatter(X[:,0],s=10,c=labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#tmp = pd.DataFrame.from_records(DBSCAN_params, columns =['Eps', 'Min_samples'])   \n#tmp['Sil_score'] = sil_score\n\n#pivot_1 = pd.pivot_table(tmp, values='Sil_score', index='Min_samples', columns='Eps')\n\n#fig, ax = plt.subplots(figsize=(18,6))\n#sns.heatmap(pivot_1, annot=True, annot_kws={\"size\": 10}, cmap=\"YlGnBu\", ax=ax)\n#plt.show()\nDBS_clustering = DBSCAN(eps=12.5, min_samples=4).fit(X)\n\nDBSCAN_clustered = X.copy()\nDBSCAN_clustered.loc[:,'Cluster'] = DBS_clustering.labels_ # append labels to points\nDBSCAN_clust_sizes = DBSCAN_clustered.groupby('Cluster').size().to_frame()\nDBSCAN_clust_sizes.columns = [\"DBSCAN_size\"]\nDBSCAN_clust_sizes","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"XGBoost"},{"metadata":{"trusted":true},"cell_type":"code","source":"eval_metric = [\"rmse\"]\nxgb_model = XGBRegressor()\nXGBRegressor(base_score=0.5, booster='gbtree',\n       importance_type='gain', learning_rate=0.3, max_delta_step=0,\n       max_depth=3, min_child_weight=1, n_estimators=100,\n       n_jobs=1,objective='reg:linear', random_state=0,\n        seed=None,silent=None, subsample=1, verbosity=1)\n\nxgb_model.fit(X, y, eval_metric = eval_metric)\n\nfrom sklearn.metrics import precision_recall_fscore_support as score, precision_score, recall_score, f1_score\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import precision_score, recall_score, accuracy_score\npred_5 = xgb_model.predict(X_test)\npred_5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pred_5.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"* from sklearn.metrics import mean_absolute_error, mean_squared_error\n* from math import sqrt\n* print(\"MAE test score:\", mean_absolute_error(y, pred_5))\n* print(\"MSE test score:\", mean_squared_error(y, pred_5))\n* print(\"RMSE test score:\", sqrt(mean_squared_error(y, pred_5)))"},{"metadata":{"trusted":true},"cell_type":"code","source":"#x_ax = range(len(y))\n#plt.plot(x_ax, y, label=\"original\")\n#plt.plot(x_ax, pred_5, label=\"predicted\")\n#plt.title(\"test and predicted data\")\n#plt.legend()\n#plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_set['time_to_eruption'] = pred_5\n#test_set['time_to_eruption'] = pred_3\n#test_set['time_to_eruption'] = pred_2\n#test_set['time_to_eruption'] = pred_1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission = pd.merge(sample_submission, test_set[['segment_id', 'time_to_eruption']], on='segment_id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission = sample_submission.drop(['time_to_eruption_x'], axis=1)\nsample_submission.columns = ['segment_id', 'time_to_eruption']\nsample_submission","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"References\n* https://www.kaggle.com/isaienkov"},{"metadata":{},"cell_type":"markdown","source":"Overall, our models are working good but we got some errors during the classification report as there was some index mismatch. We have learned a lot of things from this project, it taught how important data cleaning, data visualization, and data imputation is because you cannot drop the columns with missing values as we are losing a lot of valuable data, instead we filled it with either median or mode. Data pre-processing was time-consuming because there was a massive amount of data we had to work on but we had a good learning experience working with large datasets."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}