{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\n\nFirst and foremost, my most heartfelt thanks go to the organizers for setting up this competition! \n\nI really had a blast working on such an inspiring problem! \n\nThis is my first Kaggle competition. About a year ago I started my personnal journey into Machine learning and Data Science, starting by  reading about the topic and then completing online classes. After I first read about the competition, I thought this could be an excellent opportunity to apply the technics I learned so far.\n\nAs I detailled further, I attempted to solve this project using a **1D CNN**. \n\nGoing though the public notebooks, it appears that this solution is not the most common among the participants. Although my final score is average (at least based on the current standing on the leaderboard :), more on that later), I thought it would be of some interest if I documented this attempt and discussed the concept with the community. \n\nAs such your comments are welcome!\n\n\n"},{"metadata":{},"cell_type":"markdown","source":"# General Concept\n\nAs I read about the problem description I thought I could try to detect and use **patterns in the waveform recordings** in order to solve this prediction problem. \n\nThis idea is not new among Vulcanologist (see articles I reference at the end of the notebook)\n\nHere is how I would personnally summurrize the idea:\n* In and around the volcano *seismic events* happen all the time.\n* The *seismic events* are captured by the different sensors in the form of *seismic waveforms measurements*.\n* Different *waveforms* resulting from different *seismic events* will show *similarities* in amplitude, duration and shape. I will call them *patterns*.\n* The study of the *patterns* can inform us on what is happening in the volcano, and, help make *predictions* on the time to the next eruption.\n\n\nIn a nutshell: **Seismic events --> Waveforms --> Patterns --> Predictions**\n\n\nI made a few hypothesis to guide my choice for the type of machine learning model to be used:\n* The Machine learning model should be able to independantly define the most relevant patterns --> I would not personally define what the patterns are and look like.\n* Patterns can be very limited duration events: patterns may last only a few seconds in a 600sec measurement. Therefore the model should keep the time components of the sensor recordings when doing its analysis.\n* The patterns can happen at anytime within the 600sec recording provided. The pattern can be seen at the beginning, middle or end of the recording, and therefore the model should not be sensitive to the pattern location within the recording.\n* The pattern can be defined by the dynamic within the waveform: For example the waveform can have a sharp increase in amplitude followed by a slow decay. The model need to be able to account for that.\n* One or multiple sensors could record waveforms resulting from the same seismic event. In this case, the recordings from different sensors may complement each other --> Therefore the model needs to be able to analyse multiple sensors together.\n\n\nAll of this convinced me that a **CONV1D based machine learning model** would be a strong candidate. But why ?\n* CNNs are good at independantly detecting commonalities/patterns within the input features. \n* 1D CNNs especially efficient for time series anaylsis.\n* CNNs are very good are treating a large number of input features with a low number of parameters. In this problem, If I were to use the time series as input, each instance could contain up to 60001 (time serie dimension) x 10 (number of sensors) = 600010 input features.\n\n\nOther deciding factors I accounted for:\n* Computation limitation: I will be running everything on my laptop with a Core I7 processor, without GPUs.\n* The inputs of the machine learning model should be manageable in size (couple Gbs at the most)\n* The machine learning model need to be trained within a couple hours at the most on my installation, this will ensure I can quickly iterate through the model structures.\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom scipy import signal","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 1: Data Processing\n\n\n![](https://storage.googleapis.com/kagglesdsdata/datasets/1070142/1803331/2021-01-01_17h25_10.jpg?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=gcp-kaggle-com%40kaggle-161607.iam.gserviceaccount.com%2F20210101%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20210101T163347Z&X-Goog-Expires=259199&X-Goog-SignedHeaders=host&X-Goog-Signature=1fd2c920ef2d579f3141334024e5169f8c86dd91ad649adeb8700e6257161918e436003541f68843559620d5c7525abf0ad653c520241db8764ab0d6371072eb922e6c802a10611399b6e3afb0ded6c845b647ea84253d4d066abae21fc082d3291d622eaf668731d9718dc35e748055fa343699e2a61aac3883e5127ad9e0c8e0c0e7627f138997740b0ef96739fdc9ffa3a48b295b39acac3618d9ebb7be66075b8fe59a67c3e5b8177ff7ca3bbbcf274b8f6c082c23e55cf0bc76b2776feefd813fabb919f7986dfe3c629894d108dbf6055b46ca7331a02e61208240243eee344b499cc24ea8b697fbe146916b733c2fd3943b44620ac4409f2ba998a763)\n\nAlthough I could technically directly pass the raw waveforms time series as inputs to the 1D CNN, I realised a data processing step was necessary in order to:\n* reduce the size of the data, i.e reduce the dimension of the 4431x10 time series of dimensions (600001,1) into a lower dimensional space, \n* rearrange the individual time series (arrays from .csv files) in a way they could be easily consume by the 1D CNN (np.matrices) during the training phase,\n* apply some normalisation and some data augmentation technics (not described here).\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# select an instance, and load the file\nfile = '1969647810'\nfile_path = '../input/predict-volcanic-eruptions-ingv-oe/train/' + file + '.csv'\ndf = pd.read_csv(file_path)      # Load the csv file\n\n# Select a Sensor i_sensor = [0-9]\ni_sensor = 1   \nsensor_raw = df['sensor_' + str(i_sensor+1)]\nprint(np.shape(sensor_raw))\n\n# Time serie parameters:\nfs = 100                                # frequency signal aquisition\nN=60001                                 # duration of the recording\ntime_vect = np.arange(N) / float(fs)\n\n# plot the time series: \nplt.figure()\nplt.plot(time_vect, sensor_raw)\nplt.ylim([-5000,5000])\nplt.ylabel('Amplitude [-]')\nplt.xlabel('Time [sec]')\nplt.title('Raw time serie')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Given the nature for the time series (waveforms) I decided to first proceed with a Short Time Fast Fourier Transformation (STFFT). \n\nThe parameters of the STFFT (time windows and overlap factors) where selected in order to reduce the size of the time dimension (reduction from 60001 to 202) while keeping an acceptable time resolution (3sec).\n\nAn example of the STFFT is given below for one instance and one sensor (1969647810, sensor 2).\nFor the rest of the analysis, only the amplitude of the individual frequency is kept, the phase is not used (this maybe of some later interest, IDK...).\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define parameters for the STFFT'''\nnperseg = 400                           # Time window: number of time points to consider for the 'STFFT analysis\noverlap_factor = 0.25                    # Factor for overlapping the time window\nnoverlap = overlap_factor*nperseg\n\n# stFFT transformation \nf, t, Zxx = signal.stft( sensor_raw, fs, nperseg=nperseg, noverlap=noverlap)       \n\n# Cleaning up the nan and looking at absolute values (amplitude)\nZxx = np.nan_to_num(Zxx)\nstFFT_amp = np.abs(Zxx)\nprint(np.shape(stFFT_amp))\n\n# plotting of the stFFT results \nplt.figure()\nplt.pcolormesh(t, f, stFFT_amp, cmap='inferno', shading='flatgouraud')\nplt.ylabel('Frequency [hz]')\nplt.xlabel('Time [sec]')\nplt.title('STFFT result')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"While the STFFT effectively reduced the time dimension, it created of a new frequency dimension covering the (0-50hz) range with a 0,25Hz resolution. \n\nBoth the range and resolution where not really practical for the application:\n* Most of the sensors would be excited by lower frequencies (0-15Hz) range.\n* Sensor 6,7,8 are exception and will typically showed an higher frequency content, but mostly up to 25hz.\n* The small resolution, brings precision at the expense of the 1D CNN model size and complexity. A 1Hz resolution seemed a better compromise.\n\nTherefore, the next steps of the data processing targeted the reduction of the frequency dimension:\n* First, I discarded the values of frequency 0hz (average of the signal over the time window).\n* Then, the frequency is reduced by a factor of 4, by averaging the value of 4 continuous frequencies.\n* finally, only the frequencies in a (1-25Hz) range are kept, the rest is discarded.\n\nWe end up with a matrice of size (25,202) (frenquency and time dimensions respectively) --> 5050 input features per instance and sensor, an ~88% reduction (1-5050/60000) over the raw time waveforms.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# stFFT parameters\nm = 4431                                                # number of instances in the training dataset\nn_t = 202                                               # Size out of the stFFT in the time dimension\nn_f = 201                                               # Size out of the stFFT in the frequency dimension \nn_s = 10                                                # Size out of the stFFT in the sensor dimension\n\n# dimension reduction parameters\nn_t_reduc = n_t                                         # Size after the dimension reduction in the time dimension\nn_f_reduc = 50                                          # Size after the dimension reduction in the frequency dimension \nn_s_reduc = n_s                                         # Size after the dimension reduction in the sensor dimension\nreduc_f= 4                                              # reduction factor on the frequency\nf = range(1,n_f_reduc+1)                                # new freqency array (used for plotting)\n\n# Output dimension ( after cutting the frequency dimension)\nf_max_out = 25                                          # index of thf last frequency to be outputed\nn_s_out = 10                                            # size for the output\nn_f_out = 25                                            # size for the output\nn_t_out = 202                                           # size for the output\n\n# Redimensionning in the frequency dimension '''  \nstFFT_amp = stFFT_amp[1:,:]   # removal of the 1st row of the frequency (Frequency = 0hz)\nstFFT_amp = np.mean(stFFT_amp.reshape(int(n_f_reduc),reduc_f,int(n_t_reduc)), axis=1)    # averaging on the frequency axis\nstFFT_amp = stFFT_amp[:f_max_out,:]     # removal of the last rows of the frequency\nprint(np.shape(stFFT_amp))\n\n# plotting of the stFFT results \nplt.figure()\nplt.pcolormesh(stFFT_amp, cmap='inferno', shading='flatgouraud')\nplt.title('STFFT result')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above defined data processing steps need to be repeated for each sensor of every instance. When completed, we end up with a matrice of dimension (4431, 25, 202, 10), (instance, frequency, time and sensor dimensions respectively). The matrice can then reshaped, transformed into a panda database and exported into an .hdf file. \n\nThe whole process is not described here as it takes a substantial time to compute, I provide the final result in the data: [*STFFT_train_4431_10_25_202_v5-1.hdf*](https://www.kaggle.com/adriencossa/files-ingv?select=STFFT_train_4431_10_25_202_v5-1.hdf). \n\nA description of the reshaping is done below.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Data import from the .hdf file \nm = 4431                    # number of instances in the dataset\nn_t = 202                   # Size of the stFFT in the time dimension\nn_f = 25                    # Size of the stFFT in the frequency dimension \nn_s = 10                    # Size of the stFFT in the sensor dimension\n\ndf = pd.read_hdf('../input/files-ingv/STFFT_train_4431_10_25_202_v5-1.hdf')  \ndf = df.set_index('segment_id')          # set the index for the dataframe\n        \n# Create a Input (X) and labelling (Y) dataset\nY = df[\"time_to_eruption\"]      # Select only the time to eruption \nX = df.drop(labels = [\"time_to_eruption\"],axis = 1)            # Drop 'time to eruption' label\ndel df      # delete unnecessary variables  \n   \n# Reshape the input data '''\nprint(X.shape)  \nX = X.values.reshape(m, n_s, n_f, n_t)      # reshape the database into a matric of m, n_s, n_f, n_t dimensions --> See input file order      \nX = np.transpose(X, axes=[0, 3, 2, 1])      # reshape the database into a matric of m, n_t, n_f, n_s dimensions --> necessary for machine learning\nprint(X.shape)\n  \n# 2D reshaping:\n# reshape the database into a matric of m, n_t, n_f*n_s dimensions =Stacking the different sensors on top of each other\nX =    np.concatenate((X[:,:,:,0], X[:,:,:,1], X[:,:,:,2], X[:,:,:,3], X[:,:,:,4], X[:,:,:,5], X[:,:,:,6], X[:,:,:,7], X[:,:,:,8], X[:,:,:,9]), axis=2 ) \nprint(X.shape)\n\n# regularisation\nX_max = np.max(X)\nX_min = np.min(X)   # in case the min is not 0  \nprint('  X max : ', X_max, 'X min : ', X_min)\nX = (X-X_min) / (X_max-X_min)\nX_max = np.max(X)\nX_min = np.min(X)   # in case the min is not 0  \nprint('  X max : ', X_max, 'X min : ', X_min)\n\n# check: Ploting one element of the resulting matrice\nfig = plt.figure()\nplt.pcolormesh(X[1,:,:].T, cmap='inferno', shading='flatgouraud')         # Note: transposition is necessary to match the representation\nplt.title('Input of the 1D CNN')\nplt.show()\n    \n# Creating Train / Dev / Test sets '''\nfrom sklearn.model_selection import train_test_split\nrandom_seed = 0 \ntest_size = 0.2     # 20% of the dataset is left for validation\nX_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size = test_size, random_state=random_seed ) \n\ntest_size = 0.5     # Validation is splitted evenly between tain and dev\nX_dev, X_test, Y_dev, Y_test = train_test_split(X_val, Y_val, test_size = test_size, random_state=random_seed ) \n\n# Setting the format to a dataframe \nY_train = Y_train.to_frame()\nY_dev = Y_dev.to_frame()\nY_test = Y_test.to_frame()\ndel X_val, Y_val      # delete unnecessary variables                                              \n                                                  \n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2: CNN definition and training\n\n![-](https://storage.googleapis.com/kagglesdsdata/datasets/1070142/1803331/2021-01-01_17h25_37.jpg?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=gcp-kaggle-com%40kaggle-161607.iam.gserviceaccount.com%2F20210101%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20210101T163502Z&X-Goog-Expires=259199&X-Goog-SignedHeaders=host&X-Goog-Signature=428e14fb804d2eabb697299f5e2df0cc8ea194dfa450a95ac9235001a3f7c48ac37a35716f3385513da5a074005198cf3f4d4f987a5b3b3d5d659ded0fbb39fbab611c9a24172a55741b0cc3dd36f565d05f6ac4366cdd143dd8948d653d56226ee27a30f9f07312cc2d37afff46c9aa34bce8a1bbc8d74f2d6e0d2103afcab2cb4fce19f2b2efb87ae9513b047de05db535b660b5f2a9ac1d16dc6f331dc7fe0ca16b8da26ee28495aeb5b07647306b59275a6458a458262a8d8df272f9870eea5e98a96e040bfc9de30742dc1bfa9efb9af7fd1f48418057ecfb374bd24d51ec72d64fd508d424fc06d75ecf5b3791b4e56a1946412bc42fa21411ebf2a4b5)\n\nThe 1D CNN followed a traditional structure of convolutional, pooling and fully connected layers.\n\nThe structure of the model was defined with the goals of optimising the model predictive accuracy, generalisation, while keeping the complexity in check. After playing with multiple structures, I found the following compromise:\n* The number of convolutional layer is limited to 4\n* the number of filters for each layer is around 100.\n* The size of the filters in the first layer is large than in the following layers (20 vs 5), in order to enable a large pattern detection in the time dimension at an early stage.\n* A global average pooling layer (instead of flattens) reduces the dimensions of the FCs and keeps the number of parameters of the model reasonnable.\n* A few FCs, with a large dropout allow for an ok generalisation.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten, Conv1D, MaxPool1D, GlobalAveragePooling1D\n\n# Model definition\nmodel = Sequential()\n\nmodel.add(Conv1D(filters = 100, \n                 kernel_size = 20,\n                 padding = \"same\",\n                 strides=1,\n                 activation =\"relu\",\n                 input_shape = (n_t, 10*n_f)) )\n\nmodel.add(Conv1D(filters = 100, \n                 kernel_size = 5,\n                 padding = \"same\",\n                 strides=1,\n                 activation =\"relu\" ))\n\nmodel.add(MaxPool1D(pool_size= 2))\n\nmodel.add(Conv1D(filters = 100, \n                 kernel_size = 5,\n                 padding = \"same\",\n                 strides=1,\n                 activation =\"relu\" ))\n\nmodel.add(Conv1D(filters = 100, \n                 kernel_size = 5,\n                 padding = \"same\",\n                 strides=1,\n                 activation =\"relu\" ))\n\nmodel.add(MaxPool1D(pool_size= 2))\n\nmodel.add(GlobalAveragePooling1D()) # averaging along the time axis\n\nmodel.add(Dropout(0.20))\n\nmodel.add(Dense(10,activation = \"relu\"))\n\nmodel.add(Dense(1, activation = \"linear\"))\n\n\n# Print model summary: \nprint(model.summary())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The model is trained with and Adam optimizer and a variable learning rate:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define the  Adam optimizer\nfrom keras.optimizers import Adam\nlr = 0.1 # learning rate start value ---> See the reduction\noptimizer = Adam(learning_rate=lr,\n                 beta_1=0.9,\n                 beta_2=0.999,\n                 epsilon=1e-07)\n\n# Define Loss function and metrics \nloss = \"mean_squared_error\"\nmetrics= \"mean_squared_error\", \"mean_absolute_error\"\nmodel.compile(optimizer = optimizer , loss = loss, metrics=metrics)\n\n\nepochs = 15 # to be adjusted for complete training  \nbatch_size = 512\n\n# Set a learning rate reduction. learning rate is multiplied y factor 0.5 if val mean squared error is not improved 5 times in a row\nfrom keras.callbacks import ReduceLROnPlateau\nlearning_rate_reduction = ReduceLROnPlateau(monitor='val_mean_squared_error', \n                                            patience=5, \n                                            verbose=2, \n                                            factor=0.5, \n                                            min_lr=0.001)\n\nmodel_training_history = model.fit(    X_train, Y_train, \n                                       batch_size = batch_size, \n                                       epochs = epochs,\n                                       callbacks=[learning_rate_reduction], \n                                       validation_data = (X_test, Y_test), \n                                       verbose = 1)\n\n# Plotting the model training metrics\nfig = plt.figure()\nplt.plot(model_training_history.history['mean_squared_error'], color='r')\nplt.plot(model_training_history.history['val_mean_squared_error'], color='b')\nplt.title('Model training')\nplt.show()\n\nprint('Final Training Mean Squared error : ', model_training_history.history['mean_squared_error'][-1])\nprint('Final Validation Mean Squared error : ', model_training_history.history['val_mean_squared_error'][-1])\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3: Results discussion\n\nFor simplicity, I provide an example of the model trained over 160 epochs (Training time = 1300sec on my machine).\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.models import load_model\nmodel = load_model('../input/files-ingv/Model_9-all-7_input_5-1reg.h5')\n\n\n# Model Evaluation\ndef evaluation (X, Y, title=''): \n    # Standard function for plotting the evaluation\n    print(title)\n    \n    # computation of the predion\n    Y_pred = Y # intialisation\n    Y_pred['time_to_eruption_pred'] = np.reshape(model.predict(X),(-1,)).tolist()       # add the result of the prediction    \n    Y_pred['Error'] = Y_pred['time_to_eruption'] - Y_pred['time_to_eruption_pred']    \n    Y_pred['Abs_Error'] = abs(Y_pred['Error'])\n    Y_pred['Squared_Error'] = Y_pred['Abs_Error']*Y_pred['Abs_Error']\n    \n    # Print Mean Abs Error\n    print('Mean Abs Error: \\n', Y_pred['Abs_Error'].mean() ,'\\n')    \n    \n    # print Mean square error  \n    print('Mean Sqarred Error: \\n', Y_pred['Squared_Error'].mean() ,'\\n')    \n    \n    # Plotting the Y / Ypred curve\n    fig , ((ax1, ax2)) = plt.subplots(ncols=1, nrows=2, sharex=False,  figsize=(12,8), constrained_layout=True, gridspec_kw={'height_ratios':[1,1]})        \n    ax1.set_title(title)\n    ax1.set(xlabel='Real value', ylabel='predion')   \n    ax1.plot([0, 50000000],[0, 50000000],color='b')     # print red line\n    ax1.scatter(Y_pred['time_to_eruption'],Y_pred['time_to_eruption_pred'])\n    ax1.set_ylim(0, 50000000)\n    ax1.set_xlim(0, 50000000)\n    ax1.grid()\n    \n    ax2.set(xlabel='Real value', ylabel='predion error')   \n    ax2.plot([0, 50000000],[0,0],color='b')         # print red line\n    ax2.scatter(Y_pred['time_to_eruption'],Y_pred['Error'])\n    ax2.set_xlim(0, 50000000)\n    ax2.set_ylim(-25000000, 25000000)\n    ax2.grid()\n    \n    plt.show()          \n    \n    return \n\n\n# Evaluation of the Training, Dev and Testing Set\n_ = evaluation (X_train, Y_train, title='Evaluation of the Train dataset')\n_ = evaluation (X_dev, Y_dev, title='Evaluation of the Dev dataset')\n_ = evaluation (X_test, Y_test, title='Evaluation of the Test dataset')\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The evaluation of the error on the training, testing and dev datasets show that:\n* The model was successfull at tendancially predicting the time to eruption.\n* The residual error is largely independant of the real time to eruption. The model does not overfit one part of the dataset\n* The model is moderately overfitting, the mean average square error of the dev, test and train datasets are comparable\n\nWas it however sufficient to achieve a high score in the kaggle leaderboard ? Not really...\n\nWhen predicting the Kaggle testing dataset, I achieved a score around 6,4E+06 which is ... average. \n\nMoreover, after playing with the data processing, data augmentation, model definition and training, I found very little opportunity for improvement.\n\nHence the following questions:\n* Are my starting hypothesis valid on their own? Do the waveforms contain clear recogniseable patterns, or are those too random ?\n* Is the study of patterns in the waveforms possible on 600sec measurements? would it be more effective on longer measurements or in another context ?\n* Is the number of instances provided for the training (4431) sufficient to train this 1D CNN? Some studies suggest a 1D CNN is hard to train with a low number of instances\n"},{"metadata":{},"cell_type":"markdown","source":"# Cool reads:\n\nAlong the way I have found a lot of cool articles and blog posts, that I used for inspiration on this task:\n\nVolcano Stuff and discussion on the frequency content of the signals:\n* https://www.usgs.gov/natural-hazards/volcano-hazards/monitoring-volcano-seismicity-provides-insight-volcanic-structure\n* https://avo.alaska.edu/about/infrasound.php\n* https://en.wikipedia.org/wiki/Harmonic_tremor\n\nWork of Bernard Chouet on patterns in volcano measurement, and in particular Long-Period Events:\n* http://www.bbc.co.uk/science/horizon/2001/volcanohell.shtml\n* https://www.pbs.org/wgbh/nova/volcano/chouet.html\n* https://www.researchgate.net/profile/Phillip_Dawson/publication/253644552_A_characterization_of_long-period_events_recorded_during_the_eruptive_activity_of_Mount_Etna_Italy_1992/links/00b49527d4a4b6c1bd000000.pdf\n* https://reader.elsevier.com/reader/sd/pii/S0377027301002992?token=12418CC88C9DB11F8A6B6DC796539E5D7EA87DE7902EB62DD0D5841B1BE8B217BFF05FDE179B2260F09F62FBEB349EBF\n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}