{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Projet 8 : G2Net Gravitational Wave Detection\n\nThis work was an attempt to participate in a Kaggle competition : https://www.kaggle.com/c/g2net-gravitational-wave-detection/overview\n\nThe aim of the chosen competition is to build a classifier to detect gravitational wave in a signal. This is an oportunity to work on time series and to participate in an effort to improve gravitational wave detection using machine learning. The competition host would love if Kaggle winners were able to outdo the standard approaches.\n\n","metadata":{}},{"cell_type":"markdown","source":"Kaggle competition is a great way to learn state of the art techniques. I have spent a lot of time reading kernels and comments to know what works and what doesn't. I started the competition 7 days before the end so I had to go directly in the right direction.\n\nI have come to the conclusion that :\n* Almost everyone transform time series into images to work with CNN\n* The optimal transformation for this project is CQT transform\n* A lot of high score use transfer learning models : EfficientNetB7 seems to be the best\n\nThis notebook was inspired by useful kernels: \n\nhttps://www.kaggle.com/coldfir3/cqt-dataset-generator-rgb-jpg\n    \nhttps://www.kaggle.com/esratmaria/gravitational-wave-detection-simple-cnn-model\n\n","metadata":{}},{"cell_type":"markdown","source":"## Plan\n* Visualisation\n* Preprocessing\n* Modelling\n* Results","metadata":{}},{"cell_type":"code","source":"%%capture\n!python -m pip install gwpy\n!pip install astropy==4.2.1","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:56:56.012514Z","iopub.execute_input":"2021-09-30T09:56:56.013172Z","iopub.status.idle":"2021-09-30T09:57:09.761518Z","shell.execute_reply.started":"2021-09-30T09:56:56.013046Z","shell.execute_reply":"2021-09-30T09:57:09.760617Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport shutil\nfrom glob import glob\nfrom tqdm.auto import tqdm\nfrom joblib import Parallel, delayed\nfrom pathlib import Path\n\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nfrom tensorflow.keras.metrics import AUC\nfrom sklearn.model_selection import train_test_split\n\nfrom gwpy.timeseries import TimeSeries\nfrom gwpy.plot import Plot\nfrom scipy import signal\nfrom PIL import Image\n\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:09.764300Z","iopub.execute_input":"2021-09-30T09:57:09.764935Z","iopub.status.idle":"2021-09-30T09:57:13.780385Z","shell.execute_reply.started":"2021-09-30T09:57:09.764887Z","shell.execute_reply":"2021-09-30T09:57:13.779593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualisation","metadata":{}},{"cell_type":"code","source":"train_path = glob('../input/g2net-gravitational-wave-detection/train/*/*/*/*')\ntest_path = glob('../input/g2net-gravitational-wave-detection/test/*/*/*/*')","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:13.781618Z","iopub.execute_input":"2021-09-30T09:57:13.781848Z","iopub.status.idle":"2021-09-30T09:57:19.940104Z","shell.execute_reply.started":"2021-09-30T09:57:13.781824Z","shell.execute_reply":"2021-09-30T09:57:19.939170Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Number of train files : {len(train_path)} which represents {len(train_path)/(len(train_path) + len(test_path))*100:.2f} % of all data')\nprint(f'Number of test files : {len(test_path)} which represents {len(test_path)/(len(train_path) + len(test_path))*100:.2f} % of all data')","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:19.941451Z","iopub.execute_input":"2021-09-30T09:57:19.942364Z","iopub.status.idle":"2021-09-30T09:57:19.947429Z","shell.execute_reply.started":"2021-09-30T09:57:19.942325Z","shell.execute_reply":"2021-09-30T09:57:19.946526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Loading a file","metadata":{}},{"cell_type":"code","source":"file_path = '../input/g2net-gravitational-wave-detection/train/0/0/0/000a5b6e5c.npy'\nx_series = np.load(file_path)\nx_series.shape","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:19.949520Z","iopub.execute_input":"2021-09-30T09:57:19.949788Z","iopub.status.idle":"2021-09-30T09:57:19.964582Z","shell.execute_reply.started":"2021-09-30T09:57:19.949758Z","shell.execute_reply":"2021-09-30T09:57:19.963677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Each file is composed of 3 signals, one per gravitational waves detector. ","metadata":{}},{"cell_type":"code","source":"colors = ['red', 'green', 'blue']\nsignal_names = ['LIGO Hanford', 'LIGO Livingston', 'Virgo']\n\nplt.figure(figsize=(16, 7))\nfor i in range(3):\n    plt.subplot(3, 1, i+1)\n    plt.plot(x_series[i], color=colors[i])\n    plt.legend([signal_names[i]], fontsize=12, loc=\"lower right\")\n","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:19.966086Z","iopub.execute_input":"2021-09-30T09:57:19.966376Z","iopub.status.idle":"2021-09-30T09:57:20.756321Z","shell.execute_reply.started":"2021-09-30T09:57:19.966345Z","shell.execute_reply":"2021-09-30T09:57:20.755540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Each signal correspond to a time series of 2 seconds duration and have a sampling of 2048 Hz.","metadata":{}},{"cell_type":"markdown","source":"### Spectrogram","metadata":{}},{"cell_type":"markdown","source":"One way of working with times series is to transform the signal from the time domain to the frequency domain. The we can work on a spectrogram and apply machine learning model used for image classification.\n\nIn this competition all of the competitors used a CQT transform which seems to gave the best results. The advantage of CQT transform is that the y axis corresponding to the frequency has a log scale. This technique is used for audio signal and a log scale give a representation wich is close to the human perception.","metadata":{}},{"cell_type":"code","source":"def sig2rgb(fname, whiten = True, window=0.2, bandpass=True, f_range = (20,500), q_range = (16,32), q_max = 10):\n    \n    # Load the file \n    data = np.load(fname)\n    # Split each chanel and convert to TimeSeries\n    data = map(lambda x: TimeSeries(x, sample_rate=2048), data)\n    # Whiten the signal and apply a tukey window\n    data = map(lambda x: x.whiten(window=(\"tukey\", window)), data)\n    # (optional) bandpass filter\n    if bandpass:\n        data = map(lambda x: x.bandpass(*f_range), data)\n    # Q-transform\n    data = map(lambda x: x.q_transform(qrange=q_range, frange=f_range, logf=True, whiten=False), data)\n    # Convert to RGB image\n    img = np.stack(list(data), axis = -1)\n    img = np.clip(img, 0, q_max)/q_max * 255\n    img = img.astype(np.uint8)\n    img = Image.fromarray(img).rotate(90, expand=1)\n    img = img.resize((512,512), Image.ANTIALIAS)\n    return img","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:20.757397Z","iopub.execute_input":"2021-09-30T09:57:20.758281Z","iopub.status.idle":"2021-09-30T09:57:20.768377Z","shell.execute_reply.started":"2021-09-30T09:57:20.758234Z","shell.execute_reply":"2021-09-30T09:57:20.767336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In this notebook I used this function to generate spectrogram, inspired from this kernel : https://www.kaggle.com/coldfir3/cqt-dataset-generator-rgb-jpg","metadata":{}},{"cell_type":"code","source":"sig2rgb('../input/g2net-gravitational-wave-detection/train/0/0/0/000a5b6e5c.npy')","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:20.769746Z","iopub.execute_input":"2021-09-30T09:57:20.770017Z","iopub.status.idle":"2021-09-30T09:57:21.344573Z","shell.execute_reply.started":"2021-09-30T09:57:20.769985Z","shell.execute_reply":"2021-09-30T09:57:21.343371Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This RGB spectrogram is a superposition of the 3 signal of all 3 gravitational wave interferometers (one per color Red, Green, Blue). It is a clever idea because it condense the information so we can work on smaller images and also we can used a transfer learning approach because pre-trained networks are trained on RGB images.","metadata":{}},{"cell_type":"markdown","source":"## Preprocessing","metadata":{}},{"cell_type":"markdown","source":"The idea of this part is to generate a dataset of RGB spectrogram using the sig2rgb function. One important note :\n* Applying a filter to remove certain frequency is a key element to filter out the noise\n* According to a discussion in the competition the 20 to 500 Hz band is the optimal band for gravitational waves","metadata":{}},{"cell_type":"code","source":"def save_img(x, folder_out, **kwargs):\n    fname = Path('../input/g2net-gravitational-wave-detection/' + folder_out.split('_')[0] + '/' + '/'.join([x[0], x[1], x[2], x]) + '.npy')\n    file_out = folder_out + '/' + fname.with_suffix('.jpg').name\n    x = sig2rgb(fname, **kwargs)\n    x.save(file_out)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:21.346469Z","iopub.execute_input":"2021-09-30T09:57:21.346846Z","iopub.status.idle":"2021-09-30T09:57:21.353467Z","shell.execute_reply.started":"2021-09-30T09:57:21.346795Z","shell.execute_reply":"2021-09-30T09:57:21.352535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fast_sub = False # set this to False to generate the whole dataset\ntrain = False # set this to True to generate the train set\ntest = False # set this to True to generate the test set","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:21.355076Z","iopub.execute_input":"2021-09-30T09:57:21.355529Z","iopub.status.idle":"2021-09-30T09:57:21.366563Z","shell.execute_reply.started":"2021-09-30T09:57:21.355484Z","shell.execute_reply":"2021-09-30T09:57:21.365417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.read_csv('../input/g2net-gravitational-wave-detection/training_labels.csv')\nif not os.path.isdir('train_cqt_rgb'):\n    if fast_sub: train_ids = train_df['id'][:10000]\n    else: train_ids = train_df['id']\n    if train:\n        os.makedirs('train_cqt_rgb', exist_ok = True)\n        o = Parallel(n_jobs=-1)(delayed(save_img)(x, 'train_cqt_rgb') for x in tqdm(train_ids))","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:21.367955Z","iopub.execute_input":"2021-09-30T09:57:21.368652Z","iopub.status.idle":"2021-09-30T09:57:21.728565Z","shell.execute_reply.started":"2021-09-30T09:57:21.368605Z","shell.execute_reply":"2021-09-30T09:57:21.727500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv('../input/g2net-gravitational-wave-detection/sample_submission.csv')\nif not os.path.isdir('test_cqt_rgb'):\n    if fast_sub: test_ids = test_df['id'][:10000]\n    else: test_ids = test_df['id']\n    if test:\n        os.makedirs('test_cqt_rgb', exist_ok = True)\n        o = Parallel(n_jobs=-1)(delayed(save_img)(x, 'test_cqt_rgb') for x in tqdm(test_ids))","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:21.729756Z","iopub.execute_input":"2021-09-30T09:57:21.729996Z","iopub.status.idle":"2021-09-30T09:57:21.878115Z","shell.execute_reply.started":"2021-09-30T09:57:21.729969Z","shell.execute_reply":"2021-09-30T09:57:21.877299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Modelling","metadata":{}},{"cell_type":"code","source":"results_dataset_1percent_path = '../input/results-second-try/'\nresults_dataset_100percent_path = '../input/resultsthirdtry/'\nos.makedirs(results_dataset_1percent_path, exist_ok=True)\nos.makedirs(results_dataset_100percent_path, exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:21.879402Z","iopub.execute_input":"2021-09-30T09:57:21.879663Z","iopub.status.idle":"2021-09-30T09:57:21.886768Z","shell.execute_reply.started":"2021-09-30T09:57:21.879634Z","shell.execute_reply":"2021-09-30T09:57:21.885878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_size = (512, 512)\nimg_shape = (512, 512, 3)\nbatch_size = 16\nimg_path = './train_cqt_rgb/'","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:21.890781Z","iopub.execute_input":"2021-09-30T09:57:21.891046Z","iopub.status.idle":"2021-09-30T09:57:21.896240Z","shell.execute_reply.started":"2021-09-30T09:57:21.891015Z","shell.execute_reply":"2021-09-30T09:57:21.895496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv('../input/g2net-gravitational-wave-detection/training_labels.csv')\n# df = df[:10000]","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:21.897746Z","iopub.execute_input":"2021-09-30T09:57:21.898352Z","iopub.status.idle":"2021-09-30T09:57:22.236338Z","shell.execute_reply.started":"2021-09-30T09:57:21.898309Z","shell.execute_reply":"2021-09-30T09:57:22.235250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X = df['id']\ny = df['target'].astype('int8').values","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.237721Z","iopub.execute_input":"2021-09-30T09:57:22.237944Z","iopub.status.idle":"2021-09-30T09:57:22.242430Z","shell.execute_reply.started":"2021-09-30T09:57:22.237918Z","shell.execute_reply":"2021-09-30T09:57:22.241847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x_train, x_valid, y_train, y_valid = train_test_split(X, y, random_state = 42, stratify = y)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.243563Z","iopub.execute_input":"2021-09-30T09:57:22.243952Z","iopub.status.idle":"2021-09-30T09:57:22.581780Z","shell.execute_reply.started":"2021-09-30T09:57:22.243924Z","shell.execute_reply":"2021-09-30T09:57:22.581167Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_filepath(id_, is_train=True):\n    path = ''\n    if is_train:\n        return f'./train_cqt_rgb/{id_}.jpg'\n    else:\n        return f'./test_cqt_rgb/{id_}.jpg'","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.582915Z","iopub.execute_input":"2021-09-30T09:57:22.583216Z","iopub.status.idle":"2021-09-30T09:57:22.588418Z","shell.execute_reply.started":"2021-09-30T09:57:22.583183Z","shell.execute_reply":"2021-09-30T09:57:22.587495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def imgFromPath(file_path: tf.Tensor, y=None, input_shape=img_shape):\n    file = tf.io.read_file(file_path)\n    x = tf.image.decode_image(file)\n    x = tf.ensure_shape(x, input_shape)\n    if y is None:\n        return x\n    else:\n        return x, y","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.589818Z","iopub.execute_input":"2021-09-30T09:57:22.590169Z","iopub.status.idle":"2021-09-30T09:57:22.600372Z","shell.execute_reply.started":"2021-09-30T09:57:22.590100Z","shell.execute_reply":"2021-09-30T09:57:22.599514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def makeDataset(x_train, x_valid, batch_size):\n\n    train_dataset = tf.data.Dataset.from_tensor_slices((x_train.apply(get_filepath).values, y_train))\n    # shuffle the dataset\n    train_dataset = train_dataset.shuffle(len(x_train))\n    train_dataset = train_dataset.map(imgFromPath)\n    train_dataset = train_dataset.batch(batch_size)\n    train_dataset = train_dataset.prefetch(tf.data.AUTOTUNE)\n\n    valid_dataset = tf.data.Dataset.from_tensor_slices((x_valid.apply(get_filepath).values, y_valid))\n    valid_dataset = valid_dataset.map(imgFromPath)\n    valid_dataset = valid_dataset.batch(batch_size)\n    valid_dataset = valid_dataset.prefetch(tf.data.AUTOTUNE)\n   \n    return train_dataset, valid_dataset","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.601974Z","iopub.execute_input":"2021-09-30T09:57:22.602306Z","iopub.status.idle":"2021-09-30T09:57:22.613530Z","shell.execute_reply.started":"2021-09-30T09:57:22.602265Z","shell.execute_reply":"2021-09-30T09:57:22.612462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def makeModel(img_shape):\n      \n   model_kwargs = dict(\n       include_top=False,\n       weights='imagenet',\n       input_tensor=None,\n       input_shape=img_shape,\n       pooling=None,\n       classes=1000)\n   \n   base_model = tf.keras.applications.efficientnet.EfficientNetB7(**model_kwargs)\n   base_model.trainable = False\n   base_model.summary()\n      \n   data_augmentation = tf.keras.Sequential([\n           tf.keras.layers.experimental.preprocessing.RandomFlip(\"horizontal\")])\n#            tf.keras.layers.experimental.preprocessing.RandomRotation(0.2), \n#            tf.keras.layers.experimental.preprocessing.RandomZoom(height_factor=(0.2, 0.3), width_factor=(0.2, 0.3)),\n#            tf.keras.layers.experimental.preprocessing.RandomTranslation(0.3, 0.3, fill_mode='reflect', interpolation='bilinear')])\n\n   \n   global_avg_layer = tf.keras.layers.GlobalAveragePooling2D()\n   prediction_layer = tf.keras.layers.Dense(1, activation='sigmoid')\n   \n   inputs = tf.keras.Input(img_shape)\n   prepro = tf.keras.applications.efficientnet.preprocess_input(inputs)\n   augmented = data_augmentation(prepro)    \n   x = base_model(augmented, training=False)\n   x = global_avg_layer(x)\n   outputs = prediction_layer(x)\n   model = tf.keras.Model(inputs, outputs)\n   \n   model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=1e-4),\n             loss='binary_crossentropy',\n             metrics=[[AUC(), 'accuracy']])\n   \n   return model","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.614548Z","iopub.execute_input":"2021-09-30T09:57:22.614776Z","iopub.status.idle":"2021-09-30T09:57:22.629470Z","shell.execute_reply.started":"2021-09-30T09:57:22.614752Z","shell.execute_reply":"2021-09-30T09:57:22.628697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def evaluateModel(model, train_dataset, valid_dataset, results_path):\n    \n    os.makedirs(results_path, exist_ok=True)\n\n    if os.path.exists(results_path + 'history.npy'):\n        history_dict = np.load(results_path + 'history.npy',allow_pickle='TRUE').item()\n    else:\n        history = model.fit(\n        train_dataset,\n        epochs=3,\n        validation_data=valid_dataset)\n\n        history_dict = history.history\n        np.save(results_path + 'history.npy',history_dict)\n        model.save(results_path + 'model.h5')\n        \n    return history_dict","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.630584Z","iopub.execute_input":"2021-09-30T09:57:22.630790Z","iopub.status.idle":"2021-09-30T09:57:22.646619Z","shell.execute_reply.started":"2021-09-30T09:57:22.630768Z","shell.execute_reply":"2021-09-30T09:57:22.645764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dataset, valid_dataset = makeDataset(x_train, x_valid, batch_size)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:22.647732Z","iopub.execute_input":"2021-09-30T09:57:22.648072Z","iopub.status.idle":"2021-09-30T09:57:23.104948Z","shell.execute_reply.started":"2021-09-30T09:57:22.648044Z","shell.execute_reply":"2021-09-30T09:57:23.104348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dataset","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:23.106051Z","iopub.execute_input":"2021-09-30T09:57:23.106316Z","iopub.status.idle":"2021-09-30T09:57:23.113304Z","shell.execute_reply.started":"2021-09-30T09:57:23.106289Z","shell.execute_reply":"2021-09-30T09:57:23.112305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = makeModel(img_shape)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:23.114740Z","iopub.execute_input":"2021-09-30T09:57:23.115542Z","iopub.status.idle":"2021-09-30T09:57:33.684515Z","shell.execute_reply.started":"2021-09-30T09:57:23.115498Z","shell.execute_reply":"2021-09-30T09:57:33.683554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history_dataset_1percent = evaluateModel(model, train_dataset, valid_dataset, results_dataset_1percent_path)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:33.685918Z","iopub.execute_input":"2021-09-30T09:57:33.686176Z","iopub.status.idle":"2021-09-30T09:57:33.703859Z","shell.execute_reply.started":"2021-09-30T09:57:33.686122Z","shell.execute_reply":"2021-09-30T09:57:33.702852Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history_dataset_100percent = evaluateModel(model, train_dataset, valid_dataset, results_dataset_100percent_path)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:33.705647Z","iopub.execute_input":"2021-09-30T09:57:33.705867Z","iopub.status.idle":"2021-09-30T09:57:33.718714Z","shell.execute_reply.started":"2021-09-30T09:57:33.705843Z","shell.execute_reply":"2021-09-30T09:57:33.717748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Results","metadata":{}},{"cell_type":"code","source":"acc = history_dataset_1percent['accuracy']\nval_acc = history_dataset_1percent['val_accuracy']\n\nloss = history_dataset_1percent['loss']\nval_loss = history_dataset_1percent['val_loss']\n\nplt.figure(figsize=(8, 8))\nplt.subplot(2, 1, 1)\nplt.plot(acc, label='Training Accuracy')\nplt.plot(val_acc, label='Validation Accuracy')\nplt.legend(loc='lower right')\nplt.ylabel('Accuracy')\nplt.title('Training and Validation Accuracy')\n\nplt.subplot(2, 1, 2)\nplt.plot(loss, label='Training Loss')\nplt.plot(val_loss, label='Validation Loss')\nplt.legend(loc='upper right')\nplt.ylabel('Binary Cross Entropy')\nplt.title('Training and Validation Loss')\nplt.xlabel('epoch')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:33.720290Z","iopub.execute_input":"2021-09-30T09:57:33.720714Z","iopub.status.idle":"2021-09-30T09:57:34.561362Z","shell.execute_reply.started":"2021-09-30T09:57:33.720665Z","shell.execute_reply":"2021-09-30T09:57:34.560292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"acc = history_dataset_100percent['accuracy']\nval_acc = history_dataset_100percent['val_accuracy']\n\nauc = history_dataset_100percent['auc_1']\nval_auc = history_dataset_100percent['val_auc_1']\n\nloss = history_dataset_100percent['loss']\nval_loss = history_dataset_100percent['val_loss']\n\nplt.figure(figsize=(10, 14))\nplt.subplot(3, 1, 1)\nplt.plot(acc, label='Training Accuracy')\nplt.plot(val_acc, label='Validation Accuracy')\nplt.legend(loc='lower right')\nplt.ylabel('Accuracy')\nplt.title('Training and Validation Accuracy')\n\nplt.subplot(3, 1, 2)\nplt.plot(auc, label='Training AUC')\nplt.plot(val_auc, label='Validation AUC')\nplt.legend(loc='upper right')\nplt.ylabel('AUC')\nplt.title('Training and Validation Loss')\n\nplt.subplot(3, 1, 3)\nplt.plot(loss, label='Training Loss')\nplt.plot(val_loss, label='Validation Loss')\nplt.legend(loc='upper right')\nplt.ylabel('Binary Cross Entropy')\nplt.title('Training and Validation Loss')\nplt.xlabel('epoch')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:34.562653Z","iopub.execute_input":"2021-09-30T09:57:34.562909Z","iopub.status.idle":"2021-09-30T09:57:35.382009Z","shell.execute_reply.started":"2021-09-30T09:57:34.562879Z","shell.execute_reply":"2021-09-30T09:57:35.380987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"acc_1p = history_dataset_1percent['accuracy']\nval_acc_1p = history_dataset_1percent['val_accuracy']\nacc_100p = history_dataset_100percent['accuracy']\nval_acc_100p = history_dataset_100percent['val_accuracy']\n\nloss_1p = history_dataset_1percent['loss']\nval_loss_1p = history_dataset_1percent['val_loss']\nloss_100p = history_dataset_100percent['loss']\nval_loss_100p = history_dataset_100percent['val_loss']\n\nplt.figure(figsize=(8, 8))\nplt.subplot(2, 1, 1)\nplt.plot(acc_1p, label='Training dataset 1%')\nplt.plot(val_acc_1p, label='Validation dataset 1%')\nplt.plot(acc_100p, label='Training dataset 100%')\nplt.plot(val_acc_100p, label='Validation dataset 100%')\nplt.legend(loc='lower right')\nplt.ylabel('Accuracy')\nplt.title('Training and Validation Accuracy')\n\nplt.subplot(2, 1, 2)\nplt.plot(loss_1p, label='Training dataset 1%')\nplt.plot(val_loss_1p, label='Validation dataset 1%')\nplt.plot(loss_100p, label='Training dataset 100%')\nplt.plot(val_loss_100p, label='Validation dataset 100%')\nplt.legend(loc='upper right')\nplt.ylabel('Binary Cross Entropy')\nplt.title('Training and Validation Loss')\nplt.xlabel('epoch')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:35.383538Z","iopub.execute_input":"2021-09-30T09:57:35.383858Z","iopub.status.idle":"2021-09-30T09:57:36.017026Z","shell.execute_reply.started":"2021-09-30T09:57:35.383818Z","shell.execute_reply":"2021-09-30T09:57:36.016135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Prediction","metadata":{}},{"cell_type":"code","source":"x_test = pd.read_csv('../input/g2net-gravitational-wave-detection/sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:36.018328Z","iopub.execute_input":"2021-09-30T09:57:36.018585Z","iopub.status.idle":"2021-09-30T09:57:36.171891Z","shell.execute_reply.started":"2021-09-30T09:57:36.018555Z","shell.execute_reply":"2021-09-30T09:57:36.170911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test dataset\ntest_dataset = tf.data.Dataset.from_tensor_slices((x_test['id'].apply(get_filepath, is_train=False).values))\ntest_dataset = test_dataset.map(imgFromPath)\ntest_dataset = test_dataset.batch(batch_size)\ntest_dataset = test_dataset.prefetch(tf.data.AUTOTUNE)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:57:36.175590Z","iopub.execute_input":"2021-09-30T09:57:36.175853Z","iopub.status.idle":"2021-09-30T09:57:36.311830Z","shell.execute_reply.started":"2021-09-30T09:57:36.175822Z","shell.execute_reply":"2021-09-30T09:57:36.310927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predict = False\n\nif predict:\n    prediction = model.predict(test_dataset)\n    submission = pd.DataFrame({'id': x_test.id, 'target': prediction.flatten()})\n    submission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T09:58:29.889614Z","iopub.execute_input":"2021-09-30T09:58:29.889913Z","iopub.status.idle":"2021-09-30T09:58:29.895054Z","shell.execute_reply.started":"2021-09-30T09:58:29.889885Z","shell.execute_reply":"2021-09-30T09:58:29.894141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}