{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7392733,"sourceType":"datasetVersion","datasetId":4297749},{"sourceId":7465251,"sourceType":"datasetVersion","datasetId":4317718}],"dockerImageVersionId":30636,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"mport numpy as np  # Add this line to import NumPy\n\nfrom sklearn.metrics import precision_recall_fscore_support\n\n# Assuming ⁠ true_labels ⁠ are the true labels and ⁠ predicted_labels ⁠ are the predicted labels\n# These should be arrays of shape (num_samples,)\ntrue_labels = np.argmax('true', axis=0)  # Convert one-hot encoded true labels to categorical labels if needed\npredicted_labels = np.argmax(pred, axis=0)  # Convert one-hot encoded predictions to categorical labels if needed\n\n# Calculate precision, recall, F1 score, and support\nprecision, recall, f1_score, support = precision_recall_fscore_support(true_labels, predicted_labels, average='weighted')\n\n# Print the results\nprint(f'Precision: {precision:.4f}')\nprint(f'Recall: {recall:.4f}')\nprint(f'F1 Score: {f1_score:.4f}')\nprint(f'Support: {support}')# WaveNet Starter using RAW EEG Features!\nThis notebook is a WaveNet starter for Kaggle's Brain comp. It achieves `CV 0.91` and `LB 0.66`. Note that submitting train means achieves `CV 1.26` and `LB 0.97` [[here][1]]. So this notebook's WaveNet is successfully learning to predict brain events from raw EEG waveforms!\n\nThis model only uses two features. We can engineer more features and/or modify the model architecture to improve CV score and LB score. Furthermore we can build 1 model which inputs both spectrogram images and eeg waveforms. The two EEG features in this notebook are:\n* feature 1 : `Fp1 minus O1`\n* feature 2 : `Fp2 minus O2`\n\nFeature 1 is the beginning of the montage chains `LL` and `LP` minus the ending of montage `LL` and `LP`. And feature 2 is the beginning of the montage chains `RL` and `RP` minus the ending of montage `RL` and `RP`.\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Jan-2024/montage.png)\n\n# UPDATE\nIn version 7 and 8, we add more features and update the model architecture to evaluate each montage chain separately and then concatenate the features. This new architecture is motivated by the discovery of a better formula to utilize EEG explained in discussion [here][2]\n\n* Version 5,6: Use 2 features - CV 0.91 LB 0.66\n* Version 7,8: Use 8 features grouped as 4 chains. Downsample time 5x - **CV 0.81 LB 0.53**, wow!\n\nWe train our new model in version 7, then save model weights. Then load model into version 8 to submit to LB.\n\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Jan-2024/wave-model.png)\n\n# Train and Infer Tricks\nWe train the fold models in version 7 of the notebook and submit to Kaggle LB in version 8 of the notebook. This makes submission faster because we train the fold models for 30 minutes in version 7 then save them. In version 8, we just load the models without needing to retrain models during Kaggle submit. (And we train our old model in 5 annd submit in 6).\n\nVersion 4 uses `1xP100` GPU with full precision and takes 1 hour to train 5 folds 5 epochs of WaveNet. Version 5 uses `2xT4` GPU with mixed precision and takes 30 minutes to train 5 folds 5 epochs of WaveNet. \n\n[1]: https://www.kaggle.com/code/seshurajup/eda-train-csv\n[2]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/469760","metadata":{}},{"cell_type":"markdown","source":"# Load Train Data","metadata":{}},{"cell_type":"code","source":"import pandas as pd, numpy as np, os\nimport matplotlib.pyplot as plt\n\ntrain = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\nprint( train.shape )\ndisplay( train.head() )\n\n# CHOICE TO CREATE OR LOAD EEGS FROM NOTEBOOK VERSION 1\nCREATE_EEGS = False\nTRAIN_MODEL = False ","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:20:26.91065Z","iopub.execute_input":"2024-06-26T11:20:26.910927Z","iopub.status.idle":"2024-06-26T11:20:28.132614Z","shell.execute_reply.started":"2024-06-26T11:20:26.910901Z","shell.execute_reply":"2024-06-26T11:20:28.131585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Raw EEG Features","metadata":{}},{"cell_type":"code","source":"df = pd.read_parquet('/kaggle/input/hms-harmful-brain-activity-classification/train_eegs/1000913311.parquet')\nFEATS = df.columns\nprint(f'There are {len(FEATS)} raw eeg features')\nprint( list(FEATS) )","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:20:31.036289Z","iopub.execute_input":"2024-06-26T11:20:31.036649Z","iopub.status.idle":"2024-06-26T11:20:31.235859Z","shell.execute_reply.started":"2024-06-26T11:20:31.03661Z","shell.execute_reply":"2024-06-26T11:20:31.234894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"print('We will use the following subset of raw EEG features:')\nFEATS = ['Fp1',  'C3',   'T3', 'O1', 'Fp2',  'C4', 'T4', 'O2']\nFEAT2IDX = {x:y for x,y in zip(FEATS,range(len(FEATS)))}\nprint( list(FEATS) )","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:20:33.075983Z","iopub.execute_input":"2024-06-26T11:20:33.076794Z","iopub.status.idle":"2024-06-26T11:20:33.084154Z","shell.execute_reply.started":"2024-06-26T11:20:33.07675Z","shell.execute_reply":"2024-06-26T11:20:33.083233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def eeg_from_parquet(parquet_path, display=False):\n    \n    # EXTRACT MIDDLE 50 SECONDS\n    eeg = pd.read_parquet(parquet_path, columns=FEATS)\n    rows = len(eeg)\n    offset = (rows-10_000)//2\n    eeg = eeg.iloc[offset:offset+10_000] #iloc is a Pandas method used for integer-location based indexing./offset is the starting index from where the slicing begins./offset + 10_000 specifies the ending index (exclusive) up to which the slicing continues.\n    \n    if display: \n        plt.figure(figsize=(10,5))\n        offset = 0\n    \n    # CONVERT TO NUMPY\n    data = np.zeros((10_000,len(FEATS)))\n    for j,col in enumerate(FEATS):\n        \n        # FILL NAN\n        x = eeg[col].values.astype('float32')\n        m = np.nanmean(x)\n        if np.isnan(x).mean()<1: x = np.nan_to_num(x,nan=m)\n        else: x[:] = 0\n            \n        data[:,j] = x\n        \n        if display: \n            if j!=0: offset += x.max()\n            plt.plot(range(10_000),x-offset,label=col)\n            offset -= x.min()\n            \n    if display:\n        plt.legend()\n        name = parquet_path.split('/')[-1]\n        name = name.split('.')[0]\n        plt.title(f'EEG {name}',size=16)\n        plt.show()\n        \n    return data","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:20:35.241297Z","iopub.execute_input":"2024-06-26T11:20:35.242366Z","iopub.status.idle":"2024-06-26T11:20:35.258749Z","shell.execute_reply.started":"2024-06-26T11:20:35.242321Z","shell.execute_reply":"2024-06-26T11:20:35.257756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n\nall_eegs = {}\nDISPLAY = 4\nEEG_IDS = train.eeg_id.unique()\nPATH = '/kaggle/input/hms-harmful-brain-activity-classification/train_eegs/'\n\nfor i,eeg_id in enumerate(EEG_IDS):\n    if (i%100==0)&(i!=0): print(i,', ',end='') \n    \n    # SAVE EEG TO PYTHON DICTIONARY OF NUMPY ARRAYS\n    data = eeg_from_parquet(f'{PATH}{eeg_id}.parquet', display=i<DISPLAY)              \n    all_eegs[eeg_id] = data\n    \n    if i==DISPLAY:\n        if CREATE_EEGS:\n            print(f'Processing {train.eeg_id.nunique()} eeg parquets... ',end='')\n        else:\n            print(f'Reading {len(EEG_IDS)} eeg NumPys from disk.')\n            break\n            \nif CREATE_EEGS: \n    np.save('eegs',all_eegs)\nelse:\n    all_eegs = np.load('/kaggle/input/brain-eegs/eegs.npy',allow_pickle=True).item()","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:20:37.015906Z","iopub.execute_input":"2024-06-26T11:20:37.016577Z","iopub.status.idle":"2024-06-26T11:22:25.681129Z","shell.execute_reply.started":"2024-06-26T11:20:37.016539Z","shell.execute_reply":"2024-06-26T11:22:25.680151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Deduplicate Train EEG Id","metadata":{}},{"cell_type":"code","source":"# LOAD TRAIN \ndf = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\nTARGETS = df.columns[-6:]\nTARS = {'Seizure':0, 'LPD':1, 'GPD':2, 'LRDA':3, 'GRDA':4, 'Other':5}\n\nTARS2 = {x:y for y,x in TARS.items()}\n\n\ntrain = df.groupby('eeg_id')[['patient_id']].agg('first')\n\ntmp = df.groupby('eeg_id')[TARGETS].agg('sum')\nfor t in TARGETS:\n    train[t] = tmp[t].values\n    \ny_data = train[TARGETS].values\ny_data = y_data / y_data.sum(axis=1,keepdims=True)\ntrain[TARGETS] = y_data\n\ntmp = df.groupby('eeg_id')[['expert_consensus']].agg('first')\ntrain['target'] = tmp\n\ntrain = train.reset_index()\ntrain = train.loc[train.eeg_id.isin(EEG_IDS)]\nprint('Train Data with unique eeg_id shape:', train.shape )\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:22:25.682985Z","iopub.execute_input":"2024-06-26T11:22:25.683701Z","iopub.status.idle":"2024-06-26T11:22:25.941938Z","shell.execute_reply.started":"2024-06-26T11:22:25.683662Z","shell.execute_reply":"2024-06-26T11:22:25.941051Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Butter Low-Pass Filter","metadata":{}},{"cell_type":"code","source":"from scipy.signal import butter, lfilter\n\ndef butter_lowpass_filter(data, cutoff_freq=20, sampling_rate=200, order=4):\n    nyquist = 0.5 * sampling_rate\n    normal_cutoff = cutoff_freq / nyquist\n    b, a = butter(order, normal_cutoff, btype='low', analog=False)\n    filtered_data = lfilter(b, a, data, axis=0)\n    return filtered_data","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:22:30.1961Z","iopub.execute_input":"2024-06-26T11:22:30.19646Z","iopub.status.idle":"2024-06-26T11:22:30.643773Z","shell.execute_reply.started":"2024-06-26T11:22:30.196429Z","shell.execute_reply":"2024-06-26T11:22:30.642731Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"FREQS = [1,2,4,8,16][::-1]\nx = [all_eegs[EEG_IDS[0]][:,0]]\nfor k in FREQS:\n    x.append( butter_lowpass_filter(x[0], cutoff_freq=k) )\n\nplt.figure(figsize=(20,20))\nplt.plot(range(10_000),x[0], label='without filter')\nfor k in range(1,len(x)):\n    plt.plot(range(10_000),x[k]-k*(x[0].max()-x[0].min()), label=f'with filter {FREQS[k-1]}Hz')\nplt.legend()\nplt.title('Butter Low-Pass Filter Examples',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-06-26T10:28:01.44882Z","iopub.execute_input":"2024-06-26T10:28:01.449701Z","iopub.status.idle":"2024-06-26T10:28:02.313313Z","shell.execute_reply.started":"2024-06-26T10:28:01.449653Z","shell.execute_reply":"2024-06-26T10:28:02.312308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"FREQS = [1, 2, 4, 8, 16][::-1]\nchannels = ['Fp1',  'C3',   'T3', 'O1', 'Fp2',  'C4', 'T4', 'O2']\nchannel_indices = [FEAT2IDX[channel] for channel in channels]\n\nplt.figure(figsize=(20, 20))\n\nfor i, channel_idx in enumerate(channel_indices):\n    x = [all_eegs[EEG_IDS[0]][:, channel_idx]]\n    for k in FREQS:\n        x.append(butter_lowpass_filter(x[0], cutoff_freq=k))\n\n    plt.subplot(8, 1, i + 1)\n    plt.plot(range(10_000), x[0], label='without filter')\n    for k in range(1, len(x)):\n        plt.plot(range(10_000), x[k] - k * (x[0].max() - x[0].min()), label=f'with filter {FREQS[k-1]}Hz')\n    plt.legend()\n    plt.title(f'Butter Low-Pass Filter Examples for {channels[i]}', size=18)\n    if i == 7:\n        plt.xlabel('Time (samples)')\n\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:22:32.431098Z","iopub.execute_input":"2024-06-26T11:22:32.431471Z","iopub.status.idle":"2024-06-26T11:22:37.11154Z","shell.execute_reply.started":"2024-06-26T11:22:32.431441Z","shell.execute_reply":"2024-06-26T11:22:37.110382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Loader with Butter Low-Pass Filter","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\n\nclass DataGenerator(tf.keras.utils.Sequence):\n    'Generates data for Keras'\n    def __init__(self, data, batch_size=32, shuffle=False, eegs=all_eegs, mode='train',\n                 downsample=5): \n\n        self.data = data\n        self.batch_size = batch_size\n        self.shuffle = shuffle\n        self.eegs = eegs\n        self.mode = mode\n        self.downsample = downsample\n        self.on_epoch_end()\n        \n    def __len__(self):\n        'Denotes the number of batches per epoch'\n        ct = int( np.ceil( len(self.data) / self.batch_size ) )\n        return ct\n\n    def __getitem__(self, index):\n        'Generate one batch of data'\n        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n        X, y = self.__data_generation(indexes)\n        return X[:,::self.downsample,:], y\n\n    def on_epoch_end(self):\n        'Updates indexes after each epoch'\n        self.indexes = np.arange( len(self.data) )\n        if self.shuffle: np.random.shuffle(self.indexes)\n                        \n    def __data_generation(self, indexes):\n        'Generates data containing batch_size samples' \n    \n        X = np.zeros((len(indexes),10_000,8),dtype='float32')\n        y = np.zeros((len(indexes),6),dtype='float32')\n        \n        sample = np.zeros((10_000,X.shape[-1]))\n        for j,i in enumerate(indexes):\n            row = self.data.iloc[i]      \n            data = self.eegs[row.eeg_id]\n            \n            # FEATURE ENGINEER\n            sample[:,0] = data[:,FEAT2IDX['Fp1']] - data[:,FEAT2IDX['T3']]\n            sample[:,1] = data[:,FEAT2IDX['T3']] - data[:,FEAT2IDX['O1']]\n            \n            sample[:,2] = data[:,FEAT2IDX['Fp1']] - data[:,FEAT2IDX['C3']]\n            sample[:,3] = data[:,FEAT2IDX['C3']] - data[:,FEAT2IDX['O1']]\n            \n            sample[:,4] = data[:,FEAT2IDX['Fp2']] - data[:,FEAT2IDX['C4']]\n            sample[:,5] = data[:,FEAT2IDX['C4']] - data[:,FEAT2IDX['O2']]\n            \n            sample[:,6] = data[:,FEAT2IDX['Fp2']] - data[:,FEAT2IDX['T4']]\n            sample[:,7] = data[:,FEAT2IDX['T4']] - data[:,FEAT2IDX['O2']]\n            \n            # STANDARDIZE\n            sample = np.clip(sample,-1024,1024)\n            sample = np.nan_to_num(sample, nan=0) / 32.0\n            \n            # BUTTER LOW-PASS FILTER\n            sample = butter_lowpass_filter(sample)\n            \n            X[j,] = sample\n            if self.mode!='test':\n                y[j] = row[TARGETS]\n            \n        return X,y","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:22:45.014497Z","iopub.execute_input":"2024-06-26T11:22:45.014865Z","iopub.status.idle":"2024-06-26T11:22:56.327498Z","shell.execute_reply.started":"2024-06-26T11:22:45.014827Z","shell.execute_reply":"2024-06-26T11:22:56.3267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Display Data Loader","metadata":{}},{"cell_type":"code","source":"gen = DataGenerator(train, shuffle=False)\n\nfor x,y in gen:\n    for k in range(4):\n        plt.figure(figsize=(20,4))\n        offset = 0\n        for j in range(x.shape[-1]):\n            if j!=0: offset -= x[k,:,j].min()\n            plt.plot(range(2_000),x[k,:,j]+offset,label=f'feature {j+1}')\n            offset += x[k,:,j].max()\n        tt = f'{y[k][0]:0.1f}'\n        for t in y[k][1:]:\n            tt += f', {t:0.1f}'\n        plt.title(f'EEG_Id = {EEG_IDS[k]}\\nTarget = {tt}',size=14)\n        plt.legend()\n        plt.show()\n    break","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:23:12.851718Z","iopub.execute_input":"2024-06-26T11:23:12.852763Z","iopub.status.idle":"2024-06-26T11:23:14.67209Z","shell.execute_reply.started":"2024-06-26T11:23:12.852729Z","shell.execute_reply":"2024-06-26T11:23:14.671094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Assuming all_eegs is already loaded\neeg_data = all_eegs[EEG_IDS[0]]  # Use the first EEG signal for this example\n\n# Use the first channel of the EEG data\nfiltered_eeg_data = butter_lowpass_filter(eeg_data)\nFEATS = ['Fp1', 'T3', 'C3', 'O1', 'Fp2', 'C4', 'T4', 'O2']\nFEAT2IDX = {x: y for y, x in enumerate(FEATS)}\n\nsample = np.zeros((filtered_eeg_data.shape[0], 8))\nsample[:,0] = data[:,FEAT2IDX['Fp1']] - data[:,FEAT2IDX['T3']]\nsample[:,1] = data[:,FEAT2IDX['T3']] - data[:,FEAT2IDX['O1']]\nsample[:,2] = data[:,FEAT2IDX['Fp1']] - data[:,FEAT2IDX['C3']]\nsample[:,3] = data[:,FEAT2IDX['C3']] - data[:,FEAT2IDX['O1']]\nsample[:,4] = data[:,FEAT2IDX['Fp2']] - data[:,FEAT2IDX['C4']]\nsample[:,5] = data[:,FEAT2IDX['C4']] - data[:,FEAT2IDX['O2']]\nsample[:,6] = data[:,FEAT2IDX['Fp2']] - data[:,FEAT2IDX['T4']]\nsample[:,7] = data[:,FEAT2IDX['T4']] - data[:,FEAT2IDX['O2']]\nplt.figure(figsize=(20, 10))\nfor i in range(8):\n    plt.subplot(8, 1, i + 1)\n    plt.plot(range(10_000), sample[:, i], label=f'Feature {i + 1}')\n    plt.legend()\n    plt.title(f'Feature {i + 1}', size=18)\n\nplt.xlabel('Time (samples)')\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:43:10.431859Z","iopub.execute_input":"2024-06-26T11:43:10.432606Z","iopub.status.idle":"2024-06-26T11:43:12.744455Z","shell.execute_reply.started":"2024-06-26T11:43:10.432576Z","shell.execute_reply":"2024-06-26T11:43:12.743588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"combined1 = (sample[:, 0] + sample[:, 1]) / 2\ncombined2 = (sample[:, 2] + sample[:, 3]) / 2\ncombined3 = (sample[:, 4] + sample[:, 5]) / 2\ncombined4 = (sample[:, 6] + sample[:, 7]) / 2\n\n\nplt.figure(figsize=(20, 20))\n\nplt.subplot(4, 1, 1)\nplt.plot(range(10_000), combined1, label='Combined Feature 1')\nplt.legend()\nplt.title('Combined Feature 1', size=18)\n\nplt.subplot(4, 1, 2)\nplt.plot(range(10_000), combined2, label='Combined Feature 2')\nplt.legend()\nplt.title('Combined Feature 2', size=18)\n\nplt.subplot(4, 1, 3)\nplt.plot(range(10_000), combined3, label='Combined Feature 3')\nplt.legend()\nplt.title('Combined Feature 3', size=18)\n\nplt.subplot(4, 1, 4)\nplt.plot(range(10_000), combined4, label='Combined Feature 4')\nplt.legend()\nplt.title('Combined Feature 4', size=18)\n\nplt.xlabel('Time (samples)')\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:43:42.611546Z","iopub.execute_input":"2024-06-26T11:43:42.612192Z","iopub.status.idle":"2024-06-26T11:43:43.989883Z","shell.execute_reply.started":"2024-06-26T11:43:42.612156Z","shell.execute_reply":"2024-06-26T11:43:43.988938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom scipy.signal import stft, spectrogram, welch, butter, lfilter\nimport matplotlib.colors\n\n# Load the data as per your code above\ndf = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\nTARGETS = df.columns[-6:]\nTARS = {'Seizure': 0, 'LPD': 1, 'GPD': 2, 'LRDA': 3, 'GRDA': 4, 'Other': 5}\nTARS2 = {x: y for y, x in TARS.items()}\n\ntrain = df.groupby('eeg_id')[['patient_id']].agg('first')\n\ntmp = df.groupby('eeg_id')[TARGETS].agg('sum')\nfor t in TARGETS:\n    train[t] = tmp[t].values\n\ny_data = train[TARGETS].values\ny_data = y_data / y_data.sum(axis=1, keepdims=True)\ntrain[TARGETS] = y_data\n\ntmp = df.groupby('eeg_id')[['expert_consensus']].agg('first')\ntrain['target'] = tmp\n\ntrain = train.reset_index()\nEEG_IDS = train['eeg_id'].values  # Assuming EEG_IDS is the unique EEG ids from train data\n\n# Assuming all_eegs is already loaded\neeg_data = all_eegs[EEG_IDS[0]]  # Use the first EEG signal for this example\n\n# Use the first channel of the EEG data\nx = eeg_data[:, 0]\n\n# Apply the low-pass filter\nfiltered_x = butter_lowpass_filter(x)\n\n\n# Compute and plot the spectrogram using STFT\nf, t, Zxx = stft(filtered_x, fs=200, nperseg=1000)\n","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:44:01.12133Z","iopub.execute_input":"2024-06-26T11:44:01.122259Z","iopub.status.idle":"2024-06-26T11:44:01.332791Z","shell.execute_reply.started":"2024-06-26T11:44:01.122214Z","shell.execute_reply":"2024-06-26T11:44:01.331822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, t, Zxx = stft(combined1, fs=200, window='hann', nperseg=200, noverlap=int(200 * 0.95), nfft=None, detrend=False, return_onesided=True, boundary='zeros', padded=True)\n\n# Convert the magnitude to dB\nZxx_db = 10 * np.log10(np.abs(Zxx)**2)\nnoverlap = 200 / 2\nplt.figure(figsize=(10,7), facecolor='w')\nf, Pxx_den =welch(combined1, fs=200, nperseg=200, window='hanning', detrend=False, noverlap=noverlap)\n\n\n\nPxx_den_db = 10*np.log10(Pxx_den)\nmn = np.min(Pxx_den_db)\nmx = np.max(Pxx_den_db)\n\n\n\nf, t, Sxx = spectrogram(np.array(combined1), fs=200, nperseg=200, noverlap=int(200*0.95), scaling='density', return_onesided=True)\n\nnormalize_color= matplotlib.colors.Normalize(vmin=mn, vmax=mx)\nSxx = 10*np.log10(Sxx)\nplt.pcolormesh(t, f, Sxx,cmap='jet', norm=normalize_color)# Plot the result\nplt.colorbar(label='Power (dB)')","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:45:30.141018Z","iopub.execute_input":"2024-06-26T11:45:30.141401Z","iopub.status.idle":"2024-06-26T11:45:30.582165Z","shell.execute_reply.started":"2024-06-26T11:45:30.141372Z","shell.execute_reply":"2024-06-26T11:45:30.580954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, t, Zxx = stft(combined2, fs=200, window='hann', nperseg=200, noverlap=int(200 * 0.95), nfft=None, detrend=False, return_onesided=True, boundary='zeros', padded=True)\n\n# Convert the magnitude to dB\nZxx_db = 10 * np.log10(np.abs(Zxx)**2)\nnoverlap = 200 / 2\nplt.figure(figsize=(10,7), facecolor='w')\nf, Pxx_den =welch(filtered_x, fs=200, nperseg=200, window='hanning', detrend=False, noverlap=noverlap)\n\n\n\nPxx_den_db = 10*np.log10(Pxx_den)\nmn = np.min(Pxx_den_db)\nmx = np.max(Pxx_den_db)\n\n\n\nf, t, Sxx = spectrogram(np.array(filtered_x), fs=200, nperseg=200, noverlap=int(200*0.95), scaling='density', return_onesided=True)\n\nnormalize_color= matplotlib.colors.Normalize(vmin=mn, vmax=mx)\nSxx = 10*np.log10(Sxx)\nplt.pcolormesh(t, f, Sxx, norm=normalize_color)# Plot the result\nplt.colorbar(label='Power (dB)')","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:44:07.016334Z","iopub.execute_input":"2024-06-26T11:44:07.0167Z","iopub.status.idle":"2024-06-26T11:44:07.556023Z","shell.execute_reply.started":"2024-06-26T11:44:07.016674Z","shell.execute_reply":"2024-06-26T11:44:07.554694Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, t, Zxx = stft(combined3, fs=200, window='hann', nperseg=200, noverlap=int(200 * 0.95), nfft=None, detrend=False, return_onesided=True, boundary='zeros', padded=True)\n\n# Convert the magnitude to dB\nZxx_db = 10 * np.log10(np.abs(Zxx)**2)\nnoverlap = 200 / 2\nplt.figure(figsize=(10,7), facecolor='w')\nf, Pxx_den =welch(filtered_x, fs=200, nperseg=200, window='hanning', detrend=False, noverlap=noverlap)\n\n\n\nPxx_den_db = 10*np.log10(Pxx_den)\nmn = np.min(Pxx_den_db)\nmx = np.max(Pxx_den_db)\n\n\n\nf, t, Sxx = spectrogram(np.array(filtered_x), fs=200, nperseg=200, noverlap=int(200*0.95), scaling='density', return_onesided=True)\n\nnormalize_color= matplotlib.colors.Normalize(vmin=mn, vmax=mx)\nSxx = 10*np.log10(Sxx)\nplt.pcolormesh(t, f, Sxx, norm=normalize_color)# Plot the result\nplt.colorbar(label='Power (dB)')","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:40:51.796851Z","iopub.execute_input":"2024-06-26T11:40:51.797312Z","iopub.status.idle":"2024-06-26T11:40:52.242031Z","shell.execute_reply.started":"2024-06-26T11:40:51.797272Z","shell.execute_reply":"2024-06-26T11:40:52.241238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Assuming all_eegs is already loaded\neeg_data = all_eegs[EEG_IDS[0]]  # Use the first EEG signal for this example\n\n# Use the first channel of the EEG data\nfiltered_eeg_data = butter_lowpass_filter(eeg_data)\nFEATS = ['Fp1', 'T3', 'C3', 'O1', 'Fp2', 'C4', 'T4', 'O2']\nFEAT2IDX = {x: y for y, x in enumerate(FEATS)}\n\nsample = np.zeros((filtered_eeg_data.shape[0], 2))\nfeature1=sample[:, 2] = filtered_eeg_data[:, FEAT2IDX['Fp1']] - filtered_eeg_data[:, FEAT2IDX['C3']]\nfeature2=sample[:, 3] = filtered_eeg_data[:, FEAT2IDX['C3']] - filtered_eeg_data[:, FEAT2IDX['O1']]\nplt.figure(figsize=(20, 10))\n\nplt.subplot(2, 1, 1)\nplt.plot(range(10_000), sample[:, 2], label='Fp1 - C3')\nplt.legend()\nplt.title('Feature: Fp1 - C3', size=18)\n\nplt.subplot(2, 1, 2)\nplt.plot(range(10_000), sample[:, 3], label='C3 - O1')\nplt.legend()\nplt.title('Feature: C3 - O1', size=18)\nplt.xlabel('Time (samples)')\n\nplt.tight_layout()\nplt.show()\n\ncombined=(feature1+feature2)/2\nplt.plot(combined)","metadata":{"execution":{"iopub.status.busy":"2024-06-26T11:29:12.253376Z","iopub.execute_input":"2024-06-26T11:29:12.254301Z","iopub.status.idle":"2024-06-26T11:29:12.309492Z","shell.execute_reply.started":"2024-06-26T11:29:12.25426Z","shell.execute_reply":"2024-06-26T11:29:12.308341Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from scipy.signal import spectrogram\nimport numpy as np\n\nclass DataGenerator(tf.keras.utils.Sequence):\n    'Generates data for Keras'\n    def __init__(self, data, batch_size=32, shuffle=False, eegs=all_eegs, mode='train', downsample=5, fs=256):\n        self.data = data\n        self.batch_size = batch_size\n        self.shuffle = shuffle\n        self.eegs = eegs\n        self.mode = mode\n        self.downsample = downsample\n        self.fs = fs\n        self.on_epoch_end()\n        \n    def __len__(self):\n        'Denotes the number of batches per epoch'\n        ct = int(np.ceil(len(self.data) / self.batch_size))\n        return ct\n\n    def __getitem__(self, index):\n        'Generate one batch of data'\n        indexes = self.indexes[index * self.batch_size:(index + 1) * self.batch_size]\n        X, y = self.__data_generation(indexes)\n        return X[:, ::self.downsample, :], y\n\n    def on_epoch_end(self):\n        'Updates indexes after each epoch'\n        self.indexes = np.arange(len(self.data))\n        if self.shuffle:\n            np.random.shuffle(self.indexes)\n                        \n    def __data_generation(self, indexes):\n        'Generates data containing batch_size samples'\n        X = np.zeros((len(indexes), 10_000, 8), dtype='float32')\n        y = np.zeros((len(indexes), 6), dtype='float32')\n        \n        sample = np.zeros((10_000, X.shape[-1]))\n        for j, i in enumerate(indexes):\n            row = self.data.iloc[i]      \n            data = self.eegs[row.eeg_id]\n            \n            # FEATURE ENGINEER\n            sample[:, 0] = data[:, FEAT2IDX['Fp1']] - data[:, FEAT2IDX['T3']]\n            sample[:, 1] = data[:, FEAT2IDX['T3']] - data[:, FEAT2IDX['O1']]\n            \n            sample[:, 2] = data[:, FEAT2IDX['Fp1']] - data[:, FEAT2IDX['C3']]\n            sample[:, 3] = data[:, FEAT2IDX['C3']] - data[:, FEAT2IDX['O1']]\n            \n            sample[:, 4] = data[:, FEAT2IDX['Fp2']] - data[:, FEAT2IDX['C4']]\n            sample[:, 5] = data[:, FEAT2IDX['C4']] - data[:, FEAT2IDX['O2']]\n            \n            sample[:, 6] = data[:, FEAT2IDX['Fp2']] - data[:, FEAT2IDX['T4']]\n            sample[:, 7] = data[:, FEAT2IDX['T4']] - data[:, FEAT2IDX['O2']]\n            \n            # STANDARDIZE\n            sample = np.clip(sample, -1024, 1024)\n            sample = np.nan_to_num(sample, nan=0) / 32.0\n            \n            # BUTTER LOW-PASS FILTER\n            sample = butter_lowpass_filter(sample)\n            \n            X[j,] = sample\n            if self.mode != 'test':\n                y[j] = row[TARGETS]\n        \n        return X, y\n\n    def compute_spectrogram(self, sample):\n        'Computes spectrogram for a given sample'\n        spectrograms = []\n        for i in range(sample.shape[-1]):\n            f, t, Sxx = spectrogram(sample[:, i], self.fs)\n            spectrograms.append((f, t, Sxx))\n        return spectrograms\n","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.229752Z","iopub.status.idle":"2024-06-25T08:55:30.23026Z","shell.execute_reply.started":"2024-06-25T08:55:30.229998Z","shell.execute_reply":"2024-06-25T08:55:30.230022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom scipy.signal import spectrogram\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\n\n# Load the required data\ntrain = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\nEEG_IDS = train.eeg_id.unique()\nPATH = '/kaggle/input/hms-harmful-brain-activity-classification/train_eegs/'\nFEATS = ['Fp1','T3','C3','O1','Fp2','C4','T4','O2']\nFEAT2IDX = {x: y for x, y in zip(FEATS, range(len(FEATS)))}\n\n# Function to load EEG data from parquet files\ndef eeg_from_parquet(parquet_path):\n    eeg = pd.read_parquet(parquet_path, columns=FEATS)\n    rows = len(eeg)\n    offset = (rows - 10_000) // 2\n    eeg = eeg.iloc[offset:offset + 10_000]\n    data = np.zeros((10_000, len(FEATS)))\n    for j, col in enumerate(FEATS):\n        x = eeg[col].values.astype('float32')\n        m = np.nanmean(x)\n        if np.isnan(x).mean() < 1:\n            x = np.nan_to_num(x, nan=m)\n        else:\n            x[:] = 0\n        data[:, j] = x\n    return data\n\n# Load all EEGs\nall_eegs = {eeg_id: eeg_from_parquet(f'{PATH}{eeg_id}.parquet') for eeg_id in EEG_IDS}\n\n# Define a butter low-pass filter\ndef butter_lowpass_filter(data, cutoff_freq=20, sampling_rate=200, order=4):\n    nyquist = 0.5 * sampling_rate\n    normal_cutoff = cutoff_freq / nyquist\n    b, a = butter(order, normal_cutoff, btype='low', analog=False)\n    filtered_data = lfilter(b, a, data, axis=0)\n    return filtered_data\n\n# Define DataGenerator class\nclass DataGenerator(tf.keras.utils.Sequence):\n    def __init__(self, data, batch_size=32, shuffle=False, eegs=all_eegs, mode='train', downsample=5, fs=200):\n        self.data = data\n        self.batch_size = batch_size\n        self.shuffle = shuffle\n        self.eegs = eegs\n        self.mode = mode\n        self.downsample = downsample\n        self.fs = fs\n        self.on_epoch_end()\n        \n    def __len__(self):\n        return int(np.ceil(len(self.data) / self.batch_size))\n\n    def __getitem__(self, index):\n        indexes = self.indexes[index * self.batch_size:(index + 1) * self.batch_size]\n        X, y = self.__data_generation(indexes)\n        return X[:, ::self.downsample, :], y\n\n    def on_epoch_end(self):\n        self.indexes = np.arange(len(self.data))\n        if self.shuffle:\n            np.random.shuffle(self.indexes)\n                        \n    def __data_generation(self, indexes):\n        X = np.zeros((len(indexes), 10_000, 8), dtype='float32')\n        y = np.zeros((len(indexes), 6), dtype='float32')\n        \n        sample = np.zeros((10_000, X.shape[-1]))\n        for j, i in enumerate(indexes):\n            row = self.data.iloc[i]\n            data = self.eegs[row.eeg_id]\n            \n            sample[:, 0] = data[:, FEAT2IDX['Fp1']] - data[:, FEAT2IDX['T3']]\n            sample[:, 1] = data[:, FEAT2IDX['T3']] - data[:, FEAT2IDX['O1']]\n            sample[:, 2] = data[:, FEAT2IDX['Fp1']] - data[:, FEAT2IDX['C3']]\n            sample[:, 3] = data[:, FEAT2IDX['C3']] - data[:, FEAT2IDX['O1']]\n            sample[:, 4] = data[:, FEAT2IDX['Fp2']] - data[:, FEAT2IDX['C4']]\n            sample[:, 5] = data[:, FEAT2IDX['C4']] - data[:, FEAT2IDX['O2']]\n            sample[:, 6] = data[:, FEAT2IDX['Fp2']] - data[:, FEAT2IDX['T4']]\n            sample[:, 7] = data[:, FEAT2IDX['T4']] - data[:, FEAT2IDX['O2']]\n            \n            sample = np.clip(sample, -1024, 1024)\n            sample = np.nan_to_num(sample, nan=0) / 32.0\n            sample = butter_lowpass_filter(sample)\n            \n            X[j, ] = sample\n            if self.mode != 'test':\n                y[j] = row[TARGETS]\n        \n        return X, y\n    \n    def compute_spectrogram(self, sample):\n        spectrograms = []\n        for i in range(sample.shape[-1]):\n            f, t, Sxx = spectrogram(sample[:, i], self.fs)\n            spectrograms.append((f, t, Sxx))\n        return spectrograms\n\n# Instantiate the data generator\ngen = DataGenerator(train, shuffle=False, eegs=all_eegs)\n\n# Plot spectrograms for a batch of data\nfor x, y in gen:\n    for k in range(4):\n        plt.figure(figsize=(20, 8))\n        \n        spectrograms = gen.compute_spectrogram(x[k, :, :])\n        \n        for j in range(len(spectrograms)):\n            f, t, Sxx = spectrograms[j]\n            plt.subplot(2, 4, j + 1)\n            plt.pcolormesh(t, f, Sxx, shading='gouraud')\n            plt.ylabel('Frequency [Hz]')\n            plt.xlabel('Time [sec]')\n            plt.title(f'Feature {j + 1}')\n        \n        tt = f'{y[k][0]:0.1f}'\n        for t in y[k][1:]:\n            tt += f', {t:0.1f}'\n        \n        plt.suptitle(f'EEG_Id = {train.iloc[k].eeg_id}\\nTarget = {tt}', size=14)\n        plt.show()\n    break\n","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.231597Z","iopub.status.idle":"2024-06-25T08:55:30.23211Z","shell.execute_reply.started":"2024-06-25T08:55:30.231831Z","shell.execute_reply":"2024-06-25T08:55:30.231853Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom scipy.signal import spectrogram\nfrom scipy.signal import butter, lfilter\nimport tensorflow as tf\n\n# Load training data\ntrain = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\nprint(train.shape)\ndisplay(train.head())\n\n# Constants\nFEATS = ['Fp1','T3','C3','O1','Fp2','C4','T4','O2']\nFEAT2IDX = {x: y for x, y in zip(FEATS, range(len(FEATS)))}\nPATH = '/kaggle/input/hms-harmful-brain-activity-classification/train_eegs/'\n\n# Function to load EEG data from parquet files\ndef eeg_from_parquet(parquet_path, display=False):\n    eeg = pd.read_parquet(parquet_path, columns=FEATS)\n    rows = len(eeg)\n    offset = (rows - 10_000) // 2\n    eeg = eeg.iloc[offset:offset + 10_000]\n    \n    if display:\n        plt.figure(figsize=(10, 5))\n        offset = 0\n    \n    data = np.zeros((10_000, len(FEATS)))\n    for j, col in enumerate(FEATS):\n        x = eeg[col].values.astype('float32')\n        m = np.nanmean(x)\n        if np.isnan(x).mean() < 1:\n            x = np.nan_to_num(x, nan=m)\n        else:\n            x[:] = 0\n        data[:, j] = x\n        \n        if display:\n            if j != 0:\n                offset += x.max()\n            plt.plot(range(10_000), x - offset, label=col)\n            offset -= x.min()\n    \n    if display:\n        plt.legend()\n        name = parquet_path.split('/')[-1].split('.')[0]\n        plt.title(f'EEG {name}', size=16)\n        plt.show()\n        \n    return data\n\n# Load all EEGs\nall_eegs = {}\nDISPLAY = 4\nEEG_IDS = train.eeg_id.unique()\n\nfor i, eeg_id in enumerate(EEG_IDS):\n    if i % 100 == 0 and i != 0:\n        print(i, ', ', end='')\n    data = eeg_from_parquet(f'{PATH}{eeg_id}.parquet', display=i < DISPLAY)\n    all_eegs[eeg_id] = data\n    if i == DISPLAY:\n        break\n\n# Define a butter low-pass filter\ndef butter_lowpass_filter(data, cutoff_freq=20, sampling_rate=200, order=4):\n    nyquist = 0.5 * sampling_rate\n    normal_cutoff = cutoff_freq / nyquist\n    b, a = butter(order, normal_cutoff, btype='low', analog=False)\n    filtered_data = lfilter(b, a, data, axis=0)\n    return filtered_data\n\n# Define DataGenerator class\nclass DataGenerator(tf.keras.utils.Sequence):\n    def __init__(self, data, batch_size=32, shuffle=False, eegs=all_eegs, mode='train', downsample=5, fs=200):\n        self.data = data\n        self.batch_size = batch_size\n        self.shuffle = shuffle\n        self.eegs = eegs\n        self.mode = mode\n        self.downsample = downsample\n        self.fs = fs\n        self.on_epoch_end()\n        \n    def __len__(self):\n        return int(np.ceil(len(self.data) / self.batch_size))\n\n    def __getitem__(self, index):\n        indexes = self.indexes[index * self.batch_size:(index + 1) * self.batch_size]\n        X, y = self.__data_generation(indexes)\n        return X[:, ::self.downsample, :], y\n\n    def on_epoch_end(self):\n        self.indexes = np.arange(len(self.data))\n        if self.shuffle:\n            np.random.shuffle(self.indexes)\n                        \n    def __data_generation(self, indexes):\n        X = np.zeros((len(indexes), 10_000, 8), dtype='float32')\n        y = np.zeros((len(indexes), 6), dtype='float32')\n        \n        sample = np.zeros((10_000, X.shape[-1]))\n        for j, i in enumerate(indexes):\n            row = self.data.iloc[i]\n            data = self.eegs[row.eeg_id]\n            \n            sample[:, 0] = data[:, FEAT2IDX['Fp1']] - data[:, FEAT2IDX['T3']]\n            sample[:, 1] = data[:, FEAT2IDX['T3']] - data[:, FEAT2IDX['O1']]\n            sample[:, 2] = data[:, FEAT2IDX['Fp1']] - data[:, FEAT2IDX['C3']]\n            sample[:, 3] = data[:, FEAT2IDX['C3']] - data[:, FEAT2IDX['O1']]\n            sample[:, 4] = data[:, FEAT2IDX['Fp2']] - data[:, FEAT2IDX['C4']]\n            sample[:, 5] = data[:, FEAT2IDX['C4']] - data[:, FEAT2IDX['O2']]\n            sample[:, 6] = data[:, FEAT2IDX['Fp2']] - data[:, FEAT2IDX['T4']]\n            sample[:, 7] = data[:, FEAT2IDX['T4']] - data[:, FEAT2IDX['O2']]\n            \n            sample = np.clip(sample, -1024, 1024)\n            sample = np.nan_to_num(sample, nan=0) / 32.0\n            sample = butter_lowpass_filter(sample)\n            \n            X[j, ] = sample\n            if self.mode != 'test':\n                y[j] = row[TARGETS]\n        \n        return X, y\n    \n    def compute_spectrogram(self, sample):\n        spectrograms = []\n        for i in range(sample.shape[-1]):\n            f, t, Sxx = spectrogram(sample[:, i], self.fs)\n            spectrograms.append((f, t, Sxx))\n        return spectrograms\n\n# Instantiate the data generator\ngen = DataGenerator(train, shuffle=False, eegs=all_eegs)\n\n# Plot spectrograms for a batch of data\nfor x, y in gen:\n    for k in range(4):\n        plt.figure(figsize=(20, 8))\n        \n        spectrograms = gen.compute_spectrogram(x[k, :, :])\n        \n        for j in range(len(spectrograms)):\n            f, t, Sxx = spectrograms[j]\n            plt.subplot(2, 4, j + 1)\n            plt.pcolormesh(t, f, Sxx, shading='gouraud')\n            plt.ylabel('Frequency [Hz]')\n            plt.xlabel('Time [sec]')\n            plt.title(f'Feature {j + 1}')\n        \n        tt = f'{y[k][0]:0.1f}'\n        for t in y[k][1:]:\n            tt += f', {t:0.1f}'\n        \n        plt.suptitle(f'EEG_Id = {train.iloc[k].eeg_id}\\nTarget = {tt}', size=14)\n        plt.show()\n    break\n","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.233951Z","iopub.status.idle":"2024-06-25T08:55:30.234455Z","shell.execute_reply.started":"2024-06-25T08:55:30.234182Z","shell.execute_reply":"2024-06-25T08:55:30.234204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# Instantiate the data generator\ngen = DataGenerator(train, shuffle=False)\n\n# Generate a batch of data\nfor x, y in gen:\n    # Iterate over samples in the batch\n    for k in range(4):\n        plt.figure(figsize=(20, 4))\n        \n        # Compute spectrogram for each feature\n        spectrograms = gen.compute_spectrogram(x[k, :, :])\n        \n        for j in range(len(spectrograms)):\n            f, t, Sxx = spectrograms[j]\n            plt.subplot(2, 4, j + 1)\n            plt.pcolormesh(t, f, Sxx, shading='gouraud')\n            plt.ylabel('Frequency [Hz]')\n            plt.xlabel('Time [sec]')\n            plt.title(f'Feature {j+1}')\n        \n        # Display target values for the current sample\n        tt = f'{y[k][0]:0.1f}'\n        for t in y[k][1:]:\n            tt += f', {t:0.1f}'\n        \n        plt.suptitle(f'EEG_Id = {EEG_IDS[k]}\\nTarget = {tt}', size=14)\n        plt.show()\n    break\n","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.235536Z","iopub.status.idle":"2024-06-25T08:55:30.236061Z","shell.execute_reply.started":"2024-06-25T08:55:30.235769Z","shell.execute_reply":"2024-06-25T08:55:30.235791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Initialize GPUs","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nos.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0,1\"\nimport tensorflow as tf\nprint('TensorFlow version =',tf.__version__)\n\n# USE MULTIPLE GPUS\ngpus = tf.config.list_physical_devices('GPU')\nif len(gpus)<=1: \n    strategy = tf.distribute.OneDeviceStrategy(device=\"/gpu:0\")\n    print(f'Using {len(gpus)} GPU')\nelse: \n    strategy = tf.distribute.MirroredStrategy()\n    print(f'Using {len(gpus)} GPUs')","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.238071Z","iopub.status.idle":"2024-06-25T08:55:30.238571Z","shell.execute_reply.started":"2024-06-25T08:55:30.238302Z","shell.execute_reply":"2024-06-25T08:55:30.238343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# USE MIXED PRECISION\nMIX = True\nif MIX:\n    tf.config.optimizer.set_experimental_options({\"auto_mixed_precision\": True})\n    print('Mixed precision enabled')\nelse:\n    print('Using full precision')","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.240238Z","iopub.status.idle":"2024-06-25T08:55:30.240769Z","shell.execute_reply.started":"2024-06-25T08:55:30.240495Z","shell.execute_reply":"2024-06-25T08:55:30.240518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Build WaveNet Model","metadata":{}},{"cell_type":"code","source":"# TRAIN SCHEDULE\ndef lrfn(epoch):\n        return [1e-3,1e-3,1e-4,1e-4,1e-5][epoch]\nLR = tf.keras.callbacks.LearningRateScheduler(lrfn, verbose = True)\nEPOCHS = 5","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.242689Z","iopub.status.idle":"2024-06-25T08:55:30.243335Z","shell.execute_reply.started":"2024-06-25T08:55:30.242913Z","shell.execute_reply":"2024-06-25T08:55:30.242936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.layers import Input, Dense, Multiply, Add, Conv1D, Concatenate\n\ndef wave_block(x, filters, kernel_size, n):\n    dilation_rates = [2**i for i in range(n)]\n    x = Conv1D(filters = filters,\n               kernel_size = 1,\n               padding = 'same')(x)\n    res_x = x\n    for dilation_rate in dilation_rates:\n        tanh_out = Conv1D(filters = filters,\n                          kernel_size = kernel_size,\n                          padding = 'same', \n                          activation = 'tanh', \n                          dilation_rate = dilation_rate)(x)\n        sigm_out = Conv1D(filters = filters,\n                          kernel_size = kernel_size,\n                          padding = 'same',\n                          activation = 'sigmoid', \n                          dilation_rate = dilation_rate)(x)\n        x = Multiply()([tanh_out, sigm_out])\n        x = Conv1D(filters = filters,\n                   kernel_size = 1,\n                   padding = 'same')(x)\n        res_x = Add()([res_x, x])\n    return res_x","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.245387Z","iopub.status.idle":"2024-06-25T08:55:30.245898Z","shell.execute_reply.started":"2024-06-25T08:55:30.245627Z","shell.execute_reply":"2024-06-25T08:55:30.245651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Jan-2024/wave-model.png)","metadata":{}},{"cell_type":"code","source":"def build_model():\n        \n    # INPUT \n    inp = tf.keras.Input(shape=(2_000,8))\n    \n    ############\n    # FEATURE EXTRACTION SUB MODEL\n    inp2 = tf.keras.Input(shape=(2_000,1))\n    x = wave_block(inp2, 8, 3, 12)\n    x = wave_block(x, 16, 3, 8)\n    x = wave_block(x, 32, 3, 4)\n    x = wave_block(x, 64, 3, 1)\n    model2 = tf.keras.Model(inputs=inp2, outputs=x)\n    ###########\n    \n    # LEFT TEMPORAL CHAIN\n    x1 = model2(inp[:,:,0:1])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = model2(inp[:,:,1:2])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z1 = tf.keras.layers.Average()([x1,x2])\n    \n    # LEFT PARASAGITTAL CHAIN\n    x1 = model2(inp[:,:,2:3])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = model2(inp[:,:,3:4])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z2 = tf.keras.layers.Average()([x1,x2])\n    \n    # RIGHT PARASAGITTAL CHAIN\n    x1 = model2(inp[:,:,4:5])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = model2(inp[:,:,5:6])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z3 = tf.keras.layers.Average()([x1,x2])\n    \n    # RIGHT TEMPORAL CHAIN\n    x1 = model2(inp[:,:,6:7])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = model2(inp[:,:,7:8])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z4 = tf.keras.layers.Average()([x1,x2])\n    \n    # COMBINE CHAINS\n    y = tf.keras.layers.Concatenate()([z1,z2,z3,z4])\n    y = tf.keras.layers.Dense(64, activation='relu')(y)\n    y = tf.keras.layers.Dense(6,activation='softmax', dtype='float32')(y)\n    \n    # COMPILE MODEL\n    model = tf.keras.Model(inputs=inp, outputs=y)\n    opt = tf.keras.optimizers.Adam(learning_rate = 1e-3)\n    loss = tf.keras.losses.KLDivergence()\n    model.compile(loss=loss, optimizer = opt)\n    \n    return model","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.2475Z","iopub.status.idle":"2024-06-25T08:55:30.247957Z","shell.execute_reply.started":"2024-06-25T08:55:30.247718Z","shell.execute_reply":"2024-06-25T08:55:30.247739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train Group KFold","metadata":{}},{"cell_type":"code","source":"VERBOSE = 1\nFOLDS_TO_TRAIN = 5\nif not os.path.exists('WaveNet_Model'):\n    os.makedirs('WaveNet_Model')\n\nfrom sklearn.model_selection import KFold, GroupKFold\nimport tensorflow.keras.backend as K, gc\n\nall_oof = []; all_oof2 = []; all_true = []\ngkf = GroupKFold(n_splits=5)\nfor i, (train_index, valid_index) in enumerate(gkf.split(train, train.target, train.patient_id)):   \n    \n    print('#'*25)\n    print(f'### Fold {i+1}')\n    train_gen = DataGenerator(train.iloc[train_index], shuffle=True, batch_size=32)\n    valid_gen = DataGenerator(train.iloc[valid_index], shuffle=False, batch_size=64, mode='valid')\n    print(f'### train size {len(train_index)}, valid size {len(valid_index)}')\n    print('#'*25)\n    \n    # TRAIN MODEL\n    K.clear_session()\n    with strategy.scope():\n        model = build_model()\n    if TRAIN_MODEL:\n        model.fit(train_gen, verbose=VERBOSE,\n              validation_data = valid_gen,\n              epochs=EPOCHS, callbacks = [LR])\n        model.save_weights(f'WaveNet_Model/WaveNet_fold{i}.h5')\n    else:\n        model.load_weights(f'/kaggle/input/brain-eegs/WaveNet_Model/WaveNet_fold{i}.h5')\n    \n    # WAVENET OOF\n    oof = model.predict(valid_gen, verbose=VERBOSE)\n    all_oof.append(oof)\n    all_true.append(train.iloc[valid_index][TARGETS].values)\n    \n    # TRAIN MEAN OOF\n    y_train = train.iloc[train_index][TARGETS].values\n    y_valid = train.iloc[valid_index][TARGETS].values\n    oof = y_valid.copy()\n    for j in range(6):\n        oof[:,j] = y_train[:,j].mean()\n    oof = oof / oof.sum(axis=1,keepdims=True)\n    all_oof2.append(oof)\n    \n    del model, oof, y_train, y_valid\n    gc.collect()\n    \n    if i==FOLDS_TO_TRAIN-1: break\n    \nall_oof = np.concatenate(all_oof)\nall_oof2 = np.concatenate(all_oof2)\nall_true = np.concatenate(all_true)","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.249405Z","iopub.status.idle":"2024-06-25T08:55:30.249902Z","shell.execute_reply.started":"2024-06-25T08:55:30.249645Z","shell.execute_reply":"2024-06-25T08:55:30.249666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# CV Score for WaveNet","metadata":{}},{"cell_type":"code","source":"import sys\nsys.path.append('/kaggle/input/kaggle-kl-div')\nfrom kaggle_kl_div import score\n\noof = pd.DataFrame(all_oof.copy())\noof['id'] = np.arange(len(oof))\n\ntrue = pd.DataFrame(all_true.copy())\ntrue['id'] = np.arange(len(true))\n\ncv = score(solution=true, submission=oof, row_id_column_name='id')\nprint('CV Score with WaveNet Raw EEG =',cv)","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.251743Z","iopub.status.idle":"2024-06-25T08:55:30.252286Z","shell.execute_reply.started":"2024-06-25T08:55:30.251965Z","shell.execute_reply":"2024-06-25T08:55:30.251987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# CV Score using Train Means","metadata":{}},{"cell_type":"code","source":"oof = pd.DataFrame(all_oof2.copy())\noof['id'] = np.arange(len(oof))\n\ntrue = pd.DataFrame(all_true.copy())\ntrue['id'] = np.arange(len(true))\n\ncv = score(solution=true, submission=oof, row_id_column_name='id')\nprint('CV Score with Train Means =',cv)","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.253976Z","iopub.status.idle":"2024-06-25T08:55:30.254359Z","shell.execute_reply.started":"2024-06-25T08:55:30.254161Z","shell.execute_reply":"2024-06-25T08:55:30.254177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submit to Kaggle LB","metadata":{}},{"cell_type":"code","source":"del all_eegs, train; gc.collect()\ntest = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/test.csv')\nprint('Test shape:',test.shape)\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.255592Z","iopub.status.idle":"2024-06-25T08:55:30.255932Z","shell.execute_reply.started":"2024-06-25T08:55:30.255761Z","shell.execute_reply":"2024-06-25T08:55:30.255777Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"all_eegs2 = {}\nDISPLAY = 1\nEEG_IDS2 = test.eeg_id.unique()\nPATH2 = '/kaggle/input/hms-harmful-brain-activity-classification/test_eegs/'\n\nprint('Processing Test EEG parquets...'); print()\nfor i,eeg_id in enumerate(EEG_IDS2):\n        \n    # SAVE EEG TO PYTHON DICTIONARY OF NUMPY ARRAYS\n    data = eeg_from_parquet(f'{PATH2}{eeg_id}.parquet', i<DISPLAY)\n    all_eegs2[eeg_id] = data","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.256874Z","iopub.status.idle":"2024-06-25T08:55:30.257196Z","shell.execute_reply.started":"2024-06-25T08:55:30.257035Z","shell.execute_reply":"2024-06-25T08:55:30.257049Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# INFER MLP ON TEST\npreds = []\nmodel = build_model()\ntest_gen = DataGenerator(test, shuffle=False, batch_size=64, eegs=all_eegs2, mode='test')\n\nprint('Inferring test... ',end='')\nfor i in range(FOLDS_TO_TRAIN):\n    print(f'fold {i+1}, ',end='')\n    if TRAIN_MODEL:\n        model.load_weights(f'WaveNet_Model/WaveNet_fold{i}.h5')\n    else:\n        model.load_weights(f'/kaggle/input/brain-eegs/WaveNet_Model/WaveNet_fold{i}.h5')\n    pred = model.predict(test_gen, verbose=0)\n    preds.append(pred)\npred = np.mean(preds,axis=0)\nprint()\nprint('Test preds shape',pred.shape)","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.258929Z","iopub.status.idle":"2024-06-25T08:55:30.259289Z","shell.execute_reply.started":"2024-06-25T08:55:30.259104Z","shell.execute_reply":"2024-06-25T08:55:30.25912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CREATE SUBMISSION.CSV\nfrom IPython.display import display\n\nsub = pd.DataFrame({'eeg_id':test.eeg_id.values})\nsub[TARGETS] = pred\nsub.to_csv('submission.csv',index=False)\nprint('Submission shape',sub.shape)\ndisplay( sub.head() )\n\n# SANITY CHECK TO CONFIRM PREDICTIONS SUM TO ONE\nprint('Sub row 0 sums to:',sub.iloc[0,-6:].sum())","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.260443Z","iopub.status.idle":"2024-06-25T08:55:30.260827Z","shell.execute_reply.started":"2024-06-25T08:55:30.260644Z","shell.execute_reply":"2024-06-25T08:55:30.260661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np  # Add this line to import NumPy\n\nfrom sklearn.metrics import precision_recall_fscore_support\n\n# Assuming ⁠ true_labels ⁠ are the true labels and ⁠ predicted_labels ⁠ are the predicted labels\n# These should be arrays of shape (num_samples,)\ntrue_labels = np.argmax('true', axis=0)  # Convert one-hot encoded true labels to categorical labels if needed\npredicted_labels = np.argmax(pred, axis=0)  # Convert one-hot encoded predictions to categorical labels if needed\n\n# Calculate precision, recall, F1 score, and support\nprecision, recall, f1_score, support = precision_recall_fscore_support(true_labels, predicted_labels, average='weighted')\n\n# Print the results\nprint(f'Precision: {precision:.4f}')\nprint(f'Recall: {recall:.4f}')\nprint(f'F1 Score: {f1_score:.4f}')\nprint(f'Support: {support}')","metadata":{"execution":{"iopub.status.busy":"2024-06-25T08:55:30.26246Z","iopub.status.idle":"2024-06-25T08:55:30.262801Z","shell.execute_reply.started":"2024-06-25T08:55:30.262633Z","shell.execute_reply":"2024-06-25T08:55:30.26265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}