{"metadata":{"kaggle":{"accelerator":"none","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7752462,"sourceType":"datasetVersion","datasetId":4382744},{"sourceId":7946649,"sourceType":"datasetVersion","datasetId":4417235}],"dockerImageVersionId":30636,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":390.813775,"end_time":"2024-03-11T19:50:37.750668","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-03-11T19:44:06.936893","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"| MODEL | DATA TYPE | CV | LB |TRAINING TIME| NOTES |\n|----------------|-----------|----------|---|--|--|\n| EfficientNetB2 | K+E+KE | | 0.36 | 8 hours and 42 minutes |K+E+KE: Data spectrogram augmentation|\n\nGreat discussion [here][5] by @KOLOO that led to the latest scores!\n\nFeatures+Head Starter uses Chris Deotte's Kaggle dataset [here][1]. Also Uses Chris's EEG spectrograms [here][3] (modified version). The Raw EEG signals can be found [here][6]. This notebook is a direct descendent of Chris's notebooks [EfficientNet][2] and [WaveNet][4]\n\n[1]: https://www.kaggle.com/datasets/cdeotte/brain-spectrograms\n[2]: https://www.kaggle.com/code/cdeotte/efficientnetb2-starter-lb-0-57\n[3]: https://www.kaggle.com/datasets/nartaa/eeg-spectrograms\n[4]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/468684\n[5]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/477461\n[6]: https://www.kaggle.com/datasets/nartaa/hms-eeg","metadata":{"papermill":{"duration":0.008291,"end_time":"2024-03-11T19:44:10.402144","exception":false,"start_time":"2024-03-11T19:44:10.393853","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# DATA GENERATOR, UTILITY AND MODEL FUNCTIONS","metadata":{"papermill":{"duration":0.007349,"end_time":"2024-03-11T19:44:25.360162","exception":false,"start_time":"2024-03-11T19:44:25.352813","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import librosa\nimport os, random\nimport tensorflow\nimport tensorflow as tf\nimport albumentations as albu\nimport pandas as pd, numpy as np\nfrom scipy.signal import butter, lfilter\nimport tensorflow.keras.backend as K, gc\nfrom tensorflow.keras.models import load_model\nfrom tensorflow.keras.layers import Input, Dense, Multiply, Add, Conv1D, Concatenate, LayerNormalization\n\nLOAD_BACKBONE_FROM = '/kaggle/input/efficientnetb-tf-keras/EfficientNetB2.h5'\nLOAD_MODELS_FROM = '/kaggle/input/features-head-starter-models'\nMODEL = {'K+E+KE': 59}\nfor DATA_TYPE in MODEL: pass\nTARGETS = ['seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']\nFEATS2 = ['Fp1','T3','C3','O1','Fp2','C4','T4','O2']\nFEAT2IDX = {x:y for x,y in zip(FEATS2,range(len(FEATS2)))}\nFEATS = [['Fp1','F7','T3','T5','O1'],\n         ['Fp1','F3','C3','P3','O1'],\n         ['Fp2','F8','T4','T6','O2'],\n         ['Fp2','F4','C4','P4','O2']]\n    \nclass DataGenerator():\n    'Generates data for Keras'\n    def __init__(self, data, specs=None, eeg_specs=None, raw_eegs=None , augment=False, mode='train', data_type=DATA_TYPE): \n        self.augment = augment\n        self.mode = mode\n        self.data_type = data_type\n        self.data = self.build_data(data.copy())\n        self.specs = specs\n        self.eeg_specs = eeg_specs\n        self.raw_eegs = raw_eegs\n        self.on_epoch_end()\n    \n    def build_data(self,data):\n        if self.data_type in ['K+E']:\n            data_dup = pd.concat([data] * 2, ignore_index=True)\n            data_dup.loc[:len(data),'data_type'] = 'K'\n            data_dup.loc[len(data):,'data_type'] = 'E'\n            data = data_dup\n        elif self.data_type in ['K+E+KE']:\n            data_trp = pd.concat([data] * 3, ignore_index=True)\n            data_trp.loc[:len(data),'data_type'] = 'K'\n            data_trp.loc[len(data):len(data)*2,'data_type'] = 'E'\n            data_trp.loc[len(data)*2:,'data_type'] = 'KE'\n            data = data_trp\n        else:\n            data['data_type'] = self.data_type\n        return data\n        \n    def __len__(self):\n        return self.data.shape[0]\n\n    def __getitem__(self, index):\n        X, y = self.data_generation(index)\n        if self.augment: X = self.augmentation(X)\n        return X, y\n    \n    def __call__(self):\n        for i in range(self.__len__()):\n            yield self.__getitem__(i)\n            \n            if i == self.__len__()-1:\n                self.on_epoch_end()\n                \n    def on_epoch_end(self):\n        if self.mode=='train': \n            self.data = self.data.sample(frac=1).reset_index(drop=True)\n    \n    def data_generation(self, index):\n        row = self.data.iloc[index]\n        if row.data_type == 'KE':\n            X,y = self.generate_all_specs(index)\n        elif row.data_type in ['K','E']:\n            X,y = self.generate_specs(index)\n        elif row.data_type == 'R':\n            X,y = self.generate_raw(index)\n        elif row.data_type in ['ER','KR']:\n            X1,y = self.generate_specs(index)\n            X2,y = self.generate_raw(index)\n            X = (X1,X2)\n        elif row.data_type in ['KER']:\n            X1,y = self.generate_all_specs(index)\n            X2,y = self.generate_raw(index)\n            X = (X1,X2)\n        return X,y\n    \n    def generate_all_specs(self, index):\n        X = np.zeros((512,512,3),dtype='float32')\n        y = np.zeros((6,),dtype='float32')\n        \n        row = self.data.iloc[index]\n        if self.mode=='test': \n            offset = 0\n        else:\n            offset = int(row.offset/2)\n        \n        eeg = self.eeg_specs[row.eeg_id]\n        spec = self.specs[row.spec_id]\n        \n        imgs = [spec[offset:offset+300,k*100:(k+1)*100].T for k in [0,2,1,3]] # to match kaggle with eeg\n        img = np.stack(imgs,axis=-1)\n        # LOG TRANSFORM SPECTROGRAM\n        img = np.clip(img,np.exp(-4),np.exp(8))\n        img = np.log(img)\n            \n        # STANDARDIZE PER IMAGE\n        img = np.nan_to_num(img, nan=0.0)    \n            \n        mn = img.flatten().min()\n        mx = img.flatten().max()\n        ep = 1e-5\n        img = 255 * (img - mn) / (mx - mn + ep)\n        \n        X[0_0+56:100+56,:256,0] = img[:,22:-22,0] # LL_k\n        X[100+56:200+56,:256,0] = img[:,22:-22,2] # RL_k\n        X[0_0+56:100+56,:256,1] = img[:,22:-22,1] # LP_k\n        X[100+56:200+56,:256,1] = img[:,22:-22,3] # RP_k\n        X[0_0+56:100+56,:256,2] = img[:,22:-22,2] # RL_k\n        X[100+56:200+56,:256,2] = img[:,22:-22,1] # LP_k\n        \n        X[0_0+56:100+56,256:,0] = img[:,22:-22,0] # LL_k\n        X[100+56:200+56,256:,0] = img[:,22:-22,2] # RL_k\n        X[0_0+56:100+56,256:,1] = img[:,22:-22,1] # LP_k\n        X[100+56:200+56,256:,1] = img[:,22:-22,3] # RP_K\n        \n        # EEG\n        img = eeg\n        mn = img.flatten().min()\n        mx = img.flatten().max()\n        ep = 1e-5\n        img = 255 * (img - mn) / (mx - mn + ep)\n        X[200+56:300+56,:256,0] = img[:,22:-22,0] # LL_e\n        X[300+56:400+56,:256,0] = img[:,22:-22,2] # RL_e\n        X[200+56:300+56,:256,1] = img[:,22:-22,1] # LP_e\n        X[300+56:400+56,:256,1] = img[:,22:-22,3] # RP_e\n        X[200+56:300+56,:256,2] = img[:,22:-22,2] # RL_e\n        X[300+56:400+56,:256,2] = img[:,22:-22,1] # LP_e\n        \n        X[200+56:300+56,256:,0] = img[:,22:-22,0] # LL_e\n        X[300+56:400+56,256:,0] = img[:,22:-22,2] # RL_e\n        X[200+56:300+56,256:,1] = img[:,22:-22,1] # LP_e\n        X[300+56:400+56,256:,1] = img[:,22:-22,3] # RP_e\n\n        if self.mode!='test':\n            y[:] = row[TARGETS]\n        \n        return X,y\n    \n    def generate_specs(self, index):\n        X = np.zeros((512,512,3),dtype='float32')\n        y = np.zeros((6,),dtype='float32')\n        \n        row = self.data.iloc[index]\n        if self.mode=='test': \n            offset = 0\n        else:\n            offset = int(row.offset/2)\n        \n        if row.data_type in ['E','ER']:\n            img = self.eeg_specs[row.eeg_id]\n        elif row.data_type in ['K','KR']:\n            spec = self.specs[row.spec_id]\n            imgs = [spec[offset:offset+300,k*100:(k+1)*100].T for k in [0,2,1,3]] # to match kaggle with eeg\n            img = np.stack(imgs,axis=-1)\n            # LOG TRANSFORM SPECTROGRAM\n            img = np.clip(img,np.exp(-4),np.exp(8))\n            img = np.log(img)\n            \n            # STANDARDIZE PER IMAGE\n            img = np.nan_to_num(img, nan=0.0)    \n            \n        mn = img.flatten().min()\n        mx = img.flatten().max()\n        ep = 1e-5\n        img = 255 * (img - mn) / (mx - mn + ep)\n        \n        X[0_0+56:100+56,:256,0] = img[:,22:-22,0]\n        X[100+56:200+56,:256,0] = img[:,22:-22,2]\n        X[0_0+56:100+56,:256,1] = img[:,22:-22,1]\n        X[100+56:200+56,:256,1] = img[:,22:-22,3]\n        X[0_0+56:100+56,:256,2] = img[:,22:-22,2]\n        X[100+56:200+56,:256,2] = img[:,22:-22,1]\n        \n        X[0_0+56:100+56,256:,0] = img[:,22:-22,0]\n        X[100+56:200+56,256:,0] = img[:,22:-22,1]\n        X[0_0+56:100+56,256:,1] = img[:,22:-22,2]\n        X[100+56:200+56,256:,1] = img[:,22:-22,3]\n        \n        X[200+56:300+56,:256,0] = img[:,22:-22,0]\n        X[300+56:400+56,:256,0] = img[:,22:-22,1]\n        X[200+56:300+56,:256,1] = img[:,22:-22,2]\n        X[300+56:400+56,:256,1] = img[:,22:-22,3]\n        X[200+56:300+56,:256,2] = img[:,22:-22,3]\n        X[300+56:400+56,:256,2] = img[:,22:-22,2]\n        \n        X[200+56:300+56,256:,0] = img[:,22:-22,0]\n        X[300+56:400+56,256:,0] = img[:,22:-22,2]\n        X[200+56:300+56,256:,1] = img[:,22:-22,1]\n        X[300+56:400+56,256:,1] = img[:,22:-22,3]\n        \n        if self.mode!='test':\n            y[:] = row[TARGETS]\n        \n        return X,y\n    \n    def generate_raw(self,index):\n        if USE_PROCESSED and self.mode!='test':\n            X = np.zeros((2_000,8),dtype='float32')\n            y = np.zeros((6,),dtype='float32')\n            row = self.data.iloc[index]\n            X = self.raw_eegs[row.eeg_id]\n            y[:] = row[TARGETS]\n            return X,y\n        \n        X = np.zeros((10_000,8),dtype='float32')\n        y = np.zeros((6,),dtype='float32')\n        \n        row = self.data.iloc[index]\n        eeg = self.raw_eegs[row.eeg_id]\n            \n        # FEATURE ENGINEER\n        X[:,0] = eeg[:,FEAT2IDX['Fp1']] - eeg[:,FEAT2IDX['T3']]\n        X[:,1] = eeg[:,FEAT2IDX['T3']] - eeg[:,FEAT2IDX['O1']]\n            \n        X[:,2] = eeg[:,FEAT2IDX['Fp1']] - eeg[:,FEAT2IDX['C3']]\n        X[:,3] = eeg[:,FEAT2IDX['C3']] - eeg[:,FEAT2IDX['O1']]\n            \n        X[:,4] = eeg[:,FEAT2IDX['Fp2']] - eeg[:,FEAT2IDX['C4']]\n        X[:,5] = eeg[:,FEAT2IDX['C4']] - eeg[:,FEAT2IDX['O2']]\n            \n        X[:,6] = eeg[:,FEAT2IDX['Fp2']] - eeg[:,FEAT2IDX['T4']]\n        X[:,7] = eeg[:,FEAT2IDX['T4']] - eeg[:,FEAT2IDX['O2']]\n            \n        # STANDARDIZE\n        X = np.clip(X,-1024,1024)\n        X = np.nan_to_num(X, nan=0) / 32.0\n            \n        # BUTTER LOW-PASS FILTER\n        X = self.butter_lowpass_filter(X)\n        # Downsample\n        X = X[::5,:]\n        \n        if self.mode!='test':\n            y[:] = row[TARGETS]\n                \n        return X,y\n        \n    def butter_lowpass_filter(self, data, cutoff_freq=20, sampling_rate=200, order=4):\n        nyquist = 0.5 * sampling_rate\n        normal_cutoff = cutoff_freq / nyquist\n        b, a = butter(order, normal_cutoff, btype='low', analog=False)\n        filtered_data = lfilter(b, a, data, axis=0)\n        return filtered_data\n    \n    def resize(self, img,size):\n        composition = albu.Compose([\n                albu.Resize(size[0],size[1])\n            ])\n        return composition(image=img)['image']\n            \n    def augmentation(self, img):\n        composition = albu.Compose([\n                albu.HorizontalFlip(p=0.4)\n            ])\n        return composition(image=img)['image']\n\ndef spectrogram_from_eeg(parquet_path):\n    \n    # LOAD MIDDLE 50 SECONDS OF EEG SERIES\n    eeg = pd.read_parquet(parquet_path)\n    middle = (len(eeg)-10_000)//2\n    eeg = eeg.iloc[middle:middle+10_000]\n    \n    # VARIABLE TO HOLD SPECTROGRAM\n    img = np.zeros((100,300,4),dtype='float32')\n\n    for k in range(4):\n        COLS = FEATS[k]\n        \n        for kk in range(4):\n            # FILL NANS\n            x1 = eeg[COLS[kk]].values\n            x2 = eeg[COLS[kk+1]].values\n            m = np.nanmean(x1)\n            if np.isnan(x1).mean()<1: x1 = np.nan_to_num(x1,nan=m)\n            else: x1[:] = 0\n            m = np.nanmean(x2)\n            if np.isnan(x2).mean()<1: x2 = np.nan_to_num(x2,nan=m)\n            else: x2[:] = 0\n                \n            # COMPUTE PAIR DIFFERENCES\n            x = x1 - x2\n\n            # RAW SPECTROGRAM\n            mel_spec = librosa.feature.melspectrogram(y=x, sr=200, hop_length=len(x)//300, \n                  n_fft=1024, n_mels=100, fmin=0, fmax=20, win_length=128)\n            \n            # LOG TRANSFORM\n            width = (mel_spec.shape[1]//30)*30\n            mel_spec_db = librosa.power_to_db(mel_spec, ref=np.max).astype(np.float32)[:,:width]\n            img[:,:,k] += mel_spec_db\n                \n        # AVERAGE THE 4 MONTAGE DIFFERENCES\n        img[:,:,k] /= 4.0\n          \n    return img\n\ndef eeg_from_parquet(parquet_path):\n\n    eeg = pd.read_parquet(parquet_path, columns=FEATS2)\n    rows = len(eeg)\n    offset = (rows-10_000)//2\n    eeg = eeg.iloc[offset:offset+10_000]\n    data = np.zeros((10_000,len(FEATS2)))\n    for j,col in enumerate(FEATS2):\n        \n        # FILL NAN\n        x = eeg[col].values.astype('float32')\n        m = np.nanmean(x)\n        if np.isnan(x).mean()<1: x = np.nan_to_num(x,nan=m)\n        else: x[:] = 0\n        \n        data[:,j] = x\n\n    return data\n\ndef build_spec_model(hybrid=False):  \n    inp = tf.keras.layers.Input((512,512,3))\n    base_model = load_model(f'{LOAD_BACKBONE_FROM}')    \n    x = base_model(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    if not hybrid:\n        x = tf.keras.layers.Dense(6,activation='softmax', dtype='float32')(x)\n    model = tf.keras.Model(inputs=inp, outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate = 1e-3)\n    loss = tf.keras.losses.KLDivergence()\n    model.compile(loss=loss, optimizer=opt)  \n    return model\n\ndef dataset(data, mode='train', batch_size=32, data_type=DATA_TYPE, \n            augment=False, specs=None, eeg_specs=None, raw_eegs=None):\n    \n    gen = DataGenerator(data,mode=mode, data_type=data_type, augment=augment,\n                       specs=specs, eeg_specs=eeg_specs, raw_eegs=raw_eegs)\n    inp = tf.TensorSpec(shape=(512,512,3), dtype=tf.float32)     \n    output_signature = (inp,tf.TensorSpec(shape=(6,), dtype=tf.float32))\n    dataset = tf.data.Dataset.from_generator(generator=gen, output_signature=output_signature).batch(\n        batch_size)\n    return dataset\n\ndef predict(models, params, fold, models_path=None):\n    preds = []\n    if models_path is None: models_path = LOAD_MODELS_FROM\n    model = build_spec_model()\n    for data_type in models:\n        data = params['data']\n        ver = models[data_type]\n        ds = dataset(data_type=data_type, **params)\n        model.load_weights(f'{models_path}/model_{data_type}_{ver}_{fold}.weights.h5')\n        pred = model.predict(ds)\n        if data_type in ['K+E+KE']:\n            pred = (pred[:len(data)] + pred[len(data):len(data)*2] + pred[len(data)*2:])/3\n        preds.append(pred)\n    pred = np.mean(preds,axis=0)\n    del model\n    gc.collect()\n    return pred","metadata":{"papermill":{"duration":0.077498,"end_time":"2024-03-11T19:44:25.445288","exception":false,"start_time":"2024-03-11T19:44:25.36779","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# LOAD TEST DATA","metadata":{"papermill":{"duration":0.007417,"end_time":"2024-03-11T19:44:25.716452","exception":false,"start_time":"2024-03-11T19:44:25.709035","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# READ ALL SPECTROGRAMS\ntest = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/test.csv')\nPATH2 = '/kaggle/input/hms-harmful-brain-activity-classification/test_spectrograms'\nfiles2 = os.listdir(PATH2)\nspectrograms2 = {}\nfor i,f in enumerate(files2):\n    if i%100==0: print(i,', ',end='')\n    tmp = pd.read_parquet(f'{PATH2}/{f}')\n    name = int(f.split('.')[0])\n    spectrograms2[name] = tmp.iloc[:,1:].values\n    \n# RENAME FOR DATA GENERATOR\ntest = test.rename({'spectrogram_id':'spec_id'},axis=1)\n\n# READ ALL EEG SPECTROGRAMS\nPATH2 = '/kaggle/input/hms-harmful-brain-activity-classification/test_eegs'\nEEG_IDS2 = test.eeg_id.unique()\nall_eegs2 = {}\nfor i,eeg_id in enumerate(EEG_IDS2):\n        \n    # CREATE SPECTROGRAM FROM EEG PARQUET\n    img = spectrogram_from_eeg(f'{PATH2}/{eeg_id}.parquet')\n    all_eegs2[eeg_id] = img\n\n# READ ALL RAW EEG SIGNALS\nall_raw_eegs2 = {}\nfor i,eeg_id in enumerate(EEG_IDS2):\n        \n    # SAVE EEG TO PYTHON DICTIONARY OF NUMPY ARRAYS\n    data = eeg_from_parquet(f'{PATH2}/{eeg_id}.parquet')\n    all_raw_eegs2[eeg_id] = data","metadata":{"papermill":{"duration":0.199131,"end_time":"2024-03-11T19:44:25.954285","exception":false,"start_time":"2024-03-11T19:44:25.755154","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# PREDICT","metadata":{}},{"cell_type":"code","source":"preds = []\nparams = {'data':test,'mode':'test','specs':spectrograms2, 'eeg_specs':all_eegs2, 'raw_eegs':all_raw_eegs2}\n\nfor i in range(5):\n    print(f'Fold {i+1}')\n    pred = predict(MODEL,params,i)\n    preds.append(pred)\n    \npred = np.mean(preds,axis=0)","metadata":{"papermill":{"duration":358.501214,"end_time":"2024-03-11T19:50:34.134586","exception":false,"start_time":"2024-03-11T19:44:35.633372","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# SAVE SUBMISSION","metadata":{}},{"cell_type":"code","source":"sub = pd.DataFrame({'eeg_id':test.eeg_id.values})\nsub[TARGETS] = pred\nsub.to_csv('submission.csv',index=False)","metadata":{"papermill":{"duration":0.057709,"end_time":"2024-03-11T19:50:34.209921","exception":false,"start_time":"2024-03-11T19:50:34.152212","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]}]}