{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7392733,"sourceType":"datasetVersion","datasetId":4297749},{"sourceId":7392775,"sourceType":"datasetVersion","datasetId":4297782},{"sourceId":7402356,"sourceType":"datasetVersion","datasetId":4304475},{"sourceId":7403069,"sourceType":"datasetVersion","datasetId":4304949},{"sourceId":7447509,"sourceType":"datasetVersion","datasetId":4334995},{"sourceId":7450712,"sourceType":"datasetVersion","datasetId":4336944},{"sourceId":158958765,"sourceType":"kernelVersion"}],"dockerImageVersionId":31260,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true},"papermill":{"default_parameters":{},"duration":401.606014,"end_time":"2024-01-21T23:23:54.700520","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-01-21T23:17:13.094506","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"527ddb18","cell_type":"markdown","source":"# EfficientNetB2 Starter for Brain Comp\nThis is an EfficientNetB2 starter notebook for Kaggle's brain comp. We use only spectrogram features. (The model does not use eeg features yet). This EfficientNet Starter achieves CV 0.72 and LB 0.56! This is +0.10 better than my CatBoost starter. My CatBoost starter is [here][1] (with discussion [here][3]) and achieves CV 0.82 and LB 0.67. \n\nBoth my EfficientNetB2 starter and CatBoost starter use my Kaggle dataset [here][2]. This dataset is a single file which contains all of Kaggle's 11,138 spectrogram parquets. Reading this single file is much faster than reading 11k separate files. Thank you everyonne for upvoting my Kaggle [dataset][2]!\n\n# Exciting UPDATE!\nVersion 5 of this notebook trains EfficientNet using **both** Kaggle spectrograms and my new **EEG spectrograms** from my Kaggle dataset [here][4] (which were created from my spectrogram starter [here][5]). Version 5 also uses KL Div loss (suggested by Awsaf in comments) instead of Cross Entropy loss. We also use EfficientNetB0 to save time versus EfficientNetB2 and train for 4 epochs instead of 3.\n* Version 1-2: **CV 0.73 LB 0.57** Kaggle spec only, CE loss\n* Version 3-4: EEG spec only **CV 0.63**, Kaggle spec only **CV 0.65**, KL-Div loss\n* Version 5-6: **CV 0.59 LB 0.44** Both spec, KL-Div loss\n\n[1]: https://www.kaggle.com/code/cdeotte/catboost-starter-lb-0-67\n[2]: https://www.kaggle.com/datasets/cdeotte/brain-spectrograms\n[3]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/467576\n[4]: https://www.kaggle.com/datasets/cdeotte/brain-eeg-spectrograms\n[5]: https://www.kaggle.com/code/cdeotte/how-to-make-spectrogram-from-eeg","metadata":{"papermill":{"duration":0.009678,"end_time":"2024-01-21T23:17:17.807383","exception":false,"start_time":"2024-01-21T23:17:17.797705","status":"completed"},"tags":[]}},{"id":"c1041f1c","cell_type":"markdown","source":"# Kaggle Spectrograms vs. EEG Spectrograms\nIn this competition, Kaggle provides us with both spectrograms and raw eeg waveforms. The Kaggle spectrograms are 10 minutes long and the eeg waveforms are 50 seconds. The middle 50 seconds of both data are the same time window observing the same event. (i.e. the middle 50 seconds is same information displayed two different ways). Spectrograms are just visual representations of raw waveforms.\n\nI published a spectrogram starter notebook [here][1] that converts the raw eeg waveform into a spectrogram. There is a discussion [here][3]. The eeg spectrograms (from version 4) have been uploaded to my Kaggle dataset [here][2]. Version 4 (of my spectrogram starter notebook) onward uses a new powerful formula. The below table shows that the new created eeg spectrograms are equal or better than the Kaggle spectrograms:\n\n| Spectrogram | EffNet 5Fold CV | EffNet LB | Notebook version | Model | Loss | Epochs | Data Augmention |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Kaggle Spectrograms | 0.73 | 0.57 | ver 1 | EffNetB2 | CE | 3 | none |\n| Kaggle Spectrograms | 0.66 | ??? | ver 4 | EffNetB0 | KL-Div | 4 | none |\n| EEG Spectrograms | 0.63 | ??? | ver 3 | EffNetB0 | KL-Div | 4 | none |\n| **Both Spectrograms** | **0.59** | **0.44** | ver 5| EffNetB0 | KL-Div | 4 | none |\n\n\nIn different versions of this (efficientnet starter) notebook, we will train three EfficientNet models. Version 3 is using only EEG spectrograms. Version 4 is using only Kaggle spectrograms. And version 5 is a single model using both EEG spectrograms and Kaggle spectrograms. Then we save the model weights into a Kaggle dataset [here][4] and we make a submit notebook in version 6 for the version 5 model trained on both spectrograms. We achieve **CV 0.59 LB 0.44**! Wow!\n\n[1]: https://www.kaggle.com/code/cdeotte/how-to-make-spectrogram-from-eeg\n[2]: https://www.kaggle.com/datasets/cdeotte/brain-eeg-spectrograms\n[3]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/469760\n[4]: https://www.kaggle.com/datasets/cdeotte/brain-efficientnet-models-v3-v4-v5","metadata":{"papermill":{"duration":0.008813,"end_time":"2024-01-21T23:17:17.825386","exception":false,"start_time":"2024-01-21T23:17:17.816573","status":"completed"},"tags":[]}},{"id":"ed147943","cell_type":"markdown","source":"# Initialize 2xT4 GPUs\nWe will use both Kaggle T4 GPUs and we will use mixed precision.","metadata":{"papermill":{"duration":0.008844,"end_time":"2024-01-21T23:17:17.843075","exception":false,"start_time":"2024-01-21T23:17:17.834231","status":"completed"},"tags":[]}},{"id":"15c4db17","cell_type":"code","source":"import os, gc\nos.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0,1\"\nimport tensorflow as tf\nimport pandas as pd, numpy as np\nimport matplotlib.pyplot as plt\nprint('TensorFlow version =',tf.__version__)\n\n# USE MULTIPLE GPUS\ngpus = tf.config.list_physical_devices('GPU')\nif len(gpus)<=1: \n strategy = tf.distribute.OneDeviceStrategy(device=\"/gpu:0\")\n print(f'Using {len(gpus)} GPU')\nelse: \n strategy = tf.distribute.MirroredStrategy()\n print(f'Using {len(gpus)} GPUs')\n\nVER = 5\n\n# IF THIS EQUALS NONE, THEN WE TRAIN NEW MODELS\n# IF THIS EQUALS DISK PATH, THEN WE LOAD PREVIOUSLY TRAINED MODELS\nLOAD_MODELS_FROM = '/kaggle/input/datasets/cdeotte/brain-efficientnet-models-v3-v4-v5/'\n\nUSE_KAGGLE_SPECTROGRAMS = True\nUSE_EEG_SPECTROGRAMS = True","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:45:03.410091Z","iopub.execute_input":"2026-02-18T07:45:03.410887Z","iopub.status.idle":"2026-02-18T07:45:19.956538Z","shell.execute_reply.started":"2026-02-18T07:45:03.410829Z","shell.execute_reply":"2026-02-18T07:45:19.955722Z"},"papermill":{"duration":20.981231,"end_time":"2024-01-21T23:17:38.833175","exception":false,"start_time":"2024-01-21T23:17:17.851944","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"66de1d27","cell_type":"code","source":"# USE MIXED PRECISION\nMIX = True\nif MIX:\n tf.config.optimizer.set_experimental_options({\"auto_mixed_precision\": True})\n print('Mixed precision enabled')\nelse:\n print('Using full precision')","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:45:19.957949Z","iopub.execute_input":"2026-02-18T07:45:19.958426Z","iopub.status.idle":"2026-02-18T07:45:19.963417Z","shell.execute_reply.started":"2026-02-18T07:45:19.958400Z","shell.execute_reply":"2026-02-18T07:45:19.962528Z"},"papermill":{"duration":0.018456,"end_time":"2024-01-21T23:17:38.861499","exception":false,"start_time":"2024-01-21T23:17:38.843043","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"9803bad5","cell_type":"markdown","source":"# Load Train Data","metadata":{"papermill":{"duration":0.008945,"end_time":"2024-01-21T23:17:38.879766","exception":false,"start_time":"2024-01-21T23:17:38.870821","status":"completed"},"tags":[]}},{"id":"0d993a24","cell_type":"code","source":"df = pd.read_csv('/kaggle/input/competitions/hms-harmful-brain-activity-classification/train.csv')\nTARGETS = df.columns[-6:]\nprint('Train shape:', df.shape )\nprint('Targets', list(TARGETS))\ndf.head()","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.status.busy":"2026-02-18T07:45:19.964302Z","iopub.execute_input":"2026-02-18T07:45:19.964548Z","iopub.status.idle":"2026-02-18T07:45:20.162370Z","shell.execute_reply.started":"2026-02-18T07:45:19.964519Z","shell.execute_reply":"2026-02-18T07:45:20.161609Z"},"papermill":{"duration":0.300842,"end_time":"2024-01-21T23:17:39.191159","exception":false,"start_time":"2024-01-21T23:17:38.890317","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"c3dafec1","cell_type":"markdown","source":"# Create Non-Overlapping Eeg Id Train Data\nThe competition data description says that test data does not have multiple crops from the same `eeg_id`. Therefore we will train and validate using only 1 crop per `eeg_id`. There is a discussion about this [here][1].\n\n[1]: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/467021","metadata":{"papermill":{"duration":0.009625,"end_time":"2024-01-21T23:17:39.210837","exception":false,"start_time":"2024-01-21T23:17:39.201212","status":"completed"},"tags":[]}},{"id":"2e5d8461","cell_type":"code","source":"train = df.groupby('eeg_id')[['spectrogram_id','spectrogram_label_offset_seconds']].agg(\n {'spectrogram_id':'first','spectrogram_label_offset_seconds':'min'})\ntrain.columns = ['spec_id','min']\n\ntmp = df.groupby('eeg_id')[['spectrogram_id','spectrogram_label_offset_seconds']].agg(\n {'spectrogram_label_offset_seconds':'max'})\ntrain['max'] = tmp\n\ntmp = df.groupby('eeg_id')[['patient_id']].agg('first')\ntrain['patient_id'] = tmp\n\ntmp = df.groupby('eeg_id')[TARGETS].agg('sum')\nfor t in TARGETS:\n train[t] = tmp[t].values\n \ny_data = train[TARGETS].values\ny_data = y_data / y_data.sum(axis=1,keepdims=True)\ntrain[TARGETS] = y_data\n\ntmp = df.groupby('eeg_id')[['expert_consensus']].agg('first')\ntrain['target'] = tmp\n\ntrain = train.reset_index()\nprint('Train non-overlapp eeg_id shape:', train.shape )\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:45:20.164069Z","iopub.execute_input":"2026-02-18T07:45:20.164348Z","iopub.status.idle":"2026-02-18T07:45:20.233657Z","shell.execute_reply.started":"2026-02-18T07:45:20.164326Z","shell.execute_reply":"2026-02-18T07:45:20.233050Z"},"papermill":{"duration":0.127954,"end_time":"2024-01-21T23:17:39.348387","exception":false,"start_time":"2024-01-21T23:17:39.220433","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"f41ac1b5","cell_type":"markdown","source":"# Read Train Spectrograms \n\nFirst we need to read in all 11k train spectrogram files. Reading thousands of files takes 11 minutes with Pandas. Instead, we can read 1 file from my [Kaggle dataset here][1] which contains all the 11k spectrograms in less than 1 minute! To use my Kaggle dataset, set variable `READ_SPEC_FILES = False`. Thank you for upvoting my helpful [dataset][1] :-)\n\n[1]: https://www.kaggle.com/datasets/cdeotte/brain-spectrograms","metadata":{"papermill":{"duration":0.010194,"end_time":"2024-01-21T23:17:39.369012","exception":false,"start_time":"2024-01-21T23:17:39.358818","status":"completed"},"tags":[]}},{"id":"d0ceb028","cell_type":"markdown","source":"# Read EEG Spectrograms\nIn version 4 onward, we use EEG spectrograms in addition to Kaggle spectrograms. The EEG spectrograms come from my Kaggle dataset [here][4] (which were created from my spectrogram starter [here][5]). Thank you for upvoting my Kaggle dataset!\n\n[4]: https://www.kaggle.com/datasets/cdeotte/brain-eeg-spectrograms\n[5]: https://www.kaggle.com/code/cdeotte/how-to-make-spectrogram-from-eeg","metadata":{"papermill":{"duration":0.010074,"end_time":"2024-01-21T23:18:45.908322","exception":false,"start_time":"2024-01-21T23:18:45.898248","status":"completed"},"tags":[]}},{"id":"3a73ce72","cell_type":"code","source":"%%time\nREAD_SPEC_FILES = False\n\n# READ ALL SPECTROGRAMS\nPATH = '/kaggle/input/competitions/hms-harmful-brain-activity-classification/train_spectrograms/'\nfiles = os.listdir(PATH)\nprint(f'There are {len(files)} spectrogram parquets')\n\nif READ_SPEC_FILES: \n spectrograms = {}\n for i,f in enumerate(files):\n if i%100==0: print(i,', ',end='')\n tmp = pd.read_parquet(f'{PATH}{f}')\n name = int(f.split('.')[0])\n spectrograms[name] = tmp.iloc[:,1:].values\nelse:\n spectrograms = np.load('/kaggle/input/datasets/cdeotte/brain-spectrograms/specs.npy',allow_pickle=True).item()","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:45:20.234393Z","iopub.execute_input":"2026-02-18T07:45:20.234599Z","iopub.status.idle":"2026-02-18T07:46:01.013574Z","shell.execute_reply.started":"2026-02-18T07:45:20.234579Z","shell.execute_reply":"2026-02-18T07:46:01.012883Z"},"papermill":{"duration":66.508928,"end_time":"2024-01-21T23:18:45.888033","exception":false,"start_time":"2024-01-21T23:17:39.379105","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"ced599f0","cell_type":"code","source":"%%time\nREAD_EEG_SPEC_FILES = False\n\nif READ_EEG_SPEC_FILES:\n all_eegs = {}\n for i,e in enumerate(train.eeg_id.values):\n if i%100==0: print(i,', ',end='')\n x = np.load(f'/kaggle/input/brain-eeg-spectrograms/EEG_Spectrograms/{e}.npy')\n all_eegs[e] = x\nelse:\n all_eegs = np.load('/kaggle/input/datasets/cdeotte/brain-eeg-spectrograms/eeg_specs.npy',allow_pickle=True).item()","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:46:01.014467Z","iopub.execute_input":"2026-02-18T07:46:01.014736Z","iopub.status.idle":"2026-02-18T07:47:00.924918Z","shell.execute_reply.started":"2026-02-18T07:46:01.014701Z","shell.execute_reply":"2026-02-18T07:47:00.924244Z"},"papermill":{"duration":81.341541,"end_time":"2024-01-21T23:20:07.259959","exception":false,"start_time":"2024-01-21T23:18:45.918418","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"ccd5c440","cell_type":"markdown","source":"# Train DataLoader\nThis dataloader outputs 4 spectrogram images as a 4 channel image of size 128x256x4 per train sample. This notebook version is not using data augmention but the code is available below to experiment with albumentations data augmention. Just add `augment = True` when creating the train data loader. And consider adding new transformations to the augment function below.\n\nUPDATE: In version 4 onward, our dataloader outputs both Kaggle spectrograms and EEG spectrogams as 8 channel image of size 128x256x8.","metadata":{"papermill":{"duration":0.010682,"end_time":"2024-01-21T23:20:07.281145","exception":false,"start_time":"2024-01-21T23:20:07.270463","status":"completed"},"tags":[]}},{"id":"16eab5b6","cell_type":"code","source":"import albumentations as albu\nTARS = {'Seizure':0, 'LPD':1, 'GPD':2, 'LRDA':3, 'GRDA':4, 'Other':5}\nTARS2 = {x:y for y,x in TARS.items()}\n\nclass DataGenerator(tf.keras.utils.Sequence):\n 'Generates data for Keras'\n def __init__(self, data, batch_size=32, shuffle=False, augment=False, mode='train',\n specs = spectrograms, eeg_specs = all_eegs): \n\n self.data = data\n self.batch_size = batch_size\n self.shuffle = shuffle\n self.augment = augment\n self.mode = mode\n self.specs = specs\n self.eeg_specs = eeg_specs\n self.on_epoch_end()\n \n def __len__(self):\n 'Denotes the number of batches per epoch'\n ct = int( np.ceil( len(self.data) / self.batch_size ) )\n return ct\n\n def __getitem__(self, index):\n 'Generate one batch of data'\n indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n X, y = self.__data_generation(indexes)\n if self.augment: X = self.__augment_batch(X) \n return X, y\n\n def on_epoch_end(self):\n 'Updates indexes after each epoch'\n self.indexes = np.arange( len(self.data) )\n if self.shuffle: np.random.shuffle(self.indexes)\n \n def __data_generation(self, indexes):\n 'Generates data containing batch_size samples' \n \n X = np.zeros((len(indexes),128,256,8),dtype='float32')\n y = np.zeros((len(indexes),6),dtype='float32')\n img = np.ones((128,256),dtype='float32')\n \n for j,i in enumerate(indexes):\n row = self.data.iloc[i]\n if self.mode=='test': \n r = 0\n else: \n r = int( (row['min'] + row['max'])//4 )\n\n for k in range(4):\n # EXTRACT 300 ROWS OF SPECTROGRAM\n img = self.specs[row.spec_id][r:r+300,k*100:(k+1)*100].T\n \n # LOG TRANSFORM SPECTROGRAM\n img = np.clip(img,np.exp(-4),np.exp(8))\n img = np.log(img)\n \n # STANDARDIZE PER IMAGE\n ep = 1e-6\n m = np.nanmean(img.flatten())\n s = np.nanstd(img.flatten())\n img = (img-m)/(s+ep)\n img = np.nan_to_num(img, nan=0.0)\n \n # CROP TO 256 TIME STEPS\n X[j,14:-14,:,k] = img[:,22:-22] / 2.0\n \n # EEG SPECTROGRAMS\n img = self.eeg_specs[row.eeg_id]\n X[j,:,:,4:] = img\n \n if self.mode!='test':\n y[j,] = row[TARGETS]\n \n return X,y\n \n def __random_transform(self, img):\n composition = albu.Compose([\n albu.HorizontalFlip(p=0.5),\n #albu.CoarseDropout(max_holes=8,max_height=32,max_width=32,fill_value=0,p=0.5),\n ])\n return composition(image=img)['image']\n \n def __augment_batch(self, img_batch):\n for i in range(img_batch.shape[0]):\n img_batch[i, ] = self.__random_transform(img_batch[i, ])\n return img_batch","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:47:00.925761Z","iopub.execute_input":"2026-02-18T07:47:00.926058Z","iopub.status.idle":"2026-02-18T07:47:37.778358Z","shell.execute_reply.started":"2026-02-18T07:47:00.926034Z","shell.execute_reply":"2026-02-18T07:47:37.777362Z"},"papermill":{"duration":3.823507,"end_time":"2024-01-21T23:20:11.115983","exception":false,"start_time":"2024-01-21T23:20:07.292476","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"47ec013d","cell_type":"markdown","source":"# Display DataLoader\nBelow we display example dataloader spectrogram images.","metadata":{"papermill":{"duration":0.010412,"end_time":"2024-01-21T23:20:11.137316","exception":false,"start_time":"2024-01-21T23:20:11.126904","status":"completed"},"tags":[]}},{"id":"c8113087","cell_type":"code","source":"gen = DataGenerator(train, batch_size=32, shuffle=False)\nROWS=2; COLS=3; BATCHES=2\n\nfor i,(x,y) in enumerate(gen):\n plt.figure(figsize=(20,8))\n for j in range(ROWS):\n for k in range(COLS):\n plt.subplot(ROWS,COLS,j*COLS+k+1)\n t = y[j*COLS+k]\n img = x[j*COLS+k,:,:,0][::-1,]\n mn = img.flatten().min()\n mx = img.flatten().max()\n img = (img-mn)/(mx-mn)\n plt.imshow(img)\n tars = f'[{t[0]:0.2f}'\n for s in t[1:]: tars += f', {s:0.2f}'\n eeg = train.eeg_id.values[i*32+j*COLS+k]\n plt.title(f'EEG = {eeg}\\nTarget = {tars}',size=12)\n plt.yticks([])\n plt.ylabel('Frequencies (Hz)',size=14)\n plt.xlabel('Time (sec)',size=16)\n plt.show()\n if i==BATCHES-1: break","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:47:37.779593Z","iopub.execute_input":"2026-02-18T07:47:37.780162Z","iopub.status.idle":"2026-02-18T07:47:39.685665Z","shell.execute_reply.started":"2026-02-18T07:47:37.780136Z","shell.execute_reply":"2026-02-18T07:47:39.684908Z"},"papermill":{"duration":2.839929,"end_time":"2024-01-21T23:20:13.987468","exception":false,"start_time":"2024-01-21T23:20:11.147539","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"d2e22a58","cell_type":"markdown","source":"# Train Scheduler\nWe will train our model with a Step Train Schedule for 4 epochs. First 2 epochs are LR=1e-3. Then epochs 3 and 4 use LR=1e-4 and 1e-5 respectively. (Below we also provide a Cosine Train Schedule if you want to experiment with it. Note it is not used in this notebook).","metadata":{"papermill":{"duration":0.028191,"end_time":"2024-01-21T23:20:14.044243","exception":false,"start_time":"2024-01-21T23:20:14.016052","status":"completed"},"tags":[]}},{"id":"e808dcb4","cell_type":"code","source":"import math\nLR_START = 1e-6\nLR_MAX = 1e-3\nLR_MIN = 1e-6\nLR_RAMPUP_EPOCHS = 0\nLR_SUSTAIN_EPOCHS = 0\nEPOCHS2 = 10\n\ndef lrfn(epoch):\n if epoch < LR_RAMPUP_EPOCHS:\n lr = (LR_MAX - LR_START) / LR_RAMPUP_EPOCHS * epoch + LR_START\n elif epoch < LR_RAMPUP_EPOCHS + LR_SUSTAIN_EPOCHS:\n lr = LR_MAX\n else:\n decay_total_epochs = EPOCHS2 - LR_RAMPUP_EPOCHS - LR_SUSTAIN_EPOCHS - 1\n decay_epoch_index = epoch - LR_RAMPUP_EPOCHS - LR_SUSTAIN_EPOCHS\n phase = math.pi * decay_epoch_index / decay_total_epochs\n cosine_decay = 0.5 * (1 + math.cos(phase))\n lr = (LR_MAX - LR_MIN) * cosine_decay + LR_MIN\n return lr\n\nrng = [i for i in range(EPOCHS2)]\nlr_y = [lrfn(x) for x in rng]\nplt.figure(figsize=(10, 4))\nplt.plot(rng, lr_y, '-o')\nplt.xlabel('epoch',size=14); plt.ylabel('learning rate',size=14)\nplt.title('Cosine Training Schedule',size=16); plt.show()\n\nLR2 = tf.keras.callbacks.LearningRateScheduler(lrfn, verbose = True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2026-02-18T07:47:39.687232Z","iopub.execute_input":"2026-02-18T07:47:39.688047Z","iopub.status.idle":"2026-02-18T07:47:39.828784Z","shell.execute_reply.started":"2026-02-18T07:47:39.688010Z","shell.execute_reply":"2026-02-18T07:47:39.828000Z"},"papermill":{"duration":0.253836,"end_time":"2024-01-21T23:20:14.325961","exception":false,"start_time":"2024-01-21T23:20:14.072125","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"6e8d5d6b","cell_type":"code","source":"LR_START = 1e-4\nLR_MAX = 1e-3\nLR_RAMPUP_EPOCHS = 0\nLR_SUSTAIN_EPOCHS = 1\nLR_STEP_DECAY = 0.1\nEVERY = 1\nEPOCHS = 4\n\ndef lrfn(epoch):\n if epoch < LR_RAMPUP_EPOCHS:\n lr = (LR_MAX - LR_START) / LR_RAMPUP_EPOCHS * epoch + LR_START\n elif epoch < LR_RAMPUP_EPOCHS + LR_SUSTAIN_EPOCHS:\n lr = LR_MAX\n else:\n lr = LR_MAX * LR_STEP_DECAY**((epoch - LR_RAMPUP_EPOCHS - LR_SUSTAIN_EPOCHS)//EVERY)\n return lr\n\nrng = [i for i in range(EPOCHS)]\ny = [lrfn(x) for x in rng]\nplt.figure(figsize=(10, 4))\nplt.plot(rng, y, 'o-'); \nplt.xlabel('epoch',size=14); plt.ylabel('learning rate',size=14)\nplt.title('Step Training Schedule',size=16); plt.show()\n\nLR = tf.keras.callbacks.LearningRateScheduler(lrfn, verbose = True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2026-02-18T07:47:39.831536Z","iopub.execute_input":"2026-02-18T07:47:39.831956Z","iopub.status.idle":"2026-02-18T07:47:39.964546Z","shell.execute_reply.started":"2026-02-18T07:47:39.831927Z","shell.execute_reply":"2026-02-18T07:47:39.963911Z"},"papermill":{"duration":0.255335,"end_time":"2024-01-21T23:20:14.609248","exception":false,"start_time":"2024-01-21T23:20:14.353913","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"fb6c494f","cell_type":"markdown","source":"## Build EfficientNet Model\nVersion 1-3 uses EfficientNet B2. Version 4 uses EfficientNet B0. Our models receives both Kaggle spectrograms and EEG spectrograms from our data loader. We then reshape these 8 spectrograms into 1 large flat image and feed it into EfficientNet.","metadata":{"papermill":{"duration":0.028746,"end_time":"2024-01-21T23:20:14.668270","exception":false,"start_time":"2024-01-21T23:20:14.639524","status":"completed"},"tags":[]}},{"id":"c08c9c5e","cell_type":"code","source":"!pip install --no-index --find-links=/kaggle/input/notebooks/cdeotte/tf-efficientnet-whl-files /kaggle/input/notebooks/cdeotte/tf-efficientnet-whl-files/efficientnet-1.1.1-py3-none-any.whl","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2026-02-18T07:47:39.965481Z","iopub.execute_input":"2026-02-18T07:47:39.965718Z","iopub.status.idle":"2026-02-18T07:47:44.706668Z","shell.execute_reply.started":"2026-02-18T07:47:39.965686Z","shell.execute_reply":"2026-02-18T07:47:44.705764Z"},"papermill":{"duration":15.251348,"end_time":"2024-01-21T23:20:29.990332","exception":false,"start_time":"2024-01-21T23:20:14.738984","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"780a83bd-b441-4838-a2d7-c822d7b89d8e","cell_type":"code","source":"import numpy as np\nfrom sklearn.metrics import accuracy_score","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-18T07:50:38.689819Z","iopub.execute_input":"2026-02-18T07:50:38.690692Z","iopub.status.idle":"2026-02-18T07:50:38.694307Z","shell.execute_reply.started":"2026-02-18T07:50:38.690658Z","shell.execute_reply":"2026-02-18T07:50:38.693573Z"}},"outputs":[],"execution_count":null},{"id":"8d143248","cell_type":"code","source":"import efficientnet.tfkeras as efn\n\ndef build_model():\n \n inp = tf.keras.Input(shape=(128,256,8))\n base_model = efn.EfficientNetB0(include_top=False, weights=None, input_shape=None)\n base_model.load_weights('/kaggle/input/datasets/cdeotte/tf-efficientnet-imagenet-weights/efficientnet-b0_weights_tf_dim_ordering_tf_kernels_autoaugment_notop.h5')\n \n # RESHAPE INPUT 128x256x8 => 512x512x3 MONOTONE IMAGE\n # KAGGLE SPECTROGRAMS\n x1 = [inp[:,:,:,i:i+1] for i in range(4)]\n x1 = tf.keras.layers.Concatenate(axis=1)(x1)\n # EEG SPECTROGRAMS\n x2 = [inp[:,:,:,i+4:i+5] for i in range(4)]\n x2 = tf.keras.layers.Concatenate(axis=1)(x2)\n # MAKE 512X512X3\n if USE_KAGGLE_SPECTROGRAMS & USE_EEG_SPECTROGRAMS:\n x = tf.keras.layers.Concatenate(axis=2)([x1,x2])\n elif USE_EEG_SPECTROGRAMS: x = x2\n else: x = x1\n x = tf.keras.layers.Concatenate(axis=3)([x,x,x])\n \n # OUTPUT\n x = base_model(x)\n x = tf.keras.layers.GlobalAveragePooling2D()(x)\n x = tf.keras.layers.Dense(6,activation='softmax', dtype='float32')(x)\n \n # COMPILE MODEL\n model = tf.keras.Model(inputs=inp, outputs=x)\n opt = tf.keras.optimizers.Adam(learning_rate = 1e-3)\n loss = tf.keras.losses.KLDivergence()\n\n model.compile(loss=loss, optimizer = opt, metrics=['accuracy']) \n \n return model","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:47:44.708025Z","iopub.execute_input":"2026-02-18T07:47:44.708351Z","iopub.status.idle":"2026-02-18T07:47:44.791512Z","shell.execute_reply.started":"2026-02-18T07:47:44.708315Z","shell.execute_reply":"2026-02-18T07:47:44.790963Z"},"papermill":{"duration":0.055031,"end_time":"2024-01-21T23:20:30.074915","exception":false,"start_time":"2024-01-21T23:20:30.019884","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"b58788eb","cell_type":"markdown","source":"# Train Model\nWe train using Group KFold on patient id. If `LOAD_MODELS_FROM = None`, then we will train new models in this notebook version. Otherwise we will load saved models from the path `LOAD_MODELS_FROM`.","metadata":{"papermill":{"duration":0.02914,"end_time":"2024-01-21T23:20:30.133819","exception":false,"start_time":"2024-01-21T23:20:30.104679","status":"completed"},"tags":[]}},{"id":"24f5b7a2","cell_type":"code","source":"from sklearn.model_selection import KFold, GroupKFold\nimport tensorflow.keras.backend as K, gc\n\nall_oof = []\nall_true = []\n\ngkf = GroupKFold(n_splits=5)\nfor i, (train_index, valid_index) in enumerate(gkf.split(train, train.target, train.patient_id)): \n \n print('#'*25)\n print(f'### Fold {i+1}')\n \n train_gen = DataGenerator(train.iloc[train_index], shuffle=True, batch_size=32, augment=False)\n valid_gen = DataGenerator(train.iloc[valid_index], shuffle=False, batch_size=64, mode='valid')\n \n print(f'### train size {len(train_index)}, valid size {len(valid_index)}')\n print('#'*25)\n \n K.clear_session()\n with strategy.scope():\n model = build_model()\n if LOAD_MODELS_FROM is None:\n model.fit(train_gen, verbose=1,\n validation_data = valid_gen,\n epochs=EPOCHS, callbacks = [LR])\n model.save_weights(f'EffNet_v{VER}_f{i}.h5')\n else:\n model.load_weights(f'{LOAD_MODELS_FROM}EffNet_v{VER}_f{i}.h5')\n \n oof = model.predict(valid_gen, verbose=1)\n # --- BURAYI EKLEYİN ---\n # O anki fold için gerçek etiketleri al (argmax ile sınıfa çevir)\n fold_true = np.argmax(train.iloc[valid_index][TARGETS].values, axis=1)\n # Modelin o anki fold için tahminlerini al (argmax ile sınıfa çevir)\n fold_pred = np.argmax(oof, axis=1)\n \n # Doğruluğu hesapla\n fold_acc = accuracy_score(fold_true, fold_pred)\n print(f'>>> Fold {i+1} Validation Accuracy: {fold_acc:.4f}')\n # ----------------------\n all_oof.append(oof)\n all_true.append(train.iloc[valid_index][TARGETS].values)\n \n del model, oof\n gc.collect()\n \nall_oof = np.concatenate(all_oof)\nall_true = np.concatenate(all_true)","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:50:44.162460Z","iopub.execute_input":"2026-02-18T07:50:44.162996Z","iopub.status.idle":"2026-02-18T07:53:47.681044Z","shell.execute_reply.started":"2026-02-18T07:50:44.162967Z","shell.execute_reply":"2026-02-18T07:53:47.680341Z"},"papermill":{"duration":178.356498,"end_time":"2024-01-21T23:23:28.519481","exception":false,"start_time":"2024-01-21T23:20:30.162983","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"7dc940d0","cell_type":"markdown","source":"# CV Score for EfficientNet\nThis is CV score for our EfficientNet model.","metadata":{"papermill":{"duration":0.053816,"end_time":"2024-01-21T23:23:28.626516","exception":false,"start_time":"2024-01-21T23:23:28.572700","status":"completed"},"tags":[]}},{"id":"9f62e585","cell_type":"code","source":"import sys\nsys.path.append('/kaggle/input/datasets/cdeotte/kaggle-kl-div')\nfrom kaggle_kl_div import score\n\noof = pd.DataFrame(all_oof.copy())\noof['id'] = np.arange(len(oof))\n\ntrue = pd.DataFrame(all_true.copy())\ntrue['id'] = np.arange(len(true))\n\ncv = score(solution=true, submission=oof, row_id_column_name='id')\nprint('CV Score KL-Div for EfficientNetB2 =',cv)","metadata":{"execution":{"iopub.status.busy":"2026-02-18T08:01:17.002784Z","iopub.execute_input":"2026-02-18T08:01:17.003676Z","iopub.status.idle":"2026-02-18T08:01:17.071325Z","shell.execute_reply.started":"2026-02-18T08:01:17.003642Z","shell.execute_reply":"2026-02-18T08:01:17.070416Z"},"papermill":{"duration":0.141291,"end_time":"2024-01-21T23:23:28.819098","exception":false,"start_time":"2024-01-21T23:23:28.677807","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"7aa655d0","cell_type":"markdown","source":"# Infer Test and Create Submission CSV\nBelow we use our 5 EfficientNet fold models to infer the test data and create a `submission.csv` file.","metadata":{"papermill":{"duration":0.050827,"end_time":"2024-01-21T23:23:28.921375","exception":false,"start_time":"2024-01-21T23:23:28.870548","status":"completed"},"tags":[]}},{"id":"02fd7457","cell_type":"code","source":"del all_eegs, spectrograms; gc.collect()\ntest = pd.read_csv('/kaggle/input/competitions/hms-harmful-brain-activity-classification/test.csv')\nprint('Test shape',test.shape)\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:48:55.133266Z","iopub.status.idle":"2026-02-18T07:48:55.133598Z","shell.execute_reply.started":"2026-02-18T07:48:55.133466Z","shell.execute_reply":"2026-02-18T07:48:55.133483Z"},"papermill":{"duration":0.267789,"end_time":"2024-01-21T23:23:29.239157","exception":false,"start_time":"2024-01-21T23:23:28.971368","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"a646bae6","cell_type":"code","source":"# READ ALL SPECTROGRAMS\nPATH2 = '/kaggle/input/competitions/hms-harmful-brain-activity-classification/test_spectrograms/'\nfiles2 = os.listdir(PATH2)\nprint(f'There are {len(files2)} test spectrogram parquets')\n \nspectrograms2 = {}\nfor i,f in enumerate(files2):\n if i%100==0: print(i,', ',end='')\n tmp = pd.read_parquet(f'{PATH2}{f}')\n name = int(f.split('.')[0])\n spectrograms2[name] = tmp.iloc[:,1:].values\n \n# RENAME FOR DATALOADER\ntest = test.rename({'spectrogram_id':'spec_id'},axis=1)","metadata":{"execution":{"iopub.status.busy":"2026-02-18T07:48:55.135035Z","iopub.status.idle":"2026-02-18T07:48:55.135381Z","shell.execute_reply.started":"2026-02-18T07:48:55.135205Z","shell.execute_reply":"2026-02-18T07:48:55.135226Z"},"papermill":{"duration":0.444751,"end_time":"2024-01-21T23:23:29.736674","exception":false,"start_time":"2024-01-21T23:23:29.291923","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"f2d479b0","cell_type":"code","source":"import pywt, librosa\n\nUSE_WAVELET = None \n\nNAMES = ['LL','LP','RP','RR']\n\nFEATS = [['Fp1','F7','T3','T5','O1'],\n ['Fp1','F3','C3','P3','O1'],\n ['Fp2','F8','T4','T6','O2'],\n ['Fp2','F4','C4','P4','O2']]\n\n# DENOISE FUNCTION\ndef maddest(d, axis=None):\n return np.mean(np.absolute(d - np.mean(d, axis)), axis)\n\ndef denoise(x, wavelet='haar', level=1): \n coeff = pywt.wavedec(x, wavelet, mode=\"per\")\n sigma = (1/0.6745) * maddest(coeff[-level])\n\n uthresh = sigma * np.sqrt(2*np.log(len(x)))\n coeff[1:] = (pywt.threshold(i, value=uthresh, mode='hard') for i in coeff[1:])\n\n ret=pywt.waverec(coeff, wavelet, mode='per')\n \n return ret\n\ndef spectrogram_from_eeg(parquet_path, display=False):\n \n # LOAD MIDDLE 50 SECONDS OF EEG SERIES\n eeg = pd.read_parquet(parquet_path)\n middle = (len(eeg)-10_000)//2\n eeg = eeg.iloc[middle:middle+10_000]\n \n # VARIABLE TO HOLD SPECTROGRAM\n img = np.zeros((128,256,4),dtype='float32')\n \n if display: plt.figure(figsize=(10,7))\n signals = []\n for k in range(4):\n COLS = FEATS[k]\n \n for kk in range(4):\n \n # COMPUTE PAIR DIFFERENCES\n x = eeg[COLS[kk]].values - eeg[COLS[kk+1]].values\n\n # FILL NANS\n m = np.nanmean(x)\n if np.isnan(x).mean()<1: x = np.nan_to_num(x,nan=m)\n else: x[:] = 0\n\n # DENOISE\n if USE_WAVELET:\n x = denoise(x, wavelet=USE_WAVELET)\n signals.append(x)\n\n # RAW SPECTROGRAM\n mel_spec = librosa.feature.melspectrogram(y=x, sr=200, hop_length=len(x)//256, \n n_fft=1024, n_mels=128, fmin=0, fmax=20, win_length=128)\n\n # LOG TRANSFORM\n width = (mel_spec.shape[1]//32)*32\n mel_spec_db = librosa.power_to_db(mel_spec, ref=np.max).astype(np.float32)[:,:width]\n\n # STANDARDIZE TO -1 TO 1\n mel_spec_db = (mel_spec_db+40)/40 \n img[:,:,k] += mel_spec_db\n \n # AVERAGE THE 4 MONTAGE DIFFERENCES\n img[:,:,k] /= 4.0\n \n if display:\n plt.subplot(2,2,k+1)\n plt.imshow(img[:,:,k],aspect='auto',origin='lower')\n plt.title(f'EEG {eeg_id} - Spectrogram {NAMES[k]}')\n \n if display: \n plt.show()\n plt.figure(figsize=(10,5))\n offset = 0\n for k in range(4):\n if k>0: offset -= signals[3-k].min()\n plt.plot(range(10_000),signals[k]+offset,label=NAMES[3-k])\n offset += signals[3-k].max()\n plt.legend()\n plt.title(f'EEG {eeg_id} Signals')\n plt.show()\n print(); print('#'*25); print()\n \n return img","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2026-02-18T07:48:55.136892Z","iopub.status.idle":"2026-02-18T07:48:55.137543Z","shell.execute_reply.started":"2026-02-18T07:48:55.137313Z","shell.execute_reply":"2026-02-18T07:48:55.137339Z"},"papermill":{"duration":0.088509,"end_time":"2024-01-21T23:23:29.880636","exception":false,"start_time":"2024-01-21T23:23:29.792127","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"63d69742","cell_type":"code","source":"# READ ALL EEG SPECTROGRAMS\nPATH2 = '/kaggle/input/competitions/hms-harmful-brain-activity-classification/test_eegs/'\nDISPLAY = 1\nEEG_IDS2 = test.eeg_id.unique()\nall_eegs2 = {}\n\nprint('Converting Test EEG to Spectrograms...'); print()\nfor i,eeg_id in enumerate(EEG_IDS2):\n \n # CREATE SPECTROGRAM FROM EEG PARQUET\n img = spectrogram_from_eeg(f'{PATH2}{eeg_id}.parquet', i