{"metadata":{"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7392733,"sourceType":"datasetVersion","datasetId":4297749},{"sourceId":7392775,"sourceType":"datasetVersion","datasetId":4297782},{"sourceId":7465251,"sourceType":"datasetVersion","datasetId":4317718},{"sourceId":7517324,"sourceType":"datasetVersion","datasetId":4378712},{"sourceId":7570342,"sourceType":"datasetVersion","datasetId":4407194},{"sourceId":7752462,"sourceType":"datasetVersion","datasetId":4382744},{"sourceId":7776446,"sourceType":"datasetVersion","datasetId":4550181},{"sourceId":7818976,"sourceType":"datasetVersion","datasetId":4417235},{"sourceId":7819029,"sourceType":"datasetVersion","datasetId":4581021},{"sourceId":160674831,"sourceType":"kernelVersion"},{"sourceId":160700706,"sourceType":"kernelVersion"},{"sourceId":161586765,"sourceType":"kernelVersion"}],"dockerImageVersionId":30636,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":32.293878,"end_time":"2024-02-05T15:16:39.983407","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-02-05T15:16:07.689529","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Notes - This notebook is an extension of the discussion forund here: https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/484124\n\n##### This aids to show th benefits of diversity of different modeling. In this notebook we see LB of 0.41, 0.37, and 0.36 yet when we average them we see far better scores of 0.31. However this does come with its warnings. This notebook uses a combonation of 2 stage and 1 stage which \"could\" be dramatically overfitting the LB depending on the outcome of the distribution of voters in the test set. I would personally suggest a deep review of the methods used in this notebook before you opt to use any strategies from it. Please reference the many warnings about this in the discussions for more details.\n\n#### Lastly, please do check out the hard work of those who have created these datasets and notebooks in the first place! Or who combined them in @Majiaqi111 . A high LB score does not always mean a good solution!","metadata":{}},{"cell_type":"markdown","source":"# Model 1 - @andreasbis 0.41 \nhttps://www.kaggle.com/code/andreasbis/hms-inference-lb-0-41","metadata":{}},{"cell_type":"code","source":"# Importing essential libraries\nimport gc\nimport os\nimport random\nimport warnings\nimport numpy as np\nimport pandas as pd\nfrom IPython.display import display\n\n# PyTorch for deep learning\nimport timm\nimport torch\nimport torch.nn as nn  \nimport torch.optim as optim\nimport torch.nn.functional as F\n\n# torchvision for image processing and augmentation\nimport torchvision.transforms as transforms\n\n# Suppressing minor warnings to keep the output clean\nwarnings.filterwarnings('ignore', category=Warning)\n\n# Reclaim memory no longer in use.\ngc.collect()","metadata":{"execution":{"iopub.execute_input":"2024-02-05T15:16:12.369929Z","iopub.status.busy":"2024-02-05T15:16:12.369295Z","iopub.status.idle":"2024-02-05T15:16:21.495098Z","shell.execute_reply":"2024-02-05T15:16:21.493703Z"},"papermill":{"duration":9.135464,"end_time":"2024-02-05T15:16:21.498701","exception":false,"start_time":"2024-02-05T15:16:12.363237","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Config:\n    seed=42\n    image_transform=transforms.Resize((512, 512))\n    num_folds=5\n    \n# Set the seed for reproducibility across multiple libraries\ndef set_seed(seed):\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = True\n    torch.manual_seed(seed)\n    np.random.seed(seed)\n    random.seed(seed)\n    \nset_seed(Config.seed)","metadata":{"execution":{"iopub.execute_input":"2024-02-05T15:16:21.523632Z","iopub.status.busy":"2024-02-05T15:16:21.523103Z","iopub.status.idle":"2024-02-05T15:16:21.537653Z","shell.execute_reply":"2024-02-05T15:16:21.535824Z"},"papermill":{"duration":0.02447,"end_time":"2024-02-05T15:16:21.540525","exception":false,"start_time":"2024-02-05T15:16:21.516055","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load and store the trained models for each fold into a list\nmodels = []\n\n# Load ResNet34d\nfor i in range(Config.num_folds):\n    # Create the same model architecture as during training\n    model_resnet = timm.create_model('resnet34d', pretrained=False, num_classes=6, in_chans=1)\n    \n    # Load the trained weights from the corresponding file\n    model_resnet.load_state_dict(torch.load(f'/kaggle/input/hms-train-resnet34d/resnet34d_fold{i}.pth', map_location=torch.device('cpu')))\n    \n    # Append the loaded model to the models list\n    models.append(model_resnet)\n\n# Reclaim memory no longer in use.\ngc.collect()\n\n# Load EfficientNetB0\nfor j in range(Config.num_folds):\n    # Create the same model architecture as during training\n    model_effnet_b0 = timm.create_model('efficientnet_b0', pretrained=False, num_classes=6, in_chans=1)\n    \n    # Load the trained weights from the corresponding file\n    model_effnet_b0.load_state_dict(torch.load(f'/kaggle/input/hms-train-efficientnetb0/efficientnet_b0_fold{j}.pth', map_location=torch.device('cpu')))\n    \n    # Append the loaded model to the models list\n    models.append(model_effnet_b0)\n    \n# Reclaim memory no longer in use.\ngc.collect()\n    \n# Load EfficientNetB1\nfor k in range(Config.num_folds):\n    # Create the same model architecture as during training\n    model_effnet_b1 = timm.create_model('efficientnet_b1', pretrained=False, num_classes=6, in_chans=1)\n    \n    # Load the trained weights from the corresponding file\n    model_effnet_b1.load_state_dict(torch.load(f'/kaggle/input/hms-train-efficientnetb1/efficientnet_b1_fold{k}.pth', map_location=torch.device('cpu')))\n    \n    # Append the loaded model to the models list\n    models.append(model_effnet_b1)\n\n# Reclaim memory no longer in use.\ngc.collect()","metadata":{"execution":{"iopub.execute_input":"2024-02-05T15:16:21.560516Z","iopub.status.busy":"2024-02-05T15:16:21.560032Z","iopub.status.idle":"2024-02-05T15:16:33.298155Z","shell.execute_reply":"2024-02-05T15:16:33.296986Z"},"papermill":{"duration":11.746399,"end_time":"2024-02-05T15:16:33.300965","exception":false,"start_time":"2024-02-05T15:16:21.554566","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load test data and sample submission dataframe\ntest_df = pd.read_csv(\"/kaggle/input/hms-harmful-brain-activity-classification/test.csv\")\nsubmission = pd.read_csv(\"/kaggle/input/hms-harmful-brain-activity-classification/sample_submission.csv\")\n\n# Merge the submission dataframe with the test data on EEG IDs\nsubmission = submission.merge(test_df, on='eeg_id', how='left')\n\n# Generate file paths for each spectrogram based on the EEG data in the submission dataframe\nsubmission['path'] = submission['spectrogram_id'].apply(lambda x: f\"/kaggle/input/hms-harmful-brain-activity-classification/test_spectrograms/{x}.parquet\")\n\n# Display the first few rows of the submission dataframe\ndisplay(submission.head())\n\n# Reclaim memory no longer in use\ngc.collect()","metadata":{"execution":{"iopub.execute_input":"2024-02-05T15:16:33.31318Z","iopub.status.busy":"2024-02-05T15:16:33.312356Z","iopub.status.idle":"2024-02-05T15:16:33.62502Z","shell.execute_reply":"2024-02-05T15:16:33.623722Z"},"papermill":{"duration":0.322518,"end_time":"2024-02-05T15:16:33.627798","exception":false,"start_time":"2024-02-05T15:16:33.30528","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Define the weights for each model\nweight_resnet34d = 0.26\nweight_effnetb0 = 0.48\nweight_effnetb1 = 0.26\n\n# Get file paths for test spectrograms\npaths = submission['path'].values\ntest_preds = []\n\n# Generate predictions for each spectrogram using all models\nfor path in paths:\n    eps = 1e-6\n    # Read and preprocess spectrogram data\n    data = pd.read_parquet(path)\n    data = data.fillna(-1).values[:, 1:].T\n    data = np.clip(data, np.exp(-6), np.exp(10))\n    data = np.log(data)\n    \n    # Normalize the data\n    data_mean = data.mean(axis=(0, 1))\n    data_std = data.std(axis=(0, 1))\n    data = (data - data_mean) / (data_std + eps)\n    data_tensor = torch.unsqueeze(torch.Tensor(data), dim=0)\n    data = Config.image_transform(data_tensor)\n\n    test_pred = []\n    \n    # Generate predictions using all models\n    for model in models:\n        model.eval()\n        with torch.no_grad():\n            pred = F.softmax(model(data.unsqueeze(0)))[0]\n            pred = pred.detach().cpu().numpy()\n        test_pred.append(pred)\n        \n    # Combine predictions from all models using weighted voting\n    weighted_pred = weight_resnet34d * np.mean(test_pred[:Config.num_folds], axis=0) + \\\n                     weight_effnetb0 * np.mean(test_pred[Config.num_folds:2*Config.num_folds], axis=0) + \\\n                     weight_effnetb1 * np.mean(test_pred[2*Config.num_folds:], axis=0)\n    \n    test_preds.append(weighted_pred)\n\n# Convert the list of predictions to a NumPy array for further processing\ntest_preds = np.array(test_preds)\n\n# Reclaim memory no longer in use\ngc.collect()","metadata":{"execution":{"iopub.execute_input":"2024-02-05T15:16:33.646459Z","iopub.status.busy":"2024-02-05T15:16:33.646007Z","iopub.status.idle":"2024-02-05T15:16:37.957169Z","shell.execute_reply":"2024-02-05T15:16:37.956139Z"},"papermill":{"duration":4.319949,"end_time":"2024-02-05T15:16:37.959646","exception":false,"start_time":"2024-02-05T15:16:33.639697","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load the sample submission file and update it with model predictions for each label\nsub1 = pd.read_csv(\"/kaggle/input/hms-harmful-brain-activity-classification/sample_submission.csv\")\nlabels = ['seizure', 'lpd', 'gpd', 'lrda', 'grda', 'other']\n\n# Assign model predictions to respective columns in the submission DataFrame\nfor i in range(len(labels)):\n    sub1[f'{labels[i]}_vote'] = test_preds[:, i]\n\ndisplay(sub1.head())\n\n# Reclaim memory no longer in use.\ngc.collect()","metadata":{"execution":{"iopub.execute_input":"2024-02-05T15:16:37.979237Z","iopub.status.busy":"2024-02-05T15:16:37.978679Z","iopub.status.idle":"2024-02-05T15:16:38.245997Z","shell.execute_reply":"2024-02-05T15:16:38.244647Z"},"papermill":{"duration":0.275437,"end_time":"2024-02-05T15:16:38.248291","exception":false,"start_time":"2024-02-05T15:16:37.972854","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(sub1.iloc[:,-6:].sum(axis=1).to_string())","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model 2 - @minhsienweng 0.37\nhttps://www.kaggle.com/code/minhsienweng/infer-features-head-starter","metadata":{}},{"cell_type":"code","source":"import os, random, sys\nimport tensorflow as tf\nimport tensorflow\nimport tensorflow.keras.backend as K\nimport pandas as pd, numpy as np\nimport matplotlib.pyplot as plt\nfrom tensorflow.keras.models import load_model\nimport albumentations as albu\nfrom scipy.signal import butter, lfilter\nimport librosa\nfrom sklearn.model_selection import KFold, GroupKFold\nimport tensorflow.keras.backend as K, gc\nfrom tensorflow.keras.layers import Input, Dense, Multiply, Add, Conv1D, Concatenate, LayerNormalization","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"LOAD_MODELS_FROM = '/kaggle/input/features-head-starter-models'\nTARGETS = ['seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Seed the same seed to all \ndef seed_everything(seed=42):\n    np.random.seed(42)\n    random.seed(42)\n    tf.random.set_seed(42)\n\nSEED = 42\nseed_everything(SEED)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import ctypes\nlibc = ctypes.CDLL(\"libc.so.6\")\ndef clear_memory():\n    libc.malloc_trim(0)\n    gc.collect()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# USE SINGLE GPU, MULTIPLE GPUS \ngpus = tf.config.list_physical_devices('GPU')\n# WE USE MIXED PRECISION\ntf.config.optimizer.set_experimental_options({\"auto_mixed_precision\": True})\nif len(gpus)>1:\n    strategy = tf.distribute.MirroredStrategy()\n    print(f'Using {len(gpus)} GPUs')\nelse:\n    strategy = tf.distribute.OneDeviceStrategy(device=\"/gpu:0\")\n    print(f'Using {len(gpus)} GPU')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from abc import ABC, abstractmethod \n  \nFEATS2 = ['Fp1','T3','C3','O1','Fp2','C4','T4','O2']\nFEAT2IDX = {x:y for x,y in zip(FEATS2,range(len(FEATS2)))}\nFEATS = [['Fp1','F7','T3','T5','O1'],\n         ['Fp1','F3','C3','P3','O1'],\n         ['Fp2','F8','T4','T6','O2'],\n         ['Fp2','F4','C4','P4','O2']]\nUSE_PROCESSED = True # Use processed downsampled Raw EEG \n    \nclass BaseDataGenerator():\n    'Generates data for Keras'\n    def __init__(self, data, specs, eeg_specs, raw_eegs, augment, mode, data_type): \n        self.data = data\n        self.augment = augment\n        self.mode = mode\n        self.data_type = data_type\n        self.specs = specs\n        self.eeg_specs = eeg_specs\n        self.raw_eegs = raw_eegs\n        self.on_epoch_end()\n        \n    def __len__(self):\n        return self.data.shape[0]\n\n    def __getitem__(self, index):\n        X, y = self.data_generation(index)\n        if self.augment: X = self.augmentation(X)\n        return X, y\n    \n    def __call__(self):\n        for i in range(self.__len__()):\n            yield self.__getitem__(i)\n            \n            if i == self.__len__()-1:\n                self.on_epoch_end()\n                \n    def on_epoch_end(self):\n        if self.mode=='train': \n            self.data = self.data.sample(frac=1).reset_index(drop=True)\n    \n    # Abstract method generate data based on the trained data type\n    @abstractmethod\n    def data_generation(self, index):\n        pass\n        \n    def butter_lowpass_filter(self, data, cutoff_freq=20, sampling_rate=200, order=4):\n        nyquist = 0.5 * sampling_rate\n        normal_cutoff = cutoff_freq / nyquist\n        b, a = butter(order, normal_cutoff, btype='low', analog=False)\n        filtered_data = lfilter(b, a, data, axis=0)\n        return filtered_data\n    \n    def resize(self, img,size):\n        composition = albu.Compose([\n                albu.Resize(size[0],size[1])\n            ])\n        return composition(image=img)['image']\n            \n    def augmentation(self, img):\n        composition = albu.Compose([\n                albu.HorizontalFlip(p=0.4)\n            ])\n        return composition(image=img)['image']","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Implementation class generates the data based on the data_type \nclass DataGenerator(BaseDataGenerator):\n    'Generates data for Keras'\n    def __init__(self, data, specs, eeg_specs, raw_eegs, augment, \n                 mode, data_type): \n        super().__init__(data, specs, eeg_specs, raw_eegs, augment, mode, data_type)\n    \n    def data_generation(self, index):\n        if self.data_type == 'KE':\n            X,y = self.generate_all_specs(index)\n        elif self.data_type == 'E' or self.data_type == 'K':\n            X,y = self.generate_specs(index)\n        elif self.data_type == 'R':\n            X,y = self.generate_raw(index)\n        elif self.data_type in ['ER','KR']:\n            X1,y = self.generate_specs(index)\n            X2,y = self.generate_raw(index)\n            X = (X1,X2)\n        elif self.data_type == 'KER':\n            X1,y = self.generate_all_specs(index)\n            X2,y = self.generate_raw(index)\n            X = (X1,X2)\n        return X,y\n    \n    def generate_all_specs(self, index):\n        X = np.zeros((512,512,3),dtype='float32')\n        y = np.zeros((6,),dtype='float32')\n        \n        row = self.data.iloc[index]\n        if self.mode=='test': \n            offset = 0\n        else:\n            offset = int(row.offset/2)\n        \n        eeg = self.eeg_specs[row.eeg_id]\n        spec = self.specs[row.spec_id]\n        \n        imgs = [spec[offset:offset+300,k*100:(k+1)*100].T for k in [0,2,1,3]] # to match kaggle with eeg\n        img = np.stack(imgs,axis=-1)\n        # LOG TRANSFORM SPECTROGRAM\n        img = np.clip(img,np.exp(-4),np.exp(8))\n        img = np.log(img)\n            \n        # STANDARDIZE PER IMAGE\n        img = np.nan_to_num(img, nan=0.0)    \n            \n        mn = img.flatten().min()\n        mx = img.flatten().max()\n        ep = 1e-5\n        img = 255 * (img - mn) / (mx - mn + ep)\n        \n        X[0_0+56:100+56,:256,0] = img[:,22:-22,0] # LL_k\n        X[100+56:200+56,:256,0] = img[:,22:-22,2] # RL_k\n        X[0_0+56:100+56,:256,1] = img[:,22:-22,1] # LP_k\n        X[100+56:200+56,:256,1] = img[:,22:-22,3] # RP_k\n        X[0_0+56:100+56,:256,2] = img[:,22:-22,2] # RL_k\n        X[100+56:200+56,:256,2] = img[:,22:-22,1] # LP_k\n        \n        X[0_0+56:100+56,256:,0] = img[:,22:-22,0] # LL_k\n        X[100+56:200+56,256:,0] = img[:,22:-22,2] # RL_k\n        X[0_0+56:100+56,256:,1] = img[:,22:-22,1] # LP_k\n        X[100+56:200+56,256:,1] = img[:,22:-22,3] # RP_K\n        \n        # EEG\n        img = eeg\n        mn = img.flatten().min()\n        mx = img.flatten().max()\n        ep = 1e-5\n        img = 255 * (img - mn) / (mx - mn + ep)\n        X[200+56:300+56,:256,0] = img[:,22:-22,0] # LL_e\n        X[300+56:400+56,:256,0] = img[:,22:-22,2] # RL_e\n        X[200+56:300+56,:256,1] = img[:,22:-22,1] # LP_e\n        X[300+56:400+56,:256,1] = img[:,22:-22,3] # RP_e\n        X[200+56:300+56,:256,2] = img[:,22:-22,2] # RL_e\n        X[300+56:400+56,:256,2] = img[:,22:-22,1] # LP_e\n        \n        X[200+56:300+56,256:,0] = img[:,22:-22,0] # LL_e\n        X[300+56:400+56,256:,0] = img[:,22:-22,2] # RL_e\n        X[200+56:300+56,256:,1] = img[:,22:-22,1] # LP_e\n        X[300+56:400+56,256:,1] = img[:,22:-22,3] # RP_e\n\n        if self.mode!='test':\n            y[:] = row[TARGETS]\n        \n        return X,y\n    \n    def generate_specs(self, index):\n        X = np.zeros((512,512,3),dtype='float32')\n        y = np.zeros((6,),dtype='float32')\n        \n        row = self.data.iloc[index]\n        if self.mode=='test': \n            offset = 0\n        else:\n            offset = int(row.offset/2)\n            \n        if self.data_type in ['E','ER']:\n            img = self.eeg_specs[row.eeg_id]\n        elif self.data_type in ['K','KR']:\n            spec = self.specs[row.spec_id]\n            imgs = [spec[offset:offset+300,k*100:(k+1)*100].T for k in [0,2,1,3]] # to match kaggle with eeg\n            img = np.stack(imgs,axis=-1)\n            # LOG TRANSFORM SPECTROGRAM\n            img = np.clip(img,np.exp(-4),np.exp(8))\n            img = np.log(img)\n            \n            # STANDARDIZE PER IMAGE\n            img = np.nan_to_num(img, nan=0.0)    \n            \n        mn = img.flatten().min()\n        mx = img.flatten().max()\n        ep = 1e-5\n        img = 255 * (img - mn) / (mx - mn + ep)\n        \n        X[0_0+56:100+56,:256,0] = img[:,22:-22,0]\n        X[100+56:200+56,:256,0] = img[:,22:-22,2]\n        X[0_0+56:100+56,:256,1] = img[:,22:-22,1]\n        X[100+56:200+56,:256,1] = img[:,22:-22,3]\n        X[0_0+56:100+56,:256,2] = img[:,22:-22,2]\n        X[100+56:200+56,:256,2] = img[:,22:-22,1]\n        \n        X[0_0+56:100+56,256:,0] = img[:,22:-22,0]\n        X[100+56:200+56,256:,0] = img[:,22:-22,1]\n        X[0_0+56:100+56,256:,1] = img[:,22:-22,2]\n        X[100+56:200+56,256:,1] = img[:,22:-22,3]\n        \n        X[200+56:300+56,:256,0] = img[:,22:-22,0]\n        X[300+56:400+56,:256,0] = img[:,22:-22,1]\n        X[200+56:300+56,:256,1] = img[:,22:-22,2]\n        X[300+56:400+56,:256,1] = img[:,22:-22,3]\n        X[200+56:300+56,:256,2] = img[:,22:-22,3]\n        X[300+56:400+56,:256,2] = img[:,22:-22,2]\n        \n        X[200+56:300+56,256:,0] = img[:,22:-22,0]\n        X[300+56:400+56,256:,0] = img[:,22:-22,2]\n        X[200+56:300+56,256:,1] = img[:,22:-22,1]\n        X[300+56:400+56,256:,1] = img[:,22:-22,3]\n        \n        if self.mode!='test':\n            y[:] = row[TARGETS]\n        \n        return X,y\n    \n    def generate_raw(self,index):\n        if USE_PROCESSED and self.mode!='test':\n            X = np.zeros((2_000,8),dtype='float32')\n            y = np.zeros((6,),dtype='float32')\n            row = self.data.iloc[index]\n            X = self.raw_eegs[row.eeg_id]\n            y[:] = row[TARGETS]\n            return X,y\n        \n        X = np.zeros((10_000,8),dtype='float32')\n        y = np.zeros((6,),dtype='float32')\n        \n        row = self.data.iloc[index]\n        eeg = self.raw_eegs[row.eeg_id]\n            \n        # FEATURE ENGINEER\n        X[:,0] = eeg[:,FEAT2IDX['Fp1']] - eeg[:,FEAT2IDX['T3']]\n        X[:,1] = eeg[:,FEAT2IDX['T3']] - eeg[:,FEAT2IDX['O1']]\n            \n        X[:,2] = eeg[:,FEAT2IDX['Fp1']] - eeg[:,FEAT2IDX['C3']]\n        X[:,3] = eeg[:,FEAT2IDX['C3']] - eeg[:,FEAT2IDX['O1']]\n            \n        X[:,4] = eeg[:,FEAT2IDX['Fp2']] - eeg[:,FEAT2IDX['C4']]\n        X[:,5] = eeg[:,FEAT2IDX['C4']] - eeg[:,FEAT2IDX['O2']]\n            \n        X[:,6] = eeg[:,FEAT2IDX['Fp2']] - eeg[:,FEAT2IDX['T4']]\n        X[:,7] = eeg[:,FEAT2IDX['T4']] - eeg[:,FEAT2IDX['O2']]\n            \n        # STANDARDIZE\n        X = np.clip(X,-1024,1024)\n        X = np.nan_to_num(X, nan=0) / 32.0\n            \n        # BUTTER LOW-PASS FILTER\n        X = self.butter_lowpass_filter(X)\n        # Downsample\n        X = X[::5,:]\n        \n        if self.mode!='test':\n            y[:] = row[TARGETS]\n                \n        return X,y","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_dataset(data, mode, data_type, specs, eeg_specs, raw_eegs, augment=False,\n                   batch_size=8):    \n    if data_type in ['K','E','KE', 'K+E']: \n        inp = tf.TensorSpec(shape=(512,512,3), dtype=tf.float32)\n    elif data_type in ['KR','ER','KER']:\n        inp = (tf.TensorSpec(shape=(512,512,3), dtype=tf.float32), \n               tf.TensorSpec(shape=(2000,8), dtype=tf.float32))\n    elif data_type in ['R']:\n        inp = tf.TensorSpec(shape=(2000,8), dtype=tf.float32)\n\n    output_signature = (inp, tf.TensorSpec(shape=(6,), dtype=tf.float32))\n    \n    \n    # Create the data generator\n    gen = DataGenerator(data, specs, eeg_specs, raw_eegs, augment, mode, data_type)\n    # Create the dataset from data generator\n    dataset = tf.data.Dataset.from_generator(generator=gen,\n                                             output_signature=output_signature).batch(batch_size * strategy.num_replicas_in_sync)\n    return dataset","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def spectrogram_from_eeg(parquet_path):    \n    # LOAD MIDDLE 50 SECONDS OF EEG SERIES\n    eeg = pd.read_parquet(parquet_path)\n    middle = (len(eeg)-10_000)//2\n    eeg = eeg.iloc[middle:middle+10_000]\n    \n    # VARIABLE TO HOLD SPECTROGRAM\n    img = np.zeros((100,300,4) ,dtype='float32')\n\n    for k in range(4):\n        COLS = FEATS[k]\n        \n        for kk in range(4):\n            # FILL NANS\n            x1 = eeg[COLS[kk]].values\n            x2 = eeg[COLS[kk+1]].values\n            m = np.nanmean(x1)\n            if np.isnan(x1).mean()<1: \n                x1 = np.nan_to_num(x1,nan=m)\n            else: \n                x1[:] = 0\n            m = np.nanmean(x2)\n            if np.isnan(x2).mean()<1: \n                x2 = np.nan_to_num(x2,nan=m)\n            else: \n                x2[:] = 0\n                \n            # COMPUTE PAIR DIFFERENCES\n            x = x1 - x2\n\n            # RAW SPECTROGRAM\n            mel_spec = librosa.feature.melspectrogram(y=x, sr=200, hop_length=len(x)//300, \n                                  n_fft=1024, n_mels=100, fmin=0, fmax=20, win_length=128)\n            \n            # LOG TRANSFORM\n            width = (mel_spec.shape[1]//30)*30\n            mel_spec_db = librosa.power_to_db(mel_spec, ref=np.max).astype(np.float32)[:,:width]\n            img[:,:,k] += mel_spec_db\n                \n        # AVERAGE THE 4 MONTAGE DIFFERENCES\n        img[:,:,k] /= 4.0\n    return img\n# Read EEG from par files\ndef eeg_from_parquet(parquet_path):\n    eeg = pd.read_parquet(parquet_path, columns=FEATS2)\n    rows = len(eeg)\n    offset = (rows-10_000)//2\n    eeg = eeg.iloc[offset:offset+10_000]\n    data = np.zeros((10_000,len(FEATS2)))\n    for j,col in enumerate(FEATS2):\n        \n        # FILL NAN\n        x = eeg[col].values.astype('float32')\n        m = np.nanmean(x)\n        if np.isnan(x).mean()<1: x = np.nan_to_num(x,nan=m)\n        else: x[:] = 0\n        \n        data[:,j] = x\n\n    return data","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def build_spec_model(hybrid=False):\n    LOAD_BACKBONE_FROM = '/kaggle/input/efficientnetb-tf-keras/EfficientNetB2.h5'\n    base_model = load_model(f'{LOAD_BACKBONE_FROM}')\n    \n    inp = tf.keras.layers.Input((512,512,3))        \n    x = base_model(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    if not hybrid:\n        x = tf.keras.layers.Dense(6,activation='softmax', dtype='float32')(x)\n    # Create the model\n    model_spec = tf.keras.Model(inputs=inp, outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate = 1e-3)\n    loss = tf.keras.losses.KLDivergence()\n    model_spec.compile(loss=loss, optimizer=opt)\n    return model_spec","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def wave_block(x, filters, kernel_size, n):\n    dilation_rates = [2**i for i in range(n)]\n    x = Conv1D(filters = filters,\n               kernel_size = 1,\n               padding = 'same')(x)\n    res_x = x\n    for dilation_rate in dilation_rates:\n        tanh_out = Conv1D(filters = filters,\n                          kernel_size = kernel_size,\n                          padding = 'same', \n                          activation = 'tanh', \n                          dilation_rate = dilation_rate)(x)\n        sigm_out = Conv1D(filters = filters,\n                          kernel_size = kernel_size,\n                          padding = 'same',\n                          activation = 'sigmoid', \n                          dilation_rate = dilation_rate)(x)\n        x = Multiply()([tanh_out, sigm_out])\n        x = Conv1D(filters = filters,\n                   kernel_size = 1,\n                   padding = 'same')(x)\n        res_x = Add()([res_x, x])\n    return res_x\n        \ndef build_wave_model(hybrid=False):\n    # INPUT \n    inp = tf.keras.Input(shape=(2_000,8))\n\n    ############\n    # FEATURE EXTRACTION SUB MODEL\n    inp2 = tf.keras.Input(shape=(2_000,1))\n    x = wave_block(inp2, 8, 4, 6)\n    x = wave_block(x, 16, 4, 6)\n    x = wave_block(x, 32, 4, 6)\n    x = wave_block(x, 64, 4, 6)\n    base_model = tf.keras.Model(inputs=inp2, outputs=x)\n    ###########\n\n    # LEFT TEMPORAL CHAIN\n    x1 = base_model(inp[:,:,0:1])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = base_model(inp[:,:,1:2])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z1 = tf.keras.layers.Average()([x1,x2])\n\n    # LEFT PARASAGITTAL CHAIN\n    x1 = base_model(inp[:,:,2:3])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = base_model(inp[:,:,3:4])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z2 = tf.keras.layers.Average()([x1,x2])\n\n    # RIGHT PARASAGITTAL CHAIN\n    x1 = base_model(inp[:,:,4:5])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = base_model(inp[:,:,5:6])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z3 = tf.keras.layers.Average()([x1,x2])\n\n    # RIGHT TEMPORAL CHAIN\n    x1 = base_model(inp[:,:,6:7])\n    x1 = tf.keras.layers.GlobalAveragePooling1D()(x1)\n    x2 = base_model(inp[:,:,7:8])\n    x2 = tf.keras.layers.GlobalAveragePooling1D()(x2)\n    z4 = tf.keras.layers.Average()([x1,x2])\n\n    # COMBINE CHAINS\n    y = tf.keras.layers.Concatenate()([z1,z2,z3,z4])\n    if not hybrid:\n        y = tf.keras.layers.Dense(64, activation='relu')(y)\n        y = tf.keras.layers.Dense(6,activation='softmax', dtype='float32')(y)\n\n    # COMPILE MODEL\n    model_wave = tf.keras.Model(inputs=inp, outputs=y)\n    opt = tf.keras.optimizers.Adam(learning_rate = 1e-3)\n    loss = tf.keras.losses.KLDivergence()\n    model_wave.compile(loss=loss, optimizer = opt)\n    return model_wave","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def build_hyrbid_model():\n    model_spec = build_spec_model(hybrid=True)\n    model_wave = build_wave_model(hybrid=True)\n    inputs = [model_spec.input, model_wave.input]\n    x = [model_spec.output, model_wave.output]\n    x = tf.keras.layers.Concatenate()(x)\n    x = tf.keras.layers.Dense(6,activation='softmax', dtype='float32')(x)\n\n    # COMPILE MODEL\n    model_hybrid = tf.keras.Model(inputs=inputs, outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate = 1e-3)\n    loss = tf.keras.losses.KLDivergence()\n    model_hybrid.compile(loss=loss, optimizer = opt)\n    return model_hybrid","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_spec = build_spec_model(hybrid=False)\nmodel_wave = build_wave_model(hybrid=False)\nmodel_hybrid = build_hyrbid_model()\n# print(model_spec)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"VERS ={\n    'K': 43, # Kaggle spectrogram\n    'E': 42, # EEG spectrogram \n    'R': 37, # Raw EEG \n    'KE': 47, # Kaggle + EEG spectrogram\n    'KR': 48, # Kaggle spectrogram + Raw EEG \n    'ER': 49, # EEG spectrogram + Raw EEG  \n    'KER': 50, # Kaggle + EGG spectrogram + Raw EEG \n    'K+E': 51, # Kaggle + EEG spectrogram + Data augumentation\n}","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import KFold, GroupKFold\nimport tensorflow.keras.backend as K, gc\n\n# Load training data\ndef add_kl(data):\n    labels = data[TARGETS].values + 1e-5 \n    data['kl'] = tf.keras.losses.KLDivergence(reduction='none')(np.array([[1/6]*6]*len(data)), labels)\n    return data\n    \ndef load_train_data():\n    # Load training features\n    train = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\n    META = ['spectrogram_id','spectrogram_label_offset_seconds','patient_id','expert_consensus']\n    train = train.groupby('eeg_id')[META+TARGETS].agg({**{m:'first' for m in META},**{t:'sum' for t in TARGETS}}).reset_index() \n    train[TARGETS] = train[TARGETS]/train[TARGETS].values.sum(axis=1,keepdims=True)\n    train.columns = ['eeg_id','spec_id','offset','patient_id','target'] + TARGETS\n    train = add_kl(train)\n    # display(train.head(3))\n    # Read all spectrograms\n    train_specs = np.load('/kaggle/input/brain-spectrograms/specs.npy',allow_pickle=True).item()\n    # Read all EEG Spectrograms\n    train_eegs = np.load('/kaggle/input/eeg-spectrograms/eeg_specs.npy',allow_pickle=True).item()\n    # Read all raw EGG signals\n    train_raw_eegs = np.load('/kaggle/input/hms-eeg/eegs_processed.npy',allow_pickle=True).item()\n    return train, train_specs, train_eegs, train_raw_eegs\n\n# Compute the score using KLDivergence \n# KL measures how much prediction prob distribution differs from actual prob distribution\ndef compute_score(y_true, y_pred):\n    kl = tf.keras.metrics.KLDivergence()\n    return kl(y_true, y_pred)\n\n# Submission ON TEST with an individual\ndef preds_with_a_model(data_type):\n    VER=VERS[data_type]\n    train, train_specs, train_eegs, train_raw_eegs = load_train_data()\n    # Make the predictions with 5 fold models\n    all_oof = []\n    all_true = []\n    gkf = GroupKFold(n_splits=5)\n    for i, (_, valid_index) in enumerate(gkf.split(train, train.target, train.patient_id)):   \n        print(f'### Fold {i+1}')\n        true_values = train.iloc[valid_index][TARGETS].values  \n        all_true.append(true_values)\n        # Valid dataset\n        val_df = train.iloc[valid_index]\n        val_dataset = create_dataset(val_df, mode='train', data_type=data_type,\n                                     specs=train_specs, eeg_specs=train_eegs, \n                                     raw_eegs=train_raw_eegs)\n        print(f'valid size {len(valid_index)}')\n        model = None\n        if data_type in ['K','E','KE']:\n            model = model_spec\n        elif data_type in ['R']:\n            model = model_wave\n        elif data_type in ['KR','ER','KER']:\n            model = model_hybrid\n        # EEG's, Kaggle's spectrograms and Raw model\n        model.load_weights(f'{LOAD_MODELS_FROM}/model_{data_type}_{VER}_{i}.weights.h5')\n        oof = model.predict(val_dataset, verbose=1)\n        print(f\"oof shape = {np.array(oof).shape}\")\n        all_oof.append(oof)           \n        del val_df, val_dataset\n        clear_memory()\n\n    # Compute the score with predictions and actual labels\n    all_oof = np.concatenate(all_oof)\n    all_true = np.concatenate(all_true)\n    print(f\"all_oof shape = {all_oof.shape} and all_true shape = {all_true.shape}\")\n    print(f'CV KL SCORE of {data_type} model: {compute_score(all_true,all_oof)}')\n    del train, train_specs, train_eegs, train_raw_eegs\n    clear_memory()\n    \n# # Compute the cross validation score for a model\n# preds_with_a_model(data_type='KE')\n# sys.exit(0)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load testing features\ntest = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/test.csv')\n# Rename\ntest = test.rename({'spectrogram_id':'spec_id'},axis=1)\nprint('Test shape',test.shape)\ntest.head()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Read all spectrograms\nPATH = '/kaggle/input/hms-harmful-brain-activity-classification/test_spectrograms'\nfiles = os.listdir(PATH)\nprint(f'There are {len(files)} test spectrogram parquets')\ntest_specs = {}\nfor i,f in enumerate(files):\n    tmp = pd.read_parquet(f'{PATH}/{f}')\n    name = int(f.split('.')[0])\n    test_specs[name] = tmp.iloc[:,1:].values","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Read all EEG Spectrograms\nPATH = '/kaggle/input/hms-harmful-brain-activity-classification/test_eegs'\nDISPLAY = 0\nEEG_IDS = test.eeg_id.unique()\ntest_eeg_specs = {}\nprint('Converting Test EEG to Spectrograms...')\nfor i,eeg_id in enumerate(EEG_IDS):\n    # CREATE SPECTROGRAM FROM EEG PARQUET\n    test_eeg_specs[eeg_id] = spectrogram_from_eeg(f'{PATH}/{eeg_id}.parquet')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Read all RAW EEG Signals\ntest_raw_eegs = {}\nEEG_IDS = test.eeg_id.unique()\nPATH = '/kaggle/input/hms-harmful-brain-activity-classification/test_eegs'\nprint('Processing Test EEG parquets...')\nfor i,eeg_id in enumerate(EEG_IDS):\n    # SAVE EEG TO PYTHON DICTIONARY OF NUMPY ARRAYS\n    test_raw_eegs[eeg_id] = eeg_from_parquet(f'{PATH}/{eeg_id}.parquet')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params = {'specs':test_specs, 'eeg_specs':test_eeg_specs, 'raw_eegs':test_raw_eegs}\ntest_datasets ={\n    'K': create_dataset(test, data_type='K', mode='test', **params),\n    'E': create_dataset(test, data_type='E', mode='test', **params),\n    'R': create_dataset(test, data_type='R', mode='test', **params),\n    'KE': create_dataset(test, data_type='KE', mode='test', **params),\n    'KR': create_dataset(test, data_type='KR', mode='test', **params),\n    'ER': create_dataset(test, data_type='ER', mode='test', **params),\n    'KER': create_dataset(test, data_type='KER', mode='test', **params),\n}","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"LBs = [0.41,0.39,0.41,0.37,0.39,0.38,0.36] # K|E|R|KE|KR|ER|KER for weighted ensemble we use LBs of each model\n\ndef make_preds(i, data_type):\n    VER = VERS[data_type]\n    test_dataset = test_datasets[data_type]\n    model_preds = None\n    if data_type in ['K','E','KE']:\n        model_spec.load_weights(f'{LOAD_MODELS_FROM}/model_{data_type}_{VER}_{i}.weights.h5')\n        model_preds = model_spec.predict(test_dataset, verbose=1)\n    elif data_type in ['R']:\n        model_wave.load_weights(f'{LOAD_MODELS_FROM}/model_{data_type}_{VER}_{i}.weights.h5')\n        model_preds = model_wave.predict(test_dataset, verbose=1)\n    elif data_type in ['KR','ER','KER']:\n        model_hybrid.load_weights(f'{LOAD_MODELS_FROM}/model_{data_type}_{VER}_{i}.weights.h5')\n        model_preds = model_hybrid.predict(test_dataset, verbose=1)\n    clear_memory()\n    return model_preds\n\n# Submission ON TEST with ensemble\ndef preds_with_ensemble():\n    preds = []\n    # LB SCORE WEIGHTS FOR EACH MODEL\n    lbs = 1 - np.array(LBs)\n    weights = lbs/lbs.sum()\n    for i in range(5):\n        print(f'Fold {i+1}')\n        # 'KE' model (Kaggle and EEG spectrogram)\n        preds.append(make_preds(i, data_type='KE'))\n        \n        ## 'K' model (Kaggle spectrograms)\n#         model_spec.load_weights(f'{LOAD_MODELS_FROM}/model_K_{VER_K}_{i}.weights.h5')\n#         pred_K = model_spec.predict(test_dataset_K, verbose=1)\n        ## EEG's spectrogram model\n#         model_spec.load_weights(f'{LOAD_MODELS_FROM}/model_E_{VER_E}_{i}.weights.h5')\n#         pred_E = model_spec.predict(test_dataset_E, verbose=1)\n        # EEG Raw wavenet model\n#         model_wave.load_weights(f'{LOAD_MODELS_FROM}/model_R_{VER_R}_{i}.weights.h5')\n#         pred_R = model_wave.predict(test_dataset_R, verbose=1)\n       \n#         # Kaggle's spectrogram and Raw model\n#         model_hybrid.load_weights(f'{LOAD_MODELS_FROM}/model_KR_{VER_KR}_{i}.weights.h5')\n#         pred_KR = model_hybrid.predict(test_dataset_KR, verbose=1)\n#         # EEG's spectrogram and Raw model\n#         model_hybrid.load_weights(f'{LOAD_MODELS_FROM}/model_ER_{VER_ER}_{i}.weights.h5')\n#         pred_ER = model_hybrid.predict(test_dataset_ER, verbose=1)\n#         # EEG's, Kaggle's spectrograms and Raw model\n#         model_hybrid.load_weights(f'{LOAD_MODELS_FROM}/model_KER_{VER_KER}_{i}.weights.h5')\n#         pred_KER = model_hybrid.predict(test_dataset_KER, verbose=1)\n        # Combine the predictions from all the model with different weights \n#         pred = np.array([pred_K,pred_E,pred_R,pred_KE,pred_KR,pred_ER,pred_KER])\n#         pred = np.average(pred,axis=0,weights=weights)\n        \n#         preds.append(pred)\n    # Average the prediction of all five fold models\n    avg_pred = np.mean(preds, axis=0)\n    clear_memory()\n    return avg_pred\n# Prediction with \npred_final = preds_with_ensemble()\nprint('Test preds shape', pred_final.shape)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub2 = pd.DataFrame({'eeg_id':test.eeg_id.values})\nsub2[TARGETS] = pred_final\nprint('Submissionn shape',sub2.shape)\ndisplay(sub2.head())","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(sub2.iloc[:,-6:].sum(axis=1).to_string())","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model 3 @konstantinboyko   0.36\nhttps://www.kaggle.com/code/konstantinboyko/hms-full-validation-2-stage-s-train-infer/notebook?scriptVersionId=166580594","metadata":{}},{"cell_type":"code","source":"import os\nimport gc\nimport sys\nimport math\nimport time\nimport random\nimport datetime as dt\nimport numpy as np\nimport pandas as pd\n\nfrom glob import glob\nfrom pathlib import Path\nfrom typing import Dict, List, Union\nfrom scipy.signal import butter, lfilter, freqz\nfrom matplotlib import pyplot as plt\nfrom tqdm.auto import tqdm\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.optim import Adam, SGD, AdamW\nfrom torch.utils.data import DataLoader, Dataset\n\nsys.path.append(\"/kaggle/input/kaggle-kl-div\")\nfrom kaggle_kl_div import score\n\nimport warnings\n\nwarnings.filterwarnings(\"ignore\")\n\ndevice = torch.device(\"cuda\")\nos.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0,1\"\n\n!cat /etc/os-release | grep -oP \"PRETTY_NAME=\\\"\\K([^\\\"]*)\"\nprint(f\"BUILD_DATE={os.environ['BUILD_DATE']}, CONTAINER_NAME={os.environ['CONTAINER_NAME']}\")\n\ntry:\n    print(\n        f\"PyTorch Version:{torch.__version__}, CUDA is available:{torch.cuda.is_available()}, Version CUDA:{torch.version.cuda}\"\n    )\n    print(\n        f\"Device Capability:{torch.cuda.get_device_capability()}, {torch.cuda.get_arch_list()}\"\n    )\n    print(\n        f\"CuDNN Enabled:{torch.backends.cudnn.enabled}, Version:{torch.backends.cudnn.version()}\"\n    )\nexcept Exception:\n    pass","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class APP:\n    jupyter = \"ipykernel\" in globals()\n    if not jupyter:\n        try:\n            if \"IPython\" in globals().get(\"__doc__\", \"\"):\n                jupyter = True\n        except Exception as inst:\n            print(inst)\n\n    kaggle = os.environ.get(\"KAGGLE_KERNEL_RUN_TYPE\", \"\") != \"\"\n    local = os.environ.get(\"DOCKER_USING\", \"\") == \"LOCAL\"\n    date_time_start = dt.datetime.now()\n    dt_start_ymd_hms = date_time_start.strftime(\"%Y.%m.%d_%H-%M-%S\")\n\n    file_run_path = \"\"\n    if jupyter:\n        try:\n            file_run_path = Path(globals().get(\"__vsc_ipynb_file__\", \"\"))\n        except Exception as inst:\n            print(inst)\n\n    else:\n        try:\n            file_run_path = Path(__file__)\n        except Exception as inst:\n            print(inst)\n\n    file_run_name = file_run_path.stem\n    path_app = file_run_path.parent\n    path_run = Path(os.getcwd())\n    path_out = (\n        Path(\"/kaggle/working\")\n        if kaggle\n        else file_run_path / f\"{file_run_name}_{dt_start_ymd_hms}\"\n    )\n\n\nprint(f\"jupyter:{APP.jupyter}, kaggle:{APP.kaggle}, local:{APP.local}\")\nprint(APP.file_run_path)\nprint(APP.path_out)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class CFG:\n    VERSION = 93\n\n    model_name = \"resnet1d_gru\"\n\n    seed = 2024\n    batch_size = 32\n    num_workers = 0\n\n    fixed_kernel_size = 5\n    # kernels = [3, 5, 7, 9]\n    # linear_layer_features = 424\n    kernels = [3, 5, 7, 9, 11]\n    #linear_layer_features = 448  # Full Signal = 10_000\n    #linear_layer_features = 352  # Half Signal = 5_000\n    linear_layer_features = 304   # 1/5  Signal = 2_000\n\n    seq_length = 50  # Second's\n    sampling_rate = 200  # Hz\n    nsamples = seq_length * sampling_rate  # Число семплов\n    out_samples = nsamples // 5\n\n    # bandpass_filter = {\"low\": 0.5, \"high\": 20, \"order\": 2}\n    # rand_filter = {\"probab\": 0.1, \"low\": 10, \"high\": 20, \"band\": 1.0, \"order\": 2}\n    freq_channels = []  # [(8.0, 12.0)]; [(0.5, 4.5)]\n    filter_order = 2\n    random_close_zone = 0.0  # 0.2\n        \n    target_cols = [\n        \"seizure_vote\",\n        \"lpd_vote\",\n        \"gpd_vote\",\n        \"lrda_vote\",\n        \"grda_vote\",\n        \"other_vote\",\n    ]\n\n    # target_preds = [x + \"_pred\" for x in target_cols]\n    # label_to_num = {\"Seizure\": 0, \"LPD\": 1, \"GPD\": 2, \"LRDA\": 3, \"GRDA\": 4, \"Other\": 5}\n    # num_to_label = {v: k for k, v in label_to_num.items()}\n\n    map_features = [\n        (\"Fp1\", \"T3\"),\n        (\"T3\", \"O1\"),\n        (\"Fp1\", \"C3\"),\n        (\"C3\", \"O1\"),\n        (\"Fp2\", \"C4\"),\n        (\"C4\", \"O2\"),\n        (\"Fp2\", \"T4\"),\n        (\"T4\", \"O2\"),\n        #('Fz', 'Cz'), ('Cz', 'Pz'),        \n    ]\n\n    eeg_features = [\"Fp1\", \"T3\", \"C3\", \"O1\", \"Fp2\", \"C4\", \"T4\", \"O2\"]  # 'Fz', 'Cz', 'Pz']\n        # 'F3', 'P3', 'F7', 'T5', 'Fz', 'Cz', 'Pz', 'F4', 'P4', 'F8', 'T6', 'EKG']                    \n    feature_to_index = {x: y for x, y in zip(eeg_features, range(len(eeg_features)))}\n    simple_features = []  # 'Fz', 'Cz', 'Pz', 'EKG'\n\n    # eeg_features = [row for row in feature_to_index]\n    # eeg_feat_size = len(eeg_features)\n    \n    n_map_features = len(map_features)\n    in_channels = n_map_features + n_map_features * len(freq_channels) + len(simple_features)\n    target_size = len(target_cols)\n    \n    PATH = \"/kaggle/input/hms-harmful-brain-activity-classification/\"\n    test_eeg = \"/kaggle/input/hms-harmful-brain-activity-classification/test_eegs/\"\n    test_csv = \"/kaggle/input/hms-harmful-brain-activity-classification/test.csv\"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"koef_1 = 1.0\nmodel_weights = [\n    {\n        'bandpass_filter':{'low':0.5, 'high':20, 'order':2}, \n        'file_data': \n        [\n            #{'koef':koef_1, 'file_mask':\"/kaggle/input/hms-resnet1d-gru-weights-v98/pop_1_weight_oof/*_full.pth\"},\n            #{'koef':koef_1, 'file_mask':\"/kaggle/input/hms-resnet1d-gru-weights-v98/*_full.pth\"},\n            {'koef':koef_1, 'file_mask':\"/kaggle/input/hms-resnet1d-gru-weights-v98/pop_2_weight_oof/*_full.pth\"},\n            #{'koef':koef_1, 'file_mask':\"/kaggle/input/hms-resnet1d-gru-weights-v98/pop_3_weight_oof/*_full.pth\"},\n        ]\n    },\n]","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def init_logger(log_file=\"./test.log\"):\n    from logging import getLogger, INFO, FileHandler, Formatter, StreamHandler\n\n    logger = getLogger(__name__)\n    logger.setLevel(INFO)\n    handler1 = StreamHandler()\n    handler1.setFormatter(Formatter(\"%(message)s\"))\n    handler2 = FileHandler(filename=log_file)\n    handler2.setFormatter(Formatter(\"%(message)s\"))\n    logger.addHandler(handler1)\n    logger.addHandler(handler2)\n    return logger\n\n\ndef asMinutes(s):\n    m = math.floor(s / 60)\n    s -= m * 60\n    return \"%dm %ds\" % (m, s)\n\n\ndef timeSince(since, percent):\n    now = time.time()\n    s = now - since\n    es = s / (percent)\n    rs = es - s\n    return \"%s (remain %s)\" % (asMinutes(s), asMinutes(rs))\n\n\ndef quantize_data(data, classes):\n    mu_x = mu_law_encoding(data, classes)\n    return mu_x  # quantized\n\n\ndef mu_law_encoding(data, mu):\n    mu_x = np.sign(data) * np.log(1 + mu * np.abs(data)) / np.log(mu + 1)\n    return mu_x\n\n\ndef mu_law_expansion(data, mu):\n    s = np.sign(data) * (np.exp(np.abs(data) * np.log(mu + 1)) - 1) / mu\n    return s\n\n\ndef butter_bandpass(lowcut, highcut, fs, order=5):\n    return butter(order, [lowcut, highcut], fs=fs, btype=\"band\")\n\n\ndef butter_bandpass_filter(data, lowcut, highcut, fs, order=5):\n    b, a = butter_bandpass(lowcut, highcut, fs, order=order)\n    y = lfilter(b, a, data)\n    return y\n\n\ndef butter_lowpass_filter(\n    data, cutoff_freq=20, sampling_rate=CFG.sampling_rate, order=4\n):\n    nyquist = 0.5 * sampling_rate\n    normal_cutoff = cutoff_freq / nyquist\n    b, a = butter(order, normal_cutoff, btype=\"low\", analog=False)\n    filtered_data = lfilter(b, a, data, axis=0)\n    return filtered_data\n\n\ndef denoise_filter(x):\n    # Частота дискретизации и желаемые частоты среза (в Гц).\n    # Отфильтруйте шумный сигнал\n    y = butter_bandpass_filter(x, CFG.lowcut, CFG.highcut, CFG.sampling_rate, order=6)\n    y = (y + np.roll(y, -1) + np.roll(y, -2) + np.roll(y, -3)) / 4\n    y = y[0:-1:4]\n    return y","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def eeg_from_parquet(\n    parquet_path: str, display: bool = False, seq_length=CFG.seq_length\n) -> np.ndarray:\n    \"\"\"\n    Эта функция читает файл паркета и извлекает средние 50 секунд показаний. Затем он заполняет значения NaN\n    со средним значением (игнорируя NaN).\n        :param parquet_path: путь к файлу паркета.\n        :param display: отображать графики ЭЭГ или нет.\n        :return data: np.array формы (time_steps, eeg_features) -> (10_000, 8)\n    \"\"\"\n\n    # Вырезаем среднюю 50 секундную часть\n    eeg = pd.read_parquet(parquet_path, columns=CFG.eeg_features)\n    rows = len(eeg)\n\n    # начало смещения данных, чтобы забрать середину\n    offset = (rows - CFG.nsamples) // 2\n\n    # средние 50 секунд, имеет одинаковое количество показаний слева и справа\n    eeg = eeg.iloc[offset : offset + CFG.nsamples]\n\n    if display:\n        plt.figure(figsize=(10, 5))\n        offset = 0\n\n    # Конвертировать в numpy\n\n    # создать заполнитель той же формы с нулями\n    data = np.zeros((CFG.nsamples, len(CFG.eeg_features)))\n\n    for index, feature in enumerate(CFG.eeg_features):\n        x = eeg[feature].values.astype(\"float32\")  # конвертировать в float32\n\n        # Вычисляет среднее арифметическое вдоль указанной оси, игнорируя NaN.\n        mean = np.nanmean(x)\n        nan_percentage = np.isnan(x).mean()  # percentage of NaN values in feature\n\n        # Заполнение значения Nan\n        # Поэлементная проверка на NaN и возврат результата в виде логического массива.\n        if nan_percentage < 1:  # если некоторые значения равны Nan, но не все\n            x = np.nan_to_num(x, nan=mean)\n        else:  # если все значения — Nan\n            x[:] = 0\n        data[:, index] = x\n\n        if display:\n            if index != 0:\n                offset += x.max()\n            plt.plot(range(CFG.nsamples), x - offset, label=feature)\n            offset -= x.min()\n\n    if display:\n        plt.legend()\n        name = parquet_path.split(\"/\")[-1].split(\".\")[0]\n        plt.yticks([])\n        plt.title(f\"EEG {name}\", size=16)\n        plt.show()\n    return data","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class EEGDataset(Dataset):\n    def __init__(\n        self,\n        df: pd.DataFrame,\n        batch_size: int,\n        eegs: Dict[int, np.ndarray],\n        mode: str = \"train\",\n        downsample: int = None,\n        bandpass_filter: Dict[str, Union[int, float]] = None,\n        rand_filter: Dict[str, Union[int, float]] = None,\n    ):\n        self.df = df\n        self.batch_size = batch_size\n        self.mode = mode\n        self.eegs = eegs\n        self.downsample = downsample\n        self.bandpass_filter = bandpass_filter\n        self.rand_filter = rand_filter\n        \n    def __len__(self):\n        \"\"\"\n        Length of dataset.\n        \"\"\"\n        # Обозначает количество пакетов за эпоху\n        return len(self.df)\n\n    def __getitem__(self, index):\n        \"\"\"\n        Get one item.\n        \"\"\"\n        # Сгенерировать один пакет данных\n        X, y_prob = self.__data_generation(index)\n        if self.downsample is not None:\n            X = X[:: self.downsample, :]\n        output = {\n            \"eeg\": torch.tensor(X, dtype=torch.float32),\n            \"labels\": torch.tensor(y_prob, dtype=torch.float32),\n        }\n        return output\n\n    def __data_generation(self, index):\n        # Генерирует данные, содержащие образцы размера партии\n        X = np.zeros(\n            (CFG.out_samples, CFG.in_channels), dtype=\"float32\"\n        )  # Size=(10000, 14)\n\n        row = self.df.iloc[index]  # Строка Pandas\n        data = self.eegs[row.eeg_id]  # Size=(10000, 8)\n        if CFG.nsamples != CFG.out_samples:\n            if self.mode != \"train\":\n                offset = (CFG.nsamples - CFG.out_samples) // 2\n            else:\n                #offset = random.randint(0, CFG.nsamples - CFG.out_samples)                \n                offset = ((CFG.nsamples - CFG.out_samples) * random.randint(0, 1000)) // 1000\n            data = data[offset:offset+CFG.out_samples,:]\n\n        for i, (feat_a, feat_b) in enumerate(CFG.map_features):\n            if self.mode == \"train\" and CFG.random_close_zone > 0 and random.uniform(0.0, 1.0) <= CFG.random_close_zone:\n                continue\n                \n            diff_feat = (\n                data[:, CFG.feature_to_index[feat_a]]\n                - data[:, CFG.feature_to_index[feat_b]]\n            )  # Size=(10000,)\n\n            if not self.bandpass_filter is None:\n                diff_feat = butter_bandpass_filter(\n                    diff_feat,\n                    self.bandpass_filter[\"low\"],\n                    self.bandpass_filter[\"high\"],\n                    CFG.sampling_rate,\n                    order=self.bandpass_filter[\"order\"],\n                )\n                    \n            if (\n                self.mode == \"train\"\n                and not self.rand_filter is None\n                and random.uniform(0.0, 1.0) <= self.rand_filter[\"probab\"]\n            ):\n                lowcut = random.randint(\n                    self.rand_filter[\"low\"], self.rand_filter[\"high\"]\n                )\n                highcut = lowcut + self.rand_filter[\"band\"]\n                diff_feat = butter_bandpass_filter(\n                    diff_feat,\n                    lowcut,\n                    highcut,\n                    CFG.sampling_rate,\n                    order=self.rand_filter[\"order\"],\n                )\n\n            X[:, i] = diff_feat\n\n        n = CFG.n_map_features\n        if len(CFG.freq_channels) > 0:\n            for i in range(CFG.n_map_features):\n                diff_feat = X[:, i]\n                for j, (lowcut, highcut) in enumerate(CFG.freq_channels):\n                    band_feat = butter_bandpass_filter(\n                        diff_feat, lowcut, highcut, CFG.sampling_rate, order=CFG.filter_order,  # 6\n                    )\n                    X[:, n] = band_feat\n                    n += 1\n\n        for spml_feat in CFG.simple_features:\n            feat_val = data[:, CFG.feature_to_index[spml_feat]]\n            \n            if not self.bandpass_filter is None:\n                feat_val = butter_bandpass_filter(\n                    feat_val,\n                    self.bandpass_filter[\"low\"],\n                    self.bandpass_filter[\"high\"],\n                    CFG.sampling_rate,\n                    order=self.bandpass_filter[\"order\"],\n                )\n\n            if (\n                self.mode == \"train\"\n                and not self.rand_filter is None\n                and random.uniform(0.0, 1.0) <= self.rand_filter[\"probab\"]\n            ):\n                lowcut = random.randint(\n                    self.rand_filter[\"low\"], self.rand_filter[\"high\"]\n                )\n                highcut = lowcut + self.rand_filter[\"band\"]\n                feat_val = butter_bandpass_filter(\n                    feat_val,\n                    lowcut,\n                    highcut,\n                    CFG.sampling_rate,\n                    order=self.rand_filter[\"order\"],\n                )\n\n            X[:, n] = feat_val\n            n += 1\n            \n        # Обрезать края превышающие значения [-1024, 1024]\n        X = np.clip(X, -1024, 1024)\n\n        # Замените NaN нулем и разделить все на 32\n        X = np.nan_to_num(X, nan=0) / 32.0\n\n        # обрезать полосовым фильтром верхнюю границу в 20 Hz.\n        X = butter_lowpass_filter(X, order=CFG.filter_order)  # 4\n\n        y_prob = np.zeros(CFG.target_size, dtype=\"float32\")  # Size=(6,)\n        if self.mode != \"test\":\n            y_prob = row[CFG.target_cols].values.astype(np.float32)\n\n        return X, y_prob","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ResNet_1D_Block(nn.Module):\n    def __init__(\n        self,\n        in_channels,\n        out_channels,\n        kernel_size,\n        stride,\n        padding,\n        downsampling,\n        dilation=1,\n        groups=1,\n        dropout=0.0,\n    ):\n        super(ResNet_1D_Block, self).__init__()\n\n        self.bn1 = nn.BatchNorm1d(num_features=in_channels)\n        # self.relu = nn.ReLU(inplace=False)\n        # self.relu_1 = nn.PReLU()\n        # self.relu_2 = nn.PReLU()\n        self.relu_1 = nn.Hardswish()\n        self.relu_2 = nn.Hardswish()\n\n        self.dropout = nn.Dropout(p=dropout, inplace=False)\n        self.conv1 = nn.Conv1d(\n            in_channels=in_channels,\n            out_channels=out_channels,\n            kernel_size=kernel_size,\n            stride=stride,\n            padding=padding,\n            dilation=dilation,\n            groups=groups,\n            bias=False,\n        )\n\n        self.bn2 = nn.BatchNorm1d(num_features=out_channels)\n        self.conv2 = nn.Conv1d(\n            in_channels=out_channels,\n            out_channels=out_channels,\n            kernel_size=kernel_size,\n            stride=stride,\n            padding=padding,\n            dilation=dilation,\n            groups=groups,\n            bias=False,\n        )\n\n        self.maxpool = nn.MaxPool1d(\n            kernel_size=2,\n            stride=2,\n            padding=0,\n            dilation=dilation,\n        )\n        self.downsampling = downsampling\n\n    def forward(self, x):\n        identity = x\n\n        out = self.bn1(x)\n        out = self.relu_1(out)\n        out = self.dropout(out)\n        out = self.conv1(out)\n        out = self.bn2(out)\n        out = self.relu_2(out)\n        out = self.dropout(out)\n        out = self.conv2(out)\n\n        out = self.maxpool(out)\n        identity = self.downsampling(x)\n\n        out += identity\n        return out\n\n\nclass EEGNet(nn.Module):\n    def __init__(\n        self,\n        kernels,\n        in_channels,\n        fixed_kernel_size,\n        num_classes,\n        linear_layer_features,\n        dilation=1,\n        groups=1,\n    ):\n        super(EEGNet, self).__init__()\n        self.kernels = kernels\n        self.planes = 24\n        self.parallel_conv = nn.ModuleList()\n        self.in_channels = in_channels\n\n        for i, kernel_size in enumerate(list(self.kernels)):\n            sep_conv = nn.Conv1d(\n                in_channels=in_channels,\n                out_channels=self.planes,\n                kernel_size=(kernel_size),\n                stride=1,\n                padding=0,\n                dilation=dilation,\n                groups=groups,\n                bias=False,\n            )\n            self.parallel_conv.append(sep_conv)\n\n        self.bn1 = nn.BatchNorm1d(num_features=self.planes)\n        # self.relu = nn.ReLU(inplace=False)\n        # self.relu_1 = nn.ReLU()\n        # self.relu_2 = nn.ReLU()\n        self.relu_1 = nn.SiLU()\n        self.relu_2 = nn.SiLU()\n\n        self.conv1 = nn.Conv1d(\n            in_channels=self.planes,\n            out_channels=self.planes,\n            kernel_size=fixed_kernel_size,\n            stride=2,\n            padding=2,\n            dilation=dilation,\n            groups=groups,\n            bias=False,\n        )\n\n        self.block = self._make_resnet_layer(\n            kernel_size=fixed_kernel_size,\n            stride=1,\n            dilation=dilation,\n            groups=groups,\n            padding=fixed_kernel_size // 2,\n        )\n        self.bn2 = nn.BatchNorm1d(num_features=self.planes)\n        self.avgpool = nn.AvgPool1d(kernel_size=6, stride=6, padding=2)\n\n        self.rnn = nn.GRU(\n            input_size=self.in_channels,\n            hidden_size=128,\n            num_layers=1,\n            bidirectional=True,\n            # dropout=0.2,\n        )\n\n        self.fc = nn.Linear(in_features=linear_layer_features, out_features=num_classes)\n\n    def _make_resnet_layer(\n        self,\n        kernel_size,\n        stride,\n        dilation=1,\n        groups=1,\n        blocks=9,\n        padding=0,\n        dropout=0.0,\n    ):\n        layers = []\n        downsample = None\n        base_width = self.planes\n\n        for i in range(blocks):\n            downsampling = nn.Sequential(\n                nn.MaxPool1d(kernel_size=2, stride=2, padding=0)\n            )\n            layers.append(\n                ResNet_1D_Block(\n                    in_channels=self.planes,\n                    out_channels=self.planes,\n                    kernel_size=kernel_size,\n                    stride=stride,\n                    padding=padding,\n                    downsampling=downsampling,\n                    dilation=dilation,\n                    groups=groups,\n                    dropout=dropout,\n                )\n            )\n        return nn.Sequential(*layers)\n\n    def extract_features(self, x):\n        x = x.permute(0, 2, 1)\n        out_sep = []\n\n        for i in range(len(self.kernels)):\n            sep = self.parallel_conv[i](x)\n            out_sep.append(sep)\n\n        out = torch.cat(out_sep, dim=2)\n        out = self.bn1(out)\n        out = self.relu_1(out)\n        out = self.conv1(out)\n\n        out = self.block(out)\n        out = self.bn2(out)\n        out = self.relu_2(out)\n        out = self.avgpool(out)\n\n        out = out.reshape(out.shape[0], -1)\n        rnn_out, _ = self.rnn(x.permute(0, 2, 1))\n        new_rnn_h = rnn_out[:, -1, :]  # <~~\n\n        new_out = torch.cat([out, new_rnn_h], dim=1)\n        return new_out\n\n    def forward(self, x):\n        new_out = self.extract_features(x)\n        result = self.fc(new_out)\n        return result","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def inference_function(test_loader, model, device):\n    model.eval()  # set model in evaluation mode\n    softmax = nn.Softmax(dim=1)\n    prediction_dict = {}\n    preds = []\n    with tqdm(test_loader, unit=\"test_batch\", desc=\"Inference\") as tqdm_test_loader:\n        for step, batch in enumerate(tqdm_test_loader):\n            X = batch.pop(\"eeg\").to(device)  # send inputs to `device`\n            batch_size = X.size(0)\n            with torch.no_grad():\n                y_preds = model(X)  # forward propagation pass\n            y_preds = softmax(y_preds)\n            preds.append(y_preds.to(\"cpu\").numpy())  # save predictions\n\n    prediction_dict[\"predictions\"] = np.concatenate(\n        preds\n    )  # np.array() of shape (fold_size, target_cols)\n    return prediction_dict","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv(CFG.test_csv)\nprint(f\"Test dataframe shape is: {test_df.shape}\")\ntest_df.head()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_eeg_parquet_paths = glob(CFG.test_eeg + \"*.parquet\")\ntest_eeg_df = pd.read_parquet(test_eeg_parquet_paths[0])\ntest_eeg_features = test_eeg_df.columns\nprint(f\"There are {len(test_eeg_features)} raw eeg features\")\nprint(list(test_eeg_features))\ndel test_eeg_df\n_ = gc.collect()\n\n# %%time\nall_eegs = {}\neeg_ids = test_df.eeg_id.unique()\nfor i, eeg_id in tqdm(enumerate(eeg_ids)):\n    # Save EEG to Python dictionary of numpy arrays\n    eeg_path = CFG.test_eeg + str(eeg_id) + \".parquet\"\n    data = eeg_from_parquet(eeg_path)\n    all_eegs[eeg_id] = data","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"koef_sum = 0\nkoef_count = 0\npredictions = []\nfiles = []\n    \nfor model_block in model_weights:\n    test_dataset = EEGDataset(\n        df=test_df,\n        batch_size=CFG.batch_size,\n        mode=\"test\",\n        eegs=all_eegs,\n        bandpass_filter=model_block['bandpass_filter']\n    )\n\n    if len(predictions) == 0:\n        output = test_dataset[0]\n        X = output[\"eeg\"]\n        print(f\"X shape: {X.shape}\")\n                \n    test_loader = DataLoader(\n        test_dataset,\n        batch_size=CFG.batch_size,\n        shuffle=False,\n        num_workers=CFG.num_workers,\n        pin_memory=True,\n        drop_last=False,\n    )\n\n    model = EEGNet(\n        kernels=CFG.kernels,\n        in_channels=CFG.in_channels,\n        fixed_kernel_size=CFG.fixed_kernel_size,\n        num_classes=CFG.target_size,\n        linear_layer_features=CFG.linear_layer_features,\n    )\n\n    for file_line in model_block['file_data']:\n        koef = file_line['koef']\n        for weight_model_file in glob(file_line['file_mask']):\n            files.append(weight_model_file)\n            checkpoint = torch.load(weight_model_file, map_location=device)\n            model.load_state_dict(checkpoint[\"model\"])\n            model.to(device)\n            prediction_dict = inference_function(test_loader, model, device)\n            predict = prediction_dict[\"predictions\"]\n            predict *= koef\n            koef_sum += koef\n            koef_count += 1\n            predictions.append(predict)\n            torch.cuda.empty_cache()\n            gc.collect()\n\npredictions = np.array(predictions)\nkoef_sum /= koef_count\npredictions /= koef_sum\npredictions = np.mean(predictions, axis=0)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(koef_count, koef_sum)\ndisplay(files)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub3 = pd.DataFrame({\"eeg_id\": test_df.eeg_id.values})\nsub3[CFG.target_cols] = predictions\n\nsub3.to_csv(f\"submission.csv\", index=False)\nprint(f\"Submission shape: {sub3.shape}\")\ndisplay(sub3.head())","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(sub3.iloc[:,-6:].sum(axis=1).to_string())","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission","metadata":{}},{"cell_type":"code","source":"sub_final = pd.DataFrame({\"eeg_id\": test_df.eeg_id.values})\nlabels = ['seizure', 'lpd', 'gpd', 'lrda', 'grda', 'other']\n\nfor i in range(len(labels)):\n    sub1[f'{labels[i]}_vote'] = test_preds[:, i]\nsub2[TARGETS] = pred_final\nsub3[CFG.target_cols] = predictions\n    \nfor label in labels:\n    sub_final[f'{label}_vote'] = (sub1[f'{label}_vote'] + sub2[f'{label}_vote'] + sub3[f'{label}_vote']) / 3.0 \n\nsub_final.to_csv(f\"submission.csv\", index=False)\nprint(f\"Submission shape: {sub_final.shape}\")\ndisplay(sub_final.head())","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(sub_final.iloc[:,-6:].sum(axis=1).to_string())","metadata":{},"execution_count":null,"outputs":[]}]}