{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7392733,"sourceType":"datasetVersion","datasetId":4297749},{"sourceId":7392775,"sourceType":"datasetVersion","datasetId":4297782},{"sourceId":7447509,"sourceType":"datasetVersion","datasetId":4334995},{"sourceId":7465251,"sourceType":"datasetVersion","datasetId":4317718},{"sourceId":7585255,"sourceType":"datasetVersion","datasetId":4415285},{"sourceId":7658681,"sourceType":"datasetVersion","datasetId":4465437},{"sourceId":7758086,"sourceType":"datasetVersion","datasetId":4536784},{"sourceId":7766155,"sourceType":"datasetVersion","datasetId":4494069},{"sourceId":7774266,"sourceType":"datasetVersion","datasetId":4430326},{"sourceId":7775601,"sourceType":"datasetVersion","datasetId":4527518},{"sourceId":154204277,"sourceType":"kernelVersion"}],"dockerImageVersionId":30648,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Brain Solver Python Training Notebook\n\nThis notebook utilizes the custom `brain_solver` package for analyzing brain activity data. Our data sources include official datasets from Kaggle competitions and additional datasets for enhanced model training and evaluation.\n\n## Data Sources\n\n### Official:\n\n- **HMS - Harmful Brain Activity Classification**\n  - **Source:** [Kaggle Competition](https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification)\n  - **Description:** This competition focuses on classifying harmful brain activity. It includes a comprehensive dataset for training and testing models.\n\n- **Brain-Spectrograms**\n  - **Source:** [Kaggle Dataset](https://www.kaggle.com/datasets/cdeotte/brain-spectrograms)\n  - **Description:** The `specs.npy` file contains all the spectrograms from the HMS competition, offering a detailed view of brain activity through visual representations.\n\n### Additional:\n\n- **Brain-EEG-Spectrograms**\n  - **Source:** [Kaggle Dataset](https://www.kaggle.com/datasets/cdeotte/brain-eeg-spectrograms)\n  - **Description:** The `EEG_Spectrograms` folder includes one NumPy file per EEG ID, with each array shaped as (128x256x4), representing (frequency, time, montage chain). This dataset provides a more nuanced understanding of brain activity through EEG spectrograms.\n\n- **hms_efficientnetb0_pt_ckpts**\n  - **Source:** [Kaggle Dataset](https://www.kaggle.com/datasets/crackle/hms-efficientnetb0-pt-ckpts)\n  - **Description:** This dataset offers pre-trained checkpoints for EfficientNetB0 models, tailored for the HMS competition. It's intended for use in fine-tuning models on the specific task of harmful brain activity classification.\n","metadata":{}},{"cell_type":"code","source":"!pip install d2l --no-index --find-links=file:///kaggle/input/d2l-package/d2l/\n!pip install /kaggle/input/brain-solver/brain_solver-0.9.0-py3-none-any.whl","metadata":{"execution":{"iopub.status.busy":"2024-03-06T12:35:04.168616Z","iopub.execute_input":"2024-03-06T12:35:04.168916Z","iopub.status.idle":"2024-03-06T12:35:53.831523Z","shell.execute_reply.started":"2024-03-06T12:35:04.168890Z","shell.execute_reply":"2024-03-06T12:35:53.830331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os, sys\nimport gc\nimport numpy as np\nimport pandas as pd\nimport torch\nfrom torch.utils.data import DataLoader\nimport pytorch_lightning as pl\nfrom brain_solver import Helpers as hp, Trainer as tr, BrainModel as br, EEGDataset, Wav2Vec2 as w2v\n\n# Suppress warnings if desired\nimport warnings\n\nwarnings.filterwarnings(\"ignore\")\n\n# Setup for CUDA device selection\nos.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0,1\"\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\ndevice","metadata":{"execution":{"iopub.status.busy":"2024-03-06T12:35:53.833897Z","iopub.execute_input":"2024-03-06T12:35:53.834296Z","iopub.status.idle":"2024-03-06T12:36:06.267373Z","shell.execute_reply.started":"2024-03-06T12:35:53.834257Z","shell.execute_reply":"2024-03-06T12:36:06.266328Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Config Class Summary\n\nThe `Config` class manages configurations for a brain activity classification project. It includes:\n\n- **Data and Model Paths**: Centralizes paths for data (e.g., EEG, spectrograms) and model checkpoints.\n- **Training Parameters**: Configures training details like epochs, batch size, and learning rate.\n- **Feature Flags**: Toggles for using wavelets, spectrograms, and reading options.\n\nDesigned for easy adjustments to facilitate model development and experimentation.\n","metadata":{}},{"cell_type":"code","source":"from brain_solver import Config\n# full_path = \"/home/osloup/NoodleNappers/brain/data/\"\n# full_path = \"C:/Users/tygof/Documents/Semester 8/MLiP/NoodleNappers/brain/data/\"\n# config = Config(full_path,  full_path + \"out/\", USE_EEG_SPECTROGRAMS=True, USE_KAGGLE_SPECTROGRAMS=True, should_read_brain_spectograms=False, should_read_eeg_spectrogram_files=False, USE_PRETRAINED_MODEL=False)\n\nfull_path = \"/kaggle/input/\"\nconfig = Config(full_path, \"/kaggle/working/\", USE_EEG_SPECTROGRAMS=False, VER=5, USE_KAGGLE_SPECTROGRAMS=True, should_read_brain_spectograms=False, should_read_eeg_spectrogram_files=False, USE_PRETRAINED_MODEL=False)\n\nimport sys\nsys.path.append(full_path + 'kaggle-kl-div')\nfrom kaggle_kl_div import score","metadata":{"tags":[],"execution":{"iopub.status.busy":"2024-03-06T12:36:06.268652Z","iopub.execute_input":"2024-03-06T12:36:06.269113Z","iopub.status.idle":"2024-03-06T12:36:06.295681Z","shell.execute_reply.started":"2024-03-06T12:36:06.269086Z","shell.execute_reply":"2024-03-06T12:36:06.294827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create Output folder if does not exist\nif not os.path.exists(config.output_path):\n    os.makedirs(config.output_path)\n\n# Initialize random environment\npl.seed_everything(config.seed, workers=True)","metadata":{"execution":{"iopub.status.busy":"2024-03-06T12:36:06.298243Z","iopub.execute_input":"2024-03-06T12:36:06.298862Z","iopub.status.idle":"2024-03-06T12:36:06.308060Z","shell.execute_reply.started":"2024-03-06T12:36:06.298826Z","shell.execute_reply":"2024-03-06T12:36:06.307229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df: pd.DataFrame = hp.load_csv(config.data_train_csv)\n\nif train_df is None:\n    print(\"Failed to load the CSV file.\")\n    exit()\nelse:\n    EEG_IDS = train_df.eeg_id.unique()\n    TARGETS = train_df.columns[-6:]\n    TARS = {\"Seizure\": 0, \"LPD\": 1, \"GPD\": 2, \"LRDA\": 3, \"GRDA\": 4, \"Other\": 5}\n    TARS_INV = {x: y for y, x in TARS.items()}\n    print(\"Train shape:\", train_df.shape)","metadata":{"execution":{"iopub.status.busy":"2024-03-06T12:36:06.309185Z","iopub.execute_input":"2024-03-06T12:36:06.309500Z","iopub.status.idle":"2024-03-06T12:36:06.572742Z","shell.execute_reply.started":"2024-03-06T12:36:06.309476Z","shell.execute_reply":"2024-03-06T12:36:06.571782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv(config.data_test_csv)\nprint(\"Test shape\", test_df.shape)\ntest_df.head()","metadata":{"execution":{"iopub.status.busy":"2024-03-06T12:36:06.573861Z","iopub.execute_input":"2024-03-06T12:36:06.574139Z","iopub.status.idle":"2024-03-06T12:36:06.591870Z","shell.execute_reply.started":"2024-03-06T12:36:06.574098Z","shell.execute_reply":"2024-03-06T12:36:06.590810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# READ ALL EEG SPECTROGRAMS\nDISPLAY = 1\nEEG_IDS2 = test_df.eeg_id.unique()\nall_eegs2 = {}\n\nprint(\"Converting Test EEG to Spectrograms...\")\nprint()\nfor i, eeg_id in enumerate(EEG_IDS2):\n\n    # CREATE SPECTROGRAM FROM EEG PARQUET\n    img = hp.spectrogram_from_eeg(\n        f\"{config.data_eeg_test}{eeg_id}.parquet\", i < DISPLAY, config.use_wavelet\n    )\n    all_eegs2[eeg_id] = img","metadata":{"execution":{"iopub.status.busy":"2024-03-06T12:36:06.593146Z","iopub.execute_input":"2024-03-06T12:36:06.593516Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv(config.data_test_csv)\nprint(\"Test shape\", test_df.shape)\ntest_df.head()\nfiles = [str(f) + \".parquet\" for f in test_df[\"spectrogram_id\"]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spectrograms = {}\n\nfor i, f in enumerate(files):\n    name = f[:-8]\n\n    if i % 100 == 0:\n        print(i, \", \", end=\"\")\n\n    try:\n        parquet_file = pd.read_parquet(os.path.join(config.data_spectograms_test, f))\n        \n        parquet_file = parquet_file.iloc[:, 1:].values\n        \n        # replace NaN values with 0\n        parquet_file = np.nan_to_num(parquet_file, nan=0)\n            \n        parquet_file = w2v.wav2vec2(parquet_file, proc_eegs=False, model_path=\"/kaggle/input/w2v-base-960h/\")\n        spectrograms[int(name)] = np.array([parquet_file[0].values])\n\n    except Exception as e:\n        print(f\"ERROR: An unexpected error occurred for {name}: {e}\")\n\n# Continue with renaming for DataLoader\ntest_df = test_df.rename({\"spectrogram_id\": \"spec_id\"}, axis=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# READ ALL EEG SPECTROGRAMS\nDISPLAY = 1\nEEG_IDS2 = test_df.eeg_id.unique()\nall_eegs2 = {}\n\nprint(\"Converting Test EEG to Spectrograms...\")\nprint()\nfor i, eeg_id in enumerate(EEG_IDS2):\n\n    # CREATE SPECTROGRAM FROM EEG PARQUET\n    img = hp.spectrogram_from_eeg(\n        f\"{config.data_eeg_test}{eeg_id}.parquet\", i < DISPLAY, config.use_wavelet\n    )\n    all_eegs2[eeg_id] = img","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# INFER EFFICIENTNET ON TEST\npreds = []\ntest_ds = EEGDataset(\n    test_df, specs=spectrograms, eeg_specs=None, targets=TARGETS, mode=\"test\", w2v_enabled=True, model_eegs=False\n)\ntest_loader = DataLoader(test_ds, shuffle=False, batch_size=64, num_workers=3)\n\nweights = [1]\n\nfor i in range(5):\n    print(\"#\" * 25)\n    print(f\"### Testing Fold {i+1}\")\n\n    ckpt_file = (\n        f\"EffNet_version{config.VER}_fold{i+1}.pth\"\n        if config.trained_model_path is None\n        else f\"{config.trained_model_path}/EffNet_v{config.VER}_f{i}.ckpt\"\n    )\n    \n    model_preds = []\n    for m in [\"no_w2v_model_lowpass/\"]:\n        \n        model = torch.load(config.full_path + \"models-first-wav2vec-training/\" + m + ckpt_file)\n        model = model.to(device).eval()\n        fold_preds = []\n\n        with torch.inference_mode():\n            for test_batch in test_loader:\n                test_batch = test_batch.to(device)\n                pred = torch.softmax(model(test_batch), dim=1).cpu().numpy()\n                fold_preds.append(pred)\n\n                # Delete variables not needed to free up memory\n                del test_batch, pred\n                gc.collect()  # Manually collect garbage\n\n                if device.type == \"cuda\":  # Optionally clear CUDA cache if using GPU\n                    torch.cuda.empty_cache()\n\n            fold_preds = np.concatenate(fold_preds)\n        model_preds.append(fold_preds)\n\n    preds.append(np.average(model_preds, axis=0, weights=weights))\n\n    del model\n    gc.collect()\n    if device.type == \"cuda\":\n        torch.cuda.empty_cache()\n\n\npred = np.mean(preds, axis=0)\nprint()\nprint(\"Test preds shape\", pred.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred[0][2]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.DataFrame({\"eeg_id\": test_df.eeg_id.values})\nsub[TARGETS] = pred\nsub.to_csv(\"submission.csv\", index=False)\nprint(\"Submissionn shape\", sub.shape)\nsub.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# SANITY CHECK TO CONFIRM PREDICTIONS SUM TO ONE\nsub.iloc[:, -6:].sum(axis=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}