{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7392733,"sourceType":"datasetVersion","datasetId":4297749},{"sourceId":7392775,"sourceType":"datasetVersion","datasetId":4297782},{"sourceId":7447509,"sourceType":"datasetVersion","datasetId":4334995},{"sourceId":7585255,"sourceType":"datasetVersion","datasetId":4415285},{"sourceId":7649528,"sourceType":"datasetVersion","datasetId":4430326},{"sourceId":154204277,"sourceType":"kernelVersion"}],"dockerImageVersionId":30646,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Brain Solver Python Training Notebook\n\nThis notebook utilizes the custom `brain_solver` package for analyzing brain activity data. Our data sources include official datasets from Kaggle competitions and additional datasets for enhanced model training and evaluation.\n\n## Data Sources\n\n### Official:\n\n- **HMS - Harmful Brain Activity Classification**\n  - **Source:** [Kaggle Competition](https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification)\n  - **Description:** This competition focuses on classifying harmful brain activity. It includes a comprehensive dataset for training and testing models.\n\n- **Brain-Spectrograms**\n  - **Source:** [Kaggle Dataset](https://www.kaggle.com/datasets/cdeotte/brain-spectrograms)\n  - **Description:** The `specs.npy` file contains all the spectrograms from the HMS competition, offering a detailed view of brain activity through visual representations.\n\n### Additional:\n\n- **Brain-EEG-Spectrograms**\n  - **Source:** [Kaggle Dataset](https://www.kaggle.com/datasets/cdeotte/brain-eeg-spectrograms)\n  - **Description:** The `EEG_Spectrograms` folder includes one NumPy file per EEG ID, with each array shaped as (128x256x4), representing (frequency, time, montage chain). This dataset provides a more nuanced understanding of brain activity through EEG spectrograms.\n\n- **hms_efficientnetb0_pt_ckpts**\n  - **Source:** [Kaggle Dataset](https://www.kaggle.com/datasets/crackle/hms-efficientnetb0-pt-ckpts)\n  - **Description:** This dataset offers pre-trained checkpoints for EfficientNetB0 models, tailored for the HMS competition. It's intended for use in fine-tuning models on the specific task of harmful brain activity classification.\n","metadata":{}},{"cell_type":"code","source":"!pip install /kaggle/input/brain-solver/brain_solver-0.9.0-py3-none-any.whl","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:06.471314Z","iopub.execute_input":"2024-02-18T14:17:06.472076Z","iopub.status.idle":"2024-02-18T14:17:20.063185Z","shell.execute_reply.started":"2024-02-18T14:17:06.472042Z","shell.execute_reply":"2024-02-18T14:17:20.062071Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os, sys\nimport gc\nimport numpy as np\nimport pandas as pd\nimport torch\nfrom torch.utils.data import DataLoader\nimport pytorch_lightning as pl\nfrom brain_solver import Helpers as hp, Trainer as tr, BrainModel as br, EEGDataset\nfrom brain_solver import Wav2Vec2 as w2v\n\n# Suppress warnings if desired\nimport warnings\n\nwarnings.filterwarnings(\"ignore\")\n\n# Setup for CUDA device selection\nos.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0,1\"\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\ndevice","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:20.065247Z","iopub.execute_input":"2024-02-18T14:17:20.065602Z","iopub.status.idle":"2024-02-18T14:17:20.076925Z","shell.execute_reply.started":"2024-02-18T14:17:20.065574Z","shell.execute_reply":"2024-02-18T14:17:20.076026Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Config Class Summary\n\nThe `Config` class manages configurations for a brain activity classification project. It includes:\n\n- **Data and Model Paths**: Centralizes paths for data (e.g., EEG, spectrograms) and model checkpoints.\n- **Training Parameters**: Configures training details like epochs, batch size, and learning rate.\n- **Feature Flags**: Toggles for using wavelets, spectrograms, and reading options.\n\nDesigned for easy adjustments to facilitate model development and experimentation.\n","metadata":{}},{"cell_type":"code","source":"from brain_solver import Config\n# full_path = \"/home/osloup/NoodleNappers/brain/data/\"\n# config = Config(full_path,  full_path + \"out/\", USE_EEG_SPECTROGRAMS=True, USE_KAGGLE_SPECTROGRAMS=True, should_read_brain_spectograms=False, should_read_eeg_spectrogram_files=False)\n\n\nfull_path = \"/kaggle/input/\"\nconfig = Config(full_path, \"/kaggle/working/\", USE_EEG_SPECTROGRAMS=True, USE_KAGGLE_SPECTROGRAMS=True, should_read_brain_spectograms=False, should_read_eeg_spectrogram_files=False)\n\nimport sys\nsys.path.append(full_path + 'kaggle-kl-div')\nfrom kaggle_kl_div import score\n","metadata":{"tags":[],"execution":{"iopub.status.busy":"2024-02-18T14:17:20.078077Z","iopub.execute_input":"2024-02-18T14:17:20.078416Z","iopub.status.idle":"2024-02-18T14:17:20.088993Z","shell.execute_reply.started":"2024-02-18T14:17:20.078386Z","shell.execute_reply":"2024-02-18T14:17:20.088233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create Output folder if does not exist\nif not os.path.exists(config.output_path):\n    os.makedirs(config.output_path)\n\n# Initialize random environment\npl.seed_everything(config.seed, workers=True)","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:20.091611Z","iopub.execute_input":"2024-02-18T14:17:20.091908Z","iopub.status.idle":"2024-02-18T14:17:20.102644Z","shell.execute_reply.started":"2024-02-18T14:17:20.091886Z","shell.execute_reply":"2024-02-18T14:17:20.101761Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df: pd.DataFrame = hp.load_csv(config.data_train_csv)\n\nif train_df is None:\n    print(\"Failed to load the CSV file.\")\n    exit()\nelse:\n    EEG_IDS = train_df.eeg_id.unique()\n    TARGETS = train_df.columns[-6:]\n    TARS = {\"Seizure\": 0, \"LPD\": 1, \"GPD\": 2, \"LRDA\": 3, \"GRDA\": 4, \"Other\": 5}\n    TARS_INV = {x: y for y, x in TARS.items()}\n    print(\"Train shape:\", train_df.shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:20.10371Z","iopub.execute_input":"2024-02-18T14:17:20.103958Z","iopub.status.idle":"2024-02-18T14:17:20.315748Z","shell.execute_reply.started":"2024-02-18T14:17:20.103937Z","shell.execute_reply":"2024-02-18T14:17:20.314795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data_preprocessed = hp.preprocess_eeg_data(train_df, TARGETS)","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:20.317043Z","iopub.execute_input":"2024-02-18T14:17:20.317331Z","iopub.status.idle":"2024-02-18T14:17:20.384563Z","shell.execute_reply.started":"2024-02-18T14:17:20.317309Z","shell.execute_reply":"2024-02-18T14:17:20.383627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data_preprocessed.head()","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:20.385908Z","iopub.execute_input":"2024-02-18T14:17:20.386204Z","iopub.status.idle":"2024-02-18T14:17:20.405605Z","shell.execute_reply.started":"2024-02-18T14:17:20.386179Z","shell.execute_reply":"2024-02-18T14:17:20.40455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"config.num_classes = len(TARS.keys())\nconfig.num_classes","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:20.40694Z","iopub.execute_input":"2024-02-18T14:17:20.407318Z","iopub.status.idle":"2024-02-18T14:17:20.418417Z","shell.execute_reply.started":"2024-02-18T14:17:20.407284Z","shell.execute_reply":"2024-02-18T14:17:20.417323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spectrograms = hp.read_spectrograms(\n    config.data_spectograms,\n    config.path_to_brain_spectrograms_npy,\n    config.should_read_brain_spectograms,\n)\n\n#data_eeg_spectograms = hp.read_eeg_spectrograms(\n#    train_data_preprocessed,\n#    config.path_to_eeg_spectrograms_folder,\n#    config.path_to_eeg_spectrograms_npy,\n#    config.should_read_eeg_spectrogram_files,\n#)\n\ndata_eeg_spectograms = [] # So everything stops complaining.\n\n# print(spectrograms)\nprint(w2v.wav2vec2(spectrograms))","metadata":{"execution":{"iopub.status.busy":"2024-02-18T14:17:20.419839Z","iopub.execute_input":"2024-02-18T14:17:20.420197Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\n    f\"Length of spectrograms: {spectrograms.__len__()}, Length of all EEGs: {data_eeg_spectograms.__len__()}\"\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset = EEGDataset(\n    train_data_preprocessed, spectrograms, data_eeg_spectograms, TARGETS\n)\ndataloader = DataLoader(dataset, batch_size=32, shuffle=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"hp.plot_spectrograms(\n    dataloader, train_data_preprocessed, ROWS=2, COLS=3, BATCHES=2\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del dataset, dataloader\ngc.collect()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = tr(\n    config.trained_weight_file,\n    config.USE_KAGGLE_SPECTROGRAMS,\n    config.USE_EEG_SPECTROGRAMS,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"all_oof, all_true, valid_loaders = br.cross_validate_eeg(\n    config,\n    train_data_preprocessed=train_data_preprocessed,\n    spectrograms=spectrograms,\n    data_eeg_spectograms=data_eeg_spectograms,\n    TARGETS=TARGETS,\n    n_splits=5,\n    batch_size_train=32,\n    batch_size_valid=64,\n    max_epochs=4,\n    num_workers=3,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"all_oof, all_true = br.validate_model_across_folds(config, device, all_oof, all_true, valid_loaders)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"oof = pd.DataFrame(all_oof.copy())\noof[\"id\"] = np.arange(len(oof))\n\ntrue = pd.DataFrame(all_true.copy())\ntrue[\"id\"] = np.arange(len(true))\n\n# Calculate the score\ncv = score(solution=true, submission=oof, row_id_column_name=\"id\")\nprint(\"CV Score KL-Div for EfficientNetB2 =\", cv)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del data_eeg_spectograms, spectrograms\ngc.collect()","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}