{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7526248,"sourceType":"datasetVersion","datasetId":4308295},{"sourceId":6125,"sourceType":"modelInstanceVersion","modelInstanceId":4596,"modelId":2797},{"sourceId":6127,"sourceType":"modelInstanceVersion","modelInstanceId":4598,"modelId":2797},{"sourceId":6160,"sourceType":"modelInstanceVersion","modelInstanceId":4622,"modelId":2799}],"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.14","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"papermill":{"default_parameters":{},"duration":3846.080383,"end_time":"2024-01-14T04:20:19.064569","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-01-14T03:16:12.984186","version":"2.4.0"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"08983a9c6aff42578980f4f7113c3ee2":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4411aefc021d46d0ada7b645eb53ec48","placeholder":"โ€‹","style":"IPY_MODEL_09a10a8cf9334c51857397ed50398c8e","value":"Searching best thr : 100%"}},"09a10a8cf9334c51857397ed50398c8e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1f3989a0c01248328e16875075e9d1c4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_08983a9c6aff42578980f4f7113c3ee2","IPY_MODEL_22cfcc0a7cc6455fbf3bb7c788c8a4e1","IPY_MODEL_c8392e8075224e3b8a020a16c1a08447"],"layout":"IPY_MODEL_6cec9a2c2fac450d87248aed8dd62f86"}},"22cfcc0a7cc6455fbf3bb7c788c8a4e1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_dffe80502d954bdea0bbb6353dbf5515","max":20,"min":0,"orientation":"horizontal","style":"IPY_MODEL_7ce1b34a4f864a42a6619eec82311eb0","value":20}},"4411aefc021d46d0ada7b645eb53ec48":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6cec9a2c2fac450d87248aed8dd62f86":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7ce1b34a4f864a42a6619eec82311eb0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"83fe40a0b8f047cc8602206909d42361":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9384babdb7054d55aecdf3e989ddc926":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c8392e8075224e3b8a020a16c1a08447":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_83fe40a0b8f047cc8602206909d42361","placeholder":"โ€‹","style":"IPY_MODEL_9384babdb7054d55aecdf3e989ddc926","value":" 20/20 [04:34<00:00, 12.66s/it]"}},"dffe80502d954bdea0bbb6353dbf5515":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}}},"version_major":2,"version_minor":0}}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\"Keras
\nThis starter notebook is provided by the Keras team.
","metadata":{"execution":{"iopub.execute_input":"2024-01-10T05:24:31.308329Z","iopub.status.busy":"2024-01-10T05:24:31.307595Z","iopub.status.idle":"2024-01-10T05:24:31.313088Z","shell.execute_reply":"2024-01-10T05:24:31.312113Z","shell.execute_reply.started":"2024-01-10T05:24:31.308287Z"},"papermill":{"duration":0.011755,"end_time":"2024-01-14T03:16:16.447481","exception":false,"start_time":"2024-01-14T03:16:16.435726","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# HMS - Harmful Brain Activity Classification with [KerasCV](https://github.com/keras-team/keras-cv) and [Keras](https://github.com/keras-team/keras)\n\n> The objective of this competition is to classify seizures and other patterns of harmful brain activity in critically ill patients\n\nThis notebook guides you through the process of training and inferring a Deep Learning model, specifically EfficientNetV2, using KerasCV on the competition dataset. Specificaclly, this notebook uses spectrogram of the eeg data to classify the patterns.\n\nFun fact: This notebook is backend-agnostic, supporting TensorFlow, PyTorch, and JAX. Utilizing KerasCV and Keras allows us to choose our preferred backend. Explore more details on [Keras](https://keras.io/keras_core/announcement/).\n\nIn this notebook, you will learn:\n\n* Loading the data efficiently using [`tf.data`](https://www.tensorflow.org/guide/data).\n* Creating the model using KerasCV presets.\n* Training the model.\n* Inference and Submission on test data.\n\n**Note**: For a more in-depth understanding of KerasCV, refer to the [KerasCV guides](https://keras.io/guides/keras_cv/).","metadata":{}},{"cell_type":"markdown","source":"# ๐Ÿ›  | Install Libraries \n\nSince internet access is **disabled** during inference, we cannot install libraries in the usual `!pip install ` manner. Instead, we need to install libraries from local files. In the following cell, we will install libraries from our local files. The installation code stays very similar - we just use the `filepath` instead of the `filename` of the library. So now the code is `!pip install `. \n\n> The `filepath` of these local libraries look quite complicated, but don't be intimidated! Also `--no-deps` argument ensures that we are not installing any additional libraries.","metadata":{"papermill":{"duration":0.011416,"end_time":"2024-01-14T03:16:16.470167","exception":false,"start_time":"2024-01-14T03:16:16.458751","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#!pip install -q /kaggle/input/kerasv3-lib-ds/keras_cv-0.8.2-py3-none-any.whl --no-deps\n#!pip install -q /kaggle/input/kerasv3-lib-ds/tensorflow-2.15.0.post1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl --no-deps\n#!pip install -q /kaggle/input/kerasv3-lib-ds/keras-3.0.4-py3-none-any.whl --no-deps","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:43:34.739318Z","iopub.execute_input":"2024-10-14T17:43:34.740216Z","iopub.status.idle":"2024-10-14T17:43:34.746651Z","shell.execute_reply.started":"2024-10-14T17:43:34.740175Z","shell.execute_reply":"2024-10-14T17:43:34.745744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“š | Import Libraries ","metadata":{"papermill":{"duration":0.010878,"end_time":"2024-01-14T03:17:49.510159","exception":false,"start_time":"2024-01-14T03:17:49.499281","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import os\nos.environ[\"KERAS_BACKEND\"] = \"jax\" # you can also use tensorflow or torch\n\nimport keras_cv\nimport keras\nfrom keras import ops\nimport tensorflow as tf\n\nimport cv2\nimport pandas as pd\nimport numpy as np\nfrom glob import glob\nfrom tqdm.notebook import tqdm\nimport joblib\n\n\nimport matplotlib.pyplot as plt ","metadata":{"papermill":{"duration":10.671979,"end_time":"2024-01-14T03:18:00.193134","exception":false,"start_time":"2024-01-14T03:17:49.521155","status":"completed"},"tags":[],"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-14T17:43:34.763560Z","iopub.execute_input":"2024-10-14T17:43:34.763893Z","iopub.status.idle":"2024-10-14T17:43:52.192116Z","shell.execute_reply.started":"2024-10-14T17:43:34.763860Z","shell.execute_reply":"2024-10-14T17:43:52.191090Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#pip install pywt","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:43:52.194268Z","iopub.execute_input":"2024-10-14T17:43:52.194818Z","iopub.status.idle":"2024-10-14T17:43:52.198657Z","shell.execute_reply.started":"2024-10-14T17:43:52.194784Z","shell.execute_reply":"2024-10-14T17:43:52.197600Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Library Versions","metadata":{"papermill":{"duration":0.010958,"end_time":"2024-01-14T03:18:00.215704","exception":false,"start_time":"2024-01-14T03:18:00.204746","status":"completed"},"tags":[]}},{"cell_type":"code","source":"print(\"TensorFlow:\", tf.__version__)\nprint(\"Keras:\", keras.__version__)\nprint(\"KerasCV:\", keras_cv.__version__)","metadata":{"papermill":{"duration":0.019435,"end_time":"2024-01-14T03:18:00.246368","exception":false,"start_time":"2024-01-14T03:18:00.226933","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:43:52.199911Z","iopub.execute_input":"2024-10-14T17:43:52.200264Z","iopub.status.idle":"2024-10-14T17:43:52.295661Z","shell.execute_reply.started":"2024-10-14T17:43:52.200221Z","shell.execute_reply":"2024-10-14T17:43:52.294638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# โš™๏ธ | Configuration","metadata":{"papermill":{"duration":0.010922,"end_time":"2024-01-14T03:18:00.26855","exception":false,"start_time":"2024-01-14T03:18:00.257628","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class CFG:\n verbose = 1 # Verbosity\n seed = 42 # Random seed\n #preset = \"efficientnetv2_b2_imagenet\" # Name of pretrained classifier\n preset= \"efficientnetv2_b0_imagenet\"\n #preset = \"mobilenet_v3_large_imagenet_classifier\"\n image_size = [400, 300] # Input image size\n #image_size = [224, 224] # Input image size\n epochs = 100 # Training epochs\n batch_size = 64 # Batch size\n lr_mode = \"cos\" # LR scheduler mode from one of \"cos\", \"step\", \"exp\"\n drop_remainder = True # Drop incomplete batches\n num_classes = 6 # Number of classes in the dataset\n fold = 0 # Which fold to set as validation data\n class_names = ['Seizure', 'LPD', 'GPD', 'LRDA','GRDA', 'Other']\n label2name = dict(enumerate(class_names))\n name2label = {v:k for k, v in label2name.items()}","metadata":{"papermill":{"duration":0.018795,"end_time":"2024-01-14T03:18:00.298534","exception":false,"start_time":"2024-01-14T03:18:00.279739","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:43:52.298493Z","iopub.execute_input":"2024-10-14T17:43:52.298975Z","iopub.status.idle":"2024-10-14T17:43:52.307566Z","shell.execute_reply.started":"2024-10-14T17:43:52.298933Z","shell.execute_reply":"2024-10-14T17:43:52.306660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# โ™ป๏ธ | Reproducibility \nSets value for random seed to produce similar result in each run.","metadata":{"papermill":{"duration":0.010907,"end_time":"2024-01-14T03:18:00.32063","exception":false,"start_time":"2024-01-14T03:18:00.309723","status":"completed"},"tags":[]}},{"cell_type":"code","source":"keras.utils.set_random_seed(CFG.seed)","metadata":{"papermill":{"duration":0.018371,"end_time":"2024-01-14T03:18:00.350074","exception":false,"start_time":"2024-01-14T03:18:00.331703","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:43:52.308823Z","iopub.execute_input":"2024-10-14T17:43:52.309979Z","iopub.status.idle":"2024-10-14T17:43:52.317831Z","shell.execute_reply.started":"2024-10-14T17:43:52.309935Z","shell.execute_reply":"2024-10-14T17:43:52.317040Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“ | Dataset Path ","metadata":{"papermill":{"duration":0.010888,"end_time":"2024-01-14T03:18:00.372053","exception":false,"start_time":"2024-01-14T03:18:00.361165","status":"completed"},"tags":[]}},{"cell_type":"code","source":"BASE_PATH = \"/kaggle/input/hms-harmful-brain-activity-classification\"\n\nSPEC_DIR = \"/tmp/dataset/hms-hbac\"\nos.makedirs(SPEC_DIR+'/train_spectrograms', exist_ok=True)\nos.makedirs(SPEC_DIR+'/test_spectrograms', exist_ok=True)","metadata":{"papermill":{"duration":0.017704,"end_time":"2024-01-14T03:18:00.400852","exception":false,"start_time":"2024-01-14T03:18:00.383148","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:43:52.318885Z","iopub.execute_input":"2024-10-14T17:43:52.319167Z","iopub.status.idle":"2024-10-14T17:43:52.328254Z","shell.execute_reply.started":"2024-10-14T17:43:52.319137Z","shell.execute_reply":"2024-10-14T17:43:52.327380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“– | Meta Data ","metadata":{"papermill":{"duration":0.011434,"end_time":"2024-01-14T03:18:00.472401","exception":false,"start_time":"2024-01-14T03:18:00.460967","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Train + Valid\ndf = pd.read_csv(f'{BASE_PATH}/train.csv')\ndf['eeg_path'] = f'{BASE_PATH}/train_eegs/'+df['eeg_id'].astype(str)+'.parquet'\ndf['spec_path'] = f'{BASE_PATH}/train_spectrograms/'+df['spectrogram_id'].astype(str)+'.parquet'\ndf['spec2_path'] = f'{SPEC_DIR}/train_spectrograms/'+df['spectrogram_id'].astype(str)+'.npy'\ndf['class_name'] = df.expert_consensus.copy()\ndf['class_label'] = df.expert_consensus.map(CFG.name2label)\ndisplay(df.head(2))\n\n# Test\ntest_df = pd.read_csv(f'{BASE_PATH}/test.csv')\ntest_df['eeg_path'] = f'{BASE_PATH}/test_eegs/'+test_df['eeg_id'].astype(str)+'.parquet'\ntest_df['spec_path'] = f'{BASE_PATH}/test_spectrograms/'+test_df['spectrogram_id'].astype(str)+'.parquet'\ntest_df['spec2_path'] = f'{SPEC_DIR}/test_spectrograms/'+test_df['spectrogram_id'].astype(str)+'.npy'\ndisplay(test_df.head(2))","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:43:52.329366Z","iopub.execute_input":"2024-10-14T17:43:52.329778Z","iopub.status.idle":"2024-10-14T17:43:53.152334Z","shell.execute_reply.started":"2024-10-14T17:43:52.329738Z","shell.execute_reply":"2024-10-14T17:43:53.151414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Convert `.parquet` to `.npy`\n\nTo facilitate easier data loading, we will convert the EEG spectrograms from `parquet` to `npy` format. This process involves saving the spectrogram data, and since the content of the files remains the same, no significant changes are made. \n\n> It's worth noting that the `time` column is excluded, as it is not part of the spectrogram.","metadata":{}},{"cell_type":"code","source":"# Define a function to process a single eeg_id\ndef process_spec(spec_id, split=\"train\"):\n spec_path = f\"{BASE_PATH}/{split}_spectrograms/{spec_id}.parquet\"\n spec = pd.read_parquet(spec_path)\n spec = spec.fillna(0).values[:, 1:].T # fill NaN values with 0, transpose for (Time, Freq) -> (Freq, Time)\n spec = spec.astype(\"float32\")\n np.save(f\"{SPEC_DIR}/{split}_spectrograms/{spec_id}.npy\", spec)\n\n# Get unique spec_ids of train and valid data\nspec_ids = df[\"spectrogram_id\"].unique()\n\n# Parallelize the processing using joblib for training data\n_ = joblib.Parallel(n_jobs=-1, backend=\"loky\")(\n joblib.delayed(process_spec)(spec_id, \"train\")\n for spec_id in tqdm(spec_ids, total=len(spec_ids))\n)\n\n# Get unique spec_ids of test data\ntest_spec_ids = test_df[\"spectrogram_id\"].unique()\n\n# Parallelize the processing using joblib for test data\n_ = joblib.Parallel(n_jobs=-1, backend=\"loky\")(\n joblib.delayed(process_spec)(spec_id, \"test\")\n for spec_id in tqdm(test_spec_ids, total=len(test_spec_ids))\n)","metadata":{"papermill":{"duration":0.86264,"end_time":"2024-01-14T03:18:01.346487","exception":false,"start_time":"2024-01-14T03:18:00.483847","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:43:53.153460Z","iopub.execute_input":"2024-10-14T17:43:53.153811Z","iopub.status.idle":"2024-10-14T17:46:42.853162Z","shell.execute_reply.started":"2024-10-14T17:43:53.153778Z","shell.execute_reply":"2024-10-14T17:46:42.852255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿš | DataLoader\n\nThis DataLoader first reads `npy` spectrogram files and extracts labeled subsamples using specified `offset` values. Then, it converts the spectrogram data into `log spectrogram` and applies the popular signal augmentation `MixUp`.\n\n> Note that, we are converting the mono channel signal to a 3-channel signal for using \"ImageNet\" weights of pretrained model.","metadata":{"papermill":{"duration":0.011843,"end_time":"2024-01-14T03:18:01.457956","exception":false,"start_time":"2024-01-14T03:18:01.446113","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def build_augmenter(dim=CFG.image_size):\n augmenters = [\n keras_cv.layers.MixUp(alpha=2.0),\n keras_cv.layers.RandomCutout(height_factor=(1.0, 1.0),\n width_factor=(0.06, 0.1)), # freq-masking\n keras_cv.layers.RandomCutout(height_factor=(0.06, 0.1),\n width_factor=(1.0, 1.0)), # time-masking\n ]\n \n def augment(img, label):\n data = {\"images\":img, \"labels\":label}\n for augmenter in augmenters:\n if tf.random.uniform([]) < 0.5:\n data = augmenter(data, training=True)\n return data[\"images\"], data[\"labels\"]\n \n return augment\n\n\ndef build_decoder(with_labels=True, target_size=CFG.image_size, dtype=32):\n def decode_signal(path, offset=None):\n # Read .npy files and process the signal\n file_bytes = tf.io.read_file(path)\n sig = tf.io.decode_raw(file_bytes, tf.float32)\n sig = sig[1024//dtype:] # Remove header tag\n sig = tf.reshape(sig, [400, -1])\n \n # Extract labeled subsample from full spectrogram using \"offset\"\n if offset is not None: \n offset = offset // 2 # Only odd values are given\n sig = sig[:, offset:offset+300]\n \n # Pad spectrogram to ensure the same input shape of [400, 300]\n pad_size = tf.math.maximum(0, 300 - tf.shape(sig)[1])\n sig = tf.pad(sig, [[0, 0], [0, pad_size]])\n sig = tf.reshape(sig, [400, 300])\n \n # Log spectrogram \n sig = tf.clip_by_value(sig, tf.math.exp(-4.0), tf.math.exp(8.0)) # avoid 0 in log\n sig = tf.math.log(sig)\n \n # Normalize spectrogram\n sig -= tf.math.reduce_mean(sig)\n sig /= tf.math.reduce_std(sig) + 1e-6\n \n # Mono channel to 3 channels to use \"ImageNet\" weights\n sig = tf.tile(sig[..., None], [1, 1, 3])\n return sig\n \n def decode_label(label):\n label = tf.one_hot(label, CFG.num_classes)\n label = tf.cast(label, tf.float32)\n label = tf.reshape(label, [CFG.num_classes])\n return label\n \n def decode_with_labels(path, offset=None, label=None):\n sig = decode_signal(path, offset)\n label = decode_label(label)\n return (sig, label)\n \n return decode_with_labels if with_labels else decode_signal\n\n\ndef build_dataset(paths, offsets=None, labels=None, batch_size=32, cache=True,\n decode_fn=None, augment_fn=None,\n augment=False, repeat=True, shuffle=1024, \n cache_dir=\"\", drop_remainder=False):\n if cache_dir != \"\" and cache is True:\n os.makedirs(cache_dir, exist_ok=True)\n \n if decode_fn is None:\n decode_fn = build_decoder(labels is not None)\n \n if augment_fn is None:\n augment_fn = build_augmenter()\n \n AUTO = tf.data.experimental.AUTOTUNE\n slices = (paths, offsets) if labels is None else (paths, offsets, labels)\n \n ds = tf.data.Dataset.from_tensor_slices(slices)\n ds = ds.map(decode_fn, num_parallel_calls=AUTO)\n ds = ds.cache(cache_dir) if cache else ds\n ds = ds.repeat() if repeat else ds\n if shuffle: \n ds = ds.shuffle(shuffle, seed=CFG.seed)\n opt = tf.data.Options()\n opt.experimental_deterministic = False\n ds = ds.with_options(opt)\n ds = ds.batch(batch_size, drop_remainder=drop_remainder)\n ds = ds.map(augment_fn, num_parallel_calls=AUTO) if augment else ds\n ds = ds.prefetch(AUTO)\n return ds","metadata":{"papermill":{"duration":0.039133,"end_time":"2024-01-14T03:18:01.509017","exception":false,"start_time":"2024-01-14T03:18:01.469884","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:46:42.854649Z","iopub.execute_input":"2024-10-14T17:46:42.855009Z","iopub.status.idle":"2024-10-14T17:46:42.876834Z","shell.execute_reply.started":"2024-10-14T17:46:42.854974Z","shell.execute_reply":"2024-10-14T17:46:42.876013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ”ช | Data Split\n\nIn the following code snippet, the data is divided into `5` folds. Note that, the `groups` argument is used to prevent any overlap of patients between the training and validation sets, thus avoiding potential **data leakage** issues. Additionally, each split is stratified based on the `class_label`, ensuring a uniform distribution of class labels in each fold.","metadata":{"papermill":{"duration":0.012174,"end_time":"2024-01-14T03:18:01.538524","exception":false,"start_time":"2024-01-14T03:18:01.52635","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.model_selection import StratifiedGroupKFold\n\nsgkf = StratifiedGroupKFold(n_splits=5, shuffle=True, random_state=CFG.seed)\n\ndf[\"fold\"] = -1\ndf.reset_index(drop=True, inplace=True)\nfor fold, (train_idx, valid_idx) in enumerate(\n sgkf.split(df, y=df[\"class_label\"], groups=df[\"patient_id\"])\n):\n df.loc[valid_idx, \"fold\"] = fold\ndf.groupby([\"fold\", \"class_name\"])[[\"eeg_id\"]].count().T","metadata":{"papermill":{"duration":0.037496,"end_time":"2024-01-14T03:18:01.587924","exception":false,"start_time":"2024-01-14T03:18:01.550428","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:46:42.880999Z","iopub.execute_input":"2024-10-14T17:46:42.881292Z","iopub.status.idle":"2024-10-14T17:46:44.732082Z","shell.execute_reply.started":"2024-10-14T17:46:42.881260Z","shell.execute_reply":"2024-10-14T17:46:44.731118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Build Train & Valid Dataset\n\nOnly first sample for each `spectrogram_id` is used in order to keep the dataset size managable. Feel free to train on full data.","metadata":{"papermill":{"duration":0.011875,"end_time":"2024-01-14T03:18:01.611955","exception":false,"start_time":"2024-01-14T03:18:01.60008","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Sample from full data\nsample_df = df.groupby(\"spectrogram_id\").head(1).reset_index(drop=True)\ntrain_df = sample_df[sample_df.fold != CFG.fold]\nvalid_df = sample_df[sample_df.fold == CFG.fold]\nprint(f\"# Num Train: {len(train_df)} | Num Valid: {len(valid_df)}\")\n\n# Train\ntrain_paths = train_df.spec2_path.values\ntrain_offsets = train_df.spectrogram_label_offset_seconds.values.astype(int)\ntrain_labels = train_df.class_label.values\ntrain_ds = build_dataset(train_paths, train_offsets, train_labels, batch_size=CFG.batch_size,\n repeat=True, shuffle=True, augment=True, cache=True)\n\n# Valid\nvalid_paths = valid_df.spec2_path.values\nvalid_offsets = valid_df.spectrogram_label_offset_seconds.values.astype(int)\nvalid_labels = valid_df.class_label.values\nvalid_ds = build_dataset(valid_paths, valid_offsets, valid_labels, batch_size=CFG.batch_size,\n repeat=False, shuffle=False, augment=False, cache=True)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:46:44.733299Z","iopub.execute_input":"2024-10-14T17:46:44.733842Z","iopub.status.idle":"2024-10-14T17:46:47.894561Z","shell.execute_reply.started":"2024-10-14T17:46:44.733807Z","shell.execute_reply":"2024-10-14T17:46:47.893744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Dataset Check\n\nLet's visualize some samples from the dataset.","metadata":{}},{"cell_type":"code","source":"imgs, tars = next(iter(train_ds))\n\nnum_imgs = 8\nplt.figure(figsize=(4*4, num_imgs//4*5))\nfor i in range(num_imgs):\n plt.subplot(num_imgs//4, 4, i + 1)\n img = imgs[i].numpy()[...,0] # Adjust as per your image data format\n img -= img.min()\n img /= img.max() + 1e-4\n tar = CFG.label2name[np.argmax(tars[i].numpy())]\n plt.imshow(img)\n plt.title(f\"Target: {tar}\")\n plt.axis('off')\n \nplt.tight_layout()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-14T17:46:47.895740Z","iopub.execute_input":"2024-10-14T17:46:47.896057Z","iopub.status.idle":"2024-10-14T17:46:51.220094Z","shell.execute_reply.started":"2024-10-14T17:46:47.896023Z","shell.execute_reply":"2024-10-14T17:46:51.217369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pywt\nimport numpy as np\nimport matplotlib.pyplot as plt\n\ndef compute_wavelet_transform(signal, scales, waveletname='cmor'):\n \"\"\"\n Compute the Continuous Wavelet Transform (CWT) of the input signal.\n \n Args:\n signal (np.array): 1D EEG signal to be transformed.\n scales (np.array): Scales to be used in the CWT.\n waveletname (str): Name of the wavelet function to use (default is 'cmor').\n\n Returns:\n np.array: 2D scalogram (time-frequency representation of the signal).\n \"\"\"\n coeffs, _ = pywt.cwt(signal, scales, waveletname)\n return coeffs","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:46:51.221374Z","iopub.execute_input":"2024-10-14T17:46:51.221768Z","iopub.status.idle":"2024-10-14T17:46:51.722409Z","shell.execute_reply.started":"2024-10-14T17:46:51.221728Z","shell.execute_reply":"2024-10-14T17:46:51.721210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_wavelet(eeg_signal, scales, waveletname='cmor'):\n \"\"\"\n Process the EEG signal into a wavelet scalogram.\n \n Args:\n eeg_signal (np.array): 1D EEG signal.\n scales (np.array): Scales for the CWT.\n waveletname (str): Wavelet function name.\n \n Returns:\n np.array: Wavelet scalogram (converted to float32).\n \"\"\"\n scalogram = compute_wavelet_transform(eeg_signal, scales, waveletname)\n scalogram = np.abs(scalogram) # Use absolute value for power\n scalogram = scalogram.astype(\"float32\")\n return scalogram","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:46:51.726040Z","iopub.execute_input":"2024-10-14T17:46:51.726888Z","iopub.status.idle":"2024-10-14T17:46:51.732716Z","shell.execute_reply.started":"2024-10-14T17:46:51.726844Z","shell.execute_reply":"2024-10-14T17:46:51.731658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Update your data preprocessing pipeline to convert EEG signals into wavelet scalograms\ndef process_eeg_wavelet(eeg_path, split=\"train_eeg\", scales=None):\n eeg_signal = pd.read_parquet(eeg_path) # Load EEG signal\n eeg_signal = eeg_signal.values[:, 1] # Assuming the first column is time, second is signal\n \n # Compute the Wavelet Transform\n if scales is None:\n scales = np.arange(1, 128) # Define scales (you can tweak this based on your needs)\n \n wavelet_scalogram = process_wavelet(eeg_signal, scales, waveletname='cmor')\n \n # Save the wavelet scalogram as .npy\n np.save(f\"/kaggle/working/{os.path.basename(eeg_path).replace('.parquet', '.npy')}\", wavelet_scalogram)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:46:51.733984Z","iopub.execute_input":"2024-10-14T17:46:51.734309Z","iopub.status.idle":"2024-10-14T17:46:51.759315Z","shell.execute_reply.started":"2024-10-14T17:46:51.734272Z","shell.execute_reply":"2024-10-14T17:46:51.758426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"'''\n# Get unique spec_ids of train and valid data\nspec_ids = df[\"spectrogram_id\"].unique()\n'''\n\n# Apply this to all training and test data\n\neeg_paths = df[\"eeg_path\"].unique()\n_ = joblib.Parallel(n_jobs=-1, backend=\"loky\")(\n joblib.delayed(process_eeg_wavelet)(eeg_path, \"train_eeg\")\n #for spec_id in tqdm(spec_ids, total=len(spec_ids))\n for eeg_path in tqdm(eeg_paths, total=len(eeg_paths))\n)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:46:51.760377Z","iopub.execute_input":"2024-10-14T17:46:51.760719Z","iopub.status.idle":"2024-10-14T17:58:28.212256Z","shell.execute_reply.started":"2024-10-14T17:46:51.760662Z","shell.execute_reply":"2024-10-14T17:58:28.210297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"def decode_wavelet(path):\n # Read wavelet scalogram from .npy file\n sig = np.load(path.numpy()) # Load the numpy array\n sig = tf.convert_to_tensor(sig, dtype=tf.float32)\n \n # Log-transform and normalize\n sig = tf.clip_by_value(sig, 1e-6, tf.reduce_max(sig)) # Avoid log(0)\n sig = tf.math.log(sig)\n sig = (sig - tf.reduce_mean(sig)) / (tf.math.reduce_std(sig) + 1e-6)\n \n # Add a channel dimension and convert to 3 channels\n sig = tf.tile(sig[..., None], [1, 1, 3])\n return sig\n\ndef build_decoder_wavelet(with_labels=True, target_size=CFG.image_size):\n def decode_with_wavelet(path, label=None):\n sig = decode_wavelet(path)\n label = tf.one_hot(label, CFG.num_classes) if label is not None else None\n return sig, label if label is not None else sig\n \n return decode_with_wavelet if with_labels else decode_wavelet","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.213217Z","iopub.status.idle":"2024-10-14T17:58:28.213578Z","shell.execute_reply.started":"2024-10-14T17:58:28.213401Z","shell.execute_reply":"2024-10-14T17:58:28.213418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Apply wavelet transform on all EEG signals in parallel using joblib\ntrain_eeg_ids = os.listdir(EEG_TRAIN_PATH) # Assuming files are named by EEG ID\n_ = joblib.Parallel(n_jobs=-1, backend=\"loky\")(\n joblib.delayed(process_eeg_wavelet)(eeg_id.replace('.parquet', ''), \"train\")\n for eeg_id in tqdm(train_eeg_ids, total=len(train_eeg_ids))\n)\n\n# Repeat similar process for test data\ntest_eeg_ids = os.listdir(EEG_TEST_PATH)\n_ = joblib.Parallel(n_jobs=-1, backend=\"loky\")(\n joblib.delayed(process_eeg_wavelet)(eeg_id.replace('.parquet', ''), \"test\")\n for eeg_id in tqdm(test_eeg_ids, total=len(test_eeg_ids))\n)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.215471Z","iopub.status.idle":"2024-10-14T17:58:28.216358Z","shell.execute_reply.started":"2024-10-14T17:58:28.216095Z","shell.execute_reply":"2024-10-14T17:58:28.216122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow as tf\nimport numpy as np\nimport os\n\nclass DataGenerator(tf.keras.utils.Sequence):\n def __init__(self, ids, batch_size=32, dim=(128, 128), n_channels=1, n_classes=6, shuffle=True, data_dir=\"\"):\n self.ids = ids\n self.batch_size = batch_size\n self.dim = dim\n self.n_channels = n_channels\n self.n_classes = n_classes\n self.shuffle = shuffle\n self.data_dir = data_dir\n self.on_epoch_end()\n\n def __len__(self):\n # Number of batches per epoch\n return int(np.floor(len(self.ids) / self.batch_size))\n\n def __getitem__(self, index):\n # Generate indices of the batch\n batch_ids = self.ids[index * self.batch_size:(index + 1) * self.batch_size]\n\n # Generate data\n X, y = self.__data_generation(batch_ids)\n\n return X, y\n\n def on_epoch_end(self):\n # Shuffle the data after each epoch\n if self.shuffle:\n np.random.shuffle(self.ids)\n\n def __data_generation(self, batch_ids):\n # Initialize arrays for the batch\n X = np.empty((self.batch_size, *self.dim, self.n_channels))\n y = np.empty((self.batch_size), dtype=int)\n\n for i, eeg_id in enumerate(batch_ids):\n # Load wavelet scalogram from .npy file\n X[i,] = np.load(f\"{self.data_dir}/{eeg_id}.npy\")\n # Assign corresponding label (assuming label files exist)\n y[i] = get_label(eeg_id) # Implement this function to return the label for the EEG ID\n\n return X, tf.keras.utils.to_categorical(y, num_classes=self.n_classes)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.217542Z","iopub.status.idle":"2024-10-14T17:58:28.217926Z","shell.execute_reply.started":"2024-10-14T17:58:28.217747Z","shell.execute_reply":"2024-10-14T17:58:28.217767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"'''\nimport tensorflow as tf\nfrom tensorflow.keras import layers, models\n\n# Define the CNN model\ndef create_cnn_model(input_shape, num_classes):\n model = models.Sequential()\n \n # First Conv Block\n model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=input_shape))\n model.add(layers.MaxPooling2D((2, 2)))\n\n # Second Conv Block\n model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n model.add(layers.MaxPooling2D((2, 2)))\n\n # Fully Connected Layer\n model.add(layers.Flatten())\n model.add(layers.Dense(64, activation='relu'))\n model.add(layers.Dense(num_classes, activation='softmax'))\n\n return model\n\n# Define learning rate scheduler\nlr_schedule = tf.keras.optimizers.schedules.CosineDecay(\n initial_learning_rate=0.001, decay_steps=1000, alpha=0.0)\n\n# Compile the model\ninput_shape = (128, 128, 1) # Adjust based on wavelet transform dimensions\nnum_classes = 6 # Based on your dataset\nmodel = create_cnn_model(input_shape, num_classes)\n\nmodel.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=lr_schedule),\n loss='categorical_crossentropy',\n metrics=['accuracy'])\n\nmodel.summary()\n'''","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.219653Z","iopub.status.idle":"2024-10-14T17:58:28.220133Z","shell.execute_reply.started":"2024-10-14T17:58:28.219907Z","shell.execute_reply":"2024-10-14T17:58:28.219934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# Use the same `build_dataset` function as in your code but modify it to use the wavelet decoder\ntrain_ds = build_dataset(train_paths, offsets=train_offsets, labels=train_labels, \n decode_fn=build_decoder_wavelet(), \n batch_size=CFG.batch_size, augment=True, shuffle=True)\n\nvalid_ds = build_dataset(valid_paths, offsets=valid_offsets, labels=valid_labels, \n decode_fn=build_decoder_wavelet(), \n batch_size=CFG.batch_size, augment=False, shuffle=False)\n\n# Create data generators\ntrain_generator = DataGenerator(train_ids, batch_size=32, dim=(128, 128), n_channels=1, n_classes=6, data_dir=SAVE_DIR)\nvalid_generator = DataGenerator(valid_ids, batch_size=32, dim=(128, 128), n_channels=1, n_classes=6, data_dir=SAVE_DIR)\n\n","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.221568Z","iopub.status.idle":"2024-10-14T17:58:28.221934Z","shell.execute_reply.started":"2024-10-14T17:58:28.221759Z","shell.execute_reply":"2024-10-14T17:58:28.221778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\n# Model Training Code (keep the same as in your original script)\nmodel = keras.models.Sequential([\n keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(400, 300, 3)),\n keras.layers.MaxPooling2D((2, 2)),\n keras.layers.Conv2D(64, (3, 3), activation='relu'),\n keras.layers.MaxPooling2D((2, 2)),\n keras.layers.Conv2D(128, (3, 3), activation='relu'),\n keras.layers.Flatten(),\n keras.layers.Dense(128, activation='relu'),\n keras.layers.Dense(CFG.num_classes, activation='softmax')\n])\n\n# Compile the model\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n\n","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.223183Z","iopub.status.idle":"2024-10-14T17:58:28.223529Z","shell.execute_reply.started":"2024-10-14T17:58:28.223356Z","shell.execute_reply":"2024-10-14T17:58:28.223375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{"papermill":{"duration":0.016849,"end_time":"2024-01-14T03:18:38.613991","exception":false,"start_time":"2024-01-14T03:18:38.597142","status":"completed"},"tags":[]}},{"cell_type":"code","source":"history = model.fit(\n train_generator, \n epochs=CFG.epochs, \n steps_per_epoch=len(train_generator)//CFG.batch_size,\n validation_data= valid_generator, \n verbose=CFG.verbose\n)","metadata":{"papermill":{"duration":10.446166,"end_time":"2024-01-14T03:18:49.186176","exception":false,"start_time":"2024-01-14T03:18:38.74001","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:58:28.225008Z","iopub.status.idle":"2024-10-14T17:58:28.225366Z","shell.execute_reply.started":"2024-10-14T17:58:28.225193Z","shell.execute_reply":"2024-10-14T17:58:28.225211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{"papermill":{"duration":0.016209,"end_time":"2024-01-14T03:18:49.21924","exception":false,"start_time":"2024-01-14T03:18:49.203031","status":"completed"},"tags":[]}},{"cell_type":"code","source":"","metadata":{"papermill":{"duration":0.028945,"end_time":"2024-01-14T03:18:49.264535","exception":false,"start_time":"2024-01-14T03:18:49.23559","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"papermill":{"duration":0.297147,"end_time":"2024-01-14T03:18:49.578089","exception":false,"start_time":"2024-01-14T03:18:49.280942","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ’พ | Model Checkpointing","metadata":{"papermill":{"duration":0.017199,"end_time":"2024-01-14T03:18:49.613648","exception":false,"start_time":"2024-01-14T03:18:49.596449","status":"completed"},"tags":[]}},{"cell_type":"code","source":"","metadata":{"papermill":{"duration":0.024529,"end_time":"2024-01-14T03:18:49.655708","exception":false,"start_time":"2024-01-14T03:18:49.631179","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿš‚ | Training","metadata":{"papermill":{"duration":0.01671,"end_time":"2024-01-14T03:18:49.689354","exception":false,"start_time":"2024-01-14T03:18:49.672644","status":"completed"},"tags":[]}},{"cell_type":"code","source":"history = model.fit(\n train_ds, \n epochs=CFG.epochs,\n callbacks=[lr_cb, ckpt_cb], \n steps_per_epoch=len(train_df)//CFG.batch_size,\n validation_data=valid_ds, \n verbose=CFG.verbose\n)\n","metadata":{"papermill":{"duration":3374.692199,"end_time":"2024-01-14T04:15:04.398389","exception":false,"start_time":"2024-01-14T03:18:49.70619","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:58:28.227051Z","iopub.status.idle":"2024-10-14T17:58:28.227431Z","shell.execute_reply.started":"2024-10-14T17:58:28.227261Z","shell.execute_reply":"2024-10-14T17:58:28.227279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿงช | Prediction","metadata":{"papermill":{"duration":0.693309,"end_time":"2024-01-14T04:15:05.731839","exception":false,"start_time":"2024-01-14T04:15:05.03853","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Load Best Model","metadata":{"papermill":{"duration":0.632183,"end_time":"2024-01-14T04:15:06.991143","exception":false,"start_time":"2024-01-14T04:15:06.35896","status":"completed"},"tags":[]}},{"cell_type":"code","source":"model.load_weights(\"best_model.keras\")","metadata":{"papermill":{"duration":20.428261,"end_time":"2024-01-14T04:15:28.044401","exception":false,"start_time":"2024-01-14T04:15:07.61614","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-10-14T17:58:28.229145Z","iopub.status.idle":"2024-10-14T17:58:28.229525Z","shell.execute_reply.started":"2024-10-14T17:58:28.229333Z","shell.execute_reply":"2024-10-14T17:58:28.229358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Build Test Dataset","metadata":{"papermill":{"duration":0.703901,"end_time":"2024-01-14T04:20:09.745279","exception":false,"start_time":"2024-01-14T04:20:09.041378","status":"completed"},"tags":[]}},{"cell_type":"code","source":"test_paths = test_df.spec2_path.values\ntest_ds = build_dataset(test_paths, batch_size=min(CFG.batch_size, len(test_df)),\n repeat=False, shuffle=False, cache=False, augment=False)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.230961Z","iopub.status.idle":"2024-10-14T17:58:28.231284Z","shell.execute_reply.started":"2024-10-14T17:58:28.231121Z","shell.execute_reply":"2024-10-14T17:58:28.231138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Inference","metadata":{}},{"cell_type":"code","source":"preds = model.predict(test_ds)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.233337Z","iopub.status.idle":"2024-10-14T17:58:28.233727Z","shell.execute_reply.started":"2024-10-14T17:58:28.233522Z","shell.execute_reply":"2024-10-14T17:58:28.233541Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot the training and validation loss at each epoch\n\n\n#3loss = history.history['loss']\n#val_loss = history.history['val_loss']\n#epochs = range(1, len(loss) + 1)\n#plt.figure(figsize=(8, 6)) \n#plt.plot(epochs, loss, 'y', label='Training Loss')\n#plt.plot(epochs, val_loss, 'r', label='Validation Loss')\n#plt.title('Loss Curve (Training and validation loss)')\n#plt.xlabel('Epochs')\n#plt.ylabel('Loss')\n#plt.legend()\n#plt.show()","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.234856Z","iopub.status.idle":"2024-10-14T17:58:28.235180Z","shell.execute_reply.started":"2024-10-14T17:58:28.235017Z","shell.execute_reply":"2024-10-14T17:58:28.235034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss = history.history['loss']\nepochs = range(1, len(loss) + 1)","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.236205Z","iopub.status.idle":"2024-10-14T17:58:28.236531Z","shell.execute_reply.started":"2024-10-14T17:58:28.236359Z","shell.execute_reply":"2024-10-14T17:58:28.236376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#loss = history.history['loss']\n#epochs = range(1, len(loss) + 1)\ntr_acc = history.history['accuracy']\ntr_loss = history.history['loss']\ntr_per = history.history['precision']\ntr_recall = history.history['recall']\nval_acc = history.history['val_accuracy']\nval_loss = history.history['val_loss']\nval_per = history.history['val_precision']\nval_recall = history.history['val_recall']\n\nindex_loss = np.argmin(val_loss)\nval_lowest = val_loss[index_loss]\nindex_acc = np.argmax(val_acc)\nacc_highest = val_acc[index_acc]\nindex_precision = np.argmax(val_per)\nper_highest = val_per[index_precision]\nindex_recall = np.argmax(val_recall)\nrecall_highest = val_recall[index_recall]\n\n#Epochs = [i + 1 for i in range(len(tr_acc))]\n#loss_label = f'Best epoch = {str(index_loss + 1)}'\n#acc_label = f'Best epoch = {str(index_acc + 1)}'\n#per_label = f'Best epoch = {str(index_precision + 1)}'\n#recall_label = f'Best epoch = {str(index_recall + 1)}\n\nplt.figure(figsize=(20, 12))\n#plt.style.use('fivethirtyeight')\n\nplt.subplot(2, 2, 1)\nplt.plot(epochs, tr_loss, 'r', label='Training loss')\nplt.plot(epochs, val_loss, 'g', label='Validation loss')\n#plt.scatter(index_loss + 1, val_lowest, s=150, c='blue', label=loss_label)\nplt.title('Training and Validation Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\n\nplt.legend()\nplt.grid(True)\n\nplt.subplot(2, 2, 2)\nplt.plot(epochs, tr_acc, 'r', label='Training Accuracy')\nplt.plot(epochs, val_acc, 'g', label='Validation Accuracy')\n#plt.scatter(index_acc + 1, acc_highest, s=150, c='blue', label=acc_label)\nplt.title('Training and Validation Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.legend()\nplt.grid(True)\n\nplt.subplot(2, 2, 3)\nplt.plot(epochs, tr_per, 'r', label='Precision')\nplt.plot(epochs, val_per, 'g', label='Validation Precision')\n#plt.scatter(index_precision + 1, per_highest, s=150, c='blue', label=per_label)\nplt.title('Precision and Validation Precision')\nplt.xlabel('Epochs')\nplt.ylabel('Precision')\nplt.legend()\nplt.grid(True)\n\nplt.subplot(2, 2, 4)\nplt.plot(epochs, tr_recall, 'r', label='Recall')\nplt.plot(epochs, val_recall, 'g', label='Validation Recall')\n#plt.scatter(index_recall + 1, recall_highest, s=150, c='blue', label=recall_label)\nplt.title('Recall and Validation Recall')\nplt.xlabel('Epochs')\nplt.ylabel('Recall')\nplt.legend()\nplt.grid(True)\n\n#plt.suptitle('Model Training Metrics Over Epochs', fontsize=16)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.237798Z","iopub.status.idle":"2024-10-14T17:58:28.238122Z","shell.execute_reply.started":"2024-10-14T17:58:28.237956Z","shell.execute_reply":"2024-10-14T17:58:28.237973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“ฉ | Submission","metadata":{}},{"cell_type":"code","source":"#pred_df = test_df[[\"eeg_id\"]].copy()\n#target_cols = [x.lower()+'_vote' for x in CFG.class_names]\n#pred_df[target_cols] = preds.tolist()\n\n#sub_df = pd.read_csv(f'{BASE_PATH}/sample_submission.csv')\n#sub_df = sub_df[[\"eeg_id\"]].copy()\n#sub_df = sub_df.merge(pred_df, on=\"eeg_id\", how=\"left\")\n#sub_df.to_csv(\"submission.csv\", index=False)\n#sub_df.head()","metadata":{"execution":{"iopub.status.busy":"2024-10-14T17:58:28.239726Z","iopub.status.idle":"2024-10-14T17:58:28.240081Z","shell.execute_reply.started":"2024-10-14T17:58:28.239904Z","shell.execute_reply":"2024-10-14T17:58:28.239923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“Œ | Reference\n* [HMS-HBAC: ResNet34d Baseline [Training]](https://www.kaggle.com/code/ttahara/hms-hbac-resnet34d-baseline-training) \n* [EfficientNetB2 Starter - [LB 0.57]](https://www.kaggle.com/code/cdeotte/efficientnetb2-starter-lb-0-57)","metadata":{}}]}