{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":5819812,"sourceType":"datasetVersion","datasetId":3344234},{"sourceId":7781194,"sourceType":"datasetVersion","datasetId":4553461},{"sourceId":21279,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17615},{"sourceId":21282,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17618},{"sourceId":21287,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17623},{"sourceId":21497,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17803},{"sourceId":21507,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17811},{"sourceId":21511,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17815},{"sourceId":21520,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17824},{"sourceId":21525,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17829},{"sourceId":21670,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":17949}],"dockerImageVersionId":30665,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Setup","metadata":{}},{"cell_type":"code","source":"!pip uninstall timm -y\n!pip install /kaggle/input/timm-0613/timm-0.6.13-py3-none-any.whl -qq","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:19:32.993559Z","iopub.execute_input":"2024-03-30T16:19:32.994301Z","iopub.status.idle":"2024-03-30T16:20:09.077462Z","shell.execute_reply.started":"2024-03-30T16:19:32.994269Z","shell.execute_reply":"2024-03-30T16:20:09.076264Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nimport timm\nfrom fastai.vision.all import *\nfrom fastcore.parallel import *\n\npath = Path('/kaggle/input/hms-harmful-brain-activity-classification')\n\npath.ls()","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-03-30T16:21:17.404395Z","iopub.execute_input":"2024-03-30T16:21:17.404766Z","iopub.status.idle":"2024-03-30T16:21:27.795641Z","shell.execute_reply.started":"2024-03-30T16:21:17.404734Z","shell.execute_reply":"2024-03-30T16:21:27.794741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Background","metadata":{}},{"cell_type":"markdown","source":"In this notebook I submit predictions from all 3-model ensembles of the 9 large models I've trained (3 `convnext_large_in22k`, 3 `vit_large_patch16_224` and 3 `swinv2_base_window12_192_22k`).","metadata":{}},{"cell_type":"markdown","source":"## Submission Results","metadata":{}},{"cell_type":"markdown","source":"In an effort to save time (as I'm running out of days and submissions), I've only submitted 10 three-model combinations. Here are the results:\n\n|Ensemble|Public Score|\n|:-:|:-:|\n|2 x convnext 1|1.39|\n|2 x convnext 3|1.40|\n|2 x swinv2 J|1.40|\n|All 9 models|1.41|\n|2 x convnext 4|1.41|\n|2 x vit AW|1.41|\n|2 x swinv2 AB|1.41|\n|2 x vit BB|1.42|\n|2 x vit AU|1.43|\n|2 x swinv2 AU|1.44|\n\n\n\nSummarizing the five top models:\n\n|Model Name|item method|item img size|batch_tfms|Public Score|\n|:-:|:-:|:-:|:-:|:-:|\n|convnext 1|squish|(311, 400)|None|1.39|\n|convnext 3|crop|(400, 311)|None|1.40|\n|swinv2 J|crop|(400, 311)|aug_transforms(size=192, min_scale=0.75)|1.40|\n|convnext 4|squish|(400, 311)|None|1.41|\n|vit AW|crop|256 --> (400, 311) --> (320, 512)|RandomResizedCropGPU(size=224, min_scale=1.0)|1.41|\n\nMy efforts to train even larger models didn't pan out (see [this notebook](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-large-pt-2-train#Background)) and instead of letting 5 submissions go to waste, I submitted ensembles of these 5 models with each model weighted twice. Here are the Kaggle Public scores:\n\n|Model Name|Public Score|\n|:-:|:-:|\n|||\n|||\n|||\n|||\n|||","metadata":{}},{"cell_type":"markdown","source":"## Generate Test Data Images","metadata":{}},{"cell_type":"markdown","source":"My models are image classifiers trained on spectrogram images so I need to convert the test parquet data to images. I'm referencing the following notebooks:\n\n- [HMS - HBAC - Fastai Starter](https://www.kaggle.com/code/sonujha090/hms-hbac-fastai-starter)\n- [HMS-HBAC: KerasCV Starter Notebook](https://www.kaggle.com/code/awsaf49/hms-hbac-kerascv-starter-notebook)","metadata":{}},{"cell_type":"code","source":"# create temporary folders to hold spectrograms\nSPEC_DIR = \"/tmp/dataset/hms-hbac\"\nos.makedirs(SPEC_DIR+'/test_spectrograms', exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:04.357653Z","iopub.execute_input":"2024-03-30T16:22:04.358669Z","iopub.status.idle":"2024-03-30T16:22:04.363375Z","shell.execute_reply.started":"2024-03-30T16:22:04.358636Z","shell.execute_reply":"2024-03-30T16:22:04.362288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_spec(spec_id, split=\"train\"):\n    # read the data\n    data = pd.read_parquet(path/f'{split}_spectrograms'/f'{spec_id}.parquet')\n    \n    # replace NA with 0\n    data = data.fillna(0)\n    \n    # convert DataFrame to array\n    data = data.values[:, 1:]\n    \n    # transpose\n    data = data.T\n    data = data.astype(\"float32\")\n    \n    # convert array to PILImage\n    im = PILImage.create(Image.fromarray((data * 255).astype(np.uint8)))\n    im.save(f\"{SPEC_DIR}/{split}_spectrograms/{spec_id}.png\")","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:05.043028Z","iopub.execute_input":"2024-03-30T16:22:05.043847Z","iopub.status.idle":"2024-03-30T16:22:05.049786Z","shell.execute_reply.started":"2024-03-30T16:22:05.043812Z","shell.execute_reply":"2024-03-30T16:22:05.048841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv(path/'test.csv')\ntest_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:06.741208Z","iopub.execute_input":"2024-03-30T16:22:06.741583Z","iopub.status.idle":"2024-03-30T16:22:06.767203Z","shell.execute_reply.started":"2024-03-30T16:22:06.741555Z","shell.execute_reply":"2024-03-30T16:22:06.765982Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spec_ids = test_df['spectrogram_id'].unique()\nlen(spec_ids)","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:07.430196Z","iopub.execute_input":"2024-03-30T16:22:07.430991Z","iopub.status.idle":"2024-03-30T16:22:07.441441Z","shell.execute_reply.started":"2024-03-30T16:22:07.430960Z","shell.execute_reply":"2024-03-30T16:22:07.440462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"warnings.filterwarnings(\"ignore\")\nparallel(process_spec, spec_ids, split='test', n_workers=4)\nwarnings.filterwarnings(\"default\")","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:08.029715Z","iopub.execute_input":"2024-03-30T16:22:08.030349Z","iopub.status.idle":"2024-03-30T16:22:08.401239Z","shell.execute_reply.started":"2024-03-30T16:22:08.030322Z","shell.execute_reply":"2024-03-30T16:22:08.400190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PILImage.create(Path('/tmp/dataset/hms-hbac/test_spectrograms').ls()[0])","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:08.446354Z","iopub.execute_input":"2024-03-30T16:22:08.446671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Creating the `DataLoaders` Object","metadata":{}},{"cell_type":"markdown","source":"I already have created a dataset with training images, so I'll load that into my notebook and create a training path `trn_path` to use in my `DataLoaders`.","metadata":{}},{"cell_type":"code","source":"trn_path = Path('/kaggle/input/hms-hbac-training-spectrogram-images/train_spectrograms')","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:10.434027Z","iopub.execute_input":"2024-03-30T16:22:10.434389Z","iopub.status.idle":"2024-03-30T16:22:10.438638Z","shell.execute_reply.started":"2024-03-30T16:22:10.434359Z","shell.execute_reply":"2024-03-30T16:22:10.437615Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Each model was trained with different `item_tfms` and `batch_tfms` so I'll list them out here.","metadata":{}},{"cell_type":"code","source":"item1 = Resize((311, 400), method='squish') # convnext 1\nitem2 = Resize((400, 311), method='crop') # convnext 3\nitem3 = Resize((400, 311), method='squish') # convnext 4\nitem4 = Resize((320, 512), method='crop') # vit AW\nitem5 = Resize((320, 512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros) # vit BB \nitem6 = Resize((320, 512), method='squish') # vit AU\nitem7 = Resize((400, 311), method='crop') # swinv2 J\nitem8 = Resize((400, 311), method='crop') # swinv2 AB\nitem9 = Resize((320, 512), method='squish') # swinv2 AU","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:23.430036Z","iopub.execute_input":"2024-03-30T16:22:23.430400Z","iopub.status.idle":"2024-03-30T16:22:23.438137Z","shell.execute_reply.started":"2024-03-30T16:22:23.430372Z","shell.execute_reply":"2024-03-30T16:22:23.437242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"batch1 = None # convnext 1\nbatch2 = None # convnext 3\nbatch3 = None # convnext 4\nbatch4 = RandomResizedCropGPU(size=224, min_scale=1.0) # vit AW\nbatch5 = aug_transforms(size=224, min_scale=0.75) # vit BB\nbatch6 = aug_transforms(size=224, min_scale=0.75) # vit AU\nbatch7 = aug_transforms(size=192, min_scale=0.75) # swinv2 J\nbatch8 = RandomResizedCropGPU(size=192, min_scale=1.0) # swinv2 AB\nbatch9 = aug_transforms(size=192, min_scale=0.75) # swinv2 AU","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:32.318739Z","iopub.execute_input":"2024-03-30T16:22:32.319213Z","iopub.status.idle":"2024-03-30T16:22:32.336032Z","shell.execute_reply.started":"2024-03-30T16:22:32.319172Z","shell.execute_reply":"2024-03-30T16:22:32.335074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ensemble","metadata":{}},{"cell_type":"markdown","source":"I'll create a list of `DataLoaders`, one for each of the five models that I am using in this ensemble.","metadata":{}},{"cell_type":"code","source":"dls_list = []\nitems = [item1, item2, item3, item4, item5, item6, item7, item8, item9]\nbatches = [batch1, batch2, batch3, batch4, batch5, batch6, batch7, batch8, batch9]\n\nfor i in range(9):\n    dls = ImageDataLoaders.from_folder(\n        trn_path, \n        valid_pct=0.2, \n        item_tfms=items[i],\n        batch_tfms=batches[i],\n        bs=16)\n    \n    dls_list.append(dls)","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:22:35.187708Z","iopub.execute_input":"2024-03-30T16:22:35.188068Z","iopub.status.idle":"2024-03-30T16:23:13.128971Z","shell.execute_reply.started":"2024-03-30T16:22:35.188040Z","shell.execute_reply":"2024-03-30T16:23:13.127985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls_list","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:23:16.845495Z","iopub.execute_input":"2024-03-30T16:23:16.845868Z","iopub.status.idle":"2024-03-30T16:23:16.852506Z","shell.execute_reply.started":"2024-03-30T16:23:16.845841Z","shell.execute_reply":"2024-03-30T16:23:16.851482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next, I'll create a list of `Learner`s, one for each model.","metadata":{}},{"cell_type":"code","source":"model_paths = [\n    Path('/kaggle/input/hms-hbac-convnext_large/pytorch/1/1/hms_hbac_convnext_large_in22k_1'),\n    Path('/kaggle/input/hms-hbac-convnext_large/pytorch/3/1/hms_hbac_convnext_large_in22k_3'),\n    Path('/kaggle/input/hms-hbac-convnext_large/pytorch/4/1/hms_hbac_convnext_large_in22k_4'),\n    Path('/kaggle/input/hms-hbac-vit_large/pytorch/aw/1/hms_hbac_vit_large_patch16_224_AW'),\n    Path('/kaggle/input/hms-hbac-vit_large/pytorch/bb/1/hms_hbac_vit_large_patch16_224_BB'),\n    Path('/kaggle/input/hms-hbac-vit_large/pytorch/au/1/hms_hbac_vit_large_patch16_224_AU'),\n    Path('/kaggle/input/hms-hbac-swinv2_large/pytorch/j/1/hms_hbac_swinv2_large_window12_192_22k_J'),\n    Path('/kaggle/input/hms-hbac-swinv2_large/pytorch/ab/1/hms_hbac_swinv2_large_window12_192_22k_AB'),\n    Path('/kaggle/input/hms-hbac-swinv2_large/pytorch/au/1/hms_hbac_swinv2_large_window12_192_22k_AU'),\n]","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:23:25.026541Z","iopub.execute_input":"2024-03-30T16:23:25.027281Z","iopub.status.idle":"2024-03-30T16:23:25.033872Z","shell.execute_reply.started":"2024-03-30T16:23:25.027247Z","shell.execute_reply":"2024-03-30T16:23:25.032827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"archs = ['convnext_large_in22k', 'convnext_large_in22k', 'convnext_large_in22k', \n         'vit_large_patch16_224', 'vit_large_patch16_224', 'vit_large_patch16_224',\n        'swinv2_large_window12_192_22k', 'swinv2_large_window12_192_22k', 'swinv2_large_window12_192_22k']","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:23:28.678141Z","iopub.execute_input":"2024-03-30T16:23:28.678857Z","iopub.status.idle":"2024-03-30T16:23:28.683664Z","shell.execute_reply.started":"2024-03-30T16:23:28.678823Z","shell.execute_reply":"2024-03-30T16:23:28.682424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learners = []\n\nfor idx, model_path in enumerate(model_paths):\n    learn = vision_learner(dls_list[idx], archs[idx], pretrained=False)\n    learn.model_dir = '/kaggle/working/'\n    learn.load(model_path)\n    learners.append(learn)","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:23:32.777272Z","iopub.execute_input":"2024-03-30T16:23:32.778242Z","iopub.status.idle":"2024-03-30T16:25:47.945007Z","shell.execute_reply.started":"2024-03-30T16:23:32.778187Z","shell.execute_reply":"2024-03-30T16:25:47.943973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learners","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:25:50.212811Z","iopub.execute_input":"2024-03-30T16:25:50.213551Z","iopub.status.idle":"2024-03-30T16:25:50.220352Z","shell.execute_reply.started":"2024-03-30T16:25:50.213517Z","shell.execute_reply":"2024-03-30T16:25:50.219427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next, I'll get a `DataFrame` of probabilities for each test image from each `Learner` in my 3-model ensemble.","metadata":{}},{"cell_type":"code","source":"probs_df_list = []\n\ntst_files = get_image_files(SPEC_DIR+'/test_spectrograms')\n\nfor idx in range(9):\n    # create test DataLoader\n    tst_dl = dls_list[idx].test_dl(tst_files)\n    \n    # get TTA predictions\n    probs,_= learners[idx].tta(dl=tst_dl)\n    \n    # formatting\n    probs_df = pd.DataFrame(probs, columns=dls_list[idx].vocab)\n    probs_df['eeg_id'] = test_df['eeg_id']\n    probs_df = probs_df[['eeg_id', 'seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']]\n    \n    probs_df_list.append(probs_df)","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:26:06.540659Z","iopub.execute_input":"2024-03-30T16:26:06.541560Z","iopub.status.idle":"2024-03-30T16:26:38.672249Z","shell.execute_reply.started":"2024-03-30T16:26:06.541526Z","shell.execute_reply":"2024-03-30T16:26:38.670856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Concatenate all `DataFrames`:","metadata":{}},{"cell_type":"code","source":"all_probs_df = pd.concat([probs_df_list[i] for i in [0, 6]])\nall_probs_df.shape","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:26:50.342394Z","iopub.execute_input":"2024-03-30T16:26:50.342787Z","iopub.status.idle":"2024-03-30T16:26:50.352638Z","shell.execute_reply.started":"2024-03-30T16:26:50.342750Z","shell.execute_reply":"2024-03-30T16:26:50.351738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Take the mean value of each vote by `eeg_id`","metadata":{}},{"cell_type":"code","source":"final_probs = all_probs_df.groupby('eeg_id').mean().reset_index()","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:26:53.668666Z","iopub.execute_input":"2024-03-30T16:26:53.669067Z","iopub.status.idle":"2024-03-30T16:26:53.680653Z","shell.execute_reply.started":"2024-03-30T16:26:53.669034Z","shell.execute_reply":"2024-03-30T16:26:53.679471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_probs.head()","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:26:55.213257Z","iopub.execute_input":"2024-03-30T16:26:55.214175Z","iopub.status.idle":"2024-03-30T16:26:55.230239Z","shell.execute_reply.started":"2024-03-30T16:26:55.214135Z","shell.execute_reply":"2024-03-30T16:26:55.229091Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And export it:","metadata":{}},{"cell_type":"code","source":"final_probs.to_csv('submission.csv', index=False)\n!head submission.csv","metadata":{"execution":{"iopub.status.busy":"2024-03-30T16:26:58.317255Z","iopub.execute_input":"2024-03-30T16:26:58.317647Z","iopub.status.idle":"2024-03-30T16:26:59.455425Z","shell.execute_reply.started":"2024-03-30T16:26:58.317619Z","shell.execute_reply":"2024-03-30T16:26:59.453912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}