{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7550085,"sourceType":"datasetVersion","datasetId":4397253},{"sourceId":162349255,"sourceType":"kernelVersion"}],"dockerImageVersionId":30648,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# HMS - PyTorch Baseline Inference\n\n**Comments welcome!**\n\nOne of my goals in this competition is to learn more PyTorch.\n\nThis is an **inference** notebook; the respetive training notebook is [HMS - PyTorch Baseline Training](https://www.kaggle.com/code/morodertobias/hms-pytorch-baseline-training/notebook), and its trained models have been registered as a versioned dataset [HMS - PyTorch Baseline Training Dataset](https://www.kaggle.com/datasets/morodertobias/hms-pytorch-baseline-training-dataset).\n\nThe model uses squashed spectrograms, as done in the reference notebooks. I try to use my way of coding, but naturally it is similar. \n\nThis version uses the current version of the notebook, version 1 of dataset, and the last successful notebook run, version 8, hence 10 models in total. Each one is an EfficientNetB0 which have been fined-tuned from noisy student weights.","metadata":{}},{"cell_type":"markdown","source":"## Core References\n- [HMS: Inference (LB: 0.42)](https://www.kaggle.com/code/andreasbis/hms-inference-lb-0-42)\n- [HMS-HBAC: ResNet34d Baseline [Inference]](https://www.kaggle.com/code/ttahara/hms-hbac-resnet34d-baseline-inference)\n- [HMS: Train EfficientNetB0](https://www.kaggle.com/code/andreasbis/hms-train-efficientnetb0)\n- [HMS baseline_resnet34d(512*512 Training 5 folds)](https://www.kaggle.com/code/yunsuxiaozi/hms-baseline-resnet34d-512-512-training-5-folds)\n- [https://www.kaggle.com/code/ttahara/hms-hbac-resnet34d-baseline-training/](https://www.kaggle.com/code/ttahara/hms-hbac-resnet34d-baseline-training/)","metadata":{}},{"cell_type":"markdown","source":"## Table of Contents\n- [Imports](#Imports)\n- [Config](#Config)\n- [Prepare data](#Prepare-data)\n- [Prepare model](#Prepare-model)\n- [Predict](#Predict)\n- [Finalize submission](#Finalize-submission)","metadata":{}},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"import os\nimport pathlib\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom tqdm.notebook import tqdm\nimport torch\nimport torch.nn.functional as F\nimport torchvision.transforms as transforms\nimport timm\nfrom torch.utils.data import Dataset, DataLoader","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-02-10T20:22:32.451489Z","iopub.execute_input":"2024-02-10T20:22:32.451851Z","iopub.status.idle":"2024-02-10T20:22:41.417267Z","shell.execute_reply.started":"2024-02-10T20:22:32.451819Z","shell.execute_reply":"2024-02-10T20:22:41.416470Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Config","metadata":{}},{"cell_type":"code","source":"class CFG:\n    base_dir = pathlib.Path(\"/kaggle/input/hms-harmful-brain-activity-classification\")\n    path_test = base_dir / \"test.csv\"\n    path_submission = base_dir / \"sample_submission.csv\"\n    spec_dir = base_dir / \"test_spectrograms\"\n    model_name = \"tf_efficientnet_b0_ns\"\n    model_weights = sorted(\n        list(pathlib.Path(\"/kaggle/input/hms-pytorch-baseline-training-dataset\").glob(\"*.pt\"))\n        + list(pathlib.Path(\"/kaggle/input/hms-pytorch-baseline-training\").glob(\"*.pt\"))\n    )\n    transform = transforms.Resize((512, 512), antialias=False)\n    batch_size = 16\n    label_columns = [\n        \"seizure_vote\",\n        \"lpd_vote\",\n        \"gpd_vote\",\n        \"lrda_vote\",\n        \"grda_vote\",\n        \"other_vote\",\n    ]\n\n\nCFG.model_weights","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:22:41.419041Z","iopub.execute_input":"2024-02-10T20:22:41.419691Z","iopub.status.idle":"2024-02-10T20:22:41.437072Z","shell.execute_reply.started":"2024-02-10T20:22:41.419655Z","shell.execute_reply":"2024-02-10T20:22:41.436196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prepare data\n- Load test dataframe.\n- Prepare Dataset and DataLoader.\n- Check one example to see that everything is correct.","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv(CFG.path_test)\nsubmission = pd.read_csv(CFG.path_submission)\nsubmission = pd.merge(submission, test, how=\"inner\", on=\"eeg_id\")\nsubmission[\"path\"] = submission[\"spectrogram_id\"].map(lambda x: CFG.spec_dir / f\"{x}.parquet\")\nsubmission","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:24:54.901302Z","iopub.execute_input":"2024-02-10T20:24:54.901992Z","iopub.status.idle":"2024-02-10T20:24:54.941640Z","shell.execute_reply.started":"2024-02-10T20:24:54.901959Z","shell.execute_reply":"2024-02-10T20:24:54.940661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def preprocess(x):\n    x = np.clip(x, np.exp(-6), np.exp(10))\n    x = np.log(x)\n    m, s = x.mean(), x.std()\n    x = (x - m) / (s + 1e-6)\n    return x\n\n\nclass SpecDataset(Dataset):\n    \n    def __init__(self, df, transform=CFG.transform):\n        self.df = df\n        self.transform = transform\n    \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, index):\n        row = self.df.iloc[index]\n        # input\n        x = pd.read_parquet(row.path)\n        x = x.fillna(-1).values[:, 1:].T\n        x = preprocess(x)\n        x = torch.Tensor(x[None, :])\n        if self.transform:\n            x = self.transform(x)\n        # output\n        y = np.array(row.loc[CFG.label_columns].values, 'float32')\n        y = torch.Tensor(y)\n        return x, y","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:24:55.230226Z","iopub.execute_input":"2024-02-10T20:24:55.230976Z","iopub.status.idle":"2024-02-10T20:24:55.240453Z","shell.execute_reply.started":"2024-02-10T20:24:55.230947Z","shell.execute_reply":"2024-02-10T20:24:55.239340Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_ds = SpecDataset(df=submission)\ndata_loader = DataLoader(dataset=data_ds, num_workers=os.cpu_count())\ndata_loader","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:24:55.869939Z","iopub.execute_input":"2024-02-10T20:24:55.870817Z","iopub.status.idle":"2024-02-10T20:24:55.878501Z","shell.execute_reply.started":"2024-02-10T20:24:55.870783Z","shell.execute_reply":"2024-02-10T20:24:55.877461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x, y = next(iter(data_loader))\nx.shape, x","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:24:56.277644Z","iopub.execute_input":"2024-02-10T20:24:56.278574Z","iopub.status.idle":"2024-02-10T20:24:56.720950Z","shell.execute_reply.started":"2024-02-10T20:24:56.278539Z","shell.execute_reply":"2024-02-10T20:24:56.719833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(x[0, 0])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:24:56.998435Z","iopub.execute_input":"2024-02-10T20:24:56.999172Z","iopub.status.idle":"2024-02-10T20:24:57.289134Z","shell.execute_reply.started":"2024-02-10T20:24:56.999135Z","shell.execute_reply":"2024-02-10T20:24:57.288230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prepare model","metadata":{}},{"cell_type":"code","source":"DEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nprint(f\"DEVICE: {DEVICE}\")","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:24:58.269853Z","iopub.execute_input":"2024-02-10T20:24:58.270259Z","iopub.status.idle":"2024-02-10T20:24:58.297887Z","shell.execute_reply.started":"2024-02-10T20:24:58.270217Z","shell.execute_reply":"2024-02-10T20:24:58.296833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = timm.create_model(model_name=CFG.model_name, pretrained=False, num_classes=6, in_chans=1)\nmodel.to(DEVICE)\nnum_parameter = sum(x.numel() for x in model.parameters())\nprint(f\"Model has {num_parameter} parameters.\")","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:24:58.829588Z","iopub.execute_input":"2024-02-10T20:24:58.829942Z","iopub.status.idle":"2024-02-10T20:24:59.108922Z","shell.execute_reply.started":"2024-02-10T20:24:58.829915Z","shell.execute_reply":"2024-02-10T20:24:59.108020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict\n- Load weights and compute individual predictions.\n- Note, the output of the model are logits.\n- Final predicition is the ensemble of all invidiual predictions.","metadata":{}},{"cell_type":"code","source":"prediction = pd.DataFrame(0.0, columns=CFG.label_columns, index=submission.index)\nfor i, path_weight in enumerate(CFG.model_weights):\n    print(f\"Model {i}: {path_weight}\")\n    model.load_state_dict(torch.load(path_weight))\n    model.eval()\n    with torch.no_grad():\n        res = []\n        for x, y in data_loader:\n            x = x.to(DEVICE)\n            pred = model(x)\n            pred = F.softmax(pred, dim=1)\n            pred = pred.detach().cpu().numpy()\n            res.append(pred)\n        res = np.concatenate(res)\n        res = pd.DataFrame(res, columns=CFG.label_columns, index=submission.index)\n        display(res)\n        prediction = prediction + res\n        print(\"\\n\")\nprediction = prediction / len(CFG.model_weights)","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:25:00.246903Z","iopub.execute_input":"2024-02-10T20:25:00.247402Z","iopub.status.idle":"2024-02-10T20:25:06.101856Z","shell.execute_reply.started":"2024-02-10T20:25:00.247374Z","shell.execute_reply":"2024-02-10T20:25:06.100741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prediction","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:25:06.103717Z","iopub.execute_input":"2024-02-10T20:25:06.104047Z","iopub.status.idle":"2024-02-10T20:25:06.117070Z","shell.execute_reply.started":"2024-02-10T20:25:06.104017Z","shell.execute_reply":"2024-02-10T20:25:06.116074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Finalize submission","metadata":{"execution":{"iopub.status.busy":"2024-02-04T07:41:51.765545Z","iopub.execute_input":"2024-02-04T07:41:51.765946Z","iopub.status.idle":"2024-02-04T07:41:51.774172Z","shell.execute_reply.started":"2024-02-04T07:41:51.765901Z","shell.execute_reply":"2024-02-04T07:41:51.773122Z"}}},{"cell_type":"code","source":"submission[CFG.label_columns] = prediction\nsubmission = submission[[\"eeg_id\"] + CFG.label_columns]\nsubmission","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:25:14.990684Z","iopub.execute_input":"2024-02-10T20:25:14.991066Z","iopub.status.idle":"2024-02-10T20:25:15.007577Z","shell.execute_reply.started":"2024-02-10T20:25:14.991034Z","shell.execute_reply":"2024-02-10T20:25:15.006576Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv(\"submission.csv\", index=None)","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:25:15.837623Z","iopub.execute_input":"2024-02-10T20:25:15.837978Z","iopub.status.idle":"2024-02-10T20:25:15.846953Z","shell.execute_reply.started":"2024-02-10T20:25:15.837951Z","shell.execute_reply":"2024-02-10T20:25:15.846100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!head submission.csv","metadata":{"execution":{"iopub.status.busy":"2024-02-10T20:25:16.797988Z","iopub.execute_input":"2024-02-10T20:25:16.798362Z","iopub.status.idle":"2024-02-10T20:25:17.757063Z","shell.execute_reply.started":"2024-02-10T20:25:16.798335Z","shell.execute_reply":"2024-02-10T20:25:17.755819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}