{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Making FoG predictions through segmentation and ensemble of 1D-CNN Models","metadata":{"execution":{"iopub.status.busy":"2023-06-09T19:42:11.787285Z","iopub.execute_input":"2023-06-09T19:42:11.787731Z","iopub.status.idle":"2023-06-09T19:42:11.793559Z","shell.execute_reply.started":"2023-06-09T19:42:11.787697Z","shell.execute_reply":"2023-06-09T19:42:11.792322Z"}}},{"cell_type":"markdown","source":"### Intro\nThis notebook is heavily based on the following two notebooks:\n* [[Practicum] - CNN skeleton](https://www.kaggle.com/code/ernestglukhov/practicum-cnn-skeleton)\n* [Locating steps. Wavelets -> step rate feature](https://www.kaggle.com/code/vrbaryshev/locating-steps-wavelets-step-rate-feature)\n\nIn this Kaggle notebook, we present a comprehensive approach aimed at predicting Freezing of Gait (FoG) using an ensemble of 1D-CNN models and a sophisticated segmentation process. The segmentation process helps capture distinct patterns in the acceleration data, while the ensemble strategy involves training multiple 1D-CNN models using K-fold cross-validation. Our model architecture consists of multiple Convolutional Blocks, each containing convolutional layers with different kernel sizes and dilations. The models in the ensemble are trained separately on the \"tdcsfog\" and \"defog\" datasets. By treating the `tdcsfog` and `defog` datasets separately, we aimed to optimize the models' performance for each specific type of data.\n\nAs a result, our best model enabled us to achieve a score of 0.404 on the public leaderboard, with a private leaderboard score of 0.246. \\\nWe were disappointed by the notable divergence in scores between the public and private datasets, which we believe is likely due to differences in their composition. Despite this, we must emphasize the educational value of the project, which offered a rich, hands-on learning experience. As a competition, however, it did leave a taste of disappointment due to the unexpected discrepancy in results across different data sets.","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.status.busy":"2023-06-10T10:37:09.868218Z","iopub.execute_input":"2023-06-10T10:37:09.868567Z","iopub.status.idle":"2023-06-10T10:37:09.879828Z","shell.execute_reply.started":"2023-06-10T10:37:09.868537Z","shell.execute_reply":"2023-06-10T10:37:09.878879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import gc\nimport os\nimport numpy as np\nimport pandas as pd\nimport pathlib\n\nimport pywt\n\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import average_precision_score\nfrom sklearn.preprocessing import MinMaxScaler\n\nimport pytorch_lightning as pl\nfrom pytorch_lightning.callbacks import ModelCheckpoint\n\nimport torch\nfrom torch import nn\nfrom torch.nn import functional as F\nfrom torch.utils.data import Dataset, DataLoader","metadata":{"papermill":{"duration":15.148726,"end_time":"2023-06-06T21:36:56.776822","exception":false,"start_time":"2023-06-06T21:36:41.628096","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-06-10T10:37:09.882011Z","iopub.execute_input":"2023-06-10T10:37:09.882410Z","iopub.status.idle":"2023-06-10T10:37:22.995052Z","shell.execute_reply.started":"2023-06-10T10:37:09.882379Z","shell.execute_reply":"2023-06-10T10:37:22.994178Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Segmentation process and feature engineering\n\n**Segmentation process**:\n\nWe have devised a segmentation process on raw acceleration data using a sliding window approach. This approach allows us to capture temporal patterns and variations in the acceleration signals.\nTo accomplish this, we implemented a custom function that divides the continuous time series data into smaller, fixed-length segments. Each segment represents a specific time interval, capturing a snapshot of the acceleration data within that interval. Furthermore, we have incorporated padding techniques within our segmentation function to ensure consistent segment lengths. The padding operation fills incomplete segments caused by irregular data lengths, enabling uniform processing and compatibility with downstream tasks.\nWe used the following parameters for the segmentation process:\n\n    Window size: 72 seconds (9216 data points) for events from `tdscfog` set\n    Window size: 10.24 seconds (1024 data points) for events from `defog` set\n    \n**Feature engineering**:\n\nTo capture different aspects of the data, we employed various techniques, including rolling window operations and exponential weighted moving averages (EWMA). The computation of these features is performed separately for each acceleration channel, namely AccV, AccML, and AccAP. These techniques allow us to derive meaningful insights about the temporal characteristics of the acceleration data. In addtition to temporal data analysis, we have also included a spectral based image of data. Technically, all sprectal based features are convolutions, so a sufficiently large and trained convolutional neural net should be able to recognize them without explicit help from feature engineering. However, including some spectral features equals to pretraining the model for easier recognizing repetitive patterns in particual frequency bands, such as steps and step-related mechanical processes. In this work we have included an eight pixed wide wavelet image covering frequencies from 0.4Hz to 3Hz as premade features.","metadata":{}},{"cell_type":"code","source":"class SegmentedDataset(Dataset):\n    def __init__(self, folder_path, files, is_train=True, is_defog=False, segment_length=9216):\n        self.folder_path = folder_path\n        self.is_train = is_train\n        self.is_defog = is_defog\n        self.segment_length = segment_length\n        self.data = []\n        self.time = []\n        self.ids = []\n        self.targets = []\n        \n        if self.is_defog:\n            self.segment_length = 1024\n        \n        # Read and segment all data at initialization\n        for file_name in files:\n            file_path = os.path.join(folder_path, file_name)\n            data = pd.read_csv(file_path)\n            \n            acc_cols = ['AccV', 'AccML', 'AccAP']\n            window_size = self.segment_length\n\n            \n            # Create new features based on accelerometer data columns\n            wavelet_scales = np.exp(np.arange(2, 6, 0.5))\n            wavelet_list = [ f'wave_{scale}' for scale in wavelet_scales ] \n            wavelet='morl'\n            \n            coeff, freq = pywt.cwt(data['AccML'], wavelet_scales, wavelet)\n            for i, scale in enumerate(wavelet_scales):\n                data[f'wave_{scale}'] = coeff[i] \n            \n            for acc in acc_cols:\n                data[f'{acc}_cumsum'] = data[acc].cumsum()\n                data[f'{acc}_running_sum'] = data[acc].rolling(window=window_size, min_periods=8).sum()\n                data[f'{acc}_rolling_mean'] = data[acc].rolling(window=window_size, min_periods=8).mean()\n                data[f'{acc}_rolling_std'] = data[acc].rolling(window=window_size, min_periods=8).std()\n                data[f'{acc}_rolling_min'] = data[acc].rolling(window=window_size, min_periods=8).min()\n                data[f'{acc}_rolling_max'] = data[acc].rolling(window=window_size, min_periods=8).max()\n                data[f'{acc}_rolling_delta'] = data[f'{acc}_rolling_max'] - data[f'{acc}_rolling_min']        \n                data[f'{acc}_EWMA_02'] = data[acc].ewm(alpha=0.2).mean()\n\n                \n            data.fillna(method='backfill', inplace=True)\n\n            file_id = file_name.replace('.csv', '')\n\n            time = data['Time'].values\n            features_cols = [col for col in data.columns if col not in ['Time', 'StartHesitation', 'Turn', 'Walking', 'Valid', 'Task']]\n            features_cols_nowave = [col for col in data.columns if col not in ['Time', 'StartHesitation', 'Turn', 'Walking', 'Valid', 'Task', 'wave']]\n            \n            scaler = MinMaxScaler()\n            scaled_features = scaler.fit_transform(data[features_cols_nowave])\n            data[features_cols_nowave] = scaler.fit_transform(data[features_cols_nowave])\n            \n            features = data[features_cols].values.astype(np.float32)\n            \n            if self.is_train:\n                targets = data[['StartHesitation', 'Turn', 'Walking']].values\n                # add extra target column for \"Normal\" case\n                targets = np.concatenate([targets, (1 - targets.sum(axis=1)).reshape(-1,1)], axis=1)\n                self.segment_and_pad_data(file_id, features, time, targets)\n            else:\n                self.segment_and_pad_data(file_id, features, time)\n                \n                \n    # Segments the data into fixed-length segments and handles padding for the last segment if it's shorter than the desired length\n    def segment_and_pad_data(self, file_id, features, time, targets=None):\n        num_segments = len(features) // self.segment_length\n        remainder = len(features) % self.segment_length\n        \n            \n        # First, handle the full segments\n        for i in range(num_segments):\n            feature_segment = features[i*self.segment_length:(i+1)*self.segment_length]\n            \n            self.data.append(feature_segment)\n            \n            time_segment = time[i*self.segment_length:(i+1)*self.segment_length]\n            self.time.append(time_segment)\n            \n            \n            if self.is_train:\n                target_segment = targets[i*self.segment_length:(i+1)*self.segment_length]\n                self.targets.append(target_segment)\n                \n            self.ids.append(file_id)\n\n        # Now handle the last segment if it's shorter than segment_length\n        if remainder > 0:\n            padding_length = self.segment_length - remainder\n\n            feature_segment = np.pad(features[-remainder:], ((0, padding_length), (0, 0)), mode='constant')\n            self.data.append(feature_segment)\n            \n            time_segment = np.pad(time[-remainder:], (0, padding_length), mode='constant', constant_values=-1)\n\n            self.time.append(time_segment)\n            \n            if self.is_train:\n                target_segment = np.pad(targets[-remainder:], ((0, padding_length), (0, 0)), mode='constant')\n                self.targets.append(target_segment)\n                \n            self.ids.append(file_id)\n            \n\n\n    def get_segment_indices(self):\n        return self.segment_indices\n\n    def get_segment_padding(self):\n        return self.segment_padding\n\n    def __len__(self):\n        return len(self.data)\n\n    def __getitem__(self, idx):\n        if self.is_train:\n            return torch.Tensor(self.data[idx]), torch.Tensor(np.argmax(self.targets[idx],axis=1)).to(torch.int64)\n        else:\n            return torch.Tensor(self.data[idx]), self.ids[idx], self.time[idx]","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":0.035485,"end_time":"2023-06-06T21:36:56.816901","exception":false,"start_time":"2023-06-06T21:36:56.781416","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-06-10T10:37:22.996465Z","iopub.execute_input":"2023-06-10T10:37:22.996811Z","iopub.status.idle":"2023-06-10T10:37:23.035707Z","shell.execute_reply.started":"2023-06-10T10:37:22.996778Z","shell.execute_reply":"2023-06-10T10:37:23.033309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1D-Convolutional Neural Network (1D-Conv NN)\n\nTo make predictions for the classification task on each segment, we decided to choose a 1D-Convolutional Neural Network with flexible hyperparameter tuning.\n\nOur model consists of multiple Convolutional Blocks (the optimal number of ConvBlocks based on the training speed and validation score turned out to be 3), each containing convolutional layers with different kernel sizes and dilations. These blocks allow the network to capture various temporal patterns and features present in the acceleration data. We apply batch normalization and dropout regularization within each block to enhance the model's generalization capability.\nThe output of the convolutional blocks is fed into a linear layer, which maps the learned features to the corresponding number of classes. To handle class imbalance, we use a weighted negative log-likelihood loss function. The model's predictions are obtained by applying a softmax activation function to the output logits.\n\nDuring training, we optimize the model parameters using the Adam optimizer with a learning rate of 0.001. We monitor the training and validation losses throughout the epochs to assess the model's performance. Additionally, we evaluate the mean average precision score as a performance metric for the validation set.","metadata":{}},{"cell_type":"markdown","source":"In the output of the model we have LogSoftmax and 4 classes - `StartHesitation`, `Turn`, `Walking` and `Normal` with weights: 1.0, 1.0, 1.0, 0.1.","metadata":{}},{"cell_type":"code","source":"import itertools\n\nclass ConvBlock(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_sizes, dropout_rate, dilations):\n        super(ConvBlock, self).__init__()\n\n        self.convs = nn.ModuleList([\n            nn.Conv1d(in_channels, out_channels, kernel_size, padding=(kernel_size+(kernel_size -1)*(dilation-1))//2, dilation=dilation)\n            for kernel_size, dilation in itertools.product(kernel_sizes, dilations)\n        ])\n\n        self.batch_norm = nn.BatchNorm1d(out_channels * len(kernel_sizes)*len(dilations))\n        self.dropout = nn.Dropout(dropout_rate)\n\n        self.skip_connection = nn.Conv1d(in_channels, out_channels * len(kernel_sizes)*len(dilations), kernel_size=1) \\\n            if in_channels != out_channels * len(kernel_sizes)*len(dilations) else nn.Identity()\n\n\n    def forward(self, x):\n        x = x.transpose(1, 2)\n        skip = self.skip_connection(x)\n\n        x = torch.cat([conv(x) for conv in self.convs], dim=1)\n\n        x = self.batch_norm(x + skip)\n        x = F.relu(x)\n        x = self.dropout(x)\n        x = x.transpose(1, 2)\n\n        return x\n        \n\nclass Model(pl.LightningModule):\n    def __init__(self, in_channels, out_channels, kernel_sizes, dilations, dropout_rate, num_blocks, lr=0.001):\n        super(Model, self).__init__()\n        self.train_loss_history = []\n        self.val_loss_history = []\n        self.val_score_history = []\n        self.lr = lr\n        self.blocks = nn.Sequential(*[\n            ConvBlock(in_channels if i == 0 else out_channels * len(kernel_sizes)*len(dilations), \n                      out_channels, \n                      kernel_sizes, \n                      dropout_rate,\n                      dilations)\n            for i in range(num_blocks)\n        ])\n\n        num_classes = 4\n        self.linear = nn.Linear(out_channels * len(kernel_sizes)*len(dilations), num_classes)\n        weights = torch.tensor([1.0, 1.0, 1.0, 0.1])\n        self.loss = torch.nn.NLLLoss(weight=weights)\n        self.logsoftmax = nn.LogSoftmax(dim=2)\n        self.output = []\n\n    def forward(self, x):\n        x = self.blocks(x)\n        x = self.linear(x)\n        return x\n\n    def training_step(self, batch, batch_idx):\n        x, y = batch\n        y_hat = self(x)\n        y_hat = self.logsoftmax(y_hat)\n        \n        # Reshape the outputs and labels\n        loss = self.loss(y_hat.view(-1, y_hat.size(-1)), y.view(-1))\n        \n        self.log('train_loss', loss, on_step=False, on_epoch=True, prog_bar=True, logger=True)\n        self.train_loss_history.append(loss.item())\n\n        return loss\n\n    def validation_step(self, batch, batch_idx):\n        x, y = batch\n        y_hat = self(x)\n        y_hat = self.logsoftmax(y_hat)\n        \n        # Reshape the outputs and labels\n        loss = self.loss(y_hat.view(-1, y_hat.size(-1)), y.view(-1))\n        self.log('val_loss', loss, on_step=False, on_epoch=True, prog_bar=True, logger=True)\n        \n        num_classes = 4\n        \n        # Create an one-hot encoded array of y\n        ohe_y = np.zeros((len(y.view(-1)), num_classes))\n        ohe_y[np.arange(len(y.view(-1))), y.view(-1).cpu()] = 1\n        \n        y_hat_probs = y_hat.view(-1, y_hat.size(-1)).detach().cpu().numpy()\n\n        scores = []\n        for i in range(3):\n            if np.sum(ohe_y[:, i]) > 0:\n                score = average_precision_score(ohe_y[:, i], y_hat_probs[:, i])\n            else:\n                score = 0.0  # Set default score if no positive class\n            scores.append(score)\n    \n        mean_score = np.mean(scores)  \n        self.log('val_score', mean_score, on_step=False, on_epoch=True, prog_bar=True, logger=True)\n        self.val_loss_history.append(loss.item())\n        self.val_score_history.append(mean_score)\n        return loss\n    \n    \n    def test_step(self, batch, batch_idx):\n        x, file_id, time = batch\n        y_hat = self.forward(x)\n        y_hat = F.softmax(y_hat, dim=2) \n        self.output.append((y_hat, file_id, time))\n        \n    def configure_optimizers(self):\n        optimizer = torch.optim.Adam(self.parameters(), lr=self.lr)\n        return optimizer","metadata":{"papermill":{"duration":0.031656,"end_time":"2023-06-06T21:36:56.852778","exception":false,"start_time":"2023-06-06T21:36:56.821122","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-06-10T10:37:23.041373Z","iopub.execute_input":"2023-06-10T10:37:23.041696Z","iopub.status.idle":"2023-06-10T10:37:23.085805Z","shell.execute_reply.started":"2023-06-10T10:37:23.041667Z","shell.execute_reply":"2023-06-10T10:37:23.084697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission = pd.read_csv(\"/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/sample_submission.csv\")\nsample_submission = sample_submission.set_index('Id')","metadata":{"papermill":{"duration":0.301528,"end_time":"2023-06-06T21:36:57.158370","exception":false,"start_time":"2023-06-06T21:36:56.856842","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-06-10T10:37:23.088032Z","iopub.execute_input":"2023-06-10T10:37:23.088590Z","iopub.status.idle":"2023-06-10T10:37:23.493226Z","shell.execute_reply.started":"2023-06-10T10:37:23.088535Z","shell.execute_reply":"2023-06-10T10:37:23.492226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ensemble of 1D-Convolutional Neural Network (1D-Conv NN) Models\n\nIn addition to the 1D-Convolutional Neural Network architecture, we decided to further improve the model's performance by using an ensemble of models. We employed a K-fold cross-validation strategy with `N_SPLITS` splits to train and evaluate multiple models.\n\nFor each fold, we split the dataset into training and validation sets using the K-fold indices. We created separate datasets and data loaders for both the training and validation sets. Then, we instantiated a new instance of the model for each fold, using the same hyperparameters as before.\n\nUsing KFold, we created an ensemble of 5 group models, whose validation scores were averaged. For each dataset, we took 5 models with 9 combinations of kernel sizes and dilations. Kernel sizes, dropouts, and dilation rates have been determined through trial and error. Optimal parameters have been selected.","metadata":{}},{"cell_type":"markdown","source":"**It's important to note** that we made predictions for the `tdcsfog` and `defog` datasets separately. This means that we trained the ensemble models on the `tdcsfog` dataset and evaluated their performance on the corresponding validation set. Likewise, we trained another ensemble on the `defog` dataset and evaluated its performance on the corresponding validation set.\n\nBy treating the `tdcsfog` and `defog` datasets separately, we aimed to optimize the models' performance for each specific type of data. This approach allowed us to capture the unique characteristics and patterns present in both datasets, leading to improved predictions for each dataset individually.","metadata":{}},{"cell_type":"code","source":"N_SPLITS = 5\n\nmodel_params = {\n    'in_channels': 35,\n    'out_channels': 7,\n    'kernel_sizes': [3, 5, 7],\n    'dilations': [2, 4, 8],\n    'dropout_rate': 0.1,\n    'num_blocks': 3,  \n}\n","metadata":{"execution":{"iopub.status.busy":"2023-06-10T10:37:23.498384Z","iopub.execute_input":"2023-06-10T10:37:23.500639Z","iopub.status.idle":"2023-06-10T10:37:23.507286Z","shell.execute_reply.started":"2023-06-10T10:37:23.500606Z","shell.execute_reply":"2023-06-10T10:37:23.506471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Implementing ensemble of 1D-CNN models for the `tdcsfog` dataset","metadata":{}},{"cell_type":"code","source":"folder_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/tdcsfog/'\nall_files = [file for file in os.listdir(folder_path)]\n\ntest_folder_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog/'\ntest_files = [file for file in os.listdir(test_folder_path)]\ntest_dataset = SegmentedDataset(test_folder_path, test_files, is_train=False)\ntest_loader = torch.utils.data.DataLoader(test_dataset, batch_size=256, shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2023-06-10T10:37:23.508447Z","iopub.execute_input":"2023-06-10T10:37:23.508967Z","iopub.status.idle":"2023-06-10T10:37:23.765747Z","shell.execute_reply.started":"2023-06-10T10:37:23.508909Z","shell.execute_reply":"2023-06-10T10:37:23.764868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kf = KFold(n_splits=N_SPLITS, random_state=404, shuffle=True)\n\nscores = []\n\nfor fold, (train_idx, val_idx) in enumerate(kf.split(all_files)):\n    train_files = [all_files[i] for i in train_idx]\n    val_files = [all_files[i] for i in val_idx]\n    \n    train_dataset = SegmentedDataset(folder_path, train_files)\n    val_dataset = SegmentedDataset(folder_path, val_files)\n\n    train_loader = DataLoader(train_dataset, batch_size=256, num_workers=2)\n    val_loader = DataLoader(val_dataset, batch_size=256, num_workers=2)\n    \n    model = Model(**model_params)\n\n    checkpoint_callback = ModelCheckpoint(\n        dirpath=f'fold_{fold}_checkpoints',\n        filename='best_model',\n        monitor='val_score',\n        mode='max',\n        save_top_k=1\n    )\n\n    \n    trainer = pl.Trainer(max_epochs=10, callbacks=[checkpoint_callback], accelerator=\"gpu\")\n    trainer.fit(model, train_loader, val_loader)\n    print(f'Model {fold}, score: ',  checkpoint_callback.best_model_score.item())\n    scores.append(checkpoint_callback.best_model_score.item())\n    \n    # Load the best model checkpoint\n    best_model_path = os.path.join(checkpoint_callback.dirpath, f'{checkpoint_callback.filename}.ckpt')\n    best_model = Model.load_from_checkpoint(best_model_path, **model_params)\n    \n    # Evaluate the best model on the test set\n    trainer.test(best_model, test_loader)\n\n    result = pd.DataFrame()\n    for batch in best_model.output:\n        preds = batch[0].cpu().numpy()\n        ids = batch[1]\n        times = batch[2].cpu().numpy()\n        for i, time, pred in zip(ids, times, preds):\n            id_time = [f'{i}_{t}' for t in time]\n            tmp = pd.DataFrame(pred[:,:3], index=id_time).loc[[t != -1 for t in time]]\n            result = pd.concat([result, tmp])\n\n    result.columns = ['StartHesitation', 'Turn', 'Walking']\n\n    sample_submission = sample_submission.add(result, fill_value=0)\n    \n    del train_dataset, val_dataset, val_loader, train_loader, checkpoint_callback, trainer, best_model, result\n    torch.cuda.empty_cache()\n    gc.collect()\n    \nprint('Mean score: ', sum(scores)/N_SPLITS)","metadata":{"papermill":{"duration":1136.999965,"end_time":"2023-06-06T21:55:54.180083","exception":false,"start_time":"2023-06-06T21:36:57.180118","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-06-10T10:37:23.769279Z","iopub.execute_input":"2023-06-10T10:37:23.772254Z","iopub.status.idle":"2023-06-10T10:43:47.843418Z","shell.execute_reply.started":"2023-06-10T10:37:23.772221Z","shell.execute_reply":"2023-06-10T10:43:47.842300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Implementing ensemble of 1D-CNN models for the `defog` dataset","metadata":{}},{"cell_type":"code","source":"folder_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/defog/'\nall_files = [file for file in os.listdir(folder_path)]\n\ntest_folder_path = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/defog/'\ntest_files = [file for file in os.listdir(test_folder_path)]\ntest_dataset = SegmentedDataset(test_folder_path, test_files, is_train=False, is_defog=True)\ntest_loader = torch.utils.data.DataLoader(test_dataset, batch_size=256, shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2023-06-10T10:43:47.846845Z","iopub.execute_input":"2023-06-10T10:43:47.847232Z","iopub.status.idle":"2023-06-10T10:43:49.717007Z","shell.execute_reply.started":"2023-06-10T10:43:47.847196Z","shell.execute_reply":"2023-06-10T10:43:49.715993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kf = KFold(n_splits=N_SPLITS, random_state=404, shuffle=True)\n\nscores = []\n\nfor fold, (train_idx, val_idx) in enumerate(kf.split(all_files)):\n    train_files = [all_files[i] for i in train_idx]\n    val_files = [all_files[i] for i in val_idx]\n    \n    train_dataset = SegmentedDataset(folder_path, train_files, is_defog=True)\n    val_dataset = SegmentedDataset(folder_path, val_files, is_defog=True)\n\n    train_loader = DataLoader(train_dataset, batch_size=256, num_workers=2)\n    val_loader = DataLoader(val_dataset, batch_size=256, num_workers=2)\n    \n    model = Model(**model_params)\n    \n    checkpoint_callback = ModelCheckpoint(\n        dirpath=f'fold_{fold}_checkpoints_defog',\n        filename='best_model',\n        monitor='val_score',\n        mode='max',\n        save_top_k=1\n    )\n\n    \n    trainer = pl.Trainer(max_epochs=10, callbacks=[checkpoint_callback], accelerator=\"gpu\")\n    trainer.fit(model, train_loader, val_loader)\n    print(f'Model {fold}, score: ',  checkpoint_callback.best_model_score.item())\n    scores.append(checkpoint_callback.best_model_score.item())\n        \n    # Load the best model checkpoint\n    best_model_path = os.path.join(checkpoint_callback.dirpath, f'{checkpoint_callback.filename}.ckpt')\n    best_model = Model.load_from_checkpoint(best_model_path, **model_params)\n    \n    # Evaluate the best model on the test set\n    trainer.test(best_model, test_loader)\n\n    result = pd.DataFrame()\n  #  for batch in model.output:\n    for batch in best_model.output:\n        preds = batch[0].cpu().numpy()\n        ids = batch[1]\n        times = batch[2].cpu().numpy()\n        for i, time, pred in zip(ids, times, preds):\n            id_time = [f'{i}_{t}' for t in time]\n            tmp = pd.DataFrame(pred[:,:3], index=id_time).loc[[t != -1 for t in time]]\n            result = pd.concat([result, tmp])\n\n    result.columns = ['StartHesitation', 'Turn', 'Walking']\n\n    sample_submission = sample_submission.add(result, fill_value=0)\n    \n    del train_dataset, val_dataset, val_loader, train_loader, checkpoint_callback, trainer, best_model, result\n    torch.cuda.empty_cache()\n    gc.collect()\n    \nprint('Mean score: ', sum(scores)/N_SPLITS)","metadata":{"papermill":{"duration":777.392226,"end_time":"2023-06-06T22:08:52.030509","exception":false,"start_time":"2023-06-06T21:55:54.638283","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-06-10T10:43:49.718712Z","iopub.execute_input":"2023-06-10T10:43:49.719107Z","iopub.status.idle":"2023-06-10T10:51:23.215295Z","shell.execute_reply.started":"2023-06-10T10:43:49.719073Z","shell.execute_reply":"2023-06-10T10:51:23.214129Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Finally, we record all the results in the submission. The submission is created from the average over each fold.","metadata":{}},{"cell_type":"code","source":"sample_submission = sample_submission.div(N_SPLITS)\nsample_submission = sample_submission.reset_index()\nsample_submission.columns = ['Id', 'StartHesitation', 'Turn', 'Walking']\nsample_submission.to_csv('submission.csv', index=False)","metadata":{"papermill":{"duration":2.659951,"end_time":"2023-06-06T22:08:54.725347","exception":false,"start_time":"2023-06-06T22:08:52.065396","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-06-10T10:51:23.218116Z","iopub.execute_input":"2023-06-10T10:51:23.218704Z","iopub.status.idle":"2023-06-10T10:51:25.206476Z","shell.execute_reply.started":"2023-06-10T10:51:23.218666Z","shell.execute_reply":"2023-06-10T10:51:25.205341Z"},"trusted":true},"execution_count":null,"outputs":[]}]}