{
  "id": 418008,
  "title": "11th place solution: LSTM-CNN + rolling features",
  "url": "/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/418008",
  "author_name": "Ismail",
  "post_date": "2023-06-18T10:03:43.429000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<h1>Features</h1>\n<p>I used the following features as model inputs, generated from AccV, AccML, AccAP</p>\n<ul>\n<li>Lags</li>\n<li>Global mean, median, max, average, std and quantiles</li>\n<li>Rolling mean, median, max, average, std and quantiles (+all of these applied to reversed time series)</li>\n<li>Rolling mean, median, max, average, std and quantiles of first difference of time series (+all of these applied to reversed time series)</li>\n<li>Mean number of sign changes over rolling window (+reversed time series)</li>\n<li>Exponentially weighted mean of first difference of time series</li>\n<li>Portion of time past</li>\n</ul>\n<p>My code for the feature generation:</p>\n<pre><code> () -&gt; pd.DataFrame:\n\n     back:\n        rolling = dt[cols][::-].rolling(step, min_periods=)\n        suffix = \n    :\n        rolling = dt[cols].rolling(step, min_periods=)\n        suffix = \n\n     aggfunc.startswith():\n        quantile = (aggfunc.split()[]) / \n\n         (\n            rolling.quantile(quantile)\n            .add_suffix(suffix))\n    :\n         (\n            rolling.agg(aggfunc)\n            .add_suffix(suffix))\n\n ():\n    cols = [, , ]\n    dt = data.copy()\n\n      defog:\n        data[cols] = data[cols] / \n    dt[] = (defog)\n     verbose: ()\n     aggfunc  [, , , , ]:\n        dt = dt.join(\n            dt[cols].groupby(dt.assign(dummy=).dummy)\n            .transform(aggfunc).add_suffix()\n        )\n\n    step1 = \n    step2 = \n     verbose: ()\n     shift  [, , -, -]:\n\n         shift &gt; :\n            suffix_name = \n            fill_data = dt[cols].iloc[: shift]\n        :\n            suffix_name = \n            fill_data = dt[cols].iloc[shift:]\n\n        dt = dt.join(\n            dt[cols]\n            .shift(shift)\n            .fillna(fill_data)\n            .add_suffix(suffix_name)\n        )\n\n    aggfuncs = [\n        , , , , , \n        , , \n    ]\n     verbose: ()\n\n     aggfunc  aggfuncs:\n\n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols)\n        )\n\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        funcname = aggfunc  (aggfunc, )  aggfunc.__name__\n        dt = dt.join(\n            rolling_agg(dt, step2, aggfunc, cols)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols, back=)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt[::-], step2, aggfunc, cols, back=)\n        )\n     verbose: ()\n\n    diff = dt[cols].transform().add_suffix()\n    diff = diff.fillna(diff.iloc[])\n    cols = [, , ]\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols)\n        )\n\n     verbose: ()\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols)\n        )\n\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols, back=)\n        )\n\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols, back=)\n        )\n\n     verbose: ()\n    sign_change = (\n        diff.apply(np.sign)\n        .transform()\n        .apply(np.)\n        .divide()\n        .fillna()\n        .add_suffix()\n    )\n\n    cols = [, , ]\n    aggfuncs = []\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols, back=)\n        )\n     verbose: ()\n\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols, back=)\n        )\n\n     verbose: ()\n\n    dt[] = dt.Time.divide(dt.Time.())\n     dt.drop(, axis=).fillna()\n</code></pre>\n<p>After those transformations, I scaled data and divided all time series into parts of length 10000 in separate files of feather format</p>\n<h1>Model</h1>\n<p>I used LSTM-CNN model: the input is first fed into three parallel blocks of Conv1D with the different kernel sizes of 3, 5 and 7. Then the input is concatenated with the output of those conv layers and passed to two sequential layers of LSTM. After all - there is a linear layer that does the classification</p>\n<p>Here's a code for model definition</p>\n<pre><code> torch.nn  nn\n torch.nn.functional  F\n\n ():\n     nn.Sequential(\n        nn.Conv1d(, , kernel_size, padding=),\n        nn.ReLU(),\n        nn.Conv1d(, , kernel_size, padding=),\n        nn.ReLU(),\n    )\n\n (nn.Module):\n     ():\n        self.kernels = kernels\n        ().__init__()\n        self.conv_nets = nn.ModuleList([\n            block(i)  i  kernels\n        ])\n        self.lstm = nn.LSTM(\n             * (self.kernels) + , \n            , , batch_first=, bidirectional=, dropout=)\n        self.linear = nn.Linear( * , )\n\n     ():\n        conv_res = []\n         net  self.conv_nets:\n            conv_res.append(net(x))\n        conv_res.append(x)\n\n        conv_res_tensor = torch.concat(conv_res, axis=)\n        lstm_out, _ = self.lstm(conv_res_tensor.transpose(, ))\n        res = self.linear(lstm_out).transpose(, )\n         res\n</code></pre>\n<p>I did not manage to create a good validation pipeline, so i didn't use any folds. I trained the model for 30 epoch, monitoring loss on validation data (approx. 10% of subjects)</p>",
  "messages": [
    {
      "id": 2307604,
      "postDate": "2023-06-18T10:03:43.430Z",
      "content": "<h1>Features</h1>\n<p>I used the following features as model inputs, generated from AccV, AccML, AccAP</p>\n<ul>\n<li>Lags</li>\n<li>Global mean, median, max, average, std and quantiles</li>\n<li>Rolling mean, median, max, average, std and quantiles (+all of these applied to reversed time series)</li>\n<li>Rolling mean, median, max, average, std and quantiles of first difference of time series (+all of these applied to reversed time series)</li>\n<li>Mean number of sign changes over rolling window (+reversed time series)</li>\n<li>Exponentially weighted mean of first difference of time series</li>\n<li>Portion of time past</li>\n</ul>\n<p>My code for the feature generation:</p>\n<pre><code> () -&gt; pd.DataFrame:\n\n     back:\n        rolling = dt[cols][::-].rolling(step, min_periods=)\n        suffix = \n    :\n        rolling = dt[cols].rolling(step, min_periods=)\n        suffix = \n\n     aggfunc.startswith():\n        quantile = (aggfunc.split()[]) / \n\n         (\n            rolling.quantile(quantile)\n            .add_suffix(suffix))\n    :\n         (\n            rolling.agg(aggfunc)\n            .add_suffix(suffix))\n\n ():\n    cols = [, , ]\n    dt = data.copy()\n\n      defog:\n        data[cols] = data[cols] / \n    dt[] = (defog)\n     verbose: ()\n     aggfunc  [, , , , ]:\n        dt = dt.join(\n            dt[cols].groupby(dt.assign(dummy=).dummy)\n            .transform(aggfunc).add_suffix()\n        )\n\n    step1 = \n    step2 = \n     verbose: ()\n     shift  [, , -, -]:\n\n         shift &gt; :\n            suffix_name = \n            fill_data = dt[cols].iloc[: shift]\n        :\n            suffix_name = \n            fill_data = dt[cols].iloc[shift:]\n\n        dt = dt.join(\n            dt[cols]\n            .shift(shift)\n            .fillna(fill_data)\n            .add_suffix(suffix_name)\n        )\n\n    aggfuncs = [\n        , , , , , \n        , , \n    ]\n     verbose: ()\n\n     aggfunc  aggfuncs:\n\n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols)\n        )\n\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        funcname = aggfunc  (aggfunc, )  aggfunc.__name__\n        dt = dt.join(\n            rolling_agg(dt, step2, aggfunc, cols)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols, back=)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt[::-], step2, aggfunc, cols, back=)\n        )\n     verbose: ()\n\n    diff = dt[cols].transform().add_suffix()\n    diff = diff.fillna(diff.iloc[])\n    cols = [, , ]\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols)\n        )\n\n     verbose: ()\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols)\n        )\n\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols, back=)\n        )\n\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols, back=)\n        )\n\n     verbose: ()\n    sign_change = (\n        diff.apply(np.sign)\n        .transform()\n        .apply(np.)\n        .divide()\n        .fillna()\n        .add_suffix()\n    )\n\n    cols = [, , ]\n    aggfuncs = []\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols)\n        )\n     verbose: ()\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols, back=)\n        )\n     verbose: ()\n\n\n     aggfunc  aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols, back=)\n        )\n\n     verbose: ()\n\n    dt[] = dt.Time.divide(dt.Time.())\n     dt.drop(, axis=).fillna()\n</code></pre>\n<p>After those transformations, I scaled data and divided all time series into parts of length 10000 in separate files of feather format</p>\n<h1>Model</h1>\n<p>I used LSTM-CNN model: the input is first fed into three parallel blocks of Conv1D with the different kernel sizes of 3, 5 and 7. Then the input is concatenated with the output of those conv layers and passed to two sequential layers of LSTM. After all - there is a linear layer that does the classification</p>\n<p>Here's a code for model definition</p>\n<pre><code> torch.nn  nn\n torch.nn.functional  F\n\n ():\n     nn.Sequential(\n        nn.Conv1d(, , kernel_size, padding=),\n        nn.ReLU(),\n        nn.Conv1d(, , kernel_size, padding=),\n        nn.ReLU(),\n    )\n\n (nn.Module):\n     ():\n        self.kernels = kernels\n        ().__init__()\n        self.conv_nets = nn.ModuleList([\n            block(i)  i  kernels\n        ])\n        self.lstm = nn.LSTM(\n             * (self.kernels) + , \n            , , batch_first=, bidirectional=, dropout=)\n        self.linear = nn.Linear( * , )\n\n     ():\n        conv_res = []\n         net  self.conv_nets:\n            conv_res.append(net(x))\n        conv_res.append(x)\n\n        conv_res_tensor = torch.concat(conv_res, axis=)\n        lstm_out, _ = self.lstm(conv_res_tensor.transpose(, ))\n        res = self.linear(lstm_out).transpose(, )\n         res\n</code></pre>\n<p>I did not manage to create a good validation pipeline, so i didn't use any folds. I trained the model for 30 epoch, monitoring loss on validation data (approx. 10% of subjects)</p>",
      "rawMarkdown": "# Features\nI used the following features as model inputs, generated from AccV, AccML, AccAP\n\n- Lags\n- Global mean, median, max, average, std and quantiles\n- Rolling mean, median, max, average, std and quantiles (+all of these applied to reversed time series)\n- Rolling mean, median, max, average, std and quantiles of first difference of time series (+all of these applied to reversed time series)\n- Mean number of sign changes over rolling window (+reversed time series)\n- Exponentially weighted mean of first difference of time series\n- Portion of time past\n\nMy code for the feature generation:\n```python\ndef rolling_agg(\n    dt: pd.DataFrame, step: int, \n    aggfunc: str, cols: list, back: bool = False) -> pd.DataFrame:\n    \n    if back:\n        rolling = dt[cols][::-1].rolling(step, min_periods=0)\n        suffix = f\"_back_rolling_{step}_{aggfunc}\"\n    else:\n        rolling = dt[cols].rolling(step, min_periods=0)\n        suffix = f\"_rolling_{step}_{aggfunc}\"\n        \n    if aggfunc.startswith(\"quantile\"):\n        quantile = int(aggfunc.split(\"_\")[1]) / 100\n        \n        return (\n            rolling.quantile(quantile)\n            .add_suffix(suffix))\n    else:\n        return (\n            rolling.agg(aggfunc)\n            .add_suffix(suffix))\n\ndef create_dataset(data, defog=False, verbose=False):\n    cols = ['AccV', 'AccML', 'AccAP']\n    dt = data.copy()\n    \n    if not defog:\n        data[cols] = data[cols] / 9.80665\n    dt[\"defog\"] = int(defog)\n    if verbose: print(\"Global stats\")\n    for aggfunc in [\"mean\", \"max\", \"min\", \"std\", \"median\"]:\n        dt = dt.join(\n            dt[cols].groupby(dt.assign(dummy=1).dummy)\n            .transform(aggfunc).add_suffix(f\"_{aggfunc}\")\n        )\n    \n    step1 = 500\n    step2 = 100\n    if verbose: print(\"Shifts stats\")\n    for shift in [1, 2, -1, -2]:\n        \n        if shift > 0:\n            suffix_name = f\"_lag_{shift}\"\n            fill_data = dt[cols].iloc[: shift]\n        else:\n            suffix_name = f\"_lead_{abs(shift)}\"\n            fill_data = dt[cols].iloc[shift:]\n        \n        dt = dt.join(\n            dt[cols]\n            .shift(shift)\n            .fillna(fill_data)\n            .add_suffix(suffix_name)\n        )\n    \n    aggfuncs = [\n        \"mean\", \"std\", \"max\", \"min\", \"median\", \n        \"quantile_75\", \"quantile_25\", \"quantile_99\"\n    ]\n    if verbose: print(\"Rolling stats, step 1\")\n\n    for aggfunc in aggfuncs:\n        \n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols)\n        )\n    \n    if verbose: print(\"Rolling stats, step 2\")\n\n    for aggfunc in aggfuncs:\n        funcname = aggfunc if isinstance(aggfunc, str) else aggfunc.__name__\n        dt = dt.join(\n            rolling_agg(dt, step2, aggfunc, cols)\n        )\n    if verbose: print(\"Back Rolling stats, step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols, back=True)\n        )\n    if verbose: print(\"Back Rolling stats, step 2\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt[::-1], step2, aggfunc, cols, back=True)\n        )\n    if verbose: print(\"Calculating diffs\")\n\n    diff = dt[cols].transform(\"diff\").add_suffix(\"_diff\")\n    diff = diff.fillna(diff.iloc[0])\n    cols = [\"AccV_diff\", \"AccML_diff\", \"AccAP_diff\"]\n    if verbose: print(\"Diff rolling stat step 1\")\n    \n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols)\n        )\n        \n    if verbose: print(\"Diff rolling stat step 2\")\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols)\n        )\n        \n    if verbose: print(\"Back Diff rolling stat step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols, back=True)\n        )\n    \n    if verbose: print(\"Back Diff rolling stat step 2\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols, back=True)\n        )\n    \n    if verbose: print(\"Sign change\")\n    sign_change = (\n        diff.apply(np.sign)\n        .transform(\"diff\")\n        .apply(np.abs)\n        .divide(2)\n        .fillna(0)\n        .add_suffix(\"_sc\")\n    )\n    \n    cols = [\"AccV_diff_sc\", \"AccML_diff_sc\", \"AccAP_diff_sc\"]\n    aggfuncs = [\"mean\"]\n    if verbose: print(\"Sign change rolling stat step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols)\n        )\n    if verbose: print(\"Sign change rolling stat step 2\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols)\n        )\n    if verbose: print(\"Back Sign change rolling stat step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols, back=True)\n        )\n    if verbose: print(\"Back Sign change rolling stat step 2\")\n\n        \n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols, back=True)\n        )\n        \n    if verbose: print(\"time spent\")\n    \n    dt[\"time_spent\"] = dt.Time.divide(dt.Time.max())\n    return dt.drop(\"Time\", axis=1).fillna(0)\n```\n\nAfter those transformations, I scaled data and divided all time series into parts of length 10000 in separate files of feather format\n\n# Model\n\nI used LSTM-CNN model: the input is first fed into three parallel blocks of Conv1D with the different kernel sizes of 3, 5 and 7. Then the input is concatenated with the output of those conv layers and passed to two sequential layers of LSTM. After all - there is a linear layer that does the classification\n\nHere's a code for model definition\n\n```python\nimport torch.nn as nn\nimport torch.nn.functional as F\n\ndef block(kernel_size):\n    return nn.Sequential(\n        nn.Conv1d(240, 128, kernel_size, padding=\"same\"),\n        nn.ReLU(),\n        nn.Conv1d(128, 64, kernel_size, padding=\"same\"),\n        nn.ReLU(),\n    )\n\nclass ParkinsonModel(nn.Module):\n    def __init__(self, kernels=[3, 5, 7]):\n        self.kernels = kernels\n        super().__init__()\n        self.conv_nets = nn.ModuleList([\n            block(i) for i in kernels\n        ])\n        self.lstm = nn.LSTM(\n            64 * len(self.kernels) + 240, \n            128, 2, batch_first=True, bidirectional=True, dropout=.1)\n        self.linear = nn.Linear(128 * 2, 4)\n\n    def forward(self, x):\n        conv_res = []\n        for net in self.conv_nets:\n            conv_res.append(net(x))\n        conv_res.append(x)\n            \n        conv_res_tensor = torch.concat(conv_res, axis=1)\n        lstm_out, _ = self.lstm(conv_res_tensor.transpose(2, 1))\n        res = self.linear(lstm_out).transpose(2, 1)\n        return res\n```\n\nI did not manage to create a good validation pipeline, so i didn't use any folds. I trained the model for 30 epoch, monitoring loss on validation data (approx. 10% of subjects)",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2307604": "# Features\nI used the following features as model inputs, generated from AccV, AccML, AccAP\n\n- Lags\n- Global mean, median, max, average, std and quantiles\n- Rolling mean, median, max, average, std and quantiles (+all of these applied to reversed time series)\n- Rolling mean, median, max, average, std and quantiles of first difference of time series (+all of these applied to reversed time series)\n- Mean number of sign changes over rolling window (+reversed time series)\n- Exponentially weighted mean of first difference of time series\n- Portion of time past\n\nMy code for the feature generation:\n```python\ndef rolling_agg(\n    dt: pd.DataFrame, step: int, \n    aggfunc: str, cols: list, back: bool = False) -> pd.DataFrame:\n    \n    if back:\n        rolling = dt[cols][::-1].rolling(step, min_periods=0)\n        suffix = f\"_back_rolling_{step}_{aggfunc}\"\n    else:\n        rolling = dt[cols].rolling(step, min_periods=0)\n        suffix = f\"_rolling_{step}_{aggfunc}\"\n        \n    if aggfunc.startswith(\"quantile\"):\n        quantile = int(aggfunc.split(\"_\")[1]) / 100\n        \n        return (\n            rolling.quantile(quantile)\n            .add_suffix(suffix))\n    else:\n        return (\n            rolling.agg(aggfunc)\n            .add_suffix(suffix))\n\ndef create_dataset(data, defog=False, verbose=False):\n    cols = ['AccV', 'AccML', 'AccAP']\n    dt = data.copy()\n    \n    if not defog:\n        data[cols] = data[cols] / 9.80665\n    dt[\"defog\"] = int(defog)\n    if verbose: print(\"Global stats\")\n    for aggfunc in [\"mean\", \"max\", \"min\", \"std\", \"median\"]:\n        dt = dt.join(\n            dt[cols].groupby(dt.assign(dummy=1).dummy)\n            .transform(aggfunc).add_suffix(f\"_{aggfunc}\")\n        )\n    \n    step1 = 500\n    step2 = 100\n    if verbose: print(\"Shifts stats\")\n    for shift in [1, 2, -1, -2]:\n        \n        if shift > 0:\n            suffix_name = f\"_lag_{shift}\"\n            fill_data = dt[cols].iloc[: shift]\n        else:\n            suffix_name = f\"_lead_{abs(shift)}\"\n            fill_data = dt[cols].iloc[shift:]\n        \n        dt = dt.join(\n            dt[cols]\n            .shift(shift)\n            .fillna(fill_data)\n            .add_suffix(suffix_name)\n        )\n    \n    aggfuncs = [\n        \"mean\", \"std\", \"max\", \"min\", \"median\", \n        \"quantile_75\", \"quantile_25\", \"quantile_99\"\n    ]\n    if verbose: print(\"Rolling stats, step 1\")\n\n    for aggfunc in aggfuncs:\n        \n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols)\n        )\n    \n    if verbose: print(\"Rolling stats, step 2\")\n\n    for aggfunc in aggfuncs:\n        funcname = aggfunc if isinstance(aggfunc, str) else aggfunc.__name__\n        dt = dt.join(\n            rolling_agg(dt, step2, aggfunc, cols)\n        )\n    if verbose: print(\"Back Rolling stats, step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt, step1, aggfunc, cols, back=True)\n        )\n    if verbose: print(\"Back Rolling stats, step 2\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(dt[::-1], step2, aggfunc, cols, back=True)\n        )\n    if verbose: print(\"Calculating diffs\")\n\n    diff = dt[cols].transform(\"diff\").add_suffix(\"_diff\")\n    diff = diff.fillna(diff.iloc[0])\n    cols = [\"AccV_diff\", \"AccML_diff\", \"AccAP_diff\"]\n    if verbose: print(\"Diff rolling stat step 1\")\n    \n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols)\n        )\n        \n    if verbose: print(\"Diff rolling stat step 2\")\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols)\n        )\n        \n    if verbose: print(\"Back Diff rolling stat step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step1, aggfunc, cols, back=True)\n        )\n    \n    if verbose: print(\"Back Diff rolling stat step 2\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(diff, step2, aggfunc, cols, back=True)\n        )\n    \n    if verbose: print(\"Sign change\")\n    sign_change = (\n        diff.apply(np.sign)\n        .transform(\"diff\")\n        .apply(np.abs)\n        .divide(2)\n        .fillna(0)\n        .add_suffix(\"_sc\")\n    )\n    \n    cols = [\"AccV_diff_sc\", \"AccML_diff_sc\", \"AccAP_diff_sc\"]\n    aggfuncs = [\"mean\"]\n    if verbose: print(\"Sign change rolling stat step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols)\n        )\n    if verbose: print(\"Sign change rolling stat step 2\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols)\n        )\n    if verbose: print(\"Back Sign change rolling stat step 1\")\n\n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step1, aggfunc, cols, back=True)\n        )\n    if verbose: print(\"Back Sign change rolling stat step 2\")\n\n        \n    for aggfunc in aggfuncs:\n        dt = dt.join(\n            rolling_agg(sign_change, step2, aggfunc, cols, back=True)\n        )\n        \n    if verbose: print(\"time spent\")\n    \n    dt[\"time_spent\"] = dt.Time.divide(dt.Time.max())\n    return dt.drop(\"Time\", axis=1).fillna(0)\n```\n\nAfter those transformations, I scaled data and divided all time series into parts of length 10000 in separate files of feather format\n\n# Model\n\nI used LSTM-CNN model: the input is first fed into three parallel blocks of Conv1D with the different kernel sizes of 3, 5 and 7. Then the input is concatenated with the output of those conv layers and passed to two sequential layers of LSTM. After all - there is a linear layer that does the classification\n\nHere's a code for model definition\n\n```python\nimport torch.nn as nn\nimport torch.nn.functional as F\n\ndef block(kernel_size):\n    return nn.Sequential(\n        nn.Conv1d(240, 128, kernel_size, padding=\"same\"),\n        nn.ReLU(),\n        nn.Conv1d(128, 64, kernel_size, padding=\"same\"),\n        nn.ReLU(),\n    )\n\nclass ParkinsonModel(nn.Module):\n    def __init__(self, kernels=[3, 5, 7]):\n        self.kernels = kernels\n        super().__init__()\n        self.conv_nets = nn.ModuleList([\n            block(i) for i in kernels\n        ])\n        self.lstm = nn.LSTM(\n            64 * len(self.kernels) + 240, \n            128, 2, batch_first=True, bidirectional=True, dropout=.1)\n        self.linear = nn.Linear(128 * 2, 4)\n\n    def forward(self, x):\n        conv_res = []\n        for net in self.conv_nets:\n            conv_res.append(net(x))\n        conv_res.append(x)\n            \n        conv_res_tensor = torch.concat(conv_res, axis=1)\n        lstm_out, _ = self.lstm(conv_res_tensor.transpose(2, 1))\n        res = self.linear(lstm_out).transpose(2, 1)\n        return res\n```\n\nI did not manage to create a good validation pipeline, so i didn't use any folds. I trained the model for 30 epoch, monitoring loss on validation data (approx. 10% of subjects)"
  }
}