{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":106809,"databundleVersionId":13056355,"sourceType":"competition"},{"sourceId":14384171,"sourceType":"datasetVersion","datasetId":9173035}],"dockerImageVersionId":31234,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**NOTE : If you use this dataset, simulations, or the LRNN + Adaptive Alignment Loss framework for any research, publication, or academic purpose, please cite:**\n\n---\n\nDixit, N. (2026). Liquid Recurrent Neural Network with Adaptive Alignment Loss for Brain-to-Text Decoding. Kaggle Dataset. Licensed under CC BY-NC-ND 4.0.","metadata":{}},{"cell_type":"markdown","source":"# Liquid Recurrent Neural Network with Adaptive Alignment Loss\n---\n\n## Abstract\n\n**LRNN (Liquid Recurrent Neural Network)** and **AAL (Adaptive Alignment Loss)** constitute a co-designed neural architecture and optimization framework engineered for sequence-to-sequence modeling under conditions of extreme temporal uncertainty, specifically optimized for **brain-to-text and neural signal decoding**. Conventional recurrent architectures and alignment objectives, such as Connectionist Temporal Classification (CTC), often fail to capture the non-stationary temporal dynamics and high noise profiles inherent in neural interfaces due to rigid recurrence and framewise independence assumptions.\n\nThe proposed **LRNN** addresses these limitations by modeling hidden states as continuous-time dynamic nodes utilizing learnable, input-dependent decay and mixing coefficients. This architecture bifurcates temporal processing into a **Liquid Neural Network (LNN)** component for fast-scale adaptive dynamics and an **RNN-style gated** component for long-term stability. Complementing this architecture, the **Adaptive Alignment Loss (AAL)** replaces fixed, heuristic alignment rules with context-aware transition modeling conditioned directly on the LRNN hidden states. By learning optimal frame-to-token transitions and adaptive blank-symbol behaviors, the AAL framework strictly generalizes CTC while providing superior handling of repeated tokens and signal drift. This unified system offers a principled approach to decoding complex neural time-series data, facilitating high-fidelity text reconstruction from noisy, non-stationary signals.\n\n---\n\n## 1. Introduction\n\nThe decoding of linguistic intent from neural or brain-derived signals represents a formidable frontier in sequence modeling, primarily due to the **non-stationary temporal dynamics**, pervasive noise, and the inherent lack of deterministic alignments between high-dimensional input frames and discrete output symbols. Conventional recurrent neural architectures—including LSTMs and GRUs—and standard alignment objectives like **Connectionist Temporal Classification (CTC)** are fundamentally constrained by rigid recurrence dynamics and the assumption of framewise independence. These limitations often result in a failure to generalize across the stochastic fluctuations characteristic of neuro-prosthetic interfaces.\n\nTo transcend these barriers, we introduce a unified framework comprising a **Liquid Recurrent Neural Network (LRNN)** integrated with an **Adaptive Alignment Loss (AAL)**. The LRNN architecture leverages continuous-time-inspired, input-dependent adaptive dynamics to track rapid signal variances, while the AAL replaces static, heuristic alignment rules with a learned, context-aware transition model. By synthesizing fast-scale signal tracking with robust long-term dependency preservation, this system enables the autonomous acquisition of optimal frame-to-token alignments directly from raw, unsegmented neural data.\n\n---\n\n## 2. Liquid Recurrent Neural Network (LRNN)\n\n### 2.1 Definition and Theoretical Motivation\n\nThe **Liquid Recurrent Neural Network (LRNN)** represents a hybrid computational paradigm that synthesizes the principles of **Liquid State Machines (LSM)** with the robust gradient-based learning of classical **Recurrent Neural Networks (RNNs)**. At its core, the LRNN treats hidden states as **continuous-time dynamic nodes** governed by **learnable, input-dependent decay and mixing**. This formulation allows for significantly richer temporal representations compared to the rigid, discrete-time transitions found in vanilla RNN, LSTM, or GRU architectures.  \n\nThe architecture is characterized by a strategic **time-scale separation** achieved through the fusion of two discrete components:  \n\n* **Liquid Neural Network (LNN) Component**: This module operates on a **fast time-scale**, providing high-fidelity, adaptive dynamics designed to track rapid, transient fluctuations in neural or acoustic signal manifolds.  \n* **RNN-Style Gated Component**: This module operates on a **slow time-scale**, ensuring stable recurrence and the preservation of long-term dependencies essential for maintaining global temporal structure.  \n\nThis dual-pathway design is specifically engineered for the exigencies of **brain-to-text decoding**, where signal statistics are inherently non-stationary and the system must remain resilient to abrupt, stochastic state transitions. By interpolating between fast-scale adaptation and slow-scale stability, the LRNN provides a more biologically plausible and mathematically flexible framework for decoding high-entropy neural time-series data.  \n\n---\n\n### 2.2 Input–Output Formulation\n\nLet the input sequence be:  \n\n$$\n\\mathbf{x} = {x_1, x_2, \\ldots, x_T}, \\quad x_t \\in \\mathbb{R}^{D_x}\n$$\n\nDefine:  \n\n* Liquid state:  \n  $$\n  s_t \\in \\mathbb{R}^{D_s}\n  $$\n\n* Stable recurrent hidden state:  \n  $$\n  h_t \\in \\mathbb{R}^{D_h}\n  $$\n\n* Output logits over the vocabulary (\\mathcal{V}) (including the blank symbol):  \n  $$\n  z_t \\in \\mathbb{R}^{|\\mathcal{V}|}\n  $$\n\n---\n\n### 2.3 LRNN Dynamics\n\n#### 2.3.1 Adaptive Mixing Coefficient\n\nThe update rate of the liquid state is controlled by an **adaptive mixing coefficient** computed at every timestep:  \n\n$$\n\\alpha_t = \\sigma(W_\\alpha x_t + U_\\alpha h_{t-1} + b_\\alpha), \\quad \\alpha_t \\in (0,1)\n$$\n\nThis coefficient induces a time-varying time constant:  \n\n$$\n\\tau_t = \\frac{1}{\\alpha_t}\n$$\n\nA small $(\\alpha_t$ results in slow state evolution, while a large $(\\alpha_t)$ enables rapid adaptation to changing input dynamics.  \n\n---\n\n#### 2.3.2 Liquid State Update (Fast Time-Scale)\n\nThe liquid state is updated as:  \n\n$$\ns_t = (1 - \\alpha_t) \\odot s_{t-1} + \\alpha_t \\odot f(W_{in} x_t + W_{rec} s_{t-1})\n$$  \n\nwhere:  \n\n* $(f(\\cdot))$ is a bounded nonlinearity such as (\\tanh) or ReLU,  \n* $(W_{in})$ and $(W_{rec})$ are learnable input and recurrent weight matrices,  \n* $(\\odot)$ denotes element-wise multiplication.  \n\nThe adaptive mixing term allows the liquid state to smoothly interpolate between memory retention and rapid updating, based on input context.  \n\n---\n\n#### 2.3.3 Gated Recurrent Update (Slow Time-Scale)\n\nThe stable recurrent hidden state is updated via:  \n\n$$\nh_t = \\sigma(W_h s_t + U_h h_{t-1} + b_h)\n$$\n\nThis gated recurrence stabilizes the fast liquid dynamics and preserves long-term temporal dependencies, preventing chaotic drift while maintaining contextual memory.  \n\n---\n\n### 2.4 Output Emission\n\nFor token or acoustic modeling, the LRNN produces framewise logits:  \n\n$$\nz_t = W_o h_t + b_o\n$$\n\nFramewise probabilities over the vocabulary and blank symbol are obtained using softmax:  \n\n$$\np_t = \\text{softmax}(z_t)\n$$\n\n---\n\n# 2.5 Computational Advantages of the LRNN Framework\n\nThe LRNN architecture provides critical advantages over monolithic recurrent models (LSTMs, GRUs) and attention-based Transformers in the context of neural decoding:    \n\n•\tDynamic Time-Constant Adaptation: Unlike fixed-gated RNNs, LRNN’s $\\alpha_t$ allows the network to adapt its internal time constant $\\tau_t$ in real-time, effectively performing a localized \"time-warping\" that aligns with the variable firing rates of neural populations. \n\n•\tMultiscale Temporal Integration: The explicit bifurcation into $s_t$ (fast-scale) and $h_t$ (slow-scale) enables the model to simultaneously resolve sub-millisecond signal transients and multi-second linguistic contexts without gradient vanishing.  \n\n•\tManifold Smoothness: The continuous-time formulation acts as an implicit regularizer, ensuring that hidden state transitions are topologically smooth. This is essential for alignment-based objectives like AAL, as it prevents the \"spiky\" probability distributions common in discrete-time RNNs.  \n\n•\tParameter Efficiency and Generalization: By leveraging liquid dynamics, the model achieves high expressive power with fewer parameters, significantly reducing the risk of catastrophic forgetting and overfitting on typically sparse brain-computer interface (BCI) datasets.  \n\n\n---\n\n## 3. Adaptive Alignment Loss (AAL)\n\n### 3.1 Motivation\n\nConnectionist Temporal Classification (CTC) is widely used for sequence alignment without frame-level supervision but suffers from several limitations:  \n\n* Assumption of conditional independence across frames.  \n* Heuristic and fixed blank symbol behavior.  \n* Weak handling of repeated tokens.  \n* Lack of context-aware alignment modeling.  \n \n**Adaptive Alignment Loss (AAL)** addresses these issues by learning alignment transitions conditioned on the LRNN hidden state.  \n\n---\n\n### 3.2 Emission Scores\n\nFor each frame $(t)$ and candidate symbol $(a \\in \\mathcal{V} \\cup {\\text{blank}})$, the emission score is defined as:\n\n$$\ne_t(a) = \\log p(a \\mid h_t)\n$$\n\n---\n\n### 3.3 Transition Modeling\n\nAAL introduces learnable, context-dependent transition scores between consecutive alignment symbols:\n\n$$\n\\text{tr}*t(a*{t-1} \\to a_t) = f_\\theta(h_t, \\text{embed}(a_{t-1}), \\text{embed}(a_t))\n$$\n\nwhere:\n\n* $(f_\\theta)$ is a small learnable MLP,  \n* $(\\text{embed}(\\cdot))$ denotes token embeddings,  \n* Blank transitions are explicitly modeled and learned.  \n\nThis formulation allows explicit modeling of repeated tokens and adaptive decisions about staying on blank or advancing the output sequence.  \n\n---\n\n### 3.4 Alignment Score\n\nFor an alignment path:  \n\n$$\nA = (a_1, a_2, \\ldots, a_T)\n$$\n\nthe total alignment score is:  \n\n$$\n\\log S(A) = \\sum_{t=1}^{T} \\left( e_t(a_t) + \\text{tr}*t(a*{t-1} \\to a_t) \\right)\n$$\n\n---\n\n### 3.5 Sequence Probability\n\nLet $(\\mathcal{A}(y))$ denote the set of all valid alignments corresponding to target sequence $(y)$. The conditional probability is:  \n\n$$\nP(y \\mid x) = \\sum_{A \\in \\mathcal{A}(y)} \\exp(\\log S(A))\n$$\n\n---\n\n### 3.6 Adaptive Alignment Loss\n\nThe AAL objective is defined as:  \n\n$$\n\\mathcal{L}_{AAL} = - \\log P(y \\mid x)\n$$\n\nThis formulation strictly generalizes CTC by incorporating learned transition dynamics.  \n\n---\n\n### 3.7 Regularization Terms\n\nOptional but effective regularizers include:  \n\n**Temporal Smoothness**:  \n \n$$\n\\mathcal{R}*{smooth} = \\sum_t | \\text{softmax}(e_t) - \\text{softmax}(e*{t+1}) |^2\n$$\n\n**Repeat Consistency**:  \n\n$$\n\\text{tr}_t(a \\to a) \\leftarrow \\text{tr}*t(a \\to a) - \\gamma \\cdot \\phi(\\Delta t*{last\\ advance})\n$$\n\n**Language Model Prior**:\n\n$$\n\\mathcal{L}*{total} = \\mathcal{L}*{AAL} + \\lambda_{smooth} \\mathcal{R}*{smooth} - \\lambda*{LM} \\log P_{LM}(y)\n$$\n\n---\n\n## 4. LRNN + AAL: Full System Architecture\n\n```\nInput signals x_t  \n      |\n      V\n   LRNN (Liquid + RNN)  \n      |\n      |--- hidden state h_t  \n      V \nFramewise logits z_t  \n      | \n      V\nAAL Module  \n  - Compute emission e_t  \n  - Compute transition tr_t using h_t  \n  - Forward-backward dynamic programming  \n      |\n      V\nLoss: L_AAL  \n      |\nOptimizer updates LRNN and transition MLP parameters  \n```\n\n---\n\n## Training Protocol and Algorithmic Decoding\nTo ensure numerical stability and optimal convergence, the training of the LRNN+AAL system is conducted in a multi-stage pipeline:  \n\n1.\tBootstrap Pre-training: The LRNN backbone is initially optimized using a standard Connectionist Temporal Classification (CTC) loss. This phase stabilizes the recurrent weights and establishes a baseline alignment manifold before introducing learned transitions.\n  \n2.\tJoint AAL Optimization: The transition MLP ($f_\\theta$) is unfrozen, and the system is trained end-to-end using the AAL objective. During this phase, the model learns to refine the blank-symbol duration and token-to-token transition probabilities based on the hidden dynamics $h_t$.    \n3.\tInference via AAL-Augmented Beam Search: Decoding is performed using a modified beam search algorithm. Unlike standard CTC decoding which assumes $P(a_t | a_{t-1}) = \\text{constant}$, our decoder incorporates the learned transition scores $\\text{tr}_t(a_{t-1} \\to a_t)$, allowing the model to dynamically prune unlikely paths based on temporal context.  \n4.\tNeural-Linguistic Integration: An optional language model (LM) rescoring step or joint LM regularization term is applied to enforce phonotactic and semantic constraints on the final output string.  \n\n\n---\n\n# End to End Implementation Pipeline :\n\n---\n\n```\nHDF5 Neural Data (512 × T)\n        ↓\nPreprocessing + Normalization\n        ↓\nLRNN Encoder (Liquid + RNN)\n        ↓\nFramewise Phoneme Logits (T × |V|)\n        ↓\nAAL (Adaptive Alignment Loss)  [TRAIN TIME]\n        ↓\nBeam Search Decoder            [INFERENCE]\n        ↓\n(Optional) LM Rescoring\n        ↓\nFinal Text Sentences\n        ↓\nsubmission.csv (id, text)\n\n```\n\n---\n\n## Comparative Analysis: LRNN+AAL vs. Classical CTC\n\nThe proposed framework addresses the fundamental \"Independence Assumption\" flaw inherent in Connectionist Temporal Classification (CTC) by introducing a context-aware transition geometry. By conditioning transitions on the latent hidden state , AAL effectively mitigates the \"peaky distribution\" problem where traditional CTC models assign disproportionately high probabilities to narrow temporal windows.  \n\n| Feature | CTC (Baseline) | LRNN + AAL (Proposed) |\n| --- | --- | --- |\n| **Frame Dependency** | Conditional Independence | State-Conditioned Transitions |\n| **Blank Modeling** | Heuristic & Static | Adaptive & Learnable |\n| **Temporal Drift** | Highly Sensitive | Robust via Liquid Adaptation |\n| **Repeated Tokens** | Requires Forced Blanks | Naturally Modeled via MLP |\n| **State Dynamics** | Discrete and Rigid | Continuous and Liquid |\n\n### Key Differentiators:\n\n* **Contextual Transitions**: Unlike CTC, which assumes  is constant, AAL uses the LRNN's hidden dynamics to decide if a transition is linguistically or physiologically probable at that specific millisecond.  \n* **Error Reduction**: This architecture specifically targets and reduces insertion, deletion, and repetition errors, leading to significantly lower Word Error Rates (WER) in high-noise brain-computer interface (BCI) environments.  \n* **Adaptive Blank Modeling**: The system dynamically learns how long to \"stay\" on a blank symbol based on the stability of the neural signal, rather than relying on fixed heuristic rules.  \n\n---\n\n## 7. Conclusion\n\nThis document presented a complete, formal, and equation-consistent description of **LRNN** and **AAL**, a co-designed architecture and loss framework for adaptive temporal modeling and alignment. By combining liquid dynamics with recurrent stability and replacing heuristic alignment assumptions with learned transition modeling, the proposed system provides a principled and powerful approach for decoding text from complex neural time-series data.  \n\n# Citations \n\n@misc{brain-to-text-25,\n    author = {Nicholas Card and Maitreyee Wairagkar and Carrina Iacobacci and Xianda Hou and Tyler Singer-Clark and Francis R. Willett and Erin M. Kunz and Chaofei Fan and Maryam Vahdati Nia and Darrel R. Deo and Aparna Srinivasan and Eun Young Choi and Matthew F. Glasser and Leigh R. Hochberg and Jaimie M. Henderson and Kiarash Shahlaie and Sergey D. Stavisky* and David M. Brandman*},\n    title = {Brain-to-text '25},\n    year = {2025},\n    howpublished = {\\url{https://www.kaggle.com/competitions/brain-to-text-25}},\n    note = {Kaggle Competition}\n}\n\n---\n\n@article{hasani2021liquid,\n    author = {Ramin Hasani and Mathias Lechner and Alexander Amini and Daniela Rus and Radu Grosu},\n    title = {Liquid Time-constant Networks},\n    journal = {AAAI Conference on Artificial Intelligence},\n    volume = {35},\n    number = {9},\n    pages = {7657-7666},\n    year = {2021},\n    url = {https://ojs.aaai.org/index.php/AAAI/article/view/16936}\n}\n\n---\n\n@article{lechner2020neural,\n    author = {Mathias Lechner and Ramin Hasani and Daniela Rus and Radu Grosu},\n    title = {Neural circuit policies enabling self-driving cars},\n    journal = {Nature Machine Intelligence},\n    volume = {2},\n    pages = {642–652},\n    year = {2020},\n    publisher = {Nature Publishing Group},\n    doi = {10.1038/s42256-020-00237-3}\n}\n\n---\n\n@inproceedings{hasani2022closed,\n    author = {Ramin Hasani and Mathias Lechner and Alexander Amini and Lucas Liebenwein and Aaron Ray and Max Tschaikowski and Gerald Teschl and Daniela Rus},\n    title = {Closed-form Continuous-time Neural Networks},\n    booktitle = {Nature Machine Intelligence},\n    year = {2022},\n    url = {https://www.nature.com/articles/s42256-022-00556-7}\n}","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}}]}