Firstly, huge thanks to Kaggle and Jane Street for organizing such an interesting competition. Also, a very sincere thanks to the Kaggle community; the discussions here helped me a lot to understand mistakes and I learned a lot.
\nThe solution is a hybrid model, combining an adaptive neural network with a static GBDT ensemble to address the non-stationary nature of the financial time-series data.
\nThe final prediction is a linear blend of two models:
\n\ny_final = ( * y_tabm_pred) + ( * y_lgb_pred)\ny_final = np.clip(y_final, -, )\nThis model was designed for continuous adaptation to new market data.
\n1e-5.This model, provided by my teammate, served as a stable baseline.
\nresponder values from the previous day.This is my first write-up, please correct me if I wrote something wrong.
", "messages": [ { "id": "3249104", "postDate": "07/15/2025 17:49:35", "content": "Firstly, huge thanks to Kaggle and Jane Street for organizing such an interesting competition. Also, a very sincere thanks to the Kaggle community; the discussions here helped me a lot to understand mistakes and I learned a lot.
\nThe solution is a hybrid model, combining an adaptive neural network with a static GBDT ensemble to address the non-stationary nature of the financial time-series data.
\nThe final prediction is a linear blend of two models:
\n\ny_final = ( * y_tabm_pred) + ( * y_lgb_pred)\ny_final = np.clip(y_final, -, )\nThis model was designed for continuous adaptation to new market data.
\n1e-5.This model, provided by my teammate, served as a stable baseline.
\nresponder values from the previous day.This is my first write-up, please correct me if I wrote something wrong.
", "rawMarkdown": "Firstly, huge thanks to Kaggle and Jane Street for organizing such an interesting competition. Also, a very sincere thanks to the Kaggle community; the discussions here helped me a lot to understand mistakes and I learned a lot.\n\n#### **1. Overview**\n\nThe solution is a hybrid model, combining an adaptive neural network with a static GBDT ensemble to address the non-stationary nature of the financial time-series data.\n\n#### **2. Final Architecture: Weighted Ensemble**\n\nThe final prediction is a linear blend of two models:\n\n* **Model A (70% weight):** An online learning TabM neural network (my contribution).\n* **Model B (30% weight):** A static LightGBM ensemble (my teammate's contribution).\n\n```python\n# Final prediction logic\ny_final = (0.7 * y_tabm_pred) + (0.3 * y_lgb_pred)\ny_final = np.clip(y_final, -5, 5)\n```\n\n---\n\n#### **3. Model A: Online Learning TabM (PyTorch)**\n\nThis model was designed for continuous adaptation to new market data.\n\n* **Architecture:**\n * TabM-based MLP (3 hidden layers, 512 units, ReLU activation).\n * Data I/O and processing were handled with the Polars library for performance.\n* **Online Learning Mechanism:**\n * The model was continuously fine-tuned on a rolling window of the most recent 3-4 days of data.\n * A retraining cycle was triggered daily after new data became available.\n * Fine-tuning was performed for 10 epochs using an AdamW optimizer with a learning rate of `1e-5`.\n * A smaller learning rate and mixing old data helped avoid catastrophic forgetting.\n\n---\n\n#### **4. Model B: Static LightGBM Ensemble (Teammate Contribution)**\n\nThis model, provided by my teammate, served as a stable baseline.\n\n* **Architecture:** An ensemble of multiple LightGBM models, with predictions averaged.\n* **Features:** Utilized the 79 base features plus 9 lagged `responder` values from the previous day.\n* **Purpose:** Trained once on a large historical dataset to capture long-term, stable patterns in the data.\n\n---\n\n#### **5. Validation Strategy**\n\n* During training, I was not able to find a good LB/CV correlation. I kept the last 100 days for validation, then trained on the whole dataset again for submission.\n* A local simulation of the Kaggle API was used to ensure robustness and avoid timeouts, based on this [reference notebook](https://www.kaggle.com/code/chumajin/janestreet-updated-simulator-for-time-series-api).\n * This allowed for end-to-end local testing of the entire inference and retraining pipeline. It was critical for debugging and validating the online learning strategy before submission.\n\n*This is my first write-up, please correct me if I wrote something wrong.*", "votes": null }, { "id": "3249318", "postDate": "07/16/2025 08:32:59", "content": "Nice, thank you for posting.
\nWhat was your approach for data pre-processing?
", "rawMarkdown": "Nice, thank you for posting.\n\nWhat was your approach for data pre-processing?", "votes": null }, { "id": "3249627", "postDate": "07/16/2025 19:36:04", "content": "Thanks for the question!
\nAs such no pre-processing was done, just fill null with 3 , got this from discussions and with our experiments as well this fill was working good, we tried [-5,5].
\nwe tried normalizing the data as shown in some public notebooks , but this model was performing good on local tests but in LB it was not performing.
Nice, thank you for posting.
\nWhat was your approach for data pre-processing?
", "votes": null, "replies": [ { "id": 3249627, "author_name": "zeroday19", "author_url": "", "post_date": "07/16/2025 19:36:04", "content": "Thanks for the question!
\nAs such no pre-processing was done, just fill null with 3 , got this from discussions and with our experiments as well this fill was working good, we tried [-5,5].
\nwe tried normalizing the data as shown in some public notebooks , but this model was performing good on local tests but in LB it was not performing.