![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2112326%2Fffd820b539d7dcc6c4dd644082219c8c%2FCleanShot%202026-09-22%20at%203.59.07%20PM2x.png?generation=1790117983863541&alt=media\")
I wanted to better understand Patrick Yam's 2nd place solution to this competition, but I couldn't find any code that reproduced it. However I did find his winning solution presentation video recording and slides, and used them along with Codex and GPT 6 Astra to produce the following repo with reproduction code: https://github.com/cweill/jane-street-causal-forecasting
\nMy best local replay score, with online learning enabled, is 0.01942, compared with Patrick’s reported validation score of 0.02059. Note that I haven’t confirmed the exact reproduction details with him:
Furthermore, I was able to observe a similar pattern of improvement in eval from online learning versus a using frozen model:
\nHis: ![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2112326%2F13c4ab18dd7d2fb5f5c45e557bd693ba%2FCleanShot%202026-09-22%20at%204.01.23%20PM2x.png?generation=1790118099409411&alt=media\")
Ours: \n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2112326%2F1940988bd2f9bcda0ae97acc37fa7561%2FCleanShot%202026-09-22%20at%204.00.01%20PM2x.png?generation=1790118081414654&alt=media\")
Our replay also shows a sustained benefit from online learning, although the gap is smaller than in Patrick’s chart. Our frozen model remains stronger late in the replay, and the peak near date 1500 is larger. I’m still investigating which training or online-update choices explain these differences.
\n Configuration Local weighted zero-mean R²\n ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Frozen 17-model ensemble 0.01413\n ───────────────────────────────────────────── ─────────────────────────────\n Same ensemble with online learning, LR 1e-4 0.01942\n ───────────────────────────────────────────── ─────────────────────────────\n Absolute improvement +0.00529\n\nNOTE: offline training through date 1379; replay from 1380; scoring on dates 1500–1698.\nIf Patrick or anyone who reproduced this solution spots differences in the implementation - particularly the online learning rate, optimizer-state handling, or training duration - I’d appreciate the feedback.
", "messages": [ { "id": 3527086, "postDate": "2026-09-22T23:04:12.503Z", "content": "I wanted to better understand Patrick Yam's 2nd place solution to this competition, but I couldn't find any code that reproduced it. However I did find his winning solution presentation video recording and slides, and used them along with Codex and GPT 6 Astra to produce the following repo with reproduction code: https://github.com/cweill/jane-street-causal-forecasting
\nMy best local replay score, with online learning enabled, is 0.01942, compared with Patrick’s reported validation score of 0.02059. Note that I haven’t confirmed the exact reproduction details with him:
Furthermore, I was able to observe a similar pattern of improvement in eval from online learning versus a using frozen model:
\nHis:
Ours: \n
Our replay also shows a sustained benefit from online learning, although the gap is smaller than in Patrick’s chart. Our frozen model remains stronger late in the replay, and the peak near date 1500 is larger. I’m still investigating which training or online-update choices explain these differences.
\n Configuration Local weighted zero-mean R²\n ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Frozen 17-model ensemble 0.01413\n ───────────────────────────────────────────── ─────────────────────────────\n Same ensemble with online learning, LR 1e-4 0.01942\n ───────────────────────────────────────────── ─────────────────────────────\n Absolute improvement +0.00529\n\nNOTE: offline training through date 1379; replay from 1380; scoring on dates 1500–1698.\nIf Patrick or anyone who reproduced this solution spots differences in the implementation - particularly the online learning rate, optimizer-state handling, or training duration - I’d appreciate the feedback.
", "rawMarkdown": "I wanted to better understand [Patrick Yam's](https://www.kaggle.com/wimwim) 2nd place solution to this competition, but I couldn't find any code that reproduced it. However I did find his winning solution presentation [video recording](https://www.youtube.com/watch?v=lfzzPZZyzjE) and [slides](https://drive.google.com/file/d/1AdzXrOZN699SGHAyb32slLBkEMxBCYiJ/view), and used them along with Codex and GPT 6 Astra to produce the following repo with reproduction code: https://github.com/cweill/jane-street-causal-forecasting\n\nMy best local replay score, with online learning enabled, is 0.01942, compared with Patrick’s reported validation score of 0.02059. Note that I haven’t confirmed the exact reproduction details with him: \n\nFurthermore, I was able to observe a similar pattern of improvement in eval from online learning versus a using frozen model:\n\nHis: \n\nOurs: \n\n\nOur replay also shows a sustained benefit from online learning, although the gap is smaller than in Patrick’s chart. Our frozen model remains stronger late in the replay, and the peak near date 1500 is larger. I’m still investigating which training or online-update choices explain these differences. \n\n```md\n Configuration Local weighted zero-mean R²\n ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Frozen 17-model ensemble 0.01413\n ───────────────────────────────────────────── ─────────────────────────────\n Same ensemble with online learning, LR 1e-4 0.01942\n ───────────────────────────────────────────── ─────────────────────────────\n Absolute improvement +0.00529\n\nNOTE: offline training through date 1379; replay from 1380; scoring on dates 1500–1698.\n```\n\nIf Patrick or anyone who reproduced this solution spots differences in the implementation - particularly the online learning rate, optimizer-state handling, or training duration - I’d appreciate the feedback.", "votes": 1 } ], "comments": [], "raw_markdown_by_id": { "3527086": "I wanted to better understand [Patrick Yam's](https://www.kaggle.com/wimwim) 2nd place solution to this competition, but I couldn't find any code that reproduced it. However I did find his winning solution presentation [video recording](https://www.youtube.com/watch?v=lfzzPZZyzjE) and [slides](https://drive.google.com/file/d/1AdzXrOZN699SGHAyb32slLBkEMxBCYiJ/view), and used them along with Codex and GPT 6 Astra to produce the following repo with reproduction code: https://github.com/cweill/jane-street-causal-forecasting\n\nMy best local replay score, with online learning enabled, is 0.01942, compared with Patrick’s reported validation score of 0.02059. Note that I haven’t confirmed the exact reproduction details with him: \n\nFurthermore, I was able to observe a similar pattern of improvement in eval from online learning versus a using frozen model:\n\nHis: \n\nOurs: \n\n\nOur replay also shows a sustained benefit from online learning, although the gap is smaller than in Patrick’s chart. Our frozen model remains stronger late in the replay, and the peak near date 1500 is larger. I’m still investigating which training or online-update choices explain these differences. \n\n```md\n Configuration Local weighted zero-mean R²\n ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Frozen 17-model ensemble 0.01413\n ───────────────────────────────────────────── ─────────────────────────────\n Same ensemble with online learning, LR 1e-4 0.01942\n ───────────────────────────────────────────── ─────────────────────────────\n Absolute improvement +0.00529\n\nNOTE: offline training through date 1379; replay from 1380; scoring on dates 1500–1698.\n```\n\nIf Patrick or anyone who reproduced this solution spots differences in the implementation - particularly the online learning rate, optimizer-state handling, or training duration - I’d appreciate the feedback." } }