First of all, huge thanks to the organizers and Kaggle for hosting this competition! It’s been a great experience working on the challenge and having such active discussions with everyone.
\nOur work is based on AlphaFold 3 [1]. The final model adopts a single-model deep learning based approach. Basically, we leverage the RNA foundation model AIDO.RNA [3], integrate its representations into Protenix [2] , and finetune the model on the RNA3DB database [5].
\nThings that didn't work
\ncfg_99 was the best checkpoint for us.First of all, huge thanks to the organizers and Kaggle for hosting this competition! It’s been a great experience working on the challenge and having such active discussions with everyone.
\nOur work is based on AlphaFold 3 [1]. The final model adopts a single-model deep learning based approach. Basically, we leverage the RNA foundation model AIDO.RNA [3], integrate its representations into Protenix [2] , and finetune the model on the RNA3DB database [5].
\nThings that didn't work
\ncfg_99 was the best checkpoint for us.Features embeddings from AIDO.RNA is huge work!
", "rawMarkdown": "Features embeddings from AIDO.RNA is huge work!", "votes": null }, { "id": "3296107", "postDate": "09/30/2025 07:35:20", "content": "Not sure I got you right. But I implemented the code to generate embeddings from AIDO.RNA on the fly, rather than precomputing and storing them.
", "rawMarkdown": "Not sure I got you right. But I implemented the code to generate embeddings from AIDO.RNA on the fly, rather than precomputing and storing them.", "votes": null } ], "comments": [ { "id": 3295775, "author_name": "sweetyheehee", "author_url": "", "post_date": "09/29/2025 14:17:49", "content": "Features embeddings from AIDO.RNA is huge work!
", "votes": null, "replies": [ { "id": 3296107, "author_name": "zoushuxian", "author_url": "", "post_date": "09/30/2025 07:35:20", "content": "Not sure I got you right. But I implemented the code to generate embeddings from AIDO.RNA on the fly, rather than precomputing and storing them.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3295583": "First of all, huge thanks to the organizers and Kaggle for hosting this competition! It’s been a great experience working on the challenge and having such active discussions with everyone.\n\n# Our solution\n## Solution overview\nOur work is based on AlphaFold 3 [1]. The final model adopts a single-model deep learning based approach. Basically, we leverage the RNA foundation model **AIDO.RNA** [3], integrate its representations into **Protenix** [2] , and finetune the model on the **RNA3DB** database [5].\n\n## Detailed solution\n- We augment Protenix with embeddings from AIDO.RNA, a language model pretrained on 42 millions of non-coding RNA sequences.\n - We extract the output hidden states from [AIDO.RNA-650M](https://huggingface.co/genbio-ai/AIDO.RNA-650M) as embeddings using [AIDO.ModelGenerator](https://github.com/genbio-ai/ModelGenerator) [4];\n - We then project the embeddings to the space of single representation of Protenix using a linear transformation.\n- We finetune the model with AIDO.RNA frozen on RNA3DB, a well-curated non-redundant RNA 3D structure database with sequence-based clusters. \n - we use all data in the 2024-12-04 RNA3DB release\n - lr=5e-4, warmup_steps=200, max_steps=10,000, train_crop_size=640, global_batch_size=16\n - exponential moving average (EMA) of model weights with decay rate 0.999\n - no MSAs during training\n- For inference, we use the EMA checkpoint saved at the 1600 training step and the default inference setting in Protenix\n - seed=101, n_cycle=10, n_sample=5, n_step=200\n - we use MSAs during inference\n\n## Peformance overview\n- Public LB: 0.42849 -> ranked 12th\n- Private LB: 0.49758 -> ranked 6th\n\n***\n\n# Takeaways\n\n\n**Things that didn't work**\n- Ensembling: Finetuned Protenix for sequences > 350 nucleotides, DRFold2 [6] for sequences <= 350 nucleotides\n - DRFold2 `cfg_99` was the best checkpoint for us.\n - DRFold2 custom C1 position estimation performed better compared to custom C1 estimation based on P, C4, and N1/N9 atom coordinates.\n - More DRFold2 cycles, 12 seemed to be the best.\n- Generate 20 candidate structures, select one as the reference, and use USalign to align the remaining candidates to the reference. Then, calculate the average coordinates.\n- We trained a ranker based on pairwise dRMAE and TM-scores between structures. Use map@5 as evaluation metric. LGBM ranker did not produce any significant boost, OOF map@5 = 0.7x for 20 candidates.\n\n\n# Reference\n1. Accurate structure prediction of biomolecular interactions with AlphaFold 3. *Google DeepMind. Nature, 2024.* \n2. Protenix-advancing structure prediction through a comprehensive AlphaFold3 reproduction. *ByteDance AML AI4Science Team. bioRxiv, 2025.* \n3. [A large-scale foundation model for rna function and structure prediction. ](https://www.biorxiv.org/content/10.1101/2024.11.28.625345v1)*Zou et al. bioRxiv, 2024.* [[Hugging Face Models]](https://huggingface.co/collections/genbio-ai/aidorna-6747516bb48ed96c847f5dd8)\n4. [Rapid and Reproducible Multimodal Biological Foundation Model Development with AIDO.ModelGenerator. ](https://www.biorxiv.org/content/10.1101/2025.06.30.662437v1) *Caleb et al. bioRxiv, 2025.* \n5. RNA3DB: A structurally-dissimilar dataset split for training and benchmarking deep learning models for RNA structure prediction. *Szikszai et al. bioRxiv, 2024.* \n6. Ab initio RNA structure prediction with composite language model and denoised end-to-end learning. *Li et al. bioRxiv, 2025.*", "3295775": "Features embeddings from AIDO.RNA is huge work!", "3296107": "Not sure I got you right. But I implemented the code to generate embeddings from AIDO.RNA on the fly, rather than precomputing and storing them." }, "source": "meta" }