Thank you hosts for this amazing competition.
\nTo be a little help, we briefly share the strategies we used below.
\n▪ Seq. < 400
\n • drfold2\n • Protenix finetuned RNA MSA ()\n▪ Seq. >= 400
\n • Protenix finetuned RNA MSA ()\n • Protenix finetuned MSA ()\n • Protenix baseline model\nBelow 800 because OOM on 96GB VRAM: GH200.
\nRNA only: polymer composition RNA
\nALL: v2 data + recent pdb release
\nWe included Casp 16 data so that bumps up the public score.
\nIf we don't use the casp 16 data, we get about 0.42.
rMSA pipeline MSA data (used the codes hosts provided):
\nhttps://drive.google.com/drive/folders/15bhXWsR6QuDQo6U4j8Ii8-OE4B4SmC7i?usp=drive_link
Training Data (for protenix finetuning) and labels:
\nhttps://drive.google.com/drive/folders/1XKYzk2oCcHPt6DB_wLNYL-7w3s1R793n?usp=sharing
Codes used from :
\nProtenix inference
\nhttps://www.kaggle.com/code/geraseva/protenix
\n@geraseva
Protenix finetuning
\nhttps://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/573495
\n@lihaoweicvch
Boltz inference
\nhttps://www.kaggle.com/code/youhanlee/boltz-1-inference-submission
\n@youhanlee
Thank you hosts for this amazing competition.
\nTo be a little help, we briefly share the strategies we used below.
\n▪ Seq. < 400
\n • drfold2\n • Protenix finetuned RNA MSA ()\n▪ Seq. >= 400
\n • Protenix finetuned RNA MSA ()\n • Protenix finetuned MSA ()\n • Protenix baseline model\nBelow 800 because OOM on 96GB VRAM: GH200.
\nRNA only: polymer composition RNA
\nALL: v2 data + recent pdb release
\nWe included Casp 16 data so that bumps up the public score.
\nIf we don't use the casp 16 data, we get about 0.42.
rMSA pipeline MSA data (used the codes hosts provided):
\nhttps://drive.google.com/drive/folders/15bhXWsR6QuDQo6U4j8Ii8-OE4B4SmC7i?usp=drive_link
Training Data (for protenix finetuning) and labels:
\nhttps://drive.google.com/drive/folders/1XKYzk2oCcHPt6DB_wLNYL-7w3s1R793n?usp=sharing
Codes used from :
\nProtenix inference
\nhttps://www.kaggle.com/code/geraseva/protenix
\n@geraseva
Protenix finetuning
\nhttps://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/573495
\n@lihaoweicvch
Boltz inference
\nhttps://www.kaggle.com/code/youhanlee/boltz-1-inference-submission
\n@youhanlee
Very helpful, your solution is amazing! thx
", "rawMarkdown": "Very helpful, your solution is amazing! thx", "votes": null }, { "id": "3225269", "postDate": "06/16/2025 07:54:16", "content": "Thank you for sharing, but could your fine-tuning strategy potentially lead to excessive leakage? It's possible that only 3 from DrFold2 and 1 from the ProteinX baseline model actually works, hard to say, live or die?
", "rawMarkdown": "Thank you for sharing, but could your fine-tuning strategy potentially lead to excessive leakage? It's possible that only **3 from DrFold2** and **1 from the ProteinX baseline model** actually works, hard to say, live or die?", "votes": null }, { "id": "3225312", "postDate": "06/16/2025 08:40:30", "content": "Well, we did try fine-tuning without the leakage data and it still improved.
\nBaseline model with no fine-tuning had a public test score of 3.7,
\nwhile fine-tuned model with v1 data + MSA had a public test score of 4.12.
\nSo this suggests that fine-tuning with MSAs does work, so we used all the available data for training (more data the better..?)
One thing to notice was if we fine-tune just 1-2 epoch (1 or 2 full iterations of the training dataset) the performance dropped (possibility due to the optimizer as @shujun717 suggested).
\nSo we had to train for more than 5 epochs for performance boost.
Meanwhile we did work on 2 strategies but the drfold2 one didn't go through for submissions because we didn't select that one :(
", "rawMarkdown": "Well, we did try fine-tuning without the leakage data and it still improved.\n\nBaseline model with no fine-tuning had a public test score of 3.7,\nwhile fine-tuned model with v1 data + MSA had a public test score of 4.12.\nSo this suggests that fine-tuning with MSAs does work, so we used all the available data for training (more data the better..?)\n\nOne thing to notice was if we fine-tune just 1-2 epoch (1 or 2 full iterations of the training dataset) the performance dropped (possibility due to the optimizer as @shujun717 suggested). \nSo we had to train for more than 5 epochs for performance boost.\n\nMeanwhile we did work on 2 strategies but the drfold2 one didn't go through for submissions because we didn't select that one :(", "votes": null }, { "id": "3226043", "postDate": "06/17/2025 07:22:59", "content": "Very helpful work
", "rawMarkdown": "Very helpful work", "votes": null } ], "comments": [ { "id": 3223918, "author_name": "", "author_url": "", "post_date": "06/14/2025 05:46:25", "content": "Very helpful, your solution is amazing! thx
", "votes": null, "replies": [] }, { "id": 3225269, "author_name": "sweetyheehee", "author_url": "", "post_date": "06/16/2025 07:54:16", "content": "Thank you for sharing, but could your fine-tuning strategy potentially lead to excessive leakage? It's possible that only 3 from DrFold2 and 1 from the ProteinX baseline model actually works, hard to say, live or die?
", "votes": null, "replies": [ { "id": 3225312, "author_name": "yekim102", "author_url": "", "post_date": "06/16/2025 08:40:30", "content": "Well, we did try fine-tuning without the leakage data and it still improved.
\nBaseline model with no fine-tuning had a public test score of 3.7,
\nwhile fine-tuned model with v1 data + MSA had a public test score of 4.12.
\nSo this suggests that fine-tuning with MSAs does work, so we used all the available data for training (more data the better..?)
One thing to notice was if we fine-tune just 1-2 epoch (1 or 2 full iterations of the training dataset) the performance dropped (possibility due to the optimizer as @shujun717 suggested).
\nSo we had to train for more than 5 epochs for performance boost.
Meanwhile we did work on 2 strategies but the drfold2 one didn't go through for submissions because we didn't select that one :(
", "votes": null, "replies": [] } ] }, { "id": 3226043, "author_name": "starlitdreamstar", "author_url": "", "post_date": "06/17/2025 07:22:59", "content": "Very helpful work
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3223690": "Thank you hosts for this amazing competition.\n\nTo be a little help, we briefly share the strategies we used below.\n\n1. Drfold2 + Protenix (Public 0.589)\n\n▪ Seq. < 400\n\n • 3 from drfold2\n • 2 from Protenix fine-tuned RNA only + MSA (<800)\n\n▪ Seq. >= 400\n\n • 2 from Protenix fine-tuned RNA only + MSA (<800)\n • 2 from Protenix fine-tuned All + MSA (<800)\n • 1 from Protenix baseline model\n\n2. Protenix + Boltz (Public 0.615)\n▪ 2 from Protenix fine-tuned RNA only + MSA (<800)\n▪ 1 from Protenix fine-tuned ALL + MSA (<800)\n▪ 1 from Protenix baseline\n▪ 1 from Boltz baseline\n\nBelow 800 because OOM on 96GB VRAM: GH200.\n\nRNA only: polymer composition RNA\nALL: v2 data + recent pdb release\nWe included Casp 16 data so that bumps up the public score.\nIf we don't use the casp 16 data, we get about 0.42.\n\nrMSA pipeline MSA data (used the codes hosts provided):\nhttps://drive.google.com/drive/folders/15bhXWsR6QuDQo6U4j8Ii8-OE4B4SmC7i?usp=drive_link\n\nTraining Data (for protenix finetuning) and labels:\nhttps://drive.google.com/drive/folders/1XKYzk2oCcHPt6DB_wLNYL-7w3s1R793n?usp=sharing\n\n\nCodes used from :\nProtenix inference\nhttps://www.kaggle.com/code/geraseva/protenix\n@geraseva \n\nProtenix finetuning\nhttps://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/573495\n@lihaoweicvch \n\nBoltz inference\nhttps://www.kaggle.com/code/youhanlee/boltz-1-inference-submission\n@youhanlee", "3223918": "Very helpful, your solution is amazing! thx", "3225269": "Thank you for sharing, but could your fine-tuning strategy potentially lead to excessive leakage? It's possible that only **3 from DrFold2** and **1 from the ProteinX baseline model** actually works, hard to say, live or die?", "3225312": "Well, we did try fine-tuning without the leakage data and it still improved.\n\nBaseline model with no fine-tuning had a public test score of 3.7,\nwhile fine-tuned model with v1 data + MSA had a public test score of 4.12.\nSo this suggests that fine-tuning with MSAs does work, so we used all the available data for training (more data the better..?)\n\nOne thing to notice was if we fine-tune just 1-2 epoch (1 or 2 full iterations of the training dataset) the performance dropped (possibility due to the optimizer as @shujun717 suggested). \nSo we had to train for more than 5 epochs for performance boost.\n\nMeanwhile we did work on 2 strategies but the drfold2 one didn't go through for submissions because we didn't select that one :(", "3226043": "Very helpful work" }, "source": "meta" }