![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe472f8b074b82f9936fb925d1f2fa88f%2FSelection_162.png?generation=1743218349778687&alt=media\")
As is discussed extensively already in these forums, AlpahFold 3 should provide a powerful baseline for our RNA structure prediction competition.
\nUnfortunately, due to licensing restrictions, you cannot run AlphaFold3 in your notebooks.
\nStill, as hosts, we can run AlphaFold on the leaderboard targets and provide scores, and @alissahummer has done this!
\nYou can now find AF3_noMSA and AF3_rMSA scores on the leaderboard, with scores of 0.259 (no MSA) and 0.397 (with alignments generated with rMSA, the pipeline we are using for default MSA's in this competition).
Good luck as you hill-climb towards and past these baselines!
", "messages": [ { "id": "3160619", "postDate": "03/27/2025 00:58:53", "content": "As is discussed extensively already in these forums, AlpahFold 3 should provide a powerful baseline for our RNA structure prediction competition.
\nUnfortunately, due to licensing restrictions, you cannot run AlphaFold3 in your notebooks.
\nStill, as hosts, we can run AlphaFold on the leaderboard targets and provide scores, and @alissahummer has done this!
\nYou can now find AF3_noMSA and AF3_rMSA scores on the leaderboard, with scores of 0.259 (no MSA) and 0.397 (with alignments generated with rMSA, the pipeline we are using for default MSA's in this competition).
Good luck as you hill-climb towards and past these baselines!
", "rawMarkdown": "As is discussed extensively already in these forums, AlpahFold 3 should provide a powerful baseline for our RNA structure prediction competition.\n\nUnfortunately, due to licensing restrictions, you cannot run AlphaFold3 in your notebooks.\n\nStill, as hosts, we can run AlphaFold on the leaderboard targets and provide scores, and @alissahummer has done this!\n\nYou can now find `AF3_noMSA` and `AF3_rMSA` scores on the [leaderboard](https://www.kaggle.com/competitions/stanford-rna-3d-folding/leaderboard), with scores of 0.259 (no MSA) and 0.397 (with alignments generated with rMSA, the pipeline we are using for default MSA's in this competition). \n\nGood luck as you hill-climb towards and past these baselines!", "votes": null }, { "id": "3160623", "postDate": "03/27/2025 01:13:11", "content": "Thanks for providing such a strong baseline. May I ask if we are licensed to use Protenix (a AF3 rework) in this competition?
", "rawMarkdown": "Thanks for providing such a strong baseline. May I ask if we are licensed to use Protenix (a AF3 rework) in this competition?", "votes": null }, { "id": "3160676", "postDate": "03/27/2025 03:20:42", "content": "i aslo share out team submission (public test casp16)
\nproteinX (alpha3 clone)
\nnoMAS = lb 0.327 (float16 kaggle notebook 2x t4), 0.328 (bfloat16 offline)
also, if you pull results from CASP16 server, alphafold3 server has lb0.44+
\nso the submission to casp16 server still have better msa?
\n", "rawMarkdown": ">The Protenix project, including code and model parameters, is made available under the Apache 2.0 License, it is free for both academic research and commercial use.", "votes": null }, { "id": "3160743", "postDate": "03/27/2025 05:31:21", "content": "The Protenix project, including code and model parameters, is made available under the Apache 2.0 License, it is free for both academic research and commercial use.
\n
In my understanding, the use of the AlphaFold model requires permission from DeepMind, so it is not accessible to everyone. However, Protenix does not have this restriction—it is open-source and available to all. Many SOTA models have similar limitations as AlphaFold, which disqualifies them from meeting the criteria I think.
\n\n", "rawMarkdown": "In my understanding, the use of the AlphaFold model requires permission from DeepMind, so it is not accessible to everyone. However, Protenix does not have this restriction—it is open-source and available to all. Many SOTA models have similar limitations as AlphaFold, which disqualifies them from meeting the criteria I think.\n\n>Obtaining Model Parameters\nThis repository contains all necessary code for AlphaFold 3 inference. To request access to the AlphaFold 3 model parameters, please complete this form. Access will be granted at Google DeepMind’s sole discretion. We will aim to respond to requests within 2–3 business days. You may only use AlphaFold 3 model parameters if received directly from Google. Use is subject to these terms of use.", "votes": null }, { "id": "3160747", "postDate": "03/27/2025 05:34:53", "content": "Obtaining Model Parameters
\n
\n This repository contains all necessary code for AlphaFold 3 inference. To request access to the AlphaFold 3 model parameters, please complete this form. Access will be granted at Google DeepMind’s sole discretion. We will aim to respond to requests within 2–3 business days. You may only use AlphaFold 3 model parameters if received directly from Google. Use is subject to these terms of use.
Sigh, the game is too hard.
", "rawMarkdown": "Sigh, the game is too hard.", "votes": null }, { "id": "3160748", "postDate": "03/27/2025 05:40:51", "content": "The AlphaFold 3 model parameters and output are only available for non-commercial use by, or on behalf of non-commercial organizations (i.e., universities, non-profit organizations and research institutes, educational, journalism and government bodies). If you are a researcher affiliated with a non-commercial organization, provided you are not a commercial organization or acting on behalf of a commercial organization, this means you can use it for your non-commercial affiliated research.
\nYou must not use nor allow others to use:
\nAlphaFold 3 model parameters or output in connection with any commercial activities, including research on behalf of commercial organizations; or
\nAlphaFold 3 output to train machine learning models or related technology for biomolecular structure prediction similar to AlphaFold 3.
\nYou must not publish or share AlphaFold 3 model parameters, except sharing these within your organization in accordance with the Terms.
\nYou can publish, share and adapt AlphaFold 3 output in accordance with the Terms, including the requirements to provide clear notice of any modifications you make and that ongoing use of AlphaFold 3 output and derivatives are subject to the AlphaFold 3 Output Terms of Use.
", "rawMarkdown": "The AlphaFold 3 model parameters and output are only available for non-commercial use by, or on behalf of non-commercial organizations (i.e., universities, non-profit organizations and research institutes, educational, journalism and government bodies). If you are a researcher affiliated with a non-commercial organization, provided you are not a commercial organization or acting on behalf of a commercial organization, this means you can use it for your non-commercial affiliated research.\n\nYou must not use nor allow others to use:\n\nAlphaFold 3 model parameters or output in connection with any commercial activities, including research on behalf of commercial organizations; or\n\nAlphaFold 3 output to train machine learning models or related technology for biomolecular structure prediction similar to AlphaFold 3. \n\nYou must not publish or share AlphaFold 3 model parameters, except sharing these within your organization in accordance with the Terms. \n\nYou can publish, share and adapt AlphaFold 3 output in accordance with the Terms, including the requirements to provide clear notice of any modifications you make and that ongoing use of AlphaFold 3 output and derivatives are subject to the AlphaFold 3 Output Terms of Use.", "votes": null }, { "id": "3160752", "postDate": "03/27/2025 05:42:59", "content": "There are other clone like boltz and chai-1. If any other team are using these and can share their results, we can save time and avoid spending time to setup all
", "rawMarkdown": "There are other clone like boltz and chai-1. If any other team are using these and can share their results, we can save time and avoid spending time to setup all", "votes": null }, { "id": "3160753", "postDate": "03/27/2025 05:50:53", "content": "Does ProteinX support long sequence processing on Kaggle? I've encountered OOM issues with RhoFold and NuFold. How did you manage to handle this?
\nBTW, authors of Nufold shared their MSA in CASP15. I will write a topic in discussion later.
\nhttps://kiharalab.org/nufold/CASP15_MSA/
@hengck23 Is it an offline submission or using kaggle GPU?
", "rawMarkdown": "hengck23 Is it an offline submission or using kaggle GPU?", "votes": null }, { "id": "3160775", "postDate": "03/27/2025 06:20:59", "content": "@biancochiu
\nfyi: trROSEttaRNA CASP15 msa files:
\n(BLASTN and Infernal on the RNAcentral ?)
\nhttps://yanglab.qd.sdu.edu.cn/trRosettaRNA/benchmark/
Validation scores from chai-1 server: 0.382 without MSAs / 0.342 (???) MMseqs2 MSAs
", "rawMarkdown": "Validation scores from chai-1 server: 0.382 without MSAs / 0.342 (???) MMseqs2 MSAs", "votes": null }, { "id": "3160810", "postDate": "03/27/2025 07:06:16", "content": "@ogurtsov thanks for the feedback. how did you get 0.382 for CASP16? (this is not shown in the leaderboard) or 0.382 is for CASP15?
", "rawMarkdown": "ogurtsov thanks for the feedback. how did you get 0.382 for CASP16? (this is not shown in the leaderboard) or 0.382 is for CASP15?", "votes": null }, { "id": "3160822", "postDate": "03/27/2025 07:21:56", "content": "@hengck23 it's for validation RNAs (CASP15). The score is surprisingly low.
", "rawMarkdown": "hengck23 it's for validation RNAs (CASP15). The score is surprisingly low.", "votes": null }, { "id": "3160824", "postDate": "03/27/2025 07:23:22", "content": "https://www.kaggle.com/datasets/ogurtsov/chai1-preds
\nhttps://www.kaggle.com/datasets/ogurtsov/chai-preds-msa-mmseqs2
\nI have made prediction datasets public
\n\nso the submission to casp16 server still have better msa?
\n
I don't think that's the right way to think about it. Remember, in CASP competitions there are no training and test datasets that are distinct in terms of sequence. Whatever needs to be predicted is a test dataset, and predictors can use literally whatever they want for training their predictors. If their predictors have already been trained on a sequence that is similar to CASP targets, they will naturally do better.
\nAlso, CASP targets are not necessarily chosen with any particular sequence length or difficulty level in mind. Whatever RNAs are currently in the pipeline for structure determination will be included as targets, and some of them may be quite easy.
", "rawMarkdown": "> so the submission to casp16 server still have better msa?\n\nI don't think that's the right way to think about it. Remember, in CASP competitions there are no training and test datasets that are distinct in terms of sequence. Whatever needs to be predicted is a test dataset, and predictors can use literally whatever they want for training their predictors. If their predictors have already been trained on a sequence that is similar to CASP targets, they will naturally do better.\n\nAlso, CASP targets are not necessarily chosen with any particular sequence length or difficulty level in mind. Whatever RNAs are currently in the pipeline for structure determination will be included as targets, and some of them may be quite easy.", "votes": null }, { "id": "3161606", "postDate": "03/28/2025 07:07:47", "content": "\ni check chai-1 paper. chai-1 server: 0.382 for CASP15 is what they report in their paper.
\nhave u submit chai-1 for hidden public test in kaggle (casp16)?
\n\nhave u submit chai-1 for hidden public test in kaggle (casp16)?
\n
not yet, I have found only 9 ground truth PDBs for RNA here https://predictioncenter.org/casp16/targetlist.cgi?view=rna (and 2 of them are RNA-protein complexes)
", "rawMarkdown": ">have u submit chai-1 for hidden public test in kaggle (casp16)?\n\nnot yet, I have found only 9 ground truth PDBs for RNA here https://predictioncenter.org/casp16/targetlist.cgi?view=rna (and 2 of them are RNA-protein complexes)", "votes": null }, { "id": "3161632", "postDate": "03/28/2025 07:38:07", "content": "you can compute all casp16 rna target on your local pc or webserver (i think there are about 50+ of them).
\nthen make submit as follows:
\nhttps://www.kaggle.com/code/hengck23/vfold-baseline-offline
kaggle can give your casp16 score.
\n(chack also comment of the notebook link)
your chai-1 msa results seems wrong
\n
msa_depth.pdf shows the msa seq used but it is empty
\n(for a correct msa_depth.pdf, run chai-1 example for protein prediction with msa)
maybe msa is disabled for non protein, etc in chai-1 … or some speical setting needs to be set …
\nplease consult chai-1 repo or author,etc
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fd5a4adf1774f4b999b4b8c6b7a022d96%2FSelection_163.png?generation=1743218867535876&alt=media\")
server only search for protein msa
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F41caf9fd8568bec759988513d118f621%2FSelection_164.png?generation=1743222053279788&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F8e18bab1a916c3c2b4e41da0486530b4%2FSelection_165.png?generation=1743229869226358&alt=media\")
@hengck23 nothing wrong here, chai-1 server makes prediction according to its documentation (btw this aspect is not very clear - When MSAs are disabled, Chai-1 uses a language model to capture evolutionary statistics about the protein). I didn't check if authors trained chai-1 using RNA MSAs, but if they didn't, as you have mentioned above, it's no way to improve predictions without retraining/finetuning.
\nProtenix looks much better: the same score without MSAs and higher score with MSA (0.413 CASP15).
it turns out that NONE of the AF3 clone support MSA for RNA structure prediction!!!!
\nhttps://github.com/google-deepmind/alphafold3/blob/main/docs/input.md
\nquote\"The custom AlphaFold 3 format allows:
Specifying protein, RNA, and DNA chains, including modified residues.
\nSpecifying custom multiple sequence alignment (MSA) for protein and RNA chains.\"
https://github.com/chaidiscovery/chai-lab/issues/22
\nquote \"Note however, that this version of Chai-1 isn't trained with DNA/RNA MSAs so may not provide optimal performance for these.\".
\nif you go through the code, MSA is only for protein
https://github.com/jwohlwend/boltz/blob/5f515995cfca71521dfac39d910a86cd4b5bb8e6/docs/prediction.md
\nquote \"msa: MSA_PATH # only for protein\"
https://github.com/bytedance/Protenix/issues/90
\n\"… Yes, MSA for RNA is not available yet . Although Protenix have already support most of the code for RNA MSAs，due to the hit rate of RNA is very low, we did not find a significant improvement in our experiments. In order to simplify the whole pipeline, we did not use RNA MSA for both training and inference in our released model.\"
", "rawMarkdown": "it turns out that NONE of the AF3 clone support MSA for RNA structure prediction!!!!\n\n---\n##AF3: \nhttps://github.com/google-deepmind/alphafold3/blob/main/docs/input.md\nquote\"The custom AlphaFold 3 format allows:\n\nSpecifying protein, RNA, and DNA chains, including modified residues.\nSpecifying custom multiple sequence alignment (MSA) for protein and RNA chains.\"\n\n---\n\n##chai-1: \nhttps://github.com/chaidiscovery/chai-lab/issues/22\nquote \"Note however, that this version of Chai-1 isn't trained with DNA/RNA MSAs so may not provide optimal performance for these.\".\nif you go through the code, MSA is only for protein\n\n---\n\n##boltz: \nhttps://github.com/jwohlwend/boltz/blob/5f515995cfca71521dfac39d910a86cd4b5bb8e6/docs/prediction.md\nquote \"msa: MSA_PATH # only for protein\"\n\n\n---\n\n##proteinx: \nhttps://github.com/bytedance/Protenix/issues/90\n\n\"... Yes, MSA for RNA is not available yet . Although Protenix have already support most of the code for RNA MSAs，due to the hit rate of RNA is very low, we did not find a significant improvement in our experiments. In order to simplify the whole pipeline, we did not use RNA MSA for both training and inference in our released model.\"", "votes": null }, { "id": "3162467", "postDate": "03/29/2025 08:50:45", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F682731%2Fbbe7bfe557bc71090085b6d3a10f32e6%2Fprotenix.PNG?generation=1743238147278424&alt=media\")
so enabling MSA does nothing, but disabling esm improves score (0.382 --> 0.413 CASP15)
", "rawMarkdown": "\n\nso enabling MSA does nothing, but disabling esm improves score (0.382 --> 0.413 CASP15)", "votes": null }, { "id": "3162473", "postDate": "03/29/2025 08:59:10", "content": "not sure about server code. but local repo code for proteinx doesn't seems to use msa for rna.
\nin particular:
\nhttps://github.com/bytedance/Protenix/blob/main/configs/configs_data.py
\"msa\": {\n \"enable\": ,\n \"enable_rna_msa\": ,\n\n \"prot\": {\n \"pairing_db\": \"uniref100\",\n \"non_pairing_db\": \"mmseqs_other\",\n \"pdb_mmseqs_dir\": os.path.(DATA_ROOT_DIR, \"mmcif_msa\"),\n \"seq_to_pdb_idx_path\": os.path.(DATA_ROOT_DIR, \"seq_to_pdb_index.json\"),\n \"indexing_method\": \"sequence\",\n },\n \"rna\": {\n \"seq_to_pdb_idx_path\": \"\",\n \"rna_msa_dir\": \"\",\n \"indexing_method\": \"sequence\",\n },\nYes IIRC they mentioned they didn’t use MSA for RNA training part.
", "rawMarkdown": "Yes IIRC they mentioned they didn’t use MSA for RNA training part.", "votes": null }, { "id": "3163878", "postDate": "03/31/2025 08:59:32", "content": "gotta go study af2 and af3. Thx a lot for opening this great compeition. this might make another alphafold moment.
", "rawMarkdown": "gotta go study af2 and af3. Thx a lot for opening this great compeition. this might make another alphafold moment.", "votes": null }, { "id": "3166811", "postDate": "04/01/2025 03:58:04", "content": "just out
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Ff258eec759ad0910198db88a47e78d30%2FSelection_999(7880).png?generation=1743479882346230&alt=media\")
I'd like to share my evolving understanding of protein folding and the role of long non-coding RNAs (lncRNAs) as potential chaperones. Protein folding often required cooperating chaperones, or chaperone proteins as used in old days. In recent years, it is becoming clear that lncRNA also can function as chaperone to modulate protein expression and folding. Here is one example: https://pubmed.ncbi.nlm.nih.gov/39706163/
\nThe take home message is that, considering RNA folding in the context of protein folding dynamics, or considering protein folding in the context of RNA folding dynamics, is the way to go!
", "rawMarkdown": "I'd like to share my evolving understanding of protein folding and the role of long non-coding RNAs (lncRNAs) as potential chaperones. Protein folding often required cooperating chaperones, or chaperone proteins as used in old days. In recent years, it is becoming clear that lncRNA also can function as chaperone to modulate protein expression and folding. Here is one example: https://pubmed.ncbi.nlm.nih.gov/39706163/ \n\nThe take home message is that, considering RNA folding in the context of protein folding dynamics, or considering protein folding in the context of RNA folding dynamics, is the way to go!", "votes": null }, { "id": "3293775", "postDate": "09/24/2025 16:05:04", "content": "Good comp BTW, thank you host.
", "rawMarkdown": "Good comp BTW, thank you host.", "votes": null } ], "comments": [ { "id": 3160623, "author_name": "zacchaeus", "author_url": "", "post_date": "03/27/2025 01:13:11", "content": "Thanks for providing such a strong baseline. May I ask if we are licensed to use Protenix (a AF3 rework) in this competition?
", "votes": null, "replies": [ { "id": 3160739, "author_name": "sweetyheehee", "author_url": "", "post_date": "03/27/2025 05:22:46", "content": "\n", "votes": null, "replies": [ { "id": 3160743, "author_name": "sweetyheehee", "author_url": "", "post_date": "03/27/2025 05:31:21", "content": "The Protenix project, including code and model parameters, is made available under the Apache 2.0 License, it is free for both academic research and commercial use.
\n
In my understanding, the use of the AlphaFold model requires permission from DeepMind, so it is not accessible to everyone. However, Protenix does not have this restriction—it is open-source and available to all. Many SOTA models have similar limitations as AlphaFold, which disqualifies them from meeting the criteria I think.
\n\n", "votes": null, "replies": [ { "id": 3160748, "author_name": "sweetyheehee", "author_url": "", "post_date": "03/27/2025 05:40:51", "content": "Obtaining Model Parameters
\n
\n This repository contains all necessary code for AlphaFold 3 inference. To request access to the AlphaFold 3 model parameters, please complete this form. Access will be granted at Google DeepMind’s sole discretion. We will aim to respond to requests within 2–3 business days. You may only use AlphaFold 3 model parameters if received directly from Google. Use is subject to these terms of use.
The AlphaFold 3 model parameters and output are only available for non-commercial use by, or on behalf of non-commercial organizations (i.e., universities, non-profit organizations and research institutes, educational, journalism and government bodies). If you are a researcher affiliated with a non-commercial organization, provided you are not a commercial organization or acting on behalf of a commercial organization, this means you can use it for your non-commercial affiliated research.
\nYou must not use nor allow others to use:
\nAlphaFold 3 model parameters or output in connection with any commercial activities, including research on behalf of commercial organizations; or
\nAlphaFold 3 output to train machine learning models or related technology for biomolecular structure prediction similar to AlphaFold 3.
\nYou must not publish or share AlphaFold 3 model parameters, except sharing these within your organization in accordance with the Terms.
\nYou can publish, share and adapt AlphaFold 3 output in accordance with the Terms, including the requirements to provide clear notice of any modifications you make and that ongoing use of AlphaFold 3 output and derivatives are subject to the AlphaFold 3 Output Terms of Use.
", "votes": null, "replies": [] }, { "id": 3160752, "author_name": "hengck23", "author_url": "", "post_date": "03/27/2025 05:42:59", "content": "There are other clone like boltz and chai-1. If any other team are using these and can share their results, we can save time and avoid spending time to setup all
", "votes": null, "replies": [ { "id": 3160804, "author_name": "ogurtsov", "author_url": "", "post_date": "03/27/2025 06:56:39", "content": "Validation scores from chai-1 server: 0.382 without MSAs / 0.342 (???) MMseqs2 MSAs
", "votes": null, "replies": [ { "id": 3160810, "author_name": "hengck23", "author_url": "", "post_date": "03/27/2025 07:06:16", "content": "@ogurtsov thanks for the feedback. how did you get 0.382 for CASP16? (this is not shown in the leaderboard) or 0.382 is for CASP15?
", "votes": null, "replies": [ { "id": 3160822, "author_name": "ogurtsov", "author_url": "", "post_date": "03/27/2025 07:21:56", "content": "@hengck23 it's for validation RNAs (CASP15). The score is surprisingly low.
", "votes": null, "replies": [ { "id": 3160824, "author_name": "ogurtsov", "author_url": "", "post_date": "03/27/2025 07:23:22", "content": "https://www.kaggle.com/datasets/ogurtsov/chai1-preds
\nhttps://www.kaggle.com/datasets/ogurtsov/chai-preds-msa-mmseqs2
\nI have made prediction datasets public
i check chai-1 paper. chai-1 server: 0.382 for CASP15 is what they report in their paper.
\nhave u submit chai-1 for hidden public test in kaggle (casp16)?
\n\nhave u submit chai-1 for hidden public test in kaggle (casp16)?
\n
not yet, I have found only 9 ground truth PDBs for RNA here https://predictioncenter.org/casp16/targetlist.cgi?view=rna (and 2 of them are RNA-protein complexes)
", "votes": null, "replies": [ { "id": 3161632, "author_name": "hengck23", "author_url": "", "post_date": "03/28/2025 07:38:07", "content": "you can compute all casp16 rna target on your local pc or webserver (i think there are about 50+ of them).
\nthen make submit as follows:
\nhttps://www.kaggle.com/code/hengck23/vfold-baseline-offline
kaggle can give your casp16 score.
\n(chack also comment of the notebook link)
your chai-1 msa results seems wrong
\n
msa_depth.pdf shows the msa seq used but it is empty
\n(for a correct msa_depth.pdf, run chai-1 example for protein prediction with msa)
maybe msa is disabled for non protein, etc in chai-1 … or some speical setting needs to be set …
\nplease consult chai-1 repo or author,etc
server only search for protein msa
\n
@hengck23 nothing wrong here, chai-1 server makes prediction according to its documentation (btw this aspect is not very clear - When MSAs are disabled, Chai-1 uses a language model to capture evolutionary statistics about the protein). I didn't check if authors trained chai-1 using RNA MSAs, but if they didn't, as you have mentioned above, it's no way to improve predictions without retraining/finetuning.
\nProtenix looks much better: the same score without MSAs and higher score with MSA (0.413 CASP15).
i aslo share out team submission (public test casp16)
\nproteinX (alpha3 clone)
\nnoMAS = lb 0.327 (float16 kaggle notebook 2x t4), 0.328 (bfloat16 offline)
Does ProteinX support long sequence processing on Kaggle? I've encountered OOM issues with RhoFold and NuFold. How did you manage to handle this?
\nBTW, authors of Nufold shared their MSA in CASP15. I will write a topic in discussion later.
\nhttps://kiharalab.org/nufold/CASP15_MSA/
@biancochiu
\nfyi: trROSEttaRNA CASP15 msa files:
\n(BLASTN and Infernal on the RNAcentral ?)
\nhttps://yanglab.qd.sdu.edu.cn/trRosettaRNA/benchmark/
@hengck23 Is it an offline submission or using kaggle GPU?
", "votes": null, "replies": [] } ] }, { "id": 3160678, "author_name": "hengck23", "author_url": "", "post_date": "03/27/2025 03:26:56", "content": "also, if you pull results from CASP16 server, alphafold3 server has lb0.44+
\nso the submission to casp16 server still have better msa?
\n\nso the submission to casp16 server still have better msa?
\n
I don't think that's the right way to think about it. Remember, in CASP competitions there are no training and test datasets that are distinct in terms of sequence. Whatever needs to be predicted is a test dataset, and predictors can use literally whatever they want for training their predictors. If their predictors have already been trained on a sequence that is similar to CASP targets, they will naturally do better.
\nAlso, CASP targets are not necessarily chosen with any particular sequence length or difficulty level in mind. Whatever RNAs are currently in the pipeline for structure determination will be included as targets, and some of them may be quite easy.
", "votes": null, "replies": [] } ] }, { "id": 3160747, "author_name": "sweetyheehee", "author_url": "", "post_date": "03/27/2025 05:34:53", "content": "Sigh, the game is too hard.
", "votes": null, "replies": [] }, { "id": 3162459, "author_name": "hengck23", "author_url": "", "post_date": "03/29/2025 08:34:48", "content": "it turns out that NONE of the AF3 clone support MSA for RNA structure prediction!!!!
\nhttps://github.com/google-deepmind/alphafold3/blob/main/docs/input.md
\nquote\"The custom AlphaFold 3 format allows:
Specifying protein, RNA, and DNA chains, including modified residues.
\nSpecifying custom multiple sequence alignment (MSA) for protein and RNA chains.\"
https://github.com/chaidiscovery/chai-lab/issues/22
\nquote \"Note however, that this version of Chai-1 isn't trained with DNA/RNA MSAs so may not provide optimal performance for these.\".
\nif you go through the code, MSA is only for protein
https://github.com/jwohlwend/boltz/blob/5f515995cfca71521dfac39d910a86cd4b5bb8e6/docs/prediction.md
\nquote \"msa: MSA_PATH # only for protein\"
https://github.com/bytedance/Protenix/issues/90
\n\"… Yes, MSA for RNA is not available yet . Although Protenix have already support most of the code for RNA MSAs，due to the hit rate of RNA is very low, we did not find a significant improvement in our experiments. In order to simplify the whole pipeline, we did not use RNA MSA for both training and inference in our released model.\"
", "votes": null, "replies": [ { "id": 3162467, "author_name": "ogurtsov", "author_url": "", "post_date": "03/29/2025 08:50:45", "content": "
so enabling MSA does nothing, but disabling esm improves score (0.382 --> 0.413 CASP15)
", "votes": null, "replies": [ { "id": 3162473, "author_name": "hengck23", "author_url": "", "post_date": "03/29/2025 08:59:10", "content": "not sure about server code. but local repo code for proteinx doesn't seems to use msa for rna.
\nin particular:
\nhttps://github.com/bytedance/Protenix/blob/main/configs/configs_data.py
\"msa\": {\n \"enable\": ,\n \"enable_rna_msa\": ,\n\n \"prot\": {\n \"pairing_db\": \"uniref100\",\n \"non_pairing_db\": \"mmseqs_other\",\n \"pdb_mmseqs_dir\": os.path.(DATA_ROOT_DIR, \"mmcif_msa\"),\n \"seq_to_pdb_idx_path\": os.path.(DATA_ROOT_DIR, \"seq_to_pdb_index.json\"),\n \"indexing_method\": \"sequence\",\n },\n \"rna\": {\n \"seq_to_pdb_idx_path\": \"\",\n \"rna_msa_dir\": \"\",\n \"indexing_method\": \"sequence\",\n },\nYes IIRC they mentioned they didn’t use MSA for RNA training part.
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3163878, "author_name": "beckpro", "author_url": "", "post_date": "03/31/2025 08:59:32", "content": "gotta go study af2 and af3. Thx a lot for opening this great compeition. this might make another alphafold moment.
", "votes": null, "replies": [] }, { "id": 3166811, "author_name": "hengck23", "author_url": "", "post_date": "04/01/2025 03:58:04", "content": "just out
\n
I'd like to share my evolving understanding of protein folding and the role of long non-coding RNAs (lncRNAs) as potential chaperones. Protein folding often required cooperating chaperones, or chaperone proteins as used in old days. In recent years, it is becoming clear that lncRNA also can function as chaperone to modulate protein expression and folding. Here is one example: https://pubmed.ncbi.nlm.nih.gov/39706163/
\nThe take home message is that, considering RNA folding in the context of protein folding dynamics, or considering protein folding in the context of RNA folding dynamics, is the way to go!
", "votes": null, "replies": [] }, { "id": 3293775, "author_name": "tom99763", "author_url": "", "post_date": "09/24/2025 16:05:04", "content": "Good comp BTW, thank you host.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3160619": "As is discussed extensively already in these forums, AlpahFold 3 should provide a powerful baseline for our RNA structure prediction competition.\n\nUnfortunately, due to licensing restrictions, you cannot run AlphaFold3 in your notebooks.\n\nStill, as hosts, we can run AlphaFold on the leaderboard targets and provide scores, and @alissahummer has done this!\n\nYou can now find `AF3_noMSA` and `AF3_rMSA` scores on the [leaderboard](https://www.kaggle.com/competitions/stanford-rna-3d-folding/leaderboard), with scores of 0.259 (no MSA) and 0.397 (with alignments generated with rMSA, the pipeline we are using for default MSA's in this competition). \n\nGood luck as you hill-climb towards and past these baselines!", "3160623": "Thanks for providing such a strong baseline. May I ask if we are licensed to use Protenix (a AF3 rework) in this competition?", "3160676": "i aslo share out team submission (public test casp16)\nproteinX (alpha3 clone)\nnoMAS = lb 0.327 (float16 kaggle notebook 2x t4), 0.328 (bfloat16 offline)", "3160678": "also, if you pull results from CASP16 server, alphafold3 server has lb0.44+\nso the submission to casp16 server still have better msa?", "3160739": ">The Protenix project, including code and model parameters, is made available under the Apache 2.0 License, it is free for both academic research and commercial use.", "3160743": "In my understanding, the use of the AlphaFold model requires permission from DeepMind, so it is not accessible to everyone. However, Protenix does not have this restriction—it is open-source and available to all. Many SOTA models have similar limitations as AlphaFold, which disqualifies them from meeting the criteria I think.\n\n>Obtaining Model Parameters\nThis repository contains all necessary code for AlphaFold 3 inference. To request access to the AlphaFold 3 model parameters, please complete this form. Access will be granted at Google DeepMind’s sole discretion. We will aim to respond to requests within 2–3 business days. You may only use AlphaFold 3 model parameters if received directly from Google. Use is subject to these terms of use.", "3160747": "Sigh, the game is too hard.", "3160748": "The AlphaFold 3 model parameters and output are only available for non-commercial use by, or on behalf of non-commercial organizations (i.e., universities, non-profit organizations and research institutes, educational, journalism and government bodies). If you are a researcher affiliated with a non-commercial organization, provided you are not a commercial organization or acting on behalf of a commercial organization, this means you can use it for your non-commercial affiliated research.\n\nYou must not use nor allow others to use:\n\nAlphaFold 3 model parameters or output in connection with any commercial activities, including research on behalf of commercial organizations; or\n\nAlphaFold 3 output to train machine learning models or related technology for biomolecular structure prediction similar to AlphaFold 3. \n\nYou must not publish or share AlphaFold 3 model parameters, except sharing these within your organization in accordance with the Terms. \n\nYou can publish, share and adapt AlphaFold 3 output in accordance with the Terms, including the requirements to provide clear notice of any modifications you make and that ongoing use of AlphaFold 3 output and derivatives are subject to the AlphaFold 3 Output Terms of Use.", "3160752": "There are other clone like boltz and chai-1. If any other team are using these and can share their results, we can save time and avoid spending time to setup all", "3160753": "Does ProteinX support long sequence processing on Kaggle? I've encountered OOM issues with RhoFold and NuFold. How did you manage to handle this?\nBTW, authors of Nufold shared their MSA in CASP15. I will write a topic in discussion later. \nhttps://kiharalab.org/nufold/CASP15_MSA/", "3160759": "hengck23 Is it an offline submission or using kaggle GPU?", "3160775": "biancochiu \nfyi: trROSEttaRNA CASP15 msa files:\n(BLASTN and Infernal on the RNAcentral ?)\nhttps://yanglab.qd.sdu.edu.cn/trRosettaRNA/benchmark/", "3160804": "Validation scores from chai-1 server: 0.382 without MSAs / 0.342 (???) MMseqs2 MSAs", "3160810": "ogurtsov thanks for the feedback. how did you get 0.382 for CASP16? (this is not shown in the leaderboard) or 0.382 is for CASP15?", "3160822": "hengck23 it's for validation RNAs (CASP15). The score is surprisingly low.", "3160824": "https://www.kaggle.com/datasets/ogurtsov/chai1-preds \nhttps://www.kaggle.com/datasets/ogurtsov/chai-preds-msa-mmseqs2\nI have made prediction datasets public", "3161415": "> so the submission to casp16 server still have better msa?\n\nI don't think that's the right way to think about it. Remember, in CASP competitions there are no training and test datasets that are distinct in terms of sequence. Whatever needs to be predicted is a test dataset, and predictors can use literally whatever they want for training their predictors. If their predictors have already been trained on a sequence that is similar to CASP targets, they will naturally do better.\n\nAlso, CASP targets are not necessarily chosen with any particular sequence length or difficulty level in mind. Whatever RNAs are currently in the pipeline for structure determination will be included as targets, and some of them may be quite easy.", "3161606": "ogurtsov \n\ni check chai-1 paper. chai-1 server: 0.382 for CASP15 is what they report in their paper.\nhave u submit chai-1 for hidden public test in kaggle (casp16)?", "3161623": ">have u submit chai-1 for hidden public test in kaggle (casp16)?\n\nnot yet, I have found only 9 ground truth PDBs for RNA here https://predictioncenter.org/casp16/targetlist.cgi?view=rna (and 2 of them are RNA-protein complexes)", "3161632": "you can compute all casp16 rna target on your local pc or webserver (i think there are about 50+ of them).\nthen make submit as follows:\nhttps://www.kaggle.com/code/hengck23/vfold-baseline-offline\n\nkaggle can give your casp16 score.\n(chack also comment of the notebook link)", "3162271": "ogurtsov \n\nyour chai-1 msa results seems wrong\n\n\n\nmsa_depth.pdf shows the msa seq used but it is empty\n(for a correct msa_depth.pdf, run chai-1 example for protein prediction with msa)\n\nmaybe msa is disabled for non protein, etc in chai-1 ... or some speical setting needs to be set ...\nplease consult chai-1 repo or author,etc\n\n\n\nserver only search for protein msa\n\n\n\n", "3162392": "hengck23 nothing wrong here, chai-1 server makes prediction according to its documentation (btw this aspect is not very clear - `When MSAs are disabled, Chai-1 uses a language model to capture evolutionary statistics about the protein`). I didn't check if authors trained chai-1 using RNA MSAs, but if they didn't, as you have mentioned above, it's no way to improve predictions without retraining/finetuning.\nProtenix looks much better: the same score without MSAs and higher score with MSA (0.413 CASP15).", "3162459": "it turns out that NONE of the AF3 clone support MSA for RNA structure prediction!!!!\n\n---\n##AF3: \nhttps://github.com/google-deepmind/alphafold3/blob/main/docs/input.md\nquote\"The custom AlphaFold 3 format allows:\n\nSpecifying protein, RNA, and DNA chains, including modified residues.\nSpecifying custom multiple sequence alignment (MSA) for protein and RNA chains.\"\n\n---\n\n##chai-1: \nhttps://github.com/chaidiscovery/chai-lab/issues/22\nquote \"Note however, that this version of Chai-1 isn't trained with DNA/RNA MSAs so may not provide optimal performance for these.\".\nif you go through the code, MSA is only for protein\n\n---\n\n##boltz: \nhttps://github.com/jwohlwend/boltz/blob/5f515995cfca71521dfac39d910a86cd4b5bb8e6/docs/prediction.md\nquote \"msa: MSA_PATH # only for protein\"\n\n\n---\n\n##proteinx: \nhttps://github.com/bytedance/Protenix/issues/90\n\n\"... Yes, MSA for RNA is not available yet . Although Protenix have already support most of the code for RNA MSAs，due to the hit rate of RNA is very low, we did not find a significant improvement in our experiments. In order to simplify the whole pipeline, we did not use RNA MSA for both training and inference in our released model.\"", "3162467": "\n\nso enabling MSA does nothing, but disabling esm improves score (0.382 --> 0.413 CASP15)", "3162473": "not sure about server code. but local repo code for proteinx doesn't seems to use msa for rna.\nin particular:\nhttps://github.com/bytedance/Protenix/blob/main/configs/configs_data.py\n\n```\n\n \"msa\": {\n \"enable\": True,\n \"enable_rna_msa\": False,\n\n \"prot\": {\n \"pairing_db\": \"uniref100\",\n \"non_pairing_db\": \"mmseqs_other\",\n \"pdb_mmseqs_dir\": os.path.join(DATA_ROOT_DIR, \"mmcif_msa\"),\n \"seq_to_pdb_idx_path\": os.path.join(DATA_ROOT_DIR, \"seq_to_pdb_index.json\"),\n \"indexing_method\": \"sequence\",\n },\n \"rna\": {\n \"seq_to_pdb_idx_path\": \"\",\n \"rna_msa_dir\": \"\",\n \"indexing_method\": \"sequence\",\n },\n```", "3162498": "Yes IIRC they mentioned they didn’t use MSA for RNA training part.", "3163878": "gotta go study af2 and af3. Thx a lot for opening this great compeition. this might make another alphafold moment.", "3166811": "just out\n.png?generation=1743479882346230&alt=media)", "3175251": "I'd like to share my evolving understanding of protein folding and the role of long non-coding RNAs (lncRNAs) as potential chaperones. Protein folding often required cooperating chaperones, or chaperone proteins as used in old days. In recent years, it is becoming clear that lncRNA also can function as chaperone to modulate protein expression and folding. Here is one example: https://pubmed.ncbi.nlm.nih.gov/39706163/ \n\nThe take home message is that, considering RNA folding in the context of protein folding dynamics, or considering protein folding in the context of RNA folding dynamics, is the way to go!", "3293775": "Good comp BTW, thank you host." }, "source": "meta" }