Welcome to Stanford RNA 3D Folding! As a result of our previous competition Ribonanza, we developed RibonanzaNet, a foundation model which achieved SOTA performance in a number of RNA tasks such as RNA secondary structure, RNA degradation, and RNA chemical mapping dropout. Here I have created starter notebooks finetuning RibonanzaNet to directly predict xyz coordinates of RNA nucleotides:
\ntraining:
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-finetune
inference:
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-inference
Best of luck and have fun modeling!
\nAF2 uses invariant point attention as their structure module to do iterative refinement, here we can do something similar with RIbonanzaNet.
\nI have added a structure module that looks at the distance matrix and makes xyz updates to the finetuning notebook and it improves the score from 0.177 to 0.19.
\nfinetuning and inference notebooks with a structure module:
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-finetune-add-structure-module
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-inference-add-structure-module?scriptVersionId=228564059
Starter training notebook w Rnet2: https://www.kaggle.com/code/shujun717/ribonanzanet2-ddpm-training
\nStarter inference notebook w Rnet2 (after training for 50 epochs): https://www.kaggle.com/code/shujun717/ribonanzanet2-ddpm-inference
link to our paper:
\nhttps://www.biorxiv.org/content/10.1101/2024.02.24.581671v1
previous competition:
\nhttps://www.kaggle.com/competitions/stanford-ribonanza-rna-folding
Welcome to Stanford RNA 3D Folding! As a result of our previous competition Ribonanza, we developed RibonanzaNet, a foundation model which achieved SOTA performance in a number of RNA tasks such as RNA secondary structure, RNA degradation, and RNA chemical mapping dropout. Here I have created starter notebooks finetuning RibonanzaNet to directly predict xyz coordinates of RNA nucleotides:
\ntraining:
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-finetune
inference:
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-inference
Best of luck and have fun modeling!
\nAF2 uses invariant point attention as their structure module to do iterative refinement, here we can do something similar with RIbonanzaNet.
\nI have added a structure module that looks at the distance matrix and makes xyz updates to the finetuning notebook and it improves the score from 0.177 to 0.19.
\nfinetuning and inference notebooks with a structure module:
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-finetune-add-structure-module
\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-inference-add-structure-module?scriptVersionId=228564059
Starter training notebook w Rnet2: https://www.kaggle.com/code/shujun717/ribonanzanet2-ddpm-training
\nStarter inference notebook w Rnet2 (after training for 50 epochs): https://www.kaggle.com/code/shujun717/ribonanzanet2-ddpm-inference
link to our paper:
\nhttps://www.biorxiv.org/content/10.1101/2024.02.24.581671v1
previous competition:
\nhttps://www.kaggle.com/competitions/stanford-ribonanza-rna-folding
@shujun717 does the sequence representation in RibonanzaNet play a similar role as the MSA representation in AlphaFold?
AFAIK, AlphaFold searches against external databases while RibonanzaNet learns the embedding representations used for the sequence representation. Is my understanding correct? Help is appreciated!
", "rawMarkdown": "@shujun717 does the sequence representation in `RibonanzaNet` play a similar role as the `MSA` representation in AlphaFold? \n\nAFAIK, AlphaFold searches against external databases while RibonanzaNet learns the embedding representations used for the sequence representation. Is my understanding correct? Help is appreciated! ", "votes": 1 }, { "id": 3135835, "postDate": "2025-02-27T22:29:25.727Z", "content": "Someone will probably do it soon, but it chould have been really great if you also submitted this inference notebook, demonstrating submission and establishing a public baseline.
\nAlso: nice to see you again, haha
i could submit it but then I would show up on the lb 😀
", "rawMarkdown": "i could submit it but then I would show up on the lb 😀", "votes": 1, "replies": [ { "id": 3135842, "postDate": "2025-02-27T22:50:06.347Z", "content": "I see, so it's already ready for submission! Very good job. I have a huge appreciation for conpetitions that provide a solid baseline to start with.
\nFor those wondering: It's 0.173, ~8 min for scoring.
Thanks for submitting it!
", "rawMarkdown": "Thanks for submitting it!" }, { "id": 3160532, "postDate": "2025-03-26T22:11:37.530Z", "content": "How could I have overlooked this until now
", "rawMarkdown": "How could I have overlooked this until now", "votes": 13 } ] }, { "id": 3135937, "postDate": "2025-02-28T03:15:45.583Z", "content": "Your shared code from before has many valuable lessons to learn from, and you come across as both passionate and humorous. I personally believe that you will put in tremendous effort for this competition and ultimately secure the gold medal. I would like to congratulate you in advance for becoming a Kaggle Competitions Grandmaster.
", "rawMarkdown": "Your shared code from before has many valuable lessons to learn from, and you come across as both passionate and humorous. I personally believe that you will put in tremendous effort for this competition and ultimately secure the gold medal. I would like to congratulate you in advance for becoming a Kaggle Competitions Grandmaster.", "votes": -3 } ] }, { "id": 3142676, "postDate": "2025-03-06T15:37:53.860Z", "content": "I'll implement a GNN modification of RibonanzaNet.
", "rawMarkdown": "I'll implement a GNN modification of RibonanzaNet.", "votes": 1, "replies": [ { "id": 3184133, "postDate": "2025-04-21T17:48:08.393Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 3201662, "postDate": "2025-05-14T08:11:07.600Z", "content": "How do we use invoke RibonanzaNet on a training or test sequence and get the output (i.e. a secondary structure matrix)? As I understand it, the network should output something like a contact matrix of dimensions (len(seq),len(seq)), where elements [i,j] are high (near 1) when i and j likely form a base pair, and low (near 0) when they probably don't, which can then be used as an input to a 3D stage (whether this involves rigid-body positioning stems as coarse grained cylinders, all-atom refinement, or a second neural network). I imagine it's a NxN matrix because something 1D like dot-bracket notation is no good for representing pseudoknotted structures, and also can't represent low-confidence pairs that shouldn't get a full \"yes\". But my question is what does the function call look like to do this, once RibonanzaNet is imported into a notebook, and is the return type of this call in fact a NxN matrix of floats in [0,1], where N is the sequence length?
", "rawMarkdown": "How do we use invoke RibonanzaNet on a training or test sequence and get the output (i.e. a secondary structure matrix)? As I understand it, the network should output something like a contact matrix of dimensions (len(seq),len(seq)), where elements [i,j] are high (near 1) when i and j likely form a base pair, and low (near 0) when they probably don't, which can then be used as an input to a 3D stage (whether this involves rigid-body positioning stems as coarse grained cylinders, all-atom refinement, or a second neural network). I imagine it's a NxN matrix because something 1D like dot-bracket notation is no good for representing pseudoknotted structures, and also can't represent low-confidence pairs that shouldn't get a full \"yes\". But my question is what does the function call look like to do this, once RibonanzaNet is imported into a notebook, and is the return type of this call in fact a NxN matrix of floats in [0,1], where N is the sequence length? ", "replies": [ { "id": 3202138, "postDate": "2025-05-14T22:56:54.747Z", "content": "you may try using Rnet-2 finetuned to predict secondary structure. The notebook is here: https://www.kaggle.com/code/shujun717/rnet2-alpha-2d-structure-inference
", "rawMarkdown": "you may try using Rnet-2 finetuned to predict secondary structure. The notebook is here: https://www.kaggle.com/code/shujun717/rnet2-alpha-2d-structure-inference" }, { "id": 3204787, "postDate": "2025-05-18T19:49:16.227Z", "content": "This notebook contains detailed explaination of Ribonanza Net with architecture diagrams, might be useful to you. Here
", "rawMarkdown": "This notebook contains detailed explaination of Ribonanza Net with architecture diagrams, might be useful to you. [Here](https://www.kaggle.com/code/siddhantoon/ribonanzanet-2-0-ddpm-explained)" } ] }, { "id": 3180433, "postDate": "2025-04-16T16:07:36.667Z", "content": "Thank you for your work. I truly appreciate it.
", "rawMarkdown": "Thank you for your work. I truly appreciate it.", "isDeleted": true }, { "id": 3148198, "postDate": "2025-03-12T20:24:05.427Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 3159045, "postDate": "2025-03-25T07:46:01.967Z", "content": "Yes, you can absolutely do that! Just upload your modified .pt file to a new Kaggle dataset (or replace the one in /kaggle/input/ribonanzanet-3d-finetune), and in the inference notebook, update the torch.load() path to point to your new model file. Make sure the model architecture in the inference script matches what you used during training. Once that’s set, it should work smoothly
", "rawMarkdown": "Yes, you can absolutely do that! Just upload your modified .pt file to a new Kaggle dataset (or replace the one in /kaggle/input/ribonanzanet-3d-finetune), and in the inference notebook, update the torch.load() path to point to your new model file. Make sure the model architecture in the inference script matches what you used during training. Once that’s set, it should work smoothly" } ] }, { "id": 3175690, "postDate": "2025-04-10T13:39:45.550Z", "content": "Thank you!
", "rawMarkdown": "Thank you!", "votes": 2 }, { "id": 3198896, "postDate": "2025-05-10T06:42:36.003Z", "content": "great!!!!!
", "rawMarkdown": "great!!!!!" }, { "id": 3181809, "postDate": "2025-04-18T11:40:24.217Z", "content": "Thank you for your work.
", "rawMarkdown": "Thank you for your work." }, { "id": 3160441, "postDate": "2025-03-26T18:58:40.427Z", "content": "Thank you!
", "rawMarkdown": "Thank you!" } ], "comments": [ { "id": 3149820, "author_name": "moth", "author_url": "", "post_date": "2025-03-14T18:03:00.667000", "content": "@shujun717 does the sequence representation in RibonanzaNet play a similar role as the MSA representation in AlphaFold?
AFAIK, AlphaFold searches against external databases while RibonanzaNet learns the embedding representations used for the sequence representation. Is my understanding correct? Help is appreciated!
", "votes": 1, "replies": [] }, { "id": 3135835, "author_name": "greySnow", "author_url": "", "post_date": "2025-02-27T22:29:25.727000", "content": "Someone will probably do it soon, but it chould have been really great if you also submitted this inference notebook, demonstrating submission and establishing a public baseline.
\nAlso: nice to see you again, haha
i could submit it but then I would show up on the lb 😀
", "votes": 1, "replies": [ { "id": 3135842, "author_name": "greySnow", "author_url": "", "post_date": "2025-02-27T22:50:06.347000", "content": "I see, so it's already ready for submission! Very good job. I have a huge appreciation for conpetitions that provide a solid baseline to start with.
\nFor those wondering: It's 0.173, ~8 min for scoring.
Thanks for submitting it!
", "votes": 0, "replies": [] }, { "id": 3160532, "author_name": "", "author_url": "", "post_date": "2025-03-26T22:11:37.530000", "content": "How could I have overlooked this until now
", "votes": 13, "replies": [] } ] }, { "id": 3135937, "author_name": "bestwater", "author_url": "", "post_date": "2025-02-28T03:15:45.583000", "content": "Your shared code from before has many valuable lessons to learn from, and you come across as both passionate and humorous. I personally believe that you will put in tremendous effort for this competition and ultimately secure the gold medal. I would like to congratulate you in advance for becoming a Kaggle Competitions Grandmaster.
", "votes": -3, "replies": [] } ] }, { "id": 3142676, "author_name": "Tom", "author_url": "", "post_date": "2025-03-06T15:37:53.860000", "content": "I'll implement a GNN modification of RibonanzaNet.
", "votes": 1, "replies": [ { "id": 3184133, "author_name": "", "author_url": "", "post_date": "2025-04-21T17:48:08.393000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 3201662, "author_name": "Andrew Rosko", "author_url": "", "post_date": "2025-05-14T08:11:07.600000", "content": "How do we use invoke RibonanzaNet on a training or test sequence and get the output (i.e. a secondary structure matrix)? As I understand it, the network should output something like a contact matrix of dimensions (len(seq),len(seq)), where elements [i,j] are high (near 1) when i and j likely form a base pair, and low (near 0) when they probably don't, which can then be used as an input to a 3D stage (whether this involves rigid-body positioning stems as coarse grained cylinders, all-atom refinement, or a second neural network). I imagine it's a NxN matrix because something 1D like dot-bracket notation is no good for representing pseudoknotted structures, and also can't represent low-confidence pairs that shouldn't get a full \"yes\". But my question is what does the function call look like to do this, once RibonanzaNet is imported into a notebook, and is the return type of this call in fact a NxN matrix of floats in [0,1], where N is the sequence length?
", "votes": 0, "replies": [ { "id": 3202138, "author_name": "Shujun", "author_url": "", "post_date": "2025-05-14T22:56:54.747000", "content": "you may try using Rnet-2 finetuned to predict secondary structure. The notebook is here: https://www.kaggle.com/code/shujun717/rnet2-alpha-2d-structure-inference
", "votes": 0, "replies": [] }, { "id": 3204787, "author_name": "Siddhantoon", "author_url": "", "post_date": "2025-05-18T19:49:16.227000", "content": "This notebook contains detailed explaination of Ribonanza Net with architecture diagrams, might be useful to you. Here
", "votes": 0, "replies": [] } ] }, { "id": 3180433, "author_name": "", "author_url": "", "post_date": "2025-04-16T16:07:36.667000", "content": "Thank you for your work. I truly appreciate it.
", "votes": 0, "replies": [] }, { "id": 3148198, "author_name": "", "author_url": "", "post_date": "2025-03-12T20:24:05.427000", "content": "", "votes": 0, "replies": [ { "id": 3159045, "author_name": "Fae Gaze", "author_url": "", "post_date": "2025-03-25T07:46:01.967000", "content": "Yes, you can absolutely do that! Just upload your modified .pt file to a new Kaggle dataset (or replace the one in /kaggle/input/ribonanzanet-3d-finetune), and in the inference notebook, update the torch.load() path to point to your new model file. Make sure the model architecture in the inference script matches what you used during training. Once that’s set, it should work smoothly
", "votes": 0, "replies": [] } ] }, { "id": 3175690, "author_name": "", "author_url": "", "post_date": "2025-04-10T13:39:45.550000", "content": "Thank you!
", "votes": 2, "replies": [] }, { "id": 3198896, "author_name": "Zhiai Fang", "author_url": "", "post_date": "2025-05-10T06:42:36.003000", "content": "great!!!!!
", "votes": 0, "replies": [] }, { "id": 3181809, "author_name": "chencheng hua", "author_url": "", "post_date": "2025-04-18T11:40:24.217000", "content": "Thank you for your work.
", "votes": 0, "replies": [] }, { "id": 3160441, "author_name": "Julia Iskandarova", "author_url": "", "post_date": "2025-03-26T18:58:40.427000", "content": "Thank you!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3135830": "Welcome to Stanford RNA 3D Folding! As a result of our previous competition Ribonanza, we developed RibonanzaNet, a foundation model which achieved SOTA performance in a number of RNA tasks such as RNA secondary structure, RNA degradation, and RNA chemical mapping dropout. Here I have created starter notebooks finetuning RibonanzaNet to directly predict xyz coordinates of RNA nucleotides:\n\ntraining:\n[https://www.kaggle.com/code/shujun717/ribonanzanet-3d-finetune](https://www.kaggle.com/code/shujun717/ribonanzanet-3d-finetune)\n\ninference:\n[https://www.kaggle.com/code/shujun717/ribonanzanet-3d-inference](https://www.kaggle.com/code/shujun717/ribonanzanet-3d-inference)\n\nBest of luck and have fun modeling! \n\n### update to the finetuning notebook with a structure module:\nAF2 uses invariant point attention as their structure module to do iterative refinement, here we can do something similar with RIbonanzaNet.\n\nI have added a structure module that looks at the distance matrix and makes xyz updates to the finetuning notebook and it improves the score from 0.177 to 0.19.\n\nfinetuning and inference notebooks with a structure module:\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-finetune-add-structure-module\nhttps://www.kaggle.com/code/shujun717/ribonanzanet-3d-inference-add-structure-module?scriptVersionId=228564059\n\n### update with Rnet2 Alpha finetuning notebook with diffusion (lb 0.3):\n\nStarter training notebook w Rnet2: https://www.kaggle.com/code/shujun717/ribonanzanet2-ddpm-training\nStarter inference notebook w Rnet2 (after training for 50 epochs): https://www.kaggle.com/code/shujun717/ribonanzanet2-ddpm-inference\n\n### links\nlink to our paper:\n[https://www.biorxiv.org/content/10.1101/2024.02.24.581671v1](https://www.biorxiv.org/content/10.1101/2024.02.24.581671v1)\n\nprevious competition:\n[https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding](https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding)", "3149820": "@shujun717 does the sequence representation in `RibonanzaNet` play a similar role as the `MSA` representation in AlphaFold? \n\nAFAIK, AlphaFold searches against external databases while RibonanzaNet learns the embedding representations used for the sequence representation. Is my understanding correct? Help is appreciated! ", "3135835": "Someone will probably do it soon, but it chould have been really great if you also submitted this inference notebook, demonstrating submission and establishing a public baseline.\nAlso: nice to see you again, haha", "3142676": "I'll implement a GNN modification of RibonanzaNet.", "3201662": "How do we use invoke RibonanzaNet on a training or test sequence and get the output (i.e. a secondary structure matrix)? As I understand it, the network should output something like a contact matrix of dimensions (len(seq),len(seq)), where elements [i,j] are high (near 1) when i and j likely form a base pair, and low (near 0) when they probably don't, which can then be used as an input to a 3D stage (whether this involves rigid-body positioning stems as coarse grained cylinders, all-atom refinement, or a second neural network). I imagine it's a NxN matrix because something 1D like dot-bracket notation is no good for representing pseudoknotted structures, and also can't represent low-confidence pairs that shouldn't get a full \"yes\". But my question is what does the function call look like to do this, once RibonanzaNet is imported into a notebook, and is the return type of this call in fact a NxN matrix of floats in [0,1], where N is the sequence length? ", "3180433": "Thank you for your work. I truly appreciate it.", "3148198": "", "3175690": "Thank you!", "3198896": "great!!!!!", "3181809": "Thank you for your work.", "3160441": "Thank you!" } }