{ "id": 460252, "title": "GraphAttention solution approach", "url": "/competitions/stanford-ribonanza-rna-folding/discussion/460252", "author_name": "VITALIY", "post_date": "2023-12-08T12:12:59.053000", "votes": 10, "comment_count": 0, "views": 0, "content": "

Hi there,thanks Stanford University for this competition!
\nMight be someone will found my work is interesting.
\nNote enough of time was my biggest problem,but after all this is my first medal,even though its silver, i am really happy.

\n
    \n
  1. Preprocessing RNA sequence to graph:
    \n1.1. Used Eternafold pkg for extracting secondary structure.
    \n1.2. OHE nucleotids -> node features
    \n1.3. Used as edge features -> [phosphodiester_bond,base_pairing(canonical or wobble),BPPS]
    2. \n
  2. My final solution was:
    \n2.1. Random walk as positional encoding.
    \n2.2. Architecture:
    \nCombination of:
    \nlocal attention -> GraphTransformer.
    \nglobal attention -> Attention encoder.
    \n20 layers depth nn(192 hidden dim), for me expands hidden dimension doesn't help.
    3. \n
\n

\"\"

\n

This works helps me a lot:
\nhttps://arxiv.org/abs/2009.03509
\nhttps://arxiv.org/abs/2205.12454

\n

3.
\nDefault splitting data on train and validation(10 % of SN filter =1)
\nDuring train process using rna sequence with signal to noise >= 0.8

\n

For final submission used only one model prediction no stucks, that might increase quality of prediction as i said before no time was problem.

\n

github:https://github.com/cerenov94/ribonanzaRNA
\nInstruments:
\nPytorch Geometric,Graphein

", "messages": [ { "id": 2553640, "postDate": "2023-12-08T12:12:59.053Z", "content": "

Hi there,thanks Stanford University for this competition!
\nMight be someone will found my work is interesting.
\nNote enough of time was my biggest problem,but after all this is my first medal,even though its silver, i am really happy.

\n
    \n
  1. Preprocessing RNA sequence to graph:
    \n1.1. Used Eternafold pkg for extracting secondary structure.
    \n1.2. OHE nucleotids -> node features
    \n1.3. Used as edge features -> [phosphodiester_bond,base_pairing(canonical or wobble),BPPS]
    2. \n
  2. My final solution was:
    \n2.1. Random walk as positional encoding.
    \n2.2. Architecture:
    \nCombination of:
    \nlocal attention -> GraphTransformer.
    \nglobal attention -> Attention encoder.
    \n20 layers depth nn(192 hidden dim), for me expands hidden dimension doesn't help.
    3. \n
\n

\"\"

\n

This works helps me a lot:
\nhttps://arxiv.org/abs/2009.03509
\nhttps://arxiv.org/abs/2205.12454

\n

3.
\nDefault splitting data on train and validation(10 % of SN filter =1)
\nDuring train process using rna sequence with signal to noise >= 0.8

\n

For final submission used only one model prediction no stucks, that might increase quality of prediction as i said before no time was problem.

\n

github:https://github.com/cerenov94/ribonanzaRNA
\nInstruments:
\nPytorch Geometric,Graphein

", "rawMarkdown": "Hi there,thanks Stanford University for this competition!\nMight be someone will found my work is interesting.\nNote enough of time was my biggest problem,but after all this is my first medal,even though its silver, i am really happy.\n\n\n1. Preprocessing RNA sequence to graph:\n1.1. Used Eternafold pkg for extracting secondary structure.\n1.2. OHE nucleotids -> node features\n1.3. Used as edge features -> [phosphodiester_bond,base_pairing(canonical or wobble),BPPS]\n2. My final solution was:\n2.1. Random walk as positional encoding.\n2.2. Architecture: \nCombination of:\nlocal attention -> GraphTransformer.\nglobal attention -> Attention encoder.\n20 layers depth nn(192 hidden dim), for me expands hidden dimension doesn't help.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10350612%2F29ca6292363042edcf0923d231234396%2Fd.drawio.png?generation=1702059627165899&alt=media)\n\nThis works helps me a lot:\nhttps://arxiv.org/abs/2009.03509\nhttps://arxiv.org/abs/2205.12454\n\n3.\nDefault splitting data on train and validation(10 % of SN filter =1) \nDuring train process using rna sequence with signal to noise >= 0.8\n\nFor final submission used only one model prediction no stucks, that might increase quality of prediction as i said before no time was problem.\n\n\ngithub:https://github.com/cerenov94/ribonanzaRNA\nInstruments:\nPytorch Geometric,Graphein", "votes": 10 } ], "comments": [], "raw_markdown_by_id": { "2553640": "Hi there,thanks Stanford University for this competition!\nMight be someone will found my work is interesting.\nNote enough of time was my biggest problem,but after all this is my first medal,even though its silver, i am really happy.\n\n\n1. Preprocessing RNA sequence to graph:\n1.1. Used Eternafold pkg for extracting secondary structure.\n1.2. OHE nucleotids -> node features\n1.3. Used as edge features -> [phosphodiester_bond,base_pairing(canonical or wobble),BPPS]\n2. My final solution was:\n2.1. Random walk as positional encoding.\n2.2. Architecture: \nCombination of:\nlocal attention -> GraphTransformer.\nglobal attention -> Attention encoder.\n20 layers depth nn(192 hidden dim), for me expands hidden dimension doesn't help.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10350612%2F29ca6292363042edcf0923d231234396%2Fd.drawio.png?generation=1702059627165899&alt=media)\n\nThis works helps me a lot:\nhttps://arxiv.org/abs/2009.03509\nhttps://arxiv.org/abs/2205.12454\n\n3.\nDefault splitting data on train and validation(10 % of SN filter =1) \nDuring train process using rna sequence with signal to noise >= 0.8\n\nFor final submission used only one model prediction no stucks, that might increase quality of prediction as i said before no time was problem.\n\n\ngithub:https://github.com/cerenov94/ribonanzaRNA\nInstruments:\nPytorch Geometric,Graphein" } }