{ "id": 567959, "title": "Convert UW synthetic dataset of 440k synthetic RNAs", "url": "/competitions/stanford-rna-3d-folding/discussion/567959", "author_name": "tomoo inubushi", "post_date": "2025-03-13T02:33:38.101000", "votes": 20, "comment_count": 10, "views": 0, "content": "

The host shared synthetic RNA structure dataset including 447,402 RNA sequences.

\n

I converted this dataset into the same format of this competition. You can download it from here.
\nThe conversion script is here.

\n

I am not sure it works for this competition. I appreciate if anyone share us the result.

\n

p.s.
\nDear @andrewfavor

\n\n

[UPDATE 20240314] I rewrited my conversion script and reuploaded the dataset. Please use the latest version from here
\nI also converted Stanford Ribonanza RNA Folding (data description) including 124k RNAs. The conversion script is here.

", "messages": [ { "id": 3148328, "postDate": "2025-03-13T02:33:38.100Z", "content": "

The host shared synthetic RNA structure dataset including 447,402 RNA sequences.

\n

I converted this dataset into the same format of this competition. You can download it from here.
\nThe conversion script is here.

\n

I am not sure it works for this competition. I appreciate if anyone share us the result.

\n

p.s.
\nDear @andrewfavor

\n\n

[UPDATE 20240314] I rewrited my conversion script and reuploaded the dataset. Please use the latest version from here
\nI also converted Stanford Ribonanza RNA Folding (data description) including 124k RNAs. The conversion script is here.

", "rawMarkdown": "The host shared [synthetic RNA structure dataset](https://www.kaggle.com/datasets/andrewfavor/uw-synthetic-rna-final) including 447,402 RNA sequences.\n\nI converted this dataset into the same format of this competition. You can download it from [here](https://www.kaggle.com/datasets/tomooinubushi/converted-uw-synthetic-rna-final).\nThe conversion script is [here](https://www.kaggle.com/code/tomooinubushi/convert-uw-synthetic-dataset).\n\nI am not sure it works for this competition. I appreciate if anyone share us the result.\n\np.s.\nDear @andrewfavor \n- ~~What is the difference between [uw-synthetic-rna-final](https://www.kaggle.com/datasets/andrewfavor/uw-synthetic-rna-final) and [uw-synthetic-rna-structures](https://www.kaggle.com/datasets/andrewfavor/uw-synthetic-rna-structures) ?~~ Already answered in [here](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/567539). They are identical.\n- ~~Should we concatenate these datasets? or just use final?~~ Already answered in [here](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/567539). We should just use final version.\n- What are the licenses of these datasets? Is my dataset OK?\n\n\n**[UPDATE 20240314]** I rewrited my conversion script and reuploaded the dataset. Please use the latest version from [here](https://www.kaggle.com/datasets/tomooinubushi/converted-uw-synthetic-rna-final)\nI also converted [Stanford Ribonanza RNA Folding](https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data) ([data description](https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/discussion/451853)) including 124k RNAs. The conversion script is [here](https://www.kaggle.com/code/tomooinubushi/convert-stanford-ribonanza-rna-folding-dataset).", "votes": 20 }, { "id": 3148517, "postDate": "2025-03-13T08:13:36.307Z", "content": "

i did not check your code in details but:

\n
    \n
  1. we only need C1 atom (we predict backbone structure only)
    2. \n
  2. resname and resid should be extracted from pdb file (and not running number)
    3. \n
\n

while it is safe to assume that there is only one strcuture and the chain is continous for synthetic data, it may be better to use assert check in your code for these 2 points.

\n

finally, you can verify your conversion code is correct or not by:

\n", "rawMarkdown": "i did not check your code in details but:\n1. we only need C1 atom (we predict backbone structure only)\n2. resname and resid should be extracted from pdb file (and not running number)\n\nwhile it is safe to assume that there is only one strcuture and the chain is continous for synthetic data, it may be better to use assert check in your code for these 2 points.\n\nfinally, you can verify your conversion code is correct or not by:\n- choose a pdb id from kaggle train or validation csv\n- download pdb file from protein bank\n- compare your conversion results with that from the given kaggle label csv", "votes": 4, "replies": [ { "id": 3148520, "postDate": "2025-03-13T08:20:55.750Z", "content": "
def parse_pdb_to_df(pdb_file, target_id):\n    parser = PDBParser()\n\n    structure = parser.get_structure('', pdb_file)\n\n    df=[]\n    for model in structure:\n        for chain in model:\n            print(chain)\n            chain_data = []\n            for residue in chain:\n                 print(residue)\n                if residue.get_resname() in ['A', 'U', 'G', 'C']:\n\n                     Check if the residue has a C1' atom\n                    if 'C1\\'' in residue:\n                        atom = residue['C1\\'']\n                        xyz = atom.get_coord()\n                        resname = residue.get_resname()\n                        resid = residue.get_id()[1]\n\n                            resname resid   x_1 y_1 z_1\n                         detect discontinous: resid = previous resid +1\n                        chain_data.append(dict(\n                            ID=target_id+'_'+str(resid),  change to target_id+'_'+str(chainid)+'_'+str(resid)\n                            resname=resname,\n                            resid=resid,\n                            x_1=xyz[0],\n                            y_1=xyz[1],\n                            z_1=xyz[2],\n                        ))\n                        \n\n            if len(chain_data)!=0:\n                chain_df = pd.DataFrame(chain_data)\n                df.append(chain_df)\n                \n    return df\n
", "rawMarkdown": "```\n\n\ndef parse_pdb_to_df(pdb_file, target_id):\n parser = PDBParser()\n\n structure = parser.get_structure('', pdb_file)\n\n df=[]\n for model in structure:\n for chain in model:\n print(chain)\n chain_data = []\n for residue in chain:\n # print(residue)\n if residue.get_resname() in ['A', 'U', 'G', 'C']:\n\n # Check if the residue has a C1' atom\n if 'C1\\'' in residue:\n atom = residue['C1\\'']\n xyz = atom.get_coord()\n resname = residue.get_resname()\n resid = residue.get_id()[1]\n\n #ID\tresname\tresid\tx_1\ty_1\tz_1\n #todo detect discontinous: resid = previous resid +1\n chain_data.append(dict(\n ID=target_id+'_'+str(resid), #todo: change to target_id+'_'+str(chainid)+'_'+str(resid)\n resname=resname,\n resid=resid,\n x_1=xyz[0],\n y_1=xyz[1],\n z_1=xyz[2],\n ))\n ##print(f\"Residue {resname} {resid}, Atom: {atom.get_name()}, xyz: {xyz}\")\n\n if len(chain_data)!=0:\n chain_df = pd.DataFrame(chain_data)\n df.append(chain_df)\n ##print(chain_df)\n return df\n\n\n```", "votes": 4, "replies": [ { "id": 3148580, "postDate": "2025-03-13T10:05:33.610Z", "content": "

Thank you for your comment.
\nI am now rewriting my conversion code.

\n

I downloaded 1scl.pdb from here and uploaded to here, and compared the data.

\n

I found that the sequences of these data are the same, but x, y, and z coordinates are different.

\n

According to 1scl.pdb, x, y, and z of first c1' are 11.334, -27.307, -2.027.
\nOn the other hand, according to 1SCL_A data in train_labels.csv, x, y, and z are 13.760, -25.974001, 0.102.
\nDo anybody give me a suggestion? Maybe I downloaded the wrong pdb file.

\n

1SCL_A data in train_labels.csv

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
IDresnameresidx_1y_1z_1pdb_id
1SCL_A_1G113.760-25.9740010.1021SCL_A
1SCL_A_2G29.310-29.6380002.6691SCL_A
1SCL_A_3G35.529-27.8130005.8781SCL_A
1SCL_A_4U42.678-24.9009999.7931SCL_A
1SCL_A_5G51.827-20.13600011.7931SCL_A
\n

The result of my current script from 1scl.pdb

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
resnameresidx_1y_1z_1
G111.334-27.306999-2.027
G26.168-29.6390000.850
G33.555-27.5289995.061
U41.614-24.3379999.446
G52.020-19.66700011.685
\n

\"\"

", "rawMarkdown": "Thank you for your comment. \nI am [now rewriting my conversion code](https://www.kaggle.com/code/tomooinubushi/convert-uw-synthetic-dataset).\n\nI downloaded 1scl.pdb from [here](https://www.rcsb.org/structure/1scl) and uploaded to [here](https://www.kaggle.com/datasets/tomooinubushi/1scl-pdb), and compared the data.\n\nI found that the sequences of these data are the same, but x, y, and z coordinates are different.\n\nAccording to 1scl.pdb, x, y, and z of first c1' are 11.334, -27.307, -2.027.\nOn the other hand, according to 1SCL_A data in train_labels.csv, x, y, and z are 13.760, -25.974001, 0.102.\nDo anybody give me a suggestion? Maybe I downloaded the wrong pdb file.\n\n\n1SCL_A data in train_labels.csv \n\n|ID | resname | resid | x_1 | y_1 | z_1 | pdb_id|\n| --- | --- | --- | --- | --- | --- |\n| 1SCL_A_1| G \t| 1 \t| 13.760| -25.974001 | 0.102| 1SCL_A| \n| 1SCL_A_2 | G \t| 2 \t| 9.310| -29.638000| 2.669| 1SCL_A| \n| 1SCL_A_3 | G \t| 3 \t| 5.529| -27.813000 | 5.878| 1SCL_A| \n| 1SCL_A_4 | U \t| 4 \t| 2.678| -24.900999 | 9.793| 1SCL_A| \n| 1SCL_A_5 | G \t| 5 \t| 1.827 | -20.136000 | 11.793| 1SCL_A| \n\n\n [The result of my current script](https://www.kaggle.com/code/tomooinubushi/convert-uw-synthetic-dataset) from 1scl.pdb\n\n| resname | resid | x_1 | y_1 | z_1 | \n| --- | --- | --- | --- | --- |\n| G \t| 1 \t| 11.334 |-27.306999|-2.027|\n| G \t| 2 \t| 6.168|-29.639000|0.850|\n| G \t| 3 \t| 3.555 |-27.528999|5.061|\n| U \t| 4 \t| 1.614 |-24.337999|9.446|\n| G \t| 5 \t| 2.020 |-19.667000|11.685|\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3754725%2F8a5080341d56ff7b3ce2555db7edd7a2%2F__results___6_0.png?generation=1741860220469441&alt=media)" }, { "id": 3148583, "postDate": "2025-03-13T10:09:45.403Z", "content": "

not all will target be the same. if there are some target the same, then your code is correct.

\n

e.g. in my conversion, i have
\nkaggle traget id1 = same (all residue has extact same xyz from downloaded pdb)
\nkaggle traget id2 = not same (athough there is only one coformation in downloaded pdb)

\n

(i am not sure why they are different but it could be due to different version updated? of pdb file, or maybe conformation, etc)

", "rawMarkdown": "not all will target be the same. if there are some target the same, then your code is correct.\n\ne.g. in my conversion, i have\nkaggle traget id1 = same (all residue has extact same xyz from downloaded pdb)\nkaggle traget id2 = not same (athough there is only one coformation in downloaded pdb)\n\n\n(i am not sure why they are different but it could be due to different version updated? of pdb file, or maybe conformation, etc)", "votes": 1 }, { "id": 3148632, "postDate": "2025-03-13T11:11:17.037Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 3148656, "postDate": "2025-03-13T11:37:08.440Z", "content": "

Thank you for your comment.

\n

So, train_labels.csv with only one conformation would be noisy labels.

", "rawMarkdown": "Thank you for your comment. \n\nSo, train_labels.csv with only one conformation would be noisy labels." }, { "id": 3149593, "postDate": "2025-03-14T12:23:03.693Z", "content": "

I downloaded 1rnk.pdb from here, and confirmed successful conversion.

\n

\"\"

", "rawMarkdown": "I downloaded 1rnk.pdb from [here](https://www.rcsb.org/structure/1rnk), and confirmed successful conversion.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3754725%2Fa58369a3497da65a7fa6d7cc31d6e149%2F__results___7_0.png?generation=1741954946178236&alt=media)" } ] } ] }, { "id": 3148387, "postDate": "2025-03-13T03:58:18.663Z", "content": "

Some of your questions are already answered here.

", "rawMarkdown": "Some of your questions are already answered [**here**](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/567539).", "votes": 2, "replies": [ { "id": 3148390, "postDate": "2025-03-13T04:06:45.270Z", "content": "

Thank you for your comment. I didn't check it.

", "rawMarkdown": "Thank you for your comment. I didn't check it." } ] }, { "id": 3160535, "postDate": "2025-03-26T22:14:10.547Z", "content": "

Thanks, this is helpful to me as well.

", "rawMarkdown": "Thanks, this is helpful to me as well.", "votes": 12 } ], "comments": [ { "id": 3148517, "author_name": "hengck23", "author_url": "", "post_date": "2025-03-13T08:13:36.307000", "content": "

i did not check your code in details but:

\n
    \n
  1. we only need C1 atom (we predict backbone structure only)
    2. \n
  2. resname and resid should be extracted from pdb file (and not running number)
    3. \n
\n

while it is safe to assume that there is only one strcuture and the chain is continous for synthetic data, it may be better to use assert check in your code for these 2 points.

\n

finally, you can verify your conversion code is correct or not by:

\n", "votes": 4, "replies": [ { "id": 3148520, "author_name": "hengck23", "author_url": "", "post_date": "2025-03-13T08:20:55.750000", "content": "
def parse_pdb_to_df(pdb_file, target_id):\n    parser = PDBParser()\n\n    structure = parser.get_structure('', pdb_file)\n\n    df=[]\n    for model in structure:\n        for chain in model:\n            print(chain)\n            chain_data = []\n            for residue in chain:\n                 print(residue)\n                if residue.get_resname() in ['A', 'U', 'G', 'C']:\n\n                     Check if the residue has a C1' atom\n                    if 'C1\\'' in residue:\n                        atom = residue['C1\\'']\n                        xyz = atom.get_coord()\n                        resname = residue.get_resname()\n                        resid = residue.get_id()[1]\n\n                            resname resid   x_1 y_1 z_1\n                         detect discontinous: resid = previous resid +1\n                        chain_data.append(dict(\n                            ID=target_id+'_'+str(resid),  change to target_id+'_'+str(chainid)+'_'+str(resid)\n                            resname=resname,\n                            resid=resid,\n                            x_1=xyz[0],\n                            y_1=xyz[1],\n                            z_1=xyz[2],\n                        ))\n                        \n\n            if len(chain_data)!=0:\n                chain_df = pd.DataFrame(chain_data)\n                df.append(chain_df)\n                \n    return df\n
", "votes": 4, "replies": [ { "id": 3148580, "author_name": "tomoo inubushi", "author_url": "", "post_date": "2025-03-13T10:05:33.610000", "content": "

Thank you for your comment.
\nI am now rewriting my conversion code.

\n

I downloaded 1scl.pdb from here and uploaded to here, and compared the data.

\n

I found that the sequences of these data are the same, but x, y, and z coordinates are different.

\n

According to 1scl.pdb, x, y, and z of first c1' are 11.334, -27.307, -2.027.
\nOn the other hand, according to 1SCL_A data in train_labels.csv, x, y, and z are 13.760, -25.974001, 0.102.
\nDo anybody give me a suggestion? Maybe I downloaded the wrong pdb file.

\n

1SCL_A data in train_labels.csv

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
IDresnameresidx_1y_1z_1pdb_id
1SCL_A_1G113.760-25.9740010.1021SCL_A
1SCL_A_2G29.310-29.6380002.6691SCL_A
1SCL_A_3G35.529-27.8130005.8781SCL_A
1SCL_A_4U42.678-24.9009999.7931SCL_A
1SCL_A_5G51.827-20.13600011.7931SCL_A
\n

The result of my current script from 1scl.pdb

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
resnameresidx_1y_1z_1
G111.334-27.306999-2.027
G26.168-29.6390000.850
G33.555-27.5289995.061
U41.614-24.3379999.446
G52.020-19.66700011.685
\n

\"\"

", "votes": 0, "replies": [] }, { "id": 3148583, "author_name": "hengck23", "author_url": "", "post_date": "2025-03-13T10:09:45.403000", "content": "

not all will target be the same. if there are some target the same, then your code is correct.

\n

e.g. in my conversion, i have
\nkaggle traget id1 = same (all residue has extact same xyz from downloaded pdb)
\nkaggle traget id2 = not same (athough there is only one coformation in downloaded pdb)

\n

(i am not sure why they are different but it could be due to different version updated? of pdb file, or maybe conformation, etc)

", "votes": 1, "replies": [] }, { "id": 3148632, "author_name": "", "author_url": "", "post_date": "2025-03-13T11:11:17.037000", "content": "", "votes": 1, "replies": [] }, { "id": 3148656, "author_name": "tomoo inubushi", "author_url": "", "post_date": "2025-03-13T11:37:08.440000", "content": "

Thank you for your comment.

\n

So, train_labels.csv with only one conformation would be noisy labels.

", "votes": 0, "replies": [] }, { "id": 3149593, "author_name": "tomoo inubushi", "author_url": "", "post_date": "2025-03-14T12:23:03.693000", "content": "

I downloaded 1rnk.pdb from here, and confirmed successful conversion.

\n

\"\"

", "votes": 0, "replies": [] } ] } ] }, { "id": 3148387, "author_name": "Tilii", "author_url": "", "post_date": "2025-03-13T03:58:18.663000", "content": "

Some of your questions are already answered here.

", "votes": 2, "replies": [ { "id": 3148390, "author_name": "tomoo inubushi", "author_url": "", "post_date": "2025-03-13T04:06:45.270000", "content": "

Thank you for your comment. I didn't check it.

", "votes": 0, "replies": [] } ] }, { "id": 3160535, "author_name": "", "author_url": "", "post_date": "2025-03-26T22:14:10.547000", "content": "

Thanks, this is helpful to me as well.

", "votes": 12, "replies": [] } ], "raw_markdown_by_id": { "3148328": "The host shared [synthetic RNA structure dataset](https://www.kaggle.com/datasets/andrewfavor/uw-synthetic-rna-final) including 447,402 RNA sequences.\n\nI converted this dataset into the same format of this competition. You can download it from [here](https://www.kaggle.com/datasets/tomooinubushi/converted-uw-synthetic-rna-final).\nThe conversion script is [here](https://www.kaggle.com/code/tomooinubushi/convert-uw-synthetic-dataset).\n\nI am not sure it works for this competition. I appreciate if anyone share us the result.\n\np.s.\nDear @andrewfavor \n- ~~What is the difference between [uw-synthetic-rna-final](https://www.kaggle.com/datasets/andrewfavor/uw-synthetic-rna-final) and [uw-synthetic-rna-structures](https://www.kaggle.com/datasets/andrewfavor/uw-synthetic-rna-structures) ?~~ Already answered in [here](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/567539). They are identical.\n- ~~Should we concatenate these datasets? or just use final?~~ Already answered in [here](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/567539). We should just use final version.\n- What are the licenses of these datasets? Is my dataset OK?\n\n\n**[UPDATE 20240314]** I rewrited my conversion script and reuploaded the dataset. Please use the latest version from [here](https://www.kaggle.com/datasets/tomooinubushi/converted-uw-synthetic-rna-final)\nI also converted [Stanford Ribonanza RNA Folding](https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data) ([data description](https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/discussion/451853)) including 124k RNAs. The conversion script is [here](https://www.kaggle.com/code/tomooinubushi/convert-stanford-ribonanza-rna-folding-dataset).", "3148517": "i did not check your code in details but:\n1. we only need C1 atom (we predict backbone structure only)\n2. resname and resid should be extracted from pdb file (and not running number)\n\nwhile it is safe to assume that there is only one strcuture and the chain is continous for synthetic data, it may be better to use assert check in your code for these 2 points.\n\nfinally, you can verify your conversion code is correct or not by:\n- choose a pdb id from kaggle train or validation csv\n- download pdb file from protein bank\n- compare your conversion results with that from the given kaggle label csv", "3148387": "Some of your questions are already answered [**here**](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/567539).", "3160535": "Thanks, this is helpful to me as well." } }