Dear @rhijudas das and @shujun717,
\nThank you again for organizing this exciting challenge. I’d like to revisit a question I asked earlier regarding validation, which I believe is still unclear and important for the fairness and consistency of the evaluation.
\nIn the validation_sequences.csv and validation_labels.csv files, I noticed that 2 target IDs, R1189 and R1190, have an identical RNA sequence but are treated as separate targets:
R1189 (7YR7_1) is associated with 6 chains including CsrA protein,
R1190 (7YR6_1) is associated with 4 chains of CsrA.
My questions:
\nWhy were these considered distinct target IDs, rather than treating them as the same target with multiple possible 3D conformations (i.e., multiple valid ground truths for the same sequence)?
If the rationale is based on the accompanying all_sequences (which lists macromolecule chains), how can we distinguish cases where RNA is co-crystallized with small molecules, ions, or ligands, since these are not included in all_sequences?
As previously raised in this discussion , small molecules and ions can significantly influence RNA conformation, yet we currently lack the input features needed to account for them.
\nSuggestion:
\nWould it be possible to merge all structures with the same RNA sequence into a single target ID, and consider all of their 3D conformations as valid ground truths?
\nThis could reduce ambiguity during testing and better reflect the biological reality where a single RNA sequence may adopt multiple conformations depending on context (e.g., protein, ion binding).
Thank you for considering this. Looking forward to your thoughts!
", "messages": [ { "id": "3195371", "postDate": "05/07/2025 00:45:12", "content": "Dear @rhijudas das and @shujun717,
\nThank you again for organizing this exciting challenge. I’d like to revisit a question I asked earlier regarding validation, which I believe is still unclear and important for the fairness and consistency of the evaluation.
\nIn the validation_sequences.csv and validation_labels.csv files, I noticed that 2 target IDs, R1189 and R1190, have an identical RNA sequence but are treated as separate targets:
R1189 (7YR7_1) is associated with 6 chains including CsrA protein,
R1190 (7YR6_1) is associated with 4 chains of CsrA.
My questions:
\nWhy were these considered distinct target IDs, rather than treating them as the same target with multiple possible 3D conformations (i.e., multiple valid ground truths for the same sequence)?
If the rationale is based on the accompanying all_sequences (which lists macromolecule chains), how can we distinguish cases where RNA is co-crystallized with small molecules, ions, or ligands, since these are not included in all_sequences?
As previously raised in this discussion , small molecules and ions can significantly influence RNA conformation, yet we currently lack the input features needed to account for them.
\nSuggestion:
\nWould it be possible to merge all structures with the same RNA sequence into a single target ID, and consider all of their 3D conformations as valid ground truths?
\nThis could reduce ambiguity during testing and better reflect the biological reality where a single RNA sequence may adopt multiple conformations depending on context (e.g., protein, ion binding).
Thank you for considering this. Looking forward to your thoughts!
", "rawMarkdown": "Dear @rhijudas das and @shujun717,\n\nThank you again for organizing this exciting challenge. I’d like to revisit [a question I asked earlier](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/571324) regarding validation, which I believe is still unclear and important for the fairness and consistency of the evaluation.\n\nIn the `validation_sequences.csv` and `validation_labels.csv` files, I noticed that 2 target IDs, R1189 and R1190, have an identical RNA sequence but are treated as separate targets:\n\n* R1189 (7YR7_1) is associated with 6 chains including CsrA protein,\n\n* R1190 (7YR6_1) is associated with 4 chains of CsrA.\n\nMy questions:\n* Why were these considered distinct target IDs, rather than treating them as the same target with multiple possible 3D conformations (i.e., multiple valid ground truths for the same sequence)?\n\n* If the rationale is based on the accompanying `all_sequences` (which lists macromolecule chains), how can we distinguish cases where RNA is co-crystallized with small molecules, ions, or ligands, since these are not included in `all_sequences`?\n\nAs previously raised in [this discussion](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/571324) , small molecules and ions can significantly influence RNA conformation, yet we currently lack the input features needed to account for them.\n\n**Suggestion**:\nWould it be possible to merge all structures with the same RNA sequence into a single target ID, and consider all of their 3D conformations as valid ground truths?\nThis could reduce ambiguity during testing and better reflect the biological reality where a single RNA sequence may adopt multiple conformations depending on context (e.g., protein, ion binding).\n\nThank you for considering this. Looking forward to your thoughts!", "votes": null }, { "id": "3196838", "postDate": "05/07/2025 13:39:33", "content": "Thanks for the post!
\nWe included these duplicate sequences TR1189 and TR1190 in the host provided validation because they were indeed separate targets in CASP15 and have become a widely used benchmark in the field, including for development of approaches that can simulate the protein interactions that are described in all_sequences (or ligand information or human readable
\nInformation in the description field).
However for the hidden test set and for the future targets for the current competition, we will remove targets that have duplicates in the main RNA sequence field.
\nIt’s a great idea to also curate training data sets to bring together duplicates or near-duplicates as alternative labels for a target sequence.
\nIf you or others have seen a positive effect on your models from such aggregation of labels, please post and we will considering releasing supplemental files with that rearrangement for the community, or linking to your alternative data set. Please also see the post from @ckjoshi9: https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/576556
", "rawMarkdown": "Thanks for the post! \n\nWe included these duplicate sequences TR1189 and TR1190 in the host provided validation because they were indeed separate targets in CASP15 and have become a widely used benchmark in the field, including for development of approaches that can simulate the protein interactions that are described in `all_sequences` (or ligand information or human readable\nInformation in the description field).\n\nHowever for the hidden test set and for the future targets for the current competition, we will remove targets that have duplicates in the main RNA sequence field.\n\nIt’s a great idea to also curate training data sets to bring together duplicates or near-duplicates as alternative labels for a target sequence.\n\n If you or others have seen a positive effect on your models from such aggregation of labels, please post and we will considering releasing supplemental files with that rearrangement for the community, or linking to your alternative data set. Please also see the post from @ckjoshi9: https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/576556", "votes": null }, { "id": "3196898", "postDate": "05/07/2025 15:05:02", "content": "Thanks for your clarification and the link to the useful post.
", "rawMarkdown": "Thanks for your clarification and the link to the useful post.", "votes": null } ], "comments": [ { "id": 3196838, "author_name": "rhijudas", "author_url": "", "post_date": "05/07/2025 13:39:33", "content": "Thanks for the post!
\nWe included these duplicate sequences TR1189 and TR1190 in the host provided validation because they were indeed separate targets in CASP15 and have become a widely used benchmark in the field, including for development of approaches that can simulate the protein interactions that are described in all_sequences (or ligand information or human readable
\nInformation in the description field).
However for the hidden test set and for the future targets for the current competition, we will remove targets that have duplicates in the main RNA sequence field.
\nIt’s a great idea to also curate training data sets to bring together duplicates or near-duplicates as alternative labels for a target sequence.
\nIf you or others have seen a positive effect on your models from such aggregation of labels, please post and we will considering releasing supplemental files with that rearrangement for the community, or linking to your alternative data set. Please also see the post from @ckjoshi9: https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/576556
", "votes": null, "replies": [ { "id": 3196898, "author_name": "nguyenhoa", "author_url": "", "post_date": "05/07/2025 15:05:02", "content": "Thanks for your clarification and the link to the useful post.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3195371": "Dear @rhijudas das and @shujun717,\n\nThank you again for organizing this exciting challenge. I’d like to revisit [a question I asked earlier](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/571324) regarding validation, which I believe is still unclear and important for the fairness and consistency of the evaluation.\n\nIn the `validation_sequences.csv` and `validation_labels.csv` files, I noticed that 2 target IDs, R1189 and R1190, have an identical RNA sequence but are treated as separate targets:\n\n* R1189 (7YR7_1) is associated with 6 chains including CsrA protein,\n\n* R1190 (7YR6_1) is associated with 4 chains of CsrA.\n\nMy questions:\n* Why were these considered distinct target IDs, rather than treating them as the same target with multiple possible 3D conformations (i.e., multiple valid ground truths for the same sequence)?\n\n* If the rationale is based on the accompanying `all_sequences` (which lists macromolecule chains), how can we distinguish cases where RNA is co-crystallized with small molecules, ions, or ligands, since these are not included in `all_sequences`?\n\nAs previously raised in [this discussion](https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/571324) , small molecules and ions can significantly influence RNA conformation, yet we currently lack the input features needed to account for them.\n\n**Suggestion**:\nWould it be possible to merge all structures with the same RNA sequence into a single target ID, and consider all of their 3D conformations as valid ground truths?\nThis could reduce ambiguity during testing and better reflect the biological reality where a single RNA sequence may adopt multiple conformations depending on context (e.g., protein, ion binding).\n\nThank you for considering this. Looking forward to your thoughts!", "3196838": "Thanks for the post! \n\nWe included these duplicate sequences TR1189 and TR1190 in the host provided validation because they were indeed separate targets in CASP15 and have become a widely used benchmark in the field, including for development of approaches that can simulate the protein interactions that are described in `all_sequences` (or ligand information or human readable\nInformation in the description field).\n\nHowever for the hidden test set and for the future targets for the current competition, we will remove targets that have duplicates in the main RNA sequence field.\n\nIt’s a great idea to also curate training data sets to bring together duplicates or near-duplicates as alternative labels for a target sequence.\n\n If you or others have seen a positive effect on your models from such aggregation of labels, please post and we will considering releasing supplemental files with that rearrangement for the community, or linking to your alternative data set. Please also see the post from @ckjoshi9: https://www.kaggle.com/competitions/stanford-rna-3d-folding/discussion/576556", "3196898": "Thanks for your clarification and the link to the useful post." }, "source": "meta" }