{ "id": 568012, "title": "Previous Kaggle Competitions and Winning Solutions", "url": "/competitions/stanford-rna-3d-folding/discussion/568012", "author_name": "Younus_Mohamed", "post_date": "2025-03-13T10:53:26.704000", "votes": 8, "comment_count": 0, "views": 0, "content": "

1) Open Problems - Multimodal Single-Cell Integration
\nGoal: Predict how DNA, RNA, and protein measurements co-vary during human blood cell development. The main challenge is to model one modality from another (e.g., gene expression from chromatin accessibility) across different time points in single-cell data.

\n\n
\n

2) Stanford Ribonanza RNA Folding
\nGoal: Create a model that accurately predicts RNA 3D structure by inferring chemical reactivity profiles (DMS_MaP and 2A3_MaP) for each nucleotide. This addresses fundamental questions about RNA folding and aids in developing better RNA-based medicines.

\n

What is Ribonanza?
\nRibonanza is a large-scale project and dataset aimed at understanding RNA structure through chemical mapping measurements. Organized by the Das lab and the Eterna community, it compiles chemical reactivity data (like DMS_MaP and 2A3_MaP) across synthetic and natural RNAs. Researchers leverage these data to build models predicting RNA folding and reactivity, bridging a key gap in RNA structure prediction and potentially revolutionizing RNA-based therapeutics.

\n\n

Overall, these previous competitions demonstrate the importance of domain-driven data preprocessing, creative neural network architectures, and ensembling methods that push the boundaries of biology-related data science challenges.

\n

Hope this helps. Happy Kaggling!

", "messages": [ { "id": 3148624, "postDate": "2025-03-13T10:53:26.703Z", "content": "

1) Open Problems - Multimodal Single-Cell Integration
\nGoal: Predict how DNA, RNA, and protein measurements co-vary during human blood cell development. The main challenge is to model one modality from another (e.g., gene expression from chromatin accessibility) across different time points in single-cell data.

\n\n
\n

2) Stanford Ribonanza RNA Folding
\nGoal: Create a model that accurately predicts RNA 3D structure by inferring chemical reactivity profiles (DMS_MaP and 2A3_MaP) for each nucleotide. This addresses fundamental questions about RNA folding and aids in developing better RNA-based medicines.

\n

What is Ribonanza?
\nRibonanza is a large-scale project and dataset aimed at understanding RNA structure through chemical mapping measurements. Organized by the Das lab and the Eterna community, it compiles chemical reactivity data (like DMS_MaP and 2A3_MaP) across synthetic and natural RNAs. Researchers leverage these data to build models predicting RNA folding and reactivity, bridging a key gap in RNA structure prediction and potentially revolutionizing RNA-based therapeutics.

\n\n

Overall, these previous competitions demonstrate the importance of domain-driven data preprocessing, creative neural network architectures, and ensembling methods that push the boundaries of biology-related data science challenges.

\n

Hope this helps. Happy Kaggling!

", "rawMarkdown": "**1) Open Problems - Multimodal Single-Cell Integration** \n**Goal:** Predict how DNA, RNA, and protein measurements co-vary during human blood cell development. The main challenge is to model one modality from another (e.g., gene expression from chromatin accessibility) across different time points in single-cell data.\n\n- **1st Place Solution** \n - **Multiome:** Used a tSVD-based imputation approach and ML models to correlate ATAC-seq signals with gene expression. \n - **CITEseq:** Employed correlation-based feature selection and built an ensemble of neural network models and LightGBM to predict protein levels from gene expression.\n\n- **2nd Place Solution** \n - **Data Preprocessing:** Centered log-ratio (CLR) transformation, feature correlation filtering, and partial fine-tuning for both Multiome and CITEseq. \n - **Models:** Combined LightGBM with variants of raw/normalized inputs, plus a final meta-model using a GRU or MLP for improved predictions.\n\n- **3rd Place Solution** \n - **Multiome:** Leveraged TF-IDF or Okapi BM25 transformations on ATAC, followed by SVD and W2V-based features in MLP or CatBoost. \n - **CITEseq:** Also used W2V vectorization for top genes, cluster-based features, and multiple MLP+CatBoost ensembles to refine final outputs.\n\n---\n\n**2) Stanford Ribonanza RNA Folding** \n**Goal:** Create a model that accurately predicts RNA 3D structure by inferring chemical reactivity profiles (DMS_MaP and 2A3_MaP) for each nucleotide. This addresses fundamental questions about RNA folding and aids in developing better RNA-based medicines.\n\n**What is Ribonanza?** \nRibonanza is a large-scale project and dataset aimed at understanding RNA structure through chemical mapping measurements. Organized by the Das lab and the Eterna community, it compiles chemical reactivity data (like DMS_MaP and 2A3_MaP) across synthetic and natural RNAs. Researchers leverage these data to build models predicting RNA folding and reactivity, bridging a key gap in RNA structure prediction and potentially revolutionizing RNA-based therapeutics.\n\n- **1st Place Solution** \n - **Transformer + Conv2D**: Integrated base pair probability matrices (BPPMs) with a convolutional block and used them as attention biases. \n - **Dynamic Positional Bias**: Added to handle sequences well beyond training lengths. \n - **Final Ensemble**: 27 sub-models with minor variations to tackle diverse RNA lengths and contexts.\n\n- **2nd Place Solution** \n - **Squeezeformer**: Adapted from speech recognition, layering a 2D CNN on BPPMs as an attention bias. \n - **ALiBi Positional Encoding**: Maintains extrapolation capability for longer RNAs. \n - **Weighted Loss**: Combined MAE with signal-to-noise weighting for robust training.\n\n- **3rd Place Solution** \n - **Twin Model Approach**: A smaller Squeezeformer-based model plus a larger “twin-tower” alpha-fold–inspired model, each focusing on different aspects of structure representation. \n - **Partially Emulated AlphaFold**: MSA-like single representation with pair representation for advanced 3D context, though not fully realized in the final submission. \n - **Synthetic Data**: Augmented low SNR data with model-estimated confidence, boosting coverage and performance for the smaller model.\n\nOverall, these previous competitions demonstrate the importance of domain-driven data preprocessing, creative neural network architectures, and ensembling methods that push the boundaries of biology-related data science challenges.\n\nHope this helps. Happy Kaggling!\n", "votes": 8 } ], "comments": [], "raw_markdown_by_id": { "3148624": "**1) Open Problems - Multimodal Single-Cell Integration** \n**Goal:** Predict how DNA, RNA, and protein measurements co-vary during human blood cell development. The main challenge is to model one modality from another (e.g., gene expression from chromatin accessibility) across different time points in single-cell data.\n\n- **1st Place Solution** \n - **Multiome:** Used a tSVD-based imputation approach and ML models to correlate ATAC-seq signals with gene expression. \n - **CITEseq:** Employed correlation-based feature selection and built an ensemble of neural network models and LightGBM to predict protein levels from gene expression.\n\n- **2nd Place Solution** \n - **Data Preprocessing:** Centered log-ratio (CLR) transformation, feature correlation filtering, and partial fine-tuning for both Multiome and CITEseq. \n - **Models:** Combined LightGBM with variants of raw/normalized inputs, plus a final meta-model using a GRU or MLP for improved predictions.\n\n- **3rd Place Solution** \n - **Multiome:** Leveraged TF-IDF or Okapi BM25 transformations on ATAC, followed by SVD and W2V-based features in MLP or CatBoost. \n - **CITEseq:** Also used W2V vectorization for top genes, cluster-based features, and multiple MLP+CatBoost ensembles to refine final outputs.\n\n---\n\n**2) Stanford Ribonanza RNA Folding** \n**Goal:** Create a model that accurately predicts RNA 3D structure by inferring chemical reactivity profiles (DMS_MaP and 2A3_MaP) for each nucleotide. This addresses fundamental questions about RNA folding and aids in developing better RNA-based medicines.\n\n**What is Ribonanza?** \nRibonanza is a large-scale project and dataset aimed at understanding RNA structure through chemical mapping measurements. Organized by the Das lab and the Eterna community, it compiles chemical reactivity data (like DMS_MaP and 2A3_MaP) across synthetic and natural RNAs. Researchers leverage these data to build models predicting RNA folding and reactivity, bridging a key gap in RNA structure prediction and potentially revolutionizing RNA-based therapeutics.\n\n- **1st Place Solution** \n - **Transformer + Conv2D**: Integrated base pair probability matrices (BPPMs) with a convolutional block and used them as attention biases. \n - **Dynamic Positional Bias**: Added to handle sequences well beyond training lengths. \n - **Final Ensemble**: 27 sub-models with minor variations to tackle diverse RNA lengths and contexts.\n\n- **2nd Place Solution** \n - **Squeezeformer**: Adapted from speech recognition, layering a 2D CNN on BPPMs as an attention bias. \n - **ALiBi Positional Encoding**: Maintains extrapolation capability for longer RNAs. \n - **Weighted Loss**: Combined MAE with signal-to-noise weighting for robust training.\n\n- **3rd Place Solution** \n - **Twin Model Approach**: A smaller Squeezeformer-based model plus a larger “twin-tower” alpha-fold–inspired model, each focusing on different aspects of structure representation. \n - **Partially Emulated AlphaFold**: MSA-like single representation with pair representation for advanced 3D context, though not fully realized in the final submission. \n - **Synthetic Data**: Augmented low SNR data with model-estimated confidence, boosting coverage and performance for the smaller model.\n\nOverall, these previous competitions demonstrate the importance of domain-driven data preprocessing, creative neural network architectures, and ensembling methods that push the boundaries of biology-related data science challenges.\n\nHope this helps. Happy Kaggling!\n" } }