I appreciate the organizers for creating this competition and the effort invested in preparing the dataset. However, after several days of experimenting with classical ML and mechanistic immunology models, I realized that the challenge, as currently structured, does not align with how adaptive immune repertoires actually encode information. Real immune responses depend on clonal expansion, contraction, affinity maturation, and memory dynamics, but the dataset contains only static bags of sequences with no temporal or clonal context. This forces models to treat repertoires like arbitrary text documents rather than dynamic biological systems, and the disease signal becomes extremely weak compared to the overwhelming repertoire noise.
\nMoreover, identifying tens of thousands of “important sequences” per dataset has no biological validity, real diagnostic signatures involve a handful of high-confidence clones, not 50,000 items sorted by ML coefficients. Combined with severe cross-dataset heterogeneity (depth, preprocessing, metadata gaps), the task becomes a statistical exercise rather than a biologically meaningful one. For these reasons, I’m stepping back. I genuinely hope future iterations incorporate temporal sampling, clonal abundances, and richer metadata so the community can explore immune repertoire biology in a scientifically grounded way
", "messages": [ { "id": "3359320", "postDate": "12/01/2025 17:51:21", "content": "I appreciate the organizers for creating this competition and the effort invested in preparing the dataset. However, after several days of experimenting with classical ML and mechanistic immunology models, I realized that the challenge, as currently structured, does not align with how adaptive immune repertoires actually encode information. Real immune responses depend on clonal expansion, contraction, affinity maturation, and memory dynamics, but the dataset contains only static bags of sequences with no temporal or clonal context. This forces models to treat repertoires like arbitrary text documents rather than dynamic biological systems, and the disease signal becomes extremely weak compared to the overwhelming repertoire noise.
\nMoreover, identifying tens of thousands of “important sequences” per dataset has no biological validity, real diagnostic signatures involve a handful of high-confidence clones, not 50,000 items sorted by ML coefficients. Combined with severe cross-dataset heterogeneity (depth, preprocessing, metadata gaps), the task becomes a statistical exercise rather than a biologically meaningful one. For these reasons, I’m stepping back. I genuinely hope future iterations incorporate temporal sampling, clonal abundances, and richer metadata so the community can explore immune repertoire biology in a scientifically grounded way
", "rawMarkdown": "I appreciate the organizers for creating this competition and the effort invested in preparing the dataset. However, after several days of experimenting with classical ML and mechanistic immunology models, I realized that the challenge, as currently structured, does not align with how adaptive immune repertoires actually encode information. Real immune responses depend on clonal expansion, contraction, affinity maturation, and memory dynamics, but the dataset contains only static bags of sequences with no temporal or clonal context. This forces models to treat repertoires like arbitrary text documents rather than dynamic biological systems, and the disease signal becomes extremely weak compared to the overwhelming repertoire noise.\n\nMoreover, identifying tens of thousands of “important sequences” per dataset has no biological validity, real diagnostic signatures involve a handful of high-confidence clones, not 50,000 items sorted by ML coefficients. Combined with severe cross-dataset heterogeneity (depth, preprocessing, metadata gaps), the task becomes a statistical exercise rather than a biologically meaningful one. For these reasons, I’m stepping back. I genuinely hope future iterations incorporate temporal sampling, clonal abundances, and richer metadata so the community can explore immune repertoire biology in a scientifically grounded way", "votes": null }, { "id": "3360112", "postDate": "12/01/2025 21:36:21", "content": "We agree that there are indeed many different settings that could be interesting to explore in a benchmark, including settings with temporal sampling that could allow to infer e.g. clonal expansion, contraction, affinity maturation, and memory dynamics. Different settings will require different methodologies, where mechanistic immunology models will be suited to some. We are considering the setting of large-scale association studies, with a single sample from individuals of different cohorts, similar to e.g. GWAS and EWAS studies for genetics and epigenetics. This setting we explore has been considered many times in the literature. The purpose of this challenge is to learn about which methods are best at learning to discriminate between individuals in this cohort setting. We also aim to learn to what degree state-of-the-art methodologies are able to extract the underlying signals (receptors). These signals are not necessarily 50K per training dataset. We described in the peer-reviewed pre-registered report linked to from the competition webpage, and also clarified in another discussion thread, why we ask for 50K important sequences. Instead of asking for a different number of sequences per training dataset, we uniformly ask to submit a top 50,000 ranked list per each training dataset. This is just a simple way to ensure one has submitted enough sequences to cover the evaluation. Please see this thread for more detailed explanation on that.
", "rawMarkdown": "We agree that there are indeed many different settings that could be interesting to explore in a benchmark, including settings with temporal sampling that could allow to infer e.g. clonal expansion, contraction, affinity maturation, and memory dynamics. Different settings will require different methodologies, where mechanistic immunology models will be suited to some. We are considering the setting of large-scale association studies, with a single sample from individuals of different cohorts, similar to e.g. GWAS and EWAS studies for genetics and epigenetics. This setting we explore has been considered many times in the literature. The purpose of this challenge is to learn about which methods are best at learning to discriminate between individuals in this cohort setting. We also aim to learn to what degree state-of-the-art methodologies are able to extract the underlying signals (receptors). These signals are not necessarily 50K per training dataset. We described in the [peer-reviewed pre-registered report ](https://github.com/uio-bmi/adaptive_immune_profiling_challenge_2025/blob/main/registered_report.pdf) linked to from the competition webpage, and also clarified in another [discussion thread](https://www.kaggle.com/competitions/adaptive-immune-profiling-challenge-2025/discussion/617673), why we ask for 50K important sequences. Instead of asking for a different number of sequences per training dataset, we uniformly ask to submit a **top 50,000 ranked list** per each training dataset. This is just a simple way to ensure one has submitted ***enough*** sequences to cover the evaluation. Please see [this thread](https://www.kaggle.com/competitions/adaptive-immune-profiling-challenge-2025/discussion/617673) for more detailed explanation on that.", "votes": null } ], "comments": [ { "id": 3360112, "author_name": "ckanduri", "author_url": "", "post_date": "12/01/2025 21:36:21", "content": "We agree that there are indeed many different settings that could be interesting to explore in a benchmark, including settings with temporal sampling that could allow to infer e.g. clonal expansion, contraction, affinity maturation, and memory dynamics. Different settings will require different methodologies, where mechanistic immunology models will be suited to some. We are considering the setting of large-scale association studies, with a single sample from individuals of different cohorts, similar to e.g. GWAS and EWAS studies for genetics and epigenetics. This setting we explore has been considered many times in the literature. The purpose of this challenge is to learn about which methods are best at learning to discriminate between individuals in this cohort setting. We also aim to learn to what degree state-of-the-art methodologies are able to extract the underlying signals (receptors). These signals are not necessarily 50K per training dataset. We described in the peer-reviewed pre-registered report linked to from the competition webpage, and also clarified in another discussion thread, why we ask for 50K important sequences. Instead of asking for a different number of sequences per training dataset, we uniformly ask to submit a top 50,000 ranked list per each training dataset. This is just a simple way to ensure one has submitted enough sequences to cover the evaluation. Please see this thread for more detailed explanation on that.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3359320": "I appreciate the organizers for creating this competition and the effort invested in preparing the dataset. However, after several days of experimenting with classical ML and mechanistic immunology models, I realized that the challenge, as currently structured, does not align with how adaptive immune repertoires actually encode information. Real immune responses depend on clonal expansion, contraction, affinity maturation, and memory dynamics, but the dataset contains only static bags of sequences with no temporal or clonal context. This forces models to treat repertoires like arbitrary text documents rather than dynamic biological systems, and the disease signal becomes extremely weak compared to the overwhelming repertoire noise.\n\nMoreover, identifying tens of thousands of “important sequences” per dataset has no biological validity, real diagnostic signatures involve a handful of high-confidence clones, not 50,000 items sorted by ML coefficients. Combined with severe cross-dataset heterogeneity (depth, preprocessing, metadata gaps), the task becomes a statistical exercise rather than a biologically meaningful one. For these reasons, I’m stepping back. I genuinely hope future iterations incorporate temporal sampling, clonal abundances, and richer metadata so the community can explore immune repertoire biology in a scientifically grounded way", "3360112": "We agree that there are indeed many different settings that could be interesting to explore in a benchmark, including settings with temporal sampling that could allow to infer e.g. clonal expansion, contraction, affinity maturation, and memory dynamics. Different settings will require different methodologies, where mechanistic immunology models will be suited to some. We are considering the setting of large-scale association studies, with a single sample from individuals of different cohorts, similar to e.g. GWAS and EWAS studies for genetics and epigenetics. This setting we explore has been considered many times in the literature. The purpose of this challenge is to learn about which methods are best at learning to discriminate between individuals in this cohort setting. We also aim to learn to what degree state-of-the-art methodologies are able to extract the underlying signals (receptors). These signals are not necessarily 50K per training dataset. We described in the [peer-reviewed pre-registered report ](https://github.com/uio-bmi/adaptive_immune_profiling_challenge_2025/blob/main/registered_report.pdf) linked to from the competition webpage, and also clarified in another [discussion thread](https://www.kaggle.com/competitions/adaptive-immune-profiling-challenge-2025/discussion/617673), why we ask for 50K important sequences. Instead of asking for a different number of sequences per training dataset, we uniformly ask to submit a **top 50,000 ranked list** per each training dataset. This is just a simple way to ensure one has submitted ***enough*** sequences to cover the evaluation. Please see [this thread](https://www.kaggle.com/competitions/adaptive-immune-profiling-challenge-2025/discussion/617673) for more detailed explanation on that." }, "source": "meta" }