I read with interest the background and motivation behind this competition, and I have to say there is one thing that I am missing, which is - why should we care about the x-y-z position of these particles?
\nIt seems like the competition hosts are asking participants to precisely identify the x-y-z positions of biological particles including proteins, viruses, and ribosomes. I've been working with biological data for 20 years, and I can't think of an application where I needed to know the position of a biological particle in an image. What is typically more relevant is the count (and by extension, concentration) and object-specific properties like diameter, circularity, and others. I can't remember anyone stopping by my desk and proclaiming with excitement: \"there is a nucleus at position 50x, 40y, 44z isn't that amazing?!\". And even if I knew the position of all proteins in a sample, I wouldn't know what to do with that information.
\nBy extension, suppose someone had the ground truth positions and simply shifted them in one dimension by X distance, where X is beyond the acceptable error of this competition. Then their algorithm would have 0% recall, despite getting the count exactly correct. Personally I would still consider this a win because I can see how accurate counts (and concentrations) can be actionable.
\nSo mainly for my own education, it would be great to understand why we are focused on position and not counts. Thanks for your time and for hosting this fascinating competition.
", "messages": [ { "id": "3039920", "postDate": "11/08/2024 14:15:05", "content": "I read with interest the background and motivation behind this competition, and I have to say there is one thing that I am missing, which is - why should we care about the x-y-z position of these particles?
\nIt seems like the competition hosts are asking participants to precisely identify the x-y-z positions of biological particles including proteins, viruses, and ribosomes. I've been working with biological data for 20 years, and I can't think of an application where I needed to know the position of a biological particle in an image. What is typically more relevant is the count (and by extension, concentration) and object-specific properties like diameter, circularity, and others. I can't remember anyone stopping by my desk and proclaiming with excitement: \"there is a nucleus at position 50x, 40y, 44z isn't that amazing?!\". And even if I knew the position of all proteins in a sample, I wouldn't know what to do with that information.
\nBy extension, suppose someone had the ground truth positions and simply shifted them in one dimension by X distance, where X is beyond the acceptable error of this competition. Then their algorithm would have 0% recall, despite getting the count exactly correct. Personally I would still consider this a win because I can see how accurate counts (and concentrations) can be actionable.
\nSo mainly for my own education, it would be great to understand why we are focused on position and not counts. Thanks for your time and for hosting this fascinating competition.
", "rawMarkdown": "I read with interest the background and motivation behind this competition, and I have to say there is one thing that I am missing, which is - why should we care about the x-y-z position of these particles?\n\nIt seems like the competition hosts are asking participants to precisely identify the x-y-z positions of biological particles including proteins, viruses, and ribosomes. I've been working with biological data for 20 years, and I can't think of an application where I needed to know the position of a biological particle in an image. What is typically more relevant is the count (and by extension, concentration) and object-specific properties like diameter, circularity, and others. I can't remember anyone stopping by my desk and proclaiming with excitement: \"there is a nucleus at position 50x, 40y, 44z isn't that amazing?!\". And even if I knew the position of all proteins in a sample, I wouldn't know what to do with that information.\n\nBy extension, suppose someone had the ground truth positions and simply shifted them in one dimension by X distance, where X is beyond the acceptable error of this competition. Then their algorithm would have 0% recall, despite getting the count exactly correct. Personally I would still consider this a win because I can see how accurate counts (and concentrations) can be actionable.\n\nSo mainly for my own education, it would be great to understand why we are focused on position and not counts. Thanks for your time and for hosting this fascinating competition.", "votes": null }, { "id": "3039971", "postDate": "11/08/2024 14:49:12", "content": "Hi @josephmarturano,
\nI hope some of my colleagues can explain more of the nuances here, but there are a couple of points to make:
\n1) For most of the current work in cryoET the relevance of the x-y-z positions is about the next step of processing. One of the next steps in a cryoET workflow (after finding 1000s of particles of a given type) is to perform subtomogram averaging and reconstruct the particle's structure. We need the particle positions to find all of the subtomograms to perform this reconstruction.
\n2) It can be interesting to know the positions of some of these particles relative to the rest of the cellular architecture. \"There is a particle at 50x, 40y, and 44z\" can become extra interesting if that position is also in a nuclear membrane (e.g. being able to answer colocalization questions, what a particle looks like when it is membrane-bound, etc.)
", "rawMarkdown": "Hi @josephmarturano,\n\nI hope some of my colleagues can explain more of the nuances here, but there are a couple of points to make:\n\n1) For most of the current work in cryoET the relevance of the x-y-z positions is about the next step of processing. One of the next steps in a cryoET workflow (after finding 1000s of particles of a given type) is to perform subtomogram averaging and reconstruct the particle's structure. We need the particle positions to find all of the subtomograms to perform this reconstruction. \n\n2) It can be interesting to know the positions of some of these particles relative to the rest of the cellular architecture. \"There is a particle at 50x, 40y, and 44z\" can become extra interesting if that position is also in a nuclear membrane (e.g. being able to answer colocalization questions, what a particle looks like when it is membrane-bound, etc.)", "votes": null }, { "id": "3040041", "postDate": "11/08/2024 16:16:27", "content": "Hi @josephmarturano,
\n@kharrington gives a good summary as to why we are interested in the particle locations. If you're interested in learning more about the cryoET workflow, I recommend this introductory article on cryoET preparation and processing on the cryoET in cryoET data portal's documentation.
\nIn state-of-the-art research contexts, cryoET is used to study thin sections of frozen cells with fully preserved ultrastructure. In that context, identifying the location of different protein complexes in relation to each other is interesting in order to study protein-protein interactions and pathways within the cell. Using cryoET we can even go beyond simple localization, and also determine the molecular structure and conformation of complexes we identified.
\nFor the purpose of this challenge we created a sample with partially known composition in order to be able to provide unambiguous ground truth labels. We have intentionally designed this sample to contain proteins spanning a large size range (as encountered in cellular sections), and to contain similar background as cellular sections. Our ultimate goal is to re-use methods developed as part of this challenge on other samples, particularly those containing isolated organelles or thin cellular sections.
\nIf you want to explore cellular cryoET data to get a better idea of the capabilities of this method, I recommend taking a look at the following datasets (you can visualize them in your browser by clicking \"View Tomogram\"):
\nthanks @uermel and @kharrington, good to know we're trying to achieve nanometer and sub-namometer scale imaging resolution here.
\nIt might be helpful to highlight what insights cryoET has provided that would not be possible with other techniques (e.g., XRD, sequencing, mass spec). I read a review on cryoET (Turk & Baumeister 2020) but the insights the method uncovered were not obvious to me. There seems to be excitement about high-res in situ imaging but the connection to how it could be used to advance society or health was not immediately clear.
\nIf nothing else, thanks for making me aware of the technique.
", "rawMarkdown": "thanks @uermel and @kharrington, good to know we're trying to achieve nanometer and sub-namometer scale imaging resolution here. \n\nIt might be helpful to highlight what insights cryoET has provided that would not be possible with other techniques (e.g., XRD, sequencing, mass spec). I read a review on cryoET ([Turk & Baumeister 2020](https://pubmed.ncbi.nlm.nih.gov/33020915/)) but the insights the method uncovered were not obvious to me. There seems to be excitement about high-res in situ imaging but the connection to how it could be used to advance society or health was not immediately clear.\n\nIf nothing else, thanks for making me aware of the technique.", "votes": null } ], "comments": [ { "id": 3039971, "author_name": "kharrington", "author_url": "", "post_date": "11/08/2024 14:49:12", "content": "Hi @josephmarturano,
\nI hope some of my colleagues can explain more of the nuances here, but there are a couple of points to make:
\n1) For most of the current work in cryoET the relevance of the x-y-z positions is about the next step of processing. One of the next steps in a cryoET workflow (after finding 1000s of particles of a given type) is to perform subtomogram averaging and reconstruct the particle's structure. We need the particle positions to find all of the subtomograms to perform this reconstruction.
\n2) It can be interesting to know the positions of some of these particles relative to the rest of the cellular architecture. \"There is a particle at 50x, 40y, and 44z\" can become extra interesting if that position is also in a nuclear membrane (e.g. being able to answer colocalization questions, what a particle looks like when it is membrane-bound, etc.)
", "votes": null, "replies": [] }, { "id": 3040041, "author_name": "uermel", "author_url": "", "post_date": "11/08/2024 16:16:27", "content": "Hi @josephmarturano,
\n@kharrington gives a good summary as to why we are interested in the particle locations. If you're interested in learning more about the cryoET workflow, I recommend this introductory article on cryoET preparation and processing on the cryoET in cryoET data portal's documentation.
\nIn state-of-the-art research contexts, cryoET is used to study thin sections of frozen cells with fully preserved ultrastructure. In that context, identifying the location of different protein complexes in relation to each other is interesting in order to study protein-protein interactions and pathways within the cell. Using cryoET we can even go beyond simple localization, and also determine the molecular structure and conformation of complexes we identified.
\nFor the purpose of this challenge we created a sample with partially known composition in order to be able to provide unambiguous ground truth labels. We have intentionally designed this sample to contain proteins spanning a large size range (as encountered in cellular sections), and to contain similar background as cellular sections. Our ultimate goal is to re-use methods developed as part of this challenge on other samples, particularly those containing isolated organelles or thin cellular sections.
\nIf you want to explore cellular cryoET data to get a better idea of the capabilities of this method, I recommend taking a look at the following datasets (you can visualize them in your browser by clicking \"View Tomogram\"):
\nthanks @uermel and @kharrington, good to know we're trying to achieve nanometer and sub-namometer scale imaging resolution here.
\nIt might be helpful to highlight what insights cryoET has provided that would not be possible with other techniques (e.g., XRD, sequencing, mass spec). I read a review on cryoET (Turk & Baumeister 2020) but the insights the method uncovered were not obvious to me. There seems to be excitement about high-res in situ imaging but the connection to how it could be used to advance society or health was not immediately clear.
\nIf nothing else, thanks for making me aware of the technique.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3039920": "I read with interest the background and motivation behind this competition, and I have to say there is one thing that I am missing, which is - why should we care about the x-y-z position of these particles?\n\nIt seems like the competition hosts are asking participants to precisely identify the x-y-z positions of biological particles including proteins, viruses, and ribosomes. I've been working with biological data for 20 years, and I can't think of an application where I needed to know the position of a biological particle in an image. What is typically more relevant is the count (and by extension, concentration) and object-specific properties like diameter, circularity, and others. I can't remember anyone stopping by my desk and proclaiming with excitement: \"there is a nucleus at position 50x, 40y, 44z isn't that amazing?!\". And even if I knew the position of all proteins in a sample, I wouldn't know what to do with that information.\n\nBy extension, suppose someone had the ground truth positions and simply shifted them in one dimension by X distance, where X is beyond the acceptable error of this competition. Then their algorithm would have 0% recall, despite getting the count exactly correct. Personally I would still consider this a win because I can see how accurate counts (and concentrations) can be actionable.\n\nSo mainly for my own education, it would be great to understand why we are focused on position and not counts. Thanks for your time and for hosting this fascinating competition.", "3039971": "Hi @josephmarturano,\n\nI hope some of my colleagues can explain more of the nuances here, but there are a couple of points to make:\n\n1) For most of the current work in cryoET the relevance of the x-y-z positions is about the next step of processing. One of the next steps in a cryoET workflow (after finding 1000s of particles of a given type) is to perform subtomogram averaging and reconstruct the particle's structure. We need the particle positions to find all of the subtomograms to perform this reconstruction. \n\n2) It can be interesting to know the positions of some of these particles relative to the rest of the cellular architecture. \"There is a particle at 50x, 40y, and 44z\" can become extra interesting if that position is also in a nuclear membrane (e.g. being able to answer colocalization questions, what a particle looks like when it is membrane-bound, etc.)", "3040041": "Hi @josephmarturano,\n\n@kharrington gives a good summary as to why we are interested in the particle locations. If you're interested in learning more about the cryoET workflow, I recommend this [introductory article on cryoET preparation and processing on the cryoET](https://chanzuckerberg.github.io/cryoet-data-portal/cryoet_workflow.html#cryoet-workflow) in cryoET data portal's documentation. \n\nIn state-of-the-art research contexts, cryoET is used to study thin sections of frozen cells with fully preserved ultrastructure. In that context, identifying the location of different protein complexes in relation to each other is interesting in order to study protein-protein interactions and pathways within the cell. Using cryoET we can even go beyond simple localization, and also determine the molecular structure and conformation of complexes we identified. \n\nFor the purpose of this challenge we created a sample with partially known composition in order to be able to provide unambiguous ground truth labels. We have intentionally designed this sample to contain proteins spanning a large size range (as encountered in cellular sections), and to contain similar background as cellular sections. Our ultimate goal is to re-use methods developed as part of this challenge on other samples, particularly those containing isolated organelles or thin cellular sections. \n\nIf you want to explore cellular cryoET data to get a better idea of the capabilities of this method, I recommend taking a look at the following datasets (you can visualize them in your browser by clicking \"View Tomogram\"): \n- [Thin sections of *Chlamydomonas reinhardtii* (a unicellular algae)](https://cryoetdataportal.czscience.com/datasets/10301)\n- [Thin sections of *Saccharomyces cerevisiae* (baker's yeast)](https://cryoetdataportal.czscience.com/datasets/10007)\n- [Thin sections of *Encephalitozoon hellem* spores (a microsporidian parasite)](https://cryoetdataportal.czscience.com/datasets/10438)", "3051686": "thanks @uermel and @kharrington, good to know we're trying to achieve nanometer and sub-namometer scale imaging resolution here. \n\nIt might be helpful to highlight what insights cryoET has provided that would not be possible with other techniques (e.g., XRD, sequencing, mass spec). I read a review on cryoET ([Turk & Baumeister 2020](https://pubmed.ncbi.nlm.nih.gov/33020915/)) but the insights the method uncovered were not obvious to me. There seems to be excitement about high-res in situ imaging but the connection to how it could be used to advance society or health was not immediately clear.\n\nIf nothing else, thanks for making me aware of the technique." }, "source": "meta" }