![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10704200%2F9248ab21deb99a742777743a1f27f10e%2Fribosomes.png?generation=1731368960640935&alt=media\")
EDIT: Okay, I've kind of butchered the original post, but the point is the particles are often elongated along the z-axis as an artifact of generating the tomograms. Note the better images I've included below in the comments.
\nEDIT: Here's a link to the article referenced below by @rezaparaan (Competition Host) describing why this occurs: https://www.biorxiv.org/content/10.1101/2024.04.12.589090v2.full.pdf
\nEDIT: Here's a link to the article reference below by @uermel regarding isonet correction: https://www.nature.com/articles/s41467-022-33957-8.pdf
\nIf you transpose the volumes, or at least the one I'm looking at, it's quite apparent it's easier to pick out detail from the straight on view. Any thoughts on why is true? Is this maybe because each layer is scanned at a different time and things move between scans?
\nAlso wondering what sort of model/algorithm implications this may have. For instance, data augmentation may be more difficult / less effective.
\n
EDIT: Okay, I've kind of butchered the original post, but the point is the particles are often elongated along the z-axis as an artifact of generating the tomograms. Note the better images I've included below in the comments.
\nEDIT: Here's a link to the article referenced below by @rezaparaan (Competition Host) describing why this occurs: https://www.biorxiv.org/content/10.1101/2024.04.12.589090v2.full.pdf
\nEDIT: Here's a link to the article reference below by @uermel regarding isonet correction: https://www.nature.com/articles/s41467-022-33957-8.pdf
\nIf you transpose the volumes, or at least the one I'm looking at, it's quite apparent it's easier to pick out detail from the straight on view. Any thoughts on why is true? Is this maybe because each layer is scanned at a different time and things move between scans?
\nAlso wondering what sort of model/algorithm implications this may have. For instance, data augmentation may be more difficult / less effective.
\n
Hi @davidlist, this is an excellent observation and question, and one of the reasons working with cryoET data is hard.
\ncryoET is a limited-angle tomography method. Generally, in a TEM, projection images can only be acquired with sample tilt in a range between -65 to 65 degrees due to technical constraints of the sample and microscope. In the challenge sample, images were acquired in a range from -45 to 45 degrees, with 3 degree step, to limit electron beam induced damage to the sample.
\nAs a consequence, the 3D reconstruction suffers from the so-called \"missing wedge\" artifact, a wedge-shaped region in the fourier transform of the reconstructed tomogram that doesn't contain any information. In the real-space tomogram (i.e. the images we provided), this appears as the \"stretch\" you observed in the images along the z-axis. It's worth noting that this artifact cannot be corrected for by applying affine transformations to the tomgrams.
\nIn case of the denoised, ctf-deconvolved or raw weighted back projection tomograms, we did not apply any correction of the missing wedge artifact. The isonet-corrected tomograms were processed with a method that attempts to recover information in the missing wedge region (paper, code). As such, the observed stretch should be reduced in the isonet-corrected tomograms.
\nFor a more in-depth, mathematical explanation of the relationship of projection images, tomograms and fourier transforms of tomograms, I'd recommend reading on the projection-slice theorem.
", "rawMarkdown": "Hi @davidlist, this is an excellent observation and question, and one of the reasons working with cryoET data is hard.\n\ncryoET is a limited-angle tomography method. Generally, in a TEM, projection images can only be acquired with sample tilt in a range between -65 to 65 degrees due to technical constraints of the sample and microscope. In the challenge sample, images were acquired in a range from -45 to 45 degrees, with 3 degree step, to limit electron beam induced damage to the sample.\n\nAs a consequence, the 3D reconstruction suffers from the so-called \"missing wedge\" artifact, a wedge-shaped region in the **fourier transform** of the reconstructed tomogram that doesn't contain any information. In the real-space tomogram (i.e. the images we provided), this appears as the \"stretch\" you observed in the images along the z-axis. It's worth noting that this artifact cannot be corrected for by applying affine transformations to the tomgrams. \n\nIn case of the denoised, ctf-deconvolved or raw weighted back projection tomograms, we did not apply any correction of the missing wedge artifact. The isonet-corrected tomograms were processed with a method that attempts to recover information in the missing wedge region ([paper](https://www.nature.com/articles/s41467-022-33957-8), [code](https://github.com/IsoNet-cryoET/IsoNet)). As such, the observed stretch should be reduced in the isonet-corrected tomograms. \n\nFor a more in-depth, mathematical explanation of the relationship of projection images, tomograms and fourier transforms of tomograms, I'd recommend reading on the [projection-slice theorem](https://en.wikipedia.org/wiki/Projection-slice_theorem). ", "votes": 5, "replies": [ { "id": 3044012, "postDate": "2024-11-12T22:48:30.883Z", "content": "Great! Thanks @uermel! All of that is fantastic information. BTW, my comment regarding affine transformations was in regards to synthesizing more examples of the individual particles to assist machine learning, not to correct the stretch. If for instance, you take rotation, I suspect I could generate more ribosome examples if I limit it to around the z-axis. Anything around x, or y would cause problems.
", "rawMarkdown": "Great! Thanks @uermel! All of that is fantastic information. BTW, my comment regarding affine transformations was in regards to synthesizing more examples of the individual particles to assist machine learning, not to correct the stretch. If for instance, you take rotation, I suspect I could generate more ribosome examples if I limit it to around the z-axis. Anything around x, or y would cause problems." }, { "id": 3079619, "postDate": "2024-12-23T22:57:14.453Z", "content": "Thanks @uermel - you mention isonet-corrected data should improve this effect but it doesn't seem to do much to @davidlist example images, i.e.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10813%2F58e9682a6881a398e1bf70aa1e7399bd%2FScreenshot%202024-12-24%20005458.png?generation=1734994595650122&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10813%2Fa5102370ad99953c5a0d283dc7dda4ef%2FScreenshot%202024-12-24%20005507.png?generation=1734994611332133&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10813%2F9df4cb2770b9c4e348bbcf7f9de2844f%2FScreenshot%202024-12-24%20005518.png?generation=1734994620812553&alt=media\")
is this also your experience? is this expected? and merry Christmas!
", "rawMarkdown": "Thanks @uermel - you mention isonet-corrected data should improve this effect but it doesn't seem to do much to @davidlist example images, i.e.\n\n\n\n\n\nis this also your experience? is this expected? and merry Christmas!", "votes": 2, "replies": [ { "id": 3079640, "postDate": "2024-12-23T23:45:46.310Z", "content": "Hmmm… Always meant to take a look, but hadn't gotten to it yet. The edges are maybe a little more blurred out in the isonet corrected image, but the underlying shape and detail appear unchanged.
", "rawMarkdown": "Hmmm... Always meant to take a look, but hadn't gotten to it yet. The edges are maybe a little more blurred out in the isonet corrected image, but the underlying shape and detail appear unchanged." }, { "id": 3079648, "postDate": "2024-12-24T00:11:36.857Z", "content": "@yahalom I can see an effect when I add all the ribosome images together:
\nDenoised:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10704200%2F4788f3515a8d05f0606849fa0161e275%2Fdenoised.png?generation=1734999055542528&alt=media\")
IsoNet Corrected:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10704200%2F82a348967f6985a822ced75306470d5b%2Fisonet.png?generation=1734999092412892&alt=media\")
Subtle though. And on individual images, pretty imperceptible.
", "rawMarkdown": "@yahalom I can see an effect when I add all the ribosome images together:\n\nDenoised:\n\n\nIsoNet Corrected:\n\n\nSubtle though. And on individual images, pretty imperceptible." }, { "id": 3079991, "postDate": "2024-12-24T13:15:35.070Z", "content": "thanks @davidlist- i think this effect works only if you believe really hard ;) I wonder if it could be that IsoNet is supposed to take care of actual sampling/Fourier/reflection/wedge artifacts as seen in the papers e.g.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10813%2Fb029d9419bcf17ab88458a8a7c9aa87c%2FScreenshot%202024-12-24%20151358.png?generation=1735046095339481&alt=media\")
, and that the elongations we're witnessing here are of a different source?
", "rawMarkdown": "thanks @davidlist- i think this effect works only if you believe really hard ;) I wonder if it could be that IsoNet is supposed to take care of actual sampling/Fourier/reflection/wedge artifacts as seen in the papers e.g. \n\n\n\n, and that the elongations we're witnessing here are of a different source?" }, { "id": 3080221, "postDate": "2024-12-24T21:57:38.517Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3080224, "postDate": "2024-12-24T22:06:06.623Z", "content": "@yahalom The best way to see the effect is to have each image open in a separate tab and switch back and forth between them… It's there, I promise. 😀 But if it's supposed to be as good as what you have in your image…yeah, it's not that.
", "rawMarkdown": "@yahalom The best way to see the effect is to have each image open in a separate tab and switch back and forth between them... It's there, I promise. 😀 But if it's supposed to be as good as what you have in your image...yeah, it's not that." } ] } ] }, { "id": 3053666, "postDate": "2024-11-23T18:11:43.370Z", "content": "make mean image to decide where the information lies (and decide how to model your net)
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fc9314e1def14fc55cbc52d59f72c3149%2FSelection_724.png?generation=1732385379717922&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F30d73c5811be84bce8a3bdc22897f786%2FSelection_726.png?generation=1732385391987271&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F5fbe8a0517f768c9137e62f970124bc5%2FSelection_728.png?generation=1732385405504599&alt=media\")
You can see this if viewed in 3D (napari)
", "rawMarkdown": "You can see this if viewed in 3D (napari)\n\n" }, { "id": 3043291, "postDate": "2024-11-12T09:40:50.270Z", "content": "I think from the top view is enough.
", "rawMarkdown": "I think from the top view is enough. \n" }, { "id": 3043161, "postDate": "2024-11-12T06:05:52.177Z", "content": "The following images are of a ribosome from the front, side, and top. The ring is the defined radius of 150 around the label point.
\nTop View:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10704200%2Fb9e5ddcc61e3c9fe1820b62d03a96712%2Ftop-view.png?generation=1731391538918182&alt=media\")
Side View::
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10704200%2Ff4a6e5aa1003f38bb67cadd9b69a0785%2Fside-view.png?generation=1731391485457386&alt=media\")
Straight on view:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10704200%2F35be0d2ac9b048b6f5881a04ee38b269%2Fstraight-on.png?generation=1731391413648792&alt=media\")
Actually, you might be able to get away with affine transformations so long as there's no rotation around the x or y axes.
", "rawMarkdown": "Actually, you might be able to get away with affine transformations so long as there's no rotation around the x or y axes." }, { "id": 3043008, "postDate": "2024-11-12T02:20:02.193Z", "content": "This is expected because of the missing wedge artifact in electron tomography.
", "rawMarkdown": "This is expected because of the missing wedge artifact in electron tomography. ", "isDeleted": true, "replies": [ { "id": 3043013, "postDate": "2024-11-12T02:29:20.010Z", "content": "Can you provide any detail to explain the mechanism? For instance, is the data being stretched to fill the void?
", "rawMarkdown": "Can you provide any detail to explain the mechanism? For instance, is the data being stretched to fill the void?", "votes": 1, "replies": [ { "id": 3044004, "postDate": "2024-11-12T22:34:19.353Z", "content": "Here's one of many references on this topic:
\nVan Veen, D. et al. Missing Wedge Completion via Unsupervised Learning with Coordinate Networks. Int. J. Mol. Sci. 25, (2024).
Hi @davidlist, this is an excellent observation and question, and one of the reasons working with cryoET data is hard.
\ncryoET is a limited-angle tomography method. Generally, in a TEM, projection images can only be acquired with sample tilt in a range between -65 to 65 degrees due to technical constraints of the sample and microscope. In the challenge sample, images were acquired in a range from -45 to 45 degrees, with 3 degree step, to limit electron beam induced damage to the sample.
\nAs a consequence, the 3D reconstruction suffers from the so-called \"missing wedge\" artifact, a wedge-shaped region in the fourier transform of the reconstructed tomogram that doesn't contain any information. In the real-space tomogram (i.e. the images we provided), this appears as the \"stretch\" you observed in the images along the z-axis. It's worth noting that this artifact cannot be corrected for by applying affine transformations to the tomgrams.
\nIn case of the denoised, ctf-deconvolved or raw weighted back projection tomograms, we did not apply any correction of the missing wedge artifact. The isonet-corrected tomograms were processed with a method that attempts to recover information in the missing wedge region (paper, code). As such, the observed stretch should be reduced in the isonet-corrected tomograms.
\nFor a more in-depth, mathematical explanation of the relationship of projection images, tomograms and fourier transforms of tomograms, I'd recommend reading on the projection-slice theorem.
", "votes": 5, "replies": [ { "id": 3044012, "author_name": "David List", "author_url": "", "post_date": "2024-11-12T22:48:30.883000", "content": "Great! Thanks @uermel! All of that is fantastic information. BTW, my comment regarding affine transformations was in regards to synthesizing more examples of the individual particles to assist machine learning, not to correct the stretch. If for instance, you take rotation, I suspect I could generate more ribosome examples if I limit it to around the z-axis. Anything around x, or y would cause problems.
", "votes": 0, "replies": [] }, { "id": 3079619, "author_name": "Yahalom", "author_url": "", "post_date": "2024-12-23T22:57:14.453000", "content": "Thanks @uermel - you mention isonet-corrected data should improve this effect but it doesn't seem to do much to @davidlist example images, i.e.
\n
\n
\n
is this also your experience? is this expected? and merry Christmas!
", "votes": 2, "replies": [ { "id": 3079640, "author_name": "David List", "author_url": "", "post_date": "2024-12-23T23:45:46.310000", "content": "Hmmm… Always meant to take a look, but hadn't gotten to it yet. The edges are maybe a little more blurred out in the isonet corrected image, but the underlying shape and detail appear unchanged.
", "votes": 0, "replies": [] }, { "id": 3079648, "author_name": "David List", "author_url": "", "post_date": "2024-12-24T00:11:36.857000", "content": "@yahalom I can see an effect when I add all the ribosome images together:
\nDenoised:
\n
IsoNet Corrected:
\n
Subtle though. And on individual images, pretty imperceptible.
", "votes": 0, "replies": [] }, { "id": 3079991, "author_name": "Yahalom", "author_url": "", "post_date": "2024-12-24T13:15:35.070000", "content": "thanks @davidlist- i think this effect works only if you believe really hard ;) I wonder if it could be that IsoNet is supposed to take care of actual sampling/Fourier/reflection/wedge artifacts as seen in the papers e.g.
\n
, and that the elongations we're witnessing here are of a different source?
", "votes": 0, "replies": [] }, { "id": 3080221, "author_name": "", "author_url": "", "post_date": "2024-12-24T21:57:38.517000", "content": "", "votes": 0, "replies": [] }, { "id": 3080224, "author_name": "David List", "author_url": "", "post_date": "2024-12-24T22:06:06.623000", "content": "@yahalom The best way to see the effect is to have each image open in a separate tab and switch back and forth between them… It's there, I promise. 😀 But if it's supposed to be as good as what you have in your image…yeah, it's not that.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3053666, "author_name": "hengck23", "author_url": "", "post_date": "2024-11-23T18:11:43.370000", "content": "make mean image to decide where the information lies (and decide how to model your net)
\n
\n
\n
You can see this if viewed in 3D (napari)
", "votes": 0, "replies": [] }, { "id": 3043291, "author_name": "Rustam Bazarbayev", "author_url": "", "post_date": "2024-11-12T09:40:50.270000", "content": "I think from the top view is enough.
", "votes": 0, "replies": [] }, { "id": 3043161, "author_name": "David List", "author_url": "", "post_date": "2024-11-12T06:05:52.177000", "content": "The following images are of a ribosome from the front, side, and top. The ring is the defined radius of 150 around the label point.
\nTop View:
\n
Side View::
\n
Straight on view:
\n
Actually, you might be able to get away with affine transformations so long as there's no rotation around the x or y axes.
", "votes": 0, "replies": [] }, { "id": 3043008, "author_name": "", "author_url": "", "post_date": "2024-11-12T02:20:02.193000", "content": "This is expected because of the missing wedge artifact in electron tomography.
", "votes": 0, "replies": [ { "id": 3043013, "author_name": "David List", "author_url": "", "post_date": "2024-11-12T02:29:20.010000", "content": "Can you provide any detail to explain the mechanism? For instance, is the data being stretched to fill the void?
", "votes": 1, "replies": [ { "id": 3044004, "author_name": "Reza Paraan", "author_url": "", "post_date": "2024-11-12T22:34:19.353000", "content": "Here's one of many references on this topic:
\nVan Veen, D. et al. Missing Wedge Completion via Unsupervised Learning with Coordinate Networks. Int. J. Mol. Sci. 25, (2024).