Hi there,
\nAs we are dealing with 4 modalities in our data, I want to share this review as it might be helpful to start creating architectures of deep learning models [https://link.springer.com/article/10.1007/s00371-021-02166-7]. If you have any other useful literature or approaches on how to handle the different modalities, I look forward to reading them in the comments :)
", "messages": [ { "id": "1402554", "postDate": "07/28/2021 09:40:57", "content": "Hi there,
\nAs we are dealing with 4 modalities in our data, I want to share this review as it might be helpful to start creating architectures of deep learning models [https://link.springer.com/article/10.1007/s00371-021-02166-7]. If you have any other useful literature or approaches on how to handle the different modalities, I look forward to reading them in the comments :)
", "rawMarkdown": "Hi there,\n\nAs we are dealing with 4 modalities in our data, I want to share this review as it might be helpful to start creating architectures of deep learning models [https://link.springer.com/article/10.1007/s00371-021-02166-7]. If you have any other useful literature or approaches on how to handle the different modalities, I look forward to reading them in the comments :)", "votes": null }, { "id": "1402616", "postDate": "07/28/2021 11:09:25", "content": "Thank you @fanconic that's a great resource. Let's hope more to come…
", "rawMarkdown": "Thank you @fanconic that's a great resource. Let's hope more to come...", "votes": null }, { "id": "1402742", "postDate": "07/28/2021 13:59:46", "content": "Thank you so much for sharing. I would just like to request if you come accross any resources which show practical examples of multi-modal transfer learning please do share.
\nfor instance, https://keras.io/guides/transfer_learning/, here feature extraction using transfer learning is done for 2D RGB images and in this competetion we have to focus on 3D mutli-modal transfer learning.
I would just like to mention that, I was thinking on creating nifti format image from dicom images and resize the images in 240240155 dimension and then planning on transfer learning for feature extraction but still could not find out how to do that.
", "rawMarkdown": "Thank you so much for sharing. I would just like to request if you come accross any resources which show practical examples of multi-modal transfer learning please do share.\nfor instance, https://keras.io/guides/transfer_learning/, here feature extraction using transfer learning is done for 2D RGB images and in this competetion we have to focus on 3D mutli-modal transfer learning.\n\nI would just like to mention that, I was thinking on creating nifti format image from dicom images and resize the images in 240*240*155 dimension and then planning on transfer learning for feature extraction but still could not find out how to do that.", "votes": null }, { "id": "1402743", "postDate": "07/28/2021 14:02:45", "content": "https://blog.tensorflow.org/2018/07/an-introduction-to-biomedical-image-analysis-tensorflow-dltk.html Anaother resource shared by @lucamtb
", "rawMarkdown": "https://blog.tensorflow.org/2018/07/an-introduction-to-biomedical-image-analysis-tensorflow-dltk.html Anaother resource shared by @lucamtb", "votes": null }, { "id": "1402765", "postDate": "07/28/2021 14:15:01", "content": "I am currently not familiar with 3D multi-modal transfer learning literature, I will have to search the internet a little more. Otherwise, one should start putting together the pieces available from the resources and give it a shot.
\nThe second point you mentioned seems sensible to me. Maybe be careful when transforming the NIFTI or DICOM files to images (uint8), as one might loose intensity information, which is a lot larger in MRI scans than the classic 8-bit image
", "rawMarkdown": "I am currently not familiar with 3D multi-modal transfer learning literature, I will have to search the internet a little more. Otherwise, one should start putting together the pieces available from the resources and give it a shot.\n\nThe second point you mentioned seems sensible to me. Maybe be careful when transforming the NIFTI or DICOM files to images (uint8), as one might loose intensity information, which is a lot larger in MRI scans than the classic 8-bit image", "votes": null }, { "id": "1403112", "postDate": "07/28/2021 19:38:42", "content": "Hi
\nGreat post by the way. But I'd like to know by 4 modalities are you referring to the 4 types of image scans?
Exactly. Theoretically, as they are all images they could be stacked together. However, I fear that by doing this, we might loose predictive information coming from the four different acquisition methods. I might be wrong though
", "rawMarkdown": "Exactly. Theoretically, as they are all images they could be stacked together. However, I fear that by doing this, we might loose predictive information coming from the four different acquisition methods. I might be wrong though", "votes": null }, { "id": "1404077", "postDate": "07/29/2021 14:45:09", "content": "Hello, basically I was thinking something similar to the following paper
\nhttps://www.nature.com/articles/s41598-018-37387-9
This looks very fitting on first sight
", "rawMarkdown": "This looks very fitting on first sight", "votes": null }, { "id": "1404106", "postDate": "07/29/2021 15:08:31", "content": "But I am still looking for resources where I could get some examples of implementing multi-channel 3D CNN with MRI images.
", "rawMarkdown": "But I am still looking for resources where I could get some examples of implementing multi-channel 3D CNN with MRI images.", "votes": null }, { "id": "1404147", "postDate": "07/29/2021 15:39:58", "content": "Why not try to implement it yourself? I will also give it a shot eventually
", "rawMarkdown": "Why not try to implement it yourself? I will also give it a shot eventually", "votes": null }, { "id": "1404522", "postDate": "07/30/2021 01:01:32", "content": "I read the article @fanconic and I found it super helpful!
", "rawMarkdown": "I read the article @fanconic and I found it super helpful!", "votes": null }, { "id": "1406553", "postDate": "07/31/2021 23:30:45", "content": "I just realized, that maybe it's actually not even necessary to use a multimodal approach. As I have seen, it also makes sense to use the three modalities as channels, as channels are usually processed independently of each other in a convolutional neural network. It could be interesting to try not to have any cross-connection between the channels, at least in the first layers.
", "rawMarkdown": "I just realized, that maybe it's actually not even necessary to use a multimodal approach. As I have seen, it also makes sense to use the three modalities as channels, as channels are usually processed independently of each other in a convolutional neural network. It could be interesting to try not to have any cross-connection between the channels, at least in the first layers.", "votes": null } ], "comments": [ { "id": 1402616, "author_name": "tharun2001", "author_url": "", "post_date": "07/28/2021 11:09:25", "content": "Thank you @fanconic that's a great resource. Let's hope more to come…
", "votes": null, "replies": [] }, { "id": 1402742, "author_name": "wfarzana", "author_url": "", "post_date": "07/28/2021 13:59:46", "content": "Thank you so much for sharing. I would just like to request if you come accross any resources which show practical examples of multi-modal transfer learning please do share.
\nfor instance, https://keras.io/guides/transfer_learning/, here feature extraction using transfer learning is done for 2D RGB images and in this competetion we have to focus on 3D mutli-modal transfer learning.
I would just like to mention that, I was thinking on creating nifti format image from dicom images and resize the images in 240240155 dimension and then planning on transfer learning for feature extraction but still could not find out how to do that.
", "votes": null, "replies": [ { "id": 1402765, "author_name": "fanconic", "author_url": "", "post_date": "07/28/2021 14:15:01", "content": "I am currently not familiar with 3D multi-modal transfer learning literature, I will have to search the internet a little more. Otherwise, one should start putting together the pieces available from the resources and give it a shot.
\nThe second point you mentioned seems sensible to me. Maybe be careful when transforming the NIFTI or DICOM files to images (uint8), as one might loose intensity information, which is a lot larger in MRI scans than the classic 8-bit image
", "votes": null, "replies": [] } ] }, { "id": 1402743, "author_name": "wfarzana", "author_url": "", "post_date": "07/28/2021 14:02:45", "content": "https://blog.tensorflow.org/2018/07/an-introduction-to-biomedical-image-analysis-tensorflow-dltk.html Anaother resource shared by @lucamtb
", "votes": null, "replies": [] }, { "id": 1403112, "author_name": "troublem1", "author_url": "", "post_date": "07/28/2021 19:38:42", "content": "Hi
\nGreat post by the way. But I'd like to know by 4 modalities are you referring to the 4 types of image scans?
Exactly. Theoretically, as they are all images they could be stacked together. However, I fear that by doing this, we might loose predictive information coming from the four different acquisition methods. I might be wrong though
", "votes": null, "replies": [] }, { "id": 1404077, "author_name": "wfarzana", "author_url": "", "post_date": "07/29/2021 14:45:09", "content": "Hello, basically I was thinking something similar to the following paper
\nhttps://www.nature.com/articles/s41598-018-37387-9
This looks very fitting on first sight
", "votes": null, "replies": [] }, { "id": 1404106, "author_name": "wfarzana", "author_url": "", "post_date": "07/29/2021 15:08:31", "content": "But I am still looking for resources where I could get some examples of implementing multi-channel 3D CNN with MRI images.
", "votes": null, "replies": [] }, { "id": 1404147, "author_name": "fanconic", "author_url": "", "post_date": "07/29/2021 15:39:58", "content": "Why not try to implement it yourself? I will also give it a shot eventually
", "votes": null, "replies": [] } ] }, { "id": 1404522, "author_name": "jolasa", "author_url": "", "post_date": "07/30/2021 01:01:32", "content": "I read the article @fanconic and I found it super helpful!
", "votes": null, "replies": [ { "id": 1406553, "author_name": "fanconic", "author_url": "", "post_date": "07/31/2021 23:30:45", "content": "I just realized, that maybe it's actually not even necessary to use a multimodal approach. As I have seen, it also makes sense to use the three modalities as channels, as channels are usually processed independently of each other in a convolutional neural network. It could be interesting to try not to have any cross-connection between the channels, at least in the first layers.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1402554": "Hi there,\n\nAs we are dealing with 4 modalities in our data, I want to share this review as it might be helpful to start creating architectures of deep learning models [https://link.springer.com/article/10.1007/s00371-021-02166-7]. If you have any other useful literature or approaches on how to handle the different modalities, I look forward to reading them in the comments :)", "1402616": "Thank you @fanconic that's a great resource. Let's hope more to come...", "1402742": "Thank you so much for sharing. I would just like to request if you come accross any resources which show practical examples of multi-modal transfer learning please do share.\nfor instance, https://keras.io/guides/transfer_learning/, here feature extraction using transfer learning is done for 2D RGB images and in this competetion we have to focus on 3D mutli-modal transfer learning.\n\nI would just like to mention that, I was thinking on creating nifti format image from dicom images and resize the images in 240*240*155 dimension and then planning on transfer learning for feature extraction but still could not find out how to do that.", "1402743": "https://blog.tensorflow.org/2018/07/an-introduction-to-biomedical-image-analysis-tensorflow-dltk.html Anaother resource shared by @lucamtb", "1402765": "I am currently not familiar with 3D multi-modal transfer learning literature, I will have to search the internet a little more. Otherwise, one should start putting together the pieces available from the resources and give it a shot.\n\nThe second point you mentioned seems sensible to me. Maybe be careful when transforming the NIFTI or DICOM files to images (uint8), as one might loose intensity information, which is a lot larger in MRI scans than the classic 8-bit image", "1403112": "Hi \nGreat post by the way. But I'd like to know by 4 modalities are you referring to the 4 types of image scans?", "1403501": "Exactly. Theoretically, as they are all images they could be stacked together. However, I fear that by doing this, we might loose predictive information coming from the four different acquisition methods. I might be wrong though", "1404077": "Hello, basically I was thinking something similar to the following paper\nhttps://www.nature.com/articles/s41598-018-37387-9", "1404098": "This looks very fitting on first sight", "1404106": "But I am still looking for resources where I could get some examples of implementing multi-channel 3D CNN with MRI images.", "1404147": "Why not try to implement it yourself? I will also give it a shot eventually", "1404522": "I read the article @fanconic and I found it super helpful!", "1406553": "I just realized, that maybe it's actually not even necessary to use a multimodal approach. As I have seen, it also makes sense to use the three modalities as channels, as channels are usually processed independently of each other in a convolutional neural network. It could be interesting to try not to have any cross-connection between the channels, at least in the first layers." }, "source": "meta" }