Hey, Can anyone please share any reference/code/article on how to process and feed a multi-channel(more than 3 channels) image to a model!\nI tried feeding RGB images to model but this is not giving a decent accuracy as I got only a accuracy of 0.015 till now.\nI am new to this field and I don't have enough experience on how to feed 6 channel image separately.\nPlease share any reference available, thanks in advance!
", "messages": [ { "id": "580074", "postDate": "07/19/2019 16:34:10", "content": "Hey, Can anyone please share any reference/code/article on how to process and feed a multi-channel(more than 3 channels) image to a model!\nI tried feeding RGB images to model but this is not giving a decent accuracy as I got only a accuracy of 0.015 till now.\nI am new to this field and I don't have enough experience on how to feed 6 channel image separately.\nPlease share any reference available, thanks in advance!
", "rawMarkdown": "Hey, Can anyone please share any reference/code/article on how to process and feed a multi-channel(more than 3 channels) image to a model!\nI tried feeding RGB images to model but this is not giving a decent accuracy as I got only a accuracy of 0.015 till now.\nI am new to this field and I don't have enough experience on how to feed 6 channel image separately.\nPlease share any reference available, thanks in advance!", "votes": null }, { "id": "580291", "postDate": "07/20/2019 01:16:08", "content": "If you use pytorch, this Leigh's kernel is useful.\nhttps://www.kaggle.com/leighplt/densenet121-pytorch
\n\nPlease see In[5], it defines 6(=num_channels) channel model.
", "rawMarkdown": "If you use pytorch, this Leigh's kernel is useful.\nhttps://www.kaggle.com/leighplt/densenet121-pytorch\n\nPlease see In[5], it defines 6(=num_channels) channel model.", "votes": null }, { "id": "580326", "postDate": "07/20/2019 02:24:06", "content": "Though I have worked on Keras mostly, I'll definitely give this kernel a shot. Thank you so much! :) \nSo should I start using Pytorch, is it that good?
", "rawMarkdown": "Though I have worked on Keras mostly, I'll definitely give this kernel a shot. Thank you so much! :) \nSo should I start using Pytorch, is it that good?", "votes": null }, { "id": "580373", "postDate": "07/20/2019 04:17:17", "content": "In the kernels of Human Protein Atlas Image Classification, there are some Keras models. The data set of this competition has 4 channels (R, G, B and \"Y\"). Thus, some models' input is \"4\" channels. For example, this kernel seems to use 4 channels model (see In[2]).\nhttps://www.kaggle.com/rejpalcz/cnn-128x128x4-keras-from-scratch-lb-0-328\nYou might be able to refer it.
\n\nOn a different note, I think it is also good to start from baseline model (though it is pytorch in this case).
\nIMO, if you struggle with construction of base model for Keras, it might be constructive to spend time for customize baseline model in public kernels.
Okay, sure thanks so much. Appreciate your help! ^_^
", "rawMarkdown": "Okay, sure thanks so much. Appreciate your help! ^_^", "votes": null }, { "id": "580406", "postDate": "07/20/2019 05:54:36", "content": "You can also just use a point-wise Conv2D to combine 6 channels to 3 channels.\nAnother way is to train an auto-encoder that codes 6 channels to 3 channels
", "rawMarkdown": "You can also just use a point-wise Conv2D to combine 6 channels to 3 channels.\nAnother way is to train an auto-encoder that codes 6 channels to 3 channels", "votes": null }, { "id": "580413", "postDate": "07/20/2019 06:02:29", "content": "Okay, will see that approach, thanks!
", "rawMarkdown": "Okay, will see that approach, thanks!", "votes": null }, { "id": "581053", "postDate": "07/21/2019 10:20:39", "content": "To summarize, i think there are 3 ways how you can adopt your pretrained 3 channel network for 6 channel input.
\n\nThe easiest way is to just convert 6 channel image to 3 channels using some kind of color mapping (as it is done in rxrx1-utils). \n- Pros: easy to do, can be used with any framework and network pretrained on RGB.\n- Cons: you loose some information when projecting 6 channels to 3 channels.
\n\nLearn how to convert 6 channels to 3 channels. The idea is simple: you just add another convolutional layer just before you network (in PyTorch it would be nn.Conv2d(6, 3, 1)) and learn projection kernel via backprop.\n- Pros: you can visualize how your network converts images, information loss is less severe (network learns useful conversion), can learn nonlinear conversion.\n- Cons: Still loosing information, can be challenging to do with some frameworks.
\n\nReplace pretrained network's first convolutional layer with one to match your input.\n- Pros: no information loss.\n- Cons: can be really hard to do with some frameworks, loosing pretrained weights from first layer (not a problem usually).
", "rawMarkdown": "To summarize, i think there are 3 ways how you can adopt your pretrained 3 channel network for 6 channel input.\n\nThe easiest way is to just convert 6 channel image to 3 channels using some kind of color mapping (as it is done in [rxrx1-utils](https://github.com/recursionpharma/rxrx1-utils)). \n- Pros: easy to do, can be used with any framework and network pretrained on RGB.\n- Cons: you loose some information when projecting 6 channels to 3 channels.\n\nLearn how to convert 6 channels to 3 channels. The idea is simple: you just add another convolutional layer just before you network (in PyTorch it would be nn.Conv2d(6, 3, 1)) and learn projection kernel via backprop.\n- Pros: you can visualize how your network converts images, information loss is less severe (network learns useful conversion), can learn nonlinear conversion.\n- Cons: Still loosing information, can be challenging to do with some frameworks.\n\nReplace pretrained network's first convolutional layer with one to match your input.\n- Pros: no information loss.\n- Cons: can be really hard to do with some frameworks, loosing pretrained weights from first layer (not a problem usually).", "votes": null }, { "id": "581054", "postDate": "07/21/2019 10:26:17", "content": "I would recommend not loosing information and using the right tool (PyTorch) for ease of implementation.
", "rawMarkdown": "I would recommend not loosing information and using the right tool (PyTorch) for ease of implementation.", "votes": null }, { "id": "581274", "postDate": "07/21/2019 17:35:10", "content": "Hey, thanks @vshmyhlo , I tried adding one layer before the pretrained model but it got very hard for me.\nAlso, since I have worked only on keras till now, I think this is high time to start with pytorch now!\nAlso can you clear one more issue for me, when I am training my model in the kernel, I can see the validation accuracy changing and model is learning while when I commit the same kernel, the val_acc gets stuck to only one value and loss gives 'nan' value and public score comes out to be 0, can you please share some insight on that too? Thanks in advance. :)
", "rawMarkdown": "Hey, thanks @vshmyhlo , I tried adding one layer before the pretrained model but it got very hard for me.\nAlso, since I have worked only on keras till now, I think this is high time to start with pytorch now!\nAlso can you clear one more issue for me, when I am training my model in the kernel, I can see the validation accuracy changing and model is learning while when I commit the same kernel, the val_acc gets stuck to only one value and loss gives 'nan' value and public score comes out to be 0, can you please share some insight on that too? Thanks in advance. :)", "votes": null }, { "id": "581498", "postDate": "07/22/2019 02:59:04", "content": "how to perform image augmentation with 6 channels? AFAIK neither transforms from pytorch or albumentations can transform images with 6 channels... I am always having errors
", "rawMarkdown": "how to perform image augmentation with 6 channels? AFAIK neither transforms from pytorch or albumentations can transform images with 6 channels... I am always having errors", "votes": null }, { "id": "621595", "postDate": "09/08/2019 17:31:34", "content": "@vshmyhlo I know the post is a little old, but could you explain with a little more detail what's the difference between second and third options in your post?
", "rawMarkdown": "vshmyhlo I know the post is a little old, but could you explain with a little more detail what's the difference between second and third options in your post?", "votes": null }, { "id": "622095", "postDate": "09/09/2019 09:17:48", "content": "If you use pytorch, this Leigh's kernel is useful.\nhttps://www.kaggle.com/leighplt/densenet121-pytorch
\n\nPlease see In[5], it defines 6(=num_channels) channel model.
", "votes": null, "replies": [ { "id": 580326, "author_name": "sarques", "author_url": "", "post_date": "07/20/2019 02:24:06", "content": "Though I have worked on Keras mostly, I'll definitely give this kernel a shot. Thank you so much! :) \nSo should I start using Pytorch, is it that good?
", "votes": null, "replies": [] }, { "id": 580373, "author_name": "oshidori", "author_url": "", "post_date": "07/20/2019 04:17:17", "content": "In the kernels of Human Protein Atlas Image Classification, there are some Keras models. The data set of this competition has 4 channels (R, G, B and \"Y\"). Thus, some models' input is \"4\" channels. For example, this kernel seems to use 4 channels model (see In[2]).\nhttps://www.kaggle.com/rejpalcz/cnn-128x128x4-keras-from-scratch-lb-0-328\nYou might be able to refer it.
\n\nOn a different note, I think it is also good to start from baseline model (though it is pytorch in this case).
\nIMO, if you struggle with construction of base model for Keras, it might be constructive to spend time for customize baseline model in public kernels.
Okay, sure thanks so much. Appreciate your help! ^_^
", "votes": null, "replies": [] } ] }, { "id": 580406, "author_name": "rrezaii", "author_url": "", "post_date": "07/20/2019 05:54:36", "content": "You can also just use a point-wise Conv2D to combine 6 channels to 3 channels.\nAnother way is to train an auto-encoder that codes 6 channels to 3 channels
", "votes": null, "replies": [ { "id": 580413, "author_name": "sarques", "author_url": "", "post_date": "07/20/2019 06:02:29", "content": "Okay, will see that approach, thanks!
", "votes": null, "replies": [] } ] }, { "id": 581053, "author_name": "vshmyhlo", "author_url": "", "post_date": "07/21/2019 10:20:39", "content": "To summarize, i think there are 3 ways how you can adopt your pretrained 3 channel network for 6 channel input.
\n\nThe easiest way is to just convert 6 channel image to 3 channels using some kind of color mapping (as it is done in rxrx1-utils). \n- Pros: easy to do, can be used with any framework and network pretrained on RGB.\n- Cons: you loose some information when projecting 6 channels to 3 channels.
\n\nLearn how to convert 6 channels to 3 channels. The idea is simple: you just add another convolutional layer just before you network (in PyTorch it would be nn.Conv2d(6, 3, 1)) and learn projection kernel via backprop.\n- Pros: you can visualize how your network converts images, information loss is less severe (network learns useful conversion), can learn nonlinear conversion.\n- Cons: Still loosing information, can be challenging to do with some frameworks.
\n\nReplace pretrained network's first convolutional layer with one to match your input.\n- Pros: no information loss.\n- Cons: can be really hard to do with some frameworks, loosing pretrained weights from first layer (not a problem usually).
", "votes": null, "replies": [ { "id": 581054, "author_name": "vshmyhlo", "author_url": "", "post_date": "07/21/2019 10:26:17", "content": "I would recommend not loosing information and using the right tool (PyTorch) for ease of implementation.
", "votes": null, "replies": [] }, { "id": 581274, "author_name": "sarques", "author_url": "", "post_date": "07/21/2019 17:35:10", "content": "Hey, thanks @vshmyhlo , I tried adding one layer before the pretrained model but it got very hard for me.\nAlso, since I have worked only on keras till now, I think this is high time to start with pytorch now!\nAlso can you clear one more issue for me, when I am training my model in the kernel, I can see the validation accuracy changing and model is learning while when I commit the same kernel, the val_acc gets stuck to only one value and loss gives 'nan' value and public score comes out to be 0, can you please share some insight on that too? Thanks in advance. :)
", "votes": null, "replies": [] }, { "id": 621595, "author_name": "cateek", "author_url": "", "post_date": "09/08/2019 17:31:34", "content": "@vshmyhlo I know the post is a little old, but could you explain with a little more detail what's the difference between second and third options in your post?
", "votes": null, "replies": [] }, { "id": 622095, "author_name": "valanm", "author_url": "", "post_date": "09/09/2019 09:17:48", "content": "how to perform image augmentation with 6 channels? AFAIK neither transforms from pytorch or albumentations can transform images with 6 channels... I am always having errors
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "580074": "Hey, Can anyone please share any reference/code/article on how to process and feed a multi-channel(more than 3 channels) image to a model!\nI tried feeding RGB images to model but this is not giving a decent accuracy as I got only a accuracy of 0.015 till now.\nI am new to this field and I don't have enough experience on how to feed 6 channel image separately.\nPlease share any reference available, thanks in advance!", "580291": "If you use pytorch, this Leigh's kernel is useful.\nhttps://www.kaggle.com/leighplt/densenet121-pytorch\n\nPlease see In[5], it defines 6(=num_channels) channel model.", "580326": "Though I have worked on Keras mostly, I'll definitely give this kernel a shot. Thank you so much! :) \nSo should I start using Pytorch, is it that good?", "580373": "In [the kernels of Human Protein Atlas Image Classification](https://www.kaggle.com/c/human-protein-atlas-image-classification/kernels), there are some Keras models. The data set of this competition has 4 channels (R, G, B and \"Y\"). Thus, some models' input is \"4\" channels. For example, this kernel seems to use 4 channels model (see In[2]).\nhttps://www.kaggle.com/rejpalcz/cnn-128x128x4-keras-from-scratch-lb-0-328\nYou might be able to refer it.\n\nOn a different note, I think it is also good to start from baseline model (though it is pytorch in this case). \nIMO, if you struggle with construction of base model for Keras, it might be constructive to spend time for customize baseline model in public kernels.", "580374": "Okay, sure thanks so much. Appreciate your help! ^_^", "580406": "You can also just use a point-wise Conv2D to combine 6 channels to 3 channels.\nAnother way is to train an auto-encoder that codes 6 channels to 3 channels", "580413": "Okay, will see that approach, thanks!", "581053": "To summarize, i think there are 3 ways how you can adopt your pretrained 3 channel network for 6 channel input.\n\nThe easiest way is to just convert 6 channel image to 3 channels using some kind of color mapping (as it is done in [rxrx1-utils](https://github.com/recursionpharma/rxrx1-utils)). \n- Pros: easy to do, can be used with any framework and network pretrained on RGB.\n- Cons: you loose some information when projecting 6 channels to 3 channels.\n\nLearn how to convert 6 channels to 3 channels. The idea is simple: you just add another convolutional layer just before you network (in PyTorch it would be nn.Conv2d(6, 3, 1)) and learn projection kernel via backprop.\n- Pros: you can visualize how your network converts images, information loss is less severe (network learns useful conversion), can learn nonlinear conversion.\n- Cons: Still loosing information, can be challenging to do with some frameworks.\n\nReplace pretrained network's first convolutional layer with one to match your input.\n- Pros: no information loss.\n- Cons: can be really hard to do with some frameworks, loosing pretrained weights from first layer (not a problem usually).", "581054": "I would recommend not loosing information and using the right tool (PyTorch) for ease of implementation.", "581274": "Hey, thanks @vshmyhlo , I tried adding one layer before the pretrained model but it got very hard for me.\nAlso, since I have worked only on keras till now, I think this is high time to start with pytorch now!\nAlso can you clear one more issue for me, when I am training my model in the kernel, I can see the validation accuracy changing and model is learning while when I commit the same kernel, the val_acc gets stuck to only one value and loss gives 'nan' value and public score comes out to be 0, can you please share some insight on that too? Thanks in advance. :)", "581498": "how to perform image augmentation with 6 channels? AFAIK neither transforms from pytorch or albumentations can transform images with 6 channels... I am always having errors", "621595": "vshmyhlo I know the post is a little old, but could you explain with a little more detail what's the difference between second and third options in your post?", "622095": "2. you **insert** a layer before the input layer which learns how to reduce 6 channels to 3 channels (expected input for standard CNN) \n3. you **replace** your input layer\nHope it helps." }, "source": "meta" }