{
  "id": 70219,
  "title": "How to convert 4 channels image(RGBY) to RGB",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/70219",
  "author_name": "",
  "post_date": "2018-11-01T03:21:59.392219400Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<pre><code>def load_image(path, shape):\n    image_red_ch = skimage.io.imread(path+'_red.png')\n    image_yellow_ch = skimage.io.imread(path+'_yellow.png')\n    image_green_ch = skimage.io.imread(path+'_green.png')\n    image_blue_ch = skimage.io.imread(path+'_blue.png')\n\n    image_red_ch += (image_yellow_ch/2).astype(np.uint8) \n    image_green_ch += (image_yellow_ch/2).astype(np.uint8)\n\n    image = np.stack((\n        image_red_ch, \n        image_green_ch, \n        image_blue_ch), -1)\n    image = resize(image, (shape[0], shape[1]), mode='reflect')\n    return image\n</code></pre>\n\n<p>from <a href=\"https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\">https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier</a></p>\n\n<p>Is this the best way to do convert?</p>",
  "messages": [
    {
      "id": "413492",
      "postDate": "11/01/2018 03:21:59",
      "content": "<pre><code>def load_image(path, shape):\n    image_red_ch = skimage.io.imread(path+'_red.png')\n    image_yellow_ch = skimage.io.imread(path+'_yellow.png')\n    image_green_ch = skimage.io.imread(path+'_green.png')\n    image_blue_ch = skimage.io.imread(path+'_blue.png')\n\n    image_red_ch += (image_yellow_ch/2).astype(np.uint8) \n    image_green_ch += (image_yellow_ch/2).astype(np.uint8)\n\n    image = np.stack((\n        image_red_ch, \n        image_green_ch, \n        image_blue_ch), -1)\n    image = resize(image, (shape[0], shape[1]), mode='reflect')\n    return image\n</code></pre>\n\n<p>from <a href=\"https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\">https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier</a></p>\n\n<p>Is this the best way to do convert?</p>",
      "rawMarkdown": "def load_image(path, shape):\n        image_red_ch = skimage.io.imread(path+'_red.png')\n        image_yellow_ch = skimage.io.imread(path+'_yellow.png')\n        image_green_ch = skimage.io.imread(path+'_green.png')\n        image_blue_ch = skimage.io.imread(path+'_blue.png')\n\n        image_red_ch += (image_yellow_ch/2).astype(np.uint8) \n        image_green_ch += (image_yellow_ch/2).astype(np.uint8)\n\n        image = np.stack((\n            image_red_ch, \n            image_green_ch, \n            image_blue_ch), -1)\n        image = resize(image, (shape[0], shape[1]), mode='reflect')\n        return image\n\nfrom https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\n\nIs this the best way to do convert?",
      "votes": null
    },
    {
      "id": "413506",
      "postDate": "11/01/2018 03:50:02",
      "content": "<p>create 4 channel [r,g,b,y] as input. insert a 1x1 conv to convert from 4 channel to 3 channel before pretrain model. initialise weights of 1x1 convolution so that it perform \"<a href=\"https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\">https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier</a>\" at initilisation.</p>\n\n<p>let the network learn the best combination.</p>\n\n<p>4channel ---&gt; channel_transform_net ---&gt; 3 channel ---&gt; pretrain model ---&gt; ...</p>\n\n<p>e.g. channel_transform_net  = {conv1x1}</p>\n\n<p>channel_transform_net  = {conv1x1,relu, conv1x1, .......}</p>\n\n<hr>\n\n<p>but i think feeding 4 channel input is better than 3 channel</p>",
      "rawMarkdown": "create 4 channel [r,g,b,y] as input. insert a 1x1 conv to convert from 4 channel to 3 channel before pretrain model. initialise weights of 1x1 convolution so that it perform \"https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\" at initilisation.\n\nlet the network learn the best combination.\n\n4channel ---&gt; channel_transform_net ---&gt; 3 channel ---&gt; pretrain model ---&gt; ...\n\ne.g. channel_transform_net  = {conv1x1}\n\nchannel_transform_net  = {conv1x1,relu, conv1x1, .......}\n\n\n\n---\nbut i think feeding 4 channel input is better than 3 channel",
      "votes": null
    },
    {
      "id": "413543",
      "postDate": "11/01/2018 05:25:24",
      "content": "<p>Wow，this is an excellent idea, I will try this first~\nThanks CherKeng</p>",
      "rawMarkdown": "Wow，this is an excellent idea, I will try this first~\nThanks CherKeng",
      "votes": null
    },
    {
      "id": "413547",
      "postDate": "11/01/2018 05:35:04",
      "content": "<p>Maybe this is another way：</p>\n\n<blockquote>\n  <p>One of the challenges in this competition is 4-chanel input (RGBY)\n  that limits usage of ImageNet pretrained models taking RGB input.\n  However, the input dataset is too tiny to train even a low capacity\n  model like ResNet34 from scratch. I propose to use the following way\n  to walk around this limitation. The most common way to convert RGBY to\n  RGB is just dropping Y channel while keeping RGB without modification.\n  Therefore, I replace the first convolution layer from 7x7 3-&gt;64 to\n  7x7 4-&gt;64 while keeping weights from 3-&gt;64 and setting new\n  initial weighs for Y channel to be zero. It allows using the original\n  weights to initialize the network while giving the opportunity to the\n  model to incorporate Y channel into prediction during the following\n  training (when the first layers of the model are unfreezed). In the\n  following hiden cell I put a code from fast.ai library with adding\n  several lines for the replacment of the first convolutional layer.</p>\n</blockquote>\n\n<p><a href=\"https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb\">https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb</a></p>",
      "rawMarkdown": "Maybe this is another way：\n\n&gt; One of the challenges in this competition is 4-chanel input (RGBY)\n&gt; that limits usage of ImageNet pretrained models taking RGB input.\n&gt; However, the input dataset is too tiny to train even a low capacity\n&gt; model like ResNet34 from scratch. I propose to use the following way\n&gt; to walk around this limitation. The most common way to convert RGBY to\n&gt; RGB is just dropping Y channel while keeping RGB without modification.\n&gt; Therefore, I replace the first convolution layer from 7x7 3-&gt;64 to\n&gt; 7x7 4-&gt;64 while keeping weights from 3-&gt;64 and setting new\n&gt; initial weighs for Y channel to be zero. It allows using the original\n&gt; weights to initialize the network while giving the opportunity to the\n&gt; model to incorporate Y channel into prediction during the following\n&gt; training (when the first layers of the model are unfreezed). In the\n&gt; following hiden cell I put a code from fast.ai library with adding\n&gt; several lines for the replacment of the first convolutional layer.\n\nhttps://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb",
      "votes": null
    },
    {
      "id": "414852",
      "postDate": "11/03/2018 18:29:05",
      "content": "<p>Is it possible to do it using keras?</p>",
      "rawMarkdown": "Is it possible to do it using keras?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 413506,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "11/01/2018 03:50:02",
      "content": "<p>create 4 channel [r,g,b,y] as input. insert a 1x1 conv to convert from 4 channel to 3 channel before pretrain model. initialise weights of 1x1 convolution so that it perform \"<a href=\"https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\">https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier</a>\" at initilisation.</p>\n\n<p>let the network learn the best combination.</p>\n\n<p>4channel ---&gt; channel_transform_net ---&gt; 3 channel ---&gt; pretrain model ---&gt; ...</p>\n\n<p>e.g. channel_transform_net  = {conv1x1}</p>\n\n<p>channel_transform_net  = {conv1x1,relu, conv1x1, .......}</p>\n\n<hr>\n\n<p>but i think feeding 4 channel input is better than 3 channel</p>",
      "votes": null,
      "replies": [
        {
          "id": 413543,
          "author_name": "the16throute",
          "author_url": "",
          "post_date": "11/01/2018 05:25:24",
          "content": "<p>Wow，this is an excellent idea, I will try this first~\nThanks CherKeng</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 413547,
          "author_name": "the16throute",
          "author_url": "",
          "post_date": "11/01/2018 05:35:04",
          "content": "<p>Maybe this is another way：</p>\n\n<blockquote>\n  <p>One of the challenges in this competition is 4-chanel input (RGBY)\n  that limits usage of ImageNet pretrained models taking RGB input.\n  However, the input dataset is too tiny to train even a low capacity\n  model like ResNet34 from scratch. I propose to use the following way\n  to walk around this limitation. The most common way to convert RGBY to\n  RGB is just dropping Y channel while keeping RGB without modification.\n  Therefore, I replace the first convolution layer from 7x7 3-&gt;64 to\n  7x7 4-&gt;64 while keeping weights from 3-&gt;64 and setting new\n  initial weighs for Y channel to be zero. It allows using the original\n  weights to initialize the network while giving the opportunity to the\n  model to incorporate Y channel into prediction during the following\n  training (when the first layers of the model are unfreezed). In the\n  following hiden cell I put a code from fast.ai library with adding\n  several lines for the replacment of the first convolutional layer.</p>\n</blockquote>\n\n<p><a href=\"https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb\">https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 414852,
          "author_name": "hermesrs",
          "author_url": "",
          "post_date": "11/03/2018 18:29:05",
          "content": "<p>Is it possible to do it using keras?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "413492": "def load_image(path, shape):\n        image_red_ch = skimage.io.imread(path+'_red.png')\n        image_yellow_ch = skimage.io.imread(path+'_yellow.png')\n        image_green_ch = skimage.io.imread(path+'_green.png')\n        image_blue_ch = skimage.io.imread(path+'_blue.png')\n\n        image_red_ch += (image_yellow_ch/2).astype(np.uint8) \n        image_green_ch += (image_yellow_ch/2).astype(np.uint8)\n\n        image = np.stack((\n            image_red_ch, \n            image_green_ch, \n            image_blue_ch), -1)\n        image = resize(image, (shape[0], shape[1]), mode='reflect')\n        return image\n\nfrom https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\n\nIs this the best way to do convert?",
    "413506": "create 4 channel [r,g,b,y] as input. insert a 1x1 conv to convert from 4 channel to 3 channel before pretrain model. initialise weights of 1x1 convolution so that it perform \"https://www.kaggle.com/byrachonok/pretrained-inceptionresnetv2-base-classifier\" at initilisation.\n\nlet the network learn the best combination.\n\n4channel ---&gt; channel_transform_net ---&gt; 3 channel ---&gt; pretrain model ---&gt; ...\n\ne.g. channel_transform_net  = {conv1x1}\n\nchannel_transform_net  = {conv1x1,relu, conv1x1, .......}\n\n\n\n---\nbut i think feeding 4 channel input is better than 3 channel",
    "413543": "Wow，this is an excellent idea, I will try this first~\nThanks CherKeng",
    "413547": "Maybe this is another way：\n\n&gt; One of the challenges in this competition is 4-chanel input (RGBY)\n&gt; that limits usage of ImageNet pretrained models taking RGB input.\n&gt; However, the input dataset is too tiny to train even a low capacity\n&gt; model like ResNet34 from scratch. I propose to use the following way\n&gt; to walk around this limitation. The most common way to convert RGBY to\n&gt; RGB is just dropping Y channel while keeping RGB without modification.\n&gt; Therefore, I replace the first convolution layer from 7x7 3-&gt;64 to\n&gt; 7x7 4-&gt;64 while keeping weights from 3-&gt;64 and setting new\n&gt; initial weighs for Y channel to be zero. It allows using the original\n&gt; weights to initialize the network while giving the opportunity to the\n&gt; model to incorporate Y channel into prediction during the following\n&gt; training (when the first layers of the model are unfreezed). In the\n&gt; following hiden cell I put a code from fast.ai library with adding\n&gt; several lines for the replacment of the first convolutional layer.\n\nhttps://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb",
    "414852": "Is it possible to do it using keras?"
  },
  "source": "meta"
}