![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/415069/10597/activation.png\")
some code for your visualization:
\n\nhttps://github.com/utkuozbulak/pytorch-cnn-visualizations#gradient-visualization
\n\nhttps://github.com/eugenelet/VisualBackProp-PyTorch
\n\n\n\ni hope to post mine when my results are ready
", "messages": [ { "id": "413233", "postDate": "10/31/2018 14:40:31", "content": "some code for your visualization:
\n\nhttps://github.com/utkuozbulak/pytorch-cnn-visualizations#gradient-visualization
\n\nhttps://github.com/eugenelet/VisualBackProp-PyTorch
\n\n\n\ni hope to post mine when my results are ready
", "rawMarkdown": "some code for your visualization:\n\nhttps://github.com/utkuozbulak/pytorch-cnn-visualizations#gradient-visualization\n\nhttps://github.com/eugenelet/VisualBackProp-PyTorch\n\nhttps://github.com/handong1587/handong1587.github.io/blob/master/_posts/deep_learning/2015-10-09-visulizing-interpreting-cnn.md\n\n---\n\ni hope to post mine when my results are ready", "votes": null }, { "id": "413289", "postDate": "10/31/2018 16:28:25", "content": "see also:
\n\nhttps://www.kaggle.com/c/planet-understanding-the-amazon-from-space/discussion/32953
", "rawMarkdown": "see also:\n\nhttps://www.kaggle.com/c/planet-understanding-the-amazon-from-space/discussion/32953", "votes": null }, { "id": "413323", "postDate": "10/31/2018 17:41:24", "content": "Here is how I visualize things:
\n\nfrom keras.models import load_model\nmodel_name = 'hpa_sub_19_best.hdf5'\nprint(\"LOAD: \" + model_name)\norig_model = load_model(model_name,custom_objects=keras_custom_objects)\norig_model.summary()\n\n\n
#pick a layer by name in the previous and put in the size.\nimg_width = 64\nimg_height = 64\nlr = 1.\nsteps = 400\n#get layer\nlayer_dict = dict([(layer.name, layer) for layer in orig_model.layers])\nlayer_name = 'block3'\nfilter_index = 42 # can be any integer from 0 to 511, if there are 512 filters in that layer\n\ninput_img = orig_model.input\n\n# build a loss function that maximizes the activation\n# of the nth filter of the layer considered\nlayer_output = layer_dict[layer_name].output\nloss = K.mean(layer_output[:, :, :, filter_index])\n\n# compute the gradient of the input picture wrt this loss\ngrads = K.gradients(loss, input_img)[0]\n\n# normalization trick: we normalize the gradient\ngrads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)\n\n# this function returns the loss and grads given the input picture\niterate = K.function([input_img], [loss, grads])\n\n# we start from a gray image with some noise\nif K.image_data_format() == 'channels_first':\n input_img_data = np.random.random((1, 3, img_width, img_height))\nelse:\n input_img_data = np.random.random((1, img_width, img_height, 3))\ninput_img_data = (input_img_data - 0.5) * 20 + 128\n\nfor i in range(steps):\n if i%20 == 0:\n print(\"%d/%d\" % (i+1,steps))\n loss_value, grads_value = iterate([input_img_data])\n input_img_data += grads_value * lr\n\n\n# util function to convert a tensor into a valid image\ndef deprocess_image(x):\n # normalize tensor: center on 0., ensure std is 0.1\n x -= x.mean()\n x /= (x.std() + 1e-5)\n x *= 0.1\n\n # clip to [0, 1]\n x += 0.5\n x = np.clip(x, 0, 1)\n\n # convert to RGB array\n x *= 255\n #x = x.transpose((1, 2, 0))\n x = np.clip(x, 0, 255).astype('uint8')\n return np.squeeze(x)\n\nimg = input_img_data[0]\nimg = deprocess_image(img)\nprint(str(img.shape))\nfrom scipy.misc import imsave\nimsave('%s_filter_%d.png' % (layer_name, filter_index), img)\n\nfig = plt.figure(figsize=(img_width,img_height))\nax = fig.add_subplot(1,1,1)\nax.imshow(img, cmap='gray')\n\n\n",
"rawMarkdown": "Here is how I visualize things:\n\nfrom keras.models import load_model\nmodel_name = 'hpa_sub_19_best.hdf5'\nprint(\"LOAD: \" + model_name)\norig_model = load_model(model_name,custom_objects=keras_custom_objects)\norig_model.summary()\n\n\n\n
#pick a layer by name in the previous and put in the size.\nimg_width = 64\nimg_height = 64\nlr = 1.\nsteps = 400\n#get layer\nlayer_dict = dict([(layer.name, layer) for layer in orig_model.layers])\nlayer_name = 'block3'\nfilter_index = 42 # can be any integer from 0 to 511, if there are 512 filters in that layer\n\ninput_img = orig_model.input\n\n# build a loss function that maximizes the activation\n# of the nth filter of the layer considered\nlayer_output = layer_dict[layer_name].output\nloss = K.mean(layer_output[:, :, :, filter_index])\n\n# compute the gradient of the input picture wrt this loss\ngrads = K.gradients(loss, input_img)[0]\n\n# normalization trick: we normalize the gradient\ngrads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)\n\n# this function returns the loss and grads given the input picture\niterate = K.function([input_img], [loss, grads])\n\n# we start from a gray image with some noise\nif K.image_data_format() == 'channels_first':\n input_img_data = np.random.random((1, 3, img_width, img_height))\nelse:\n input_img_data = np.random.random((1, img_width, img_height, 3))\ninput_img_data = (input_img_data - 0.5) * 20 + 128\n\nfor i in range(steps):\n if i%20 == 0:\n print(\"%d/%d\" % (i+1,steps))\n loss_value, grads_value = iterate([input_img_data])\n input_img_data += grads_value * lr\n\n\n# util function to convert a tensor into a valid image\ndef deprocess_image(x):\n # normalize tensor: center on 0., ensure std is 0.1\n x -= x.mean()\n x /= (x.std() + 1e-5)\n x *= 0.1\n\n # clip to [0, 1]\n x += 0.5\n x = np.clip(x, 0, 1)\n\n # convert to RGB array\n x *= 255\n #x = x.transpose((1, 2, 0))\n x = np.clip(x, 0, 255).astype('uint8')\n return np.squeeze(x)\n\nimg = input_img_data[0]\nimg = deprocess_image(img)\nprint(str(img.shape))\nfrom scipy.misc import imsave\nimsave('%s_filter_%d.png' % (layer_name, filter_index), img)\n\nfig = plt.figure(figsize=(img_width,img_height))\nax = fig.add_subplot(1,1,1)\nax.imshow(img, cmap='gray')\n\n\n",
"votes": null
},
{
"id": "415069",
"postDate": "11/04/2018 10:34:48",
"content": "as an example
\n\n
![\"https://image.ibb.co/bE3jnq/CAM0232b2ba-bbbd-11e8-b2ba-ac1f6b6435d0.jpg\"](\"https://image.ibb.co/bE3jnq/CAM0232b2ba-bbbd-11e8-b2ba-ac1f6b6435d0.jpg\")
I may have a bug in my CAM visualization code so I'll just post it below:
\n\nimport os\nimport sys\nsys.path.append('../../')\nfrom dependencies import *\nfrom settings import *\nfrom reproducibility import *\nfrom models.Protein.ResNet34_4chan_gap_scSE import ResNet34 as Net\n\nSIZE = 256\nFOLD = 0\nimport io\nimport requests\nfrom PIL import Image\nfrom torchvision import models, transforms\nfrom torch.autograd import Variable\nfrom torch.nn import functional as F\nimport numpy as np\nimport cv2\nimport pdb\n\ndef valid_augment(image,label, index):\n cache = None#Struct(image = image.copy())\n #image, mask = do_resize2(image, mask, SIZE, SIZE)\n #image, mask = do_center_pad_to_factor2(image, mask, factor = FACTOR)\n image = image.astype(np.float32)/255\n #image = (image-MEAN)/STD\n image = np.rollaxis(image,2,0)\n for o in range(4):\n image[o,:,:] = (image[o,:,:]-MEAN[o])/STD[o]\n gc.collect()\n return image,label,index,cache\n\nMEAN=[0.08069, 0.05258, 0.05487, 0.08282]\nSTD=[0.13704, 0.10145, 0.15313, 0.13814]\n# input image\nLABELS_URL = 'https://s3.amazonaws.com/outcome-blog/imagenet/labels.json'\nIMG_URL = 'http://media.mlive.com/news_impact/photo/9933031-large.jpg'\n\nvalid_dataset = ProteinDataset('list_valid'+str(FOLD)+'_3108', valid_augment, 'train',size=SIZE)\nvalid_loader = DataLoader(\n valid_dataset,\n sampler = RandomSampler(valid_dataset),\n batch_size = 1,\n drop_last = False,\n num_workers = 8,\n pin_memory = False,\n collate_fn = null_collate)\nnet = Net().cuda()\nROOT = \"/data/kaggle/protein/checkpoints/list_train0_27964/\"\ninitial_checkpoint = ROOT+\"ResNet34_gap_f1_OHEM_51200099500_model.pth\"\n\nprint('\\tinitial_checkpoint = %s\\n' % initial_checkpoint)\nnet.load_state_dict(torch.load(initial_checkpoint, map_location=lambda storage, loc: storage))\n\nnet.eval()\n\n# hook the feature extractor\nfeatures_blobs = []\ndef hook_feature(module, input, output):\n features_blobs.append(output.data.cpu().numpy())\n\nnet._modules.get('attention5').register_forward_hook(hook_feature)\n\n# get the softmax weight\nparams = list(net.parameters())\nweight_softmax = np.squeeze(params[-2].data.cpu().numpy())\n#weight_softmax = np.squeeze(net.logit.parameters().data.cpu().numpy())\n#print(type(params[-2]))\ndef returnCAM(feature_conv, weight_softmax, class_idx):\n # generate the class activation maps upsample to 256x256\n size_upsample = (SIZE, SIZE)\n bz, nc, h, w = feature_conv.shape\n output_cam = []\n for idx in class_idx:\n cam = weight_softmax[idx].dot(feature_conv.reshape((nc, h*w)))\n cam = cam.reshape(h, w)\n cam = cam - np.min(cam)\n cam_img = cam / np.max(cam)\n cam_img = np.uint8(255 * cam_img)\n output_cam.append(cv2.resize(cam_img, size_upsample))\n return output_cam\n\nfont = cv2.FONT_HERSHEY_SIMPLEX\nbottomLeftCornerOfText = (10,500)\nfontScale = 0.8\nfontColor = (255,255,255)\nlineType = 2\n\nfor input, truth, index, cache in tqdm(valid_loader):\n with torch.no_grad():\n logit = net(input.cuda())\n\n #print(input)\n #print(truth)\n #print(index)\n\n classes = valid_dataset.label_dict\n #print(classes)\n h_x = F.sigmoid(logit).data.squeeze()\n probs, idx = h_x.sort(0, True)\n probs = probs.cpu().numpy()\n idx = idx.cpu().numpy()\n\n # output the prediction\n #for i in range(0, 5):\n # print('{:.3f} -> {}'.format(probs[i], classes[idx[i]]))\n\n # generate class activation mapping for the top1 prediction\n CAMs = returnCAM(features_blobs[0], weight_softmax, [idx[0]])\n acc = net.metric(logit, truth)\n acc = round(acc,5)\n if True: \n # render the CAM and output\n for i in range(0, 5):\n print('{:.3f} -> {}'.format(probs[i], classes[idx[i]]))\n print('output CAM.jpg for the top1 prediction: %s'%classes[idx[0]])\n imgg = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_green.png', cv2.IMREAD_GRAYSCALE)\n imgr = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_red.png', cv2.IMREAD_GRAYSCALE)\n imgb = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_blue.png', cv2.IMREAD_GRAYSCALE)\n imgy = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_yellow.png', cv2.IMREAD_GRAYSCALE)\n img = image = np.dstack((imgr,imgg,imgb))\n heatmap = cv2.applyColorMap(cv2.resize(CAMs[0],(512, 512)), cv2.COLORMAP_JET)\n result = heatmap * 0.25 + img * 0.75\n cv2.putText(result,'Metric: '+str(acc), \n bottomLeftCornerOfText, \n font, \n fontScale,\n fontColor,\n lineType)\n cv2.imwrite('CAM/CAM'+valid_dataset.ids[index[0]]+'.jpg', result)\nSome images from another method of visualization/guided training I use. Sort of like attention...
\n\n![\"\"](\"http://brians.network/images/c1.png\")
\n![\"\"](\"http://brians.network/images/c2.png\")
\n![\"\"](\"http://brians.network/images/c3.png\")
@Brian
\n\nyour attention mask is good! this is confirmed by your high LB score
", "rawMarkdown": "Brian\n\nyour attention mask is good! this is confirmed by your high LB score", "votes": null } ], "comments": [ { "id": 413289, "author_name": "hengck23", "author_url": "", "post_date": "10/31/2018 16:28:25", "content": "see also:
\n\nhttps://www.kaggle.com/c/planet-understanding-the-amazon-from-space/discussion/32953
", "votes": null, "replies": [] }, { "id": 413323, "author_name": "ldm314", "author_url": "", "post_date": "10/31/2018 17:41:24", "content": "Here is how I visualize things:
\n\nfrom keras.models import load_model\nmodel_name = 'hpa_sub_19_best.hdf5'\nprint(\"LOAD: \" + model_name)\norig_model = load_model(model_name,custom_objects=keras_custom_objects)\norig_model.summary()\n\n\n
#pick a layer by name in the previous and put in the size.\nimg_width = 64\nimg_height = 64\nlr = 1.\nsteps = 400\n#get layer\nlayer_dict = dict([(layer.name, layer) for layer in orig_model.layers])\nlayer_name = 'block3'\nfilter_index = 42 # can be any integer from 0 to 511, if there are 512 filters in that layer\n\ninput_img = orig_model.input\n\n# build a loss function that maximizes the activation\n# of the nth filter of the layer considered\nlayer_output = layer_dict[layer_name].output\nloss = K.mean(layer_output[:, :, :, filter_index])\n\n# compute the gradient of the input picture wrt this loss\ngrads = K.gradients(loss, input_img)[0]\n\n# normalization trick: we normalize the gradient\ngrads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)\n\n# this function returns the loss and grads given the input picture\niterate = K.function([input_img], [loss, grads])\n\n# we start from a gray image with some noise\nif K.image_data_format() == 'channels_first':\n input_img_data = np.random.random((1, 3, img_width, img_height))\nelse:\n input_img_data = np.random.random((1, img_width, img_height, 3))\ninput_img_data = (input_img_data - 0.5) * 20 + 128\n\nfor i in range(steps):\n if i%20 == 0:\n print(\"%d/%d\" % (i+1,steps))\n loss_value, grads_value = iterate([input_img_data])\n input_img_data += grads_value * lr\n\n\n# util function to convert a tensor into a valid image\ndef deprocess_image(x):\n # normalize tensor: center on 0., ensure std is 0.1\n x -= x.mean()\n x /= (x.std() + 1e-5)\n x *= 0.1\n\n # clip to [0, 1]\n x += 0.5\n x = np.clip(x, 0, 1)\n\n # convert to RGB array\n x *= 255\n #x = x.transpose((1, 2, 0))\n x = np.clip(x, 0, 255).astype('uint8')\n return np.squeeze(x)\n\nimg = input_img_data[0]\nimg = deprocess_image(img)\nprint(str(img.shape))\nfrom scipy.misc import imsave\nimsave('%s_filter_%d.png' % (layer_name, filter_index), img)\n\nfig = plt.figure(figsize=(img_width,img_height))\nax = fig.add_subplot(1,1,1)\nax.imshow(img, cmap='gray')\n\n\n",
"votes": null,
"replies": []
},
{
"id": 415069,
"author_name": "hengck23",
"author_url": "",
"post_date": "11/04/2018 10:34:48",
"content": "as an example
\n\n
I may have a bug in my CAM visualization code so I'll just post it below:
\n\nimport os\nimport sys\nsys.path.append('../../')\nfrom dependencies import *\nfrom settings import *\nfrom reproducibility import *\nfrom models.Protein.ResNet34_4chan_gap_scSE import ResNet34 as Net\n\nSIZE = 256\nFOLD = 0\nimport io\nimport requests\nfrom PIL import Image\nfrom torchvision import models, transforms\nfrom torch.autograd import Variable\nfrom torch.nn import functional as F\nimport numpy as np\nimport cv2\nimport pdb\n\ndef valid_augment(image,label, index):\n cache = None#Struct(image = image.copy())\n #image, mask = do_resize2(image, mask, SIZE, SIZE)\n #image, mask = do_center_pad_to_factor2(image, mask, factor = FACTOR)\n image = image.astype(np.float32)/255\n #image = (image-MEAN)/STD\n image = np.rollaxis(image,2,0)\n for o in range(4):\n image[o,:,:] = (image[o,:,:]-MEAN[o])/STD[o]\n gc.collect()\n return image,label,index,cache\n\nMEAN=[0.08069, 0.05258, 0.05487, 0.08282]\nSTD=[0.13704, 0.10145, 0.15313, 0.13814]\n# input image\nLABELS_URL = 'https://s3.amazonaws.com/outcome-blog/imagenet/labels.json'\nIMG_URL = 'http://media.mlive.com/news_impact/photo/9933031-large.jpg'\n\nvalid_dataset = ProteinDataset('list_valid'+str(FOLD)+'_3108', valid_augment, 'train',size=SIZE)\nvalid_loader = DataLoader(\n valid_dataset,\n sampler = RandomSampler(valid_dataset),\n batch_size = 1,\n drop_last = False,\n num_workers = 8,\n pin_memory = False,\n collate_fn = null_collate)\nnet = Net().cuda()\nROOT = \"/data/kaggle/protein/checkpoints/list_train0_27964/\"\ninitial_checkpoint = ROOT+\"ResNet34_gap_f1_OHEM_51200099500_model.pth\"\n\nprint('\\tinitial_checkpoint = %s\\n' % initial_checkpoint)\nnet.load_state_dict(torch.load(initial_checkpoint, map_location=lambda storage, loc: storage))\n\nnet.eval()\n\n# hook the feature extractor\nfeatures_blobs = []\ndef hook_feature(module, input, output):\n features_blobs.append(output.data.cpu().numpy())\n\nnet._modules.get('attention5').register_forward_hook(hook_feature)\n\n# get the softmax weight\nparams = list(net.parameters())\nweight_softmax = np.squeeze(params[-2].data.cpu().numpy())\n#weight_softmax = np.squeeze(net.logit.parameters().data.cpu().numpy())\n#print(type(params[-2]))\ndef returnCAM(feature_conv, weight_softmax, class_idx):\n # generate the class activation maps upsample to 256x256\n size_upsample = (SIZE, SIZE)\n bz, nc, h, w = feature_conv.shape\n output_cam = []\n for idx in class_idx:\n cam = weight_softmax[idx].dot(feature_conv.reshape((nc, h*w)))\n cam = cam.reshape(h, w)\n cam = cam - np.min(cam)\n cam_img = cam / np.max(cam)\n cam_img = np.uint8(255 * cam_img)\n output_cam.append(cv2.resize(cam_img, size_upsample))\n return output_cam\n\nfont = cv2.FONT_HERSHEY_SIMPLEX\nbottomLeftCornerOfText = (10,500)\nfontScale = 0.8\nfontColor = (255,255,255)\nlineType = 2\n\nfor input, truth, index, cache in tqdm(valid_loader):\n with torch.no_grad():\n logit = net(input.cuda())\n\n #print(input)\n #print(truth)\n #print(index)\n\n classes = valid_dataset.label_dict\n #print(classes)\n h_x = F.sigmoid(logit).data.squeeze()\n probs, idx = h_x.sort(0, True)\n probs = probs.cpu().numpy()\n idx = idx.cpu().numpy()\n\n # output the prediction\n #for i in range(0, 5):\n # print('{:.3f} -> {}'.format(probs[i], classes[idx[i]]))\n\n # generate class activation mapping for the top1 prediction\n CAMs = returnCAM(features_blobs[0], weight_softmax, [idx[0]])\n acc = net.metric(logit, truth)\n acc = round(acc,5)\n if True: \n # render the CAM and output\n for i in range(0, 5):\n print('{:.3f} -> {}'.format(probs[i], classes[idx[i]]))\n print('output CAM.jpg for the top1 prediction: %s'%classes[idx[0]])\n imgg = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_green.png', cv2.IMREAD_GRAYSCALE)\n imgr = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_red.png', cv2.IMREAD_GRAYSCALE)\n imgb = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_blue.png', cv2.IMREAD_GRAYSCALE)\n imgy = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_yellow.png', cv2.IMREAD_GRAYSCALE)\n img = image = np.dstack((imgr,imgg,imgb))\n heatmap = cv2.applyColorMap(cv2.resize(CAMs[0],(512, 512)), cv2.COLORMAP_JET)\n result = heatmap * 0.25 + img * 0.75\n cv2.putText(result,'Metric: '+str(acc), \n bottomLeftCornerOfText, \n font, \n fontScale,\n fontColor,\n lineType)\n cv2.imwrite('CAM/CAM'+valid_dataset.ids[index[0]]+'.jpg', result)\nSome images from another method of visualization/guided training I use. Sort of like attention...
\n\n\n\n
@Brian
\n\nyour attention mask is good! this is confirmed by your high LB score
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "413233": "some code for your visualization:\n\nhttps://github.com/utkuozbulak/pytorch-cnn-visualizations#gradient-visualization\n\nhttps://github.com/eugenelet/VisualBackProp-PyTorch\n\nhttps://github.com/handong1587/handong1587.github.io/blob/master/_posts/deep_learning/2015-10-09-visulizing-interpreting-cnn.md\n\n---\n\ni hope to post mine when my results are ready", "413289": "see also:\n\nhttps://www.kaggle.com/c/planet-understanding-the-amazon-from-space/discussion/32953", "413323": "Here is how I visualize things:\n\nfrom keras.models import load_model\nmodel_name = 'hpa_sub_19_best.hdf5'\nprint(\"LOAD: \" + model_name)\norig_model = load_model(model_name,custom_objects=keras_custom_objects)\norig_model.summary()\n\n\n\n
#pick a layer by name in the previous and put in the size.\nimg_width = 64\nimg_height = 64\nlr = 1.\nsteps = 400\n#get layer\nlayer_dict = dict([(layer.name, layer) for layer in orig_model.layers])\nlayer_name = 'block3'\nfilter_index = 42 # can be any integer from 0 to 511, if there are 512 filters in that layer\n\ninput_img = orig_model.input\n\n# build a loss function that maximizes the activation\n# of the nth filter of the layer considered\nlayer_output = layer_dict[layer_name].output\nloss = K.mean(layer_output[:, :, :, filter_index])\n\n# compute the gradient of the input picture wrt this loss\ngrads = K.gradients(loss, input_img)[0]\n\n# normalization trick: we normalize the gradient\ngrads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)\n\n# this function returns the loss and grads given the input picture\niterate = K.function([input_img], [loss, grads])\n\n# we start from a gray image with some noise\nif K.image_data_format() == 'channels_first':\n input_img_data = np.random.random((1, 3, img_width, img_height))\nelse:\n input_img_data = np.random.random((1, img_width, img_height, 3))\ninput_img_data = (input_img_data - 0.5) * 20 + 128\n\nfor i in range(steps):\n if i%20 == 0:\n print(\"%d/%d\" % (i+1,steps))\n loss_value, grads_value = iterate([input_img_data])\n input_img_data += grads_value * lr\n\n\n# util function to convert a tensor into a valid image\ndef deprocess_image(x):\n # normalize tensor: center on 0., ensure std is 0.1\n x -= x.mean()\n x /= (x.std() + 1e-5)\n x *= 0.1\n\n # clip to [0, 1]\n x += 0.5\n x = np.clip(x, 0, 1)\n\n # convert to RGB array\n x *= 255\n #x = x.transpose((1, 2, 0))\n x = np.clip(x, 0, 255).astype('uint8')\n return np.squeeze(x)\n\nimg = input_img_data[0]\nimg = deprocess_image(img)\nprint(str(img.shape))\nfrom scipy.misc import imsave\nimsave('%s_filter_%d.png' % (layer_name, filter_index), img)\n\nfig = plt.figure(figsize=(img_width,img_height))\nax = fig.add_subplot(1,1,1)\nax.imshow(img, cmap='gray')\n\n\n",
"415069": "as an example\n\n ![enter image description here][1]\n\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/415069/10597/activation.png",
"416630": "![https://image.ibb.co/bE3jnq/CAM0232b2ba-bbbd-11e8-b2ba-ac1f6b6435d0.jpg][1]\n\nI may have a bug in my CAM visualization code so I'll just post it below:\n\n import os\n import sys\n sys.path.append('../../')\n\tfrom dependencies import *\n\tfrom settings import *\n\tfrom reproducibility import *\n\tfrom models.Protein.ResNet34_4chan_gap_scSE import ResNet34 as Net\n\n\tSIZE = 256\n\tFOLD = 0\n\timport io\n\timport requests\n\tfrom PIL import Image\n\tfrom torchvision import models, transforms\n\tfrom torch.autograd import Variable\n\tfrom torch.nn import functional as F\n\timport numpy as np\n\timport cv2\n\timport pdb\n\n\tdef valid_augment(image,label, index):\n\t cache = None#Struct(image = image.copy())\n\t #image, mask = do_resize2(image, mask, SIZE, SIZE)\n\t #image, mask = do_center_pad_to_factor2(image, mask, factor = FACTOR)\n\t image = image.astype(np.float32)/255\n\t #image = (image-MEAN)/STD\n\t image = np.rollaxis(image,2,0)\n\t for o in range(4):\n\t image[o,:,:] = (image[o,:,:]-MEAN[o])/STD[o]\n\t gc.collect()\n\t return image,label,index,cache\n\n\tMEAN=[0.08069, 0.05258, 0.05487, 0.08282]\n\tSTD=[0.13704, 0.10145, 0.15313, 0.13814]\n\t# input image\n\tLABELS_URL = 'https://s3.amazonaws.com/outcome-blog/imagenet/labels.json'\n\tIMG_URL = 'http://media.mlive.com/news_impact/photo/9933031-large.jpg'\n\n\tvalid_dataset = ProteinDataset('list_valid'+str(FOLD)+'_3108', valid_augment, 'train',size=SIZE)\n\tvalid_loader = DataLoader(\n\t valid_dataset,\n\t sampler = RandomSampler(valid_dataset),\n\t batch_size = 1,\n\t drop_last = False,\n\t num_workers = 8,\n\t pin_memory = False,\n\t collate_fn = null_collate)\n\tnet = Net().cuda()\n\tROOT = \"/data/kaggle/protein/checkpoints/list_train0_27964/\"\n\tinitial_checkpoint = ROOT+\"ResNet34_gap_f1_OHEM_51200099500_model.pth\"\n\n\tprint('\\tinitial_checkpoint = %s\\n' % initial_checkpoint)\n\tnet.load_state_dict(torch.load(initial_checkpoint, map_location=lambda storage, loc: storage))\n\n\tnet.eval()\n\n\t# hook the feature extractor\n\tfeatures_blobs = []\n\tdef hook_feature(module, input, output):\n\t features_blobs.append(output.data.cpu().numpy())\n\n\tnet._modules.get('attention5').register_forward_hook(hook_feature)\n\n\t# get the softmax weight\n\tparams = list(net.parameters())\n\tweight_softmax = np.squeeze(params[-2].data.cpu().numpy())\n\t#weight_softmax = np.squeeze(net.logit.parameters().data.cpu().numpy())\n\t#print(type(params[-2]))\n\tdef returnCAM(feature_conv, weight_softmax, class_idx):\n\t # generate the class activation maps upsample to 256x256\n\t size_upsample = (SIZE, SIZE)\n\t bz, nc, h, w = feature_conv.shape\n\t output_cam = []\n\t for idx in class_idx:\n\t cam = weight_softmax[idx].dot(feature_conv.reshape((nc, h*w)))\n\t cam = cam.reshape(h, w)\n\t cam = cam - np.min(cam)\n\t cam_img = cam / np.max(cam)\n\t cam_img = np.uint8(255 * cam_img)\n\t output_cam.append(cv2.resize(cam_img, size_upsample))\n\t return output_cam\n\n\tfont = cv2.FONT_HERSHEY_SIMPLEX\n\tbottomLeftCornerOfText = (10,500)\n\tfontScale = 0.8\n\tfontColor = (255,255,255)\n\tlineType = 2\n\n\tfor input, truth, index, cache in tqdm(valid_loader):\n\t with torch.no_grad():\n\t logit = net(input.cuda())\n\n\t #print(input)\n\t #print(truth)\n\t #print(index)\n\n\t classes = valid_dataset.label_dict\n\t #print(classes)\n\t h_x = F.sigmoid(logit).data.squeeze()\n\t probs, idx = h_x.sort(0, True)\n\t probs = probs.cpu().numpy()\n\t idx = idx.cpu().numpy()\n\n\t # output the prediction\n\t #for i in range(0, 5):\n\t # print('{:.3f} -> {}'.format(probs[i], classes[idx[i]]))\n\n\t # generate class activation mapping for the top1 prediction\n\t CAMs = returnCAM(features_blobs[0], weight_softmax, [idx[0]])\n\t acc = net.metric(logit, truth)\n\t acc = round(acc,5)\n\t if True: \n\t # render the CAM and output\n\t for i in range(0, 5):\n\t print('{:.3f} -> {}'.format(probs[i], classes[idx[i]]))\n\t print('output CAM.jpg for the top1 prediction: %s'%classes[idx[0]])\n\t imgg = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_green.png', cv2.IMREAD_GRAYSCALE)\n\t imgr = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_red.png', cv2.IMREAD_GRAYSCALE)\n\t imgb = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_blue.png', cv2.IMREAD_GRAYSCALE)\n\t imgy = cv2.imread('/home/alexanderliao/protein/train/'+valid_dataset.ids[index[0]]+'_yellow.png', cv2.IMREAD_GRAYSCALE)\n\t img = image = np.dstack((imgr,imgg,imgb))\n\t heatmap = cv2.applyColorMap(cv2.resize(CAMs[0],(512, 512)), cv2.COLORMAP_JET)\n\t result = heatmap * 0.25 + img * 0.75\n\t cv2.putText(result,'Metric: '+str(acc), \n\t bottomLeftCornerOfText, \n\t font, \n\t fontScale,\n\t fontColor,\n\t lineType)\n\t cv2.imwrite('CAM/CAM'+valid_dataset.ids[index[0]]+'.jpg', result)\n\n\n [1]: https://image.ibb.co/bE3jnq/CAM0232b2ba-bbbd-11e8-b2ba-ac1f6b6435d0.jpg",
"416758": "Some images from another method of visualization/guided training I use. Sort of like attention...\n\n\n\n",
"416777": "Brian\n\nyour attention mask is good! this is confirmed by your high LB score"
},
"source": "meta"
}