{"cells":[{"metadata":{},"cell_type":"markdown","source":"# ***Disclaimer:*** \nHello Kagglers! I am a Solution Architect with the Google Cloud Platform. I am a coach for this competition, the focus of my contributions is on helping users to leverage GCP components (GCS, TPUs, BigQueryetc..) in order to solve large problems. My ideas and contributions represent my own opinion, and are not representative of an official recommendation by Google. Also, I try to develop notebooks quickly in order to help users early in competitions. There may be better ways to solving particular problems, I welcome comments and suggestions. Use my contributions at your own risk, I don't garantee that they will help on winning any competition, but I am hoping to learn by collaborating with everyone."},{"metadata":{},"cell_type":"markdown","source":"# Objective:\n\n\nThe objective of this notebook is to demonstrate how make a submission by using a trained model previously stored in a private dataset. This version is optimized to be memory efficient. I have previously tried to process the images in memory, and kept running out of memory. The solution was to write the image data into a TFRecord file and then use tf.Data.Dataset to feed the TFRecord file to the model for predictions. The tf.Data.Dataset reads the data sequentially from the file and therefore no extra copy of the image is needed in memory. This was the ONLY way I found for submissions to execute succefully both in the public and private datasets.\n\nThe Keras model that is used was built and trained using TPUs as described in this notebook, the dataset was all tiles with gloms for 200 epochs:\n[https://www.kaggle.com/marcosnovaes/hubmap-unet-keras-model-fit-with-tpu/](https://www.kaggle.com/marcosnovaes/hubmap-unet-keras-model-fit-with-tpu/)\n\nI also have a GPU version that is not as powerful :\n[https://www.kaggle.com/marcosnovaes/hubmap-unet-keras-model-fit-with-gpu](https://www.kaggle.com/marcosnovaes/hubmap-unet-keras-model-fit-with-gpu)\n\nThe Unet Keras model utilized is the one proposed by a [popular paper in biomedical image segmentation](https://arxiv.org/abs/1505.04597), by (Olaf Ronneberger, Philipp Fischer, Thomas Brox).\n\nThe particular implementation used is the one proposed by by [Dr. Bradley Erickson](https://github.com/slowvak), available in the: [The Magician's Corner repository](https://github.com/RSNA/MagiciansCorner/blob/master/UNetWithTensorflow.ipynb). \n\nThe basic modification that I have made to the implementation provided by Dr. Erickson is to enable the Tensorflow distributed training strategy (tf.strategy). You will notice that the function model.fit() is used within a strategy.scope(), so that it leverages either GPU or TPU acceleration. \n\nIn previous notebooks, I demonstrated how to read the competition data and produce a TFRecord dataset tiling the images in 512x512 tiles. This Notebook will use this dataset as input to the Keras Unet model:\n--> [Link to the TFRecord Dataset used for training.](https://www.kaggle.com/marcosnovaes/hubmap-tfrecord-512)\n\nPrevious Notebooks in this competition: \n\n[https://www.kaggle.com/marcosnovaes/hubmap-3-unet-models-with-keras-cpu-gpu/](https://www.kaggle.com/marcosnovaes/hubmap-3-unet-models-with-keras-cpu-gpu/): Investigates three implementations of the Unet model\n\n[https://www.kaggle.com/marcosnovaes/hubmap-read-data-and-build-tfrecords/](https://www.kaggle.com/marcosnovaes/hubmap-read-data-and-build-tfrecords/): Demonstrates how the TFRecord Dataset was built\n\n[https://www.kaggle.com/marcosnovaes/hubmap-looking-at-tfrecords/](https://www.kaggle.com/marcosnovaes/hubmap-looking-at-tfrecords/): Explains how to read the data using the TFRecord Dataset"},{"metadata":{},"cell_type":"markdown","source":"# Setup\n\n1) You need to add your own saved model to a dataset and include it to this notebook. I am not making by model public so as not to make it too easy for everyone, but you can use the above referenced notebook to train your own model.\n2) Edit the code to use the name of your own trained model\n3) make sure internet is turned off\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls -l /kaggle/input","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport sys\nimport glob\n#import random\nimport warnings\n\nimport tifffile as tiff\nfrom tqdm.notebook import tqdm\n\nimport numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\n\n#from tqdm import tqdm\n#from itertools import chain\nfrom skimage.io import imread, imshow, imread_collection, concatenate_images\nfrom skimage.transform import resize\nimport skimage.transform\nimport skimage.measure\nfrom skimage.morphology import label\nimport cv2\n\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.layers import Input\n\nfrom tensorflow.keras.layers import Conv2D, Conv2DTranspose\nfrom tensorflow.keras.layers import MaxPooling2D, UpSampling2D\nfrom tensorflow.keras.layers import concatenate\n\nfrom tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint\nfrom tensorflow.keras import backend as K\n\nfrom tensorflow.keras import layers\n\nfrom keras.engine.topology import Layer\n\nfrom tensorflow.keras.optimizers import Adam\nfrom keras.utils.generic_utils import get_custom_objects\n\n\nfrom kaggle_datasets import KaggleDatasets\nfrom kaggle_secrets import UserSecretsClient\n\nimport tensorflow as tf\n\nimport gc","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Utilities serialize data into a TFRecord\ndef _bytes_feature(value):\n  \"\"\"Returns a bytes_list from a string / byte.\"\"\"\n  if isinstance(value, type(tf.constant(0))):\n    value = value.numpy() # BytesList won't unpack a string from an EagerTensor.\n  return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))\n\ndef _float_feature(value):\n  \"\"\"Returns a float_list from a float / double.\"\"\"\n  return tf.train.Feature(float_list=tf.train.FloatList(value=[value]))\n\ndef _int64_feature(value):\n  \"\"\"Returns an int64_list from a bool / enum / int / uint.\"\"\"\n  return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def image_example(image, tile_col_number, tile_row_number):\n    image_shape = image.shape\n    \n    img_bytes = image.tostring()\n    \n    feature = {\n        'height': _int64_feature(image_shape[0]),\n        'width': _int64_feature(image_shape[1]),\n        'num_channels': _int64_feature(image_shape[2]),\n        'img_bytes': _bytes_feature(img_bytes),\n        'tile_col_number': _int64_feature(tile_col_number),\n        'tile_row_number': _int64_feature(tile_row_number),\n    }\n    return tf.train.Example(features=tf.train.Features(feature=feature))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a dictionary describing the features.\nimage_feature_description = {\n    'height': tf.io.FixedLenFeature([], tf.int64),\n    'width': tf.io.FixedLenFeature([], tf.int64),\n    'num_channels': tf.io.FixedLenFeature([], tf.int64),\n    'img_bytes': tf.io.FixedLenFeature([], tf.string),\n    'tile_col_number': tf.io.FixedLenFeature([], tf.int64),\n    'tile_row_number': tf.io.FixedLenFeature([], tf.int64),\n}\n\ndef _parse_image(example_proto):\n    single_example = tf.io.parse_single_example(example_proto, image_feature_description)\n    img_height = single_example['height']\n    img_width = single_example['width']\n    num_channels = single_example['num_channels']\n    \n    img_bytes =  tf.io.decode_raw(single_example['img_bytes'],out_type='uint8')\n    #dynamic shape\n    #img_array = tf.reshape( img_bytes, (img_height, img_width, num_channels))\n    #fixed shape\n    img_array = tf.reshape( img_bytes, (512, 512, 3))\n    img_array = tf.cast(img_array, tf.float32) / 255.0\n    \n    mtd = dict()\n    mtd['width'] = single_example['width']\n    mtd['height'] = single_example['height']\n    mtd['tile_col_number'] = single_example['tile_col_number']\n    mtd['tile_row_number'] = single_example['tile_row_number']\n   \n    return img_array, mtd\n\ndef read_dataset(storage_file_path):\n    encoded_image_dataset = tf.data.TFRecordDataset(storage_file_path, compression_type=\"GZIP\")\n    parsed_image_dataset = encoded_image_dataset.map(_parse_image)\n    return parsed_image_dataset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def dice_coeff(y_true, y_pred):\n    # add epsilon to avoid a divide by 0 error in case a slice has no pixels set\n   # we only care about relative value, not absolute so this alteration doesn't matter\n    _epsilon = 10 ** -7\n    intersections = tf.reduce_sum(y_true * y_pred)\n    unions = tf.reduce_sum(y_true + y_pred)\n    dice_scores = (2.0 * intersections + _epsilon) / (unions + _epsilon)\n    return dice_scores\n\ndef dice_loss(y_true, y_pred):\n    loss = 1 - dice_coeff(y_true, y_pred)\n    return loss\n  \nget_custom_objects().update({\"dice\": dice_loss})\n\nclass LayerNormalization (Layer) :\n    \n    def call(self, x, mask=None, training=None) :\n        axis = list (range (1, len (x.shape)))\n        x /= K.std (x, axis = axis, keepdims = True) + K.epsilon()\n        x -= K.mean (x, axis = axis, keepdims = True)\n        return x\n        \ndef compute_output_shape(self, input_shape):\n    return input_shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def magic_unet(act_fn = 'relu', init_fn = 'he_normal', width=512, height = 512, channels = 3): \n    inputs = Input((512,512,3))\n    act_fn = 'relu'\n    init_fn = 'he_normal'\n\n    # note we use linear function before layer normalization\n    conv1 = Conv2D(8, 5, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(inputs)\n    conv1 = LayerNormalization()(conv1)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2 = Conv2D(16, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(pool1)\n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(32, 3, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(pool2)\n    conv3 = LayerNormalization()(conv3)\n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n    conv4 = Conv2D(64, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(pool3)\n    pool4 = MaxPooling2D(pool_size=(2, 2))(conv4)\n\n    conv5 = Conv2D(72, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(pool4)\n\n    up6 = Conv2D(64, 2, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv5))\n    up6 = LayerNormalization()(up6)\n    merge6 = concatenate([conv4,up6], axis = 3)\n    conv6 = Conv2D(64, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge6)\n\n    up7 = Conv2D(32, 2, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv6))\n    merge7 = concatenate([conv3,up7], axis = 3)\n    conv7 = Conv2D(32, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge7)\n\n    up8 = Conv2D(16, 2, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv7))\n    up8 = LayerNormalization()(up8)\n    merge8 = concatenate([conv2,up8], axis = 3)\n    conv8 = Conv2D(16, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge8)\n\n    up9 = Conv2D(8, 2, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv8))\n    merge9 = concatenate([conv1,up9], axis = 3)\n    conv9 = Conv2D(8, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge9)\n    conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)\n    model = Model(inputs = inputs, outputs = conv10)\n\n    return model\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"!ls -l /kaggle/input","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls -l /kaggle/input/humap-sample-models","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!head /kaggle/input/hubmap-kidney-segmentation/sample_submission.csv","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n#tf.config.experimental_connect_to_cluster(tpu)\n#tf.tpu.experimental.initialize_tpu_system(tpu)\n\n#strategy = tf.distribute.experimental.TPUStrategy(tpu)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"strategy = tf.distribute.MirroredStrategy()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tf.config.list_physical_devices()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tf.config.list_logical_devices()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.models import load_model\n\nwith strategy.scope():\n    pred_model = magic_unet()\n    #pred_model.load_weights(\"/kaggle/input/humap-sample-models/hubmap-keras-tpu-200.h5\")\n    pred_model.load_weights(\"/kaggle/input/humap-sample-models/hubmap-tpu-cortex-200.h5\")\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pred_model.inputs","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#pred_model.save('/kaggle/working/hubmap-keras-tpu-200.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def predict_mask( image_shape, dataset_path, pred_model, bool_cutoff):\n    \n    pred_mask = np.zeros((image_shape[0], image_shape[1]), dtype = 'bool')\n    #pred_mask = np.zeros((image_shape[0], image_shape[1]), dtype = 'float32')\n    #pred_mask = pred_mask.astype(bool)\n    \n    print('Reading Dataset')\n    input_dataset = read_dataset( dataset_path )\n    gc.collect()\n\n    print('Calling prediction')\n    #pred_float_masks = pred_model.predict(input_dataset.batch(1),batch_size = 8, verbose=1)\n      \n    #print(\"Filling tile masks into large mask\")\n    input_batch_size = 512\n    input_dataset = input_dataset.batch(input_batch_size)\n    \n    for input_batch, input_mtd in input_dataset.as_numpy_iterator():\n        # fill in the predicted masks into the large mask\n        pred_float_masks = pred_model.predict(input_batch,batch_size = 8, verbose=1)  \n        # overwrite the big mask with these values at the right place\n        #\n        for index in range(pred_float_masks.shape[0]):\n            tile_col = input_mtd['tile_col_number'][index]\n            tile_row = input_mtd['tile_row_number'][index]\n            \n            tile_height_begin = tile_row * tile_size\n            tile_height_end = tile_height_begin + tile_size  \n            tile_width_begin = tile_col * tile_size\n            tile_width_end = tile_width_begin + tile_size\n            #pred_bool_mask = pred_float_masks[index,:,:,0] > bool_cutoff\n            # downsample by 8x8 blocks with min to eliminate small areas\n            reduced_pred = skimage.measure.block_reduce(pred_float_masks[index,:,:,0], (4,4), np.min)\n            # upsample back to 512x512\n            resized_pred = skimage.transform.resize(reduced_pred, (512,512))\n            # apply boolean cutoff\n            #pred_bool_mask = pred_float_masks[index,:,:,0] > bool_cutoff\n            pred_bool_mask = resized_pred > bool_cutoff\n            pred_mask[tile_height_begin:tile_height_end, tile_width_begin:tile_width_end] = pred_bool_mask\n            #pred_mask[tile_height_begin:tile_height_end, tile_width_begin:tile_width_end] = pred_float_masks[index,:,:,0] \n\n    return pred_mask\n\n\ndef write_dataset( image_path, img_id, tile_size, output_path, reduce_factor=0):\n    baseimage = tiff.imread(image_path)\n    print ('original image shape',baseimage.shape)\n    baseimage = np.squeeze(baseimage)\n    if( baseimage.shape[0] == 3):\n        baseimage = baseimage.swapaxes(0,1)\n        baseimage = baseimage.swapaxes(1,2)\n        print ('swaped shape',baseimage.shape)\n    # cv2.resize is causing oom,...\n    # resize the image if required\n    if reduce_factor > 0:\n        baseimage = cv2.resize(baseimage,(baseimage.shape[1]//reduce_factor,baseimage.shape[0]//reduce_factor),\n                         interpolation = cv2.INTER_AREA)\n    print ('resized image shape',baseimage.shape)    \n        \n    img_shape = baseimage.shape\n    img_height = baseimage.shape[0]\n    img_width = baseimage.shape[1]\n    num_tile_rows = img_height // tile_size\n    num_tile_cols = img_width // tile_size\n    \n    dataset_path = ('/kaggle/working/{}_tiles.tfrecords'.format(img_id))\n    opts = tf.io.TFRecordOptions(compression_type=\"GZIP\")\n    writer = tf.io.TFRecordWriter(dataset_path, opts)\n         \n    for tile_col_number in tqdm(range(num_tile_cols),total=num_tile_cols, desc = 'filtering data by tile columns' ):\n        \n        for tile_row_number in range(num_tile_rows):\n\n            tile_height_begin = tile_row_number * tile_size\n            tile_height_end = tile_height_begin + tile_size\n            \n            tile_width_begin = tile_col_number * tile_size\n            tile_width_end = tile_width_begin + tile_size\n\n            image_tile = baseimage[tile_height_begin:tile_height_end, tile_width_begin:tile_width_end, :]\n          \n            img_hist = np.histogram(image_tile)\n            lowband_density = np.sum(img_hist[0][0:4])\n            if lowband_density > 1000:     \n                # add tile to dataset\n               \n                tf_example = image_example(image_tile, tile_col_number, tile_row_number)\n                writer.write(tf_example.SerializeToString())\n    writer.close()\n    baseimage = np.zeros(10)# clear previous variables from mem\n    gc.collect()\n    return img_shape, dataset_path","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I leverage the RLE encoder provided by @xhlulu here:\n[https://www.kaggle.com/xhlulu/efficient-mask2rle](https://www.kaggle.com/xhlulu/efficient-mask2rle)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def mask2rle(img):\n    '''\n    Efficient implementation of mask2rle, from @paulorzp\n    --\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    Source: https://www.kaggle.com/xhlulu/efficient-mask2rle\n    '''\n    pixels = img.T.flatten()\n    pixels = np.pad(pixels, ((1, 1), ))\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\n#https://www.kaggle.com/bguberfain/memory-aware-rle-encoding\n#with bug fix\ndef rle_encode_less_memory(img):\n    #watch out for the bug\n    pixels = img.T.flatten()\n    \n    # This simplified method requires first and last pixel to be zero\n    pixels[0] = 0\n    pixels[-1] = 0\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 2\n    runs[1::2] -= runs[::2]\n    \n    return ' '.join(str(x) for x in runs)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"REDUCTION_FACTOR = 4\n\n#process all images in the test directory\nimg_file_list = glob.glob('/kaggle/input/hubmap-kidney-segmentation/test/*.tiff')\n#img_file_list = img_file_list[::-1]\n#img_file_list = ['/kaggle/input/hubmap-kidney-segmentation/test/b2dc8411c.tiff']\n#img_file_list = ['/kaggle/input/hubmap-kidney-segmentation/test/26dc41664.tiff']\ntile_size = 512\nnum_images = img_file_list.__len__()\n#num_images = 1\npred_mask = []\nsubmission_df = pd.DataFrame(columns = ['id','predicted'])\n#print(\"Processing all test image files\")\nimg_shapes = []\nimg_paths = []\nimg_ids = []\nprint(\"Resizing and tiling all images\")\nfor image_index in tqdm(range(num_images),total=num_images, desc='num images processed'):\n    file_name = img_file_list[image_index]\n    prefix = file_name.split('.')\n    parts = prefix[0].split('/')\n    image_id = parts[-1]\n    image_path = img_file_list[image_index]\n    sys.stdout.flush()\n    print(\"Processing Image {}\".format(image_id))\n    # image will be reduced by REDUCTION_FACTOR\n    img_shape, dataset_path = write_dataset(image_path, image_id, 512,'/kaggle/working', REDUCTION_FACTOR)  \n    img_shapes.append(img_shape)\n    img_paths.append(dataset_path)\n    img_ids.append(image_id)\n    \nprint(\"Generating Prediction Masks\")    \nfor image_index in tqdm(range(num_images),total=num_images, desc='num masks processed'):\n    print(\"Predicting Masks {}\".format(image_id))\n    pred_mask = predict_mask( img_shapes[image_index], img_paths[image_index], pred_model, 0.5)\n    print(\"bool mask shape\",pred_mask.shape)\n    pred_mask = np.repeat(pred_mask, REDUCTION_FACTOR, axis=0)\n    pred_mask = np.repeat(pred_mask, REDUCTION_FACTOR, axis=1)\n    print(\"bool mask expanded shape\",pred_mask.shape )\n    print(\"RLE Encoding {}\".format(img_ids[image_index]))\n    #rle_string = mask2rle(pred_mask)\n    rle_string = rle_encode_less_memory(pred_mask)\n    #pred_mask = np.zeros(10)# clear previous variables from mem\n    #gc.collect()\n    submission_df = submission_df.append({'id':img_ids[image_index], 'predicted':rle_string},ignore_index=True)\n    print(\"Removing Dataset from Disk {}\".format(img_paths[image_index]))\n    os.remove(img_paths[image_index])\n\n#write csv\nprint(\"Writing Submission File\")\nsubmission_df.to_csv('/kaggle/working/submission.csv',index=False)\n\n#show last mask\n#plt.imshow(pred_mask)\nprint(\"Done\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pred_mask.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pred_mask.dtype","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls -l /kaggle/working/","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"verify_sub = pd.read_csv('/kaggle/working/submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"verify_sub.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#plt.imshow(pred_mask)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}