{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

About

\n\n This notebook uses Cycle GANs to trying to convert high resolution Day cityscapes images to Night images and vice versa. The dataset consists of high resolution Day and Night images. Cycle GANs allows us to use the dataset without any explicit image-target image pairs, by using cycle consistancy loss to ensure that the images retain the semantic and structural components. Hence Cycle GAN builds on top of Pix2Pix GANs, with an additional Cycle Consistency loss.\n \n* Pix2Pix Notebook : https://www.kaggle.com/code/virajkadam/segmenting-cityscapes-with-pix2pix-gans\n","metadata":{}},{"cell_type":"markdown","source":"

About Cycle GAN

\n\n Paper Link : https://www.cs.cmu.edu/~junyanz/projects/CycleGAN/CycleGAN.pdf\n\n\n Cycle GANs provide the approach to translate an image from source domain X to target domain Y in absence of paired examples. The goal is to learn a mapping G: X-> Y , and inverse mapping F: Y -> X , and introduce a cycle consistancy loss to enforce F(G(X)) = X and G(F(Y_hat)) = Y_hat. \n\n The problem that Cycle GANs address can be classified as Image to Image translation. While there are other methods that address the problem with image- target pairs, obtaining these pairs is difficult and expensive. This method is applied for the tasks of style transfer, object transfiguration, and attribute transfer and claims to outperform baseline approaches.\n\n\n\n

Formulation

\n\n Our goal is to learn two mapping functions G,F over two domains X,Y ; such that G: X -> Y and F : Y -> X. We can think of X as set of pictures, and Y as the set of paintings, for the task of converting Pictures to Paintings. The functions G and F are Generators. \n\n We also have adverserial discriminators Dx and Dy, where Dx has the task to discriminating between images from set X (pictures) and set of images generated by F(Y)/X_hat (i.e the image generated by the inverse mapping Generator F). Similarly Dy has the task to discriminating between images from set Y (Paintings) and set of images generated by G(X)/Y_hat (i.e the image generated by the generator G) .\n\n We guide the training of the GAN using an objective that has two components : \n 1) Adverserial loss: For mapping of input domain to target domain, and vice versa.\n 2) Cycle Consistency loss: For enforcing cycle consistency so that F(G(X)) = X and G(F(Y)) = Y . \n\n\n

Losses

\n\n 1) Adverserial loss : \n\n Loss for GAN G: LGAN(G,DY ,X,Y ) =Ey∼pdata (y)[log DY (y)] + Ex∼pdata (x)[log(1 −DY (G(x))]\n\n Loss for GAN F : LGAN(F,DX ,Y,X ) =Ex∼pdata (x)[log Dx (x)] + Ey∼pdata (y)[log(1 −Dx (F(y))]\n\n Where G tries to generate images similar to domain Y, and Dy tries to to discriminate between generated images and original images.\n Similarly, F tries to generate images similar to domain X, and Dx tries to to discriminate between generated images and original images.\n\n 2) Cycle Consistency loss: The network can learn to map input images to some random set permutations in the output domain, which can match the target distribution.Hence using only a adverserial loss would not guarantee a desired and consistant mapping. Hence we apply Cycle consistance loss to enforce this behavour. \n\n Forward Cycle Consistancy : x → G(x) → F(G(x)) ≈ x ; where x is the input image domain.\n\n Backward Cycle Consistancy : y → F(y) → G(F(y)) ≈ y ; where y is the target image domain.\n\n Cycle loss : Lcyc(G,F) =Ex∼pdata(x) [‖F(G(x)) −x‖1] + Ey∼pdata(y) [‖G(F(y)) −y‖1]\n\n\n

Objective

\n \n\n Hence the full loss is: \n\n L(G,F,DX ,DY) = LGAN(G,DY ,X,Y ) + LGAN(F,DX ,Y,X) + λ * Lcyc(G,F)\n\n

Implementation

\n\n 1) Architecture : Used the generator architecture from Johnson et al, with instance normalization. The discriminator uses a PatchGAN, which classifies wheather 70*70 patch in the image is real or fake.\n 2) Training Detail: \n a) Adverserial Loss : Replace the Log-likelyhood adverserial loss with the Least Square loss, as it is more stable during training.\n b) Total loss : Used the cycle consistency loss weighting parameter Lambda = 10 .\n c) Optimization : Used Adam optimizer with LR = 0.0002 for first 100 epoch, linealy decaying into 0 in next 100 epochs. (i.e total 200 epochs)\n d) Batch Size : 1 \n c) Training updates: Used history of training images for updating Dx and Dy, using a buffer that stores 50 previously generated images . This reduces model oscillations (from strivastava et al).\n\n\n","metadata":{}},{"cell_type":"markdown","source":"

Imports

\n","metadata":{}},{"cell_type":"code","source":"\n\nimport os\nimport pathlib\nimport time\nimport datetime\nimport imageio\nfrom glob import glob\n\n\n\nimport tensorflow as tf\nimport numpy as np \nimport tensorflow.keras.backend as K\nimport tensorflow_addons as tfa\n\n\nfrom matplotlib import pyplot as plt\nfrom IPython import display\nfrom termcolor import colored\nfrom tqdm import tqdm\nfrom IPython.display import Image\nimport PIL\nfrom PIL import ImageDraw\nfrom IPython import display\n\ndef color_print(print_str,\n print_color='green'):\n \n '''print in given color (default green)'''\n print(colored(print_str,print_color))\n\n","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:34.259074Z","iopub.execute_input":"2022-08-08T08:05:34.259414Z","iopub.status.idle":"2022-08-08T08:05:40.543228Z","shell.execute_reply.started":"2022-08-08T08:05:34.259339Z","shell.execute_reply":"2022-08-08T08:05:40.542242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ndef set_seed(seed):\n np.random.seed(seed)\n tf.random.set_seed(seed)\n os.environ['PYTHONHASHSEED'] = str(seed)\n os.environ['TF_DETERMINISTIC_OPS'] = '1'\n print(f'setting seed to {seed}')\n \n\nclass CFG:\n \n # Dimension of image\n IMG_WIDTH = 512\n IMG_HEIGHT = 512\n \n #resize image \n resize_height = 700\n resize_width = 1200\n \n #the lambda param in loss\n LAMBDA = 10\n #--------train pipe-------------\n BUFFER_SIZE = 100\n # The batch size of 1 produced better results for the U-Net in the original pix2pix experiment\n BATCH_SIZE = 2\n \n #cache \n cache= 50\n \n #learning rate \n learning_rate = 0.00025\n \n \n seed = 7 \n \n\nset_seed(CFG.seed)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:05:40.545264Z","iopub.execute_input":"2022-08-08T08:05:40.546020Z","iopub.status.idle":"2022-08-08T08:05:40.556145Z","shell.execute_reply.started":"2022-08-08T08:05:40.545974Z","shell.execute_reply":"2022-08-08T08:05:40.555124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data directories

\n","metadata":{}},{"cell_type":"code","source":"day_dir = '../input/daynight-cityview/day'\nnight_dir = '../input/daynight-cityview/night'\n\n\n#plotting a sample image \nplt.figure(figsize=(16,8))\n\nimg = plt.imread(day_dir + '/' + os.listdir(day_dir)[0])\nplt.imshow(img)\nplt.axis('off')\nplt.title('sample image')\nprint(f'Image dimensions {img.shape}')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:40.557651Z","iopub.execute_input":"2022-08-08T08:05:40.558313Z","iopub.status.idle":"2022-08-08T08:05:41.300421Z","shell.execute_reply.started":"2022-08-08T08:05:40.558265Z","shell.execute_reply":"2022-08-08T08:05:41.299507Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

building data loading pipeline

\n","metadata":{}},{"cell_type":"code","source":"def load_image(image_file):\n '''load a image file'''\n image = tf.io.read_file(image_file)\n image = tf.io.decode_jpeg(image)\n \n return image\n\n\ndef random_crop(image):\n '''randomly crop image into defined size '''\n cropped_image = tf.image.random_crop(image, size=[CFG.IMG_HEIGHT, CFG.IMG_WIDTH, 3])\n\n return cropped_image\n\n\ndef normalize(image):\n '''normalizing the images to [-1, 1]'''\n image = tf.cast(image, tf.float32)\n image = (image / 127.5) - 1\n return image\n\n\ndef de_normalize(image):\n '''De normalize the image to be in range (0,1)'''\n \n return (image * 0.5) + 0.5\n ","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:41.302318Z","iopub.execute_input":"2022-08-08T08:05:41.302630Z","iopub.status.idle":"2022-08-08T08:05:41.311388Z","shell.execute_reply.started":"2022-08-08T08:05:41.302601Z","shell.execute_reply":"2022-08-08T08:05:41.310498Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def image_augmentations(image):\n '''perform spatial augmentations (rotation and flips) on input image\n \n from : https://www.kaggle.com/code/dimitreoliveira/improving-cyclegan-monet-paintings'''\n \n \n # --------------------rotations----------\n #rotation probabliity\n p_rotate = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n \n if p_rotate > .8:\n image = tf.image.rot90(image, k=3) # rotate 270º\n elif p_rotate > .6:\n image = tf.image.rot90(image, k=2) # rotate 180º\n elif p_rotate > .4:\n image = tf.image.rot90(image, k=1) # rotate 90º\n \n \n \n # ----------------------Flips---------------------\n p_flip = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n \n if p_flip > 0.7: \n image = tf.image.random_flip_left_right(image)\n elif p_flip < 0.3:\n image = tf.image.random_flip_up_down(image)\n \n return image","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:41.312886Z","iopub.execute_input":"2022-08-08T08:05:41.313658Z","iopub.status.idle":"2022-08-08T08:05:41.517311Z","shell.execute_reply.started":"2022-08-08T08:05:41.313622Z","shell.execute_reply":"2022-08-08T08:05:41.516051Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def random_jitter(image):\n '''resize and randommly crop the input image'''\n \n# # resizing image\n image = tf.image.resize(image, size=(CFG.resize_height, CFG.resize_width),\n method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)\n\n # randomly cropping to 512,512\n image = random_crop(image)\n \n return image\n\n\ndef preprocess_image_train(image):\n image = load_image(image)\n image = random_jitter(image)\n image= image_augmentations(image)\n image = normalize(image)\n return image\n\n\n#same function, withou the augemntation\ndef preprocess_image_eval(image):\n image = load_image(image)\n image = random_jitter(image)\n image = normalize(image)\n return image\n\n","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:41.519115Z","iopub.execute_input":"2022-08-08T08:05:41.519514Z","iopub.status.idle":"2022-08-08T08:05:41.529368Z","shell.execute_reply.started":"2022-08-08T08:05:41.519470Z","shell.execute_reply":"2022-08-08T08:05:41.528370Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\ndef create_img_dataset(directory,\n image_preprocess_fn,\n image_extension = 'jpg', \n repeat=True\n ):\n '''create a tf dataset object from a directory of images'''\n img_list = glob(directory+f'/*{image_extension}')\n \n dataset = tf.data.Dataset.list_files(img_list)\n \n dataset = dataset.map(image_preprocess_fn,\n num_parallel_calls=tf.data.AUTOTUNE)\n \n if repeat :\n dataset = dataset.repeat()\n \n dataset = dataset.shuffle(CFG.BUFFER_SIZE) \n dataset = dataset.batch(CFG.BATCH_SIZE)\n return dataset\n\n\nDay_Dataset = create_img_dataset(directory = day_dir,image_preprocess_fn = preprocess_image_train)\n\n#without augmentation\nDay_eval = create_img_dataset(directory = day_dir,\n image_preprocess_fn = preprocess_image_eval)\n\n\n\nfig,ax = plt.subplots(figsize=(16,8))\n \ninp_img = next(iter(Day_Dataset))\nplt.imshow(de_normalize(inp_img[0]))\nplt.title('Sample Day image')\nplt.axis('off')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:41.530870Z","iopub.execute_input":"2022-08-08T08:05:41.531375Z","iopub.status.idle":"2022-08-08T08:05:52.977840Z","shell.execute_reply.started":"2022-08-08T08:05:41.531340Z","shell.execute_reply":"2022-08-08T08:05:52.977013Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Night_Dataset = create_img_dataset(directory = night_dir,image_preprocess_fn = preprocess_image_train)\nNight_eval = create_img_dataset(directory = night_dir,\n image_preprocess_fn = preprocess_image_eval)\nfig,ax = plt.subplots(figsize=(16,8))\n\n\ninp_img = next(iter(Night_Dataset))\nplt.imshow(de_normalize(inp_img[0]))\nplt.title('Sample Night image')\nplt.axis('off')\n\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:52.978989Z","iopub.execute_input":"2022-08-08T08:05:52.980227Z","iopub.status.idle":"2022-08-08T08:05:59.094835Z","shell.execute_reply.started":"2022-08-08T08:05:52.980186Z","shell.execute_reply":"2022-08-08T08:05:59.093993Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train data set \n\nTrain_Dataset = tf.data.Dataset.zip((Day_Dataset,Night_Dataset))","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:05:59.096031Z","iopub.execute_input":"2022-08-08T08:05:59.097448Z","iopub.status.idle":"2022-08-08T08:05:59.104496Z","shell.execute_reply.started":"2022-08-08T08:05:59.097395Z","shell.execute_reply":"2022-08-08T08:05:59.103429Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Building Model

\n","metadata":{}},{"cell_type":"markdown","source":"**Upsampling and downsampling blocks**","metadata":{}},{"cell_type":"code","source":"\n#conv weights initilaizer\nconv_initializer = tf.random_normal_initializer(mean=0.0,\n stddev=0.02)\n\n#init for intance normalization\ngamma_initializer = tf.keras.initializers.RandomNormal(mean=0.0, \n stddev=0.02)\n \n \n \n \ndef downsample(input_layer,\n filters,\n name,\n size=3, \n strides=2, \n activation=tf.keras.layers.ReLU(), \n ):\n \n '''perform a downsampling by applying a convolution,followed by instance norm and activation'''\n conv = tf.keras.layers.Conv2D(filters, \n size, \n strides=strides, \n padding='same', \n use_bias=False, \n kernel_initializer=conv_initializer, \n name=f'encoder_{name}')(input_layer)\n\n \n conv = tfa.layers.InstanceNormalization(axis=-1,gamma_initializer=gamma_initializer)(conv)\n \n conv = activation(conv)\n\n return conv\n\n\ndef upsample(input_layer,\n filters,\n name,\n size=3,\n strides=2,\n activation='relu'):\n \n res = tf.keras.layers.Conv2DTranspose(filters, size, \n strides=strides, \n padding='same', \n use_bias=False, \n kernel_initializer=conv_initializer, \n name=f'decoder_{name}')(input_layer)\n\n res = tfa.layers.InstanceNormalization(gamma_initializer=gamma_initializer)(res)\n\n res = tf.keras.layers.Activation(activation)(res)\n \n return res","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:05:59.109920Z","iopub.execute_input":"2022-08-08T08:05:59.110605Z","iopub.status.idle":"2022-08-08T08:05:59.136988Z","shell.execute_reply.started":"2022-08-08T08:05:59.110567Z","shell.execute_reply":"2022-08-08T08:05:59.136069Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Residual block**","metadata":{}},{"cell_type":"code","source":"\ndef residual_block(input_layer, \n size=3, \n strides=1, \n name='block_x'):\n '''performs 2 convolutions followed by an added skip connection with the input'''\n \n \n filters = input_layer.shape[-1]\n block = tf.keras.layers.Conv2D(filters, \n size,\n strides=strides,\n padding='same',\n use_bias=False, \n kernel_initializer=conv_initializer,\n name=f'residual_{name}')(input_layer)\n \n block = tf.keras.layers.Activation('relu')(block)\n block = tf.keras.layers.Conv2D(filters, size, strides=strides, padding='same', use_bias=False, \n kernel_initializer=conv_initializer, name=f'transformer_{name}_2')(block) \n \n #skip connection\n res = tf.keras.layers.Add()([block, input_layer])\n\n return res\n\n\ndef concat_layer(layer_1,layer_2,name):\n '''concatenation of layers for skip connections'''\n return tf.keras.layers.Concatenate(name=name)([layer_1,layer_2])\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:05:59.139135Z","iopub.execute_input":"2022-08-08T08:05:59.139827Z","iopub.status.idle":"2022-08-08T08:05:59.147982Z","shell.execute_reply.started":"2022-08-08T08:05:59.139790Z","shell.execute_reply":"2022-08-08T08:05:59.146946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Building Generator

\n","metadata":{}},{"cell_type":"code","source":"def get_generator(num_residual_connections=6):\n \n input_ = tf.keras.layers.Input(shape=(CFG.IMG_WIDTH,CFG.IMG_HEIGHT,3), \n name='input_layer')\n \n #-----------------------ENCODER-------------------------------\n #downsample images \n enc1 = downsample(input_layer = input_, filters=64, strides = 1, size=7, name='dwn_1') # (bs, 512,512, 64)\n enc2 = downsample(input_layer=enc1,filters= 128,size = 3, strides = 2, name='dwn_2') # (bs, 256, 256, 128)\n enc3 = downsample(input_layer=enc2, filters=256,size = 3, strides =2, name='dwn_3') # (bs, 128,128,256)\n enc4 = downsample(input_layer=enc3, filters=256,size = 3, strides =2, name='dwn_4') # (bs, 64,64,256)\n \n \n #-----------------------Residual connections-------------------------------\n x = enc4\n for n in range(num_residual_connections):\n x = residual_block(input_layer=x, name=f'res_block_{n+1}') # (bs, 64, 64, 256)\n\n #-----------------------DECODER-------------------------------\n #UNET like skip connection \n #upsample 1\n x_skip = concat_layer(layer_1=x,layer_2=enc4,name='skip_1') \n dec1 = upsample(x_skip,filters=256 ,name='upsam_1') # (bs, 128, 128, 256)\n \n #upsample 2\n x_skip = concat_layer(layer_1=dec1,layer_2=enc3,name='skip_2') \n dec_2 = upsample(x_skip, filters=128,name='upsam_2') # (bs, 256, 256, 128)\n \n #upsample 3\n x_skip = concat_layer(layer_1=dec_2,layer_2=enc2,name='skip_3') \n dec_3 = upsample(x_skip, filters= 64,name='upsam_3') # (bs, 512, 512, 64)\n \n #penultimate\n x_skip = concat_layer(layer_1=dec_3,\n layer_2=enc1,\n name='skip_final')\n\n output = tf.keras.layers.Conv2D(filters = 3,kernel_size = 7, strides=1, padding='same', \n kernel_initializer=conv_initializer, use_bias=False, activation='tanh', \n name='output_layer')(x_skip) \n\n \n return tf.keras.models.Model(inputs=input_,outputs=output)\n\n\n\n\n# day images -> night images \nday2night_gen = get_generator()\n\n# night images -> day images \nnight2day_gen = get_generator()\n\n\n#plot model\n# tf.keras.utils.plot_model(day2night_gen)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:05:59.149368Z","iopub.execute_input":"2022-08-08T08:05:59.150020Z","iopub.status.idle":"2022-08-08T08:06:00.139642Z","shell.execute_reply.started":"2022-08-08T08:05:59.149904Z","shell.execute_reply":"2022-08-08T08:06:00.138665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#passing in a input to generator for check \n#plot a sample output\ngen_output = night2day_gen(inp_img, training=False)[0]\nplt.subplots(1,2,figsize=(16,8))\n\n\nplt.subplot(1,2,1)\nplt.imshow(gen_output.numpy().squeeze())\nplt.title('Untrained Night2Day Generator output')\nplt.axis('off')\n\n\nplt.subplot(1,2,2)\nplt.imshow(de_normalize(inp_img[0]))\nplt.title('Original Night image')\nplt.axis('off')\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:06:00.141110Z","iopub.execute_input":"2022-08-08T08:06:00.141920Z","iopub.status.idle":"2022-08-08T08:06:08.098015Z","shell.execute_reply.started":"2022-08-08T08:06:00.141880Z","shell.execute_reply":"2022-08-08T08:06:08.097071Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Building Discriminator

\n","metadata":{}},{"cell_type":"code","source":"\n\ndef PATCH_discriminator(leak_rate = 0.2):\n '''PATCH discriminator network'''\n leaky_relu = tf.keras.layers.LeakyReLU(leak_rate)\n\n \n input_ = tf.keras.layers.Input(shape=(CFG.IMG_WIDTH,CFG.IMG_HEIGHT,3), \n name='input_layer')\n # Encoder \n # Input image 512,512\n x = downsample(input_layer = input_, filters=64, strides = 2, size=4, name='dwn_1',activation = leaky_relu) #h,w =256\n x = downsample(input_layer = x, filters=128, strides = 2, size=4, name='dwn_2',activation = leaky_relu) #h,w =128\n x = downsample(input_layer = x, filters=256, strides = 2, size=4, name='dwn_3',activation = leaky_relu) #h,w = 64\n x = downsample(input_layer = x, filters=512, strides = 2, size=4, name='dwn_4',activation = leaky_relu) #h,w = 32\n x = downsample(input_layer = x, filters=512, strides = 1, size=4, name='dwn_5',activation = leaky_relu) #h,w = 32\n \n \n output = tf.keras.layers.Conv2D(1, 4, strides=1, padding='valid', kernel_initializer=conv_initializer)(x) #(29, 29, 1)\n \n return tf.keras.models.Model(inputs=input_,outputs=output)\n\n\n\n#create instance of discriminators \nday2night_disc = PATCH_discriminator() # identify night images\nnight2day_disc = PATCH_discriminator() # identify day images\n","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:06:08.099489Z","iopub.execute_input":"2022-08-08T08:06:08.100217Z","iopub.status.idle":"2022-08-08T08:06:08.394468Z","shell.execute_reply.started":"2022-08-08T08:06:08.100163Z","shell.execute_reply":"2022-08-08T08:06:08.393500Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#check on dicriminator \n\ndisc_output = day2night_disc(inp_img, training=False)\nplt.subplots(1,1,figsize=(8,8))\n\nplt.imshow(disc_output.numpy().mean(axis=0),cmap='gray')\nplt.title('Untrained Night2Day disc output')\nplt.axis('off')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:06:08.396013Z","iopub.execute_input":"2022-08-08T08:06:08.396398Z","iopub.status.idle":"2022-08-08T08:06:08.660123Z","shell.execute_reply.started":"2022-08-08T08:06:08.396360Z","shell.execute_reply":"2022-08-08T08:06:08.658865Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def generate_cycle(gen_1,gen_2,input_image):\n '''generate a full cycle of images using given generators'''\n gen_img_1 = gen_1(input_image,training=True)\n gen_img_2 = gen_2(gen_img_1,training=True)\n \n return gen_img_1,gen_img_2\n\n\ndef calc_and_apply_gradients(tape,\n model,\n loss,\n optimizer):\n '''Apply gradients for a given model using given optimizer'''\n \n #calculate gradients of loss function\n gradients = tape.gradient(loss,model.trainable_variables)\n \n #apply gradients \n optimizer.apply_gradients(zip(gradients,model.trainable_variables))\n return ","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:06:08.661315Z","iopub.execute_input":"2022-08-08T08:06:08.661781Z","iopub.status.idle":"2022-08-08T08:06:08.678469Z","shell.execute_reply.started":"2022-08-08T08:06:08.661728Z","shell.execute_reply":"2022-08-08T08:06:08.677246Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

losses

","metadata":{}},{"cell_type":"code","source":"def discriminator_loss(real, generated):\n '''discriminator Binary CrossEntropy loss'''\n real_loss = tf.keras.losses.BinaryCrossentropy(from_logits=True)(tf.ones_like(real), real)\n\n generated_loss = tf.keras.losses.BinaryCrossentropy(from_logits=True)(tf.zeros_like(generated), generated)\n\n total_disc_loss = real_loss + generated_loss\n\n return total_disc_loss * 0.5\n\n# Generator Adverserial loss\ndef generator_loss(generated):\n '''adverserial generator loss (BCE)'''\n return tf.keras.losses.BinaryCrossentropy(from_logits=True)(tf.ones_like(generated), generated)\n\n \n# Cycle consistency loss \n \ndef calc_cycle_loss(real_image, cycled_image, LAMBDA):\n '''pixel wise cycle loss between original image and cycled image'''\n mae_loss = tf.reduce_mean(tf.abs(real_image - cycled_image))\n\n return LAMBDA * mae_loss\n\n\n# identity loss\ndef identity_loss(real_image, same_image, LAMBDA): \n mae_loss = tf.reduce_mean(tf.abs(real_image - same_image))\n return LAMBDA * 0.5 * mae_loss","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:06:08.680588Z","iopub.execute_input":"2022-08-08T08:06:08.681360Z","iopub.status.idle":"2022-08-08T08:06:08.693213Z","shell.execute_reply.started":"2022-08-08T08:06:08.681321Z","shell.execute_reply":"2022-08-08T08:06:08.692127Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Cycle GAN

\n","metadata":{}},{"cell_type":"code","source":"class CycleGAN(tf.keras.models.Model):\n def __init__(self,\n lambda_cycle=10):\n super(CycleGAN, self).__init__()\n self.gen_d2n = day2night_gen # Day -> Night \n self.gen_n2d = night2day_gen # Night -> Day\n self.disc_d2n = day2night_disc # Classifies Night Images\n self.disc_n2d = night2day_disc # Classifier Day Images \n self.lambda_cycle = lambda_cycle #lambda in cycle consistancy loss\n \n \n \n def compile(self,\n gen_loss_fn,\n disc_loss_fn,\n cycle_loss_fn,\n identity_loss_fn,\n common_opt = tf.keras.optimizers.Adam(learning_rate = CFG.learning_rate,beta_1 = 0.5)):\n \n super(CycleGAN, self).compile()\n \n # -------optimizers ---------\n self.opt_gen_d2n = common_opt\n self.opt_gen_n2d = common_opt\n self.opt_disc_d2n = common_opt\n self.opt_disc_n2d = common_opt\n \n \n # -------losses ---------\n self.gen_loss_fn = gen_loss_fn\n self.disc_loss_fn = disc_loss_fn\n self.cycle_loss_fn = cycle_loss_fn\n self.identity_loss_fn = identity_loss_fn\n \n \n def train_step(self, batch_data):\n day_image, night_image = batch_data\n \n with tf.GradientTape(persistent=True) as tape:\n \n #-----day->night->day\n fake_night,cycled_day = generate_cycle(self.gen_d2n,\n self.gen_n2d,\n day_image)\n \n\n # --------night -> day- > night\n fake_day,cycled_night = generate_cycle(self.gen_n2d,\n self.gen_d2n,\n night_image)\n \n #---------- generating itself (for identity loss)\n iden_day = self.gen_d2n(night_image, training=True)\n iden_night = self.gen_n2d(day_image, training=True)\n\n # -----------discriminator on real images\n disc_night = self.disc_d2n(night_image, training=True)\n disc_day = self.disc_n2d(day_image, training=True)\n\n # -----------discriminator on fake images-----------------\n disc_fake_night = self.disc_d2n(fake_night, training=True)\n disc_fake_day = self.disc_n2d(fake_day, training=True)\n\n # -------------------------generator loss-------------\n #---1)adverserial loss\n night_gen_loss = self.gen_loss_fn(disc_fake_night) \n day_gen_loss = self.gen_loss_fn(disc_fake_day)\n\n #---2)Cycle loss loss\n total_cycle_loss = self.cycle_loss_fn(night_image, cycled_night, self.lambda_cycle) + self.cycle_loss_fn(day_image, cycled_day, self.lambda_cycle)\n\n # +++++3) Total Gen loss (day gen and night gen)\n total_gen_d2n_loss = night_gen_loss + total_cycle_loss + self.identity_loss_fn(night_image, iden_night,self.lambda_cycle)\n total_gen_n2d_loss = day_gen_loss + total_cycle_loss + self.identity_loss_fn(day_image, iden_day, self.lambda_cycle)\n \n \n # -------------------------Discriminator loss-------------\n night_disc_loss = self.disc_loss_fn(disc_night, disc_fake_night) # check classifying generated and real night\n day_disc_loss = self.disc_loss_fn(disc_day, disc_fake_day) # check classifying generated and real day\n\n ## ------------------------- Calculating and Updating gradients------------------\n \n # day->night gen gradeints\n _ = calc_and_apply_gradients(tape=tape,\n model= self.gen_d2n,\n loss = total_gen_d2n_loss,\n optimizer = self.opt_gen_d2n)\n \n # night - >day gen gradeints\n _ = calc_and_apply_gradients(tape=tape,\n model= self.gen_n2d,\n loss = total_gen_n2d_loss,\n optimizer = self.opt_gen_n2d)\n \n # discrim gradients (classifies night images)\n _ = calc_and_apply_gradients(tape=tape,\n model= self.disc_d2n,\n loss = night_disc_loss,\n optimizer = self.opt_disc_d2n)\n \n # Day discrim gradients (classifies day images)\n _ = calc_and_apply_gradients(tape=tape,\n model= self.disc_n2d,\n loss = day_disc_loss,\n optimizer = self.opt_disc_n2d)\n \n \n return {'gen_D2N_loss': total_gen_d2n_loss,\n 'gen_N2D_loss': total_gen_n2d_loss,\n 'disc_day_loss': day_disc_loss,\n 'disc_night_loss': night_disc_loss\n }\n \n \n ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:06:08.694647Z","iopub.execute_input":"2022-08-08T08:06:08.695328Z","iopub.status.idle":"2022-08-08T08:06:08.714153Z","shell.execute_reply.started":"2022-08-08T08:06:08.695293Z","shell.execute_reply":"2022-08-08T08:06:08.713236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#creat a instance of Cycle gan \ngan = CycleGAN()\n\n\n#complie with the losses \ngan.compile(gen_loss_fn=generator_loss,\n disc_loss_fn=discriminator_loss,\n cycle_loss_fn=calc_cycle_loss,\n identity_loss_fn=identity_loss)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:06:08.715340Z","iopub.execute_input":"2022-08-08T08:06:08.715635Z","iopub.status.idle":"2022-08-08T08:06:08.736994Z","shell.execute_reply.started":"2022-08-08T08:06:08.715608Z","shell.execute_reply":"2022-08-08T08:06:08.736056Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Callbacks

\n\n For reducing Learning Rate,stopping training and Visulization","metadata":{}},{"cell_type":"code","source":"#learning rate schedule \n\ndef scheduler(epoch, \n lr,\n decay_rate = 0.05,\n warm_up_period = 10):\n \n if epoch < warm_up_period:\n return lr\n elif (epoch > warm_up_period and epoch<40):\n return lr * tf.math.exp(decay_rate)\n else:\n return lr * tf.math.exp(decay_rate*2)\n \n \n \n \nlr_scheduler = tf.keras.callbacks.LearningRateScheduler(scheduler,\n verbose = 0)\n\n\n#early stopping \n\n# early_stop = tf.keras.callbacks.EarlyStopping(monitor = 'gen_N2D_loss',\n# mode = 'min',\n# patience = 10,\n# restore_best_weights = True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:06:08.738366Z","iopub.execute_input":"2022-08-08T08:06:08.738920Z","iopub.status.idle":"2022-08-08T08:06:08.745675Z","shell.execute_reply.started":"2022-08-08T08:06:08.738886Z","shell.execute_reply":"2022-08-08T08:06:08.744777Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# early stopping : from https://stackoverflow.com/questions/64556120/early-stopping-with-multiple-conditions\n\nclass CustomEarlyStopping(tf.keras.callbacks.Callback):\n def __init__(self, patience=0):\n super(CustomEarlyStopping, self).__init__()\n self.patience = patience\n self.best_weights = None\n \n def on_train_begin(self, logs=None):\n \n # The number of epoch it has waited when loss is no longer minimum.\n self.wait = 0\n # The epoch the training stops at.\n self.stopped_epoch = 0\n # Initialize the best as infinity.\n self.n2d_loss = np.Inf\n self.d2n_loss = np.Inf\n\n def on_epoch_end(self, epoch, logs=None): \n n2d_loss=logs.get('gen_N2D_loss')\n d2n_loss=logs.get('gen_D2N_loss')\n\n # If both the conditions are met, continue training\n if (np.less(n2d_loss, self.n2d_loss) and np.less(d2n_loss, self.d2n_loss)):\n self.d2n_loss = d2n_loss\n self.n2d_loss = n2d_loss\n self.wait = 0\n # Record the best weights if current results is better (less).\n self.best_weights = self.model.get_weights()\n \n # if above xondition not met, continue training till patiance epochs\n else:\n self.wait += 1\n if self.wait >= self.patience:\n self.stopped_epoch = epoch\n self.model.stop_training = True\n print(\"Restoring model weights from the end of the best epoch.\")\n self.model.set_weights(self.best_weights)\n \n def on_train_end(self, logs=None):\n if self.stopped_epoch > 0:\n print(\"Epoch %05d: early stopping\" % (self.stopped_epoch + 1))\n","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:06:08.748787Z","iopub.execute_input":"2022-08-08T08:06:08.749757Z","iopub.status.idle":"2022-08-08T08:06:08.759902Z","shell.execute_reply.started":"2022-08-08T08:06:08.749724Z","shell.execute_reply":"2022-08-08T08:06:08.758950Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#--------------------------------------Viz Callbacks : from https://www.kaggle.com/code/dimitreoliveira/improving-cyclegan-monet-paintings -----------------------------------------\ndef display_samples(ds, n_samples):\n ds_iter = iter(ds)\n for n_sample in range(n_samples):\n example_sample = next(ds_iter)\n plt.subplot(121)\n plt.imshow(example_sample[0] * 0.5 + 0.5)\n plt.axis('off')\n plt.show()\n \ndef display_generated_samples(ds, model, n_samples):\n ds_iter = iter(ds)\n for n_sample in range(n_samples):\n example_sample = next(ds_iter)\n generated_sample = model.predict(example_sample)\n \n f = plt.figure(figsize=(16,8))\n \n plt.subplot(121)\n plt.title('Input image')\n plt.imshow(example_sample[0] * 0.5 + 0.5)\n plt.axis('off')\n \n plt.subplot(122)\n plt.title('Generated image')\n plt.imshow(generated_sample[0] * 0.5 + 0.5)\n plt.axis('off')\n plt.show()\n \ndef evaluate_cycle(ds, generator_a, generator_b, n_samples=1):\n fig, axes = plt.subplots(n_samples, 3, figsize=(22, (n_samples*6)))\n axes = axes.flatten()\n \n ds_iter = iter(ds)\n for n_sample in range(n_samples):\n idx = n_sample*3\n example_sample = next(ds_iter)\n generated_a_sample = generator_a.predict(example_sample)\n generated_b_sample = generator_b.predict(generated_a_sample)\n \n axes[idx].set_title('Input image', fontsize=18)\n axes[idx].imshow(example_sample[0] * 0.5 + 0.5)\n axes[idx].axis('off')\n \n axes[idx+1].set_title('Generated image', fontsize=18)\n axes[idx+1].imshow(generated_a_sample[0] * 0.5 + 0.5)\n axes[idx+1].axis('off')\n \n axes[idx+2].set_title('Cycled image', fontsize=18)\n axes[idx+2].imshow(generated_b_sample[0] * 0.5 + 0.5)\n axes[idx+2].axis('off')\n \n plt.show()\n \ndef predict_and_save(input_ds, generator_model, output_path):\n i = 1\n for img in input_ds:\n prediction = generator_model(img, training=False)[0].numpy() # make predition\n prediction = (prediction * 127.5 + 127.5).astype(np.uint8) # re-scale\n im = PIL.Image.fromarray(prediction)\n im.save(f'{output_path}{str(i)}.jpg')\n i += 1\n\n# Callback\nclass GANMonitor(tf.keras.callbacks.Callback):\n \"\"\"A callback to generate and save images after each epoch\"\"\"\n\n def __init__(self, \n num_img=1, \n day_paths='generated_day', \n night_paths='generated_night'):\n self.num_img = num_img\n self.day_paths = day_paths\n self.night_paths = night_paths\n \n # dir to save genereated day images\n if not os.path.exists(self.day_paths):\n os.makedirs(self.day_paths)\n \n \n # dir to save genereated night images\n if not os.path.exists(self.night_paths):\n os.makedirs(self.night_paths)\n\n \n \n def on_epoch_end(self, epoch, logs=None):\n #generated night \n for i, img in enumerate(Day_eval.take(self.num_img)):\n \n \n prediction = day2night_gen(img, training=False)[0].numpy()\n prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n prediction = PIL.Image.fromarray(prediction)\n prediction.save(f'{self.night_paths}/generated_{i}_{epoch+1}.png')\n \n # generated day images \n for i, img in enumerate(Night_eval.take(self.num_img)):\n prediction = night2day_gen(img, training=False)[0].numpy()\n prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n prediction = PIL.Image.fromarray(prediction)\n prediction.save(f'{self.day_paths}/generated_{i}_{epoch+1}.png')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:06:08.762488Z","iopub.execute_input":"2022-08-08T08:06:08.762940Z","iopub.status.idle":"2022-08-08T08:06:08.783443Z","shell.execute_reply.started":"2022-08-08T08:06:08.762901Z","shell.execute_reply":"2022-08-08T08:06:08.782508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Fitting the model

\n","metadata":{}},{"cell_type":"code","source":"EPOCHS = 75\ncallbacks = [lr_scheduler,GANMonitor(),CustomEarlyStopping(patience = 10)]\nsteps_per_epoch = 200\n\n\nhistory = gan.fit(Train_Dataset,\n epochs = EPOCHS,\n steps_per_epoch=steps_per_epoch,\n callbacks = callbacks)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:06:08.784904Z","iopub.execute_input":"2022-08-08T08:06:08.785519Z","iopub.status.idle":"2022-08-08T08:41:38.167686Z","shell.execute_reply.started":"2022-08-08T08:06:08.785464Z","shell.execute_reply":"2022-08-08T08:41:38.165229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Visualizing Training Progress

","metadata":{}},{"cell_type":"markdown","source":"

Visualizing Training Progress for Night -> Day generator

\n\n","metadata":{}},{"cell_type":"code","source":"\n\n# from https://www.kaggle.com/code/dimitreoliveira/improving-cyclegan-monet-paintings\ndef create_gif(images_path, gif_path):\n images = []\n filenames = glob(images_path)\n filenames.sort(key=lambda x: int(''.join(filter(str.isdigit, x))))\n for epoch, filename in enumerate(filenames):\n img = PIL.ImageDraw.Image.open(filename)\n ImageDraw.Draw(img).text((0, 0), # Coordinates\n f'Epoch {epoch+1}')\n images.append(img)\n imageio.mimsave(gif_path, images, fps=2) # Save gif\n \n \n \ncreate_gif('./generated_day/*.png', 'day.gif')\n\nprint('Training progress for Night -> Day Generator')\ndisplay.Image('./day.gif')","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:41:55.324778Z","iopub.execute_input":"2022-08-08T08:41:55.325203Z","iopub.status.idle":"2022-08-08T08:41:56.396977Z","shell.execute_reply.started":"2022-08-08T08:41:55.325138Z","shell.execute_reply":"2022-08-08T08:41:56.395941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"

Visualizing Training Progress for Day -> Night generator

\n","metadata":{}},{"cell_type":"code","source":"create_gif('./generated_night/*.png', 'night.gif')\n\n\nprint('Training progress for Day-> Generator')\ndisplay.Image('./night.gif')","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:00.995809Z","iopub.execute_input":"2022-08-08T08:44:00.996214Z","iopub.status.idle":"2022-08-08T08:44:02.033845Z","shell.execute_reply.started":"2022-08-08T08:44:00.996159Z","shell.execute_reply":"2022-08-08T08:44:02.032660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Results

\n","metadata":{}},{"cell_type":"markdown","source":"

Evaluating Day -> Night Generator

\n","metadata":{}},{"cell_type":"code","source":"display_generated_samples(Day_eval.take(2), day2night_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:10.140685Z","iopub.execute_input":"2022-08-08T08:44:10.141061Z","iopub.status.idle":"2022-08-08T08:44:16.371090Z","shell.execute_reply.started":"2022-08-08T08:44:10.141019Z","shell.execute_reply":"2022-08-08T08:44:16.370018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Day_eval.take(2), day2night_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:16.374076Z","iopub.execute_input":"2022-08-08T08:44:16.374502Z","iopub.status.idle":"2022-08-08T08:44:26.622378Z","shell.execute_reply.started":"2022-08-08T08:44:16.374456Z","shell.execute_reply":"2022-08-08T08:44:26.621360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Day_eval.take(2), day2night_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:26.624615Z","iopub.execute_input":"2022-08-08T08:44:26.625095Z","iopub.status.idle":"2022-08-08T08:44:36.871876Z","shell.execute_reply.started":"2022-08-08T08:44:26.625056Z","shell.execute_reply":"2022-08-08T08:44:36.870856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Evaluating Night -> Day Generator

\n","metadata":{}},{"cell_type":"code","source":"display_generated_samples(Night_eval.take(2), night2day_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:36.874853Z","iopub.execute_input":"2022-08-08T08:44:36.875425Z","iopub.status.idle":"2022-08-08T08:44:47.121123Z","shell.execute_reply.started":"2022-08-08T08:44:36.875386Z","shell.execute_reply":"2022-08-08T08:44:47.120112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Night_eval.take(2), night2day_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:47.123190Z","iopub.execute_input":"2022-08-08T08:44:47.123650Z","iopub.status.idle":"2022-08-08T08:44:53.313876Z","shell.execute_reply.started":"2022-08-08T08:44:47.123612Z","shell.execute_reply":"2022-08-08T08:44:53.312668Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Night_eval.take(2), night2day_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:53.315897Z","iopub.execute_input":"2022-08-08T08:44:53.316258Z","iopub.status.idle":"2022-08-08T08:44:59.780939Z","shell.execute_reply.started":"2022-08-08T08:44:53.316226Z","shell.execute_reply":"2022-08-08T08:44:59.779842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Evaluating Identity for Day to Night Generator so that Night -> Night

\n","metadata":{}},{"cell_type":"code","source":"display_generated_samples(Night_eval.take(2), day2night_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:44:59.783979Z","iopub.execute_input":"2022-08-08T08:44:59.784862Z","iopub.status.idle":"2022-08-08T08:45:06.361696Z","shell.execute_reply.started":"2022-08-08T08:44:59.784826Z","shell.execute_reply":"2022-08-08T08:45:06.360641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Night_eval.take(2), day2night_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:45:06.363713Z","iopub.execute_input":"2022-08-08T08:45:06.364055Z","iopub.status.idle":"2022-08-08T08:45:16.609880Z","shell.execute_reply.started":"2022-08-08T08:45:06.364023Z","shell.execute_reply":"2022-08-08T08:45:16.608827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Night_eval.take(2), day2night_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:45:16.617026Z","iopub.execute_input":"2022-08-08T08:45:16.617336Z","iopub.status.idle":"2022-08-08T08:45:26.860838Z","shell.execute_reply.started":"2022-08-08T08:45:16.617308Z","shell.execute_reply":"2022-08-08T08:45:26.859825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Evaluating Identity for Night to Day Generator so that Day -> Day

\n","metadata":{}},{"cell_type":"code","source":"display_generated_samples(Day_eval.take(2), night2day_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:45:50.670841Z","iopub.execute_input":"2022-08-08T08:45:50.671986Z","iopub.status.idle":"2022-08-08T08:46:00.916231Z","shell.execute_reply.started":"2022-08-08T08:45:50.671939Z","shell.execute_reply":"2022-08-08T08:46:00.915124Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Day_eval.take(2), night2day_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:46:00.923121Z","iopub.execute_input":"2022-08-08T08:46:00.923508Z","iopub.status.idle":"2022-08-08T08:46:06.708804Z","shell.execute_reply.started":"2022-08-08T08:46:00.923470Z","shell.execute_reply":"2022-08-08T08:46:06.707939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_generated_samples(Day_eval.take(2), night2day_gen, 2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:46:06.710978Z","iopub.execute_input":"2022-08-08T08:46:06.711938Z","iopub.status.idle":"2022-08-08T08:46:12.018091Z","shell.execute_reply.started":"2022-08-08T08:46:06.711887Z","shell.execute_reply":"2022-08-08T08:46:12.016963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Cycle : Day(real image) -> Night -> Day

\n","metadata":{}},{"cell_type":"code","source":"evaluate_cycle(Day_eval.take(2),\n day2night_gen, \n night2day_gen, \n n_samples=2)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-08T08:46:25.310537Z","iopub.execute_input":"2022-08-08T08:46:25.310901Z","iopub.status.idle":"2022-08-08T08:46:35.629665Z","shell.execute_reply.started":"2022-08-08T08:46:25.310872Z","shell.execute_reply":"2022-08-08T08:46:35.628609Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"evaluate_cycle(Day_eval.take(2),\n day2night_gen, \n night2day_gen, \n n_samples=2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:46:35.632105Z","iopub.execute_input":"2022-08-08T08:46:35.632517Z","iopub.status.idle":"2022-08-08T08:46:45.979577Z","shell.execute_reply.started":"2022-08-08T08:46:35.632478Z","shell.execute_reply":"2022-08-08T08:46:45.978568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Cycle : Day(real image) -> Night -> Day

\n","metadata":{}},{"cell_type":"code","source":"evaluate_cycle(Night_eval.take(2),\n night2day_gen, \n day2night_gen, \n n_samples=2)\n","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:46:45.981235Z","iopub.execute_input":"2022-08-08T08:46:45.982157Z","iopub.status.idle":"2022-08-08T08:46:56.298274Z","shell.execute_reply.started":"2022-08-08T08:46:45.982118Z","shell.execute_reply":"2022-08-08T08:46:56.297136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"evaluate_cycle(Night_eval.take(2),\n night2day_gen, \n day2night_gen, \n n_samples=2)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T08:46:56.301072Z","iopub.execute_input":"2022-08-08T08:46:56.301472Z","iopub.status.idle":"2022-08-08T08:47:03.046101Z","shell.execute_reply.started":"2022-08-08T08:46:56.301435Z","shell.execute_reply":"2022-08-08T08:47:03.045041Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# References and Resources \n\n* https://www.cs.cmu.edu/~junyanz/projects/CycleGAN/CycleGAN.pdf\n* https://www.tensorflow.org/tutorials/generative/cyclegan\n* https://www.kaggle.com/code/dimitreoliveira/improving-cyclegan-monet-paintings\n","metadata":{}}]}