{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Color grey-scale images using UNET 🎨\nIn the dataset there are a few images that are in greyscale. It adds complexity to the task so we may want to colorize them to make the task easier.\n\nIn this notebook, I'll try to fit a UNet to colorize these images.\n\n**This notebook uses Pytorch Lightning ⚡**<br>","metadata":{"_kg_hide-input":false}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport cv2\nimport albumentations\nimport warnings\nimport matplotlib.pyplot as plt\nimport wandb\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport pytorch_lightning as pl\nfrom tqdm import tqdm\nfrom random import random\nfrom albumentations.pytorch.transforms import ToTensorV2\nfrom torch.utils.data import Dataset\nfrom torchvision import transforms\nfrom pytorch_lightning import Callback, LightningModule, Trainer\nfrom pytorch_lightning.loggers import WandbLogger\nwarnings.filterwarnings(\"ignore\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-01T19:06:14.988319Z","iopub.execute_input":"2022-03-01T19:06:14.989409Z","iopub.status.idle":"2022-03-01T19:06:20.435292Z","shell.execute_reply.started":"2022-03-01T19:06:14.989295Z","shell.execute_reply":"2022-03-01T19:06:20.434478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Configuration ⚙️\n","metadata":{}},{"cell_type":"code","source":"class CFG:\n    SEED = 69\n    ### Dataset\n    BATCH_SIZE = 32\n    IMAGE_SIZE = 224#380\n    NUM_WORKERS = 2\n    NOISE_STD = 0.15\n    BLUR_SIGMA = (0.1, 2.0)\n    BLUR_KERNEL_SIZE = 5\n    ## Training\n    EPOCHS = 4\n    LR = 0.001\n    SCHEDULER_PATIENCE = 600\n    SCHEDULER_FACTOR = 0.1\n    MODEL_PATH=\"model.ckpt\"","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:06:20.437828Z","iopub.execute_input":"2022-03-01T19:06:20.438329Z","iopub.status.idle":"2022-03-01T19:06:20.444837Z","shell.execute_reply.started":"2022-03-01T19:06:20.438292Z","shell.execute_reply":"2022-03-01T19:06:20.443951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Logging 📄\nLogging using Weights and Biases 🪄🐝","metadata":{}},{"cell_type":"code","source":"from kaggle_secrets import UserSecretsClient\n\nuser_secrets = UserSecretsClient()\n\n# I have saved my API token with \"wandb_api\" as Label. \n# If you use some other Label make sure to change the same below. \nwandb_api = user_secrets.get_secret(\"wandb_api\") \n\nwandb.login(key=wandb_api)","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:06:20.445968Z","iopub.execute_input":"2022-03-01T19:06:20.446293Z","iopub.status.idle":"2022-03-01T19:06:21.822177Z","shell.execute_reply.started":"2022-03-01T19:06:20.446256Z","shell.execute_reply":"2022-03-01T19:06:21.821392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dataset 🖼️\nImplementing the dataset as a Pytorch Dataset as required by Pytorch Lightning. It applies some augmentations:\n* Noise: We add some gaussian noise to denoise old images\n* Gaussian blur: Can help when the quality was degraded with time","metadata":{}},{"cell_type":"code","source":"BASE_PATH = \"../input/happywhale-enhanced-dataset-light\"\n\nclass WandDID(Dataset):\n    def __init__(self, data, return_colored=True, folder=\"train_images\"):\n        self.return_colored = return_colored\n        self.base_path = os.path.join(BASE_PATH, folder)\n        if \"inference_image\" not in data.columns:\n            data[\"inference_image\"] = data[\"image\"]\n        self.data = data\n        # Augmentations\n        transformations = albumentations.Compose([\n            albumentations.Normalize(),\n            ToTensorV2(p=1.0)\n        ])\n\n        def make_transform(transform=False):\n            def f(image):\n                if transform:\n                    image_np = np.array(image)\n                    augmented = transform(image=image_np)\n                return augmented\n            return f\n\n        self.transforms = transforms.Compose([\n            transforms.Lambda(make_transform(transformations)),\n        ])\n        \n    def __getitem__(self, idx):\n        colored_image = self.preprocess(self.data[\"inference_image\"].iloc[idx])\n        #label = self.data[\"individual_id_integer\"].iloc[idx]\n        greyscale_image = colored_image.mean(0).unsqueeze(0)\n        \n        if not self.return_colored:\n            return greyscale_image\n        \n        if random() < 0.5:\n            sigma = torch.empty(1).uniform_(CFG.BLUR_SIGMA[0], CFG.BLUR_SIGMA[1]).item()\n            greyscale_image = transforms.functional.gaussian_blur(greyscale_image, CFG.BLUR_KERNEL_SIZE, [sigma, sigma])\n            \n        if random() < 0.5:\n            greyscale_image += torch.randn_like(greyscale_image) * CFG.NOISE_STD\n        \n        return colored_image, greyscale_image#, torch.tensor(label, dtype=torch.long)\n    \n    def preprocess(self, image):\n        image = os.path.join(self.base_path, image)\n        image = cv2.imread(image)[:, :, ::-1]\n        if self.transforms is not None:\n            image = self.transforms(image)[\"image\"]\n        return image\n    \n    def plot_sample(self, idx):\n        image = self.data[\"image\"].iloc[idx]\n        image = os.path.join(self.base_path, image)\n        image = cv2.imread(image)[:, :, ::-1]\n        plt.title(\"{} ({})\".format(\n            self.data[\"individual_id\"].iloc[idx],\n            self.data[\"species\"].iloc[idx]\n        ))\n        plt.imshow(image)\n        plt.show()\n    \n    def __len__(self):\n        return len(self.data)","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:06:21.823731Z","iopub.execute_input":"2022-03-01T19:06:21.823996Z","iopub.status.idle":"2022-03-01T19:06:21.841663Z","shell.execute_reply.started":"2022-03-01T19:06:21.82396Z","shell.execute_reply":"2022-03-01T19:06:21.840711Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data = pd.read_csv(os.path.join(BASE_PATH, \"train.csv\"))\ntrain_greyscale_map = np.load(\"../input/greyscale-images/train_grey_scale_mask.npy\")\ntrain_grey_data = train_data.loc[train_greyscale_map]\ntrain_data = train_data.loc[~train_greyscale_map]\n\ntest_data = pd.read_csv(os.path.join(BASE_PATH, \"sample_submission.csv\"))\ntest_greyscale_map = np.load(\"../input/greyscale-images/test_grey_scale_mask.npy\")\ntest_grey_data = test_data.loc[test_greyscale_map]\ntest_data = test_data.loc[~test_greyscale_map]\n\ntrain_dataset = WandDID(train_data)\n# Dataloader\ntrain_loader = torch.utils.data.DataLoader(\n    train_dataset,\n    batch_size=CFG.BATCH_SIZE,\n    num_workers=CFG.NUM_WORKERS,\n    pin_memory=True,\n    shuffle=True\n)","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:06:21.844577Z","iopub.execute_input":"2022-03-01T19:06:21.844867Z","iopub.status.idle":"2022-03-01T19:06:22.031995Z","shell.execute_reply.started":"2022-03-01T19:06:21.844819Z","shell.execute_reply":"2022-03-01T19:06:22.031265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model 🤖\nImplements the model as a Pytorch lightning module. We use a classical U-Net architecture\n![](https://datascientest.com/wp-content/uploads/2021/05/u-net-architecture-1024x682.png)","metadata":{}},{"cell_type":"code","source":"\"\"\"\n    https://github.com/milesial/Pytorch-UNet/tree/master/unet\n\"\"\"\n\nclass DoubleConv(nn.Module):\n    \"\"\"(convolution => [BN] => ReLU) * 2\"\"\"\n\n    def __init__(self, in_channels, out_channels, mid_channels=None):\n        super().__init__()\n        if not mid_channels:\n            mid_channels = out_channels\n        self.double_conv = nn.Sequential(\n            nn.Conv2d(in_channels, mid_channels, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(mid_channels),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(mid_channels, out_channels, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(out_channels),\n            nn.ReLU(inplace=True)\n        )\n\n    def forward(self, x):\n        return self.double_conv(x)\n\n\nclass Down(nn.Module):\n    \"\"\"Downscaling with maxpool then double conv\"\"\"\n\n    def __init__(self, in_channels, out_channels):\n        super().__init__()\n        self.maxpool_conv = nn.Sequential(\n            nn.MaxPool2d(2),\n            DoubleConv(in_channels, out_channels)\n        )\n\n    def forward(self, x):\n        return self.maxpool_conv(x)\n\n\nclass Up(nn.Module):\n    \"\"\"Upscaling then double conv\"\"\"\n\n    def __init__(self, in_channels, out_channels, bilinear=True):\n        super().__init__()\n\n        # if bilinear, use the normal convolutions to reduce the number of channels\n        if bilinear:\n            self.up = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)\n            self.conv = DoubleConv(in_channels, out_channels, in_channels // 2)\n        else:\n            self.up = nn.ConvTranspose2d(in_channels, in_channels // 2, kernel_size=2, stride=2)\n            self.conv = DoubleConv(in_channels, out_channels)\n\n    def forward(self, x1, x2):\n        x1 = self.up(x1)\n        # input is CHW\n        diffY = x2.size()[2] - x1.size()[2]\n        diffX = x2.size()[3] - x1.size()[3]\n\n        x1 = F.pad(x1, [diffX // 2, diffX - diffX // 2,\n                        diffY // 2, diffY - diffY // 2])\n        # if you have padding issues, see\n        # https://github.com/HaiyongJiang/U-Net-Pytorch-Unstructured-Buggy/commit/0e854509c2cea854e247a9c615f175f76fbb2e3a\n        # https://github.com/xiaopeng-liao/Pytorch-UNet/commit/8ebac70e633bac59fc22bb5195e513d5832fb3bd\n        x = torch.cat([x2, x1], dim=1)\n        return self.conv(x)\n\n\nclass OutConv(nn.Module):\n    def __init__(self, in_channels, out_channels):\n        super(OutConv, self).__init__()\n        self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=1)\n\n    def forward(self, x):\n        return self.conv(x)\n\nclass UNet(nn.Module):\n    def __init__(self, n_channels, n_classes, bilinear=False):\n        super(UNet, self).__init__()\n        self.n_channels = n_channels\n        self.n_classes = n_classes\n        self.bilinear = bilinear\n\n        self.inc = DoubleConv(n_channels, 64)\n        self.down1 = Down(64, 128)\n        self.down2 = Down(128, 256)\n        self.down3 = Down(256, 512)\n        factor = 2 if bilinear else 1\n        self.down4 = Down(512, 1024 // factor)\n        self.up1 = Up(1024, 512 // factor, bilinear)\n        self.up2 = Up(512, 256 // factor, bilinear)\n        self.up3 = Up(256, 128 // factor, bilinear)\n        self.up4 = Up(128, 64, bilinear)\n        self.outc = OutConv(64, n_classes)\n\n    def forward(self, x):\n        x1 = self.inc(x)\n        x2 = self.down1(x1)\n        x3 = self.down2(x2)\n        x4 = self.down3(x3)\n        x5 = self.down4(x4)\n        x = self.up1(x5, x4)\n        x = self.up2(x, x3)\n        x = self.up3(x, x2)\n        x = self.up4(x, x1)\n        logits = self.outc(x)\n        return logits","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-01T19:06:22.033345Z","iopub.execute_input":"2022-03-01T19:06:22.033615Z","iopub.status.idle":"2022-03-01T19:06:22.056432Z","shell.execute_reply.started":"2022-03-01T19:06:22.033582Z","shell.execute_reply":"2022-03-01T19:06:22.055589Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class WandDRecolorize(LightningModule):\n    def __init__(\n        self,\n        learning_rate: float = CFG.LR,\n    ) -> None:\n        \"\"\"\n            learning_rate: Learning rate\n        \"\"\"\n        super().__init__()\n        self.save_hyperparameters()\n\n        self.net = UNet(n_channels=1, n_classes=3)\n\n        self.mse = nn.MSELoss()\n        self.huber = nn.HuberLoss()\n        self.criterion = self.huber\n\n    def configure_optimizers(self):\n        opt = torch.optim.Adam(self.net.parameters(), lr=self.hparams.learning_rate)\n        return {\n            \"optimizer\": opt,\n            \"lr_scheduler\": {\n                \"scheduler\": torch.optim.lr_scheduler.ReduceLROnPlateau(opt, patience=CFG.SCHEDULER_PATIENCE, factor=CFG.SCHEDULER_FACTOR, verbose=True),\n                \"monitor\": \"train/loss\",\n                \"interval\": \"step\"\n            }\n        }\n\n    def forward(self, x):\n        x = self.net(x)\n        y = x[:, 0, :, :]\n        cb = x[:, 1, :, :]\n        cr = x[:, 2, :, :]\n        r = y + 1.402 * cr\n        g = y - 0.34414 * cb - 0.71414 * cr\n        b = y + 1.772 * cb\n        x = torch.stack([r, g, b], dim=1)\n        return x\n    \n    def training_step(self, batch, batch_idx):\n        colored_batch, greyscale_batch = batch\n\n        preds = self(greyscale_batch)\n        loss = self.criterion(preds, colored_batch)\n        \n        self.log(\"train/loss\", loss)\n        self.log(\"train/mse\", self.mse(preds, colored_batch))\n        self.log(\"train/huber\", self.huber(preds, colored_batch))\n        \n        return loss","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:06:22.058075Z","iopub.execute_input":"2022-03-01T19:06:22.058778Z","iopub.status.idle":"2022-03-01T19:06:22.07187Z","shell.execute_reply.started":"2022-03-01T19:06:22.058744Z","shell.execute_reply":"2022-03-01T19:06:22.071144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training 🏃\nCreate a Pytorch lightning trainer with our configuration, and run our model on our dataset:","metadata":{}},{"cell_type":"code","source":"\"\"\"\n    Callbacks\n\"\"\"\n\nclass WandbImageCallback(pl.Callback):\n    def __init__(self, data, display_frequency=300):\n        super().__init__()\n        self.display_frequency = display_frequency\n        self.dataset = WandDID(data, return_colored=False)\n        self.loader = torch.utils.data.DataLoader(\n            self.dataset,\n            batch_size=CFG.BATCH_SIZE,\n            num_workers=CFG.NUM_WORKERS,\n            pin_memory=True,\n            shuffle=False\n        )\n        self.mean=torch.tensor([0.485, 0.456, 0.406]).unsqueeze(-1).unsqueeze(-1)\n        self.std=torch.tensor([0.229, 0.224, 0.225]).unsqueeze(-1).unsqueeze(-1)\n        self.batch = next(iter(self.loader))\n        \n    def unnormalize(self, x):\n        x = x.detach().cpu() * self.std + self.mean\n        return (x * 255).permute(1, 2, 0).clamp(0, 255).numpy().astype(np.uint8)\n        \n    def on_batch_end(self, trainer: pl.Trainer, pl_module: LightningModule):\n        if trainer.global_step % self.display_frequency > 0:\n            return\n        \n        with torch.no_grad():\n            pl_module.eval()\n            images = pl_module(self.batch.to(pl_module.device))\n            pl_module.train()\n\n        wandb_images = []\n        for image in images:\n            image = self.unnormalize(image)\n            wandb_images.append(wandb.Image(image))\n        \n        trainer.logger.experiment.log({\n            \"val/examples\": wandb_images,\n            \"global_step\": trainer.global_step\n        })","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-01T19:06:22.073263Z","iopub.execute_input":"2022-03-01T19:06:22.073582Z","iopub.status.idle":"2022-03-01T19:06:22.085706Z","shell.execute_reply.started":"2022-03-01T19:06:22.073526Z","shell.execute_reply":"2022-03-01T19:06:22.084607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = WandDRecolorize()\nwandb_logger = WandbLogger(project=\"W&D - recolorization\")\n# Trainer \ntrainer = Trainer(\n    profiler=\"simple\", # Profiling\n    gpus=1,# Use the one GPU we have\n    max_epochs=CFG.EPOCHS,\n    logger=wandb_logger,\n    log_every_n_steps=10,\n    callbacks=[WandbImageCallback(train_grey_data)]\n)\n# Let's go ⚡\ntrainer.fit(model, train_loader)\ntrainer.save_checkpoint(CFG.MODEL_PATH)","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:06:22.087235Z","iopub.execute_input":"2022-03-01T19:06:22.087585Z","iopub.status.idle":"2022-03-01T19:06:44.524014Z","shell.execute_reply.started":"2022-03-01T19:06:22.087545Z","shell.execute_reply":"2022-03-01T19:06:44.523188Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Inference 🔮\n","metadata":{}},{"cell_type":"code","source":"mean=torch.tensor([0.485, 0.456, 0.406]).unsqueeze(-1).unsqueeze(-1)\nstd=torch.tensor([0.229, 0.224, 0.225]).unsqueeze(-1).unsqueeze(-1)\n!mkdir train_images\n!mkdir test_images","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-01T19:06:44.525596Z","iopub.execute_input":"2022-03-01T19:06:44.525874Z","iopub.status.idle":"2022-03-01T19:06:46.011228Z","shell.execute_reply.started":"2022-03-01T19:06:44.525823Z","shell.execute_reply":"2022-03-01T19:06:46.010137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_grey_dataset = WandDID(train_grey_data, return_colored=False)\ntrain_grey_loader = torch.utils.data.DataLoader(\n    train_grey_dataset,\n    batch_size=CFG.BATCH_SIZE,\n    num_workers=CFG.NUM_WORKERS,\n    pin_memory=True,\n    shuffle=False\n)\npreds = trainer.predict(model, dataloaders=train_grey_loader)\npreds = torch.cat(preds, dim=0)\n\nfor d, image in tqdm(zip(train_grey_data.iterrows(), preds)):\n    image = image.detach().cpu() * std + mean\n    image = (image * 255).permute(1, 2, 0).clamp(0, 255).numpy().astype(np.uint8)\n    img_path = os.path.join(\"train_images\", d[1][\"inference_image\"])\n    cv2.imwrite(img_path, image[:, :, ::-1])","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:06:46.016357Z","iopub.execute_input":"2022-03-01T19:06:46.018663Z","iopub.status.idle":"2022-03-01T19:07:03.446936Z","shell.execute_reply.started":"2022-03-01T19:06:46.018614Z","shell.execute_reply":"2022-03-01T19:07:03.44621Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_grey_dataset = WandDID(test_grey_data, return_colored=False, folder=\"test_images\")\ntest_grey_loader = torch.utils.data.DataLoader(\n    test_grey_dataset,\n    batch_size=CFG.BATCH_SIZE,\n    num_workers=CFG.NUM_WORKERS,\n    pin_memory=True,\n    shuffle=False\n)\npreds = trainer.predict(model, dataloaders=test_grey_loader)\npreds = torch.cat(preds, dim=0)\n\nfor d, image in tqdm(zip(test_grey_data.iterrows(), preds)):\n    image = image.detach().cpu() * std + mean\n    image = (image * 255).permute(1, 2, 0).clamp(0, 255).numpy().astype(np.uint8)\n    img_path = os.path.join(\"test_images\", d[1][\"inference_image\"])\n    cv2.imwrite(img_path, image[:, :, ::-1])","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:07:03.448555Z","iopub.execute_input":"2022-03-01T19:07:03.448811Z","iopub.status.idle":"2022-03-01T19:07:07.325889Z","shell.execute_reply.started":"2022-03-01T19:07:03.448772Z","shell.execute_reply":"2022-03-01T19:07:07.325219Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Log our model in wandb and finish the run\ntry:\n    artifact = wandb.log_artifact(CFG.MODEL_PATH, name='w_and_d-colorize', type='model') \n    wandb_logger.finalize(\"success\")\n    wandb.finish()\nexcept Exception as e:\n    print(e)","metadata":{"_kg_hide-input":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Inspect the samples 🕵️","metadata":{}},{"cell_type":"code","source":"def plot_images(batch, folder=\"train_images\", row=4, col=4):\n    \"\"\"\n        Copied and adapted from https://www.kaggle.com/awsaf49/happywhale-data-distribution\n    \"\"\"\n    plt.figure(figsize=(col*3, row*3))\n    for i in range(row*col):\n        plt.subplot(row, col, i+1)\n        path = os.path.join(BASE_PATH, folder,  batch[\"inference_image\"].iloc[i])\n        img_grey = cv2.imread(path)\n        path = os.path.join(\".\", folder,  batch[\"inference_image\"].iloc[i])\n        img_color = cv2.imread(path)\n        img = np.concatenate([img_grey, img_color], axis=1)[:, :, ::-1]\n        plt.imshow(img)\n        plt.axis('off')\n    plt.tight_layout()\n    plt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-01T19:07:07.327273Z","iopub.execute_input":"2022-03-01T19:07:07.327942Z","iopub.status.idle":"2022-03-01T19:07:07.336586Z","shell.execute_reply.started":"2022-03-01T19:07:07.327897Z","shell.execute_reply":"2022-03-01T19:07:07.335874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Train images\")\nplot_images(train_grey_data)","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:07:07.339683Z","iopub.execute_input":"2022-03-01T19:07:07.340163Z","iopub.status.idle":"2022-03-01T19:07:08.392403Z","shell.execute_reply.started":"2022-03-01T19:07:07.340124Z","shell.execute_reply":"2022-03-01T19:07:08.386823Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Test images\")\nplot_images(test_grey_data, folder=\"test_images\")","metadata":{"execution":{"iopub.status.busy":"2022-03-01T19:07:08.393354Z","iopub.execute_input":"2022-03-01T19:07:08.393622Z","iopub.status.idle":"2022-03-01T19:07:09.612038Z","shell.execute_reply.started":"2022-03-01T19:07:08.393582Z","shell.execute_reply":"2022-03-01T19:07:09.611402Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see that it's quite good but not perfect. There are sometimes unconsistant colors in the water, some shades of blue on the individuals and some pictures that stay grey-ish. We could probably improve it by cleaning our data, because their are still some images that are not proper grey-scale but almost...<br>\n\n## Some ideas for improvement\n* Adding a GAN-loss to have even more realistic results\n* Use a style-gan architecture\n* Condition the model on an embedding of the individual\n* Adding down-sample augmentations","metadata":{}},{"cell_type":"code","source":"!rm -rf ./\"W&D - recolorization\"\n!rm -rf wandb\n!zip -r test_images.zip test_images\n!zip -r train_images.zip train_images","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-03-01T19:07:09.61337Z","iopub.execute_input":"2022-03-01T19:07:09.614194Z","iopub.status.idle":"2022-03-01T19:07:32.051874Z","shell.execute_reply.started":"2022-03-01T19:07:09.614147Z","shell.execute_reply":"2022-03-01T19:07:32.051077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion 🤷\nWe know have a way to colorize and denoise greyscales images. We can still improve the method by removing the \"almost greyscale samples\"...\n\n👍 If you found this notebook helpful or learned something please consider giving an upvote, and if you disagree with the content, I'll be pleased to dicsuss it with you in the comments.\n\n😊 Happy Kaggling everyone !","metadata":{}},{"cell_type":"markdown","source":"![](https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/Thats_all_folks.svg/2560px-Thats_all_folks.svg.png)","metadata":{}}]}