{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import numpy as np\nimport torch\nfrom torch import optim\nimport torch.nn as nn\nfrom PIL import Image, ImageOps\nfrom torchvision import models\nfrom matplotlib import pyplot as plt\nfrom torch.optim import lr_scheduler\nimport os \nimport time\nimport cv2","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-08-09T02:10:57.886451Z","iopub.execute_input":"2022-08-09T02:10:57.886848Z","iopub.status.idle":"2022-08-09T02:10:57.893187Z","shell.execute_reply.started":"2022-08-09T02:10:57.886818Z","shell.execute_reply":"2022-08-09T02:10:57.891781Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Introduction\n\nIn this Notebook I will be attempting to create a GAN (Generative Adversarial Network). This is a type of model architecture for training a generator network to create an output, usually images, that is reflective of the distribution of the training data. In this example we will be giving a GAN the MNIST data as its training data and then training on that. The hope is that at the conclusion we will have a generator that produces believable images that could be part of the MNIST training set but are in fact new creations. One way that can be helpful to think about this is all the images of a number in the MNIST data represent a broad distribution of the way humans write out numbers. Now there is a limited amount of examples in the MNIST data set of a number but the way to write a number is near infinitely variable.  Our trained generator when it chooses to write that number will be creating a number from within that distribution and is capable of filling in the gaps with data not found in the dataset.\n\nThe way a typical GAN functions is that a generator network is created that uses an input of random noise to produce an image, this image is then inputted into a discriminator network that attempts to determine whether the image is a real example from the training data or a fake example produced by the generator. Both of these networks compete and get better over time. When it works this is an effective means of producing capable generator networks.","metadata":{}},{"cell_type":"markdown","source":"# Create Custom Dataset\n\nHere I am creating a custom dataset class to store the MNIST images in at training time.","metadata":{}},{"cell_type":"code","source":"from torch.utils.data import Dataset\n\nclass CustomDataset(Dataset):\n  def __init__(self, X, y, BatchSize, transform):\n    super().__init__()\n    self.BatchSize = BatchSize\n    self.y = y\n    self.X = X\n    self.transform = transform\n    \n  def num_of_batches(self):\n    \"\"\"\n    Detect the total number of batches\n    \"\"\"\n    return math.floor(len(self.list_IDs) / self.BatchSize)\n\n  def __getitem__(self,idx):\n    class_id = self.y[idx]\n    img = self.X[idx].reshape(28,28)\n    img = Image.fromarray(np.uint8(img * 255)).convert('L')\n    img = self.transform(img)\n    return img, torch.tensor(int(class_id))\n\n  def __len__(self):\n    return len(self.X)","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:10:57.979346Z","iopub.execute_input":"2022-08-09T02:10:57.979983Z","iopub.status.idle":"2022-08-09T02:10:57.988127Z","shell.execute_reply.started":"2022-08-09T02:10:57.979949Z","shell.execute_reply":"2022-08-09T02:10:57.987056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Data","metadata":{}},{"cell_type":"code","source":"from numpy import genfromtxt\ndf = genfromtxt('../input/digit-recognizer/train.csv', delimiter=',')\ntest_df = genfromtxt(\"../input/digit-recognizer/test.csv\", delimiter=',')\ndf = df[1:]\ny = df[:,0]\nX = df[:,1:]\nX = X / 255\n\ntest_X = test_df[1:] / 255","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:10:58.080770Z","iopub.execute_input":"2022-08-09T02:10:58.081185Z","iopub.status.idle":"2022-08-09T02:11:54.333085Z","shell.execute_reply.started":"2022-08-09T02:10:58.081141Z","shell.execute_reply":"2022-08-09T02:11:54.331838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"_, *test_y = range(len(test_X)) ","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:54.335490Z","iopub.execute_input":"2022-08-09T02:11:54.336244Z","iopub.status.idle":"2022-08-09T02:11:54.342813Z","shell.execute_reply.started":"2022-08-09T02:11:54.336194Z","shell.execute_reply":"2022-08-09T02:11:54.341446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Set The Dataloaders\n\nHere I will first create some transforms. At training time I will slightly move the images around to create artificial variation. This expands the size of the training set and synthetically gives the neural network more to work on. I will slightly change the brightness and contrast of each image to further increase variety.\n\nThen the transforms and images will be added to dataloaders to make them easily available in the training loop.","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\nfrom torch.utils.data import DataLoader\nfrom torchvision import transforms\n\n# Define Transforms\ntransform = transforms.Compose([\n                transforms.ToTensor(),\n                transforms.RandomAffine(degrees=0, translate=(0.05, 0.05), fill=0),\n                transforms.Normalize([0.5], [0.5])\n            ])\n\ntest_transform = transforms.Compose([\n                transforms.ToTensor(),\n                transforms.Normalize((0.5,), (0.5,)),\n            ])\n\ntrain_number = 2560\n\ndataset_stages = ['train', 'test']\nbatch_size = 640\nimage_datasets = {'train' : CustomDataset(X[:], y[:], batch_size, transform), 'test' : CustomDataset(test_X, test_y, batch_size, test_transform)}\n\ndataloaders = {x: DataLoader(image_datasets[x], batch_size=image_datasets[x].BatchSize, shuffle=False, num_workers=0) \n               for x in dataset_stages}\ndataset_sizes = {x: len(image_datasets[x]) for x in ['train']}","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:54.344668Z","iopub.execute_input":"2022-08-09T02:11:54.345025Z","iopub.status.idle":"2022-08-09T02:11:54.357033Z","shell.execute_reply.started":"2022-08-09T02:11:54.344988Z","shell.execute_reply":"2022-08-09T02:11:54.355829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Look at Dataset Image\n\nTo make sure everything is working correctly I will look at an image from the dataset. I will look at it 10 times to check that the transforms are suitable for the image. The transforms will be applied randomly with every access.","metadata":{}},{"cell_type":"code","source":"def denormalise(image):\n    image = image.numpy().transpose(1, 2, 0)  # PIL images have channel last\n    mean = [0.5, 0.5, 0.5]\n    stdd = [0.5, 0.5, 0.5]\n    image = (image * stdd + mean).clip(0, 1)\n    return image\n\nfig,ax = plt.subplots(2,5)\nfor i in range(10):\n    result = image_datasets[\"train\"][4]\n    image =  denormalise(result[0].cpu())\n    ax[i%2][i//2].imshow(image)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:54.359830Z","iopub.execute_input":"2022-08-09T02:11:54.360208Z","iopub.status.idle":"2022-08-09T02:11:55.126436Z","shell.execute_reply.started":"2022-08-09T02:11:54.360174Z","shell.execute_reply":"2022-08-09T02:11:55.125465Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Look at Training Images\n\nLet's look at a set of training images in general so we can gauge them against the generator output in later training.","metadata":{}},{"cell_type":"code","source":"fig,ax = plt.subplots(5,5)\nfor i in range(25):\n    nparray = image_datasets['train'][i][0].cpu().numpy() \n    image = transforms.ToPILImage()(image_datasets['train'][i][0].cpu()).convert(\"RGB\")\n    ax[i%5][i//5].imshow(image)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T05:04:02.342700Z","iopub.execute_input":"2022-08-09T05:04:02.343102Z","iopub.status.idle":"2022-08-09T05:04:04.322784Z","shell.execute_reply.started":"2022-08-09T05:04:02.343069Z","shell.execute_reply":"2022-08-09T05:04:04.321633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Define Generator Network","metadata":{}},{"cell_type":"code","source":"import torch.nn.functional as F\n\nclass GeneratorNet(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.trans1 = nn.ConvTranspose2d(100, 64, 4, stride=2, padding=1, bias=False)\n        self.trans2 = nn.ConvTranspose2d(64, 32, 4, stride=2, padding=1, bias=False)\n        self.trans3 = nn.ConvTranspose2d(32, 16, 4, stride=2, padding=1, bias=False)\n        self.trans4 = nn.ConvTranspose2d(16, 8, 3, stride=2, padding=1, bias=False)\n        self.trans5 = nn.ConvTranspose2d(8, 1, 2, stride=2, padding=1, bias=False)\n    def forward(self, x):\n        x = F.leaky_relu(self.trans1(x))\n        x = F.leaky_relu(self.trans2(x))\n        x = F.leaky_relu(self.trans3(x))\n        x = F.leaky_relu(self.trans4(x))\n        x = F.tanh(self.trans5(x))\n        return x","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.127780Z","iopub.execute_input":"2022-08-09T02:11:55.128140Z","iopub.status.idle":"2022-08-09T02:11:55.137759Z","shell.execute_reply.started":"2022-08-09T02:11:55.128107Z","shell.execute_reply":"2022-08-09T02:11:55.136816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generatorNet = GeneratorNet()\ngeneratorNet(torch.zeros((32,100,1,1))).shape","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.138828Z","iopub.execute_input":"2022-08-09T02:11:55.139883Z","iopub.status.idle":"2022-08-09T02:11:55.165227Z","shell.execute_reply.started":"2022-08-09T02:11:55.139840Z","shell.execute_reply":"2022-08-09T02:11:55.164459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Count Trainable Generator Parameters","metadata":{}},{"cell_type":"code","source":"model_parameters = filter(lambda p: p.requires_grad, generatorNet.parameters())\nparams = sum([np.prod(p.size()) for p in model_parameters])\nparams","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.166280Z","iopub.execute_input":"2022-08-09T02:11:55.166964Z","iopub.status.idle":"2022-08-09T02:11:55.175576Z","shell.execute_reply.started":"2022-08-09T02:11:55.166932Z","shell.execute_reply":"2022-08-09T02:11:55.174349Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Define Discriminator Network","metadata":{}},{"cell_type":"code","source":"class DiscriminatorNet(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.model = nn.Sequential(\n            nn.Linear(28*28, 300),\n            nn.LeakyReLU(0.2, inplace=True),\n            nn.Linear(300, 200),\n            nn.LeakyReLU(0.2, inplace=True),\n            nn.Linear(200, 1),\n            nn.Sigmoid()\n        )\n\n    def forward(self, x):\n        x = x.view(x.size(0), -1)\n        out = self.model(x)\n        return out.view(x.size(0))","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.177860Z","iopub.execute_input":"2022-08-09T02:11:55.178763Z","iopub.status.idle":"2022-08-09T02:11:55.187495Z","shell.execute_reply.started":"2022-08-09T02:11:55.178717Z","shell.execute_reply":"2022-08-09T02:11:55.186727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Test Discriminator\n\nI will test this discriminator by giving it a dummy input representing the same dimensions as the images it will see from the dataset and generator.","metadata":{}},{"cell_type":"code","source":"discriminatorNet = DiscriminatorNet()\ndiscriminatorNet(torch.zeros((32,1,28,28))).shape","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.188918Z","iopub.execute_input":"2022-08-09T02:11:55.190558Z","iopub.status.idle":"2022-08-09T02:11:55.204599Z","shell.execute_reply.started":"2022-08-09T02:11:55.190515Z","shell.execute_reply":"2022-08-09T02:11:55.203634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Count Trainable Discriminator Parameters","metadata":{}},{"cell_type":"code","source":"model_parameters = filter(lambda p: p.requires_grad, discriminatorNet.parameters())\nparams = sum([np.prod(p.size()) for p in model_parameters])\nparams","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.208069Z","iopub.execute_input":"2022-08-09T02:11:55.208801Z","iopub.status.idle":"2022-08-09T02:11:55.218281Z","shell.execute_reply.started":"2022-08-09T02:11:55.208758Z","shell.execute_reply":"2022-08-09T02:11:55.217099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training the Model\n\nLet's create a training loop and train this model","metadata":{}},{"cell_type":"code","source":"disc_criterion = nn.BCELoss()\ngen_criterion = nn.BCELoss()\n\nlearning_rate = 0.002\ngen_optimizer = optim.Adam(generatorNet.parameters(), lr = learning_rate, betas=(0.5, 0.999))\ndisc_optimizer = optim.Adam(discriminatorNet.parameters(), lr = learning_rate, betas=(0.5, 0.999))\n\nreal_label = 1\nfake_label = 0\n\n\ngen_exp_lr_scheduler = lr_scheduler.StepLR(gen_optimizer, step_size=1, gamma=0.95)\ndisc_exp_lr_scheduler = lr_scheduler.StepLR(disc_optimizer, step_size=1, gamma=0.95)","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.220335Z","iopub.execute_input":"2022-08-09T02:11:55.221296Z","iopub.status.idle":"2022-08-09T02:11:55.230527Z","shell.execute_reply.started":"2022-08-09T02:11:55.221252Z","shell.execute_reply":"2022-08-09T02:11:55.229446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%matplotlib inline\n\ndef get_learning_rate(optimizer):\n    for param_group in optimizer.param_groups:\n        return param_group['lr']\n    \ndef DiscriminatorTraining(discriminator, generator, disc_criterion, disc_optimizer, inputs, labels, disc_losses, disc_lrs):\n    # Discriminator Training\n    discriminator.train()\n\n    discriminator.zero_grad()\n    inputs = inputs.to(device)\n    labels = labels.to(device)\n    real_labels = torch.full((inputs.size(0),), real_label, dtype=torch.float, device=device)\n    real_output = discriminator(inputs)\n    real_labels = real_labels\n    disc_real_loss = disc_criterion(real_output, real_labels)\n    disc_real_loss.backward()\n\n    noise_vector = torch.randn((inputs.size(0),100,1,1), device=device)\n    fakes = generator(noise_vector)\n    fake_labels = torch.full((fakes.size(0),), fake_label, dtype=torch.float, device=device)\n    fake_output = discriminator(fakes)\n    fake_labels = fake_labels\n    disc_fake_loss = disc_criterion(fake_output, fake_labels)\n    disc_fake_loss.backward()\n    disc_optimizer.step()\n    disc_losses.append(disc_real_loss + disc_fake_loss)\n\n    # Record and update lr\n    disc_lrs.append(get_learning_rate(disc_optimizer))\n    \n    return (disc_real_loss + disc_fake_loss)\n\ndef GeneratorTraining(discriminator, generator, gen_criterion, gen_optimizer, gen_losses, inputs, gen_lrs):\n    generator.train()\n    generator.zero_grad()\n    noise_vector = torch.randn((inputs.size(0),100,1,1), device=device)\n    fakes = generator(noise_vector)\n    generator_output = discriminator(fakes)\n    real_labels = torch.full((inputs.size(0),), real_label, dtype=torch.float, device=device)\n    generator_loss = gen_criterion(generator_output, real_labels)\n    gen_losses.append(generator_loss)\n    generator_loss.backward()\n    gen_optimizer.step()\n    # Record and update lr\n    gen_lrs.append(get_learning_rate(gen_optimizer))\n    return generator_loss\n\ndef train_model(generator, discriminator, train_loader, gen_criterion, disc_criterion, gen_optimizer, disc_optimizer, gen_exp_lr_scheduler, disc_exp_lr_scheduler, device, epochs_per_print=10, num_epochs=200):\n    since = time.time()\n    log = {}\n    log[\"gen_loss\"] = []\n    log[\"disc_loss\"] = []\n    \n    last_disc_loss = 0\n    last_gen_loss = 0\n    for epoch in range(num_epochs):\n        # Training Phase\n        generator.train()\n        gen_losses = list()\n        gen_lrs = list()\n        disc_losses = list()\n        disc_lrs = list()\n        for inputs, labels in train_loader:\n            # Discriminator Training\n            if last_gen_loss <= last_disc_loss:\n                for i in range(10):\n                    last_disc_loss = DiscriminatorTraining(discriminator, generator, disc_criterion, disc_optimizer, inputs, labels, disc_losses, disc_lrs).item()\n                    # print(\"last disc loss\", last_disc_loss)\n                    if last_gen_loss > last_disc_loss:\n                        break\n                    if i == 9:\n                        # print(\"end disc loop\")\n                        last_gen_loss = GeneratorTraining(discriminator, generator, gen_criterion, gen_optimizer, gen_losses, inputs, gen_lrs).item()  \n            \n            if last_disc_loss <= last_gen_loss:\n                for i in range(10):\n                    last_gen_loss = GeneratorTraining(discriminator, generator, gen_criterion, gen_optimizer, gen_losses, inputs, gen_lrs).item()  \n                    # print(\"last gen loss\", last_gen_loss)     \n                    if last_disc_loss > last_gen_loss:\n                        break  \n                    if i == 9:\n                        # print(\"end gen loop\")\n                        last_disc_loss = DiscriminatorTraining(discriminator, generator, disc_criterion, disc_optimizer, inputs, labels, disc_losses, disc_lrs).item()\n         \n        # Step scheduler\n        gen_exp_lr_scheduler.step()\n        disc_exp_lr_scheduler.step()\n        \n        result = {}\n        if (len(gen_losses)) == 0:\n            result['gen_loss'] = 0\n        else:\n            result['gen_loss'] = torch.stack(gen_losses).mean().item()\n        if (len(disc_losses)) == 0:\n            result['disc_loss'] = 0\n        else:\n            result['disc_loss'] = torch.stack(disc_losses).mean().item()\n        result['gen_lrs'] = gen_lrs\n        result['disc_lrs'] = disc_lrs\n        if len(gen_lrs) > 0:\n            result['gen_last_lr'] = gen_lrs[-1]\n        else:\n            result['gen_last_lr'] = 0\n        if len(disc_lrs) > 0:\n            result['disc_last_lr'] = disc_lrs[-1]\n        else:\n            result['disc_last_lr'] = 0\n        log[\"gen_loss\"].append(result['gen_loss'])\n        log[\"disc_loss\"].append(result['disc_loss'])\n            \n        # Display images\n        if ((epoch) % (epochs_per_print) == 0):\n            print(f\"Epoch [{epoch}], gen_last_lr: {result['gen_last_lr']:.8f}, disc_last_lr: {result['disc_last_lr']:.8f}, gen_loss: {result['gen_loss']:.4f}, disc_loss: {result['disc_loss']:.4f}\")\n            fig,ax = plt.subplots(1,5)\n            for i in range(5):\n                ax[i].cla()\n                noise_vector = torch.randn((1,100,1,1), device=device)\n                fake = generator(noise_vector)\n                image =  denormalise(fake.squeeze(0).cpu().detach())\n                ax[i].imshow(image)\n            fig.show()\n            plt.show()\n        \n    time_elapsed = time.time() - since\n    print('Training complete in {:.0f}m {:.0f}s'.format(\n        time_elapsed // 60, time_elapsed % 60))\n    return generator, discriminator, result, log\n","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.232131Z","iopub.execute_input":"2022-08-09T02:11:55.232461Z","iopub.status.idle":"2022-08-09T02:11:55.277766Z","shell.execute_reply.started":"2022-08-09T02:11:55.232420Z","shell.execute_reply":"2022-08-09T02:11:55.276578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\ngeneratorNet.to(device)\ndiscriminatorNet.to(device)\ngeneratorNet, discriminatorNet, result, log = train_model(generatorNet, discriminatorNet, dataloaders['train'], gen_criterion, disc_criterion, gen_optimizer, disc_optimizer, gen_exp_lr_scheduler, disc_exp_lr_scheduler, device, 1, 20)","metadata":{"execution":{"iopub.status.busy":"2022-08-09T02:11:55.279614Z","iopub.execute_input":"2022-08-09T02:11:55.280481Z","iopub.status.idle":"2022-08-09T04:07:05.547912Z","shell.execute_reply.started":"2022-08-09T02:11:55.280433Z","shell.execute_reply":"2022-08-09T04:07:05.546151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualise Loss","metadata":{}},{"cell_type":"code","source":"import plotly.express as px\n\nfig = px.line()\nfig.add_scatter(x=[*range(len(log[\"gen_loss\"]))], y=log[\"gen_loss\"], mode='lines', name='Generator Loss', showlegend=True)\nfig.add_scatter(x=[*range(len(log[\"disc_loss\"]))], y=log[\"disc_loss\"], mode='lines', name='Discriminator Loss', showlegend=True)\nfig.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-08-09T05:24:02.164030Z","iopub.execute_input":"2022-08-09T05:24:02.164781Z","iopub.status.idle":"2022-08-09T05:24:02.221117Z","shell.execute_reply.started":"2022-08-09T05:24:02.164742Z","shell.execute_reply":"2022-08-09T05:24:02.220327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Look At Final Output","metadata":{}},{"cell_type":"code","source":"fig,ax = plt.subplots(2,5)\nfor i in range(10):\n    noise_vector = torch.randn((1,100,1,1), device=device)\n    fake = generatorNet(noise_vector)\n    image =  denormalise(fake.squeeze(0).cpu().detach())\n    ax[i%2][i//2].imshow(image)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T04:18:55.792492Z","iopub.execute_input":"2022-08-09T04:18:55.793438Z","iopub.status.idle":"2022-08-09T04:18:56.557736Z","shell.execute_reply.started":"2022-08-09T04:18:55.793383Z","shell.execute_reply":"2022-08-09T04:18:56.556582Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion\n\nI feel the generator has converged in this example. Now it is true some numbers here don't look like numbers we'd be likely to write ourselves but if you look at some of the MNIST examples you will see that there are some pretty odd numbers in them to begin with. So I feel the examples generated are representative of the data at hand. Next I hope to look at conditional GANs and using the discriminator from it to potentially classify MNIST data. ","metadata":{}}]}