{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Document scope","metadata":{}},{"cell_type":"markdown","source":"Following the guidelines | instructions provided by Tyler about this challenge | test here is how I decided to tackle the problem\n\n* Download and analyze the data locally to understand image dimensions, labels, and overall how does a random sample image looks like\n* Make a 1st image classifier in PyTorch with CPU (based on this tutorial https://www.tensorflow.org/tutorials/images/classification) to have some starting point for the ML framework (network architecture and training)\n* Return to image analysis converting the input images to different dimensions (smaller) and saving them to disk\n* Make a 2nd image classifier in Tensorflow (based on this tutorial https://keras.io/examples/vision/image_classification_from_scratch/) to test a different  ML framework (network architecture, training, and prediction)\n* Profile my solution since locally I am using a normal laptop to understand how my training times from previous approaches can be reduced\n* Create output file in the correct format given test images\n* Writing this document showing details for each step and \"aha\" moments that make me decide some approaches (like which network to use, dimensions of images, labels to use, etc.)\n","metadata":{}},{"cell_type":"markdown","source":"# Understanding data","metadata":{}},{"cell_type":"code","source":"# First thing is to read the CSV file\nimport pandas\nimport os\nusinglocalmachine = False\ndatadir = \"\"\nif (usinglocalmachine):\n    datadir =  'd:\\\\plant-pathology-2021-fgvc8\\\\'\nelse:\n    datadir = '/kaggle/input/plant-pathology-2021-fgvc8/'\ncsvfile = os.path.join(datadir, 'train.csv')\ndataframes = pandas.read_csv(csvfile)\nprint(\"File at {} contains {} rows\".format(csvfile, len(dataframes)))","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:27:11.526028Z","iopub.execute_input":"2023-02-22T21:27:11.526470Z","iopub.status.idle":"2023-02-22T21:27:11.574483Z","shell.execute_reply.started":"2023-02-22T21:27:11.526426Z","shell.execute_reply":"2023-02-22T21:27:11.573394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Then I can get a random sample from the pandas dataframes to see how an image looks\nimport matplotlib.pyplot\nimport skimage.io\nsample = dataframes.sample()\nimagefilename = sample.iloc[0][0]\nlabel = sample.iloc[0][1]\nimagefilepath = os.path.join(datadir + 'train_images/', imagefilename)\nimage = skimage.io.imread(imagefilepath)\nmatplotlib.pyplot.imshow(image)\nmatplotlib.pyplot.title(label)\nmatplotlib.pyplot.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:27:20.018146Z","iopub.execute_input":"2023-02-22T21:27:20.018993Z","iopub.status.idle":"2023-02-22T21:27:22.406219Z","shell.execute_reply.started":"2023-02-22T21:27:20.018947Z","shell.execute_reply":"2023-02-22T21:27:22.405330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here is one thing to notice, the images are commonly fairly large (some 2k - 4k x 2k - 4k pixels) and not square. That might be a problem when trying some of the common neural network layers such as convolution 2D (I am aware convolution can be made non-square but for simplicity of developing a model better for me to stick for the moment with square and not that gigantic images)","metadata":{}},{"cell_type":"code","source":"# Later I will be using a more robust approach for doing transformation but for now let me show\n# how to do image resizing|cropping in order to have images that we can use for a 1st image classifier model\n# I am using scikit-image python package for manipulating images but I could be using other ones such as\n# opencv or writing my own image reader\nimport skimage.transform\nimport numpy\ndef imageResizingAndRandomCropping(imagedata, resizesize, cropsize):\n    h, w = imagedata.shape[:2]\n    if isinstance(resizesize, int):\n        if h > w:\n            resizeh, resizew = resizesize * h / w, resizesize\n        else:\n            resizeh, resizew = resizesize, resizesize * w / h\n    else:\n        resizeh, resizew = resizesize\n    resizedimage = skimage.transform.resize(imagedata, (resizeh, resizew))\n    croph, cropw = (cropsize, cropsize)\n    top = numpy.random.randint(0, resizeh - croph)\n    left = numpy.random.randint(0, resizew - cropw)\n    croppedimage = resizedimage[top:top + croph, left:left + cropw]\n    return croppedimage\n\ntransformedimage = imageResizingAndRandomCropping(image, 256, 224)\nmatplotlib.pyplot.imshow(transformedimage)\nmatplotlib.pyplot.title(label)\nmatplotlib.pyplot.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:27:28.218692Z","iopub.execute_input":"2023-02-22T21:27:28.219042Z","iopub.status.idle":"2023-02-22T21:27:31.832757Z","shell.execute_reply.started":"2023-02-22T21:27:28.219011Z","shell.execute_reply":"2023-02-22T21:27:31.831782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And voila, I have now an image that is square and have less pixels to work with. You can notice some stuff tho\n1. The cropping is being doing randomly but with the purpose of selecting the region of where to find the anchor point at the top-left corner with 32 pixels of area (because 256 - 224 = 32). The 224 is commonly used in image classification (seems previous research has been done with that in mind)\n2. The resizing maintains the aspect ratio of the original image, which is something we want to keep to avoid learning distorted features\n3. The big 'oh wait' moment here is the image has less 'information' than the original one given that is only a section of it. That is correct but given that our first approach would be to do some CPU better to try smaller stuff.","metadata":{}},{"cell_type":"code","source":"# Another thing to check is how many labels are in the dataset?\ndef consoleLogLabelsInformation(dataframes):\n    uniquelabelsinformation = {}\n    labelsinformation = {}\n    for _, rowdata in dataframes.iterrows():\n        labels = rowdata['labels']\n        if labels not in labelsinformation:\n            labelsinformation[labels] = 0\n        labelsinformation[labels] += 1\n        unique_labels = labels.split(' ')\n        for label in unique_labels:\n            if (label not in uniquelabelsinformation):\n                uniquelabelsinformation[label] = 0\n            uniquelabelsinformation[label] += 1\n    print(\"{} labels with the following distribution\".format(len(labelsinformation)))\n    for key,value in labelsinformation.items():\n        print(\"\\t{}: {}\".format(key, value))\n    print(\"{} unique labels with the following distribution\".format(len(uniquelabelsinformation)))\n    for key,value in uniquelabelsinformation.items():\n        print(\"\\t{}: {}\".format(key, value))\n        \nconsoleLogLabelsInformation(dataframes)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:27:35.674968Z","iopub.execute_input":"2023-02-22T21:27:35.675327Z","iopub.status.idle":"2023-02-22T21:27:36.553833Z","shell.execute_reply.started":"2023-02-22T21:27:35.675297Z","shell.execute_reply":"2023-02-22T21:27:36.552757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Checking those labels we see that we have **12 categories** but **only 6 are unique** (not a surprise given the information about the dataset mentions that labels are \"...a space delimited list of all diseases found in the image. Unhealthy leaves with too many diseases to classify visually will have the complex class, and may also have a subset of the diseases identified.\" So which one should I choose, well I decided two approaches\n1. Keep the initial 12 categories and throw that into the network (this is what I did for the 1st image classifier in PyTorch)\n2. Separate the categories into labels and use that into the network (this is what I did for the 2nd image classifier in Tensorflow)\n\nThe reasoning behind 2 is for the following reasons\n* The ML frameworks (PyTorch and Tensorflow) does have a way of loading a dataset straight in few lines of code but the data folder has to have specific structure\n* Seems to me somehow bias to use those original labels since they are compounding multiple diseases (worse case is something like keys: [rust complex], [frog_eye_leaf_spot complex]) and then how the network should be able to differentiate between rust and rust complex?","metadata":{}},{"cell_type":"markdown","source":"# Creating 1st image classifier using PyTorch","metadata":{}},{"cell_type":"code","source":"# ML frameworks have a way of loading data directly if its data directory has certain structure, for now\n# let me show you how to do it by 'hand'\nimport torchvision.transforms\nimport torch.utils.data\nimport torch.utils\nimport torch\nimport skimage.io\nimport skimage.transform\n\nclass Rescale(object):\n    def __init__(self, outputsize):\n        assert isinstance(outputsize, (int, tuple))\n        self.outputsize = outputsize\n    \n    def __call__(self, data):\n        image = data['image']\n        h, w = image.shape[:2]\n        if isinstance(self.outputsize, int):\n            if h > w:\n                newh, neww = self.outputsize * h / w, self.outputsize\n            else:\n                newh, neww = self.outputsize, self.outputsize * w / h\n        else:\n            newh, neww = self.outputsize\n        newh, neww = int(newh), int(neww)\n        transformedimage = skimage.transform.resize(image, (newh, neww))\n        return {'image':transformedimage, 'label':data['label']}\n\nclass RandomCrop(object):\n    def __init__(self, outputsize):\n        assert isinstance(outputsize, (int, tuple))\n        if isinstance(outputsize, int):\n            self.outputsize = (outputsize, outputsize)\n        else:\n            assert len(outputsize) == 2\n            self.outputsize = outputsize\n\n    def __call__(self, data):\n        image = data['image']\n        h, w = image.shape[:2]\n        newh, neww = self.outputsize\n        top = numpy.random.randint(0, h - newh)\n        left = numpy.random.randint(0, w - neww)\n        image = image[top:top + newh, left:left + neww]\n        return {'image':image, 'label':data['label']}\n\nclass ToTensor(object):\n    def __call__(self, data):\n        image = data['image']\n        image = image.transpose((2, 0, 1))\n        return {'image':torch.from_numpy(image).float(), 'label':data['label']}\n\nclass PlantDataset(torch.utils.data.Dataset):\n    def __init__(self, csvfile:str, rootdirectory:str, imagesfolder:str, transform=None):\n        self.csvfile = os.path.join(rootdirectory, csvfile)\n        self.imagesdirectory = imagesfolder\n        self.transform = transform\n        self.dataframes = pandas.read_csv(self.csvfile)\n        self.classes = {}\n        self.computeclasses()\n        \n    def __len__(self):\n        return len(self.dataframes)\n\n    def __getitem__(self, index):\n        imagefilepath = os.path.join(self.imagesdirectory, self.dataframes.iloc[index][0])\n        imagedata = skimage.io.imread(imagefilepath)\n        imagelabel = self.dataframes.iloc[index][1]\n        sample = {'image':imagedata, 'label':self.classes[imagelabel]}\n        if self.transform:\n            sample = self.transform(sample)\n        return sample\n    \n    def computeclasses(self):\n        labelid = 0\n        for _,row in self.dataframes.iterrows():\n            if row['labels'] not in self.classes:\n                self.classes[row['labels']] = labelid\n                labelid += 1\n\n# A trick to avoid the gigantic images is preprocess them and save them\n# to disk, this will be in handy in later sections (if not takes forever to train on CPU)\nuseresizedimages = True\nif (useresizedimages):\n    transformationtoapply = [ToTensor()]\n    imagesfolder = \"/kaggle/input/d/alejandroguayaquil/plant-pathology-2021-fgvc8/train_images_resized/\"\nelse:\n    transformationtoapply = [Rescale(256), RandomCrop(224), ToTensor()]\n    imagesfolder = \"train_images/\"\ntraindataset = PlantDataset(\n    csvfile=\"train.csv\", \n    rootdirectory=datadir,\n    imagesfolder=imagesfolder, \n    transform=torchvision.transforms.Compose(transformationtoapply)\n)\nbatchsize = 4\nnumworkers = 0\ntrainloader = torch.utils.data.DataLoader(\n    traindataset, \n    batch_size=batchsize, \n    shuffle=True, \n    num_workers=numworkers\n    )","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:27:51.766625Z","iopub.execute_input":"2023-02-22T21:27:51.766994Z","iopub.status.idle":"2023-02-22T21:27:56.071182Z","shell.execute_reply.started":"2023-02-22T21:27:51.766964Z","shell.execute_reply":"2023-02-22T21:27:56.070132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Everything seems great but how do we know the data is loaded correctly, let me console out\n# the first batch from the trainloader variable\nfor batchIndex, batchData in enumerate(trainloader):\n    inputs, labels = batchData['image'], batchData['label']\n    print(inputs.shape)\n    print(labels)\n    break","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:27:56.072930Z","iopub.execute_input":"2023-02-22T21:27:56.073319Z","iopub.status.idle":"2023-02-22T21:27:56.266143Z","shell.execute_reply.started":"2023-02-22T21:27:56.073283Z","shell.execute_reply":"2023-02-22T21:27:56.265123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Perfect, now we have an object that contains 4 images in a batch (that was the purpose of the batch_size variable).  Some stuff to notice from previous lines of code\n1. We have created the image transformation (resize and crop) as objects given that we want to use them when the PyTorch DataLoader call the __ getitem __() method. If we don't do this when we are enumerating the trainloader variable the program can crash since PyTorch Dataloader is expecting all elements in a batch to have the same dimensions (my guess is that can be made generic but for now let's keep it like that)\n2. The labels (computed in the method computeclasses() of PlantDataset) are changed from string to numerical value. This is important since when I was creating this network at some point I run in the problem that the labels as strings are not allowed","metadata":{}},{"cell_type":"code","source":"# Now the magic part of ML, creating the neural network model\n# For now let's just put the following class\nclass ImageClassificationNN(torch.nn.Module):\n    def __init__(self):\n        super(ImageClassificationNN, self).__init__()\n        self.convolution1 = torch.nn.Conv2d(3, 6, 5)\n        self.pool = torch.nn.MaxPool2d(2, 2)\n        self.convolution2 = torch.nn.Conv2d(6, 16, 5)\n        self.fullyconnected1 = torch.nn.Linear(16 * 53 * 53, 120)\n        self.fullyconnected2 = torch.nn.Linear(120, 84)\n        self.fullyconnected3 = torch.nn.Linear(84, 12)\n\n    def forward(self, x):\n        x = self.pool(torch.nn.functional.relu(self.convolution1(x)))\n        x = self.pool(torch.nn.functional.relu(self.convolution2(x)))\n        x = x.flatten(1)\n        x = torch.nn.functional.relu(self.fullyconnected1(x))\n        x = torch.nn.functional.relu(self.fullyconnected2(x))\n        x = self.fullyconnected3(x)\n        return x\n\nneuralnetwork = ImageClassificationNN()\nprint(neuralnetwork)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:28:01.824619Z","iopub.execute_input":"2023-02-22T21:28:01.825317Z","iopub.status.idle":"2023-02-22T21:28:01.891396Z","shell.execute_reply.started":"2023-02-22T21:28:01.825281Z","shell.execute_reply":"2023-02-22T21:28:01.890267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This network shown is composed by some 2D convolutions, a pooling (by default is max), and some fully connected layers. The decision of why to use such layers is by now following tutorials but from previous experience I have the following insights\n1. Convolutions will try to acquire features of an image (for example a Sobel operator would do edge detection). So in my choice we have 2 convolutions that will try to do similar stuff\n2. The pool reduces dimensionality of the features, so for example from having 224 x 224 image will pass to be a 110 x 100 'image' after the first convolution and first pool\n\nBesides that, we are using a ReLU activation function. This is just standard to make some neurons to be active/inactive as the network starts training\n\nIn addition, an important part to notice is the (16 * 53 * 53) factor that we have in our code. That one is not magic but refers to the dimensionality of the data | 'image' at that point in the network. 53 comes from (conv2d) 224 - 2 = 222 -> (pool) 220 - 2 / 2 = 110 -> (conv2d) 110 - 2 = 108 -> (pool) 108 - 2 / 2 = 53. I found this relevant since we would have to modify such values if we use a different image size as input","metadata":{}},{"cell_type":"code","source":"# This is the last part, just define the loss function, optimizer, and then start training\n# We will not be doing a full training but just show the 1st 10 iterations to check\n# the loss being reduced (meaning our network is working)\ncriterion = torch.nn.CrossEntropyLoss()\nlearningrate = 0.001\nmomentum = 0.9\noptimizer = torch.optim.SGD(neuralnetwork.parameters(), lr=learningrate, momentum=momentum)\nepochs = 1\nminibatch = 16\nfor epoch in range(epochs):\n    sampleloss = 0.0\n    for batchIndex, batchData in enumerate(trainloader):\n        inputs, labels = batchData['image'], batchData['label']\n        optimizer.zero_grad()\n        outputs = neuralnetwork(inputs)\n        loss = criterion(outputs, labels)\n        loss.backward()\n        optimizer.step()\n        sampleloss += loss.item()\n        if (batchIndex % minibatch == (minibatch - 1)):\n            print(\"Loss at {} of {}: {}\".format(batchIndex, len(trainloader), sampleloss / minibatch))\n            sampleloss = 0.0\n        if (batchIndex > 639):\n            break","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Previous training should take long (even on Kaggle instance) because I have not enabled the GPU neither do some optimization about the data. If is taking forever previous cell feel free to interrup it.\n\nAt this point we could try to save the model and start trying to do some predictions but although the loss is reduced (in my personal laptop goes from 2.5 to around 1.8) it is still not what we are looking for an image classifier (as mentioned takes really long to train with the original images and the loss metric is kept bouncing in the vicinity of 1.8 ... that is a sign of overfitting). Let me return back to manipulate some of the data to improve timings","metadata":{}},{"cell_type":"markdown","source":"# Returning to image analysis","metadata":{}},{"cell_type":"code","source":"# At this point we know the original images are big but we have a function to resize|crop them\n# How about if we do the following\n# 1. Create a new CSV that has separated labels (in case we want to do something like previous approach but approach 2 is superior for rapid development)\n# 2. Create directories for each separated label (this is done by hand in the file explorer)\n# The structure should look something like\n# data_dir:\n#     | --- separated_labels\n#                  | ---- complex\n#                  | ---- frog_eye_leaf_spot\n#                  | ---- healthy\n#                  | ---- powdery_mildew\n#                  | ---- rust\n#                  | ---- scab\n# 3. Copy image files from original directory in the appropiate folder\nimport shutil\nimport time\nimport csv\n\ndef createCSVWithUniqueLabels(dataframes):\n    separatedlabelrow = []\n    for _, rowdata in dataframes.iterrows():\n        labels = rowdata['labels'].split(' ')\n        image = rowdata['image']\n        for label in labels:\n            separatedlabelrow.append([image, label])\n    csvfiletowrite = \"train_separated_labels.csv\"\n    print(\"Writing separated labels in {}\".format(csvfiletowrite))\n    with open(csvfiletowrite, 'w', newline='') as csvfile:\n        csvwriter = csv.writer(csvfile)\n        csvwriter.writerow(['image', 'label'])\n        csvwriter.writerows(separatedlabelrow)\n\ndef readOriginalCSVAndConvertToSeparatedLabelsCSV():\n    csvfile = \"train.csv\"\n    dataframes = pandas.read_csv(os.path.join(\"d:\\\\plant-pathology-2021-fgvc8\\\\\", csvfile))\n    print(\"Number of rows in file {}: {}\".format(csvfile, len(dataframes)))\n    consoleLogLabelsInformation(dataframes)\n    createCSVWithUniqueLabels(dataframes)\n    csvfile = \"train_separated_labels.csv\"\n    dataframes = pandas.read_csv(os.path.join(\"\", csvfile))\n    print(\"Number of rows in file {}: {}\".format(csvfile, len(dataframes)))\n\ndef copyImagesFromMixedLabelsToSeparateLabels():\n    csvfile = \"train_separated_labels.csv\"\n    sourcedatadir = \"d:\\\\plant-pathology-2021-fgvc8\\\\train_images\\\\\"\n    destinationdatadir = \"d:\\\\plant-pathology-2021-fgvc8\\\\separated_labels\\\\\"\n    dataframes = pandas.read_csv(os.path.join(\"\", csvfile))\n    print(\"Number of rows {}\".format(len(dataframes)))\n    processsubset = False\n    numberoffiletoprocess = len(dataframes)\n    if processsubset:\n        numberoffiletoprocess = 512\n        dataframes = dataframes.iloc[:numberoffiletoprocess][:]\n    print(\"Processing {} rows\".format(len(dataframes)))\n    time1 = time.time()\n    for _, rowdata in dataframes.iterrows():\n        image = rowdata['image']\n        label = rowdata['label']\n        sourceimage = os.path.join(sourcedatadir, image)\n        destinationimage = os.path.join(destinationdatadir, label + \"\\\\\" + image)\n        shutil.copy(sourceimage, destinationimage)\n    time2 = time.time()\n    print(\"Copied {} files in {} seconds\".format(numberoffiletoprocess, time2 - time1))","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Previous functions are just reference and should not be run in the kaggle environment (for one I even left the paths on my personal laptop D:\\\\ drive so the functions should throw entirely, the other is it can take quite some time for those functions to finish, copying files took around 40 minutes in my computer .... ).\n\nBut why to do this effort? Well, if you see this tutorial https://www.tensorflow.org/tutorials/load_data/images you can notice that if we have the data as what I am showing in point 2 of the comments from previous cell, Tensorflow (works similar in PyTorch) is able to load all the data for you. So let's try","metadata":{}},{"cell_type":"markdown","source":"# Doing 2nd Image Classifier now using Tensorflow","metadata":{}},{"cell_type":"code","source":"# First load tensorflow and disable warnings (they can be annoying)\nimport tensorflow\nprint(tensorflow.__version__)\ntensorflow.get_logger().setLevel('ERROR')\ntensorflow.debugging.set_log_device_placement(False)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:25:47.755281Z","iopub.execute_input":"2023-02-22T21:25:47.755734Z","iopub.status.idle":"2023-02-22T21:26:00.119929Z","shell.execute_reply.started":"2023-02-22T21:25:47.755649Z","shell.execute_reply":"2023-02-22T21:26:00.118895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pathlib\nimport math\n# data_dir_with_structure = \"/kaggle/input/plant-pathology-2021-fgvc8-image-resized/separated_labels_square_resized/\"\ndata_dir_with_structure = \"/kaggle/input/plant-dataset-subset/separated_labels_square_resized_subset/\"\nimage_width = 224\nimage_height = 224\nbatch_size = 4\n# Note that tensorflow 2.11.0 (my machine) can have subset= argument as \"both\" but\n# Kaggle is using tensorflow 2.9.2 (at least stable version) and does requires duplicating this portion\n# of code with different argument\ntrain_dataset = tensorflow.keras.utils.image_dataset_from_directory(\n    data_dir_with_structure,\n    validation_split=0.2,\n    subset=\"training\",\n    seed=123,\n    image_size=(image_width, image_height),\n    batch_size=batch_size\n)\nvalidation_dataset = tensorflow.keras.utils.image_dataset_from_directory(\n    data_dir_with_structure,\n    validation_split=0.2,\n    subset=\"validation\",\n    seed=123,\n    image_size=(image_width, image_height),\n    batch_size=batch_size\n)\nclass_names = train_dataset.class_names\nprint(class_names)\nimage_count = len(list(pathlib.Path(data_dir_with_structure).glob('*/*.jpg')))\ntraining_dataset_num_images = math.ceil(image_count * 0.8)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:26:05.482826Z","iopub.execute_input":"2023-02-22T21:26:05.483479Z","iopub.status.idle":"2023-02-22T21:26:10.827099Z","shell.execute_reply.started":"2023-02-22T21:26:05.483442Z","shell.execute_reply":"2023-02-22T21:26:10.826047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And voila, I have loaded all the data in few lines of code. Some stuff to point out\n1. Notice that I am using again the resized images of 224 x 224. To do so I combine the codes from __copyImagesFromMixedLabelsToSeparateLabels__() function and the __imageResizingAndRandomCropping__() function that is at the beginning of the notes. Be warn that doing such was time consuming, my personal laptop took 15 hours for that ... but I was using 1 thread and not optimizing anything\n2. The __image_dataset_from_directory__() method has the argument 'validation_split' which is important since takes the whole dataset and splits it into two categories: training and validation. This is important (and something I didn't add to the PyTorch implementation) since it allows the network to have some point of reference of how well its learning is going (by comparing the current state to some images that we know the actual answers)\n3. I don't even have to care about the CSV file anymore. The price to pay is the time that it takes to have the data in the correct structure for the loaders","metadata":{}},{"cell_type":"code","source":"# Let me show again an image from the dataset\nimport matplotlib.pyplot\nfor image_batch, label_batch in train_dataset:\n    print(image_batch[0].dtype)\n    print(label_batch)\n    matplotlib.pyplot.imshow(image_batch[0].numpy().astype(\"uint8\"))\n    matplotlib.pyplot.title(class_names[label_batch[0]])\n    matplotlib.pyplot.axis(\"off\")\n    break","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:26:23.089862Z","iopub.execute_input":"2023-02-22T21:26:23.090231Z","iopub.status.idle":"2023-02-22T21:26:23.660361Z","shell.execute_reply.started":"2023-02-22T21:26:23.090201Z","shell.execute_reply":"2023-02-22T21:26:23.659375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Nice, and the loader took even care of mapping the category strings into numerical value. \n\nBut one thing, why to use the __astype__(\"uint8\") method of numpy? In the image display is just to avoid a warning but is a good starting point to remember that images (channels) can be encoded in different ways, for example uint8 which refers to Unsigned Integer of 8 bits (or values that go in the range [0,255]) or float32 which is Floating of 32 bits (or values that go in the range [0,1]).\n\nNetworks like to work in floating point, this because is better to have some decimal representation when tunning values (rather than having pure integers), but how do we make the image to be in the range of [0,1] given that most likely internally our dataloader might be doing something like numpy().astype(\"uint8\")? Well, we have to normalize it (after all, range [0,255] to [0,1] is just a 'simple' mapping).\n\nI could create a function | class named __normalizeImageData__() but that can be tedious and error prone. So let tensorflow do it for you (there are two approaches for it, you can see about it here https://keras.io/examples/vision/image_classification_from_scratch/, for know I will be putting it directly in a layer).","metadata":{}},{"cell_type":"code","source":"num_classes = len(class_names)\ndef create_model(num_classes):\n    model = tensorflow.keras.models.Sequential([\n        tensorflow.keras.layers.Rescaling(1.0 / 255, input_shape=(image_width, image_height, 3)),\n        tensorflow.keras.layers.Conv2D(16, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Conv2D(32, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Conv2D(64, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Flatten(),\n        tensorflow.keras.layers.Dense(128, activation=\"relu\"),\n        tensorflow.keras.layers.Dense(num_classes)\n    ])\n    model.compile(\n        optimizer=\"adam\", \n        loss=tensorflow.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n        metrics=['accuracy']\n    )\n    return model\n\nmodel = create_model(num_classes)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:28:31.941012Z","iopub.execute_input":"2023-02-22T21:28:31.941373Z","iopub.status.idle":"2023-02-22T21:28:35.164208Z","shell.execute_reply.started":"2023-02-22T21:28:31.941342Z","shell.execute_reply":"2023-02-22T21:28:35.163220Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If you see the architecture of previous network is very similar to what we did with PyTorch, couple of 2D convolutions, couple of max poolings, some fully connected (fancy that Tensorflow call them __Dense__()) and some ReLU activation function. The optimizer and loss are also same, 'Adam' and CrossEntropy.","metadata":{}},{"cell_type":"code","source":"# Let see the network architecture in console\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:28:38.910796Z","iopub.execute_input":"2023-02-22T21:28:38.911650Z","iopub.status.idle":"2023-02-22T21:28:38.943573Z","shell.execute_reply.started":"2023-02-22T21:28:38.911608Z","shell.execute_reply":"2023-02-22T21:28:38.942707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This is a little of 'magic' for now (profiler section might try to cover some of it)\n# and makes training faster with the price of more RAM\n# Given that Kaggle could be constrained in RAM let me add a flag to enable|disable\nuse_data_optimization = False\nif use_data_optimization:\n    train_dataset = train_dataset.cache().shuffle(100).prefetch(buffer_size=tensorflow.data.AUTOTUNE)\n    validation_dataset = validation_dataset.cache().prefetch(buffer_size=tensorflow.data.AUTOTUNE)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:28:45.950997Z","iopub.execute_input":"2023-02-22T21:28:45.951374Z","iopub.status.idle":"2023-02-22T21:28:45.956551Z","shell.execute_reply.started":"2023-02-22T21:28:45.951341Z","shell.execute_reply":"2023-02-22T21:28:45.955761Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# So is time for training\nepochs = 15\nhistory = model.fit(\n    train_dataset,\n    validation_data=validation_dataset,\n    epochs=epochs,\n    verbose=1\n)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:28:48.883742Z","iopub.execute_input":"2023-02-22T21:28:48.884142Z","iopub.status.idle":"2023-02-22T21:29:35.854647Z","shell.execute_reply.started":"2023-02-22T21:28:48.884110Z","shell.execute_reply":"2023-02-22T21:29:35.853681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Using a 10% of the resized images takes 30 seconds per epoch in my personal computer, while 1s in the Kaggle instance with GPU, I also notice that the whole resized image dataset is estimated to take 30 to 60 seconds per epoch, while on my personal computer whole dataset is estimated to take around 60 hours even with data optimization...\n\nRegardless the time estimation, I have finally a first model that looks nice. Let see how was the loss and accuracy for this training","metadata":{}},{"cell_type":"code","source":"def plotTrainingMetrics(history, epochs):\n    accuracy = history.history['accuracy']\n    validation_accuracy = history.history['val_accuracy']\n    loss = history.history['loss']\n    validation_loss = history.history['val_loss']\n    epochs_range = range(epochs)\n    matplotlib.pyplot.figure(figsize=(8,8))\n    matplotlib.pyplot.subplot(1, 2, 1)\n    matplotlib.pyplot.plot(epochs_range, accuracy, label=\"training accuracy\")\n    matplotlib.pyplot.plot(epochs_range, validation_accuracy, label=\"validation accuracy\")\n    matplotlib.pyplot.legend(loc='lower right')\n    matplotlib.pyplot.title(\"training and validation accuracy\")\n    matplotlib.pyplot.subplot(1, 2, 2)\n    matplotlib.pyplot.plot(epochs_range, loss, label=\"training loss\")\n    matplotlib.pyplot.plot(epochs_range, validation_loss, label=\"validation loss\")\n    matplotlib.pyplot.legend(loc='upper right')\n    matplotlib.pyplot.title(\"training and validation loss\")\n\nplotTrainingMetrics(history, epochs)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:29:39.108944Z","iopub.execute_input":"2023-02-22T21:29:39.109327Z","iopub.status.idle":"2023-02-22T21:29:39.482572Z","shell.execute_reply.started":"2023-02-22T21:29:39.109295Z","shell.execute_reply":"2023-02-22T21:29:39.481623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And we see not bad but still room for improvement. So were to tackle? First, if you remember from the PyTorch version I was doing not just rescaling in data type but also transforming the image to have specific size. That is what we can leverage here and add some __data_augmentation__() into our solution","metadata":{}},{"cell_type":"code","source":"# Define some layers for augmentation\ndata_augmentation = tensorflow.keras.Sequential([\n    tensorflow.keras.layers.RandomFlip(\"horizontal\", input_shape=(image_width, image_height, 3)),\n    tensorflow.keras.layers.RandomRotation(0.1),\n    tensorflow.keras.layers.RandomZoom(0.1)\n])","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:26:32.624228Z","iopub.execute_input":"2023-02-22T21:26:32.624650Z","iopub.status.idle":"2023-02-22T21:26:32.826400Z","shell.execute_reply.started":"2023-02-22T21:26:32.624606Z","shell.execute_reply":"2023-02-22T21:26:32.825460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let show some of those image augmentation\nfor images, _ in train_dataset.take(1):\n  for i in range(9):\n    augmented_images = data_augmentation(images)\n    matplotlib.pyplot.subplot(3, 3, i + 1)\n    matplotlib.pyplot.imshow(augmented_images[0].numpy().astype(\"uint8\"))\n    matplotlib.pyplot.axis(\"off\")","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:26:34.742452Z","iopub.execute_input":"2023-02-22T21:26:34.743544Z","iopub.status.idle":"2023-02-22T21:26:35.576348Z","shell.execute_reply.started":"2023-02-22T21:26:34.743484Z","shell.execute_reply":"2023-02-22T21:26:35.575370Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# And now we can create an improved network\n# Notice we have our data augmentation layers and we added a Dropout layer to avoid overfitting\nnum_classes = len(class_names)\ndef create_improved_model(num_classes):\n    model = tensorflow.keras.models.Sequential([\n        data_augmentation,\n        tensorflow.keras.layers.Rescaling(1.0 / 255),\n        tensorflow.keras.layers.Conv2D(16, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Conv2D(32, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Conv2D(64, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Dropout(0.2),\n        tensorflow.keras.layers.Flatten(),\n        tensorflow.keras.layers.Dense(128, activation=\"relu\"),\n        tensorflow.keras.layers.Dense(num_classes)\n    ])\n    model.compile(\n        optimizer=\"adam\", \n        loss=tensorflow.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n        metrics=['accuracy']\n    )\n    return model\n\nimproved_model = create_improved_model(num_classes)\nimproved_model.summary()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:29:47.681097Z","iopub.execute_input":"2023-02-22T21:29:47.681483Z","iopub.status.idle":"2023-02-22T21:29:47.926366Z","shell.execute_reply.started":"2023-02-22T21:29:47.681450Z","shell.execute_reply":"2023-02-22T21:29:47.925592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# And do training again but now for the improved model\nepochs = 15\nimproved_history = improved_model.fit(\n    train_dataset,\n    validation_data=validation_dataset,\n    epochs=epochs,\n    verbose=1\n)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:29:52.436498Z","iopub.execute_input":"2023-02-22T21:29:52.437553Z","iopub.status.idle":"2023-02-22T21:31:03.323969Z","shell.execute_reply.started":"2023-02-22T21:29:52.437488Z","shell.execute_reply":"2023-02-22T21:31:03.322853Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's see metric this time\nplotTrainingMetrics(improved_history, epochs)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:31:10.418963Z","iopub.execute_input":"2023-02-22T21:31:10.420183Z","iopub.status.idle":"2023-02-22T21:31:11.142428Z","shell.execute_reply.started":"2023-02-22T21:31:10.420137Z","shell.execute_reply":"2023-02-22T21:31:11.141477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Much better, now we have a model that seems fairly good so is time to start checking with the test images","metadata":{}},{"cell_type":"code","source":"# Show test images to have a visual cue of what the network model should be outputing\nimport skimage.io\ntestimagefilename1 = os.path.join(datadir, \"test_images/85f8cb619c66b863.jpg\")\ntestimagefilename2 = os.path.join(datadir, \"test_images/ad8770db05586b59.jpg\")\ntestimagefilename3 = os.path.join(datadir, \"test_images/c7b03e718489f3ca.jpg\")\ntestimage1 = skimage.io.imread(testimagefilename1)\ntestimage2 = skimage.io.imread(testimagefilename2)\ntestimage3 = skimage.io.imread(testimagefilename3)\nmatplotlib.pyplot.figure(figsize=(20,20))\nmatplotlib.pyplot.subplot(1, 3, 1)\nmatplotlib.pyplot.imshow(testimage1)\nmatplotlib.pyplot.axis(\"off\")\nmatplotlib.pyplot.subplot(1, 3, 2)\nmatplotlib.pyplot.imshow(testimage2)\nmatplotlib.pyplot.axis(\"off\")\nmatplotlib.pyplot.subplot(1, 3, 3)\nmatplotlib.pyplot.imshow(testimage3)\nmatplotlib.pyplot.axis(\"off\")\nmatplotlib.pyplot.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:31:23.327297Z","iopub.execute_input":"2023-02-22T21:31:23.328279Z","iopub.status.idle":"2023-02-22T21:31:29.685925Z","shell.execute_reply.started":"2023-02-22T21:31:23.328241Z","shell.execute_reply":"2023-02-22T21:31:29.683476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So as human I can see the test images seems to have rust, complex, and frog_eye_leaf_spot as the diseases. Let's see if our model is capable to catch those one","metadata":{}},{"cell_type":"code","source":"def consoleLogPredictionsUsingModel(model, model_name):\n    test_images_directory = os.path.join(datadir, \"test_images/\")\n    for test_image_file in os.listdir(test_images_directory):\n        test_image_path = os.path.join(test_images_directory, test_image_file)\n        image = tensorflow.keras.utils.load_img(test_image_path, target_size=(image_width, image_height))\n        image_array = tensorflow.keras.utils.img_to_array(image)\n        image_array = tensorflow.expand_dims(image_array, 0)\n        predictions = model.predict(image_array)\n        score = tensorflow.nn.softmax(predictions[0])\n        print(\"Score predictions (using {}) for file {}\".format(model_name, test_image_file))\n        string_to_show = \"\"\n        for index,value in enumerate(score):\n            string_to_show += \"{}: {:5.2f}, \".format(class_names[index], value * 100)\n        print(string_to_show[:-2])\n\nconsoleLogPredictionsUsingModel(create_model(num_classes), \"untrained_model\")\nprint()\nconsoleLogPredictionsUsingModel(model, \"naive_model\")\nprint()\nconsoleLogPredictionsUsingModel(improved_model, \"improved_model\")","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:31:34.405368Z","iopub.execute_input":"2023-02-22T21:31:34.405818Z","iopub.status.idle":"2023-02-22T21:31:36.627151Z","shell.execute_reply.started":"2023-02-22T21:31:34.405785Z","shell.execute_reply":"2023-02-22T21:31:36.626113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And voila, we have now a decent neural network that can detect plant disease. What else to be done?\n1. Well, I have to create code to save checkpoints of the model (what happens if the training breaks or the dataset is to big that can takes days to train or we found a mistake but we don't want to start from the begining?)\n2. I can improve the network architecture with more layers to tune and see how to get better accuracy | scores for my detector (Kaggle has code submitted by contestants and it looks that is what they are doing to get the best performance possible, there is also a family of network architectures that I can rely on via this webpage https://keras.io/examples/vision/)\n3. Profile the model, after all I have been doing everything is with CPU intention and in my personal computer takes forever.\n\nLet me finish this section with point 1 and starts next section with some of 3. Point 2 was originally in my scope for the assignment but I think showing how to analyze data and work with ML frameworks, like PyTorch and Tensorflow, is sufficient given all the work from previous cells","metadata":{}},{"cell_type":"code","source":"# Save weights in directory training_checkpoints_CPU\ndef trainNetworkAndSaveWeights(\n    model,\n    epochs, \n    train_dataset,\n    validation_dataset, \n    checkpoint_path, \n    training_batch_size, \n    frequency_of_saving\n    ):\n    checkpoint_save_callback = tensorflow.keras.callbacks.ModelCheckpoint(\n        filepath=checkpoint_path,\n        save_weights_only=True,\n        verbose=1,\n        save_freq=frequency_of_saving * training_batch_size\n        )\n    model.save_weights(checkpoint_path.format(epoch=0))\n    model.fit(\n        train_dataset,\n        validation_data=validation_dataset,\n        epochs=epochs,\n        batch_size=training_batch_size,\n        callbacks=[checkpoint_save_callback],\n        verbose=0\n    )\n\nepochs = 50\nfrequency_of_epoch_saving = 5\ndevice_used = tensorflow.test.gpu_device_name()\nif not device_used:\n    device_used = \"CPU\"\nelse:\n    device_used = \"GPU\"\ncheckpoint_directory_to_save_weights = \"training_checkpoints_\" + device_used + '/'\ncheckpoint_path = checkpoint_directory_to_save_weights + \"/cp-{epoch:04d}.ckpt\"\ntrainNetworkAndSaveWeights(\n    improved_model, \n    epochs,\n    train_dataset,\n    validation_dataset,\n    checkpoint_path,\n    training_dataset_num_images // batch_size,\n    frequency_of_epoch_saving)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# And now we can load the pretrained weights\ncheckpoint_dir = os.path.dirname(checkpoint_path)\nlatest_weights = tensorflow.train.latest_checkpoint(checkpoint_dir)\npretrained_model = create_improved_model(num_classes)\npretrained_model.load_weights(latest_weights)\nconsoleLogPredictionsUsingModel(pretrained_model, \"pretrained_model\")","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Profiling the model","metadata":{}},{"cell_type":"code","source":"# First check the devices the machine has (could be even a virtual machine)\n# My personal laptop has 8gb of RAM and an Intel GPU (meaning tensorflow would not run with gpu)\n# I am not sure if the ML frameworks support other GPU that are not Nvidia, probably not but have not\n# research about it and would be nice in the future if OpenCL or Compute Shaders make that work\n# Kaggle instance support 13gb of RAM and 2 GPUs (T4? My guess is those are Titans?)\nimport tensorflow\ntensorflow.config.list_physical_devices()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import datetime\ndef trainNetworkAndLogWithTensorBoard(model, epochs, train_dataset, validation_dataset):\n    logs = \"logs/\" + datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n    tensorboard_callback = tensorflow.keras.callbacks.TensorBoard(\n        log_dir=logs,\n        histogram_freq=1,\n        profile_batch='5,20'\n        )\n    model.fit(\n        train_dataset,\n        validation_data=validation_dataset,\n        epochs=epochs,\n        callbacks=[tensorboard_callback]\n    )\n\nmodel_to_profile = create_improved_model(num_classes)\ntrainNetworkAndLogWithTensorBoard(model_to_profile, 10, train_dataset, validation_dataset)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%load_ext tensorboard\n%tensorboard --logdir=logs","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict on test images and write output","metadata":{}},{"cell_type":"code","source":"# Last part of the project, we already have the code for doing prediction given\n# an image, we only need to output the score values into the required CSV format\n# One thing to notice is that if we recall we did all the network by separating the labels\n# into unique categories, well, I have to return back to possible have a space separated output\n# I cam think of 2 approaches for it\n# 1. Redo the Tensorflow stuff but not separating labels (which I still find it bias)\n# 2. The scores I will get from my network are going to be in percentages, and I can use such\n# to decide when to output only 1 label and when multiple labels.\nimport csv\ndef createCSVOutputFromPrediction(model):\n    test_images_directory = os.path.join(datadir, \"test_images/\")\n    output = []\n    for test_image_file in os.listdir(test_images_directory):\n        test_image_path = os.path.join(test_images_directory, test_image_file)\n        image = tensorflow.keras.utils.load_img(test_image_path, target_size=(image_width, image_height))\n        image_array = tensorflow.keras.utils.img_to_array(image)\n        image_array = tensorflow.expand_dims(image_array, 0)\n        predictions = model.predict(image_array, verbose=0)\n        scores = tensorflow.nn.softmax(predictions[0]) * 100\n        scores_with_indices = []\n        for index,value in enumerate(scores):\n            scores_with_indices.append((value.numpy(), index))\n        sorted_scores = sorted(scores_with_indices)\n        topscore = sorted_scores[-1][0]\n        if (topscore >= 70):\n            labels = class_names[sorted_scores[-1][1]]\n        elif (topscore >= 50 and topscore < 70):\n            firstlabel = class_names[sorted_scores[-1][1]]\n            secondlabel = class_names[sorted_scores[-2][1]]\n            if firstlabel == 'healthy':\n                labels = secondlabel\n            elif secondlabel == 'healthy':\n                labels = firstlabel\n            else:\n                labels = firstlabel + ' ' + secondlabel\n        elif (topscore >= 30 and topscore < 50):\n            firstlabel = class_names[sorted_scores[-1][1]]\n            secondlabel = class_names[sorted_scores[-2][1]]\n            thirdlabel = class_names[sorted_scores[-3][1]]\n            if firstlabel == 'healthy':\n                labels = secondlabel + ' ' + thirdlabel\n            elif secondlabel == 'healthy':\n                labels = firstlabel + ' ' + thirdlabel\n            elif thirdlabel == 'healthy':\n                labels = firstlabel + ' ' + secondlabel\n            else:\n                labels = firstlabel + ' ' + secondlabel + ' ' + thirdlabel\n        else:\n            labels = \"unknown\"\n        print(test_image_file, labels)\n        output.append([test_image_file, labels])\n    csvfiletowrite = \"plantdisease_classification_submission.csv\"\n    print(\"Writing separated labels in {}\".format(csvfiletowrite))\n    with open(csvfiletowrite, 'w', newline='') as csvfile:\n        csvwriter = csv.writer(csvfile)\n        csvwriter.writerow(['image', 'labels'])\n        csvwriter.writerows(output)\n\n# I had to add this one because tensorflow in Kaggle is 2.9 while in\n# my personal computer is 2.11 and writing the createCSVOutputFromPrediction\n# would create a compatibility issue\nnum_classes = len(class_names)\ndef create_improved_model_legacy(num_classes):\n    model = tensorflow.keras.models.Sequential([\n        data_augmentation,\n        tensorflow.keras.layers.Rescaling(1.0 / 255),\n        tensorflow.keras.layers.Conv2D(16, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Conv2D(32, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Conv2D(64, 3, padding=\"same\", activation=\"relu\"),\n        tensorflow.keras.layers.MaxPooling2D(),\n        tensorflow.keras.layers.Dropout(0.2),\n        tensorflow.keras.layers.Flatten(),\n        tensorflow.keras.layers.Dense(128, activation=\"relu\"),\n        tensorflow.keras.layers.Dense(num_classes)\n    ])\n    model.compile(\n        optimizer=tensorflow.keras.optimizers.legacy.Adam(),\n        loss=tensorflow.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n        metrics=['accuracy']\n    )\n    return model\n\ncheckpoint_directory_to_save_weights = \"/kaggle/input/pretrainedweightsallimagesresized/training_checkpoints/\"\ncheckpoint_path = checkpoint_directory_to_save_weights + \"/cp-{epoch:04d}.ckpt\"\ncheckpoint_dir = os.path.dirname(checkpoint_path)\nlatest_weights = tensorflow.train.latest_checkpoint(checkpoint_dir)\npretrained_model = create_improved_model_legacy(num_classes)\npretrained_model.load_weights(latest_weights)\ncreateCSVOutputFromPrediction(pretrained_model)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:40:41.415779Z","iopub.execute_input":"2023-02-22T21:40:41.416159Z","iopub.status.idle":"2023-02-22T21:40:42.335390Z","shell.execute_reply.started":"2023-02-22T21:40:41.416129Z","shell.execute_reply":"2023-02-22T21:40:42.334102Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Last thing to notice is that when we are fetching the scores we have some logic to discard the 'healthy' label if we found a case where the top score is less than 70% and any of the predicted labels is 'healthy'. This because it will be weird to have a leaf that has a disease and at the same time is healthy isn't? This is a crude approach for solving those prediction errors, other stuff I can think is\n1. Warn the user that the prediction result contains a healthy result thus the 'confidence' of the result is not what expected\n2. Return to analyze the data/redo the network to acquire better confidence","metadata":{}}]}