{"metadata":{"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":21154,"databundleVersionId":1243559,"sourceType":"competition"}],"dockerImageVersionId":30648,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Overview and Initial Setup","metadata":{}},{"cell_type":"markdown","source":"When I took the fastai course (https://course.fast.ai/) one of the exercises I wanted to do was to take Jeremy Howard's Road to the Top example (https://www.kaggle.com/code/jhoward/first-steps-road-to-the-top-part-1) and use it on another set of images other than the Paddy Disease Classification images Jeremy used. That way I would gain a deeper understanding of the fastai structure. I thought this Kaggle knowledge competition would be a great way to do that, since there's no time pressure and it's an interesting set of data. \n\nThe biggest initial challenge was that this competition is designed to use TensorFlow and fastai is built on PyTorch. So the first thing I had to do was to convert the images from TFRecords files to regular JPEG files. Fortunately, Gaurav Ramse created a great notebook from the Cassava Leaf Disease Classification competition that provided a template to do this: https://www.kaggle.com/code/ramsegaurav/how-to-convert-tfrecord-to-jpg.\n\nThe other challenge I'm having is that for some reason I'm undable to submit the submission.csv file to Kaggle. I've tried numerous approaches and nothing has worked. If anyone can solve this, I would be very grateful. ","metadata":{}},{"cell_type":"markdown","source":"This is to install the Hugging Face PyTorch Image Models (timm) that were used in the fastai exercise. ","metadata":{}},{"cell_type":"code","source":"!pip install timm","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:57:31.086479Z","iopub.execute_input":"2024-02-19T07:57:31.087145Z","iopub.status.idle":"2024-02-19T07:57:44.958835Z","shell.execute_reply.started":"2024-02-19T07:57:31.087114Z","shell.execute_reply":"2024-02-19T07:57:44.957781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport tensorflow as tf\n\nfrom fastai.vision.all import *\nset_seed(42)","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.status.busy":"2024-02-19T07:57:44.960872Z","iopub.execute_input":"2024-02-19T07:57:44.961181Z","iopub.status.idle":"2024-02-19T07:57:44.994747Z","shell.execute_reply.started":"2024-02-19T07:57:44.961153Z","shell.execute_reply":"2024-02-19T07:57:44.993809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To set up the various records files needed to use Gaurav Ramse's conversion code. ","metadata":{}},{"cell_type":"code","source":"val_tfrecord_path = '/kaggle/input/tpu-getting-started/tfrecords-jpeg-512x512/val/'\ntrain_tfrecord_path = '/kaggle/input/tpu-getting-started/tfrecords-jpeg-512x512/train/'\ntest_tfrecord_path = '/kaggle/input/tpu-getting-started/tfrecords-jpeg-512x512/test/'\n\ntrain_images_from_tfrecord_path = '/kaggle/working/Train_Images/' \ntest_images_from_tfrecord_path = '/kaggle/working/Test_Images/' \n\nsubmission_file_path = '/kaggle/input/tpu-getting-started/'","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:57:44.995766Z","iopub.execute_input":"2024-02-19T07:57:44.996069Z","iopub.status.idle":"2024-02-19T07:57:45.000704Z","shell.execute_reply.started":"2024-02-19T07:57:44.996045Z","shell.execute_reply":"2024-02-19T07:57:44.999692Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Converting the Images from TFRecords","metadata":{}},{"cell_type":"markdown","source":"This is the primary code used to convert the images from TFRecords files back to JPEG files so that they can be used with fastai. To keep things simple I merged the validation images back into the test images and will use a 25% split later when I use the fastai data loader. ","metadata":{}},{"cell_type":"code","source":"# To create the required path for TRAIN images\n\nif os.path.exists(train_images_from_tfrecord_path):\n    print('Looks like already exists')\nelse:\n    os.makedirs(train_images_from_tfrecord_path)\n    print(f\"Great , You have created required folder at {train_images_from_tfrecord_path}\")\n    \n# Create a dictionary describing the features.\nimage_feature_description = {\n    'image': tf.io.FixedLenFeature([], tf.string),  \n    'id': tf.io.FixedLenFeature([], tf.string),\n    'class': tf.io.FixedLenFeature([], tf.int64),\n}\n\ndef _parse_image_function(example_proto):\n    return tf.io.parse_single_example(example_proto, image_feature_description)\n\n\n# To store the tfrecord filenames in variable, filename\ntrain_filename = [train_tfrecord_path + i for i in os.listdir(train_tfrecord_path)]\nval_filename = [val_tfrecord_path + i for i in os.listdir(val_tfrecord_path)]\ntrain_filename = train_filename + val_filename\n\n\ndf_to_store_imagename_and_info = pd.DataFrame()\nimage_name_to_df = []\nclass_name_to_df = []\n\n# To parse the tfrecords path to the TFrecordDataset function from tensorflow\nraw_image_dataset = tf.data.TFRecordDataset(filenames=train_filename)\n# To map the tfrecorf file in the shape, where it contains a dict with an array of image, \n#image_name, and target variable\nparsed_image_dataset = raw_image_dataset.map(_parse_image_function) \n\n#for image_features in tqdm(parsed_image_dataset):\nfor image_features in parsed_image_dataset:\n    image_raw = image_features['image'].numpy()        # accessing image array  \n    array = tf.io.decode_image(image_raw, dtype=tf.dtypes.uint8, expand_animations=True).numpy()\n    im = Image.fromarray(array)                        # converting as a image\n    image_path = train_images_from_tfrecord_path +str(image_features['id'].numpy(), 'utf8')+'.jpg'  \n    im.save(image_path)\n    \n    # To store the image path and target variable for later use \n    image_name_to_df.append(str(image_features['id'].numpy(), 'utf8'))\n    class_name_to_df.append(image_features['class'].numpy())\n    \ndf_to_store_imagename_and_info['id'] = image_name_to_df\ndf_to_store_imagename_and_info['label'] = class_name_to_df\n\nprint('Great you have converted from tfrecord to jpg format :)')","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:57:45.003704Z","iopub.execute_input":"2024-02-19T07:57:45.004403Z","iopub.status.idle":"2024-02-19T07:59:01.495195Z","shell.execute_reply.started":"2024-02-19T07:57:45.004367Z","shell.execute_reply":"2024-02-19T07:59:01.494141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To do the same work for the test images.","metadata":{}},{"cell_type":"code","source":"# To create the required path for the TEST images\nif os.path.exists(test_images_from_tfrecord_path):\n    print('Looks like already exists')\nelse:\n    os.makedirs(test_images_from_tfrecord_path)\n    print(f\"Great , You have created required folder at {test_images_from_tfrecord_path}\")\n    \n# To create a dictionary describing the features.\ntest_image_feature_description = {\n    'image': tf.io.FixedLenFeature([], tf.string),  \n    'id': tf.io.FixedLenFeature([], tf.string),\n    \n}\n\ndef test_parse_image_function(example_proto):\n    return tf.io.parse_single_example(example_proto, test_image_feature_description)\n\n# To store the tfrecord filename in variable, filename\ntest_filename = [test_tfrecord_path + i for i in os.listdir(test_tfrecord_path)]\n\n\ntest_df_to_store_imagename_and_info = pd.DataFrame()\ntest_image_name_to_df = []\n\n\nraw_image_dataset = tf.data.TFRecordDataset(filenames=test_filename) \nparsed_image_dataset = raw_image_dataset.map(test_parse_image_function) \n\n\n#for image_features in tqdm(parsed_image_dataset):\nfor image_features in parsed_image_dataset:\n    image_raw = image_features['image'].numpy()        # accessing image array  \n    array = tf.io.decode_image(image_raw, dtype=tf.dtypes.uint8, expand_animations=True).numpy() \n    im = Image.fromarray(array)                        # converting as a image\n    image_path = test_images_from_tfrecord_path +str(image_features['id'].numpy(), 'utf8')+'.jpg'  \n    im.save(image_path)\n    \n    test_image_name_to_df.append(str(image_features['id'].numpy(), 'utf8'))\n    \ntest_df_to_store_imagename_and_info['id'] = test_image_name_to_df\n\nprint('Great you have converted from tfrecord to jpg format :)')","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:59:01.496979Z","iopub.execute_input":"2024-02-19T07:59:01.497277Z","iopub.status.idle":"2024-02-19T07:59:35.358993Z","shell.execute_reply.started":"2024-02-19T07:59:01.497252Z","shell.execute_reply":"2024-02-19T07:59:35.358064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To view the datafile for the training information. ","metadata":{}},{"cell_type":"code","source":"df_to_store_imagename_and_info","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:59:35.360265Z","iopub.execute_input":"2024-02-19T07:59:35.360664Z","iopub.status.idle":"2024-02-19T07:59:35.383053Z","shell.execute_reply.started":"2024-02-19T07:59:35.360628Z","shell.execute_reply":"2024-02-19T07:59:35.382168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_to_store_imagename_and_info.dtypes","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:59:35.384209Z","iopub.execute_input":"2024-02-19T07:59:35.384847Z","iopub.status.idle":"2024-02-19T07:59:35.391249Z","shell.execute_reply.started":"2024-02-19T07:59:35.384812Z","shell.execute_reply":"2024-02-19T07:59:35.390412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To make sure all the class numbers are in the datafile. ","metadata":{}},{"cell_type":"code","source":"df_to_store_imagename_and_info['label'].unique()","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:59:35.392534Z","iopub.execute_input":"2024-02-19T07:59:35.392859Z","iopub.status.idle":"2024-02-19T07:59:35.404712Z","shell.execute_reply.started":"2024-02-19T07:59:35.392829Z","shell.execute_reply":"2024-02-19T07:59:35.403743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Setting Up the fastai Model","metadata":{}},{"cell_type":"code","source":"train_path = '/kaggle/working/Train_Images/'   \nfiles = get_image_files(train_path)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:59:35.405791Z","iopub.execute_input":"2024-02-19T07:59:35.406096Z","iopub.status.idle":"2024-02-19T07:59:35.538944Z","shell.execute_reply.started":"2024-02-19T07:59:35.406070Z","shell.execute_reply":"2024-02-19T07:59:35.538245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To look at the image sizes. ","metadata":{}},{"cell_type":"code","source":"from fastcore.parallel import *\n\ndef f(o): return PILImage.create(o).size\nsizes = parallel(f, files, n_workers=8)\npd.Series(sizes).value_counts()","metadata":{"execution":{"iopub.status.busy":"2024-02-19T07:59:35.542103Z","iopub.execute_input":"2024-02-19T07:59:35.542373Z","iopub.status.idle":"2024-02-19T08:00:10.748418Z","shell.execute_reply.started":"2024-02-19T07:59:35.542341Z","shell.execute_reply":"2024-02-19T08:00:10.747386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the Paddy Disease Classification competition the data was labeled by separating each type of image in separately named folders. Therefore, Jeremy used the fastai from_folder dataloader. Since this competition has the images categorized in a matching datafile, I used the fastai from_df dataloader. In the exercise Jeremy used a 20% validation split, but since in this competition the validation images were about 25% of the total, I used a 25% validation split. \n\nRunning the resnet26d model Jeremy used with this dataset created a lot of strain on the CPUs. I had to move the batch size down to 4 to make it work. For comparison purposes I also used the resizing method and the size transformation from The Road To The Top example. \n\nIt can be seen from the sample images that were printed that the dataloader appeared to successfully match the label numbers to the actual images.","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_df(df_to_store_imagename_and_info, fn_col=0,\n                                folder=train_path, label_col=1, suff='.jpg', bs=4,\n                                valid_pct=0.25, seed=42,\n                               item_tfms=Resize(480, method='squish'), \n                               batch_tfms=aug_transforms(size=128, min_scale=0.75))\n\ndls.show_batch(max_n=6)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:00:10.749537Z","iopub.execute_input":"2024-02-19T08:00:10.749814Z","iopub.status.idle":"2024-02-19T08:00:13.287105Z","shell.execute_reply.started":"2024-02-19T08:00:10.749791Z","shell.execute_reply":"2024-02-19T08:00:13.286204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"One of the methods I researched to reduce the strain on the CPU from this dataset was to empty the torch cache. When I didn't do this I had some problems with cuda out of memory errors, even with a batch size as small as 4. ","metadata":{}},{"cell_type":"code","source":"import torch\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:00:13.288383Z","iopub.execute_input":"2024-02-19T08:00:13.288745Z","iopub.status.idle":"2024-02-19T08:00:13.295662Z","shell.execute_reply.started":"2024-02-19T08:00:13.288696Z","shell.execute_reply":"2024-02-19T08:00:13.294757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here's the simple process to create the model in fastai.","metadata":{}},{"cell_type":"code","source":"learn = vision_learner(dls, 'resnet26d', metrics=error_rate, path='.').to_fp16()","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:00:13.296744Z","iopub.execute_input":"2024-02-19T08:00:13.297018Z","iopub.status.idle":"2024-02-19T08:00:14.547832Z","shell.execute_reply.started":"2024-02-19T08:00:13.296995Z","shell.execute_reply":"2024-02-19T08:00:14.546886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This formula helps set the learning rate. ","metadata":{}},{"cell_type":"code","source":"learn.lr_find(suggest_funcs=(valley, slide))","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:00:14.549280Z","iopub.execute_input":"2024-02-19T08:00:14.549783Z","iopub.status.idle":"2024-02-19T08:00:21.953063Z","shell.execute_reply.started":"2024-02-19T08:00:14.549746Z","shell.execute_reply":"2024-02-19T08:00:21.952057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Finally, the formula to actually run the model with an inital 3 batches and a 0.01 learning rate. With the high loss values and error rate my guess is the Kaggle score on this wouldn't be very high. But the point of this exercise is to help get better familiarity with fastai. ","metadata":{}},{"cell_type":"code","source":"learn.fine_tune(3, 0.01)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:00:21.954655Z","iopub.execute_input":"2024-02-19T08:00:21.955398Z","iopub.status.idle":"2024-02-19T08:16:04.638498Z","shell.execute_reply.started":"2024-02-19T08:00:21.955338Z","shell.execute_reply":"2024-02-19T08:16:04.637534Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Creating the Submission File","metadata":{}},{"cell_type":"code","source":"submit = pd.read_csv('/kaggle/input/tpu-getting-started/sample_submission.csv')\nsubmit","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:04.639899Z","iopub.execute_input":"2024-02-19T08:16:04.640217Z","iopub.status.idle":"2024-02-19T08:16:04.675261Z","shell.execute_reply.started":"2024-02-19T08:16:04.640189Z","shell.execute_reply":"2024-02-19T08:16:04.674287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To pull up the actual images in the test files. ","metadata":{}},{"cell_type":"code","source":"test_files = get_image_files(test_images_from_tfrecord_path)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:04.685109Z","iopub.execute_input":"2024-02-19T08:16:04.685521Z","iopub.status.idle":"2024-02-19T08:16:04.756232Z","shell.execute_reply.started":"2024-02-19T08:16:04.685484Z","shell.execute_reply":"2024-02-19T08:16:04.755565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_files","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:04.757275Z","iopub.execute_input":"2024-02-19T08:16:04.757626Z","iopub.status.idle":"2024-02-19T08:16:04.763596Z","shell.execute_reply.started":"2024-02-19T08:16:04.757593Z","shell.execute_reply":"2024-02-19T08:16:04.762653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"While the submission file has a list of all the test file names, I found I needed to retrieve these actual test files in a path and then place there names in a new dataframe that I could use to run the fastai model for the test files. Otherwise I wasn't able to match what Jeremy Howard did in his fastai example. After some research I found I was able to use the Python Glob function to do this. ","metadata":{}},{"cell_type":"code","source":"import glob\ntest_images = glob.glob('/kaggle/working/Test_Images/*.jpg')\nfrom pathlib import Path\nfiles = test_images\ntest_images = [Path(file).stem for file in files]\ntesting_data = pd.DataFrame()\ntesting_data['id'] = test_images\ntesting_data","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:04.764593Z","iopub.execute_input":"2024-02-19T08:16:04.764857Z","iopub.status.idle":"2024-02-19T08:16:04.876287Z","shell.execute_reply.started":"2024-02-19T08:16:04.764834Z","shell.execute_reply":"2024-02-19T08:16:04.875375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To use fastai's test dataloader for the test data. ","metadata":{}},{"cell_type":"code","source":"test_dl = dls.test_dl(test_files)\ntest_dl","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:04.877176Z","iopub.execute_input":"2024-02-19T08:16:04.877436Z","iopub.status.idle":"2024-02-19T08:16:04.886090Z","shell.execute_reply.started":"2024-02-19T08:16:04.877413Z","shell.execute_reply":"2024-02-19T08:16:04.885214Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From Jeremy Howard in the Road to the Top example: \"We can now get the probabilities of each class, and the index of the most likely class, from this test set (the 2nd thing returned by get_preds are the targets, which are blank for a test set, so we discard them).\"","metadata":{}},{"cell_type":"code","source":"probs,_,idxs = learn.get_preds(dl=test_dl, with_decoded=True)\nidxs","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:04.887182Z","iopub.execute_input":"2024-02-19T08:16:04.887472Z","iopub.status.idle":"2024-02-19T08:16:47.490378Z","shell.execute_reply.started":"2024-02-19T08:16:04.887448Z","shell.execute_reply":"2024-02-19T08:16:47.489240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From Jeremy Howard: \"We can create and apply this mapping using pandas.\"","metadata":{}},{"cell_type":"code","source":"results = pd.Series(idxs.numpy(), name=\"idxs\")\nresults","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:47.491808Z","iopub.execute_input":"2024-02-19T08:16:47.493077Z","iopub.status.idle":"2024-02-19T08:16:47.502440Z","shell.execute_reply.started":"2024-02-19T08:16:47.493043Z","shell.execute_reply":"2024-02-19T08:16:47.501475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now I can create the actual csv file that matches the testing dataframe that was created using Glob with the predictions using the fastai model. ","metadata":{}},{"cell_type":"code","source":"testing_data['label'] = results\ntesting_data","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:47.503733Z","iopub.execute_input":"2024-02-19T08:16:47.504080Z","iopub.status.idle":"2024-02-19T08:16:47.517302Z","shell.execute_reply.started":"2024-02-19T08:16:47.504047Z","shell.execute_reply":"2024-02-19T08:16:47.516268Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The order of the ids as pulled from the test data do not match the order of the ids in the submission file. So I had to align the two files. ","metadata":{}},{"cell_type":"code","source":"new_testing_data = testing_data.set_index('id')\nnew_testing_data = new_testing_data.reindex(index=submit['id'])\nnew_testing_data = new_testing_data.reset_index()\nnew_testing_data","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:16:47.518605Z","iopub.execute_input":"2024-02-19T08:16:47.519261Z","iopub.status.idle":"2024-02-19T08:16:47.537412Z","shell.execute_reply.started":"2024-02-19T08:16:47.519233Z","shell.execute_reply":"2024-02-19T08:16:47.536469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"new_testing_data.to_csv('/kaggle/working/submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2024-02-19T08:42:35.258392Z","iopub.execute_input":"2024-02-19T08:42:35.258855Z","iopub.status.idle":"2024-02-19T08:42:35.277109Z","shell.execute_reply.started":"2024-02-19T08:42:35.258824Z","shell.execute_reply":"2024-02-19T08:42:35.276123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As I mentioned at the start, Kaggle doesn't recognize the submission.csv file as existing. When I submit the file, I get a \"Successful\" message on the events tab, but there's no score listed. When I try to submit from the competition page it states that \"No Output Files found\". I can download the file to my computer and pull it up in Excel and it looks correct.","metadata":{}}]}