{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Quick fastai v4 pipeline\n\nthis pipeline is to get you started with a quick baseline in fastai. Would love comments on improving it.\n\nCheck out fastbook chapter 5 (and 4 and 2) for more info on cnn_learner.\nhttps://github.com/fastai/fastbook"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# starter cell lets run it, cause why not.\n\n# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# kaggle already is already updated to fastai v2\n\nimport fastai\nfrom fastai.vision.all import *","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"Config.config_path\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"for those starting out (like me) you can look at the data by tappping the |< icon on the top right. You can also copy path directly from there."},{"metadata":{"trusted":true},"cell_type":"code","source":"input_images = \"../input/cassava-leaf-disease-classification\"\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!dir {input_images}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# lets look at an image\n# you can do it manually too\n\nim = Image.open(input_images + \"/train_images/1000015157.jpg\")\nim.to_thumb(256,256)\n\n# nice pic !","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN = input_images + \"/train_images\"\nTEST = input_images + \"/test_images\"\nLABELS = input_images + \"/train.csv\"\nSAMPLE_SUB = input_images + \"/sample_submission.csv\"\n\n# I really like it when the data input is uncomplicated as this\n# it really makes it accessible!\n# but i dont understand the tf records part\n# would be happy if someone clarified","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# reading the training csv\n\ntrain_df = pd.read_csv(LABELS)\ntrain_df.head()\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#print(train_df.__dict__)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# very helpful Dataloader from fastai\n# further transformation can be looked into\n# checkout chapter 5 of fastbook for reference\n\ndls = ImageDataLoaders.from_df(train_df, TRAIN, item_tfms=Resize(128), batch_tfms=aug_transforms(mult=2))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dls.show_batch()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fns = get_image_files(TEST)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#verify_images(fns)\n#just in case\n\n# you can use it if you are using dirty test images","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = cnn_learner(dls, resnet18, metrics=error_rate)\n#learn = cnn_learner(dls, resnet50, metrics=error_rate)\n\nlearn.fine_tune(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"interp = ClassificationInterpretation.from_learner(learn)\ninterp.plot_confusion_matrix()\n\n# lots a error here try \n# try using resnet50","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"interp.plot_top_losses(16,nrows=8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.export(fname='/kaggle/working/export.pkl')\n\n\n# this can be a chekpoint if you are running this notebook\n# we can load the model next time we come back","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# load the pickle file you exported previously\npath = Path()\npath.ls(file_exts='.pkl')\n\nlearn_inf = load_learner(path/'export.pkl')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# submitting on test data\n\n!dir {TEST}\n\n# hmm why is there just one test image here\n# sure looks fishy I guess real data is in tfrecords\n# need to come back and parse it","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# anyway\n# lets make the submission\n\nlearn_inf.predict(\"../input/cassava-leaf-disease-classification/test_images/2216849948.jpg\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"inference from https://www.kaggle.com/sd4321/latest-fastai-training-inference-prediction"},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\ntest_image_files = Path('../input/cassava-leaf-disease-classification/test_images')\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"image_data_loader_test = dls.test_dl(get_image_files(test_image_files))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\n_,_,results = learn.get_preds(dl = image_data_loader_test, with_decoded = True)\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\nsub = pd.read_csv(\"../input/cassava-leaf-disease-classification/sample_submission.csv\")\nsub.head()\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\nsub['label'] = results\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\nsub.head()\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.to_csv('my_submission_file.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is the end of quick baseline model.\n\n1. firstof all instead of using resnet18 you can change it to resnet 50 and train it. that will definitely bump the accuracy.\n\n1. and how to parse these tfrecords. somebody got any idea ?\n\n2. check out chapter 5 of fastbook to look into various augmentations you can try with this dataset.\n\n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}