{"cells":[{"metadata":{},"cell_type":"markdown","source":"![Keeley Dog 'Lets Classify Art!'](https://i.imgur.com/mzGMRuP.jpg)\n# iMet 2020 Multilabel Artwork Image Classification\n\nHello! \n\nSo here we have images of famous, and not so famous but still museum worthy, pieces of art in the Metropolitan Museum of Art's collection. \n\nLabeled, presumably by experts, with varied amounts of labels which have varied types of meanings. \n\nOur challenge is to create a multilabel image classification model. \n\nLets explore!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Table of contents\n* [Data Vizualization](#1)\n* [FastAI Multilabel Classification](#2)","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport functools\nimport re\nfrom fastai.vision import *\n\nimport seaborn as sns\nsns.set(style=\"whitegrid\")\nfrom collections import Counter\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# First we load the csv files\npath = Path('/kaggle/input/imet-2020-fgvc7/')\nlabels_df = pd.read_csv(path/'labels.csv')\ntrain_df = pd.read_csv(path/'train.csv')\ntest_df = pd.read_csv(path/'sample_submission.csv')\ntrain_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Visualization <a id=\"1\"></a>","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Visualizing Tag Counts","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Here we will visualize the number of tags per image\ntrain_df[\"tag_count\"] = train_df[\"attribute_ids\"].apply(lambda x:len(x.split(' ')))\nsns.countplot(x=\"tag_count\",data=train_df,palette=\"Reds\",log=True)\nplt.ylabel('Number of images')\nplt.xlabel('Tag Count')\nplt.title('Tag Count per Image')\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"TAG_COUNTS = train_df['tag_count'].value_counts().reset_index().sort_values(by=['index']).set_index('index').style.background_gradient(cmap=\"cividis\")\nTAG_COUNTS","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Split attribute types and values","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# We are now replacing the attribute_type::attrbiute_value format in attribute_name with the two columns seperated, and deleting the original \nlabels_df[\"attribute_type\"]=labels_df[\"attribute_name\"].apply(lambda x:x.split(\"::\")[0])\nlabels_df[\"attribute_value\"]=labels_df[\"attribute_name\"].apply(lambda x:x.split(\"::\")[1])\nlabels_df.drop(\"attribute_name\",1,inplace=True)\nlabels_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Display index ranges for each attribute type","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# See which different types of attributes we have\nunique_attributes = labels_df.attribute_type.unique()\nunique_attributes\n# Check range in indexes for each attribute type\nprint(\"Index Ranges For Each Attribute Type\")\nfor attribute in unique_attributes:\n    all_matches = labels_df.loc[labels_df['attribute_type']==attribute]\n    print(attribute, \": \",all_matches.min().attribute_id,\"-\",all_matches.max().attribute_id)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"![Keeley Dog 'Fastai is my favorite deep learning library!'](https://imgur.com/rCVKLA3.jpg)\n# FastAI Multilabel Classification <a id=\"2\"></a>","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Creating a Data Bunch from csv","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(69)\ntfms = get_transforms()\ndata = (ImageList.from_csv(path, 'train.csv', folder='train', suffix='.png')\n       .split_by_rand_pct(0.2)\n       .label_from_df(label_delim=' ', cols=\"attribute_ids\")\n       .transform(tfms, size=128)\n       .databunch(bs=32)\n       .normalize(imagenet_stats))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data.show_batch(3, figsize=(12,12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Creating a CNN with ResNet50","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"copy pretrained weights for resnet50 to the folder fastai will search by default\nPath('/root/.cache/torch/checkpoints/').mkdir(exist_ok=True, parents=True)\n!cp '../input/resnet50/resnet50.pth' '/root/.cache/torch/checkpoints/resnet50-19c8e357.pth'\narch = models.resnet50\nacc_02 = partial(accuracy_thresh,thresh=0.2)\nf_score = partial(fbeta,thresh=0.2)\nlearn=cnn_learner(data,arch,metrics=[acc_02,f_score], model_dir=\"/kaggle\", pretrained=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Find and set optimal learning rate","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()\nlearn.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Start training with Fit One Cycle","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lr = 0.01\nlearn.fit_one_cycle(5,slice(lr))\nlearn.save('stage-1-rn50')\nlearn.export('resnet50_imet')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### If we want to work with a pre-trained model we comment out the above, and load a model instead\nThis model was created previously by the learn.export() function and saved in our models dataset","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# learn = load_learner('/kaggle/input/models/', 'resnet50_imet_2_f0.399.pkl')\n# learn","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Add test dataset and get predictions","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.data.add_test(ImageList.from_df(test_df,path,folder='test',suffix='.png'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds,y = learn.get_preds(DatasetType.Test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.DataFrame(preds.numpy()).to_csv('preds.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"# Use this one when doing predictions on a new model\npreds_df = pd.DataFrame(preds.numpy())\n\n# Use this one to load predictions calculated from a past model\n# preds_df = pd.read_csv('/kaggle/input/imet-version17/preds.csv')\n# preds_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Display predictions for a given image","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def display_Predictions(image,display):\n    #Selects top 20 attributes, display scores\n    top20preds = preds_df.iloc[image].sort_values(ascending=False)[:20].reset_index()\n    top20preds = top20preds.rename(columns={\"index\":\"attribute_id\",image:\"acc_preds\"})\n    top20preds[\"attribute_type\"] = top20preds[\"attribute_id\"].apply(lambda x : labels_df.iloc[int(x)][\"attribute_type\"])\n    top20preds[\"attribute_value\"] = top20preds[\"attribute_id\"].apply(lambda x : labels_df.iloc[int(x)][\"attribute_value\"])\n    top20preds.reindex(columns=[\"attribute_id\",\"acc_preds\",\"attribute_type\",\"attribute_value\"])\n    if display:\n        image_path = path/'test'/(test_df.iloc[image].id+'.png')\n        img = plt.imread(str(image_path))\n        plt.imshow(img)\n    return top20preds\ndisplay_Predictions(11, display=True).style.background_gradient(cmap='cividis')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualize prediction percentages across all images","execution_count":null},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"for img_index in range(len(preds_df)):\n    values = display_Predictions(img_index,display=False)['acc_preds'].round(2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Convert prediction percentages to attributes and create new dataframe with them","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"thresh = 0.2\nlabelled_preds = [' '.join([learn.data.classes[i] for i,p in enumerate(pred) if p > thresh]) for pred in preds]\ndf = pd.DataFrame({'id':test_df['id'], 'attribute_ids':labelled_preds})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Output to submission.csv","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# All finished!","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}