{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n#for dirname, _, filenames in os.walk('/kaggle/input'):\n #   for filename in filenames:\n  #      print(os.path.join(dirname, filename))\n\n# You can write up to 5GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"![](https://i.imgflip.com/d6d2z.jpg)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Have you ever gone through your vacation photos and asked yourself: What was the name of that temple I visited in China? or Who created this monument I saw in France? Landmark recognition can help! This technology can predict landmark labels directly from image pixels, to help people better understand and organize their photo collections.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Breakdown of Notebook\n\n1.  Import Libraries\n2.  Loading the dataset\n3.  EDA\n   *  Plot distribution of landmarks\n   *  Plot distribution of count of images\n   *  Barplot of top 50 frequent images\n   *  Number of images per class(log line)\n   *  Number of imges per classs scatter plot\n   *  Percentages of images in class(pie chart)\n4.  Train Data on Xception network\n5.  Test model on test data\n      ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Import all required libraries","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd \nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport plotly.figure_factory as ff\nimport plotly.graph_objects as go\nfrom scipy import stats\nimport cv2\nimport glob\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.applications import MobileNetV2\nfrom keras.utils import to_categorical\nfrom keras.layers import Dense\nfrom keras import Model\nfrom keras.callbacks import ModelCheckpoint\nfrom keras.models import load_model\nfrom tensorflow.keras.applications.xception import Xception\nfrom keras.layers import MaxPooling2D\nfrom keras.layers import GlobalAveragePooling2D\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That was a lot of work ,let's import all datasets required ","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df=pd.read_csv('../input/landmark-recognition-2020/train.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets take a look at the data","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[15,7])\nsns.distplot(train_df['landmark_id'])\nplt.xlabel('landmark_id')\nplt.title('distribution of lanmark')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"landmark_count=pd.value_counts(train_df[\"landmark_id\"])\nlandmark_count=landmark_count.reset_index()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"landmark_count.rename(columns={\"index\":'landmark_ids','landmark_id':'count'},inplace=True)\nlandmark_count","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[15,7])\nsns.distplot(landmark_count)\nplt.xlabel('landmark_id')\nplt.title('distribution of lanmark count')\nlandmark_count1=landmark_count.copy()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"\nsample = landmark_count[0:50]\nsample.rename(columns={\"index\":'landmark_ids','landmark_id':'count'},inplace=True)\nsample.sort_values(by=['count'],ascending=False,inplace=True)\nsample['landmark_ids']=sample['landmark_ids'].map(str)\nsample.info()\nsample","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"First fifty frequently occuring landmarks.","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[15,7])\nax=sns.barplot(x='landmark_ids',y='count',data=sample,order=sample['landmark_ids'],palette=sns.cubehelix_palette(50, start=9, rot=0, dark=0, light=.95, reverse=True))\nfor item in ax.get_xticklabels(): item.set_rotation(90)\n\nplt.xlabel('landmark_id')\nplt.ylabel('count of images')\nplt.title(\"Count of images per landmark_id\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since there is such a large differnce in count of landmark ids we'll need to represent it on lagarithmic scale to make sense of the distribution.","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"landmark_count1=landmark_count1.sort_values(by=['count'],ascending=False)\nfig=px.line(landmark_count1,y='count',hover_name=\"landmark_ids\",title=\"Number of images per class line\")\nfig.update_layout(yaxis_type=\"log\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig=px.scatter(landmark_count1,x='landmark_ids',y='count',title=\"Number of images per class scatter\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"its evident that most ids have a frequency below 300 hence we'll remove the top 70 rows of datafarame and plot againg to get a beeter resolution","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig=px.scatter(landmark_count1[70:],x='landmark_ids',y='count',title=\"Number of images per class scatter below 500\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sample=landmark_count1.loc[landmark_count1['count']<150]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig=px.scatter(sample,x='landmark_ids',y='count',title=\"Number of images per class scatter below 150\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sample=landmark_count1.loc[landmark_count1['count']<50]\nfig=px.scatter(sample,x='landmark_ids',y='count',title=\"Number of images per class scatter below 50\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we have a fair idea of how many number of images of a landmark is available","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"landmark_count1.loc[landmark_count1['count']<=10000,'landmark_ids']=\"below 10000 and above 500 images\"\nlandmark_count1.loc[landmark_count1['count']<=500,'landmark_ids']=\"below 500 and above 150 images\" \nlandmark_count1.loc[landmark_count1['count']<=150,'landmark_ids']=\"below 150 and above 50 images\"\nlandmark_count1.loc[landmark_count1['count']<=50,'landmark_ids']=\"below 50 images\"\n\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"landmark_count1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig=px.pie(landmark_count1,values='count',names='landmark_ids',title='Percentage of landmarks in classes')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we know majority of images have lesser than 50 samples","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_list = glob.glob('../input/landmark-recognition-2020/train/*/*/*/*')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"example = cv2.imread(train_list[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.imshow(example)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" we'll use the top 8000 most common landmarks for training","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sub = pd.read_csv(\"/kaggle/input/landmark-recognition-2020/sample_submission.csv\")\nsub[\"filename\"] = sub.id.str[0]+\"/\"+sub.id.str[1]+\"/\"+sub.id.str[2]+\"/\"+sub.id+\".jpg\"\nsub","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df[\"filename\"] = train_df.id.str[0]+\"/\"+train_df.id.str[1]+\"/\"+train_df.id.str[2]+\"/\"+train_df.id+\".jpg\"\ntrain_df[\"label\"] = train_df.landmark_id.astype(str)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from collections import Counter\n\nc = train_df.landmark_id.values\ncount = Counter(c).most_common(1000)\nprint(len(count), count[-1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# only keep 1000 classes\nkeep_labels = [i[0] for i in count]\ntrain_keep = train_df[train_df.landmark_id.isin(keep_labels)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"val_rate = 0.2\nbatch_size = 32","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"\ngen = ImageDataGenerator(validation_split=val_rate)\n\ntrain_gen = gen.flow_from_dataframe(\n    train_keep,\n    directory=\"/kaggle/input/landmark-recognition-2020/train/\",\n    x_col=\"filename\",\n    y_col=\"label\",\n    weight_col=None,\n    target_size=(299, 299),\n    color_mode=\"rgb\",\n    classes=None,\n    class_mode=\"categorical\",\n    batch_size=batch_size,\n    shuffle=True,\n    subset=\"training\",\n    interpolation=\"nearest\",\n    validate_filenames=False)\n    \nval_gen = gen.flow_from_dataframe(\n    train_keep,\n    directory=\"/kaggle/input/landmark-recognition-2020/train/\",\n    x_col=\"filename\",\n    y_col=\"label\",\n    weight_col=None,\n    target_size=(299, 299),\n    color_mode=\"rgb\",\n    classes=None,\n    class_mode=\"categorical\",\n    batch_size=batch_size,\n    shuffle=True,\n    subset=\"validation\",\n    interpolation=\"nearest\",\n    validate_filenames=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"weights_xce='../input/keras-pretrained-models/xception_weights_tf_dim_ordering_tf_kernels.h5'\nmodel  = Xception(weights=weights_xce)\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"model.compile(optimizer=\"adam\", loss=\"categorical_crossentropy\", metrics=[\"categorical_accuracy\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# training parameters\nepochs = 4 # maximum number of epochs\ntrain_steps = int(len(train_keep)*(1-val_rate))//batch_size\nval_steps = int(len(train_keep)*val_rate)//batch_size","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"model_checkpoint = ModelCheckpoint(\"best_model.h5\", save_best_only=True, verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"\nhistory = model.fit_generator(train_gen, steps_per_epoch=train_steps, epochs=epochs,validation_data=val_gen, validation_steps=val_steps, callbacks=[model_checkpoint])\n\nmodel.save(\"model.h5\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.models import load_model\nbest_model = load_model(\"model.h5\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"test_gen = ImageDataGenerator().flow_from_dataframe(\n    sub,\n    directory=\"/kaggle/input/landmark-recognition-2020/test/\",\n    x_col=\"filename\",\n    y_col=None,\n    weight_col=None,\n    target_size=(299, 299),\n    color_mode=\"rgb\",\n    classes=None,\n    class_mode=None,\n    batch_size=1,\n    shuffle=True,\n    subset=None,\n    interpolation=\"nearest\",\n    validate_filenames=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"print(\"Predicting on  available data   \")\ny_pred_one_hot = best_model.predict_generator(test_gen, verbose=1, steps=len(sub))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"y_pred = np.argmax(y_pred_one_hot, axis=-1)\ny_prob = np.max(y_pred_one_hot, axis=-1)\nprint(y_pred.shape, y_prob.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"y_uniq = np.unique(train_keep.landmark_id.values)\nprint(y_uniq)\ny_pred = [y_uniq[Y] for Y in y_pred]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"for i in range(len(sub)):\n    sub.loc[i, \"landmarks\"] = str(y_pred[i])+\" \"+str(y_prob[i])\nsub = sub.drop(columns=\"filename\")\nsub.to_csv(\"submission.csv\", index=False)\nsub","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### References\n\n* [Landmark Recognition Exploratory Data Analysis(EDA](https://www.kaggle.com/chirag9073/landmark-recognition-exploratory-data-analysis)\n\n* [Pre-trained MobileNetV2 (1000 classes, 1 epoch)](https://www.kaggle.com/socathie/pre-trained-mobilenetv2-1000-classes-1-epoch)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}