{"cells":[{"metadata":{},"cell_type":"markdown","source":"#  Google Landmark Retrieval 2020 - EDA\n\nThe purpose of this competition is to retrieve relevant database images to a given query image (ie, the model should retrieve database images containing the same landmark as the query).\n\nThe competition this year is different from previous two years in that it requires you to submit a model instead of the prediction file. From the organizer:\n\n> Your model must be named submission.zip and be compatible with TensorFlow 2.2. The submission.zip should contain all files and directories created by the tf.saved_model_save function using Tensorflow's SavedModel format.\n\nThis dataset is a large one and has [101.99 GB](https://www.kaggle.com/c/landmark-retrieval-2020/data). \n\n## Acknowledgement\n\nThis EDA kernel takes reference from the following excellent kernels:\n- https://www.kaggle.com/huangxiaoquan/google-landmarks-v2-exploratory-data-analysis-eda\n- https://www.kaggle.com/shivyshiv/exploratory-eda-for-google-landmark-retrieval-202\n- https://www.kaggle.com/seriousran/google-landmark-retrieval-2020-eda\n\n## Table of Content\n* [Import Packages](#import-packages)\n* [File Structure](#file-structure)\n* [Train File](#train-file)\n* [Landmark ID](#landmark-id)\n* [Pictures at Train, Test, and Index](#pic-train-test-index)\n* [Pictures by Class](#pic-by-class)\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"<a id=\"import-packages\"></a>\n## Import Packages","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import os\nimport glob\nimport math\nimport numpy as np \nimport pandas as pd \nimport seaborn as sns\nfrom scipy import stats\nimport matplotlib.pyplot as plt\nimport cv2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"file-structure\"></a>\n## File Structure\n\nThe dataset has one file (`train.csv`) and three folders (`train`, `test`, `index`) in the root path.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The `train`, `test`, and `index` folders each have a three-level hierarchy. The first three digits/letters of the file decides how the file is placed. From the organizer:\n\n> Each image has a unique id. Since there are a large number of images, each image is placed within three subfolders according to the first three characters of the image id (i.e. image abcdef.jpg is placed in a/b/c/abcdef.jpg).\n\nFor example, a picture named `00009069e8450638.jpg` will be placed at `0/0/0/00009069e8450638.jpg` since the first three digits/letters are `0`, `0`, and `0`. For another example, a picture named `0012f4bb3812e158.jpg` can be found at `0/0/1/0012f4bb3812e158.jpg` since the first three digits/letters are `0`, `0`, and `1`. \n\nThe file hierarchy is designed in this way probably because of the ease of searching; otherwise, we need to search in a folder consisting of 1 million images which would be slow.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/train/0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/train/0/0/","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From the following, each of the sub-folder in `train` consists of around 400 pictures. 16 folders * 16 folders * 16 folders * 400 images/folder is around 1.5 million images.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/train/0/0/0 | wc -l","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/train/0/0/1 | wc -l","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/train/9/9/f | wc -l","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"On the other hand, for `test` folder, there are not necessarily all folders from `0` to `f`. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/test/0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/landmark-retrieval-2020/test/1/0","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Last but not least, let's count the number of pictures in each main folder.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_list = glob.glob('../input/landmark-retrieval-2020/train/*/*/*/*')\ntest_list = glob.glob('../input/landmark-retrieval-2020/test/*/*/*/*')\nindex_list = glob.glob('../input/landmark-retrieval-2020/index/*/*/*/*')\nprint(\"There are {} images in train folder, {} images in test folder, and {} images in index folder.\".format(\n    len(train_list), len(test_list), len(index_list)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"train-file\"></a>\n## Train File (`train.csv`)\n\nWe don't have access to the meta-data of the testing set. Nevertheless, we know we are dealing with 1.5 million pictures. The two columns - `id` and `landmark_id` - are string and integers respectively. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_file_path = '../input/landmark-retrieval-2020/train.csv'\ndf_train = pd.read_csv(train_file_path)\n\nprint(\"Training data size:\", df_train.shape)\nprint(\"Training data columns: {}\\n\\n\".format(df_train.columns))\nprint(df_train.info())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.tail(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Great news - no N/A in `train.csv`. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"missing = df_train.isnull().sum()\npercent = missing/df_train.count()\nmissing_train_data = pd.concat([missing, percent], axis=1, keys=['Missing', 'Percent'])\nmissing_train_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"landmark-id\"></a>\n## Landmark ID \n\nFrom the following, there are 81313 classes of landmark. Please note that the landmark_id is not placed in a consecutive manner. Probably images for some of the IDs in between are skipped or taken out by the competition organizer.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Minimum of landmark_id: {}, maximum of landmark_id: {}\".format(df_train['landmark_id'].min(), df_train['landmark_id'].max()))\nprint(\"Number of unique landmark_id: {}\".format(len(df_train['landmark_id'].unique())))\nprint(df_train['landmark_id'].unique())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Consistent with the above finding, the maximum value of x-axis for the following plot is around 80,000 (i.e. there are around 80,000 unique landmark_id). From the shape of the plot, we are dealing with a very unbalanced dataset. We need to be careful on the training tactics and evaluation metric(s). ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set()\nplt.title('Training set: number of images per class(line plot)')\nsns.set_color_codes(\"pastel\")\nlandmarks_fold = pd.DataFrame(df_train['landmark_id'].value_counts())\nlandmarks_fold.reset_index(inplace=True)\nlandmarks_fold.columns = ['landmark_id','count']\nax = landmarks_fold['count'].plot(logy=True, grid=True)\nlocs, labels = plt.xticks()\nplt.setp(labels, rotation=30)\nax.set(xlabel=\"Landmarks\", ylabel=\"Number of images\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_count = pd.DataFrame(df_train.landmark_id.value_counts().sort_values(ascending=False))\ndf_count.reset_index(inplace=True)\ndf_count.columns = ['landmark_id', 'count']\ndf_count","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's take a look at the top 10 most appearing landmark_id. So the top landmark_id `138982` has more than 600 images, whereas the rest has around 1000-2000 images.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set()\nplt.figure(figsize=(9, 4))\nplt.title('Most frequent landmarks')\nsns.set_color_codes(\"pastel\")\nsns.barplot(\n    x=\"landmark_id\",\n    y=\"count\",\n    data=df_count.head(10),\n    label=\"Count\",\n    order=df_count.head(10).landmark_id)\nlocs, labels = plt.xticks()\nplt.setp(labels, rotation=45)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Following is the function to print images in a grid manner.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def print_img(class_id, df_class, figsize):\n    file_path = \"../input/landmark-retrieval-2020/train/\"\n    df = df_train[df_train['landmark_id'] == class_id].reset_index()\n    \n    print(\"Class {} - {}\".format(class_id, df_class[class_id].split(':')[-1]))\n    print(\"Number of images: {}\".format(len(df)))\n    \n    plt.rcParams[\"axes.grid\"] = False\n    no_row = math.ceil(min(len(df), 12)/3) \n    f, axarr = plt.subplots(no_row, 3, figsize=figsize)\n\n    curr_row = 0\n    len_img = min(12, len(df))\n    for i in range(len_img):\n        img_name = df['id'][i] + \".jpg\"\n        img_path = os.path.join(\n            file_path, img_name[0], img_name[1], img_name[2], img_name)\n        example = cv2.imread(img_path)\n        # uncomment the following if u wanna rotate the image\n        # example = cv2.rotate(example, cv2.ROTATE_180)\n        example = example[:,:,::-1]\n\n        col = i % 3\n        axarr[curr_row, col].imshow(example)\n        axarr[curr_row, col].set_title(\"{}. {} ({})\".format(\n            class_id, df_class[class_id].split(':')[-1], df['id'][i]))\n        if col == 2:\n            curr_row += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's get the name of each landmark category (credit goes to @sudeepshouche).","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# From: https://www.kaggle.com/sudeepshouche/identify-landmark-name-from-landmark-id\nurl = 'https://s3.amazonaws.com/google-landmark/metadata/train_label_to_category.csv'\ndf_class = pd.read_csv(url, index_col = 'landmark_id', encoding='latin', engine='python')['category'].to_dict()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Following are some sample pictures for `landmark_id = 138982`. The bottom of each picture may need to be removed for running any recognition algorithm during preprocessing. On the other hand, they might be helpful with text analysis methodologies. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 138982\nprint_img(class_id, df_class, (24, 18))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Following are some sample pictures for `landmark_id = 126637`. Unlike the above, there is no such thing as human hand-writing at the bottom. On the other hand, the category consists of a diverse topic of images such as plants and ships in different angles and orientation.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 126637\nprint_img(class_id, df_class, (24, 18))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Following this line of analysis, let's count the number images under a certain threshold count. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"threshold = [2, 3, 5, 10, 20, 50, 100, 200, 1000]\ntotal = len(df_train['landmark_id'].unique())\nfor num in threshold:\n    cnt = (df_train['landmark_id'].value_counts() < num).sum()\n    print(\"Number of classes with {} images or less: {}/{} ({:.2f}%)\".format(\n        num, cnt, total, cnt/total*100))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"pic-train-test-index\"></a>\n## Pictures at Train, Test, and Index\n\nLet's take a brief look at the pictures at Train, Test, and Index folders.\n\n### Train","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.rcParams[\"axes.grid\"] = False\nf, axarr = plt.subplots(4, 3, figsize=(24, 22))\n\ncurr_row = 0\nfor i in range(12):\n    example = cv2.imread(train_list[i])\n    # uncomment the following if u wanna rotate the image\n    # example = cv2.rotate(example, cv2.ROTATE_180)\n    example = example[:,:,::-1]\n    \n    col = i % 4\n    axarr[col, curr_row].imshow(example)\n    axarr[col, curr_row].set_title(train_list[i])\n    if col == 3:\n        curr_row += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Test","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.rcParams[\"axes.grid\"] = False\nf, axarr = plt.subplots(4, 3, figsize=(24, 22))\n\ncurr_row = 0\nfor i in range(12):\n    example = cv2.imread(test_list[i])\n    # uncomment the following if u wanna rotate the image\n    # example = cv2.rotate(example, cv2.ROTATE_180)\n    example = example[:,:,::-1]\n    \n    col = i % 4\n    axarr[col, curr_row].imshow(example)\n    axarr[col, curr_row].set_title(test_list[i])\n    if col == 3:\n        curr_row += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Index","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.rcParams[\"axes.grid\"] = False\nf, axarr = plt.subplots(4, 3, figsize=(24, 22))\n\ncurr_row = 0\nfor i in range(12):\n    example = cv2.imread(index_list[i])\n    # uncomment the following if u wanna rotate the image\n    # example = cv2.rotate(example, cv2.ROTATE_180)\n    example = example[:,:,::-1]\n    \n    col = i % 4\n    axarr[col, curr_row].imshow(example)\n    axarr[col, curr_row].set_title(index_list[i])\n    if col == 3:\n        curr_row += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"pic-by-class\"></a>\n## Pictures by Class","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 1\nprint_img(class_id, df_class, (24, 12))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 7\nprint_img(class_id, df_class, (24, 12))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 9\nprint_img(class_id, df_class, (24, 16))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 11\nprint_img(class_id, df_class, (24, 16))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 12\nprint_img(class_id, df_class, (24, 16))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class_id = 22\nprint_img(class_id, df_class, (24, 16))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To be continued .....","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}