{"cells":[{"metadata":{},"cell_type":"markdown","source":"This bit is from last year's challenge, when we still had to do the retrieval on our own. It has lost some of its relevance since the organizers are doing the retrieval during submission. But still it saves valuable GPU time when evaluating MAP scores while training.\n\n[FAISS](https://github.com/facebookresearch/faiss) from Facebook Research is a superfast package specialized for similarity search. The notebook will show how to do retrieval with faiss. It will also evaluate the difference in speeds between the scipy based search in the organizer's script and faiss based search and check how much time can you save in validation and training MAP score computations.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport cv2\nimport time\nimport numpy as np\nfrom keras.models import Model\nfrom keras.applications import VGG16,ResNet50\nfrom scipy.spatial import distance","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# a dummy model, good old resnet50\nmodel = ResNet50(weights='imagenet')\n\n# removing the final classification layer \nmodel = Model(inputs=[model.input], outputs=[model.layers[-2].output])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see how many images we have in our hand.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"index_files = []\nfor dirname, _, filenames in os.walk('../input/landmark-retrieval-2020/index/'):\n    for filename in filenames:\n        index_files.append(os.path.join(dirname, filename))\n\nquery_files = []\nfor dirname, _, filenames in os.walk('../input/landmark-retrieval-2020/test/'):\n    for filename in filenames:\n        query_files.append(os.path.join(dirname, filename))\n\nprint(\"We have a total of {} query images and a total of {} index images\".format(len(query_files), len(index_files)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Getting the embeddings from a subset of the query and index images","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"index_embeddings = []\nquery_embeddings = []\n\n# considering only the first 10,000 index images\nfor j,index_file in enumerate(index_files[:10000]):\n    im = cv2.imread(index_file)\n    im = cv2.resize(im, (224,224))\n    index_embedding = model.predict(np.array([im]))[0]\n    index_embeddings.append(index_embedding)\n\n# considering only the first 500 query images\nfor i,query_file in enumerate(query_files[:500]):\n    im = cv2.imread(query_file)\n    im = cv2.resize(im, (224,224))\n    query_embedding = model.predict(np.array([im]))[0]\n    query_embeddings.append(query_embedding)\n        ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Timing the scipy method used in organizer's script. Retrieving top 100 samples from the index images for each query image.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"start = time.time()\ndistances = distance.cdist(np.array(query_embeddings), np.array(index_embeddings), 'euclidean')\npredicted_positions = np.argpartition(distances, 100, axis=1)[:,:100]\n\nprint(predicted_positions.shape)\nprint(\"Time taken {} secs\".format(time.time() - start))\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Moving on to faiss!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# install\n!pip install faiss\n\nimport faiss                                 # make faiss available\nfaiss_index = faiss.IndexFlatL2(2048)        # build the index, need to input embedding size (last layer dimension of our model)\nprint(faiss_index.is_trained)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now timing. Retrieving top 100 samples from the index images for each query image.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# adding the index embeddings to faiss\nfaiss_index.add(np.array(index_embeddings))\n\n# check how many are added\nprint(\"total embeddings added\", faiss_index.ntotal) \n\n# now timing retrieval\nstart = time.time()\n_, I = faiss_index.search(np.array(query_embeddings), 100)\n    \nprint(I.shape)\nprint(\"Time taken {} secs\".format(time.time() - start))\n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So a whopping ~30x improvement which will only get bigger with increasing number of index and query images.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"This is taken from my last year's solutions which are available at https://github.com/mayukh18/Google-Landmark-Recognition-Retrieval-2019. Leave an upvote if you find these useful. :)","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}