{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-13T18:47:15.866860Z","iopub.execute_input":"2022-07-13T18:47:15.867357Z","iopub.status.idle":"2022-07-13T18:47:15.896794Z","shell.execute_reply.started":"2022-07-13T18:47:15.867261Z","shell.execute_reply":"2022-07-13T18:47:15.895582Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#ReadMe: This competition does not provide a training set. In accordance with the rules, any training data may be used, as long as it is disclosed in the forum by the relevant deadline.\n\nI didn't read just to confirm the stats that women don't read :).","metadata":{}},{"cell_type":"markdown","source":"![](https://opengraph.githubassets.com/b894d378a8465450410412c1049be92ad9e314e78ec3427525afb0d8083921ad/neo4j/neo4j/issues/12666)github.com","metadata":{}},{"cell_type":"markdown","source":"#All script by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook \n\n#Vote Queyrusi work, not mine.","metadata":{}},{"cell_type":"code","source":"# Lovely formatter\n!pip install nb_black > /dev/null\n%load_ext lab_black","metadata":{"execution":{"iopub.status.busy":"2022-07-13T18:47:50.479370Z","iopub.execute_input":"2022-07-13T18:47:50.479757Z","iopub.status.idle":"2022-07-13T18:48:03.999833Z","shell.execute_reply.started":"2022-07-13T18:47:50.479716Z","shell.execute_reply":"2022-07-13T18:48:03.998501Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -U image_embeddings","metadata":{"execution":{"iopub.status.busy":"2022-07-13T18:48:27.287192Z","iopub.execute_input":"2022-07-13T18:48:27.288099Z","iopub.status.idle":"2022-07-13T18:49:05.423051Z","shell.execute_reply.started":"2022-07-13T18:48:27.288058Z","shell.execute_reply":"2022-07-13T18:49:05.421728Z"},"_kg_hide-output":true,"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import image_embeddings","metadata":{"execution":{"iopub.status.busy":"2022-07-13T18:49:09.615743Z","iopub.execute_input":"2022-07-13T18:49:09.616124Z","iopub.status.idle":"2022-07-13T18:49:20.005884Z","shell.execute_reply.started":"2022-07-13T18:49:09.616093Z","shell.execute_reply":"2022-07-13T18:49:20.004810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Available datasets:\n\nNot all Datasets will work on those snippets. I was trying to get what Queyrusi made in his Embeddings: (EfficientNet - Elements of Image Embedding) Notebook.","metadata":{}},{"cell_type":"markdown","source":"Available datasets:\n\t- abstract_reasoning\n\t- accentdb\n\t- aeslc\n\t- aflw2k3d\n\t- ag_news_subset\n\t- ai2_arc\n\t- ai2_arc_with_ir\n\t- amazon_us_reviews\n\t- anli\n\t- arc\n\t- bair_robot_pushing_small\n\t- bccd\n\t- beans\n\t- big_patent\n\t- bigearthnet\n\t- billsum\n\t- binarized_mnist\n\t- binary_alpha_digits\n\t- blimp\n\t- bool_q\n\t- c4\n\t- caltech101\n\t- caltech_birds2010\n\t- caltech_birds2011\n\t- cars196\n\t- cassava\n\t- cats_vs_dogs\n\t- celeb_a\n\t- celeb_a_hq\n\t- cfq\n\t- cherry_blossoms\n\t- chexpert\n\t- cifar10\n\t- cifar100\n\t- cifar10_1\n\t- cifar10_corrupted\n\t- citrus_leaves\n\t- cityscapes\n\t- civil_comments\n\t- clevr\n\t- clic\n\t- clinc_oos\n\t- cmaterdb\n\t- cnn_dailymail\n\t- coco\n\t- coco_captions\n\t- coil100\n\t- colorectal_histology\n\t- colorectal_histology_large\n\t- common_voice\n\t- coqa\n\t- cos_e\n\t- cosmos_qa\n\t- covid19sum\n\t- crema_d\n\t- curated_breast_imaging_ddsm\n\t- cycle_gan\n\t- d4rl_mujoco_ant\n\t- d4rl_mujoco_halfcheetah\n\t- dart\n\t- davis\n\t- deep_weeds\n\t- definite_pronoun_resolution\n\t- dementiabank\n\t- diabetic_retinopathy_detection\n\t- div2k\n\t- dmlab\n\t- dolphin_number_word\n\t- downsampled_imagenet\n\t- drop\n\t- dsprites\n\t- dtd\n\t- duke_ultrasound\n\t- e2e_cleaned\n\t- efron_morris75\n\t- emnist\n\t- eraser_multi_rc\n\t- esnli\n\t- eurosat\n\t- fashion_mnist\n\t- flic\n\t- flores\n\t- food101\n\t- forest_fires\n\t- fuss\n\t- gap\n\t- geirhos_conflict_stimuli\n\t- gem\n\t- genomics_ood\n\t- german_credit_numeric\n\t- gigaword\n\t- glue\n\t- goemotions\n\t- gpt3\n\t- gref\n\t- groove\n\t- gtzan\n\t- gtzan_music_speech\n\t- hellaswag\n\t- higgs\n\t- horses_or_humans\n\t- howell\n\t- i_naturalist2017\n\t- imagenet2012\n\t- imagenet2012_corrupted\n\t- imagenet2012_real\n\t- imagenet2012_subset\n\t- imagenet_a\n\t- imagenet_r\n\t- imagenet_resized\n\t- imagenet_v2\n\t- imagenette\n\t- imagewang\n\t- imdb_reviews\n\t- irc_disentanglement\n\t- iris\n\t- kitti\n\t- kmnist\n\t- lambada\n\t- lfw\n\t- librispeech\n\t- librispeech_lm\n\t- libritts\n\t- ljspeech\n\t- lm1b\n\t- lost_and_found\n\t- lsun\n\t- lvis\n\t- malaria\n\t- math_dataset\n\t- mctaco\n\t- mlqa\n\t- mnist\n\t- mnist_corrupted\n\t- movie_lens\n\t- movie_rationales\n\t- movielens\n\t- moving_mnist\n\t- multi_news\n\t- multi_nli\n\t- multi_nli_mismatch\n\t- natural_questions\n\t- natural_questions_open\n\t- newsroom\n\t- nsynth\n\t- nyu_depth_v2\n\t- ogbg_molpcba\n\t- omniglot\n\t- open_images_challenge2019_detection\n\t- open_images_v4\n\t- openbookqa\n\t- opinion_abstracts\n\t- opinosis\n\t- opus\n\t- oxford_flowers102\n\t- oxford_iiit_pet\n\t- para_crawl\n\t- patch_camelyon\n\t- paws_wiki\n\t- paws_x_wiki\n\t- pet_finder\n\t- pg19\n\t- piqa\n\t- places365_small\n\t- plant_leaves\n\t- plant_village\n\t- plantae_k\n\t- qa4mre\n\t- qasc\n\t- quac\n\t- quickdraw_bitmap\n\t- race\n\t- radon\n\t- reddit\n\t- reddit_disentanglement\n\t- reddit_tifu\n\t- resisc45\n\t- robonet\n\t- rock_paper_scissors\n\t- rock_you\n\t- s3o4d\n\t- salient_span_wikipedia\n\t- samsum\n\t- savee\n\t- scan\n\t- scene_parse150\n\t- schema_guided_dialogue\n\t- scicite\n\t- scientific_papers\n\t- sentiment140\n\t- shapes3d\n\t- siscore\n\t- smallnorb\n\t- snli\n\t- so2sat\n\t- speech_commands\n\t- spoken_digit\n\t- squad\n\t- stanford_dogs\n\t- stanford_online_products\n\t- star_cfq\n\t- starcraft_video\n\t- stl10\n\t- story_cloze\n\t- sun397\n\t- super_glue\n\t- svhn_cropped\n\t- tao\n\t- ted_hrlr_translate\n\t- ted_multi_translate\n\t- tedlium\n\t- tf_flowers\n\t- the300w_lp\n\t- tiny_shakespeare\n\t- titanic\n\t- trec\n\t- trivia_qa\n\t- tydi_qa\n\t- uc_merced\n\t- ucf101\n\t- vctk\n\t- vgg_face2\n\t- visual_domain_decathlon\n\t- voc\n\t- voxceleb\n\t- voxforge\n\t- waymo_open_dataset\n\t- web_nlg\n\t- web_questions\n\t- wider_face\n\t- wiki40b\n\t- wiki_bio\n\t- wiki_table_questions\n\t- wiki_table_text\n\t- wikiann\n\t- wikihow\n\t- wikipedia\n\t- wikipedia_toxicity_subtypes\n\t- wine_quality\n\t- winogrande\n\t- wmt13_translate\n\t- wmt14_translate\n\t- wmt15_translate\n\t- wmt16_translate\n\t- wmt17_translate\n\t- wmt18_translate\n\t- wmt19_translate\n\t- wmt_t2t_translate\n\t- wmt_translate\n\t- wordnet\n\t- wsc273\n\t- xnli\n\t- xquad\n\t- xsum\n\t- xtreme_pawsx\n\t- xtreme_xnli\n\t- yelp_polarity_reviews\n\t- yes_no\n\t- youtube_vis","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook \n\n# Let's define some paths where to save images, tfrecords and embeddings\n# They will be stored on disk but we will keep the index in the RAM for retrieval speed.\nimport os\nfrom pathlib import Path\n\nBASEDIR = \"basedir\"\nos.makedirs(BASEDIR, exist_ok=True)\n\nDATASET = \"cassava\"\nos.makedirs(os.path.join(BASEDIR, DATASET), exist_ok=True)\n\nPATH_IMAGES = f\"{BASEDIR}/{DATASET}/images\"\nos.makedirs(PATH_IMAGES, exist_ok=True)\n\n# PATH_TFRECORDS = f\"{BASEDIR}/{DATASET}/tfrecords\" #It's a requirement for All Datasets!\n# os.makedirs(PATH_TFRECORDS, exist_ok=True) #It doesn't need to be commented\n\nPATH_EMBEDDINGS = f\"{BASEDIR}/{DATASET}/embeddings\"\nos.makedirs(PATH_EMBEDDINGS, exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T18:57:52.660406Z","iopub.execute_input":"2022-07-13T18:57:52.660757Z","iopub.status.idle":"2022-07-13T18:57:52.677991Z","shell.execute_reply.started":"2022-07-13T18:57:52.660728Z","shell.execute_reply":"2022-07-13T18:57:52.677228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nimage_embeddings.downloader.save_examples_to_folder(\n    output_folder=PATH_IMAGES, images_count=1000, dataset=DATASET\n)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T18:57:58.267198Z","iopub.execute_input":"2022-07-13T18:57:58.267541Z","iopub.status.idle":"2022-07-13T18:59:58.254508Z","shell.execute_reply.started":"2022-07-13T18:57:58.267514Z","shell.execute_reply":"2022-07-13T18:59:58.252892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Transform image to tf records\n\nTf record is an efficient format to store image, it's better to use than raw image file for inference\n\nBy Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nPATH_TFRECORDS = f\"{BASEDIR}/{DATASET}/tfrecords\"\nos.makedirs(PATH_TFRECORDS, exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:00:17.937575Z","iopub.execute_input":"2022-07-13T19:00:17.937969Z","iopub.status.idle":"2022-07-13T19:00:17.948509Z","shell.execute_reply.started":"2022-07-13T19:00:17.937939Z","shell.execute_reply":"2022-07-13T19:00:17.947597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Only 2 days ago, I learned/heard about shards for the 1st time. Now they are here! ","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nimage_embeddings.inference.write_tfrecord(\n    image_folder=PATH_IMAGES, output_folder=PATH_TFRECORDS, num_shards=10\n)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:00:24.720970Z","iopub.execute_input":"2022-07-13T19:00:24.721906Z","iopub.status.idle":"2022-07-13T19:00:27.173467Z","shell.execute_reply.started":"2022-07-13T19:00:24.721867Z","shell.execute_reply":"2022-07-13T19:00:27.172245Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Build embeddings\n\nHere, EfficientNet is used but any pretrained model would work. The input is tfrecords and the output is embeddings.\n\nBy Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nimage_embeddings.inference.run_inference(\n    tfrecords_folder=PATH_TFRECORDS, output_folder=PATH_EMBEDDINGS, batch_size=1000\n)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:01:18.503772Z","iopub.execute_input":"2022-07-13T19:01:18.504229Z","iopub.status.idle":"2022-07-13T19:01:46.369474Z","shell.execute_reply.started":"2022-07-13T19:01:18.504183Z","shell.execute_reply":"2022-07-13T19:01:46.368254Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Read the embeddings and build an index with it\n\nThe knn index is built using https://github.com/facebookresearch/faiss which makes it possible to search embeddings in  log(N) with lot of options to reduce memory footprint.\n\nBy Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\n[id_to_name, name_to_id, embeddings] = image_embeddings.knn.read_embeddings(\n    PATH_EMBEDDINGS\n)\nindex = image_embeddings.knn.build_index(embeddings)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:02:14.274914Z","iopub.execute_input":"2022-07-13T19:02:14.275326Z","iopub.status.idle":"2022-07-13T19:02:14.415932Z","shell.execute_reply.started":"2022-07-13T19:02:14.275294Z","shell.execute_reply":"2022-07-13T19:02:14.414625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Search in the index\n\nLet's pick a random product by id, retrieve its embedding and search in the index. We then display the results.\n\nBy Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\np = 121\nprint(id_to_name[p])\nimage_embeddings.knn.display_picture(PATH_IMAGES, id_to_name[p])\nresults = image_embeddings.knn.search(index, id_to_name, embeddings[p])\nimage_embeddings.knn.display_results(PATH_IMAGES, results)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:03:46.464945Z","iopub.execute_input":"2022-07-13T19:03:46.465865Z","iopub.status.idle":"2022-07-13T19:03:46.586512Z","shell.execute_reply.started":"2022-07-13T19:03:46.465824Z","shell.execute_reply":"2022-07-13T19:03:46.585240Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Combining queries\n\n\"Any vector in the same space can be used as query. For example I could have 2 image and want to find some example that are closeby to the 2, Let's just average them and see that happens!\"\n\nBy Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\np1 = 20\np2 = 70\nimage1 = id_to_name[p1]\nimage2 = id_to_name[p2]\nimage_embeddings.knn.display_picture(PATH_IMAGES, image1)\nimage_embeddings.knn.display_picture(PATH_IMAGES, image2)\nresults = image_embeddings.knn.search(\n    index, id_to_name, (embeddings[p1] + embeddings[p2]) / 2, 7\n)\nimage_embeddings.knn.display_results(PATH_IMAGES, results)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:04:34.698244Z","iopub.execute_input":"2022-07-13T19:04:34.698652Z","iopub.status.idle":"2022-07-13T19:04:34.847029Z","shell.execute_reply.started":"2022-07-13T19:04:34.698614Z","shell.execute_reply":"2022-07-13T19:04:34.845944Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Improve this is to normalize the embeddings to get a better mix.","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\ndef normalized(a, axis=-1, order=2):\n    l2 = np.atleast_1d(np.linalg.norm(a, order, axis))\n    l2[l2 == 0] = 1\n    return a / np.expand_dims(l2, axis)\n\n\nnormalized_embeddings = normalized(embeddings, 1)\nindex_normalized = image_embeddings.knn.build_index(normalized_embeddings)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:05:25.670795Z","iopub.execute_input":"2022-07-13T19:05:25.671296Z","iopub.status.idle":"2022-07-13T19:05:25.689207Z","shell.execute_reply.started":"2022-07-13T19:05:25.671257Z","shell.execute_reply":"2022-07-13T19:05:25.688290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\np1 = 20\np2 = 70\nimage1 = id_to_name[p1]\nimage2 = id_to_name[p2]\nimage_embeddings.knn.display_picture(PATH_IMAGES, image1)\nimage_embeddings.knn.display_picture(PATH_IMAGES, image2)\nresults = image_embeddings.knn.search(\n    index_normalized,\n    id_to_name,\n    (normalized_embeddings[p1] + normalized_embeddings[p2]) / 2,\n    7,\n)\nimage_embeddings.knn.display_results(PATH_IMAGES, results)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:06:05.524467Z","iopub.execute_input":"2022-07-13T19:06:05.525257Z","iopub.status.idle":"2022-07-13T19:06:05.661834Z","shell.execute_reply.started":"2022-07-13T19:06:05.525218Z","shell.execute_reply":"2022-07-13T19:06:05.660801Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Let's try on Horses vs. Humans Dataset","metadata":{}},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\n# Let's try another Dataset\n\nBASEDIR1 = \"basedir1\"\nos.makedirs(BASEDIR1, exist_ok=True)\n\nDATASET1 = \"horses_or_humans\"\nos.makedirs(os.path.join(BASEDIR1, DATASET1), exist_ok=True)\n\nPATH_IMAGES1 = f\"{BASEDIR1}/{DATASET1}/images\"\nos.makedirs(PATH_IMAGES1, exist_ok=True)\n\n# PATH_TFRECORDS = f\"{BASEDIR}/{DATASET}/tfrecords\"\n# os.makedirs(PATH_TFRECORDS, exist_ok=True)\n\nPATH_EMBEDDINGS1 = f\"{BASEDIR1}/{DATASET1}/embeddings\"\nos.makedirs(PATH_EMBEDDINGS1, exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:16:03.749671Z","iopub.execute_input":"2022-07-13T19:16:03.750138Z","iopub.status.idle":"2022-07-13T19:16:03.763022Z","shell.execute_reply.started":"2022-07-13T19:16:03.750100Z","shell.execute_reply":"2022-07-13T19:16:03.762095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nimage_embeddings.downloader.save_examples_to_folder(\n    output_folder=PATH_IMAGES1, images_count=1000, dataset=DATASET1\n)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:16:09.630460Z","iopub.execute_input":"2022-07-13T19:16:09.630832Z","iopub.status.idle":"2022-07-13T19:16:52.418626Z","shell.execute_reply.started":"2022-07-13T19:16:09.630803Z","shell.execute_reply":"2022-07-13T19:16:52.417134Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nPATH_TFRECORDS1 = f\"{BASEDIR1}/{DATASET1}/tfrecords\"\nos.makedirs(PATH_TFRECORDS1, exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:17:44.769534Z","iopub.execute_input":"2022-07-13T19:17:44.769940Z","iopub.status.idle":"2022-07-13T19:17:44.778637Z","shell.execute_reply.started":"2022-07-13T19:17:44.769910Z","shell.execute_reply":"2022-07-13T19:17:44.777486Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nimage_embeddings.inference.write_tfrecord(\n    image_folder=PATH_IMAGES1, output_folder=PATH_TFRECORDS1, num_shards=10\n)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:18:34.654813Z","iopub.execute_input":"2022-07-13T19:18:34.655693Z","iopub.status.idle":"2022-07-13T19:18:37.006024Z","shell.execute_reply.started":"2022-07-13T19:18:34.655639Z","shell.execute_reply":"2022-07-13T19:18:37.004650Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\nimage_embeddings.inference.run_inference(\n    tfrecords_folder=PATH_TFRECORDS1, output_folder=PATH_EMBEDDINGS1, batch_size=1000\n)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:19:56.517808Z","iopub.execute_input":"2022-07-13T19:19:56.518532Z","iopub.status.idle":"2022-07-13T19:20:16.758314Z","shell.execute_reply.started":"2022-07-13T19:19:56.518491Z","shell.execute_reply":"2022-07-13T19:20:16.757399Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\n[id_to_name, name_to_id, embeddings] = image_embeddings.knn.read_embeddings(\n    PATH_EMBEDDINGS1\n)\nindex = image_embeddings.knn.build_index(embeddings)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:20:47.589462Z","iopub.execute_input":"2022-07-13T19:20:47.589882Z","iopub.status.idle":"2022-07-13T19:20:47.662935Z","shell.execute_reply.started":"2022-07-13T19:20:47.589847Z","shell.execute_reply":"2022-07-13T19:20:47.661926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\np = 121\nprint(id_to_name[p])\nimage_embeddings.knn.display_picture(PATH_IMAGES1, id_to_name[p])\nresults = image_embeddings.knn.search(index, id_to_name, embeddings[p])\nimage_embeddings.knn.display_results(PATH_IMAGES1, results)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:21:27.434704Z","iopub.execute_input":"2022-07-13T19:21:27.435071Z","iopub.status.idle":"2022-07-13T19:21:27.539765Z","shell.execute_reply.started":"2022-07-13T19:21:27.435042Z","shell.execute_reply":"2022-07-13T19:21:27.538934Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\np = 113\nprint(id_to_name[p])\nimage_embeddings.knn.display_picture(PATH_IMAGES1, id_to_name[p])\nresults = image_embeddings.knn.search(index, id_to_name, embeddings[p])\nimage_embeddings.knn.display_results(PATH_IMAGES1, results)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:22:01.681435Z","iopub.execute_input":"2022-07-13T19:22:01.681862Z","iopub.status.idle":"2022-07-13T19:22:01.790673Z","shell.execute_reply.started":"2022-07-13T19:22:01.681821Z","shell.execute_reply":"2022-07-13T19:22:01.789561Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\np1 = 20\np2 = 70\nimage1 = id_to_name[p1]\nimage2 = id_to_name[p2]\nimage_embeddings.knn.display_picture(PATH_IMAGES1, image1)\nimage_embeddings.knn.display_picture(PATH_IMAGES1, image2)\nresults = image_embeddings.knn.search(\n    index, id_to_name, (embeddings[p1] + embeddings[p2]) / 2, 7\n)\nimage_embeddings.knn.display_results(PATH_IMAGES1, results)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:25:12.174845Z","iopub.execute_input":"2022-07-13T19:25:12.175326Z","iopub.status.idle":"2022-07-13T19:25:12.322704Z","shell.execute_reply.started":"2022-07-13T19:25:12.175287Z","shell.execute_reply":"2022-07-13T19:25:12.321598Z"},"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Both Horses and Humans doesn't seem real, just images alike.","metadata":{}},{"cell_type":"code","source":"def normalized(a, axis=-1, order=2):\n    l2 = np.atleast_1d(np.linalg.norm(a, order, axis))\n    l2[l2 == 0] = 1\n    return a / np.expand_dims(l2, axis)\n\n\nnormalized_embeddings = normalized(embeddings, 1)\nindex_normalized = image_embeddings.knn.build_index(normalized_embeddings)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:26:28.648680Z","iopub.execute_input":"2022-07-13T19:26:28.649903Z","iopub.status.idle":"2022-07-13T19:26:28.665414Z","shell.execute_reply.started":"2022-07-13T19:26:28.649853Z","shell.execute_reply":"2022-07-13T19:26:28.664364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook\n\np1 = 20\np2 = 70\nimage1 = id_to_name[p1]\nimage2 = id_to_name[p2]\nimage_embeddings.knn.display_picture(PATH_IMAGES1, image1)\nimage_embeddings.knn.display_picture(PATH_IMAGES1, image2)\nresults = image_embeddings.knn.search(\n    index_normalized,\n    id_to_name,\n    (normalized_embeddings[p1] + normalized_embeddings[p2]) / 2,\n    7,\n)\nimage_embeddings.knn.display_results(PATH_IMAGES1, results)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:27:04.935234Z","iopub.execute_input":"2022-07-13T19:27:04.935657Z","iopub.status.idle":"2022-07-13T19:27:05.071379Z","shell.execute_reply.started":"2022-07-13T19:27:04.935619Z","shell.execute_reply":"2022-07-13T19:27:05.070386Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Exporting the embeddings to numpy\n\n\"For easy access to the embeddings in other languages, we provide a function to export them to numpy.\"\n\nBy Queyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook","metadata":{}},{"cell_type":"code","source":"from image_embeddings.knn import embeddings_to_numpy","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:27:52.965206Z","iopub.execute_input":"2022-07-13T19:27:52.965969Z","iopub.status.idle":"2022-07-13T19:27:52.972186Z","shell.execute_reply.started":"2022-07-13T19:27:52.965924Z","shell.execute_reply":"2022-07-13T19:27:52.970911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path_embeddings_numpy = f\"{BASEDIR}/{DATASET}/embeddings_numpy\"\nembeddings_to_numpy(PATH_EMBEDDINGS, path_embeddings_numpy)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T19:28:16.309197Z","iopub.execute_input":"2022-07-13T19:28:16.309601Z","iopub.status.idle":"2022-07-13T19:28:16.376358Z","shell.execute_reply.started":"2022-07-13T19:28:16.309569Z","shell.execute_reply":"2022-07-13T19:28:16.375218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Till now, I didn't make any object detection, though I could show some images.","metadata":{}},{"cell_type":"markdown","source":"#Acknowledgemnts:\n\nQueyrusi https://www.kaggle.com/code/queyrusi/efficientnet-elements-of-image-embedding/notebook ","metadata":{}}]}