{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"The goal of this notebook was to make embeddings exploration as interactive as possible, learning [Bokeh](https://bokeh.org/) library in the process, which I found well suited for this task - it has good documentation, examples, it is flexible enough, and it allows to easily add client-side interactivity without using running python server. While I faced a few minor bugs working with it, my overall impression that it is a very nice plotting library.\n\nAs my focus was on visualization/interactivity, I do not create embeddings here, instead I base my work on [others](https://www.kaggle.com/konradb/product-embeddings) (thanks @konradb).\n\n### Challenges/known issues\n\nOne of challenges working on this notebook was how to include product thumbnails in it. Looks like there is no way to reference images from dataset directly, but what I found is that you can reference any file from notebook output files (/kaggle/working). While it does the job, it has some drawbacks:\n\n- you have to create 100K thumbnails on each notebook re-run, which takes time and space. Even if 128x128 thumbnails do not take that much space, the overall number of files is quite large, which causes slowdowns.\n- image urls seem to expire, so while you can open notebook and view images, you can't pregenerate thumbnails in another notebook once and then reference them in another notebook. Also if you open the notebook and keep it open for a while (a few hours) and then try to hover the plot, you might see that image links are broken - to fix this you'll need to reload the page.\n- it takes some time for server to respond (up to half a minute). Not because notebook itself is large, but because there are lots of tiny files\n\nSo I would like to hear if there better ways to include thumbnails into the notebook.\n\nAnother issue is that notebook output itself is quite large (currently about 40MB) as it includes almost all of `articles.csv` + embeddings (2d). It is possible to make progressive loading, extracting this data into separate file from notebook, which will make UX better, but I see this less critical problem comparing to the the first one, when you are stuck for half a minute with blank screen. Nevertheless, after the notebook is loaded it is quite fast and interactive despite 100K points thanks to Bokeh's webgl renderer.\n\n### TODO\n\n- Solve loading perf issues (✅ partially)\n- Add color legend (✅)\n- Add other embeddings\n\n### Updates \n\n*01.03.2022*
\nPartially solved loading performance issue by extracting articles.csv and embeddings to a separate file and loading it incrementally.\n\n*03.03.2022*
\nAdded a color legend.","metadata":{}},{"cell_type":"code","source":"!pip install opentsne -qq\n\nimport os\nimport umap\nimport colorcet\nimport itertools\nimport pandas as pd\nimport bokeh as bk\nimport bokeh.models as bkm\nimport bokeh.layouts as bkl\nimport bokeh.resources as bkr\nimport bokeh.transform as bkt\n\nfrom glob import glob\nfrom PIL import Image\nfrom openTSNE import TSNE\nfrom tqdm.auto import tqdm\nfrom multiprocessing import Pool\nfrom IPython.display import HTML\nfrom bokeh.plotting import output_notebook, show, figure\n\noutput_notebook(resources=bkr.INLINE, hide_banner=True)\n\ndisplay(HTML(\"\"\"\n\n\"\"\"))\n\nDEV = False\n\n# generate product image thumbnails\n\ndef resize(i):\n fname = os.path.basename(i)\n im = Image.open(i)\n im.thumbnail((128, 128))\n os.makedirs(f'thumbs', exist_ok=True)\n im.save(f'thumbs/{fname}', quality=85)\n\ndef make_thumbs(): \n images = glob('../input/h-and-m-personalized-fashion-recommendations/images/**/*.jpg')\n with Pool(4) as p:\n p.map(resize, images)\n \nif not DEV:\n make_thumbs()\n\n# load data & embeddings\n\narticles = pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/articles.csv')\nemb_shop = pd.read_csv('../input/product-embeddings/prodemb_shop_20.csv')\nemb_img = pd.read_csv('../input/product-embeddings/prodemb_img_128.csv')\nemb_img['article_id'] = emb_img['image_id'].str[-14:-4].astype('int')\nemb_img = emb_img.drop(columns='image_id').set_index('article_id')\n\n# project embedding into 2D\n\narticles_emb = articles\nemb_names = []\n\nemb_name = 'img_tsne_euclidean'\nreducer = TSNE(verbose=DEV, n_jobs=-1, metric='euclidean')\nemb2d = reducer.fit(emb_img.values)\nemb2d = pd.DataFrame(emb2d.astype('float32'), columns=[f'x_{emb_name}', f'y_{emb_name}'], index=emb_img.index)\narticles_emb = articles_emb.join(emb2d, on='article_id')\nemb_names.append(emb_name)\n\nif not DEV:\n emb_name = 'img_umap_euclidean'\n reducer = umap.UMAP(verbose=DEV, n_neighbors=30, metric='euclidean')\n emb2d = reducer.fit_transform(emb_img.values)\n emb2d = pd.DataFrame(emb2d.astype('float32'), columns=[f'x_{emb_name}', f'y_{emb_name}'], index=emb_img.index)\n articles_emb = articles_emb.join(emb2d, on='article_id')\n emb_names.append(emb_name)\n\n emb_name = 'img_tsne_cosine'\n reducer = TSNE(verbose=DEV, n_jobs=-1, metric='cosine')\n emb2d = reducer.fit(emb_img.values)\n emb2d = pd.DataFrame(emb2d.astype('float32'), columns=[f'x_{emb_name}', f'y_{emb_name}'], index=emb_img.index)\n articles_emb = articles_emb.join(emb2d, on='article_id')\n emb_names.append(emb_name)\n\n emb_name = 'img_umap_cosine'\n reducer = umap.UMAP(verbose=DEV, n_neighbors=30, metric='cosine')\n emb2d = reducer.fit_transform(emb_img.values)\n emb2d = pd.DataFrame(emb2d.astype('float32'), columns=[f'x_{emb_name}', f'y_{emb_name}'], index=emb_img.index)\n articles_emb = articles_emb.join(emb2d, on='article_id')\n emb_names.append(emb_name)\n \narticles_emb.index.name = 'index'\narticles_emb.to_csv('articles_emb.csv')\n\n# plot\n\np = figure(output_backend='webgl', sizing_mode='stretch_width', title='Product Embeddings', \n tools=['pan', 'box_zoom', 'wheel_zoom', 'reset'], active_scroll='wheel_zoom', name='figure')\ncolor_mapper = bkt.factor_cmap('product_type_name', palette=bk.palettes.Category10_10 * 10, \n factors=articles['product_type_name'].value_counts().index.tolist())\n\nif DEV:\n source = articles_emb.sample(frac=0.1)\nelse:\n source = articles_emb.head(0)\n \nscatter = p.circle(source=source.filter(regex='x_|y_|article_id|_name|detail_desc'), \n x=f'x_{emb_names[0]}', y=f'y_{emb_names[0]}', size=5, alpha=0.2, color=color_mapper)\n\np.title.align = 'center'\np.title.text_font_size='12pt'\np.toolbar.logo = None\n\n# configure controls\n\nroot_url = ''\nhover = bkm.HoverTool(attachment='above', tooltips=f\"\"\"\n
\n \n
\n @prod_name
\n Product group: @product_group_name
\n Product type: @product_type_name
\n Color group: @colour_group_name
\n @detail_desc\n
\n
\n\"\"\")\n\np.add_tools(hover)\np.select(bkm.WheelZoomTool).zoom_on_axis = False\n\nprogress_bar = bkm.Div(text=f'
articles 0/{len(articles_emb)}
', name='progress_bar', sizing_mode='stretch_width', style={'width': '100%'})\nlegend = bkm.Div(name='legend', sizing_mode='stretch_width', css_classes=['legend'], style={'width': '100%'})\n\nselect_emb = bkm.Select(title='Embedding', options=emb_names, margin=(70, 5, 5, 5))\nselect_emb.js_on_change('value', bkm.CustomJS(args={'p': p, 'scatter': scatter}, code=\"\"\"\n scatter.glyph.x.field = `x_${this.value}`;\n scatter.glyph.y.field = `y_${this.value}`;\n scatter.data_source.change.emit();\n p.reset.emit();\n\"\"\"))\n\nselect_color_dim = bkm.Select(title='Color dimension', name='color_dimension', value='product_type_name',\n options=[c for c in articles.columns if c.endswith('name') and c != 'prod_name'])\nselect_color_dim.js_on_change('value', bkm.CustomJS(args={'scatter': scatter, 'color_mapper': color_mapper['transform']}, code=\"\"\"\n const column = this.value;\n scatter.glyph.line_color.field = column;\n scatter.glyph.fill_color.field = column;\n color_mapper.factors = valueCounts(scatter.data_source.data[column]).map(x => x[0])\n color_mapper.change.emit();\n renderLegend();\n\"\"\"))\n\nselect_cmap = bkm.Select(title='Colormap', options=['bokeh', 'category10', 'category20', \n 'glasbey_bw', 'glasbey_category10', 'glasbey_dark', 'glasbey_light',\n 'spectral','set1', 'set2', 'set3'],\n name='palette_name', value='category10')\nselect_cmap.js_on_change('value', bkm.CustomJS(args={'color_mapper': color_mapper['transform']}, code=f\"\"\"\n const column = this.value;\n color_mapper.palette = palettes[this.value];\n color_mapper.change.emit();\n renderLegend();\n\"\"\"))\n\nslider_marker_size = bkm.Slider(title='Marker size', start=1, end=10, value=5, step=1)\nslider_marker_size.js_link('value', scatter.glyph, 'size')\n\nslider_marker_alpha = bkm.Slider(title='Marker alpha', start=0, end=1, value=0.2, step=0.1)\nslider_marker_alpha.js_link('value', scatter.glyph, 'fill_alpha')\nslider_marker_alpha.js_link('value', scatter.glyph, 'line_alpha')\n\nshow(bkl.row(bkl.column(select_emb, select_color_dim, select_cmap, \n slider_marker_size, slider_marker_alpha, legend, width=200), \n bkl.column(progress_bar, p, sizing_mode='stretch_width')))\n\n# palettes\n\ndisplay(HTML(f\"\"\"\n\n\"\"\"))\n\n# legend\n\ndisplay(HTML(\"\"\"\n\n\"\"\"))\n\n# incremental data loading (applicable only for rendered notebook)\n\nif not DEV:\n display(HTML(f\"\"\"\n \n \"\"\"))\n\n display(HTML(\"\"\"\n \n \"\"\"))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-02T22:16:58.483506Z","iopub.execute_input":"2022-03-02T22:16:58.483924Z","iopub.status.idle":"2022-03-02T22:21:57.631488Z","shell.execute_reply.started":"2022-03-02T22:16:58.483808Z","shell.execute_reply":"2022-03-02T22:21:57.630685Z"},"trusted":true},"execution_count":null,"outputs":[]}]}