{ "id": 372927, "title": "Visualizing 3D and 4D Embeddings", "url": "/competitions/otto-recommender-system/discussion/372927", "author_name": "", "post_date": "2022-12-18T18:54:13.961145300Z", "votes": 17, "comment_count": 1, "views": 0, "content": "

So far, we have seen some 2D representations of event embeddings.
\nBut t-SNE and UMAP allow for more than two components. Below you can see my embedding matrix, represented in 3D space.
\n\"\"
\n3D embeddings are easy to imagine, but how about 4D embeddings?
\nLike in the 3D example, we can project each 4D point onto a 2D plane and plot. The trick is to rotate two dimensions around the x-axis and two dimensions around the y-axis.
\nThe result looks like this:
\n\"\"
\nIs this more useful than the 3D example? I don't think so, but it certainly looks interesting. Try to guess if the plot is rotating forward or backward.
\nIf you want to apply this to your own data, here is the code:

\n
\nlst = []\nnp.random.seed()\n i  ():\n    mean = np.random.random()\n    lst.append(np.random.normal(mean, , (, )))\n\nembedding_matrix = np.concatenate(lst, axis=)\n\n\n\nreducer = umap.UMAP(n_neighbors=, n_components=)\nembeddings = reducer.fit_transform(embedding_matrix)\n\n\nembeddings  -= embeddings.mean(axis=)\n\n\ncolor = embeddings[:, :] \ncolor = color - color.()\ncolor = color/color.()\n\n\nfile_names = []\nplt.figure(figsize = (, ))\ncounter=\n perc  tqdm(np.linspace(, *np.pi, num=)):\n    \n    emb_x = embeddings[:,]* np.sin(perc/) + embeddings[:,] * np.cos(perc/)\n    emb_y = embeddings[:,] * np.sin(perc) + embeddings[:,] * np.cos(perc)\n    plt.scatter(emb_x, emb_y, s=, alpha=, c=color)\n    \n    plt.ylim(-, )\n    plt.xlim(-, )\n    plt.savefig()\n    plt.clf()\n    file_names.append()\n    counter+=\n\n\npng_dir = \nimages = []\n file_name  file_names:\n    file_path = os.path.join(png_dir, file_name)\n    images.append(imageio.imread(file_path))\n\nimageio.mimsave(, images)\n
\n

The code should create a gif that looks something like this:
\n\"\"

", "messages": [ { "id": "2069238", "postDate": "12/18/2022 18:54:13", "content": "

So far, we have seen some 2D representations of event embeddings.
\nBut t-SNE and UMAP allow for more than two components. Below you can see my embedding matrix, represented in 3D space.
\n\"\"
\n3D embeddings are easy to imagine, but how about 4D embeddings?
\nLike in the 3D example, we can project each 4D point onto a 2D plane and plot. The trick is to rotate two dimensions around the x-axis and two dimensions around the y-axis.
\nThe result looks like this:
\n\"\"
\nIs this more useful than the 3D example? I don't think so, but it certainly looks interesting. Try to guess if the plot is rotating forward or backward.
\nIf you want to apply this to your own data, here is the code:

\n
\nlst = []\nnp.random.seed()\n i  ():\n    mean = np.random.random()\n    lst.append(np.random.normal(mean, , (, )))\n\nembedding_matrix = np.concatenate(lst, axis=)\n\n\n\nreducer = umap.UMAP(n_neighbors=, n_components=)\nembeddings = reducer.fit_transform(embedding_matrix)\n\n\nembeddings  -= embeddings.mean(axis=)\n\n\ncolor = embeddings[:, :] \ncolor = color - color.()\ncolor = color/color.()\n\n\nfile_names = []\nplt.figure(figsize = (, ))\ncounter=\n perc  tqdm(np.linspace(, *np.pi, num=)):\n    \n    emb_x = embeddings[:,]* np.sin(perc/) + embeddings[:,] * np.cos(perc/)\n    emb_y = embeddings[:,] * np.sin(perc) + embeddings[:,] * np.cos(perc)\n    plt.scatter(emb_x, emb_y, s=, alpha=, c=color)\n    \n    plt.ylim(-, )\n    plt.xlim(-, )\n    plt.savefig()\n    plt.clf()\n    file_names.append()\n    counter+=\n\n\npng_dir = \nimages = []\n file_name  file_names:\n    file_path = os.path.join(png_dir, file_name)\n    images.append(imageio.imread(file_path))\n\nimageio.mimsave(, images)\n
\n

The code should create a gif that looks something like this:
\n\"\"

", "rawMarkdown": "So far, we have seen some 2D representations of event embeddings. \nBut t-SNE and UMAP allow for more than two components. Below you can see my embedding matrix, represented in 3D space.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10631026%2F1b9eb1bcb6a97518df68a7f2c82f72c6%2Fscatter3D.gif?generation=1671379579494032&alt=media)\n3D embeddings are easy to imagine, but how about 4D embeddings?\nLike in the 3D example, we can project each 4D point onto a 2D plane and plot. The trick is to rotate two dimensions around the x-axis and two dimensions around the y-axis.\nThe result looks like this:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10631026%2F0fc0489e65a65fbae6fd44e84bfe090a%2Fscatter4D.gif?generation=1671379491212385&alt=media)\nIs this more useful than the 3D example? I don't think so, but it certainly looks interesting. Try to guess if the plot is rotating forward or backward.\nIf you want to apply this to your own data, here is the code:\n```python\n# create random data to represent embeddings\nlst = []\nnp.random.seed(0)\nfor i in range(20):\n mean = np.random.random()\n lst.append(np.random.normal(mean, 0.02, (200, 10)))\n\nembedding_matrix = np.concatenate(lst, axis=0)\n# embedding_matrix = your data here\n\n# reduce embedding dimensionality\nreducer = umap.UMAP(n_neighbors=15, n_components=4)\nembeddings = reducer.fit_transform(embedding_matrix)\n\n# center embeddings\nembeddings -= embeddings.mean(axis=0)\n\n# add some colors based on the source data (optional)\ncolor = embeddings[:, 0:3] # or use embedding_matrix[:, 0:3]\ncolor = color - color.min()\ncolor = color/color.max()\n\n# plot and save individual projections\nfile_names = []\nplt.figure(figsize = (16, 13))\ncounter=0\nfor perc in tqdm(np.linspace(0, 4*np.pi, num=100)):\n # project each 4d point for plotting\n emb_x = embeddings[:,0]* np.sin(perc/2) + embeddings[:,1] * np.cos(perc/2)\n emb_y = embeddings[:,2] * np.sin(perc) + embeddings[:,3] * np.cos(perc)\n plt.scatter(emb_x, emb_y, s=3, alpha=0.8, c=color)\n # adjust lim for your data\n plt.ylim(-13, 13)\n plt.xlim(-16, 16)\n plt.savefig(f'image_{counter}.png')\n plt.clf()\n file_names.append(f'image_{counter}.png')\n counter+=1\n\n# load all images again and create a gif\npng_dir = '/kaggle/working/'\nimages = []\nfor file_name in file_names:\n file_path = os.path.join(png_dir, file_name)\n images.append(imageio.imread(file_path))\n\nimageio.mimsave('scatter.gif', images)\n```\nThe code should create a gif that looks something like this:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10631026%2F6734798e9406ada4fe166fb474dcf424%2Fscatter_example.gif?generation=1671387798226762&alt=media)", "votes": null }, { "id": "2074594", "postDate": "12/24/2022 11:30:49", "content": "

Its really amazing!!

", "rawMarkdown": "Its really amazing!!", "votes": null } ], "comments": [ { "id": 2074594, "author_name": "devharal", "author_url": "", "post_date": "12/24/2022 11:30:49", "content": "

Its really amazing!!

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2069238": "So far, we have seen some 2D representations of event embeddings. \nBut t-SNE and UMAP allow for more than two components. Below you can see my embedding matrix, represented in 3D space.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10631026%2F1b9eb1bcb6a97518df68a7f2c82f72c6%2Fscatter3D.gif?generation=1671379579494032&alt=media)\n3D embeddings are easy to imagine, but how about 4D embeddings?\nLike in the 3D example, we can project each 4D point onto a 2D plane and plot. The trick is to rotate two dimensions around the x-axis and two dimensions around the y-axis.\nThe result looks like this:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10631026%2F0fc0489e65a65fbae6fd44e84bfe090a%2Fscatter4D.gif?generation=1671379491212385&alt=media)\nIs this more useful than the 3D example? I don't think so, but it certainly looks interesting. Try to guess if the plot is rotating forward or backward.\nIf you want to apply this to your own data, here is the code:\n```python\n# create random data to represent embeddings\nlst = []\nnp.random.seed(0)\nfor i in range(20):\n mean = np.random.random()\n lst.append(np.random.normal(mean, 0.02, (200, 10)))\n\nembedding_matrix = np.concatenate(lst, axis=0)\n# embedding_matrix = your data here\n\n# reduce embedding dimensionality\nreducer = umap.UMAP(n_neighbors=15, n_components=4)\nembeddings = reducer.fit_transform(embedding_matrix)\n\n# center embeddings\nembeddings -= embeddings.mean(axis=0)\n\n# add some colors based on the source data (optional)\ncolor = embeddings[:, 0:3] # or use embedding_matrix[:, 0:3]\ncolor = color - color.min()\ncolor = color/color.max()\n\n# plot and save individual projections\nfile_names = []\nplt.figure(figsize = (16, 13))\ncounter=0\nfor perc in tqdm(np.linspace(0, 4*np.pi, num=100)):\n # project each 4d point for plotting\n emb_x = embeddings[:,0]* np.sin(perc/2) + embeddings[:,1] * np.cos(perc/2)\n emb_y = embeddings[:,2] * np.sin(perc) + embeddings[:,3] * np.cos(perc)\n plt.scatter(emb_x, emb_y, s=3, alpha=0.8, c=color)\n # adjust lim for your data\n plt.ylim(-13, 13)\n plt.xlim(-16, 16)\n plt.savefig(f'image_{counter}.png')\n plt.clf()\n file_names.append(f'image_{counter}.png')\n counter+=1\n\n# load all images again and create a gif\npng_dir = '/kaggle/working/'\nimages = []\nfor file_name in file_names:\n file_path = os.path.join(png_dir, file_name)\n images.append(imageio.imread(file_path))\n\nimageio.mimsave('scatter.gif', images)\n```\nThe code should create a gif that looks something like this:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10631026%2F6734798e9406ada4fe166fb474dcf424%2Fscatter_example.gif?generation=1671387798226762&alt=media)", "2074594": "Its really amazing!!" }, "source": "meta" }