{
  "id": 148324,
  "title": "Build embeddings dictionary without gensim from the embeddings zip file",
  "url": "/competitions/quora-insincere-questions-classification/discussion/148324",
  "author_name": "Chintan Gandhi",
  "post_date": "2020-05-03T22:57:37.295000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Here is a small code snippet that I referred from <a href=\"https://stackoverflow.com/a/60918920/6543250\">this post</a> on Stack Overflow that helps with reading a zip file without completely unzipping it.</p>\n\n<p>```\nimport io\nimport zipfile</p>\n\n<p>dim=300\nembedding_dict={}</p>\n\n<p>with zipfile.ZipFile(\"../input/quora-insincere-questions-classification/embeddings.zip\") as zf:\n    with io.TextIOWrapper(zf.open(\"glove.840B.300d/glove.840B.300d.txt\"), encoding=\"utf-8\") as f:\n        for line in tqdm(f):\n            values=line.split(' ') # \".split(' ')\" only for glove-840b-300d; for all other files, \".split()\" works\n            word=values[0]\n            vectors=np.asarray(values[1:],'float32')\n            embedding_dict[word]=vectors\n```</p>\n\n<p>I am positive that this code will work for other embeddings as well but is particularly helpful for reading GloVe embeddings. Hope this helps :)</p>",
  "messages": [
    {
      "id": 832160,
      "postDate": "2020-05-03T22:57:37.297Z",
      "content": "<p>Here is a small code snippet that I referred from <a href=\"https://stackoverflow.com/a/60918920/6543250\">this post</a> on Stack Overflow that helps with reading a zip file without completely unzipping it.</p>\n\n<p>```\nimport io\nimport zipfile</p>\n\n<p>dim=300\nembedding_dict={}</p>\n\n<p>with zipfile.ZipFile(\"../input/quora-insincere-questions-classification/embeddings.zip\") as zf:\n    with io.TextIOWrapper(zf.open(\"glove.840B.300d/glove.840B.300d.txt\"), encoding=\"utf-8\") as f:\n        for line in tqdm(f):\n            values=line.split(' ') # \".split(' ')\" only for glove-840b-300d; for all other files, \".split()\" works\n            word=values[0]\n            vectors=np.asarray(values[1:],'float32')\n            embedding_dict[word]=vectors\n```</p>\n\n<p>I am positive that this code will work for other embeddings as well but is particularly helpful for reading GloVe embeddings. Hope this helps :)</p>",
      "rawMarkdown": "Here is a small code snippet that I referred from [this post](https://stackoverflow.com/a/60918920/6543250) on Stack Overflow that helps with reading a zip file without completely unzipping it.\n\n```\nimport io\nimport zipfile\n\ndim=300\nembedding_dict={}\n\nwith zipfile.ZipFile(\"../input/quora-insincere-questions-classification/embeddings.zip\") as zf:\n    with io.TextIOWrapper(zf.open(\"glove.840B.300d/glove.840B.300d.txt\"), encoding=\"utf-8\") as f:\n        for line in tqdm(f):\n            values=line.split(' ') # \".split(' ')\" only for glove-840b-300d; for all other files, \".split()\" works\n            word=values[0]\n            vectors=np.asarray(values[1:],'float32')\n            embedding_dict[word]=vectors\n```\n\nI am positive that this code will work for other embeddings as well but is particularly helpful for reading GloVe embeddings. Hope this helps :)"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "832160": "Here is a small code snippet that I referred from [this post](https://stackoverflow.com/a/60918920/6543250) on Stack Overflow that helps with reading a zip file without completely unzipping it.\n\n```\nimport io\nimport zipfile\n\ndim=300\nembedding_dict={}\n\nwith zipfile.ZipFile(\"../input/quora-insincere-questions-classification/embeddings.zip\") as zf:\n    with io.TextIOWrapper(zf.open(\"glove.840B.300d/glove.840B.300d.txt\"), encoding=\"utf-8\") as f:\n        for line in tqdm(f):\n            values=line.split(' ') # \".split(' ')\" only for glove-840b-300d; for all other files, \".split()\" works\n            word=values[0]\n            vectors=np.asarray(values[1:],'float32')\n            embedding_dict[word]=vectors\n```\n\nI am positive that this code will work for other embeddings as well but is particularly helpful for reading GloVe embeddings. Hope this helps :)"
  }
}