{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Word Embeddings and Embedding Layer in Keras.","metadata":{"_uuid":"d3d6451a0f6029825c07e3efadcd77169b31db9e"}},{"cell_type":"markdown","source":"### Do we need embedding layer?\n\nEmbedding layer enables us to convert each word into a fixed length vector of defined size. The resultant vector is a dense one with having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions\n","metadata":{"_uuid":"7a5c0786e978d93aa5de2a4315cb817d66f3da38"}},{"cell_type":"markdown","source":"#### IMPORTING MODULES","metadata":{"_uuid":"d48312c5146ad38bab97d505dd35a147b946de4a"}},{"cell_type":"code","source":"# Ignore the warnings\nimport warnings\nwarnings.filterwarnings('always')\nwarnings.filterwarnings('ignore')\n\n# data visualisation and manipulation\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom matplotlib import style\nimport seaborn as sns\n#configure\n# sets matplotlib to inline and displays graphs below the corressponding cell.\n%matplotlib inline \nstyle.use('fivethirtyeight')\nsns.set(style='whitegrid',color_codes=True)\n\n\n#keras\nimport keras\nfrom keras.preprocessing.text import one_hot,Tokenizer\nfrom keras.preprocessing.sequence import pad_sequences\nfrom keras.models import Sequential\nfrom keras.layers import Dense , Flatten ,Embedding,Input\nfrom keras.models import Model","metadata":{"_uuid":"cedb06de49a94e693e8de60941b7c57d6997f63f","execution":{"iopub.status.busy":"2023-01-04T09:26:51.17455Z","iopub.execute_input":"2023-01-04T09:26:51.17495Z","iopub.status.idle":"2023-01-04T09:26:55.297992Z","shell.execute_reply.started":"2023-01-04T09:26:51.174893Z","shell.execute_reply":"2023-01-04T09:26:55.296635Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### CREATING SAMPLE CORPUS OF DOCUMENTS","metadata":{"_uuid":"604b9fdec82053d0170e98af3e76594001ee9e43"}},{"cell_type":"code","source":"text_1=\"she bought the baby a doll\"\ntext_2=\"but she wants to buy something else\"\ntext_3=\"so she bought another doll and she went back home\"\n\ndocs=[text_1,text_2,text_3]\nno_docs=len(docs)\n","metadata":{"_uuid":"b789f1783a653c9a6b74541fd27348f5680c6569","execution":{"iopub.status.busy":"2023-01-04T09:34:53.306761Z","iopub.execute_input":"2023-01-04T09:34:53.307607Z","iopub.status.idle":"2023-01-04T09:34:53.312503Z","shell.execute_reply.started":"2023-01-04T09:34:53.307548Z","shell.execute_reply":"2023-01-04T09:34:53.311599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### INTEGER ENCODING ALL THE DOCUMENTS","metadata":{"_uuid":"2fc3f964bec6dab85e0b9e5874d065f6f5637f1d"}},{"cell_type":"markdown","source":"After this all the unique words will be reprsented by an integer. For this we are using **one_hot** function from the Keras. Note that the **vocab_size** is specified large enough so as to ensure **unique integer encoding** for each and every word.","metadata":{"_uuid":"46bd3ac6a6dfb9e73708302c20cd7091092b596e"}},{"cell_type":"code","source":"vocab_size=50 \nencod_docs=[]\nfor i,doc in enumerate(docs):\n encod_docs.append(one_hot(doc,50))\n print(\"The encoding for document\",i+1,\" is : \",one_hot(doc,50))","metadata":{"_uuid":"04453f82e7bd7cb51c8a2c7d17637218060e398a","execution":{"iopub.status.busy":"2023-01-04T09:35:21.721167Z","iopub.execute_input":"2023-01-04T09:35:21.721503Z","iopub.status.idle":"2023-01-04T09:35:21.728407Z","shell.execute_reply.started":"2023-01-04T09:35:21.721443Z","shell.execute_reply":"2023-01-04T09:35:21.727248Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### PADDING THE DOCS (to make very doc of same length)","metadata":{"_uuid":"e58da092bf54cc13ea8eedb3b930b5857dc94b40"}},{"cell_type":"markdown","source":"**The Keras Embedding layer requires all individual documents to be of same length.** Therefore now in Keras Embedding layer the **'input_length'** will be equal to the length of the document with maximum length or maximum number of words.\n\nTo pad the shorter documents We use **pad_sequences** functon from the Keras library.","metadata":{"_uuid":"7dcb24072c22e932b98dcb15f17372430cd715f6"}},{"cell_type":"code","source":"# length of maximum document. will be nedded whenever create embeddings for the words\nimport nltk\nmaxlen = -1\nfor doc in docs:\n tokens=nltk.word_tokenize(doc)\n if(maxlen no of documents**\n\n**10---> each document is made of 10 words which was our maximum length of any document.**\n\n**8---> each word is 8 dimensional.**\n","metadata":{}},{"cell_type":"code","source":"embed_weights = embed_model.get_layer(\"embedding_4\").get_weights()[0]\nprint(embed_weights.shape) # same size as vocab size and embedding_dim (each word is a embedding_dim size vector)","metadata":{"_uuid":"7ebe56daafe993cd6b6845231928ec237831707c","execution":{"iopub.status.busy":"2023-01-04T10:18:44.466169Z","iopub.execute_input":"2023-01-04T10:18:44.466566Z","iopub.status.idle":"2023-01-04T10:18:44.479993Z","shell.execute_reply.started":"2023-01-04T10:18:44.466501Z","shell.execute_reply":"2023-01-04T10:18:44.478956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n ","metadata":{"_uuid":"d7cd46a4aaa6d7d1f43a81202add33c5b7f3b528"}},{"cell_type":"markdown","source":"#### GETTING ENCODING FOR A PARTICULAR WORD IN A SPECIFIC DOCUMENT","metadata":{"_uuid":"d8862c2ce0926f3fc06b813faa35d06ce72e87b6"}},{"cell_type":"code","source":"for i,doc in enumerate(embeddings_pred):\n for j,word in enumerate(doc):\n print(f\"The encoding for {j+1}th word in {i+1}th document is :\",word)\n if j == 1 :\n break\nprint(\"....\") ","metadata":{"_uuid":"6ac216640a7db94d9794a88256851d4e1e0c8858","execution":{"iopub.status.busy":"2023-01-04T10:45:30.351024Z","iopub.execute_input":"2023-01-04T10:45:30.35173Z","iopub.status.idle":"2023-01-04T10:45:30.359801Z","shell.execute_reply.started":"2023-01-04T10:45:30.351676Z","shell.execute_reply":"2023-01-04T10:45:30.358542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}