{"cells":[{"metadata":{},"cell_type":"markdown","source":"![](http://www.quickmeme.com/img/8d/8da6189f6c24b2f66498daccefdba903f70e45b1fc4e305d52848c5942e1c066.jpg)"},{"metadata":{},"cell_type":"markdown","source":"## Taking a deep dive in NLP techniques with the famous Kaggle competition Quora Insincere Questions Classification problem. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nprint(os.listdir(\"../input\"))\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nsns.set()","execution_count":14,"outputs":[{"output_type":"stream","text":"['test.csv', 'train.csv', 'sample_submission.csv', 'embeddings']\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(\"../input/train.csv\",index_col='qid')\ndf_test = pd.read_csv(\"../input/test.csv\",index_col='qid')\n#df = pd.concat([df_train ,df_test],sort=True)","execution_count":2,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.info()","execution_count":3,"outputs":[{"output_type":"stream","text":"\nIndex: 1306122 entries, 00002165364db923c7e6 to ffffed09fedb5088744a\nData columns (total 2 columns):\nquestion_text 1306122 non-null object\ntarget 1306122 non-null int64\ndtypes: int64(1), object(1)\nmemory usage: 29.9+ MB\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"# Business Problem"},{"metadata":{},"cell_type":"markdown","source":"## Business problem asks us to classify question texts given to us with the help of the target variable in the training dataset. Given the involvement of text data, it requires to be treated with NLP techniques to be able to extract useful information from text data."},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.head()","execution_count":5,"outputs":[{"output_type":"execute_result","execution_count":5,"data":{"text/plain":" question_text target\nqid \n00002165364db923c7e6 How did Quebec nationalists see their province... 0\n000032939017120e6e44 Do you have an adopted dog, how would you enco... 0\n0000412ca6e4628ce2cf Why does velocity affect time? Does velocity a... 0\n000042bf85aa498cd78e How did Otto von Guericke used the Magdeburg h... 0\n0000455dfa3e01eae3af Can I convert montra helicon D to a mountain b... 0","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
question_texttarget
qid
00002165364db923c7e6How did Quebec nationalists see their province...0
000032939017120e6e44Do you have an adopted dog, how would you enco...0
0000412ca6e4628ce2cfWhy does velocity affect time? Does velocity a...0
000042bf85aa498cd78eHow did Otto von Guericke used the Magdeburg h...0
0000455dfa3e01eae3afCan I convert montra helicon D to a mountain b...0
\n
"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.target.value_counts()","execution_count":4,"outputs":[{"output_type":"execute_result","execution_count":4,"data":{"text/plain":"0 1225312\n1 80810\nName: target, dtype: int64"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"p = sns.countplot(x=None,y= 'target',data=df_train)","execution_count":32,"outputs":[{"output_type":"display_data","data":{"text/plain":"
","image/png":"iVBORw0KGgoAAAANSUhEUgAAAYQAAAESCAYAAAD9gqKNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAEYlJREFUeJzt3Xts1fXdwPF3L5bbXKAFbMvMLmSDbj4MUpC5ZExabTd6wTgDSR/I3FidI/iwENcHtcqULVqXzrnJZfOBOJM9W+LiNOtWd0kxy9xEcLpJ5BIBWWcL1VN8FAiFnvN9/tivnWwKp6znd1p5v/6hPT/O+X2+4XfOm3PPCSEEJEkXvNxsDyBJGhkMgiQJMAiSpIhBkCQBBkGSFDEIkiTAIEiSIgZBkgQYBElSxCBIkgCDIEmKGARJEmAQJEmR/GwPkK6jR4+TSr07Ppi1qOg9JBLHsj3GsHE9I5vrGdkytZ7c3BwmTZowpPOMmiCkUuFdEwTgXbUWcD0jnesZ2UbKenzISJIEGARJUsQgSJIAgyBJihgESRJgECRJEYMgSQIMgiQpkhNCGBnviJAkneFk32nefOPkeZ03NzeHoqL3DOk8o+adyv9192O8dvR4tseQpNj8773/yZucXxDOhw8ZSZIAgyBJihgESRJgECRJEYMgSQIMgiQpYhAkSYBBkCRFDIIkCTAIkqSIQZAkAQZBkhQxCJIkwCBIkiIGQZIEGARJUsQgSJIAgyBJihgESRJgECRJEYMgSQIMgiQpYhAkSYBBkCRFDIIkCTAIkqSIQZAkAQZBkhQxCJIkwCBIkiIGQZIEGARJUsQgSJIAgyBJihgESRJgECRJEYMgSQIMgiQpElsQDh48yNKlS6murmbp0qW8/PLLce1akpSG2IKwbt06Ghoa+NWvfkVDQwN33HFHXLuWJKUhliAkEglefPFFamtrAaitreXFF1+kt7c3jt1LktIQSxC6u7u55JJLyMvLAyAvL4+pU6fS3d0dx+4lSWnwSWVJEhBTEEpKSjhy5AjJZBKAZDJJT08PJSUlcexekpSGWIJQVFREWVkZbW1tALS1tVFWVkZhYWEcu5ckpSE/rh19/etfZ+3atWzcuJH3vve9tLS0xLVrSVIaYgvC9OnTeeSRR+LanSRpiHxSWZIEGARJUsQgSJIAgyBJihgESRJgECRJEYMgSQIMgiQpYhAkSYBBkCRFDIIkCTAIkqSIQZAkAQZBkhQxCJIkwCBIkiIGQZIEGARJUsQgSJIAgyBJihgESRJgECRJEYMgSQIMgiQpYhAkSYBBkCRFDIIkCTAIkqSIQZAkAQZBkhQxCJIkwCBIkiJpB+ErX/nK256+atWqYRtGkpQ9aQdh+/btb3v6M888M2zDSJKyJ/9cf+H+++8H4PTp04M/D+js7KS0tDQzk0mSYnXOIBw+fBiAEMLgzwNKSkq46aabMjOZJClW5wzC3XffDcCcOXNYsmRJxgeSJGXHOYMwYMmSJezfv58nnniCRCLBHXfcwYEDBzh16hQzZ87M5IySpBjkhBBCOn+xvb2dO++8k6qqKtra2vjTn/7ECy+8QGtrKw899FCGx5SkC8/JvtO8+cbJ8zpvbm4ORUXvGdJ50r6H8N3vfpeHHnqImTNn0t7eDsDMmTPZs2fP0KY8T4nEMVKptNo14k2ZcjGvvvpmtscYNq5nZHM9I9tIWk/aLzvt7e1lxowZAOTk5Az+OfCzJGl0SzsIH/vYx3j88cfPOO0Xv/gFs2bNGvahJEnxS/sho9tuu40VK1bw05/+lBMnTrBixQoOHjzI1q1bMzmfJCkmaQdh+vTptLe3s23bNq688kpKSkq48sormTBhQibnkyTFJO0gAIwbN45FixZlahZJUhalHYSGhoa3fQK5oKCA4uJirr76aioqKoZ1OElSfNJ+Uvnyyy/nlVdeYd68edTX1zNv3jy6urq47LLLKCoq4tZbb+XBBx/M5KySpAxK+x7CU089xZYtW5g+ffrgaXV1daxdu5ZHHnmEqqoq1qxZQ2NjY0YGlSRlVtr3EA4cOMCll156xmnTpk3j4MGDAMyaNYtEIjG800mSYpN2EObNm8ctt9zCoUOH6Ovr49ChQzQ3N1NeXg7A3r17mTJlSsYGlSRlVtpBuOeee0ilUtTU1DB79mxqampIpVKDn4Z60UUX0dramrFBJUmZldZzCKlUit27d9PS0kJrayu9vb0UFhaSm/uPnnzoQx/K2JCSpMxL6x5Cbm4uK1eupKCggNzcXCZPnnxGDCRJo9+QnkN4/vnnMzmLJCmL0n7ZaWlpKY2NjVRWVlJcXHzGm9RWr16dkeEkSfFJOwh9fX1cddVVABw5ciRjA0mSsiPtIAy8mkiS9O40pA+3Azh27BhHjx4947R/fsOaJGn0STsIL730EjfffDN79uwhJyeHEMLg8wi7d+/O2ICSpHikHYQ777yT+fPn8/DDD1NZWUlHRwetra3MmTMnk/MNGuqXRZ9L/6k+jv7fqWG9TEkazdIOwp49e9i6dSsXXXQRIQQuvvhimpqaqK2tZfHixZmcEYAXNv83p94Yvs9KKm/6H8AgSNKAtN+HMGbMGPr7+wGYNGkSXV1dpFIpXn/99YwNJ0mKT9r3EMrLy2lvb+faa6+lurqaxsZGCgoKuOKKKzI5nyQpJmkHYdasWVx77bUArFmzhg9/+MMcP36cN954I2PDSZLik/ZDRhs2bPjHmXJzWbx4MQ0NDWzdujUjg0mS4nXOewh//OMfAUgmkzz99NOEEAa3/e1vf2PChAmZm06SFJtzBuG2224D4NSpU9x6662Dp+fk5DB58mSam5szN50kKTbnDEJHRwcATU1N3HvvvRkfSJKUHWk/h2AMJOndzW+5kSQBBkGSFDEIkiTAIEiSIgZBkgQYBElSxCBIkgCDIEmKGARJEmAQJEkRgyBJAgyCJCliECRJgEGQJEUMgiQJMAiSpIhBkCQBBkGSFDEIkiTAIEiSIgZBkgQYBElSxCBIkgCDIEmKGARJEmAQJEkRgyBJAgyCJCliECRJgEGQJEViCUJLSwsVFRXMmDGDffv2xbFLSdIQxRKEyspKfvSjHzFt2rQ4didJOg/5cexk7ty5cexGkvRv8DkESRJgECRJEYMgSQIMgiQpEksQvvGNb7BgwQIOHz7MF77wBWpqauLYrSRpCGJ5lVFzczPNzc1x7EqSdJ58yEiSBBgESVLEIEiSAIMgSYoYBEkSYBAkSRGDIEkCDIIkKWIQJEmAQZAkRQyCJAkwCJKkiEGQJAEGQZIUMQiSJMAgSJIiBkGSBBgESVLEIEiSAIMgSYoYBEkSYBAkSRGDIEkCDIIkKWIQJEmAQZAkRQyCJAkwCJKkiEGQJAEGQZIUMQiSJMAgSJIiBkGSBBgESVLEIEiSAIMgSYoYBEkSYBAkSZH8bA+Qrv+4sWVYL6//VN+wXp4kjXajJgiJxDFSqZDtMSTpXcuHjCRJgEGQJEUMgiQJMAiSpIhBkCQBBkGSFDEIkiTAIEiSIqPmjWm5uTnZHmFYuZ6RzfWMbK4nM5eZE0Lw7b+SJB8ykiT9nUGQJAEGQZIUMQiSJMAgSJIiBkGSBBgESVLEIEiSAIMgSYqM6CAcPHiQpUuXUl1dzdKlS3n55ZezMsfRo0dpbGykurqauro6Vq1aRW9vLwDPP/889fX1VFdX88UvfpFEIjF4vri3DdUDDzzAjBkz2Ldv36hfS19fH+vWraOqqoq6ujpuv/124OzHUNzbhmLbtm1cc801LF68mPr6en7961+PqvW0tLRQUVFxxvE10uYfytrebj1nu12AUXp9CiPY8uXLw2OPPRZCCOGxxx4Ly5cvz8ocR48eDU8//fTg7/fcc0+45ZZbQjKZDFdddVXYsWNHCCGEDRs2hLVr14YQQuzbhmrXrl1hxYoVYeHChWHv3r2jei0hhLB+/frwzW9+M6RSqRBCCK+++moI4ezHUNzb0pVKpcLcuXPD3r17Qwgh7N69O8yePTskk8lRs54dO3aErq6uweMrWzMO19rebj3vdLsQQvzXmeG6Po3YILz22muhvLw89Pf3hxBC6O/vD+Xl5SGRSGR5shCeeOKJ8PnPfz78+c9/DjU1NYOnJxKJMHv27BBCiH3bUPT19YUlS5aEzs7OwQN8tK4lhBCOHTsWysvLw7Fjx844/WzHUNzbhiKVSoXLL7887Ny5M4QQwjPPPBOqqqpG5XreegM6kuY/37X9c+DeauB2IYT4rzPDdX0asZ922t3dzSWXXEJeXh4AeXl5TJ06le7ubgoLC7M2VyqV4sc//jEVFRV0d3dTWlo6uK2wsJBUKsXrr78e+7aJEyemvYb777+f+vp63ve+9w2eNlrXAtDZ2cnEiRN54IEH2L59OxMmTGD16tWMHTv2HY+hEEKs24ZyzObk5PCd73yHlStXMn78eI4fP84PfvCDs14nRvJ6Boyk+Yd7bW+9XRhY62i8Po3o5xBGovXr1zN+/HiWLVuW7VHOy3PPPceuXbtoaGjI9ijDJplM0tnZyUc/+lEeffRRbr75Zm666SZOnDiR7dHOS39/P9///vfZuHEj27ZtY9OmTXz1q18dteu5EIz224UBI/YeQklJCUeOHCGZTJKXl0cymaSnp4eSkpKszdTS0sKhQ4fYvHkzubm5lJSU0NXVNbi9t7eX3NxcJk6cGPu2dO3YsYP9+/dTWVkJwOHDh1mxYgXLly8fdWsZUFJSQn5+PrW1tQB8/OMfZ9KkSYwdO/Ydj6EQQqzbhmL37t309PRQXl4OQHl5OePGjWPMmDGjcj1v/XcaKfMP59r++XZhYK2j8fo0Yu8hFBUVUVZWRltbGwBtbW2UlZVl7eGib3/72+zatYsNGzZQUFAAwGWXXcbJkyfZuXMnAD/5yU/4zGc+k5Vt6brhhhv4/e9/T0dHBx0dHRQXF7Nlyxa+9KUvjbq1DCgsLGT+/Pk89dRTwN9fPZJIJPjABz7wjsfQ2Y6vTGwbiuLiYg4fPsyBAwcA2L9/P4lEgve///2jcj0D4p4xjrW93e0CjM7bBmBkv8ropZdeCtddd12oqqoK1113Xdi/f39W5ti3b1/4yEc+EqqqqkJ9fX2or68PK1euDCGE8Oyzz4ba2tpw9dVXh+uvv37w1S3Z2HY+3vok2Whey1//+tewbNmyUFtbG6655prw5JNPhhDOfgzFvW0oHn/88VBbWxvq6upCXV1d+M1vfjOq1rN+/frwqU99KpSVlYVPfvKTYdGiRSNu/qGs7e3Wc7bbhRBG5/XJb0yTJAEj+CEjSVK8DIIkCTAIkqSIQZAkAQZBkhQxCJIkwCBIsauoqOAPf/hDtseQ/oVBkCQBBkGiu7ubVatW8YlPfIL58+dz1113kUql2LhxIwsXLuSKK66gqamJN998E4Dt27ezYMGCMy7jrf/r/973vsfq1atpampizpw51NTU8MILLwDwta99ja6uLm688UbmzJnDgw8+GO9ipbMwCLqgJZNJvvzlL1NaWkpHRwe/+93vWLRoEY8++ig/+9nPePjhh/ntb3/LiRMnuOuuu9K+3I6ODmpqati5cycVFRWsX78egG9961uUlpayefNmnnvuORobGzO1NGnIDIIuaH/5y1/o6emhqamJ8ePHM2bMGObOncvPf/5zrr/+ei699FImTJjAmjVr+OUvf0l/f39al1teXs6nP/1p8vLyWLx4MXv27MnwSqR/n0HQBW3gi0Xy88/8JPienh6mTZs2+Pu0adPo7+9P+3tqJ0+ePPjz2LFj6evrSzsmUrYYBF3QSkpK6O7u/pcb66lTp/LKK68M/t7V1UV+fj5FRUWMGzeOkydPDm5LJpNnfLm6NFoZBF3QZs2axZQpU2htbeXEiRP09fXx7LPPUltbyw9/+EM6Ozs5fvw49913H5/97GfJz8/ngx/8IH19fTz55JOcPn2aTZs2cerUqbT3OXnyZDo7OzO4Kun8GARd0PLy8ti8eTOHDh1i4cKFLFiwgPb2dj73uc9RX1/PsmXLqKyspKCggNtvvx2Aiy++mHXr1tHc3MyCBQsYN24cxcXFae/zhhtuYNOmTcydO5ctW7ZkamnSkPl9CJIkwHsIkqSIQZAkAQZBkhQxCJIkwCBIkiIGQZIEGARJUsQgSJIAgyBJivw/dLEX91qDfm4AAAAASUVORK5CYII=\n"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"![](https://cdn-images-1.medium.com/max/1200/0*dIUk8mZ4ADIqBjUj.jpg)"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.dropna(inplace=True)","execution_count":6,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Time for some NLP"},{"metadata":{},"cell_type":"markdown","source":"# Bag of Words\n The bag-of-words model is simple to understand and implement. It is a way of extracting features from the text for use in machine learning algorithms.In this approach, we use the tokenized words for each observation and find out the frequency of each token.\nLet’s take an example to understand this concept in depth.\n\n* “It was the best of times”\n* “It was the worst of times”\n* “It was the age of wisdom”\n* “It was the age of foolishness”\n\nWe treat each sentence as a separate document and we make a list of all words from all the four documents excluding the punctuation. We get,\n\n‘It’, ‘was’, ‘the’, ‘best’, ‘of’, ‘times’, ‘worst’, ‘age’, ‘wisdom’, ‘foolishness’\n\nThe next step is the create vectors. Vectors convert text that can be used by the machine learning algorithm.\n\nWe take the first document — “It was the best of times” and we check the frequency of words from the 10 unique words.\n* “it” = 1\n* “was” = 1\n* “the” = 1\n* “best” = 1\n* “of” = 1\n* “times” = 1\n* “worst” = 0\n* “age” = 0\n* “wisdom” = 0\n* “foolishness” = 0"},{"metadata":{},"cell_type":"markdown","source":"#### Rest of the documents will be:\n* “It was the best of times” = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]\n* “It was the worst of times” = [1, 1, 1, 0, 1, 1, 1, 0, 0, 0]\n* “It was the age of wisdom” = [1, 1, 1, 0, 1, 0, 0, 1, 1, 0]\n* “It was the age of foolishness” = [1, 1, 1, 0, 1, 0, 0, 1, 0, 1]"},{"metadata":{},"cell_type":"markdown","source":"#### In this approach, each word or token is called a “gram”. Creating a vocabulary of two-word pairs is called a bigram model.\n\nFor example, the bigrams in the first document : “It was the best of times” are as follows:\n* “it was”\n* “was the”\n* “the best”\n* “best of”\n* “of times”"},{"metadata":{},"cell_type":"markdown","source":"# Vectorization\n\n#### The process of converting NLP text into numbers is called vectorization in ML. Different ways to convert text into vectors are:\n\n* Counting the number of times each word appears in a document.\n* Calculating the frequency that each word appears in a document out of all the words in the document."},{"metadata":{},"cell_type":"markdown","source":"## 1. CountVectorizer\nCountVectorizer works on Terms Frequency, i.e. counting the occurrences of tokens and building a sparse matrix of documents x tokens."},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\nvect = CountVectorizer()\nvect","execution_count":7,"outputs":[{"output_type":"execute_result","execution_count":7,"data":{"text/plain":"CountVectorizer(analyzer='word', binary=False, decode_error='strict',\n dtype=, encoding='utf-8', input='content',\n lowercase=True, max_df=1.0, max_features=None, min_df=1,\n ngram_range=(1, 1), preprocessor=None, stop_words=None,\n strip_accents=None, token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n tokenizer=None, vocabulary=None)"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"## 2. TF-IDF Vectorizer\nTF-IDF stands for term frequency-inverse document frequency. TF-IDF weight is a statistical measure used to evaluate how important a word is to a document in a collection or corpus. The importance increases proportionally to the number of times a word appears in the document but is offset by the frequency of the word in the corpus.\n\n**Term Frequency (TF)**: is a scoring of the frequency of the word in the current document. Since every document is different in length, it is possible that a term would appear much more times in long documents than shorter ones. The term frequency is often divided by the document length to normalize.\n\n![](https://cdn-images-1.medium.com/max/800/1*SUAeubfQGK_w0XZWQW6V1Q.png)\n\n**Inverse Document Frequency** (IDF): is a scoring of how rare the word is across documents. IDF is a measure of how rare a term is. Rarer the term, more is the IDF score.\n\n\n![](https://cdn-images-1.medium.com/max/800/1*T57j-UDzXizqG40FUfmkLw.png)\nThus,\n\n![](https://cdn-images-1.medium.com/max/800/1*YrgmAeG7KNRB4dQcGcsdyg.png)"},{"metadata":{},"cell_type":"markdown","source":"# Implementation"},{"metadata":{},"cell_type":"markdown","source":"#### Below is a pipeline based LGBM Classifier implementation of the above NLP concept Bag of Words on the Quora business problem."},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.pipeline import Pipeline\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nimport lightgbm as lgbm\n\n\ntext_clf_lgbm = Pipeline([('tfidf', TfidfVectorizer()),\n ('clf', lgbm.LGBMClassifier()),\n])\n","execution_count":8,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nX = df_train['question_text']\ny = df_train['target']\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42,stratify=y)","execution_count":9,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"text_clf_lgbm.fit(X_train, y_train)\n# Form a prediction set\npredictions = text_clf_lgbm.predict(X_test)","execution_count":10,"outputs":[{"output_type":"stream","text":"[[400644 3710]\n [ 17155 9512]]\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"# Evaluation\n\n\n![](http://www.quickmeme.com/img/30/300ace809c3c2dca48f2f55ca39cbab24693a9bd470867d2eb4e869c645acd42.jpg)"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn import metrics\ncnf_matrix = metrics.confusion_matrix(y_test, predictions)\np = sns.heatmap(pd.DataFrame(cnf_matrix), annot=True, cmap=\"YlGnBu\" ,fmt='g')\nplt.title('Confusion matrix', y=1.1)\nplt.ylabel('Actual label')\nplt.xlabel('Predicted label')","execution_count":21,"outputs":[{"output_type":"execute_result","execution_count":21,"data":{"text/plain":"Text(0.5, 12.5, 'Predicted label')"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"
","image/png":"iVBORw0KGgoAAAANSUhEUgAAAY0AAAEcCAYAAAA7neg3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xtcjvf/B/DXfXdScbsrIkXxnQgjRJuQ87EVOcV3NNYwh2zDWA5N2JRtbMqZZpvZCZHz+bRh5TQ5RaIoWiel0um+fn/47f4ulytXuLtLr+fjcT0e3/v6XIf3dc3Xy+f6XAeFIAgCiIiIZFDquwAiIqo8GBpERCQbQ4OIiGRjaBARkWwMDSIiko2hQUREsjE0SCcePXqE8ePHo23btvD393/u7Wzfvh1jxox5iZXpT3R0NHr37q3vMoheiILPaVRtkZGRCA8PR3x8PMzNzdG0aVOMHz8eLi4uL7TdiIgI/PDDD/jpp59gaGj4kqqtuJo0aYJ9+/bB3t5e36UQ6dSr//9mkhQeHo7Vq1dj3rx56NixI4yMjHD8+HEcPHjwhUMjKSkJDg4OVSIw5CgqKuK5oFeDQFVSVlaW4OzsLOzatUtymfz8fGHBggWCm5ub4ObmJixYsEDIz88XBEEQTp06JXTq1ElYt26d8MYbbwhubm7Cb7/9JgiCIHz99ddC8+bNhWbNmgnOzs7CL7/8InzzzTfC1KlTtdtOTEwUHB0dhcLCQkEQBGHz5s1Ct27dBGdnZ6Fr167Ctm3btPN9fHy06505c0bw9vYW2rRpI3h7ewtnzpzRtr399tvCkiVLhGHDhgnOzs7C6NGjhbS0tKce2z/1r169Wlv//v37hSNHjgi9evUS2rVrJ6xYsUK7/IULF4ShQ4cKbdu2Fdzc3IR58+Zpz8WIESMER0dHoVWrVoKzs7Owc+dO7fZXrVoldOjQQZg2bZp2niAIwu3bt4V27doJMTExgiAIwr179wRXV1fh1KlTMv8LEukHQ6OKOnr0qODk5KT9S/tpli5dKgwZMkRITU0V0tLShGHDhglLliwRBOHxX7pOTk7C0qVLhYKCAuHIkSNCy5YthczMTEEQBFFIlBYaOTk5QuvWrYW4uDhBEATh/v37QmxsrCAIJUMjIyNDcHFxEbZu3SoUFhYKkZGRgouLi5Ceni4IwuPQ6N69u3Dz5k0hLy9PePvtt4XFixc/9dj+qX/ZsmVCQUGB8PPPPwuurq7CRx99JGRnZwuxsbHC66+/LiQkJAiCIAgXL14Uzp07JxQWFgqJiYlCnz59hPDwcO32HB0dhVu3bom2HxISIuTn5wt5eXklQkMQBOHnn38W+vbtK+Tm5gpjxowRFi1a9Iz/akT6x4HwKiozMxMWFhalXjKJjIzExIkTYWVlBUtLS0ycOBHbt2/XthsaGmLixIkwMjKCu7s7zMzMEB8f/1z1KJVKXL9+HY8ePYK1tTUaN24sWubIkSOwt7fHgAEDYGhoCA8PDzRq1AiHDx/WLuPt7Y2GDRuiWrVq6NOnD65cuSK5T0NDQ7z//vswMjJCv379kJGRgVGjRqF69epo3LgxXnvtNVy7dg0A0KJFCzg7O8PQ0BB2dnYYNmwYoqKinnlM/v7+MDY2RrVq1UTtQ4cORYMGDTB06FCkpKTgww8/lHu6iPSGoVFFqdVqZGRkoKioSHKZlJQU1KtXT/u7Xr16SElJKbGNf4eOqakpcnNzy1yLmZkZlixZgp9++gkdO3bE2LFjERcX98x6/qnp/v372t+1a9eWXY9arYaBgQEAaP9St7Ky0rabmJggJycHABAfH49x48bBzc0Nbdq0wZIlS5CRkVHqcVlYWMDExKTUZYYOHYrY2FiMHDkSxsbGpS5LVBEwNKqo1q1bw9jYGAcOHJBcxtraGklJSdrfycnJsLa2fq79mZqa4tGjR9rfqampJdo7deqE8PBwnDhxAo0aNcKcOXOeWc8/NdWpU+e5aiqLTz/9FI0aNcLevXtx9uxZfPjhhxCeceOhQqEotT0nJwefffYZBg8ejGXLliEzM/NllkykEwyNKqpGjRrw9/dHUFAQDhw4gLy8PBQWFuLo0aMICQkBAPTv3x8rVqxAeno60tPTERYWhrfeeuu59ufk5ISoqCgkJSUhOzsbq1at0ralpqbiwIEDyM3NhbGxMczMzKBUiv9ouru749atW4iMjERRURF27dqFGzduoEuXLs9VU1nk5OTA3Nwc5ubmiIuLw6ZNm0q016pVC4mJiWXa5sKFC9GiRQssXLgQXbp0QWBg4MssmUgnGBpV2JgxYzBz5kwsX74cb775Jrp06YKNGzeiR48eAIAJEyagRYsW8PT0hKenJ5o3b44JEyY8177c3NzQr18/eHp6wtvbG127dtW2aTQafPvtt+jUqRPat2+PqKgofPrpp6JtWFhYYOXKlQgPD4erqyvWrl2LlStXwtLS8rlqKosZM2Zgx44daNOmDebMmYN+/fqVaJ80aRJmzpwJFxcX7Nq165nbO3DgAI4fP649zpkzZ+Ly5cslxoyIKiI+3EdERLKxp0FERLIxNIiISDaGBhGRnoSGhqJJkyaIjY0FAJw/fx6enp7o3bs3xowZg7S0NO2y5d0mSc8PFxIRVUkxMTHCu+++K3Tt2lW4du2aUFxcLPTo0UOIiooSBEEQwsLChJkzZwqCIJR7W2nY0yAiegmysrJw584d0ZSVlSVatqCgAEFBQSXuEoyJiYGJiYn2ZaE+Pj7Ys2ePXtpKU+leu2naYLi+S6AKJi9hnr5LoArL8YXWLsvfNyHTOyA0NFQ0f9KkSZg8eXKJeV9//TU8PT1hZ2ennZecnFzijQeWlpbQaDTIzMws9za1Wi15nJUuNIiIyotCIf9ijK+vLwYOHCiar1KpSvw+d+4cYmJiMG3atBeuTx8YGkREEhRluFdIpVKJAuJpoqKiEBcXh+7duwMA7t27h3fffRcjR44s8Zqc9PR0KJVKqNVq2NjYlGtbaTimQUQkQaFQyp7kGjt2LE6cOIFDhw7h0KFDqFu3LtatWwc/Pz88evQI0dHRAICffvoJffr0AfD4Lcvl2VYa9jSIiCSUJQxelFKpREhICAIDA5Gfnw9bW1ssXrxYL22lqXSvEeFAOD2JA+Ek7cUGwlWN/GQvm3Vz7Qvtq7JgT4OISEJ59jQqC4YGEZEEhoYYQ4OISEJZ7p6qKhgaREQS2NMQY2gQEUlgaIgxNIiIJCgVBvouocJhaBARSWBPQ4yhQUQkgaEhxtAgIpLA0BBjaBARSWJoPImhQUQkQankX5FP4hkhIpLAh/vEGBpERBI4piHG0CAikqBQKPRdQoXD0CAiksCehhhDg4hIAsc0xBgaREQSePeUGM8IEZEE9jTEGBpERFI4piHC0CAiksCBcDGGBhGRBN5yK8bQICKSwDENMYYGEZEEhZIfYXoSQ4OISAo7GiIMDSIiKRzTEGFoEBFJYWiIMDSIiKTw8pQIQ4OISIKgZE/jSQwNIiIpDA0RhgYRkRSOaYgwNIiIpDAzRBgaRERSeHlKhKFBRCSFl6dEGBpERFIMGBpPYmgQEUnRYWZMmDABd+7cgVKphJmZGebMmYO6devi448/RkJCAoyNjWFvb4+goCBYWloCAM6fP4+5c+ciPz8ftra2WLx4MaysrHTW9jR8dIWISIKgUMieyio4OBjbt29HREQExowZg4CAACgUCvj5+WHv3r2IjIxE/fr18cUXXwAANBoNpk+fjrlz52Lv3r1wcXHRaZsUhgYRkRSlQv5URjVq1ND+74cPH0KhUECtVsPV1VU739nZGUlJSQCAmJgYmJiYwMXFBQDg4+ODPXv26KxNCi9PERFJKUMWZGVlISsrSzRfpVJBpVI9dZ1Zs2bh999/hyAIWLt2bYk2jUaDTZs2oVu3bgCA5ORk1KtXT9tuaWkJjUaDzMxMnbSp1eqn1szQICKSUobLThs2bEBoaKho/qRJkzB58uSnrrNw4UIAQEREBEJCQrBmzRpt2/z582FmZoa33367jEXrFkODiEhKGe6e8vX1xcCBA0XzpXoZ/zZgwADMnTsXGRkZsLCwQHBwMG7fvo2VK1dCqXw8imBjY6O9VAUA6enpUCqVUKvVOmmTwjENIiIpCoXsSaVSwc7OTjQ9LTRycnKQnJys/X3o0CHUrFkTarUaX331FWJiYhAWFgZjY2PtMi1atMCjR48QHR0NAPjpp5/Qp08fnbVJnhJBEITnOpl6YtpguL5LoAomL2GevkugCsvxhdZ+bdAPspe9sVn+ZaTU1FRMmDABeXl5UCqVqFmzJmbMmAFjY2N4eHjAwcEB1apVAwDY2dkhLCwMAHD27FkEBgaWuD22Vq1aOmt7GoYGVXoMDZL2gqExpAyh8WvFGnvQFV6eKmf/caiLjNgNWL90onbeMK8OuPbHN0i9Go5f1nwEi5rm2jaLmub4efVHSL0ajmt/fINhXh1KbK+WZQ18+80k3ItZh6SLaxD+9UQ8yaKmORLOrcLBzYFPremTKd7IS9iErh1bvKSjpJdh2rQv0bHjKLRpMxS9e4/Dr7/uBQBs334ErVsP0U6tWg1CkyZvISbmBgDg1Km/MHJkANq2HYZu3d4VbffOnfsYOTIArVoNQp8+4/HHH+fL9bgqlTJcnqoqOBBezpYuGI0zf93U/nZytMOyz/0w8J0QnI+JR9ii9/D1wjEYNWnZ/y8/BgWFRbBvMx6tmjtgS/jH+OtKAq7E3gEA/LT6I5y5EAfHNychN68AzZvUF+1zQcAIXLtxF8qn3Eve0N4a3v1dkXw/Q0dHTM9r3LjB+OwzfxgbGyEuLhGjRgXAyek/8PTsAk/PLtrltmw5gOXLf0bz5v8BAJiZVcOgQT3g4dEZq1b9Ktru1KmL4ezcFGvWBOLo0Wj4+y/Cvn2rYGlZs7wOrdIQ+BoREfY0ytGQt97Eg6xcHP49RjvPZ4Abdh04i9//vIqc3HzM+/IXePVpj+rm1WBmaoIBfdtj3he/ICc3H39EXcPOA2cwwrsjAKB7p9dhZ2OFTxZuRFZ2HoqKinHh0q0S+3yjbWM0d7TDd78cfWpNS+ePwezPN6GgoEhnx03Pp3FjexgbGwEAFAoFFAoFEhKSRctt3XoIAwZ0g+L//7XbsqUjBgzohvr164qWjY+/i0uX4jB58ghUq2aC3r3d4Ohoj717/9DtwVRW7GmIlFtoZGRk4MqVK7hy5QoyMqrev2prVDfFnKlDMCPo+xLznRztcPHKbe3v+NspKCgsQuNGNmjcyAZFxcW4EX9P237x8m04OdoBANq3aYzYm0lY+9UE3LmwGiciF6Cjq5N2WaVSga+CRuPDud/iaUNX3v1dkV9QiL2HeXmiovr00+Vo1WoQ+vZ9H7VrW8Dd3aVE+927KYiOvgQvr66ytnfjRgLq16+L6tXNtPOaNm2IGzcSXmrdrwxFGaYqQueXpxISEjBnzhxcvnwZ1tbWAICUlBQ0a9YM8+bNg4ODg65LqBACpw3Bhp8P4+699BLzq5tXw4PsvBLzsrJzUd28GjQaDbKeaHuQnYca5qYAAFsbS/R0b4Xx01dh7LSVGNi3PX5dOxUtOn+ItIxsTBzdB1Hnb+DcxXjRZavq5tUw7+Nh6P/fz3RwtPSyfPrpBMyZMw7nzl3Fn3/GaHse/4iIOAQXl2ZP7VU8TU5OHmrUMC8xr0YNM9y/n/bSan6l8HsaIjrvaXz88ccYNGgQTp8+jZ07d2Lnzp04ffo0vL29MWPGDF3vvkJo2cweXTu+jm/W7hK1Pcx5BFV10xLzalQ3xcOcR3iYkw9VjZJtquqmyM55HCSPHhXgVkIKNvx8BEVFxfg18iTuJKfhTRdH2NSxwITRffBpyM9PrWn2h4Px45YTSLiT+pKOknTFwMAALi7Nce9eKjZtKvlnaNu2QxgwoLvsbZmbm+Lhw9wS8x4+zIO5uZnEGlUcL0+J6LynkZmZCU9PzxLzlEolvLy8sGLFCl3vvkLo/GYz2NvVQuzJx68YqG5eDQYGSjRtbIv9Ry/gdacG2mUdGljDxNgI128mQ6MRYGhggP841EXcrceXqF5v1kA7CH7xSgL69WhTYl//XIVyafUf1LVW4+zBx2+sNK1mDNNqxoiPXoH/tJ+ALm7NYWtjhbEjewIAalup8MPyKfhqxXZ8uSJSp+eDnk9xcTESEv53qfLMmctISUlH794dSlmrpNdea4DExHt4+DBXe4nq6tV4eHi4v/R6XwlVJwtk03loqNVq7NixA/3799cO1AmCgMjISFmP178K1m08iF+3/2+g8YOxHrCvXxv+AetQu1ZNHNk6D27tm+DcxVuY+9EQbNvzJx7mPAIAbNvzJ+ZOHYL3P16NVs3t4dHTBV29H986u31PFD6f/V/8d3BnbNpyHF592sHWxhIno2ORnZOHpm7+2n0OfutNDPNywxC/L6DRCOg3fCGMjAy07SciF2LG/O85vlFBpKVl4tSpv9ClSztUq2aMP/64gJ07j+HLL6drl4mIOIRevTqUGJ8AHr/orrCwCIWFRRAEAfn5BVAoFDA2NkLDhrZwcmqEsLBN+OCDkTh27AyuXbuFZcs+Ke9DrBwMea/Qk3QeGosWLUJgYCCCgoJQp04dAMD9+/fRtGlTLFq0SNe7rxDyHhUg71GB9vfD3Ed49KgQqenZSE3Phn/AOoR/PQmWFtVx6EQMxk1dqV12yqz1WPXFOCScW4n0jIeYMmudtqeR8SAHQ979AksXjMHS+aNxLS4JQ/2+QFpGNgDg/t8PtNt5kJWLwqIi7bz0zIclaiwu1iDjQQ5ycvN1dh5IPoVCgU2bdiEwcDk0Gg1sba0REPAeund//Nrs/PwC7N594ql/2UdFXcKoUQHa3y1bDkL79i3w/fefAwC++mo6PvlkKdq1Gw4bm9r45puZvN1WgsCehki5PRGenp6ufdeKjY2N9ktUZcUnwulJfCKcpL3YE+GNxv4me9mbqwe/0L4qi3J7uM/S0vK5g4KISC+q0AC3XHwinIhICm+5FWFoEBFJ4Ti4CEODiEiKAVPjSQwNIiIJAsc0RBgaRERS2NEQYWgQEUnhQLgIQ4OISAovT4kwNIiIpPAjTCIMDSIiCQIvT4kwNIiIpDA0RBgaRERSOKYhwtAgIpLCW25FGBpERFLY0xBhaBARSeFHmEQkQ0Oj0cjagFLJk0pErya+RkRMMjSaNWum/Tzr0wiCAIVCgStXruikMCIiveO/iUUkQ+PgwYPlWQcRUcXDnoaIZGjY2tqK5mk0GqSmpsLa2lqnRRERVQh8TkNEVucrKysLU6dORcuWLdGrVy8Aj3siS5Ys0WlxRER6pVTIn6oIWaERGBiI6tWr49ChQzAyMgIAtG7dGrt379ZpcURE+iQYKGRPVYWsW25PnjyJ48ePw8jISDs4bmlpibS0NJ0WR0SkVxzTEJHV06hRowYyMjJKzEtKSkLt2rV1UhQRUYXAy1MiskJjyJAh8Pf3x6lTp6DRaHDu3DnMmDEDPj4+uq6PiEh/FGWYqghZl6fee+89mJiYICgoCEVFRQgICMCwYcPg6+ur6/qIiPSGzy6LyQoNhUIBX19fhgQRVSm6CI2MjAx8/PHHSEhIgLGxMezt7REUFARLS0vtMp988gm2bNmCs2fPwtzcHABw6NAhhISEoLi4GM2bN8fnn38OU1NTnbVJnhO5B3ry5EnMnj0bY8eOxezZs3Hy5MmynSkiokpGoVDInsqyTT8/P+zduxeRkZGoX78+vvjiC237oUOHRNvLycnBnDlzsHLlSuzfvx/m5uZYt26dztpKIys01q9fj48++gg1a9aEu7s71Go1pk6divXr18s7S0RElZBCIX+SS61Ww9XVVfvb2dkZSUlJAB73QkJDQ/HJJ5+UWOfYsWNo0aIFHBwcAAA+Pj7aRx500VYaWZenwsPDsWHDBjg6OmrneXl5YfTo0RgzZoycTRARVTplCYOsrCxkZWWJ5qtUKqhUqqeuo9FosGnTJnTr1g0AEBQUBH9/f9SoUaPEcsnJyahXr572d7169ZCcnKyzttLIfjW6vb19id/169cvU5eMiKiyUZRhTGPDhg0IDQ0VzZ80aRImT5781HXmz58PMzMzvP3229i1axeMjIzQpUuX56y2fMh6NfrkyZMREBCAyZMno27dukhOTsby5cvh7+9fLkUSEelDWf5d7Ovri4EDB4rmS/UygoODcfv2baxcuRJKpRJ//vknTp06pe11AICHhwfWrFkDGxsbnD59Wjs/KSkJNjY2AKCTttIoBEEQntbQtGlTbU/i34v8e54+Xo1u2mB4ue6PKr68hHn6LoEqLMdnL1KKZuuPyV728pjOspf96quvcO7cOaxevVrybqUmTZpo7556+PAhevXqhR9//BEODg6YNWsWbGxsMGnSJJ20lYavRicikqCLK/DXr1/HqlWr4ODgoH1A2s7ODmFhYZLrVK9eHUFBQRg3bhw0Gg2cnJwwa9YsnbWVRrKnUVGxp0FPYk+DpL1YT6PFt8dlLxvzTqcX2ldlIXsg/ODBg4iKikJGRkaJy1UhISE6KYyISN/KMhBeVcg6JaGhoQgMDIRGo8GePXugVqtx4sQJyQEeIqJXgS6e06jsZIXG5s2bsX79egQEBMDIyAgBAQFYuXIl7ty5o+v6iIj0RqmUP1UVsi5PZWVlaR/sMzIyQmFhIVq2bImoqCidFkdEpE9V6I3nsskKjQYNGuD69eto3LgxGjdujE2bNkGlUqFmzZq6ro+ISG+q0mUnuWSFxgcffIDMzEwAwNSpUzFt2jTk5uYiMDBQp8UREekTQ0NMVmi4u7tr/3erVq2wf/9+nRVERFRRKHh9SkQyNBITE2VtoH79+i+tGCKiioQ9DTHJ0OjZsycUCgVKe/ZPH68RISIqL1Xprii5JEPj6tWr5VkHEVGFw6tTYrKfCCciqmp4eUqMoUFEJIGvERFjaBARSWBPQ4yhQUQkgV8nFWNoEBFJ4N1TYpKh4e7uLitljxw58jLreaasWzPKdX9U8WmEIn2XQBXUi979xI6GmGRoLF68uDzrICKqcHjLrZhkaLRv37486yAiqnAYGmKyxzSuXLmC6Oho0Zf7pkyZopPCiIj0TamoVF/DLheyhnl+/vlnDB8+HKdOncKaNWsQGxuL8PBwJCQk6Lo+IiK9MVTIn6oKWT2NtWvXYu3atXBxcUG7du0QFhaGo0ePYteuXbquj4hIb9jTEJPV00hLS4OLi8vjFZRKaDQauLu74/DhwzotjohIn5QK+VNVIaunUbduXdy5cwd2dnZwcHDAwYMHYWFhASMjI13XR0SkN3xMQ0xWaPj5+SEuLg52dnaYMGECpkyZgsLCQsyaNUvX9RER6U1V6kHIpRBK+2CGhIKCAhQWFsLc3FwXNZWqUHO+3PdJFZuBwljfJVAFpVQ0e6H1Bx86JnvZ37p1fqF9VRayehoajabkSoaGMDQ0hEajgZLP2RPRK6oq3RUll6zQaNasmeQrRfjlPiJ6VfHuKTFZoXHw4MESv//++2+sXr0aXbt21UlRREQVAcc0xGSFhq2treh3cHAwBg8ejCFDhuikMCIifePFd7HnfjX6w4cPkZ6e/jJrISKqUNjTEJMVGtOnTy8xpvHo0SNERUXB09NTZ4UREekbxzTEZIWGvb19id+mpqbw8fFBhw4ddFIUEVFFwLunxGSFRqdOndCqVSvR/L/++gstW7Z86UUREVUE7GmIyRrnGT169FPn+/n5vdRiiIgqEr57SqzUnoZGo4EgCCWmfyQkJMDAwEDnBRIR6UtVCgO5Sg2Nfz/U16xZycfxlUolxo8fr7vKiIj0TFe33AYHB2Pv3r24e/cuIiMj4ejoCADIz8/HZ599hpMnT8LExATOzs6YP38+ACA+Ph4zZ85EZmYm1Go1goOD4eDgoLM2KaW+e+ru3bsQBAEjR47EDz/88L+VFApYWlqiWrVqz3nKnh/fPUVP4runSMqLvnvqo9OHZC/7lWs32ctGR0fD1tYW//3vf7Fy5UptaCxYsABKpRKffPIJFAoFUlNTUatWLQDAqFGjMGjQIHh5eWHbtm3YvHkzvvvuO521SSk1SG1tbWFnZ4c9e/bA2toatra2sLW1Rb169WBgYICCggLZJ4mIqLJRlmEqCxcXF9jY2JSYl5OTg4iICEyZMkV7heefwEhLS8Ply5fh4eEBAPDw8MDly5eRnp6uk7ZnnZNnGjNmDC5dulRi3qVLl/Duu+/KWZ2IqFIqy0B4VlYW7ty5I5qysrJk7SsxMRFqtRqhoaHw9vbGyJEjER0dDQBITk5GnTp1tOPIBgYGsLa2RnJysk7aSiPrlttr166Jbrlt2bIlrl69KutkEBFVRooy3HK7YcMGhIaGiuZPmjQJkydPfub6xcXFSExMRLNmzTBjxgxcuHAB48ePx/79+8tUs67JCg2VSoXU1FTUrl1bOy81NRWmpqY6K4yISN/KcveUr68vBg4cKJqvUqlkrW9jYwNDQ0Pt5aJWrVrBwsIC8fHxqFevHu7fv4/i4mIYGBiguLgYKSkpsLGxgSAIL72t1HMi52B69eqFqVOnIjY2Fnl5ebh27RpmzJiBvn37yjoZRESVUVnGNFQqFezs7EST3NCwtLSEq6srfv/9dwCP72xKS0uDvb09rKys4OTkhB07dgAAduzYAScnJ1haWuqkrTSyvtyXn5+PRYsWYcuWLSgoKICJiQm8vb0xY8YMmJiYyDohLwvvnqIn8e4pkvKid08Fnj0ge9l5bXrIXnbBggXYt28fUlNTYWFhAbVajZ07dyIxMREBAQHIzMyEoaEhPvjgA7i7uwMA4uLiMHPmTGRlZUGlUiE4OBiNGjXSWZuUMn3uVRAEZGRkwMLCAgqFQi9f7mNo0JMYGiTlRUNj3jn5oRHYWn5oVGZl+hv/n+czYmNjERwcjM6dq8Y3cYmoajIow1RVyP6eRnp6OiIjIxEREYGrV6+ibdu2mDVrli5rIyLSK76wUKzU0CgsLMShQ4ewdetWnDhxAg0aNED//v2RlJSEr7/+GlZWVuVVJxFRueO7p8RKDQ03NzcoFAp4e3tj8uTJaN68OQBg06ZN5VIcEZE+MTTESh3TaNKrPFcVAAARaklEQVSkCbKzs3HhwgVcvHgRDx48KK+6iIj0zkgpf6oqSu1pfP/997h79y4iIiKwfv16LFiwAB07dkRubi6KiorKq0YiIr3gmIbYM/PR1tYWEydOxL59+/Dtt9+idu3aUCqV8PT0REhISHnUSESkF/wIk5jsu6eAx29mdHFxwezZs7F//35EREToqi4iIr2rSrfSylWm0PiHiYkJPDw8tO9IISJ6FVWlHoRczxUaRERVgZGSYxpPYmgQEUlgT0OMoUFEJIGhIcbQICKSwNAQY2gQEUkw4HMaIgwNIiIJVehBb9kYGkREEgyZGiIMDSIiCbw8JcbQICKSwIFwMYYGEZEEhoYYQ4OISAJDQ4yhQUQkga8REWNo6MGPG/cgYutRXI9NQL/+blj4+QQAwI7I45j36RrtcoJGwKNHBfj5t8/RvHkj/Hk6BiuWb8aVy/FQqapj38HQEtvt1X0S0tIyoTR4fMuHs3MTrFn3+DvuEVuPYO7slTCpZqxdPmzFDLRv31zXh0vPKS4uEfODVuPSpZuwtFRh2nRf9Oz5Bu7eSUGPHuNgZlZNu+y7fgMxYcJQAMDu3b/juw2RuHo1Hq+/3hjffb9Au1x8/F18sXgDzp27Bo1GgxYtXsOsWX5o2Mi23I+vMuDNU2IMDT2oXdsC48Z74/cTF5CfX6Cd7/FWJ3i81Un7O2LrEaxcsQXNmjUEAJiaVsNA767o198Na1Y9/bX0ocs/xpsdWj61rZWzI77fGPQSj4R0paioGBMnfA4fn95Yt/5TREVdwoT3P8NrW76EsZERAOD0nz/A0FD88u6aNatjlO9buHnzDk6fuliiLTs7B127tcfCzybD3NwUy5f/gokTP8eu3aGi7RAvTz0Ng1QPevZyRfce7aBWVy91uW0RR+Hp1QkKxeM/ua+3fA2eXp1hZ2ddHmWSHt28eQd//50B33c8YWBggDfeaInWrZti+/ajz1y3Q4dW6NvXDdbWlqK2li0dMXhwD6jVNWBkZAhf37cQH38XGRlZujiMSs9AIX+qKtjTqKCS7v6NM9FXMH/h+2Vab8bHoRA0GjR1aoip0/+Lpk0dtG1Xr9xCxzf9ULNmdbzl2Ql+Ywc89V+qVDEJAK5fT9D+7t5tLBQKBTp0aIXpH/vCwkJV5m1GR19GrdoWz7VuVcDPvYrptafx1ltv6XP3Fdr2bcfQpq1TmXoVixZPwr4Dodh3MAztXZth3HufISsrBwDQ1sUJW7d/gWO/r8aSbz7Crl2/I3z9dl2VTy+oYUNbWFrWxLp1ESgsLMLvJ84jOuoSHuXlQ21RA7/+uhgHD63Gb5u/QE5OHqZPW1Lmfdy7l4r5Qasxc8ZoHRzBq4GfexXTeWjcuHFDcsrIyND17iut7duOwWtA5zKt06ZNU1SrZgxTUxO8N3YgVDXMcfbMVQBA/fp1YGdnDaVSCUfHBhj//iDs33taF6XTS2BkZIjQsJk4eiQanTqORnj4NvTp0wF16lrB3NwULV5/DYaGBqhVS43Zc97D77+fR87DPNnbT09/AL9352H4iD7o79Hp2StUUYYK+VNVofPLUx4eHrC1tYUgiLt5mZmZut59pXT27FX8/XcGevV+48U2pMBTzzsAKBQKyTaqGJo0ccD3PyzU/h7uMxMDBnQVLffPmJdG0Mja7oMHD+H37jx07dYO48cPeTnFvqIUVSgM5NJ5aNja2uLHH39EnTp1RG3u7u663n2FVFRUjOLiYhRrNCgu1iA/vwAGBgba8YXtEcfQo2d7mJubllhPo9GgsLAIRUXFEAQB+fkFUCqUMDI2RHJSKpLvpeL1Fq9BI2iw8Yc9yMzIRuvWTQAAx4+dg1OzhqhVS42bN+9i1YrN6NXnBUOJdOratVtwcKgHjUbAph934++/MzDQuxsuXIiFqoY57B1s8ODBQyxcuBbt27dAjRrmAIDi4mLtnzHNP39OlEoYGRni4cNcvOc3D61bN8XUqaP0fIQVHzNDTOeh0atXL9y9e/epodGzZ09d775CWrVyC1aE/ab9vSPyON6fOBgTJw1Bfn4B9u45iSVffyRaLzr6Csb4/u+W2bbOI+HSrhm+/S4QOTl5mD9vHe4k3oexsRGaOjlgxepPoLaoAQA4dSoGswJWIC/3EaysasLjrU54b+xA3R8sPbft247gt98OoKioGG3bOmHd+k9hbGyEO4n3sWTJD0hPfwDz6mbo0KEVvvjyo3+tdxQBAcu0v51bDcOAAV3x+SJ/HNh/Chcv3sCNG4mIiDisXSZyxzeoV692uR5fZcCehphCqGTXKAo15/VdAlUwBgrjZy9EVZJS0eyF1j+bulP2sm1q9X+hfVUWvOWWiEiCgrfcijA0iIgkVKVbaeViaBARSWBmiDE0iIgksKchxndPERFJUJRhKovDhw9jwIAB8PLygqenJ/bt2wcAiI+Px7Bhw9C7d28MGzYMt27d0q6ji7bnwbunqNLj3VMk5UXvnrqcuUP2ss3UHrKWEwQB7du3x8aNG+Ho6IirV69i+PDhOHPmDN555x0MGjQIXl5e2LZtGzZv3ozvvvsOADBq1KiX3vY82NMgIpKgLMNUpu0qlcjOzgYAZGdnw9raGhkZGbh8+TI8PB6Hj4eHBy5fvoz09HSkpaW99LbnxTENIiIJZRnTyMrKQlaW+BXzKpUKKtX/3iKsUCiwdOlSTJgwAWZmZsjJycHq1auRnJyMOnXqwMDg8ZshDAwMYG1tjeTkZAiC8NLbLC3Fr86Xg6FBRCShLGMVGzZsQGio+GNWkyZNwuTJk7W/i4qKsGrVKixfvhxt27bFmTNn8MEHHyAkJOQlVKx7DA0iIgllebjP19cXAweKX83z714GAFy5cgUpKSlo27YtAKBt27YwNTWFiYkJ7t+/j+LiYhgYGKC4uBgpKSmwsbGBIAgvve15cUyDiEhCWe6eUqlUsLOzE01PhkbdunVx79493Lx5EwAQFxeHtLQ02Nvbw8nJCTt2PB5837FjB5ycnGBpaQkrK6uX3vbc54R3T1Flx7unSMqL3j11MztS9rKNasj/qNz27duxZs0a7Wvt/f390aNHD8TFxWHmzJnIysqCSqVCcHAwGjVqBAA6aXseDA2q9BgaJOVFQ+P2Q/mhYV+9anyJlGMaREQS+EC4GEODiEgCv6chxtAgIpLAzBBjaBARSeALC8UYGkREEpgZYgwNIiIJSn65T4ShQUQkgQPhYgwNIiIJzAwxhgYRkQS+Z0mMoUFEJIGXp8QYGkREEhTsa4gwNIiIJCgUDI0nMTSIiCTx+tSTGBpERBIUDA0RhgYRkSSGxpMYGkREEjimIcbQICKSwLunxBgaREQSOKYhxtAgIpLEnsaTGBpERBIUfCRchKFBRCSJofEkhgYRkQSOaYgxNIiIJChgoO8SKhyGBhGRBI5piDE0iIgkMTSexNAgIpLAh/vEGBpERJLY03gSQ4OISALfPSXG0CAiksDLU2IMDSIiSbw89SSGBhGRBD7cJ8bQICKSwOc0xBgaRESSOKbxJIYGEZEEDoSLMTSIiCTw8pQYQ4OISBJ7Gk9SCIIg6LsIIiKqHBijREQkG0ODiIhkY2gQEZFsDA0iIpKNoUFERLIxNIiISDaGBhERycbQICIi2RgaREQkG0OjEoqPj8ewYcPQu3dvDBs2DLdu3dJ3SaRnwcHB6NatG5o0aYLY2Fh9l0OvMIZGJRQYGIgRI0Zg7969GDFiBObOnavvkkjPunfvjo0bN8LW1lbfpdArjqFRyaSlpeHy5cvw8PAAAHh4eODy5ctIT0/Xc2WkTy4uLrCxsdF3GVQFMDQqmeTkZNSpUwcGBgYAAAMDA1hbWyM5OVnPlRFRVcDQICIi2RgalYyNjQ3u37+P4uJiAEBxcTFSUlJ4aYKIygVDo5KxsrKCk5MTduzYAQDYsWMHnJycYGlpqefKiKgq4EeYKqG4uDjMnDkTWVlZUKlUCA4ORqNGjfRdFunRggULsG/fPqSmpsLCwgJqtRo7d+7Ud1n0CmJoEBGRbLw8RUREsjE0iIhINoYGERHJxtAgIiLZGBpERCQbQ4PK3Z07d9CkSRMUFRUBAPz8/LB161ad73fZsmWYNm3aU9tOnz6Nzp07y9rOli1bMHz48Oeq4UXWJaoIDPVdAFVM3bp1Q2pqKgwMDGBqaorOnTtjzpw5MDc3f+n7Wrt2reyaFixYgA4dOrz0GohIHvY0SNLKlStx7tw5bN26FTExMVixYoVoGUEQoNFo9FAdEekDQ4OeqU6dOujUqROuX78OABg5ciSWLFkCHx8ftGrVComJicjOzkZAQAA6duyITp06YcmSJSXejxUcHAxXV1d0794dR48eLbH9kSNH4tdff9X+/uWXX9C3b1+0bt0a/fr1w6VLlzB9+nQkJSVh/PjxaN26NdasWQMAOH/+PHx8fODi4gJPT0+cPn1au53ExES8/fbbaN26NUaPHo2MjAzZx7x69Wr06NFDW8P+/ftLtAuCgKCgILRt2xZ9+vTByZMntW2lnQuiyo6Xp+iZkpOTcezYMfTs2VM7b9u2bVizZg0aNmwIQRDwwQcfwMrKCvv27UNeXh7GjRsHGxsb+Pj44JdffsHhw4cREREBU1NTTJ48WXJfu3fvxrJlyxAWFobXX38dCQkJMDQ0xOLFi3HmzJkSl6fu37+PcePGISQkBJ06dcLJkyfh7++P3bt3w9LSEtOmTYOzszPWr1+PCxcuYOzYsejevbusY65fvz42btyI2rVrY8+ePZg+fTr27dsHa2trAMBff/2FPn364NSpU9i/fz8mTZqEgwcPQq1WY+bMmZLngqiyY0+DJE2cOBEuLi4YMWIE2rVrh/Hjx2vbBg4ciMaNG8PQ0BAPHjzA0aNHERAQADMzM1hZWeGdd97Rvvto9+7d8PX1hY2NDdRqNcaNGye5z99++w1+fn5o2bIlFAoF7O3tJb9Gt23bNnTu3Bnu7u5QKpVwc3NDixYtcPToUSQlJeHixYuYMmUKjI2N0a5dO3Tr1k32sfft2xd16tSBUqlEv379YG9vj7/++kvbbmlpCV9fXxgZGaFfv35o2LAhjhw5gtTU1FLPBVFlx54GSQoLC5McdP73q9iTkpJQVFSEjh07audpNBrtMk++ur1evXqS+0xOTkaDBg1k1ZeUlIQ9e/bg8OHD2nlFRUVwdXVFSkoKVCoVzMzMSuxX7seqIiIiEB4ejrt37wIAcnNzS1zeqlOnDhQKRYltp6SkPPNcEFV2DA16Lv/+C7Nu3bowNjbGqVOnYGgo/iNVu3btEn9Zl/YXt42NDRISEmTVYGNjAy8vLyxYsEDUdvfuXWRlZSE3N1cbHElJSSXqlnL37l3Mnj0b3377LVq3bg0DAwN4eXmVWOb+/fsQBEG7veTkZHTr1u2Z54KosuPlKXph1tbWcHNzw6JFi/Dw4UNoNBokJCTgzz//BPD4Us/333+Pe/fu4cGDB1i9erXktgYPHoz169cjJiYGgiDg9u3b2n/t16pVC4mJidplPT09cfjwYRw/fhzFxcXIz8/H6dOnce/ePdja2qJFixZYtmwZCgoKEB0dXaJHUpq8vDwoFArtN0o2b96svQngH+np6fjuu+9QWFiI3bt3Iy4uDu7u7s88F0SVHUODXoqQkBAUFhaiX79+aNeuHfz9/fH3338DAIYOHYqOHTvCy8sLAwcORK9evSS307dvX4wfPx5Tp05FmzZtMHHiRDx48AAAMHbsWKxYsQIuLi5Yt24dbGxssHz5cqxatQpvvvkm3N3dsW7dOu0twF9++SUuXLgAV1dXhIWFYcCAAbKO5bXXXsOYMWPg4+ODDh06IDY2Fm3atCmxTMuWLXH79m288cYbWLp0Kb755htYWFg881wQVXb8ngYREcnGngYREcnG0CAiItkYGkREJBtDg4iIZGNoEBGRbAwNIiKSjaFBRESyMTSIiEg2hgYREcn2f7tGezXa49CZAAAAAElFTkSuQmCC\n"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Print the overall accuracy\nprint(metrics.accuracy_score(y_test,predictions))","execution_count":22,"outputs":[{"output_type":"stream","text":"0.9515916857879314\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(metrics.classification_report(y_test,predictions))","execution_count":24,"outputs":[{"output_type":"stream","text":" precision recall f1-score support\n\n 0 0.96 0.99 0.97 404354\n 1 0.72 0.36 0.48 26667\n\n micro avg 0.95 0.95 0.95 431021\n macro avg 0.84 0.67 0.73 431021\nweighted avg 0.94 0.95 0.94 431021\n\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"# Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test['prediction'] = text_clf_lgbm.predict(df_test['question_text'])\ndf_test.head()","execution_count":27,"outputs":[{"output_type":"execute_result","execution_count":27,"data":{"text/plain":" question_text prediction\nqid \n0000163e3ea7c7a74cd7 Why do so many women become so rude and arroga... 1\n00002bd4fb5d505b9161 When should I apply for RV college of engineer... 0\n00007756b4a147d2b0b3 What is it really like to be a nurse practitio... 0\n000086e4b7e1c7146103 Who are entrepreneurs? 0\n0000c4c3fbe8785a3090 Is education really making good people nowadays? 0","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
question_textprediction
qid
0000163e3ea7c7a74cd7Why do so many women become so rude and arroga...1
00002bd4fb5d505b9161When should I apply for RV college of engineer...0
00007756b4a147d2b0b3What is it really like to be a nurse practitio...0
000086e4b7e1c7146103Who are entrepreneurs?0
0000c4c3fbe8785a3090Is education really making good people nowadays?0
\n
"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test.drop(['question_text'],axis=1,inplace=True)\ndf_test.to_csv('submission.csv',index=True)","execution_count":28,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test.head()","execution_count":30,"outputs":[{"output_type":"execute_result","execution_count":30,"data":{"text/plain":" prediction\nqid \n0000163e3ea7c7a74cd7 1\n00002bd4fb5d505b9161 0\n00007756b4a147d2b0b3 0\n000086e4b7e1c7146103 0\n0000c4c3fbe8785a3090 0","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
prediction
qid
0000163e3ea7c7a74cd71
00002bd4fb5d505b91610
00007756b4a147d2b0b30
000086e4b7e1c71461030
0000c4c3fbe8785a30900
\n
"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"A Big Thanks to:\n\n* https://medium.com/greyatom/an-introduction-to-bag-of-words-in-nlp-ac967d43b428\n* https://medium.com/@paritosh_30025/natural-language-processing-text-data-vectorization-af2520529cf7\n\n\n\n![](https://www.humorside.com/wp-content/uploads/2017/12/thank-you-meme-01.jpg)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}