{"cells":[{"metadata":{"_uuid":"e47548e57a6db7d7819e35734bdd64a5aebbf9b3"},"cell_type":"markdown","source":"I fell in love with Quora when I first used it probably 6-7 years ago. How wonderful, a website for authors and experts to share ideas worth talking about and answering questions. It wasn't too long before toxic content began to show up in the platform.\n\nThe moderators of the website had made a phenomenal product, and early users were dedicated ot the platform. They helped point out this content as negative/toxic and steered users clear of it.\n\nI knew it wouldn't be too long before things started to change...\n\n\"I'm a 14 year old PhD student who just got offers from FB and Google for $500k/yr - Why am I unhappy?\"\n\nSeveral years ago, this was the type of content that plagued the website. People seeking attention for a myriad of reasons. Before the culmination of this Fake News problem we have taken on at scale, toxic content was much easier to identify: someone just wanted attention and crafted a projectile question that would maximize interest before shooting it like a harpoon into the Quora platform.\n\nNowadays, the evolution of this content has taken a much nastier turn: In today's world many people receive tremendous amounts of information constantly, and small snippets of information slowly shift the aggregate human thought of populations every day. If done with the intent of informing the public of genuine information, that's likely a positive thing - if the intent is to derail human progress or to misinform populations, the result is likely destructive.\n\nThis is certainly a problem worth fighting, and social media platforms are the primary battleground.\n\nThis notebook is an attempt for me to discover different approaches to vectorizing language such that we can identify fake news. This is my first attempt at NLP - so here goes!"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom sklearn.model_selection import train_test_split\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"#Let's take a look at the dataset\n\ntrain = pd.read_csv('../input/train.csv')\ntest = pd.read_csv('../input/test.csv')\nprint(train.head(5))\n\nprint(\"The Columns are: \" + train.columns)\n\n#Let's see what a toxic question looks like\npd.set_option('display.max_colwidth', -1)\nprint(\"Toxic Questions: \" + train.loc[train['target']==1,:]['question_text'].head(5))\n\n#How many of the questions are toxic?\nfake = train.loc[train['target']==1,:]\n\nperc_fake= round(len(fake)/len(train)*100,4)\nprint(\"How many of the questions are toxic: \" + str(perc_fake) + \"%\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"97e56c5778d283cdf7e3683435eb81c68f5b01d3"},"cell_type":"code","source":"train.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cb1edaa4a4de28d564f3af3d7d1e5b86fe88c7ab"},"cell_type":"code","source":"\nimport pandas as pd\nimport numpy as np\nfrom tqdm import tqdm\ntqdm.pandas()\nfrom sklearn.model_selection import train_test_split, StratifiedKFold\nfrom sklearn import metrics\n\nfrom keras.preprocessing.text import Tokenizer\nfrom keras.preprocessing.sequence import pad_sequences\nfrom keras.layers import *\nfrom keras.models import *\nfrom keras import initializers, regularizers, constraints, optimizers, layers\nfrom keras.initializers import *\nfrom keras.optimizers import *\nimport keras.backend as K\nfrom keras.callbacks import *\nimport tensorflow as tf\nimport os\nimport time\nimport gc\nimport re\nfrom unidecode import unidecode","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"86cef5925dcf2ae6b75535b0f8fec8b110660a9e"},"cell_type":"code","source":"# This turns all question text into lower case letters\ntrain[\"question_text\"] = train[\"question_text\"].str.lower()\ntest[\"question_text\"] = test[\"question_text\"].str.lower()\n\n# This is all of the punctuations in a list,\n# we will use this to make punctuation easier\n# to read for the NN later.\npuncts = [',', '.', '\"', ':', ')', '(', '-', '!', '?', '|', ';', \"'\", '$', '&', '/', '[', ']', '>', '%', '=', '#', '*', '+', '\\\\', '•',  '~', '@', '£', \n '·', '_', '{', '}', '©', '^', '®', '`',  '<', '→', '°', '€', '™', '›',  '♥', '←', '×', '§', '″', '′', 'Â', '█', '½', 'à', '…', \n '“', '★', '”', '–', '●', 'â', '►', '−', '¢', '²', '¬', '░', '¶', '↑', '±', '¿', '▾', '═', '¦', '║', '―', '¥', '▓', '—', '‹', '─', \n '▒', '：', '¼', '⊕', '▼', '▪', '†', '■', '’', '▀', '¨', '▄', '♫', '☆', 'é', '¯', '♦', '¤', '▲', 'è', '¸', '¾', 'Ã', '⋅', '‘', '∞', \n '∙', '）', '↓', '、', '│', '（', '»', '，', '♪', '╩', '╚', '³', '・', '╦', '╣', '╔', '╗', '▬', '❤', 'ï', 'Ø', '¹', '≤', '‡', '√', ]\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b518fbe2fb684b369315aa471737d4e8ab88e3fe"},"cell_type":"code","source":"# This is where we allocate space around the punctuation\ndef clean_text(x):\n\n    x = str(x)\n    for punct in puncts:\n        x = x.replace(punct, f' {punct} ')\n    return x\n\n\ntrain[\"question_text\"] = train[\"question_text\"].apply(lambda x: clean_text(x))\ntest[\"question_text\"] = test[\"question_text\"].apply(lambda x: clean_text(x))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d7e1f99621224c41951d501a9a1046dd23fb9eff"},"cell_type":"code","source":"## some config values \nembed_size = 300 # how big is each word vector\nmax_features = None # how many unique words to use (i.e num rows in embedding vector)\nmaxlen = 72 # max number of words in a question to use #99.99%\n\n## fill up the missing values\nX = train[\"question_text\"].fillna(\"_na_\").values\nX_test = test[\"question_text\"].fillna(\"_na_\").values\n\n## Tokenize the sentences\ntokenizer = Tokenizer(num_words=max_features, filters='')\ntokenizer.fit_on_texts(list(X))\n\nX = tokenizer.texts_to_sequences(X)\nX_test = tokenizer.texts_to_sequences(X_test)\n\n## Pad the sentences \nX = pad_sequences(X, maxlen=maxlen)\nX_test = pad_sequences(X_test, maxlen=maxlen)\n\n## Get the target values\nY = train['target'].values\n\nsub = test[['qid']]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8b39af3a83418eeac711b0cf06d74b6682f51073"},"cell_type":"code","source":"for x in range((len(train)-1)):\n    if x==0: highest_val=len(train.question_text[0])\n    highest_val = (max(highest_val,len(train.question_text[x+1])))\n    \nprint(highest_val)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3364b4839e221cb0fe5d69d87792a489112cc108"},"cell_type":"code","source":"del train, test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4a8860c724daa62bff7cec27721ad4c3fd95a433"},"cell_type":"code","source":"word_index = tokenizer.word_index\nmax_features = len(word_index)+1\ndef load_glove(word_index):\n    EMBEDDING_FILE = '../input/embeddings/glove.840B.300d/glove.840B.300d.txt'\n    def get_coefs(word,*arr): return word, np.asarray(arr, dtype='float32')\n    embeddings_index = dict(get_coefs(*o.split(\" \")) for o in open(EMBEDDING_FILE) if o.split(\" \")[0] in word_index)\n\n    all_embs = np.stack(embeddings_index.values())\n    emb_mean,emb_std = all_embs.mean(), all_embs.std()\n    embed_size = all_embs.shape[1]\n\n    embedding_matrix = np.random.normal(emb_mean, emb_std, (max_features, embed_size))\n    for word, i in word_index.items():\n        if i >= max_features: continue\n        embedding_vector = embeddings_index.get(word)\n        if embedding_vector is not None: embedding_matrix[i] = embedding_vector\n            \n    return embedding_matrix \n    \ndef load_fasttext(word_index):    \n    EMBEDDING_FILE = '../input/embeddings/wiki-news-300d-1M/wiki-news-300d-1M.vec'\n    def get_coefs(word,*arr): return word, np.asarray(arr, dtype='float32')\n    embeddings_index = dict(get_coefs(*o.split(\" \")) for o in open(EMBEDDING_FILE) if len(o)>100 and o.split(\" \")[0] in word_index )\n    all_embs = np.stack(embeddings_index.values())\n    emb_mean,emb_std = all_embs.mean(), all_embs.std()\n    embed_size = all_embs.shape[1]\n\n    embedding_matrix = np.random.normal(emb_mean, emb_std, (max_features, embed_size))\n    for word, i in word_index.items():\n        if i >= max_features: continue\n        embedding_vector = embeddings_index.get(word)\n        if embedding_vector is not None: embedding_matrix[i] = embedding_vector\n\n    return embedding_matrix\n\ndef load_para(word_index):\n    EMBEDDING_FILE = '../input/embeddings/paragram_300_sl999/paragram_300_sl999.txt'\n    def get_coefs(word,*arr): return word, np.asarray(arr, dtype='float32')\n    embeddings_index = dict(get_coefs(*o.split(\" \")) for o in open(EMBEDDING_FILE, encoding=\"utf8\", errors='ignore') if len(o)>100 and o.split(\" \")[0] in word_index)\n\n    all_embs = np.stack(embeddings_index.values())\n    emb_mean,emb_std = all_embs.mean(), all_embs.std()\n    embed_size = all_embs.shape[1]\n    \n    embedding_matrix = np.random.normal(emb_mean, emb_std, (max_features, embed_size))\n    for word, i in word_index.items():\n        if i >= max_features: continue\n        embedding_vector = embeddings_index.get(word)\n        if embedding_vector is not None: embedding_matrix[i] = embedding_vector\n    \n    return embedding_matrix","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"41039782459437b3a70deeab561f598e33391b84"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}