{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n### Table of Contents\n\n* [Basic Overview ๐บ](#2)\n\n* [Exploratory Data Analysis ๐](#3)\n\n* [Modelling and PreProcessing๐ฑโ๐](#4)\n\n * [Bidirectional LSTM](#4.1)\n * [Bidirectional GRU](#4.2)","metadata":{}},{"cell_type":"markdown","source":"### Problem Statement","metadata":{}},{"cell_type":"markdown","source":"An insincere question is defined as a question intended to make a statement rather than look for helpful answers. Some characteristics that can signify that a question is insincere:\n\n* Has a non-neutral tone\n * Has an exaggerated tone to underscore a point about a group of people\n * Is rhetorical and meant to imply a statement about a group of people\n \n* Is disparaging or inflammatory\n * Suggests a discriminatory idea against a protected class of people, or seeks confirmation of a stereotype\n * Makes disparaging attacks/insults against a specific person or group of people\n * Based on an outlandish premise about a group of people\n * Disparages against a characteristic that is not fixable and not measurable\n \n* Isn't grounded in reality\n * Based on false information, or contains absurd assumptions\n \n* Uses sexual content (incest, bestiality, pedophilia) for shock value, and not to seek genuine answers","metadata":{}},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Basic Overview ๐บ","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport re\nimport string\nimport nltk","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:20:54.424486Z","iopub.execute_input":"2022-07-24T08:20:54.425270Z","iopub.status.idle":"2022-07-24T08:20:55.866983Z","shell.execute_reply.started":"2022-07-24T08:20:54.425177Z","shell.execute_reply":"2022-07-24T08:20:55.866056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = pd.read_csv(\"../input/quora-insincere-questions-classification/train.csv\")\ndf_test = pd.read_csv(\"../input/quora-insincere-questions-classification/test.csv\")\nsample = pd.read_csv(\"../input/quora-insincere-questions-classification/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:20:55.868888Z","iopub.execute_input":"2022-07-24T08:20:55.869396Z","iopub.status.idle":"2022-07-24T08:21:01.270206Z","shell.execute_reply.started":"2022-07-24T08:20:55.869355Z","shell.execute_reply":"2022-07-24T08:21:01.269250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train1 = df_train.copy( ) # For Visualization purpose","metadata":{"execution":{"iopub.status.busy":"2022-07-24T07:55:38.700649Z","iopub.execute_input":"2022-07-24T07:55:38.702601Z","iopub.status.idle":"2022-07-24T07:55:38.758092Z","shell.execute_reply.started":"2022-07-24T07:55:38.702568Z","shell.execute_reply":"2022-07-24T07:55:38.757113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"***** Shape of Training dataset is *****\", df_train.shape)\nprint()\ndf_train.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T17:18:41.777252Z","iopub.execute_input":"2022-07-23T17:18:41.777659Z","iopub.status.idle":"2022-07-23T17:18:41.802532Z","shell.execute_reply.started":"2022-07-23T17:18:41.777627Z","shell.execute_reply":"2022-07-23T17:18:41.801206Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"***** Shape of Test Dataset *****\",df_test.shape)\nprint()\ndf_test.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T17:18:44.964462Z","iopub.execute_input":"2022-07-23T17:18:44.964822Z","iopub.status.idle":"2022-07-23T17:18:44.978147Z","shell.execute_reply.started":"2022-07-23T17:18:44.964792Z","shell.execute_reply":"2022-07-23T17:18:44.976957Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T17:18:48.585778Z","iopub.execute_input":"2022-07-23T17:18:48.586271Z","iopub.status.idle":"2022-07-23T17:18:48.937053Z","shell.execute_reply.started":"2022-07-23T17:18:48.586238Z","shell.execute_reply":"2022-07-23T17:18:48.935812Z"},"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T17:18:52.206627Z","iopub.execute_input":"2022-07-23T17:18:52.207083Z","iopub.status.idle":"2022-07-23T17:18:52.324636Z","shell.execute_reply.started":"2022-07-23T17:18:52.207047Z","shell.execute_reply":"2022-07-23T17:18:52.323542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"corpus = \" \"\nfor i in range(0,100):\n \n corpus += \" |||| \"+ df_train['question_text'][i]\n \nprint(\"***** First five sentences in training dataset *****\")\nprint()\nprint(corpus)","metadata":{"execution":{"iopub.status.busy":"2022-07-23T17:18:55.608325Z","iopub.execute_input":"2022-07-23T17:18:55.610427Z","iopub.status.idle":"2022-07-23T17:18:55.617173Z","shell.execute_reply.started":"2022-07-23T17:18:55.610377Z","shell.execute_reply":"2022-07-23T17:18:55.616059Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate to Top](#0)\n\n### Exploratory Data Analysis ๐","metadata":{}},{"cell_type":"code","source":"def without_hue(data,feature,ax):\n \n total=float(len(data))\n bars_plot=ax.patches\n \n for bars in bars_plot:\n percentage = '{:.1f}%'.format(100 * bars.get_height()/total)\n x = bars.get_x() + bars.get_width()/2.0\n y = bars.get_height()\n ax.text(x, y,(percentage,bars.get_height()),ha='center',fontweight='bold',fontsize=10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-23T07:16:08.071830Z","iopub.execute_input":"2022-07-23T07:16:08.072529Z","iopub.status.idle":"2022-07-23T07:16:08.080286Z","shell.execute_reply.started":"2022-07-23T07:16:08.072465Z","shell.execute_reply":"2022-07-23T07:16:08.079472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(8,5))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='x',color='gray')\n\na=sns.countplot(data=df_train,x='target')\nwithout_hue(df_train,'target',a)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-23T07:16:11.715110Z","iopub.execute_input":"2022-07-23T07:16:11.715525Z","iopub.status.idle":"2022-07-23T07:16:12.041760Z","shell.execute_reply.started":"2022-07-23T07:16:11.715474Z","shell.execute_reply":"2022-07-23T07:16:12.040554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def length(text):\n length = len(text.split(\" \"))\n return(length)\n\ndf_train1['length'] = df_train1['question_text'].apply(length)\n\ndf_train1.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T07:16:15.974145Z","iopub.execute_input":"2022-07-23T07:16:15.974576Z","iopub.status.idle":"2022-07-23T07:16:17.938336Z","shell.execute_reply.started":"2022-07-23T07:16:15.974537Z","shell.execute_reply":"2022-07-23T07:16:17.937261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(15,8))\nax = plt.axes()\nax.set_facecolor(\"#F2EDD7FF\")\nfig.patch.set_facecolor(\"#F2EDD7FF\")\n\nax.spines['top'].set_visible(False)\nax.spines['right'].set_visible(False)\nax.spines['left'].set_visible(False)\nax.grid(linestyle=\"--\",axis='y',color='gray')\n\na=sns.histplot(data=df_train1,x='length',hue='target',kde=True,binwidth=2,multiple='stack')\n","metadata":{"execution":{"iopub.status.busy":"2022-07-23T07:16:22.411094Z","iopub.execute_input":"2022-07-23T07:16:22.411459Z","iopub.status.idle":"2022-07-23T07:16:28.364754Z","shell.execute_reply.started":"2022-07-23T07:16:22.411430Z","shell.execute_reply":"2022-07-23T07:16:28.363701Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from nltk.corpus import stopwords\nfrom nltk.stem import WordNetLemmatizer\n\ndef remove_stopwords(text):\n text = str(text).lower()\n text = text.split(\" \")\n lemma = WordNetLemmatizer()\n text = [lemma.lemmatize(i) for i in text if i not in set(stopwords.words('english'))]\n text = \" \".join(text)\n return(text)\n \n\ndf_train1['question_text'] = df_train1['question_text'].apply(remove_stopwords)\n\n\n\n","metadata":{"execution":{"iopub.status.busy":"2022-07-23T17:19:03.882721Z","iopub.execute_input":"2022-07-23T17:19:03.883246Z","iopub.status.idle":"2022-07-23T18:08:22.156778Z","shell.execute_reply.started":"2022-07-23T17:19:03.883211Z","shell.execute_reply":"2022-07-23T18:08:22.155365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_sin = df_train1.loc[df_train1['target']==0]\ndf_train_insin = df_train1.loc[df_train1['target']==1]\n","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:08:22.159795Z","iopub.execute_input":"2022-07-23T18:08:22.160190Z","iopub.status.idle":"2022-07-23T18:08:22.270494Z","shell.execute_reply.started":"2022-07-23T18:08:22.160154Z","shell.execute_reply":"2022-07-23T18:08:22.269229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_sin.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:08:22.271751Z","iopub.execute_input":"2022-07-23T18:08:22.272134Z","iopub.status.idle":"2022-07-23T18:08:22.286599Z","shell.execute_reply.started":"2022-07-23T18:08:22.272106Z","shell.execute_reply":"2022-07-23T18:08:22.285274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_insin.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:08:22.289690Z","iopub.execute_input":"2022-07-23T18:08:22.291125Z","iopub.status.idle":"2022-07-23T18:08:22.302509Z","shell.execute_reply.started":"2022-07-23T18:08:22.291068Z","shell.execute_reply":"2022-07-23T18:08:22.300577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_count(dataframe):\n dic = {}\n for i in dataframe['question_text']:\n for j in i.split(\" \"):\n if j not in dic:\n dic[j]=1\n else:\n dic[j]+=1\n \n dic = sorted(dic.items() , key = lambda x:x[1],reverse=True)\n return(dic)\n\n\ndic_sin = get_count(df_train_sin)\ndic_insin = get_count(df_train_insin)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:20.493861Z","iopub.execute_input":"2022-07-23T18:14:20.494252Z","iopub.status.idle":"2022-07-23T18:14:27.803326Z","shell.execute_reply.started":"2022-07-23T18:14:20.494222Z","shell.execute_reply":"2022-07-23T18:14:27.801444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dic_sin_top = []\ndic_sin_topcount = []\ndic_insin_top = []\ndic_insin_topcount = []\n\ncount = 0\nfor i in dic_sin:\n if(count<50):\n dic_sin_top.append(i[0])\n dic_sin_topcount.append(i[1])\n else:\n break\n \nfor i in dic_insin:\n if(count<50):\n dic_insin_top.append(i[0])\n dic_insin_topcount.append(i[1])\n else:\n break\n\n \n \n","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:27.806078Z","iopub.execute_input":"2022-07-23T18:14:27.806504Z","iopub.status.idle":"2022-07-23T18:14:28.139710Z","shell.execute_reply.started":"2022-07-23T18:14:27.806470Z","shell.execute_reply":"2022-07-23T18:14:28.138279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(20,10))\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor i in range(0,2):\n ax[i].set_facecolor('#F2EDD7FF')\n ax[i].spines['top'].set_visible(False)\n ax[i].spines['right'].set_visible(False)\n ax[i].spines['left'].set_visible(False)\n ax[i].grid(linestyle=\"--\",axis='y',color='gray')\n\n \n\nsns.barplot(y=dic_sin_top[0:50],x=dic_sin_topcount[0:50],ax=ax[0])\nsns.barplot(y=dic_insin_top[0:50],x=dic_insin_topcount[0:50],ax=ax[1])\nax[0].set_title(\"Top words used in sincere questions\")\nax[1].set_title(\"Top words used in insincere questions\")","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:28.141749Z","iopub.execute_input":"2022-07-23T18:14:28.142114Z","iopub.status.idle":"2022-07-23T18:14:29.946617Z","shell.execute_reply.started":"2022-07-23T18:14:28.142082Z","shell.execute_reply":"2022-07-23T18:14:29.945472Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from nltk import ngrams\n\ndef getting_bigrams(dataframe):\n \n dic = {}\n for i in dataframe['question_text']:\n bigrams = ngrams(i.split(\" \"),2)\n for j in bigrams:\n if j not in dic:\n dic[j]=1\n else:\n dic[j]+=1\n \n dic = sorted(dic.items(),key= lambda x:x[1],reverse=True)\n \n return(dic)\n \ndic_sin_bigrams = getting_bigrams(df_train_sin)\ndic_insin_bigrams = getting_bigrams(df_train_insin)","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:34.369052Z","iopub.execute_input":"2022-07-23T18:14:34.369446Z","iopub.status.idle":"2022-07-23T18:14:51.457572Z","shell.execute_reply.started":"2022-07-23T18:14:34.369414Z","shell.execute_reply":"2022-07-23T18:14:51.456617Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dic_sin_topbigrams = []\ndic_sin_topcountbigrams = []\ndic_insin_topbigrams = []\ndic_insin_topcountbigrams = []\n\ncount = 0\nfor i in dic_sin_bigrams:\n count+=1\n if(count<50):\n dic_sin_topbigrams.append(i[0])\n dic_sin_topcountbigrams.append(i[1])\n else:\n break\n\ncount1 = 0\nfor i in dic_insin_bigrams:\n count1+=1\n if(count1<50):\n dic_insin_topbigrams.append(i[0])\n dic_insin_topcountbigrams.append(i[1])\n else:\n break\n","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:51.458984Z","iopub.execute_input":"2022-07-23T18:14:51.459359Z","iopub.status.idle":"2022-07-23T18:14:51.466924Z","shell.execute_reply.started":"2022-07-23T18:14:51.459334Z","shell.execute_reply":"2022-07-23T18:14:51.465033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(0,len(dic_sin_topbigrams)):\n dic_insin_topbigrams[i] = ' '.join(dic_insin_topbigrams[i])\n dic_sin_topbigrams[i] = ' '.join(dic_sin_topbigrams[i])","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:51.469043Z","iopub.execute_input":"2022-07-23T18:14:51.469638Z","iopub.status.idle":"2022-07-23T18:14:51.481253Z","shell.execute_reply.started":"2022-07-23T18:14:51.469606Z","shell.execute_reply":"2022-07-23T18:14:51.480325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(22,15))\nfig.patch.set_facecolor('#F2EDD7FF')\n\nfor i in range(0,2):\n ax[i].set_facecolor('#F2EDD7FF')\n ax[i].spines['top'].set_visible(False)\n ax[i].spines['right'].set_visible(False)\n ax[i].spines['left'].set_visible(False)\n ax[i].grid(linestyle=\"--\",axis='y',color='gray')\n\n \n\nsns.barplot(y=dic_sin_topbigrams[0:50],x=dic_sin_topcountbigrams[0:50],ax=ax[0])\nsns.barplot(y=dic_insin_topbigrams[0:50],x=dic_insin_topcountbigrams[0:50],ax=ax[1])\nax[0].set_title(\"Top bigrams used in sincere questions\")\nax[1].set_title(\"Top bigrams used in insincere questions\")","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:51.483126Z","iopub.execute_input":"2022-07-23T18:14:51.483530Z","iopub.status.idle":"2022-07-23T18:14:52.737814Z","shell.execute_reply.started":"2022-07-23T18:14:51.483506Z","shell.execute_reply":"2022-07-23T18:14:52.736409Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from wordcloud import WordCloud\n\ndf_sin_para = \" \".join([word for word in df_train_sin['question_text']])\ndf_insin_para = \" \".join([word for word in df_train_insin['question_text']])\n","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:52.739484Z","iopub.execute_input":"2022-07-23T18:14:52.739938Z","iopub.status.idle":"2022-07-23T18:14:53.253574Z","shell.execute_reply.started":"2022-07-23T18:14:52.739890Z","shell.execute_reply":"2022-07-23T18:14:53.252309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wordcloud=WordCloud(width=2000,height=1000,background_color='#F2EDD7FF').generate(df_sin_para)\n\nplt.figure(figsize=(20,30))\nplt.imshow(wordcloud)\nplt.title(\"Quora Sincere Questions Word Cloud\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:14:53.255034Z","iopub.execute_input":"2022-07-23T18:14:53.255331Z","iopub.status.idle":"2022-07-23T18:16:41.807814Z","shell.execute_reply.started":"2022-07-23T18:14:53.255302Z","shell.execute_reply":"2022-07-23T18:16:41.801149Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wordcloud=WordCloud(width=2000,height=1000,background_color='#F2EDD7FF').generate(df_insin_para)\n\nplt.figure(figsize=(20,30))\nplt.imshow(wordcloud)\nplt.title(\"Quora Insincere Questions Word Cloud\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-23T18:16:41.810770Z","iopub.execute_input":"2022-07-23T18:16:41.811446Z","iopub.status.idle":"2022-07-23T18:17:16.621723Z","shell.execute_reply.started":"2022-07-23T18:16:41.811400Z","shell.execute_reply":"2022-07-23T18:17:16.615290Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate To Top](#0)\n\n### Modelling and Preprocessing ๐ฑโ๐","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.preprocessing.text import Tokenizer\nfrom tensorflow.keras.preprocessing.sequence import pad_sequences\nfrom tensorflow.keras.layers import LSTM , RNN , Dense , Flatten , Embedding , Bidirectional , SimpleRNN , Dropout , GlobalMaxPool1D , GRU\nfrom tensorflow.keras.models import Sequential\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import accuracy_score , classification_report , f1_score , roc_auc_score\nfrom tensorflow.keras.callbacks import ModelCheckpoint , ReduceLROnPlateau","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:14:11.673219Z","iopub.execute_input":"2022-07-24T09:14:11.673769Z","iopub.status.idle":"2022-07-24T09:14:11.681708Z","shell.execute_reply.started":"2022-07-24T09:14:11.673728Z","shell.execute_reply":"2022-07-24T09:14:11.680551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train , val = train_test_split(df_train,test_size=0.2,random_state=42)\n\ndimension_size = 300\nvocabulary_size = 50000\nmax_length = 100\n\ntrain_X = train['question_text']\nval_X = val['question_text']\ntest_X = df_test['question_text']\n\ntokenizer = Tokenizer(num_words=vocabulary_size) #Initiating Tokenizer\n\nfit_text = list(train['question_text']) #getting list of train text\ntokenizer.fit_on_texts(fit_text) #fitting on text\n\ntrain_X = tokenizer.texts_to_sequences(train_X) #Converting tokens of texts into sequence of integers\nval_X = tokenizer.texts_to_sequences(val_X)\ntest_X = tokenizer.texts_to_sequences(test_X)\n\n\ntrain_X = pad_sequences(train_X, maxlen=max_length) #Padding the text to make length of each text uniform\nval_X = pad_sequences(val_X, maxlen=max_length)\ntest_X = pad_sequences(test_X, maxlen=max_length)\n\ntrain_Y = train['target']\nval_Y = val['target']","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:21:16.092547Z","iopub.execute_input":"2022-07-24T08:21:16.093171Z","iopub.status.idle":"2022-07-24T08:22:11.429053Z","shell.execute_reply.started":"2022-07-24T08:21:16.093142Z","shell.execute_reply":"2022-07-24T08:22:11.428080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(tokenizer.word_index))","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:22:11.430905Z","iopub.execute_input":"2022-07-24T08:22:11.431281Z","iopub.status.idle":"2022-07-24T08:22:11.437200Z","shell.execute_reply.started":"2022-07-24T08:22:11.431242Z","shell.execute_reply":"2022-07-24T08:22:11.436191Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glove_embedded = \"../input/glove6b300dtxt/glove.6B.300d.txt\" ##\"../input/glove840b300dtxt/glove.840B.300d.txt\"\n\n\ndef getting_glove_embedding(filename): #Getting glove embedding in a dictionary\n file = open(filename,'r')\n lines = file.readlines()\n embedding = dict()\n for line in lines:\n parts = line.split()\n embedding[parts[0]] = np.asarray(parts[1:],dtype='float32')\n \n return(embedding)\n \n\ndef embedding_weights(vocab,raw_embedding): #raw_embedding = getting_glove_embedding(filename)\n \n vocab_size = len(vocab)+1\n weight_matrix = np.zeros((vocab_size,300)) #each word with 300 dimensions\n \n for word , i in vocab.items():\n vector = raw_embedding.get(word)\n if vector is not None:\n weight_matrix[i] = vector\n \n return(weight_matrix)\n \n \nraw_embedding = getting_glove_embedding(glove_embedded)\nprint(\"<<<<<<< Raw Embedding Loaded >>>>>>>\")\nembedding_vectors = embedding_weights(tokenizer.word_index,raw_embedding)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:22:11.438758Z","iopub.execute_input":"2022-07-24T08:22:11.439451Z","iopub.status.idle":"2022-07-24T08:22:53.631343Z","shell.execute_reply.started":"2022-07-24T08:22:11.439415Z","shell.execute_reply":"2022-07-24T08:22:53.630358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"embedding_vectors.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:22:53.634216Z","iopub.execute_input":"2022-07-24T08:22:53.634571Z","iopub.status.idle":"2022-07-24T08:22:53.642871Z","shell.execute_reply.started":"2022-07-24T08:22:53.634536Z","shell.execute_reply":"2022-07-24T08:22:53.641998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate To Top](#0)\n\n#### Bidirectional LSTM","metadata":{}},{"cell_type":"code","source":"embedding_layer = Embedding(len(tokenizer.word_index)+1,300,weights=[embedding_vectors])\nmodel = Sequential()\nmodel.add(embedding_layer)\nmodel.add(Bidirectional(LSTM(64,return_sequences=True)))\nmodel.add(GlobalMaxPool1D())\nmodel.add(Dense(16,activation='relu'))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(1,activation='sigmoid'))\n\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:43:01.062052Z","iopub.execute_input":"2022-07-24T08:43:01.062591Z","iopub.status.idle":"2022-07-24T08:43:01.805787Z","shell.execute_reply.started":"2022-07-24T08:43:01.062540Z","shell.execute_reply":"2022-07-24T08:43:01.804758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.compile(loss='binary_crossentropy',optimizer='Adam',metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:43:05.482572Z","iopub.execute_input":"2022-07-24T08:43:05.483435Z","iopub.status.idle":"2022-07-24T08:43:05.493893Z","shell.execute_reply.started":"2022-07-24T08:43:05.483387Z","shell.execute_reply":"2022-07-24T08:43:05.493010Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ncheckpoint = ModelCheckpoint(\n 'model.h5', \n monitor = 'val_acc', \n verbose = 1, \n save_best_only = True\n)\nreduce_lr = ReduceLROnPlateau(\n monitor = 'val_loss', \n factor = 0.2, \n verbose = 1, \n patience = 5, \n min_lr = 0.001\n)\n\n\nhistory=model.fit(train_X,train_Y,validation_data=(val_X,val_Y),epochs=5,batch_size=512,callbacks=[checkpoint,reduce_lr])","metadata":{"execution":{"iopub.status.busy":"2022-07-24T08:43:08.592607Z","iopub.execute_input":"2022-07-24T08:43:08.593539Z","iopub.status.idle":"2022-07-24T08:51:44.660464Z","shell.execute_reply.started":"2022-07-24T08:43:08.593491Z","shell.execute_reply":"2022-07-24T08:51:44.659498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_val = model.predict(val_X,batch_size=512,verbose=1).squeeze()\n\nfor thresh in np.arange(0.1,0.501,0.01):\n thresh = np.round(thresh,2)\n print(\"**** F1 Score with \"+str(thresh)+\" value **** \",f1_score(val_Y,(pred_val>thresh).astype(int)))\n print(\" ***** AUC Score with \"+str(thresh)+\" value **** \",roc_auc_score(val_Y,(pred_val>thresh).astype(int)))\n print()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:08:18.766594Z","iopub.execute_input":"2022-07-24T09:08:18.766937Z","iopub.status.idle":"2022-07-24T09:08:34.593056Z","shell.execute_reply.started":"2022-07-24T09:08:18.766908Z","shell.execute_reply":"2022-07-24T09:08:34.592034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds_test = model.predict(test_X,batch_size=512,verbose=1).squeeze()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:08:34.594770Z","iopub.execute_input":"2022-07-24T09:08:34.595480Z","iopub.status.idle":"2022-07-24T09:08:42.248490Z","shell.execute_reply.started":"2022-07-24T09:08:34.595437Z","shell.execute_reply":"2022-07-24T09:08:42.247611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission= pd.DataFrame({'qid':df_test['qid'],'prediction': preds_test})","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:08:42.250497Z","iopub.execute_input":"2022-07-24T09:08:42.250861Z","iopub.status.idle":"2022-07-24T09:08:42.261719Z","shell.execute_reply.started":"2022-07-24T09:08:42.250823Z","shell.execute_reply":"2022-07-24T09:08:42.260436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv(\"Submission1.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:08:42.263355Z","iopub.execute_input":"2022-07-24T09:08:42.263909Z","iopub.status.idle":"2022-07-24T09:08:43.343444Z","shell.execute_reply.started":"2022-07-24T09:08:42.263872Z","shell.execute_reply":"2022-07-24T09:08:43.342482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Navigate To Top](#0)\n\n#### Bidirectional GRU","metadata":{}},{"cell_type":"code","source":"embedding_layer = Embedding(len(tokenizer.word_index)+1,300,weights=[embedding_vectors])\nmodel = Sequential()\nmodel.add(embedding_layer)\nmodel.add(Bidirectional(GRU(64,return_sequences=True)))\nmodel.add(GlobalMaxPool1D())\nmodel.add(Dense(16,activation='relu'))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(1,activation='sigmoid'))\n\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:14:22.559995Z","iopub.execute_input":"2022-07-24T09:14:22.560671Z","iopub.status.idle":"2022-07-24T09:14:23.289109Z","shell.execute_reply.started":"2022-07-24T09:14:22.560635Z","shell.execute_reply":"2022-07-24T09:14:23.288148Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.compile(loss='binary_crossentropy',optimizer='Adam',metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:14:27.166571Z","iopub.execute_input":"2022-07-24T09:14:27.167153Z","iopub.status.idle":"2022-07-24T09:14:27.177914Z","shell.execute_reply.started":"2022-07-24T09:14:27.167121Z","shell.execute_reply":"2022-07-24T09:14:27.176677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ncheckpoint = ModelCheckpoint(\n 'model.h5', \n monitor = 'val_acc', \n verbose = 1, \n save_best_only = True\n)\nreduce_lr = ReduceLROnPlateau(\n monitor = 'val_loss', \n factor = 0.2, \n verbose = 1, \n patience = 5, \n min_lr = 0.001\n)\n\n\nhistory=model.fit(train_X,train_Y,validation_data=(val_X,val_Y),epochs=5,batch_size=512,callbacks=[checkpoint,reduce_lr])","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:14:33.643252Z","iopub.execute_input":"2022-07-24T09:14:33.644151Z","iopub.status.idle":"2022-07-24T09:22:57.504028Z","shell.execute_reply.started":"2022-07-24T09:14:33.644108Z","shell.execute_reply":"2022-07-24T09:22:57.503128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_val = model.predict(val_X,batch_size=512,verbose=1).squeeze()\n\nfor thresh in np.arange(0.1,0.501,0.01):\n thresh = np.round(thresh,2)\n print(\"**** F1 Score with \"+str(thresh)+\" value **** \",f1_score(val_Y,(pred_val>thresh).astype(int)))\n print(\" ***** AUC Score with \"+str(thresh)+\" value **** \",roc_auc_score(val_Y,(pred_val>thresh).astype(int)))\n print()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:22:57.506062Z","iopub.execute_input":"2022-07-24T09:22:57.507132Z","iopub.status.idle":"2022-07-24T09:23:08.362702Z","shell.execute_reply.started":"2022-07-24T09:22:57.507091Z","shell.execute_reply":"2022-07-24T09:23:08.361507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds_test = model.predict(test_X,batch_size=512,verbose=1).squeeze()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:23:08.364223Z","iopub.execute_input":"2022-07-24T09:23:08.364775Z","iopub.status.idle":"2022-07-24T09:23:15.605135Z","shell.execute_reply.started":"2022-07-24T09:23:08.364735Z","shell.execute_reply":"2022-07-24T09:23:15.604096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission= pd.DataFrame({'qid':df_test['qid'],'prediction': preds_test})","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:23:15.609023Z","iopub.execute_input":"2022-07-24T09:23:15.609323Z","iopub.status.idle":"2022-07-24T09:23:15.621605Z","shell.execute_reply.started":"2022-07-24T09:23:15.609298Z","shell.execute_reply":"2022-07-24T09:23:15.620576Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv(\"Submission_GRU.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:23:15.623340Z","iopub.execute_input":"2022-07-24T09:23:15.623844Z","iopub.status.idle":"2022-07-24T09:23:16.715318Z","shell.execute_reply.started":"2022-07-24T09:23:15.623806Z","shell.execute_reply":"2022-07-24T09:23:16.714241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save(\"Quora_Insincere.h5\")","metadata":{"execution":{"iopub.status.busy":"2022-07-24T09:27:49.593190Z","iopub.execute_input":"2022-07-24T09:27:49.593579Z","iopub.status.idle":"2022-07-24T09:27:50.957561Z","shell.execute_reply.started":"2022-07-24T09:27:49.593549Z","shell.execute_reply":"2022-07-24T09:27:50.956470Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Thank You for scrolling this down in my notebook . I would love to get your feeback and suggestions if any โบ.\n\n##### This dataset excites because it is related to find out toxic and harmful content on social media which can trigger chaos and disputes . I have given my little contribution to this dataset ๐.","metadata":{}}]}