{"cells":[{"metadata":{"_uuid":"23f16fa676febbf5ec86ffa98ffc0c7036c6a9f1"},"cell_type":"markdown","source":"**Quora Insincere Questions Classification**"},{"metadata":{"_uuid":"1783b4f3ed8bc30affbf1313f03eb8b05a2f93e6"},"cell_type":"markdown","source":"An insincere question is defined as a question intended to make a statement rather than look for helpful answers. Some characteristics that can signify that a question is insincere:\n*     Has a non-neutral tone\n*         Has an exaggerated tone to underscore a point about a group of people\n*         Is rhetorical and meant to imply a statement about a group of people\n*     Is disparaging or inflammatory\n*         Suggests a discriminatory idea against a protected class of people, or seeks confirmation of a stereotype\n*         Makes disparaging attacks/insults against a specific person or group of people\n*         Based on an outlandish premise about a group of people\n*         Disparages against a characteristic that is not fixable and not measurable\n*     Isn't grounded in reality\n*         Based on false information, or contains absurd assumptions\n*     Uses sexual content (incest, bestiality, pedophilia) for shock value, and not to seek genuine answers\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.\n\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cba52078b00d00ff0987f63dd3210490aa90ab2b"},"cell_type":"markdown","source":"* train.csv - the training set containing insincere questions\n* test.csv - the test set\n* sample_submission.csv - A sample submission in the correct format\n* enbeddings/ - Folder containing word embeddings."},{"metadata":{"trusted":true,"_uuid":"db674dbe4a34b8eaaa2ab46338a44dccda68c83c"},"cell_type":"code","source":"!ls ../input/embeddings/","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"66850c1b72fb0ccbe25c8464c42c37e70cf7800c"},"cell_type":"markdown","source":"various embedings that can be used in the this competition \n\n* GoogleNews-vectors-negative300 - https://code.google.com/archive/p/word2vec/\n* glove.840B.300d - https://nlp.stanford.edu/projects/glove/\n* paragram_300_sl999 - https://cogcomp.org/page/resource_view/106\n* wiki-news-300d-1M - https://fasttext.cc/docs/en/english-vectors.html"},{"metadata":{"_uuid":"88ae3e765dc87e89106155fe3cd38e5d415d9242"},"cell_type":"markdown","source":"**Importing libraries Needed**"},{"metadata":{"trusted":true,"_uuid":"8a4ce8d399f73c66ad591b024388868fe26dca01"},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport re\n\n# Keras\nfrom keras.preprocessing.text import Tokenizer\nfrom keras.preprocessing.sequence import pad_sequences\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Flatten, LSTM, Conv1D, MaxPooling1D, Dropout, Activation\nfrom keras.layers import Dense, Input, LSTM, Embedding, Dropout, Activation, CuDNNGRU, Conv1D\nfrom keras.layers.embeddings import Embedding\n\n## Plotly\nimport matplotlib.pyplot as plt\nimport plotly.offline as py\nimport plotly.graph_objs as go\npy.init_notebook_mode(connected=True)\n\n# Others\nimport nltk\nimport string\nimport numpy as np\nimport pandas as pd\nfrom nltk.corpus import stopwords\nfrom sklearn.manifold import TSNE\nfrom nltk.stem import SnowballStemmer\nfrom string import punctuation\nfrom plotly import tools\nimport seaborn as sns\n\n\nimport tensorflow as tf\nimport numpy as np\n\nfrom tensorflow.python.keras.preprocessing import sequence\nfrom tensorflow.python.keras.preprocessing import text\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nfrom sklearn.feature_selection import SelectKBest\nfrom sklearn.feature_selection import f_classif\n\nfrom sklearn.utils import shuffle\n\nimport random\n\n\n# fix random seed for reproducibility\nnp.random.seed(7)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"83b619be48a014d9020e577135b4b2c11f9b3758"},"cell_type":"markdown","source":"**Reading the CSV files**"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"train_data = pd.read_csv(\"../input/train.csv\")\ntest_data = pd.read_csv(\"../input/test.csv\")\nprint(\"Train dataset shape : \", train_data.shape)\nprint(\"Test dataset shape : \", test_data.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1a73e63de2d206a49dbeed5f930eb8d7c5a34689"},"cell_type":"code","source":"train_data.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b74942da157469e9b07599fdc3f49c2007adeb4c"},"cell_type":"code","source":"train_data.tail()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c5fbdd366d85d01b87da9a2ef74990be4f1d50e2"},"cell_type":"code","source":"train_data.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"970c989e2e7f12653827ce5747f489c35786a9d2"},"cell_type":"code","source":"train_data.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ccbb9d0b7d8c4713ce6c67bdc7ba103a8f66a0cc"},"cell_type":"code","source":"train_data['target'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"9f26840b4f75db81ebab89901570121df610418a"},"cell_type":"code","source":"train0_df0 = train_data[train_data[\"target\"]==0]\ntrain0_df0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fe3353343e092ae07fc47e0f4c2897dbc6c03bed"},"cell_type":"code","source":"train1_df1 = train_data[train_data[\"target\"]==1]\ntrain1_df1","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0e73c4e9e2e85d739c5fc7cdc95e7557176655a4"},"cell_type":"markdown","source":"**Median number of words per sample in traning dataset**"},{"metadata":{"trusted":true,"_uuid":"1a6d662a07049b17de8a33a8b7390427c8657a06"},"cell_type":"code","source":"def get_num_words_per_sample(sample_texts):\n    \"\"\"Gets the median number of words per sample given corpus.\n    # Arguments\n        sample_texts: list, sample texts.\n    # Returns\n        int, median number of words per sample.\n    \"\"\"\n    num_words = [len(s.split()) for s in sample_texts]\n    return np.median(num_words)\n\nget_num_words_per_sample(train_data['question_text'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"449392178bb0c2745826c350afac57f6d9ac259c"},"cell_type":"markdown","source":"**Median number of words per sample in test dataset**"},{"metadata":{"trusted":true,"_uuid":"3649210ff066f0119eaa7f14cc2f59bc5f08dfd4"},"cell_type":"code","source":"get_num_words_per_sample(test_data['question_text'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3e9167dcef4d035a5ade31022e36015cbf47d94b"},"cell_type":"markdown","source":"**Number of words per sample in training dataset**\n\nPlotting the number of words per question in the training dataset"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"3b0b46b7a87ceb0ae0d822a4c9ba7190125ee4c2"},"cell_type":"code","source":"def plot_sample_length_distribution(sample_texts):\n    \"\"\"Plots the sample length distribution.\n    # Arguments\n        samples_texts: list, sample texts.\n    \"\"\"\n    plt.hist([len(s) for s in sample_texts],50)\n    plt.xlabel('Length of a sample')\n    plt.ylabel('Number of samples')\n    plt.title('Sample length distribution')\nplt.show()\n\nplot_sample_length_distribution(train_data['question_text'])\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4bf6f6296f3165ae3ab45e62030cbe4acf835bec"},"cell_type":"markdown","source":"**Number of words per sample in test dataset**\n\nPlotting the number of words per question in the test dataset"},{"metadata":{"trusted":true,"_uuid":"f4df044dc6b5cf1cb804430fca07b51a18e00874"},"cell_type":"code","source":"plot_sample_length_distribution(test_data['question_text'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1d51d7daadc8e9a1305ff201cf912fd795417fe3"},"cell_type":"markdown","source":"**Histogram of Training dataset\n**\n\nTarget Distribution:\n\nFirst let us look at the distribution of the target variable to understand more about the imbalance and so on.\n"},{"metadata":{"trusted":true,"_uuid":"f352e3269fa16cbff225ebdd6978ff0b3a7cb00a"},"cell_type":"code","source":"digit_train, counts_train = np.unique(train_data['target'], return_counts = True)\n\ndistribution_train = dict(zip(digit_train, counts_train))\nprint(distribution_train )\n\nplt.bar(list(distribution_train.keys()),distribution_train.values(),width =0.6)\nplt.xlabel('Target --> 0(Sincere),1(Insincere))')\nplt.ylabel('Number of Questions')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a8458d9e5ad86c09da069c11278765ed17e9e6b3"},"cell_type":"markdown","source":"***** Percentage of Insincere Questions = 80810/(80810+1225312) = 6.595%\n\n***** Percentage of Sincere Questions = 1225312/(80810+1225312) = 93.813%\n\nFrom the above histogram it can be seen that there is only 6% of question texts are insincere,feel like the data set is unbalanced.\n\n\n\n\n"},{"metadata":{"_uuid":"f3f3a98292d3c686aa61d72cb647fc3d1f18280c"},"cell_type":"markdown","source":"**Word Frequency plot of sincere & insincere questions**"},{"metadata":{"trusted":true,"_uuid":"632727a12954c78cac822b9f5a387e32b3572ce8","scrolled":true},"cell_type":"code","source":"from collections import defaultdict\nfrom wordcloud import WordCloud, STOPWORDS\n\ntrain1_df = train_data[train_data[\"target\"]==1]\ntrain0_df = train_data[train_data[\"target\"]==0]\n\n## custom function for ngram generation ##\ndef generate_ngrams(text, n_gram=1):\n    token = [token for token in text.lower().split(\" \") if token != \"\" if token not in STOPWORDS ]\n    ngrams = zip(*[token[i:] for i in range(n_gram)])\n    return [\" \".join(ngram) for ngram in ngrams]\n\n## custom function for horizontal bar chart ##\ndef horizontal_bar_chart(df, color):\n    trace = go.Bar(\n        y=df[\"word\"].values[::-1],\n        x=df[\"wordcount\"].values[::-1],\n        showlegend=False,\n        orientation = 'h',\n        marker=dict(\n            color=color,\n        ),\n    )\n    return trace\n\n## Get the bar chart from sincere questions ##\nfreq_dict = defaultdict(int)\nfor sent in train0_df[\"question_text\"]:\n    for word in generate_ngrams(sent):\n        freq_dict[word] += 1\nfd_sorted = pd.DataFrame(sorted(freq_dict.items(), key=lambda x: x[1])[::-1])\nfd_sorted.columns = [\"word\", \"wordcount\"]\ntrace0 = horizontal_bar_chart(fd_sorted.head(50), 'blue')\n\n## Get the bar chart from insincere questions ##\nfreq_dict = defaultdict(int)\nfor sent in train1_df[\"question_text\"]:\n    for word in generate_ngrams(sent):\n        freq_dict[word] += 1\nfd_sorted = pd.DataFrame(sorted(freq_dict.items(), key=lambda x: x[1])[::-1])\nfd_sorted.columns = [\"word\", \"wordcount\"]\ntrace1 = horizontal_bar_chart(fd_sorted.head(50), 'blue')\n\n# Creating two subplots\nfig = tools.make_subplots(rows=1, cols=2, vertical_spacing=0.04,\n                          subplot_titles=[\"Frequent words of sincere questions\", \n                                          \"Frequent words of insincere questions\"])\nfig.append_trace(trace0, 1, 1)\nfig.append_trace(trace1, 1, 2)\nfig['layout'].update(height=1200, width=900, paper_bgcolor='rgb(233,233,233)', title=\"Word Count Plots\")\npy.iplot(fig, filename='word-plots')\n\n#plt.figure(figsize=(10,16))\n#sns.barplot(x=\"ngram_count\", y=\"ngram\", data=fd_sorted.loc[:50,:], color=\"b\")\n#plt.title(\"Frequent words for Insincere Questions\", fontsize=16)\n#plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9194461f136d589d6fc08986e759a571419f8c7f"},"cell_type":"markdown","source":"**Train and test(validation) split**"},{"metadata":{"trusted":true,"_uuid":"9226811d399a11d953f34fcb111806cb19741e08"},"cell_type":"code","source":"train_data = shuffle(train_data)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dc9349b22d5db7852e93e9e674149453605fbcf7"},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nsentences = train_data['question_text'].values\ntarget = train_data['target'].values\n\nX_Train, X_Val, Y_Train,Y_Val = train_test_split(sentences, target, test_size=0.20, random_state=7)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3d0df4e427faac57564286245229573704ca4d12"},"cell_type":"code","source":"shuffle_index = np.random.permutation(1044897)\nX_Train, Y_Train = X_Train[shuffle_index], Y_Train[shuffle_index]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"40080450d421380a30b8edea899ee685aa4ae859"},"cell_type":"code","source":"shuffle_index = np.random.permutation(261225)\nX_Val, Y_Val = X_Val[shuffle_index], Y_Val[shuffle_index]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"86962ea9aa2b8e1c9f10ef3efa960a024472ce6e"},"cell_type":"code","source":"print(X_Train.size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e1797382a234b886e0293600deb50707c2897f9c"},"cell_type":"code","source":"print(Y_Train.size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2c294c54d66f47b5048c6db3a9ea256b8c545a27"},"cell_type":"code","source":"print(X_Val.size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fbb7328957ec57e42a6e99f1c168c8ef89fb1d11"},"cell_type":"code","source":"print(Y_Val.size)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"53703ccbcc90d269d0aa5f4c81de08758104746d"},"cell_type":"markdown","source":"**how to do the upsampling in text classification problem**"},{"metadata":{"trusted":true,"_uuid":"0d32fe6df76a2a46f4fe9eb7890a9e96ed3d1270"},"cell_type":"code","source":"# Limit on the number of features. We use the top 20K features.\nTOP_K = 20000\n\n# Whether text should be split into word or character n-grams.\n# One of 'word', 'char'.\nTOKEN_MODE = 'word'\n\n# Minimum document/corpus frequency below which a token will be discarded.\nMIN_DOCUMENT_FREQUENCY = 2\n\n# Limit on the length of text sequences. Sequences longer than this\n# will be truncated.\nMAX_SEQUENCE_LENGTH = 100\n\n\ndef sequence_vectorize(train_texts, val_texts,test_texts):\n    \"\"\"Vectorizes texts as sequence vectors.\n    1 text = 1 sequence vector withfixed length.\n    # Arguments\n        train_texts: list, training text strings.\n        val_texts: list, validation text strings.\n        test_texts: list, validation text strings.\n    # Returns\n        x_train, x_val,x_test,tokenizer object, word_index: vectorized training and validation and test\n            texts and word index dictionary.\n    \"\"\"\n    # Create vocabulary with training texts.\n    tokenizer = text.Tokenizer(num_words=TOP_K)\n    #num_words, which is responsible for setting the size of the vocabulary.\n    tokenizer.fit_on_texts(train_texts)\n\n    # Vectorize training and validation texts.\n    x_train = tokenizer.texts_to_sequences(train_texts)\n    x_val = tokenizer.texts_to_sequences(val_texts)\n    x_test = tokenizer.texts_to_sequences(test_texts)\n\n    # Get max sequence length.\n    max_length = len(max(x_train, key=len))\n    if max_length > MAX_SEQUENCE_LENGTH:\n        max_length = MAX_SEQUENCE_LENGTH\n\n    # Fix sequence length to max value. Sequences shorter than the length are\n    # padded in the beginning and sequences longer are truncated\n    # at the beginning.\n    x_train = sequence.pad_sequences(x_train, maxlen=max_length)\n    x_val = sequence.pad_sequences(x_val, maxlen=max_length)\n    x_test = sequence.pad_sequences(x_test, maxlen=max_length)\n    return x_train, x_val,x_test,tokenizer.word_index,tokenizer\n\n\nx_train ,x_val ,x_test,word_index,tk = sequence_vectorize(X_Train,X_Val,test_data['question_text'].values)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8f66cfa28c89832526ea09c103a31f94ed5d47e9"},"cell_type":"code","source":"x_train","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c28e4e63be5c6c8476deba66c03d5abf58eed252"},"cell_type":"markdown","source":"**Creating Embedding Matrix**"},{"metadata":{"trusted":true,"_uuid":"2358f7169fa67c23b7b56d1ca883ce62fe8db17c"},"cell_type":"code","source":"\n\ndef get_coefs(word,*arr): \n    return word, np.asarray(arr, dtype='float32')\n\ndef load_embedding(file):\n    if file == '../input/embeddings/wiki-news-300d-1M/wiki-news-300d-1M.vec':\n        embeddings_index = dict(get_coefs(*o.split(\" \")) for o in open(file) if len(o)>100)\n    else:\n        embeddings_index = dict(get_coefs(*o.split(\" \")) for o in open(file, encoding='latin'))\n    return embeddings_index\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7653bbbf7ebd8002615499595b2c1b09f039522a"},"cell_type":"code","source":"def make_embedding_matrix(embedding, tokenizer, len_voc):\n    all_embs = np.stack(embedding.values())\n    emb_mean,emb_std = all_embs.mean(), all_embs.std()\n    embed_size = all_embs.shape[1]\n    word_index = tokenizer.word_index\n    embedding_matrix = np.random.normal(emb_mean, emb_std, (len_voc, embed_size))\n    \n    for word, i in word_index.items():\n        if i >= len_voc:\n            continue\n        embedding_vector = embedding.get(word)\n        if embedding_vector is not None: \n            embedding_matrix[i] = embedding_vector\n    \n    return embed_size,embedding_matrix","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1a36030d127f802b7ac5f0ab291478ebe090a017"},"cell_type":"code","source":"glove = load_embedding('../input/embeddings/glove.840B.300d/glove.840B.300d.txt')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"99369f417b229226791f55715c1c8cb48f745634"},"cell_type":"code","source":"embedding_dim,embed_mat = make_embedding_matrix(glove, tk, TOP_K)\nprint(embedding_dim)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5e3380c7b505605ea6373696e6f6c31d2dd90dca"},"cell_type":"code","source":"# Number of features will be the embedding input dimension. Add 1 for the\n    # reserved index 0.\nnum_features = min(len(word_index) + 1, TOP_K)\nprint(num_features)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"47a7c77f780d61d60a5d9251d49d3beb6cffadf4"},"cell_type":"markdown","source":"**BaseLine Model**"},{"metadata":{"trusted":true,"_uuid":"c781ee8c9098172995e35e11d72b82849e861a84"},"cell_type":"code","source":"\n\nmodel4= Sequential()\nmodel4.add(Embedding(input_dim=num_features,output_dim=embedding_dim,input_length=100, weights=[embed_mat],trainable=True))\nmodel4.add(Dropout(0.2))\nmodel4.add(Conv1D(64, 5, activation='relu'))\nmodel4.add(MaxPooling1D(pool_size=4))\nmodel4.add(LSTM(100))\nmodel4.add(Dense(1, activation='sigmoid'))\nmodel4.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy'])\nmodel4.summary()  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"299511a15d64dd7206818dfefab4adf60133b468"},"cell_type":"code","source":"history = model4.fit(x_train, Y_Train,\n                    epochs=10,\n                    verbose=True,\n                    validation_split=0.2,\n                    batch_size=1024)\nloss, accuracy = model4.evaluate(x_val, Y_Val, verbose=True)\nprint(\"Test Accuracy: {:.4f}\".format(accuracy))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0d25b90b60c546b2729d0c609d6171e9f8f79f42"},"cell_type":"code","source":"import matplotlib.pyplot as plt\nplt.style.use('ggplot')\n\ndef plot_history(history):\n    acc = history.history['acc']\n    val_acc = history.history['val_acc']\n    loss = history.history['loss']\n    val_loss = history.history['val_loss']\n    x = range(1, len(acc) + 1)\n\n    plt.figure(figsize=(12, 5))\n    plt.subplot(1, 2, 1)\n    plt.plot(x, acc, 'b', label='Training acc')\n    plt.plot(x, val_acc, 'r', label='Validation acc')\n    plt.title('Training and validation accuracy')\n    plt.legend()\n    plt.subplot(1, 2, 2)\n    plt.plot(x, loss, 'b', label='Training loss')\n    plt.plot(x, val_loss, 'r', label='Validation loss')\n    plt.title('Training and validation loss')\n    plt.legend()\n    \n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fbe8f364571848fe072cf189f1d45f8063527400"},"cell_type":"code","source":"plot_history(history)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3092337b562ec5e07cd383725c3b1a7097550714"},"cell_type":"markdown","source":"Another model"},{"metadata":{"trusted":true,"_uuid":"2ba272c8f6fe7119d13293c63d5aceb17fc4b98a","scrolled":false},"cell_type":"code","source":"model5= Sequential()\nmodel5.add(Embedding(input_dim=num_features,output_dim=embedding_dim,input_length=100, weights=[embed_mat],trainable=True))\nmodel5.add(Dropout(0.2))\nmodel5.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))\nmodel5.add(Dense(1, activation='sigmoid'))\nmodel5.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy'])\nmodel5.summary()  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"34815063645a3bfd6db944e33eeed19f6ceb6606"},"cell_type":"code","source":"history5 = model5.fit(x_train, Y_Train,\n                    epochs=10,\n                    verbose=True,\n                    validation_split=0.2,\n                    batch_size=1024)\nloss, accuracy = model5.evaluate(x_val, Y_Val, verbose=True)\nprint(\"Test Accuracy: {:.4f}\".format(accuracy))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e54bdcb97b7dd118ad8e5903e61079080936b8a4"},"cell_type":"code","source":"plot_history(history5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bfc6f5c9cb718ff9f2d76464661cc24253c67b38"},"cell_type":"code","source":"y_predict = model5.predict(x_test, batch_size=None, verbose=1, steps=None)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2e02acbd9d84615a50771d949b955d379c4272b9"},"cell_type":"code","source":"test_df = pd.read_csv(\"../input/test.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"798045b8a3fa0c8a022c6dfb127dbe401afc1989"},"cell_type":"code","source":"y_predict","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ef67b9d317f607b6a05d484617b918853fa3a7cb"},"cell_type":"code","source":"y_pred_changed = y_predict.argmax(1)\ny_pred_changed","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a55eb02801faf2282467a63630a52abb5131aaa4"},"cell_type":"code","source":"y_te = (np.array(y_predict) > 0.5).astype(np.int)\nsubmit_df = pd.DataFrame({\"qid\": test_df[\"qid\"], \"prediction\": y_te.flatten()})\nsubmit_df.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e178336899497cca10912277e8c522d3e5999643"},"cell_type":"markdown","source":"**Confusion Matrix**"},{"metadata":{"trusted":true,"_uuid":"ca1e8ba3d2e1712f6dc54d6b0828253d52dff48e"},"cell_type":"code","source":"y_predicted = model5.predict(x_val, batch_size=None, verbose=1, steps=None)\ny_predicted_changed = y_predicted.argmax(1)\n\n# Creating the Confusion Matrix\nfrom sklearn.metrics import confusion_matrix\ncm = confusion_matrix(Y_Val, y_pred_changed)\ncm","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bc54d4d8763c8f5e1cc9ca5249be204476bec10c"},"cell_type":"code","source":"from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, classification_report, confusion_matrix","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"addab8782bbdfbdf9f03d517c19fbe3cb4e6a24b"},"cell_type":"code","source":"accuracy_score(sub['prediction'], y_pred_changed)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a98f9406d80dd4e143ff8f804993832cef8f0a74"},"cell_type":"code","source":"f1_score(sub['prediction'], y_pred_changed,average='weighted')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f8f7fbef5bf47eceff65fd8ea485d03c7a21a574"},"cell_type":"code","source":"recall_score(sub['prediction'], y_pred_changed,average='weighted')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cc2e4ddf70880910efa700d5929f5fc6f676600b"},"cell_type":"code","source":"precision_score(sub['prediction'], y_pred_changed,average='weighted')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d368ddb67fa5bca5f48da0ad7f1946f3a9038c03"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}