{"cells":[{"metadata":{"_uuid":"e6055851095963d37d20b1b07cbe6b838d22c717"},"cell_type":"markdown","source":"# Quora Insincere Questions Classification:\n"},{"metadata":{"_uuid":"96c218c8c8996503b76dfbfa190486b75e9b0ebb"},"cell_type":"markdown","source":"## Detect toxic content to improve online conversations\n"},{"metadata":{"_uuid":"6028ce317c94e80cf9ccf8b8f33298d2646873f6"},"cell_type":"markdown","source":"## Problem:\n* **Handle toxic and disivie content / miseleading content**"},{"metadata":{"_uuid":"3c12803e64025a04b3d26b21c741e8519276a068"},"cell_type":"markdown","source":"* **To do that we have to develop a model that identify and flag insincere questions**"},{"metadata":{"_uuid":"f83c7e0351133c05cd9af69fec063c2c2b1e2b1f"},"cell_type":"markdown","source":"## Evaluation:"},{"metadata":{"_uuid":"3fca53946b70997b86ab3ef9e6b1755f5d7a1eac"},"cell_type":"markdown","source":"* **For each qid in the testset, predict the corresponding questions_text:**\n\n * **Is insincere => 1 (Target)**\n\n * **Is sincere => 0**\n\n* **Submissions are evaluated on F1 Score between the predicted and the observed targets**"},{"metadata":{"_uuid":"a3b58eb601419cb2a425b077a60560d19aca3a2a"},"cell_type":"markdown","source":"## Data Fields:"},{"metadata":{"_uuid":"2e2f215ffaa7b4da46dd567630e762555524f0d4"},"cell_type":"markdown","source":"* **qid : unique question identifer**\n\n* **question_text : Quora question text**\n\n* **Target : a question labeled << insincere >> has a value of 1, otherwise 0**"},{"metadata":{"trusted":true,"_uuid":"d31ff4da29bf6476e3c1049da1dd42fb8291a811"},"cell_type":"code","source":"import os\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nfrom wordcloud import WordCloud\n\n\nimport nltk\nfrom nltk.corpus import stopwords\nfrom nltk import word_tokenize\n\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nfrom sklearn.naive_bayes import MultinomialNB\nfrom sklearn.svm import LinearSVC\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import f1_score, confusion_matrix\nfrom sklearn.metrics import classification_report\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"74bf2ccf2352fc25fb6c1e9f6b881841b1b1594b"},"cell_type":"code","source":"X_train_filepath = os.path.join('..', 'input', 'train.csv')\nX_test_filepath = os.path.join('..', 'input', 'test.csv')\nsample_filepath = os.path.join('..', 'input', 'sample_submission.csv')\nX_train_filepath, X_test_filepath, sample_filepath","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ffb55642b5cd867f7919f66cf578e62ce026a833"},"cell_type":"code","source":"df_train = pd.read_csv(X_train_filepath, encoding='ISO-8859-1')\ndf_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bc05d34db38a31b5d7b6172a473afa5f79b7dcfb"},"cell_type":"code","source":"df_test = pd.read_csv(X_test_filepath, encoding='ISO-8859-1')\ndf_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2cb151b8725bcec638d2218fc0546ca43ecf5567"},"cell_type":"code","source":"df_train.shape, df_test.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d799d53b15af75eb824f319cd790de4091054b99"},"cell_type":"code","source":"df_sample = pd.read_csv(sample_filepath, encoding='ISO-8859-1')\ndf_sample.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"38a1a21840a7bff37c6ca9e336a377532a2b17e1"},"cell_type":"code","source":"df_train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"204292bfe74ab796057130de8168cb59e8ba3062"},"cell_type":"code","source":"df_train[\"target\"].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d26f1293e79aca1e66716c00e084723abd90a1f5"},"cell_type":"markdown","source":"## Exploratory Data Analysis:"},{"metadata":{"trusted":true,"_uuid":"f3cfcf6501232e7f82064f12266ff64372cf4404"},"cell_type":"code","source":"insincere = df_train[df_train[\"target\"] == 1]\nsincere = df_train[df_train[\"target\"] == 0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"66ef241d3b16fe29184f3f9fe92bbd56629a75d0"},"cell_type":"code","source":"sincere.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6140af9ed36f78a840e46581a1b7960a609f61a1"},"cell_type":"code","source":"sincere.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"38bea5bc2756975f4a7d84d4484add36c75ee0c6"},"cell_type":"code","source":"insincere.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1c8e577b384cb0d1c8ff6fa8e31cb902cd446c34"},"cell_type":"code","source":"insincere.info()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bf0161cd240eef5fe2eb1db150cb5d04b9433373"},"cell_type":"markdown","source":"## Distribution sincere/insincere plots:"},{"metadata":{"trusted":true,"_uuid":"ef6e6d055b8a6dfc863c1b6c425dd9892ad74c0d"},"cell_type":"code","source":"ax, fig = plt.subplots(figsize=(10, 7))\nquestion_class = df_train[\"target\"].value_counts()\nquestion_class.plot(kind= 'bar', color= [\"blue\", \"orange\"])\nplt.title('Bar chart')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"31feb7273bcbfe6eff48fb3072e93ee1051373d2"},"cell_type":"code","source":"print(df_train['target'].value_counts())\nprint(sum(df_train['target'] == 1) / sum(df_train['target'] == 0) * 100, \"percent of questions are insincere.\")\nprint(100 - sum(df_train['target'] == 1) / sum(df_train['target'] == 0) * 100, \"percent of questions are sincere\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2d54a35d517c23e9bf63e71fbc70b3530f913c6f"},"cell_type":"markdown","source":"**We have a Unbalenced Data**"},{"metadata":{"_uuid":"061dc1bbfff5fc73842083f85c66ff0011a0fdc3"},"cell_type":"markdown","source":"## Insincere Word cloud:"},{"metadata":{"trusted":true,"_uuid":"7bb19f480c1d37405caea865ab740642c9a6fa96"},"cell_type":"code","source":"stop_words = stopwords.words(\"english\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e736b03b1bbe33359b2694915071912e8eea6f54"},"cell_type":"code","source":"insincere_words = ''\n\nfor question in insincere.question_text:\n    text = question.lower()\n    tokens = word_tokenize(text)\n    for words in tokens:\n        insincere_words = insincere_words + words + ' '","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fb2d82ae60901a1a2a76c3c0e7a5c4b9d3568c09"},"cell_type":"code","source":"# Generate a word cloud image\ninsincere_wordcloud = WordCloud(width=600, height=400).generate(insincere_words)\n#Insincere Word cloud\nplt.figure( figsize=(10,8), facecolor='k')\nplt.imshow(insincere_wordcloud)\nplt.axis(\"off\")\nplt.tight_layout(pad=0)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5505d862e11ccd7ce3d244d674820bf908599135"},"cell_type":"markdown","source":"## Length distribution in sincere and insincere questions:"},{"metadata":{"trusted":true,"_uuid":"3e7ab55ff63f13f6008db2f2d246f5e9fc540149"},"cell_type":"code","source":"insincere[\"questions_length\"] = insincere.question_text.apply(lambda x: len(x))\nsincere[\"questions_length\"] = sincere.question_text.apply(lambda x: len(x))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9d36abf5115bc67f4d93a28a0e8ff292d03225a9"},"cell_type":"code","source":"insincere[\"questions_length\"].mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6fa9c61ab0ee49a8798de6354d307ac5d844e606"},"cell_type":"code","source":"sincere[\"questions_length\"].mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"909b0bac3d7d05a0043052386c2da2e3b043e5c8"},"cell_type":"code","source":"sns.set(rc={'figure.figsize':(11.7,8.27)})\nsns.distplot(insincere.questions_length, hist=True, label=\"insincere\")\nsns.distplot(sincere.questions_length, hist=True, label=\"sincere\");","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5ae8f385a2a9d311f69cbc74f53146341bd65ee2"},"cell_type":"markdown","source":"## Number of words distribution in sincere and insincere questions:"},{"metadata":{"trusted":true,"_uuid":"c584f1886b6e0374156e9e0cf51c624ed5a5304f"},"cell_type":"code","source":"insincere['number_words'] = insincere.question_text.apply(lambda x: len(x.split()))\nsincere['number_words'] = sincere.question_text.apply(lambda x: len(x.split()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1289d70ca45316e8c74630db042f11516cdae3bc"},"cell_type":"code","source":"insincere['number_words'].mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c57f18383f67e85a1911be04f38e67223a23b17b"},"cell_type":"code","source":"sincere['number_words'].mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6e415846c3be4faf8304b2be717643e948469ee0"},"cell_type":"code","source":"sns.set(rc={'figure.figsize':(11.7,8.27)})\nsns.distplot(insincere.number_words, hist=True, label=\"insincere\")\nsns.distplot(sincere.number_words, hist=True, label=\"sincere\");","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a96fb4cba3b0c80ed9cb0a9f9e76d96279ed7773"},"cell_type":"markdown","source":"## TfidfVectorizer :"},{"metadata":{"trusted":true,"_uuid":"cefbaa90b0274ae0704be53cd282d9813c9f5e4e"},"cell_type":"code","source":"vectorizer = TfidfVectorizer(min_df=5, ngram_range=(1,3),\n                        strip_accents='unicode',lowercase =True, \n                        stop_words = 'english',tokenizer=word_tokenize)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d3f068f9b3105e1adab355606bce1dc98d16740a"},"cell_type":"code","source":"train_vectorized = vectorizer.fit_transform(df_train.question_text.values)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a5cadfdbcba0ab06944be235c91dec371d3bc4bd"},"cell_type":"code","source":"test_vectorized = vectorizer.fit_transform(df_test.question_text.values)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ec7389c60c944673f7f5498b8d11552dedfd3a69"},"cell_type":"code","source":"X_train, X_val, y_train, y_val = train_test_split(train_vectorized, df_train.target.values, test_size=0.1, stratify = df_train.target.values)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"72bc9d23b8f9faabfa94ebd809394b48079f7e3e"},"cell_type":"markdown","source":"## Classification with Logistic Regression :"},{"metadata":{"trusted":true,"_uuid":"57990fafe82400f182aa2fed78b81725398ae412"},"cell_type":"code","source":"lr = LogisticRegression(C=10, class_weight={0:0.07 , 1:1})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3feb4ca846682b570b9fba7c9c3757cedb09760a"},"cell_type":"code","source":"lr.fit(X_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"db71812decf2c1248709504508a5e0db7e93a63b"},"cell_type":"code","source":"y_pred_train1 = lr.predict(X_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0daf26d1e662772ab2ca87c1c6600440814f8412"},"cell_type":"code","source":"print(f1_score(y_train, y_pred_train1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2cb6902748dac7c9ec36734dbde087db724a2ee5"},"cell_type":"code","source":"y_pred_val1 = lr.predict(X_val)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"23b301affd3d0f3eb8edec78350d5b542adca345"},"cell_type":"code","source":"print(f1_score(y_val, y_pred_val1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"906255767b80cdbc012e2d66e065571d15256343"},"cell_type":"code","source":"cm1 = confusion_matrix(y_val, y_pred_val1)\ncm1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1394d468031e90912c00efb8ab77e84524ae9a36"},"cell_type":"code","source":"sns.heatmap(cm1, cmap=\"Blues\", annot=True, square=True, fmt=\".0f\");","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"01afef27f308f50e0c1eca096af3c17a95d3c045"},"cell_type":"code","source":"print(classification_report(y_val, y_pred_val1))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e63b5e191ca502913ec94307accd9ab7b3a880ae"},"cell_type":"markdown","source":"## Classification with MultinomialNB:"},{"metadata":{"trusted":true,"_uuid":"fe6f694644613b44318c5189f8fed438331319d4"},"cell_type":"code","source":"mnb = MultinomialNB(alpha=0.1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"78130b702f200ba688592bc5dc449193101d56ec"},"cell_type":"code","source":"mnb.fit(X_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"13671666658291e84939a4f5620dfac7408c26c6"},"cell_type":"code","source":"y_pred_train2 = mnb.predict(X_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e5be723ad8bdbd245c54b70d37540cec58b543cd"},"cell_type":"code","source":"print(f1_score(y_train, y_pred_train2))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2531a87f7ddd01b00b03886b939cce35f22a790c"},"cell_type":"code","source":"y_pred_val2 = mnb.predict(X_val)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8a0eaae0ed772af8c44b500fe12ac6df4ce2a19d"},"cell_type":"code","source":"print(f1_score(y_val, y_pred_val2))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"07b70025920cb01536265819b900576d823762bf"},"cell_type":"code","source":"cm2 = confusion_matrix(y_val, y_pred_val2)\ncm2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b2a690eb9de2c77f3e5e543020b00fb69e577a26"},"cell_type":"code","source":"sns.heatmap(cm2, cmap=\"Blues\", annot=True, square=True, fmt=\".0f\");","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"50e3bb489ec62f1d9d70713d47ce788a1c49a6ac"},"cell_type":"code","source":"print(classification_report(y_val, y_pred_val2))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"57635ca4ed5cda71128b9ea0e4ad0372808a4818"},"cell_type":"markdown","source":"## Classification with Linear SVC:"},{"metadata":{"trusted":true,"_uuid":"37293a242d9c5677a9a1f521ab625844d1b69e34"},"cell_type":"code","source":"from sklearn.svm import LinearSVC\n\nsvc = LinearSVC(C=5, class_weight={0:0.07 , 1:1})\nsvc.fit(X_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0d92fd0a17c68506aa061dc0ecb1473b6718bbc9"},"cell_type":"code","source":"y_pred_train3 = svc.predict(X_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6d117036515b61206d8a0b467221a13c751c1e86"},"cell_type":"code","source":"print(f1_score(y_train, y_pred_train3))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"070b15e5fa7b5a837210631b21db37e5dba36ab9"},"cell_type":"code","source":"y_pred_val3 = svc.predict(X_val)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ad6900cc7c53aecdf7ccac38e559d40359d3cb09"},"cell_type":"code","source":"print(f1_score(y_val, y_pred_val3))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d9c8e76f0da3446dd2302dbce164612e680f08f5"},"cell_type":"code","source":"cm3 = confusion_matrix(y_val, y_pred_val3)\ncm3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"34888cc370a4eeb6a839f8e53795e4dfa3d97cd6"},"cell_type":"code","source":"sns.heatmap(cm3, cmap=\"Blues\", annot=True, square=True, fmt=\".0f\");","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bf44efa11fd1a3e5960d924151795eddf43671d4"},"cell_type":"code","source":"print(classification_report(y_val, y_pred_val3))","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.5"}},"nbformat":4,"nbformat_minor":1}