{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Quora Insincere Questions Classification\n\n![title](https://i.ibb.co/GpDpYcW/sincere.jpg)\n\nSide note : this is the first part of two, see the conclusion for the next part.\n\n## Context\nAn existential problem for any major website today is how to handle toxic and divisive content. Quora wants to tackle this problem head-on to keep their platform a place where users can feel safe sharing their knowledge with the world.\n\nQuora is a platform that empowers people to learn from each other. On Quora, people can ask questions and connect with others who contribute unique insights and quality answers. A key challenge is to weed out insincere questions -- those founded upon false premises, or that intend to make a statement rather than look for helpful answers.\n\nIn this competition, Kagglers will develop models that identify and flag insincere questions. To date, Quora has employed both machine learning and manual review to address this problem. With your help, they can develop more scalable methods to detect toxic and misleading content.\n\nHere's your chance to combat online trolls at scale. Help Quora uphold their policy of “Be Nice, Be Respectful” and continue to be a place for sharing and growing the world’s knowledge.\n\n\n## Goal\nPredict whether a question asked on Quora is sincere or not\n\nAn insincere question is defined as a question intended to make a statement rather than look for helpful answers. Some characteristics that can signify that a question is insincere:\n- Has a non-neutral tone\n- Is disparaging or inflammatory\n- Isn't grounded in reality\n- Uses sexual content (incest, bestiality, pedophilia) for shock value, and not to seek genuine answers\n\n## Dataset\nThe training data includes the question that was asked, and whether it was identified as insincere (target = 1). The ground-truth labels contain some amount of noise: they are not guaranteed to be perfect.\n\n\nNote that the distribution of questions in the dataset should not be taken to be representative of the distribution of questions asked on Quora. This is, in part, because of the combination of sampling procedures and sanitization measures that have been applied to the final dataset.\n\n---"},{"metadata":{},"cell_type":"markdown","source":"# Exploratory Data Analysis\n\nLibraries import"},{"metadata":{"trusted":false},"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"import numpy as np \nimport pandas as pd \n\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"from nltk.tokenize import word_tokenize\nfrom nltk.corpus import stopwords\nfrom nltk.stem import PorterStemmer\n\nfrom string import punctuation \n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\nfrom sklearn.metrics import f1_score, classification_report","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"from xgboost import XGBClassifier\nimport lightgbm as lgb","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__File descriptions__\n- train.csv - the training set\n- test.csv - the test set\n- sample_submission.csv - A sample submission in the correct format\n- enbeddings/ - (see below)"},{"metadata":{"trusted":false},"cell_type":"code","source":"df = pd.read_csv(\"../input/quora-insincere-questions-classification/train.csv\")\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__Data fields__\n- qid - unique question identifier\n- question_text - Quora question text\n- target - a question labeled \"insincere\" has a value of 1, otherwise 0"},{"metadata":{"trusted":false},"cell_type":"code","source":"pd.read_csv(\"../input/quora-insincere-questions-classification/test.csv\").head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Basic infos and analysis of the target"},{"metadata":{"trusted":false},"cell_type":"code","source":"df.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"No Nan and no duplicated line :"},{"metadata":{"trusted":false},"cell_type":"code","source":"df.duplicated().sum()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Target analysis"},{"metadata":{"trusted":false},"cell_type":"code","source":"df.target.value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df.target.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"plt.figure(figsize=(5, 4))\nsns.countplot(x='target', data=df)\nplt.title('Reparition of question by sincerity (insincere = 1)');","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"print(f'There are {df.target.sum() / df.shape[0] * 100 :.1f}% of insincere questions, which make the dataset highly unbalanced.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Word clouds"},{"metadata":{},"cell_type":"markdown","source":"Generally, though, data scientists don’t think much of word clouds, in large part because the placement of the words doesn’t mean anything other than “here’s some space where I was able to fit a word.”\nAnyway, clouds can come handy to have a frist insight of the most common words...\n\nWord clouds (also known as text clouds or tag clouds) work in a simple way: the more a specific word appears in a source of textual data (such as a speech, blog post, or database), the bigger and bolder it appears in the word cloud."},{"metadata":{"trusted":false},"cell_type":"code","source":"from wordcloud import WordCloud, STOPWORDS\nstopwords = set(STOPWORDS)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"print('Word cloud image generated from sincere questions')\nsincere_wordcloud = WordCloud(width=600, height=400, background_color ='white', min_font_size = 10).generate(str(df[df[\"target\"] == 0][\"question_text\"]))\n#Positive Word cloud\nplt.figure(figsize=(15,6), facecolor=None)\nplt.imshow(sincere_wordcloud)\nplt.axis(\"off\")\nplt.tight_layout(pad=0)\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"print('Word cloud image generated from INsincere questions')\ninsincere_wordcloud = WordCloud(width=600, height=400, background_color ='white', min_font_size = 10).generate(str(df[df[\"target\"] == 1][\"question_text\"]))\n#Positive Word cloud\nplt.figure(figsize=(15,6), facecolor=None)\nplt.imshow(insincere_wordcloud)\nplt.axis(\"off\")\nplt.tight_layout(pad=0)\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Statistics form the question texts"},{"metadata":{},"cell_type":"markdown","source":"The process of converting data to something a computer can understand is referred to as pre-processing. One of the major forms of pre-processing is to filter out useless data. In natural language processing, useless words (data), are referred to as stop words."},{"metadata":{"trusted":false},"cell_type":"code","source":"# if needed\n# nltk.download('stopwords')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Stop Words: A stop word is a commonly used word (such as “the”, “a”, “an”, “in”) that a search engine has been programmed to ignore, both when indexing entries for searching and when retrieving them as the result of a search query.\n\nWe would not want these words taking up space in our database, or taking up valuable processing time. For this, we can remove them easily, by storing a list of words that you consider to be stop words. NLTK(Natural Language Toolkit) in python has a list of stopwords stored in 16 different languages. You can find them in the nltk_data directory."},{"metadata":{"trusted":false},"cell_type":"code","source":"import nltk\nfrom nltk.corpus import stopwords\nstop_words = set(stopwords.words('english'))\nstop_words","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"def create_features(df_):\n    \"\"\"Retrieve from the text column the nb of : words, unique words, characters, stopwords,\n    punctuations, upper/lower case char, title...\"\"\"\n    \n    df_[\"nb_words\"] = df_[\"question_text\"].apply(lambda x: len(x.split()))\n    df_[\"nb_unique_words\"] = df_[\"question_text\"].apply(lambda x: len(set(str(x).split())))\n    df_[\"nb_chars\"] = df[\"question_text\"].apply(lambda x: len(str(x)))\n    df_[\"nb_stopwords\"] = df_[\"question_text\"].apply(lambda x : len([nw for nw in str(x).split() if nw.lower() in stop_words]))\n    df_[\"nb_punctuation\"] = df_[\"question_text\"].apply(lambda x : len([np for np in str(x) if np in punctuation]))\n    df_[\"nb_uppercase\"] = df_[\"question_text\"].apply(lambda x : len([nu for nu in str(x).split() if nu.isupper()]))\n    df_[\"nb_lowercase\"] = df_[\"question_text\"].apply(lambda x : len([nl for nl in str(x).split() if nl.islower()]))\n    df_[\"nb_title\"] = df_[\"question_text\"].apply(lambda x : len([nl for nl in str(x).split() if nl.istitle()]))\n    return df_","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df = create_features(df)\ndf.sample(2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's take a sample - because the data set is quite huge when run locally on a single node - and visualize pair plots :"},{"metadata":{"trusted":false},"cell_type":"code","source":"num_feat = ['nb_words', 'nb_unique_words', 'nb_chars', 'nb_stopwords', \\\n            'nb_punctuation', 'nb_uppercase', 'nb_lowercase', 'nb_title', 'target'] \n# side note : remove target if needed later\n\ndf_sample = df[num_feat].sample(n=round(df.shape[0]/6), random_state=42)\n\nplt.figure(figsize=(16,10))\nsns.pairplot(data=df_sample, hue='target')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Basic stats comparison :"},{"metadata":{"trusted":false},"cell_type":"code","source":"df_sample[df_sample['target'] == 0].describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df_sample[df_sample['target'] == 1].describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Generally speaking, insincere questions are written with more words."},{"metadata":{},"cell_type":"markdown","source":"Now with a focus on the distributions, because there is a difference in the spikes between sincere and insincre questions."},{"metadata":{"trusted":false},"cell_type":"code","source":"plt.figure(figsize=(10,10))\nplt.subplot(331)\n\ni=0\nfor c in num_feat:\n    plt.subplot(3, 3, i+1)\n    i += 1\n    sns.kdeplot(df_sample[df_sample['target'] == 0][c], shade=True)\n    sns.kdeplot(df_sample[df_sample['target'] == 1][c], shade=False)\n    plt.title(c)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Same conclusion here than shown with stats"},{"metadata":{},"cell_type":"markdown","source":"Obviously, many of these indicators are highly correlated each other but not towards the target :"},{"metadata":{"trusted":false},"cell_type":"code","source":"sns.set(style=\"white\")\n\n# Compute the correlation matrix\ncorr = df_sample[num_feat].corr()\n\n# Generate a mask for the upper triangle\nmask = np.zeros_like(corr, dtype=np.bool)\nmask[np.triu_indices_from(mask)] = True\n\n# Set up the matplotlib figure\nf, ax = plt.subplots(figsize=(7, 6))\n\n# Generate a custom diverging colormap\ncmap = sns.diverging_palette(220, 10, as_cmap=True)\n\n# Draw the heatmap with the mask and correct aspect ratio\nsns.heatmap(corr, mask=mask, cmap=cmap, vmax=.3, center=0,\n            square=True, linewidths=.5, annot=True, cbar_kws={\"shrink\": .5})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"What are the most frequent words for each type of question ?"},{"metadata":{"trusted":false},"cell_type":"code","source":"class Vocabulary(object):\n    # credits : Shankar G see https://www.kaggle.com/kaosmonkey/visualize-sincere-vs-insincere-words\n    \n    def __init__(self):\n        self.vocab = {}\n        self.STOPWORDS = set()\n        self.STOPWORDS = set(stopwords.words('english'))\n        \n    def build_vocab(self, lines):\n        for line in lines:\n            for word in line.split(' '):\n                word = word.lower()\n                if (word in self.STOPWORDS):\n                    continue\n                if (word not in self.vocab):\n                    self.vocab[word] = 0\n                self.vocab[word] +=1 \n    \n    def generate_ngrams(text, n_gram=1):\n        \"\"\"arg: text, n_gram\"\"\"\n        token = [token for token in text.lower().split(\" \") if token != \"\" if token not in STOPWORDS]\n        ngrams = zip(*[token[i:] for i in range(n_gram)])\n        return [\" \".join(ngram) for ngram in ngrams]\n    \n    def horizontal_bar_chart(df, color):\n        trace = go.Bar(\n            y=df[\"word\"].values[::-1],\n            x=df[\"wordcount\"].values[::-1],\n            showlegend=False,\n            orientation = 'h',\n            marker=dict(\n            color=color,\n            ),\n        )\n        return trace","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"sincere_vocab = Vocabulary()\nsincere_vocab.build_vocab(df[df['target'] == 0]['question_text'])\nsincere_vocabulary = sorted(sincere_vocab.vocab.items(), reverse=True, key=lambda kv: kv[1])\n    \ndf_sincere_vocab = pd.DataFrame(sincere_vocabulary, columns=['word_sincere', 'frequency'])\nsns.barplot(y='word_sincere', x='frequency', data=df_sincere_vocab[:20])","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"insincere_vocab = Vocabulary()\ninsincere_vocab.build_vocab(df[df['target'] == 1]['question_text'])\ninsincere_vocabulary = sorted(insincere_vocab.vocab.items(), reverse=True, key=lambda kv: kv[1])\n\ndf_insincere_vocab = pd.DataFrame(insincere_vocabulary, columns=['word_insincere', 'frequency'])\nsns.barplot(y='word_insincere', x='frequency', data=df_insincere_vocab[:20])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we can clearly see there are certain words (swear words, discriminatory words based on race, political figures etc) that show up a lot in insincere sentences.\n\n---"},{"metadata":{},"cell_type":"markdown","source":"# Text processing & model training"},{"metadata":{},"cell_type":"markdown","source":"## Metric : F-score\n\nThe most appropriated metric is F1-score. Explanation from [Wikipedia](https://en.wikipedia.org/wiki/F1_score):\n\n*\"In statistical analysis of binary classification, the F1 score (also F-score or F-measure) is a measure of a test's accuracy. It considers both the precision p and the recall r of the test to compute the score: p is the number of correct positive results divided by the number of all positive results returned by the classifier, and r is the number of correct positive results divided by the number of all relevant samples (all samples that should have been identified as positive). The F1 score is the harmonic mean of the precision and recall, where an F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0.\"*"},{"metadata":{},"cell_type":"markdown","source":"![title](https://upload.wikimedia.org/wikipedia/commons/thumb/2/26/Precisionrecall.svg/350px-Precisionrecall.svg.png)"},{"metadata":{},"cell_type":"markdown","source":"we'll also use a [confusion matrix](https://en.wikipedia.org/wiki/Confusion_matrix) "},{"metadata":{"trusted":false},"cell_type":"code","source":"def get_fscore_matrix(fitted_clf, model_name):\n    print(model_name, ' :')\n    \n    # get classes predictions for the classification report \n    y_train_pred, y_pred = fitted_clf.predict(X_train), fitted_clf.predict(X_test)\n    print(classification_report(y_test, y_pred), '\\n') # target_names=y\n    \n    # computes probabilities keep the ones for the positive outcome only      \n    print(f'F1-score = {f1_score(y_test, y_pred):.2f}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Text processing"},{"metadata":{"trusted":false},"cell_type":"code","source":"# if needed the first time  \n# import nltk\n# nltk.download('punkt')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__Process :__\n    - tokenization\n    - keeping only alphanumeriacl characters\n    - removing stop words (punctuation etc...)\n    - stemming or lemmatization. \n\n[source : blog.bitext.com](https://blog.bitext.com/what-is-the-difference-between-stemming-and-lemmatization/)\n\n__Tokenization :__\nGiven a character sequence and a defined document unit, tokenization is the task of chopping it up into pieces, called tokens , perhaps at the same time throwing away certain characters, such as punctuation\n\n__Stemming vs. lemmatization :__\nThe aim of both processes is the same: reducing the inflectional forms of each word into a common base or root. \n\nStemming algorithms work by cutting off the end or the beginning of the word, taking into account a list of common prefixes and suffixes that can be found in an inflected word. This indiscriminate cutting can be successful in some occasions, but not always, and that is why we affirm that this approach presents some limitations.\n\nLemmatization, on the other hand, takes into consideration the morphological analysis of the words. To do so, it is necessary to have detailed dictionaries which the algorithm can look through to link the form back to its lemma. "},{"metadata":{"trusted":false},"cell_type":"code","source":"df = df[['question_text', 'target']]\n\ndef text_processing(local_df):\n    \"\"\" return the dataframe with tokens stemmetized without numerical values & stopwords \"\"\"\n    stemmer = PorterStemmer()\n    # Perform preprocessing\n    local_df['txt_processed'] = local_df['question_text'].apply(lambda df: word_tokenize(df))\n    local_df['txt_processed'] = local_df['txt_processed'].apply(lambda x: [item for item in x if item.isalpha()])\n    local_df['txt_processed'] = local_df['txt_processed'].apply(lambda x: [item for item in x if item not in stop_words])\n    local_df['txt_processed'] = local_df['txt_processed'].apply(lambda x: [stemmer.stem(item) for item in x])\n    return local_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df = text_processing(df)\ndf.tail(2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## First method : text similarity using TF-IDF"},{"metadata":{"trusted":false},"cell_type":"code","source":"vectorizer = TfidfVectorizer(lowercase=False, analyzer=lambda x: x, min_df=0.01, max_df=0.999)\n# min_df & max_df param added for less memory usage\n\ntf_idf = vectorizer.fit_transform(df['txt_processed']).toarray()\npd.DataFrame(tf_idf, columns=vectorizer.get_feature_names()).head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"# Split the data\nX_train, X_test, y_train, y_test = train_test_split(tf_idf, df['target'], test_size=0.2, random_state=42)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"XGBoost Classifier without weigths"},{"metadata":{"trusted":false},"cell_type":"code","source":"model = XGBClassifier(objective=\"binary:logistic\")\nmodel.fit(X_train, y_train)\nget_fscore_matrix(model, 'XGB Clf withOUT weights')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"XGBoost Classifier with weigths"},{"metadata":{"trusted":false},"cell_type":"code","source":"ratio = ((len(y_train) - y_train.sum()) - y_train.sum()) / y_train.sum()\nratio","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"model = XGBClassifier(objective=\"binary:logistic\", scale_pos_weight=ratio)\nmodel.fit(X_train, y_train)\nget_fscore_matrix(model, 'XGB Clf WITH weights')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now LGBM with weights"},{"metadata":{"trusted":false},"cell_type":"code","source":"model = lgb.LGBMClassifier(n_jobs = -1, class_weight={0:y_train.sum(), 1:len(y_train) - y_train.sum()})\nmodel.fit(X_train, y_train)\nget_fscore_matrix(model, 'LGBM weighted')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"LogisticRegression"},{"metadata":{"trusted":false},"cell_type":"code","source":"model = LogisticRegression(class_weight={0:y_train.sum(), 1:len(y_train) - y_train.sum()}, C=0.5, max_iter=100, n_jobs=-1)\nmodel.fit(X_train, y_train)\nget_fscore_matrix(model, 'LogisticRegression')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Second approach : a CountVectorizer / Logistic Regression pipeline "},{"metadata":{},"cell_type":"markdown","source":"Convert a collection of text documents to a matrix of token counts\n\nThis implementation produces a sparse representation of the counts using scipy.sparse.csr_matrix."},{"metadata":{"trusted":false},"cell_type":"code","source":"df['str_processed'] = df['txt_processed'].apply(lambda x: \" \".join(x))\ndf.head(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"pipeline = Pipeline([(\"cv\", CountVectorizer(analyzer=\"word\", ngram_range=(1,4), max_df=0.9)),\n                     (\"clf\", LogisticRegression(solver=\"saga\", class_weight=\"balanced\", C=0.45, max_iter=250, verbose=1, n_jobs=-1))])","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"X_train, X_test, y_train, y_test = train_test_split(df['str_processed'], df.target, test_size=0.2, stratify = df.target.values)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"lr_model = pipeline.fit(X_train, y_train)\nlr_model","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"get_fscore_matrix(lr_model, 'lr_pipe')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Conclusion, submission and opening"},{"metadata":{},"cell_type":"markdown","source":"- First we have used TF-IDF, but the least we can say : this is not really efficient, the recall for insincere question isn't good at all, so this seems to not be the right way to go...\n- Instead, using CountVectorizer with a Logistic Regression is more efficient. \n\nNow let's see what will be the submission score ?"},{"metadata":{"trusted":false},"cell_type":"code","source":"pd.read_csv(\"../input/quora-insincere-questions-classification/sample_submission.csv\").head(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df_test = pd.read_csv(\"../input/quora-insincere-questions-classification/test.csv\", index_col='qid')\ndf_test.tail(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df_test = text_processing(df_test)\ndf_test['str_processed'] = df_test['txt_processed'].apply(lambda x: \" \".join(x))\ndf_test.head(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"y_pred_final = lr_model.predict(df_test['str_processed'])\ny_pred_final","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df_submission = pd.DataFrame({\"qid\":df_test.index, \"prediction\":y_pred_final})\ndf_submission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"df_submission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__CREDITS__ : all the people mentionned above and especially [amokrane](https://github.com/atabti) & [moneynass](https://github.com/moneynass) for their inspiring work ! thanks  :)"},{"metadata":{},"cell_type":"markdown","source":"-> *IN THE 2nd  PART  I'LL USE WORD *ENBEDDINGS* !* "},{"metadata":{},"cell_type":"markdown","source":"__if you appreciated my work, your vote is warmly welcome !__"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.3"},"toc":{"base_numbering":1,"nav_menu":{},"number_sections":true,"sideBar":true,"skip_h1_title":false,"title_cell":"Table of Contents","title_sidebar":"Contents","toc_cell":false,"toc_position":{},"toc_section_display":true,"toc_window_display":false}},"nbformat":4,"nbformat_minor":1}