{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## NLP Tutorial\n\nNLP - or *Natural Language Processing* - is shorthand for a wide array of techniques designed to help machines learn from text. Natural Language Processing powers everything from chatbots to search engines, and is used in diverse tasks like sentiment analysis and machine translation.\n\nIn this tutorial we'll look at this competition's dataset, use a simple technique to process it, build a machine learning model, and submit predictions for a score!","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom sklearn import feature_extraction, linear_model, model_selection, preprocessing","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-08-08T20:01:01.929912Z","iopub.execute_input":"2022-08-08T20:01:01.930232Z","iopub.status.idle":"2022-08-08T20:01:05.096924Z","shell.execute_reply.started":"2022-08-08T20:01:01.930176Z","shell.execute_reply":"2022-08-08T20:01:05.095946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.read_csv(\"/kaggle/input/nlp-getting-started/train.csv\")\ntest_df = pd.read_csv(\"/kaggle/input/nlp-getting-started/test.csv\")","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","execution":{"iopub.status.busy":"2022-08-08T20:01:05.099642Z","iopub.execute_input":"2022-08-08T20:01:05.099957Z","iopub.status.idle":"2022-08-08T20:01:05.179520Z","shell.execute_reply.started":"2022-08-08T20:01:05.099904Z","shell.execute_reply":"2022-08-08T20:01:05.178731Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### A quick look at our data\n\nLet's look at our data... first, an example of what is NOT a disaster tweet.","metadata":{}},{"cell_type":"code","source":"train_df[train_df[\"target\"] == 0][\"text\"].values[1]","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:05.180909Z","iopub.execute_input":"2022-08-08T20:01:05.181464Z","iopub.status.idle":"2022-08-08T20:01:05.201008Z","shell.execute_reply.started":"2022-08-08T20:01:05.181402Z","shell.execute_reply":"2022-08-08T20:01:05.199467Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And one that is:","metadata":{}},{"cell_type":"code","source":"train_df[train_df[\"target\"] == 1][\"text\"].values[1]","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:05.202655Z","iopub.execute_input":"2022-08-08T20:01:05.203050Z","iopub.status.idle":"2022-08-08T20:01:05.212438Z","shell.execute_reply.started":"2022-08-08T20:01:05.202975Z","shell.execute_reply":"2022-08-08T20:01:05.211456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Building vectors\n\nThe theory behind the model we'll build in this notebook is pretty simple: the words contained in each tweet are a good indicator of whether they're about a real disaster or not (this is not entirely correct, but it's a great place to start).\n\nWe'll use scikit-learn's `CountVectorizer` to count the words in each tweet and turn them into data our machine learning model can process.\n\nNote: a `vector` is, in this context, a set of numbers that a machine learning model can work with. We'll look at one in just a second.","metadata":{}},{"cell_type":"code","source":"count_vectorizer = feature_extraction.text.CountVectorizer()\n\n## let's get counts for the first 5 tweets in the data\nexample_train_vectors = count_vectorizer.fit_transform(train_df[\"text\"][0:5])","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:05.215835Z","iopub.execute_input":"2022-08-08T20:01:05.216275Z","iopub.status.idle":"2022-08-08T20:01:05.234046Z","shell.execute_reply.started":"2022-08-08T20:01:05.216210Z","shell.execute_reply":"2022-08-08T20:01:05.232683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## we use .todense() here because these vectors are \"sparse\" (only non-zero elements are kept to save space)\nprint(example_train_vectors[0].todense().shape)\nprint(example_train_vectors[0].todense())","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:05.235799Z","iopub.execute_input":"2022-08-08T20:01:05.236356Z","iopub.status.idle":"2022-08-08T20:01:05.243508Z","shell.execute_reply.started":"2022-08-08T20:01:05.236303Z","shell.execute_reply":"2022-08-08T20:01:05.242285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above tells us that:\n1. There are 54 unique words (or \"tokens\") in the first five tweets.\n2. The first tweet contains only some of those unique tokens - all of the non-zero counts above are the tokens that DO exist in the first tweet.\n\nNow let's create vectors for all of our tweets.","metadata":{}},{"cell_type":"code","source":"train_vectors = count_vectorizer.fit_transform(train_df[\"text\"])\n\n## note that we're NOT using .fit_transform() here. Using just .transform() makes sure\n# that the tokens in the train vectors are the only ones mapped to the test vectors - \n# i.e. that the train and test vectors use the same set of tokens.\ntest_vectors = count_vectorizer.transform(test_df[\"text\"])","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:05.245209Z","iopub.execute_input":"2022-08-08T20:01:05.245794Z","iopub.status.idle":"2022-08-08T20:01:05.588664Z","shell.execute_reply.started":"2022-08-08T20:01:05.245578Z","shell.execute_reply":"2022-08-08T20:01:05.587538Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Our model\n\nAs we mentioned above, we think the words contained in each tweet are a good indicator of whether they're about a real disaster or not. The presence of particular word (or set of words) in a tweet might link directly to whether or not that tweet is real.\n\nWhat we're assuming here is a _linear_ connection. So let's build a linear model and see!","metadata":{}},{"cell_type":"code","source":"## Our vectors are really big, so we want to push our model's weights\n## toward 0 without completely discounting different words - ridge regression \n## is a good way to do this.\nclf = linear_model.RidgeClassifier()","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:05.590392Z","iopub.execute_input":"2022-08-08T20:01:05.590827Z","iopub.status.idle":"2022-08-08T20:01:05.596589Z","shell.execute_reply.started":"2022-08-08T20:01:05.590754Z","shell.execute_reply":"2022-08-08T20:01:05.594990Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's test our model and see how well it does on the training data. For this we'll use `cross-validation` - where we train on a portion of the known data, then validate it with the rest. If we do this several times (with different portions) we can get a good idea for how a particular model or method performs.\n\nThe metric for this competition is F1, so let's use that here.","metadata":{}},{"cell_type":"code","source":"scores = model_selection.cross_val_score(clf, train_vectors, train_df[\"target\"], cv=3, scoring=\"f1\")\nscores","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:05.598205Z","iopub.execute_input":"2022-08-08T20:01:05.598723Z","iopub.status.idle":"2022-08-08T20:01:06.942928Z","shell.execute_reply.started":"2022-08-08T20:01:05.598671Z","shell.execute_reply":"2022-08-08T20:01:06.941792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above scores aren't terrible! It looks like our assumption will score roughly 0.65 on the leaderboard. There are lots of ways to potentially improve on this (TFIDF, LSA, LSTM / RNNs, the list is long!) - give any of them a shot!\n\nIn the meantime, let's do predictions on our training set and build a submission for the competition.","metadata":{}},{"cell_type":"code","source":"clf.fit(train_vectors, train_df[\"target\"])","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:06.944440Z","iopub.execute_input":"2022-08-08T20:01:06.944787Z","iopub.status.idle":"2022-08-08T20:01:07.661085Z","shell.execute_reply.started":"2022-08-08T20:01:06.944724Z","shell.execute_reply":"2022-08-08T20:01:07.660245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission = pd.read_csv(\"/kaggle/input/nlp-getting-started/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:07.662313Z","iopub.execute_input":"2022-08-08T20:01:07.662811Z","iopub.status.idle":"2022-08-08T20:01:07.676359Z","shell.execute_reply.started":"2022-08-08T20:01:07.662764Z","shell.execute_reply":"2022-08-08T20:01:07.675627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission[\"target\"] = clf.predict(test_vectors)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:07.678143Z","iopub.execute_input":"2022-08-08T20:01:07.678826Z","iopub.status.idle":"2022-08-08T20:01:07.685311Z","shell.execute_reply.started":"2022-08-08T20:01:07.678765Z","shell.execute_reply":"2022-08-08T20:01:07.684136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:07.686617Z","iopub.execute_input":"2022-08-08T20:01:07.686989Z","iopub.status.idle":"2022-08-08T20:01:07.711221Z","shell.execute_reply.started":"2022-08-08T20:01:07.686947Z","shell.execute_reply":"2022-08-08T20:01:07.710412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-08T20:01:07.712913Z","iopub.execute_input":"2022-08-08T20:01:07.713794Z","iopub.status.idle":"2022-08-08T20:01:08.286182Z","shell.execute_reply.started":"2022-08-08T20:01:07.713737Z","shell.execute_reply":"2022-08-08T20:01:08.285224Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, in the viewer, you can submit the above file to the competition! Good luck!","metadata":{}}]}