{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport json\n\nfrom IPython.display import display\n\n#local script\nfrom tfutils_py import get_answer, read_sample\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This kernel is a follow up of the EDA notebook, which can be found here: https://www.kaggle.com/snovaisg/just-another-exploratory-data-analysis"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"# Baseline Concept"},{"metadata":{},"cell_type":"markdown","source":"The idea here is to use very simple techniques to make up our baseline. We will start with making a tfidf representation of each text, then calculating the similarity with each of the long-asnwer candidates. The one with the highest value will be selected as the long-answer candidate.\n\nAlso, we've seen in the EDA that about half of the questions don't have an answer so we should prepare our pipeline to output that. In order to do that, we will study this similarity measure on our train set by comparing the each annotated answer with each of the long-answer candidates, and hopefully discover that there is a threshold value for the similarity which doesn't allow any candidate-answer to be accepted.\n\nLastly since a relative portion (~30%) of the answersable questions (questions which have a long-answer answer) also have a short-answer, we will divide our long-answer candidates into sentences and find the sentence with the highest similarity with the question. Again, we don't want to give a short answer to every question because about 70% of the questions don't have one, so as described in the paragraph above we will study the similarity between the short answers and the text and find out if a threshold value can be discovered which separates well the conclusion that a short answer can or cannot be made to each question."},{"metadata":{},"cell_type":"markdown","source":"# Hypothesis\n\nThere is one hipothesis which this baseline assumes about the dataset, which is:\n\n- All the short-answer annotations can be found in the annotated long-answer;\n\nIt's important we confirm if this is true before beginning our baseline because it's one of the assumptions it makes."},{"metadata":{},"cell_type":"markdown","source":"## Are the short answers taken from the long-answer annotation?"},{"metadata":{"trusted":true},"cell_type":"code","source":"def preprocess(n=10):\n    df = read_sample(n=n,ignore_doc_text=False)\n    df['yes_no'] = df.annotations.apply(lambda x: x[0]['yes_no_answer'])\n    df['long'] = df.annotations.apply(lambda x: [x[0]['long_answer']['start_token'], x[0]['long_answer']['end_token']])\n    df['short'] = df.annotations.apply(lambda x: x[0]['short_answers'])\n    return df\ndf = preprocess(5000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# let's keep a mask of the short answers that exist.\nmask_short_answer_exists = df.short.apply(lambda x: \"Answer Doesn't exist\" if x == [] else \"Answer Exists\") == \"Answer Exists\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Note: We can't forget that some annotations contain more than one short answer."},{"metadata":{"trusted":true},"cell_type":"code","source":"def is_short_in_long(text: str, short: dict, long: dict) -> bool:\n    \"\"\"\n    Checks if a short answer is contained inside the long answer.\n    \"\"\"\n    if short['start_token'] >= long['start_token'] and short['end_token'] <= long['end_token']:\n        return True\n    return False\n\ndef are_shorts_in_long(text: str, shorts: list, long: dict) -> list:\n    \"\"\"\n    Checks for each short answer if they are contained in the long answer.\n    \n    ------\n    Returns a list with the same size of the number of short answers and\n    each position is a boolean determining if each short answer was contained \n    withing the long answer.\n    \"\"\"\n    answer = []\n    for short in shorts:\n        answer.append(is_short_in_long(text,short,long))\n    return answer\n\ndef are_all_shorts_in_long(text: str, shorts: list, long: dict) -> bool:\n    \"\"\"\n    Gets a list of short answers and returns true if all of them were\n    contained withing the long answer and false if otherwise.\n    \"\"\"\n    shorts_in_long = are_shorts_in_long(text,shorts,long)\n    if all(shorts_in_long):\n        return True\n    return False\n\nshort_in_long = df.loc[mask_short_answer_exists].apply(\\\n                    lambda row: are_all_shorts_in_long(row.document_text, \\\n                                                       row.annotations[0]['short_answers'], \\\n                                                       row.annotations[0]['long_answer']\\\n                                                      ),axis=1)\nprint(\"Are all of the short answers annotations always contained within the long_answer annotations?\\n\",\\\n      (\"-> Yes\" if all(short_in_long.values) else \"-> No\"))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Great! We can start building the baseline."},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}