{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Problem - Original Post\nIt seems that this dataset has different examples in the `sample_submission.csv` as compared to the test set `simplified-nq-test.jsonl`. This very short kernel illustrates that difference."},{"metadata":{},"cell_type":"markdown","source":"## Update\nThanks to comments by @mourad below and @kashnitsky in [this thread](https://www.kaggle.com/c/tensorflow2-question-answering/discussion/123781https://www.kaggle.com/c/tensorflow2-question-answering/discussion/123781), I have realised this is not a data conflict but an error in my code. My code uses `pandas.read_json` without specifying data type, so the `example_id` is read as an integer type with insufficient accuracy for the 20-digit id.\n\nI have added a solution (specifying 'Object' data type) in the last cell of this notebook, in case anyone else gets stuck on a similar problem."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import pandas as pd","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample = pd.read_csv('/kaggle/input/tensorflow2-question-answering/sample_submission.csv').set_index('example_id')\nsample.sort_index().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test = pd.read_json('/kaggle/input/tensorflow2-question-answering/simplified-nq-test.jsonl', orient = 'records', lines = True).set_index('example_id')\ntest.index = test.index.astype(str)\ntest.sort_index().head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Solution\nSpecify dtype as 'Object' for `example_id`"},{"metadata":{"trusted":true},"cell_type":"code","source":"test2 = pd.read_json('/kaggle/input/tensorflow2-question-answering/simplified-nq-test.jsonl',\n                    dtype = {'example_id': 'Object'},\n                    lines = True).set_index('example_id')\ntest2.sort_index().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}