{"cells":[{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"path = '../input/riiid-test-answer-prediction/'\nimport pandas as pd\nquestion = pd.read_csv(path + 'questions.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since the organizer of this competition has announced the source of the dataset (EdNET), I'm going to discuss about some of the domain specific characteristics of the data. \n\nBased on their [EdNet paper](https://arxiv.org/pdf/1912.03072.pdf), EdNet is comprised of data collected by Santa, which is an AI tutoring service that is supposed to help students prepare for their TOEIC Listening and Reading test. TOEIC is a well-known test in South Korea for assessing students' English proficiency. \n\nTOEIC is comprised of 7 parts, which is why there are 7 unique values in `part` column of questions.csv file. "},{"metadata":{"trusted":true},"cell_type":"code","source":"question['part'].unique()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"On TOEIC, Each part can be explained as follows.\n\n**Listening** - test takers have to listen to spoken English and answer questions\n\n- Part 1: Photographs - select a spoken sentence that best describes the given picture\n- Part 2: Question-Response - select the appropriate response to the spoken question\n- Part 3: Conversations - select the appropriate answers to questions related to a spoken conversation between 2~3 people\n- Part 4: Talks - select the appropriate answers to questions related to a spoken short talk\n\n**Reading**  - test takers have to read a sentence or a passage to answer questions\n\n- Part 5: Incomplete Sentences - select the correct word that fills the blank in a sentence so it becomes complete in syntax-wise or semantic-wise\n- Part 6: Text Completion - select the correct answer that fills the blank in a passage so it becomes complete in syntax-wise and semantic-wise\n- Part 7: Reading Comprehension - select the correct answers to questions related to a given passage\n\nI'm also assuming the numbers on `part` column correspond exactly one-on-one to the part numbers on TOEIC because they show same characteristics. "},{"metadata":{},"cell_type":"markdown","source":"Reason 1: On TOEIC, Part 2 has only three available options. Part 2 on questions.csv also has only three distinct values on `correct_answer` column, namely 0, 1, and 3. "},{"metadata":{"trusted":true},"cell_type":"code","source":"question.groupby('part')['correct_answer'].unique()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Reason 2: On TOEIC, each questions comes seperately in Part 1, 2, and 5. However, the questions in Part 3, 4, 6, and 7 come in bundle. On the `question.csv`, the part numbers have the exact same characteristics. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# 1 question for each bundle\nquestion.query('part == 1').groupby('bundle_id').count().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# 1 question for each bundle\nquestion.query('part == 2').groupby('bundle_id').count().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# many questions for each bundle\nquestion.query('part == 3').groupby('bundle_id').count().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# many questions for each bundle\nquestion.query('part == 4').groupby('bundle_id').count().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# 1 question for each bundle\nquestion.query('part == 5').groupby('bundle_id').count().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# many questions for each bundle\nquestion.query('part == 6').groupby('bundle_id').count().head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# many questions for each bundle\nquestion.query('part == 7').groupby('bundle_id').count().head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Some of my thoughts listed below\n- Not sure if it is possible to reflect these kinds of TOEIC specific characteristics to make a domain specific model. \n- Not sure whether it will have a good performance.\n- Not sure whether the organizers and sponsors will like it because they are probably expecting for a domain-agnostic model. "},{"metadata":{},"cell_type":"markdown","source":"You can find more information on the TOEIC website\n\n- [TOEIC website](https://www.ets.org/toeic)\n- [TOEIC sample test by ETS](https://www.ets.org/s/toeic/pdf/toeic-listening-reading-sample-test-updated.pdf)\n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}