{"cells":[{"metadata":{"_uuid":"85c4d69f-1ae5-4f75-8543-7aad79c1d56e","_cell_guid":"37922b70-de03-4366-a46b-484ff36a9924","trusted":true},"cell_type":"code","source":"from sklearn.metrics import roc_auc_score\nfrom sklearn.model_selection import train_test_split\n\nimport pandas as pd\nfrom pandas.plotting import scatter_matrix\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport warnings\nwarnings.simplefilter('ignore')\nfrom sklearn.metrics import roc_auc_score, confusion_matrix\nfrom sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV, learning_curve\nfrom sklearn.utils import shuffle\n\nimport riiideducation","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"064bdc4f-347f-4af9-b6e9-a718659bc893","_cell_guid":"5b8bdceb-1702-436b-ae8d-5fcf447cd193","trusted":true},"cell_type":"markdown","source":"Вечер в хату, арестанты"},{"metadata":{"_uuid":"9e7ac059-6e04-4766-ab26-93bb3a419754","_cell_guid":"d6c42f31-2d3e-4e84-aa45-32efa62ef8ef","trusted":true},"cell_type":"code","source":"types = {\n        'row_id': 'int64', \n        'timestamp': 'int64', \n        'user_id': 'int32', \n        'content_id': 'int16', \n        'content_type_id': 'int8',\n        'task_container_id': 'int16', \n        'user_answer': 'int8', \n        'answered_correctly': 'int64', \n        'prior_question_elapsed_time': 'float32', \n        'prior_question_had_explanation': 'boolean'\n}","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"71253cec-1249-42f4-87ac-16abe951be36","_cell_guid":"c1d468a5-d073-40d7-865b-9b483f8de6f4","trusted":true},"cell_type":"code","source":"train = pd.read_csv(\n    '/kaggle/input/riiid-test-answer-prediction/train.csv', \n    low_memory=False, \n    nrows=10**7, \n    dtype=types\n)\nquestions=pd.read_csv('/kaggle/input/riiid-test-answer-prediction/questions.csv',\n                      dtype={'question_id': 'int16','part': 'int8','bundle_id': 'int16','tags': 'str'})\nlectures=pd.read_csv('/kaggle/input/riiid-test-answer-prediction/lectures.csv',\n                    dtype={'lectures_id': 'int16','tag': 'int16','part': 'int8','type_of': 'str'})\nex_test=pd.read_csv('/kaggle/input/riiid-test-answer-prediction/example_test.csv')\n\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"12855127-f151-4d85-90cb-072b8cb53587","_cell_guid":"50ed169b-5e84-4b41-9176-189c6489f23c","trusted":true},"cell_type":"markdown","source":"**train**\n\nrow_id: (int64) ID code for the row.\n\ntimestamp: (int64) the time in milliseconds between this user interaction and the first event completion from that user.\n\nuser_id: (int32) ID code for the user.\n\ncontent_id: (int16) ID code for the user interaction\n\ncontent_type_id: (int8) 0 if the event was a question being posed to the user, 1 if the event was the user watching a lecture.\n\ntask_container_id: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a task_container_id.\n\nuser_answer: (int8) the user's answer to the question, if any. Read -1 as null, for lectures.\n\nanswered_correctly: (int8) if the user responded correctly. Read -1 as null, for lectures.\n\nprior_question_elapsed_time: (float32) The average time in milliseconds it took a user to answer each question in the previous question bundle, ignoring any lectures in between. Is null for a user's first question bundle or lecture. Note that the time is the average time a user took to solve each question in the previous bundle.\n\nprior_question_had_explanation: (bool) Whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle, ignoring any lectures in between. The value is shared across a single question bundle, and is null for a user's first question bundle or lecture. Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback."},{"metadata":{"_uuid":"ba616d5c-a1e7-4c07-8a19-72a18bffa10c","_cell_guid":"59a43f5f-d638-42e8-9a22-2c085a3f9f2d","trusted":true},"cell_type":"code","source":"questions.tail()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2a60e113-d36d-4ae3-bbe7-a668725a9c9c","_cell_guid":"edf81645-7b08-4532-8340-7363c1166f1c","trusted":true},"cell_type":"markdown","source":"**QUESTIONS.CSV**\n\nquestion_id = content_id: foreign key for the train/test content_id column, when the content type is question (0).\n\nbundle_id: code for which questions are served together.\n\ncorrect_answer: the answer to the question. Can be compared with the train user_answer column to check if the user was right.\n\npart: top level category code for the question.\n\ntags: one or more detailed tag codes for the question. The meaning of the tags will not be provided, but these codes are sufficient for clustering the questions together."},{"metadata":{"_uuid":"9d38dcad-1c24-4128-b410-de6882779271","_cell_guid":"1ebd4044-a68e-45c6-af73-fb51d9a8a22b","trusted":true},"cell_type":"code","source":"questions.iloc[5716]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6faf6294-d57d-446f-94b1-eb3309371502","_cell_guid":"2387b146-6939-4787-a79d-0165a95559a3","trusted":true},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"380a55c5-c1d4-44b6-8199-bf73f1ce2ce2","_cell_guid":"7250a8cd-127b-418b-909c-7a5822b599ac","trusted":true},"cell_type":"code","source":"lectures.head(10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f2ea0010-6d4d-4439-a285-b09b123e3466","_cell_guid":"4eb61d7c-d15a-4fc3-ba29-e9428eecbde0","trusted":true},"cell_type":"markdown","source":"lectures.csv: metadata for the lectures watched by users as they progress in their education.\n\nlecture_id: foreign key for the train/test content_id column, when the content type is lecture (1).\n\npart: top level category code for the lecture.\n\ntag: one tag codes for the lecture. The meaning of the tags will not be provided, but these codes are sufficient for clustering the lectures together.\n\ntype_of: brief description of the core purpose of the lecture"},{"metadata":{"_uuid":"ba737c05-dc0c-491c-a53d-fe782a0ca313","_cell_guid":"cfcd9793-5cd2-4b46-bfd6-054ab66d030a","trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e19135d6-8fb2-4acc-8f4f-0bf9c07a3911","_cell_guid":"6df3b05c-7540-4b78-af64-5ebb6756bd40","trusted":true},"cell_type":"code","source":"ex_test.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a1cec7e4-2145-43bf-acd1-b4715d55138e","_cell_guid":"76d8b01d-0776-4951-8003-bfaf86250509","trusted":true},"cell_type":"code","source":"ex_test.head(50)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0c1eef0b-0e07-428c-b59a-8d5f450163a0","_cell_guid":"bee32008-5708-4851-b33f-77b1f5b21954","trusted":true},"cell_type":"markdown","source":"example_test_rows.csv Three sample groups of the test set data as it will be delivered by the time-series API. The format is largely the same as train.csv. There are two different columns that mirror what information the AI tutor actually has available at any given time, but with the user interactions grouped together for the sake of API performance rather than strictly showing information for a single user at a time. Some users will appear in the hidden test set that have NOT been presented in the train set, emulating the challenge of quickly adapting to modeling new arrivals to a website.\n\nprior_group_responses (string) provides all of the user_answer entries for previous group in a string representation of a list in the first row of the group. All other rows in each group are null. If you are using Python, you will likely want to call eval on the non-null rows. Some rows may be null, or empty lists.\n\nprior_group_answers_correct (string) provides all the answered_correctly field for previous group, with the same format and caveats as prior_group_responses. Some rows may be null, or empty lists."},{"metadata":{"_uuid":"bd29593d-9072-48f6-89d7-78b7b517591e","_cell_guid":"a409a647-632c-4ffe-afe9-e0940f5ccf2d","trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c37f667f-e005-4114-b4a2-eb35f5bf5d81","_cell_guid":"88558b34-16b1-441a-b2b9-7fed2211c814","trusted":true},"cell_type":"markdown","source":"**Какие признаки нам помогут для предсказаний?**"},{"metadata":{"_uuid":"5368f2a6-b980-43af-8f7a-f5ae94e8f081","_cell_guid":"3a216053-7b1d-4e99-9782-0bf0b3beeb56","trusted":true},"cell_type":"markdown","source":"1) запоминаем историю ответов пользователя ( долю правильных ответов конкретного пользователя = accuracy)\n\n2) запоминаем историю овтетов всех пользователей на конкретные вопросы( долю правильных ответов на конкретный вопрос)\n\n????3) запоминаем историю ответов для блоков вопросов ( но нужно ли? мб 2 уже включает это)\n\n4) смотеть, просмотрены ли лекции по материалам вопроса( через скалярное произведение tag question на tag просмотренных лекций, делёное на кол-во tag вопроса)\n\n5??5-6) запоминаем среднее время на правильный и/или неправильный ответ для конкретного пользователя(prior_question_elapsed_time делаем lag -1) или запоминаем просто среднее время\nили только ср вр на правильный ответ\n\n7??7-8) запоминаем среднее время на правильный и/или неправильный ответ для конкретного вопроса  \nили запоминаем просто среднее время или только ср вр на правильный ответ\n\n9+) запоминаем accuracy ответов для каждой из групп вопросов, полученных после класстеризации вопросов по tag questions\n\n10+)  запоминаем среднее время ответов для каждой из групп вопросов, полученных после класстеризации вопросов по tag questions"},{"metadata":{"_uuid":"35237b0d-e0a4-4ead-bfab-92257c2181fc","_cell_guid":"0effd2bd-ace2-49d4-8a9c-c0600fe554a0","trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"62956523-8159-4936-9c7a-f514f7c198fe","_cell_guid":"0f1de92a-d04a-4b89-8a89-5e0b6986e7a7","trusted":true},"cell_type":"code","source":"#env = riiideducation.make_env()\n#iter_test = env.iter_test()\n#for (test_df, sample_prediction_df) in iter_test:\n    # merge\n#    print(test_df)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"17041477-cc3a-466c-892b-fa2b2414f523","_cell_guid":"02d01b5e-e2f2-451e-98dd-79cc55a3532f","trusted":true},"cell_type":"code","source":"#test_df.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0b8baaa3-cac4-4a8b-b052-825c765b712f","_cell_guid":"03a3602b-e08d-4d9a-b334-953818ea1c82","trusted":true},"cell_type":"code","source":"#test_df.head(200)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0db7ac22-c5dd-4343-9594-7af3eb77a844","_cell_guid":"e22b83e3-a759-496b-9502-71fda1a0117b","trusted":true},"cell_type":"code","source":"train","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e8723f27-6a72-45a1-babd-4885a694a8af","_cell_guid":"94f66b1e-21ce-41ac-b8b0-b45c2c3311ba","trusted":true},"cell_type":"markdown","source":"## Стоит обратить внимание на колонки prior_group_answers_correct\tprior_group_responses, стоит  почитаь о них в discussion"},{"metadata":{"_uuid":"405c62c6-c69c-4eef-aadb-67e26cf0eb30","_cell_guid":"53553e15-bc92-4f73-b755-947891f76cfa","trusted":true},"cell_type":"markdown","source":"## FEATURE ENGINEERING and DATA CLEANING"},{"metadata":{"_uuid":"2d3c46db-af31-4768-a502-d5cacac61df8","_cell_guid":"7d9b0122-0375-416c-bf31-dd9250e0cdff","trusted":true},"cell_type":"markdown","source":"## Пока что работаем без валидационной выборки, потом добавил, если понадобится"},{"metadata":{"_uuid":"de5f814d-c65a-4315-8e4b-f8933637796b","_cell_guid":"3d1f8135-414c-46e9-baca-1ca00869c181","trusted":true},"cell_type":"code","source":"# разделяем на обучающий и валидационный наборы","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8c3200c9-ea85-4968-a7f4-30e93a7434fa","_cell_guid":"a22b4290-3f88-460e-8eb0-c140383f7ef6","trusted":true},"cell_type":"code","source":"train_questions_only_df = train[train['answered_correctly']!=-1]\ngrouped_by_user_df = train_questions_only_df.groupby('user_id')\n\n\nuser_ans_df = grouped_by_user_df.agg(\n    {\n        'answered_correctly': [\n            'mean', \n            'count']}\n).copy()\n\nuser_ans_df.columns = [\n    'mean_user_accuracy',\n    'questions_answered'\n]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"96e683d1-2cf9-4fbe-94f6-b51dc7e4985c","_cell_guid":"bf89936c-0f53-46f3-88f3-ec88e2518e75","trusted":true},"cell_type":"code","source":"user_ans_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5e08de85-3dce-4b7f-86e7-2444ccb2db78","_cell_guid":"ad4953c3-7416-4f6b-b57f-9a608244f071","trusted":true},"cell_type":"code","source":"## пока что просто среднее время на ответ без разграничения на правильность\nuser_ans_time_df = grouped_by_user_df.agg(\n    {\n        'prior_question_elapsed_time': [\n            'mean']}\n).copy()\n\nuser_ans_time_df.columns = [\n    'mean_user_ans_time'\n]\n\nuser_ans_time_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"f","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3955fad3-775c-4521-a426-69914b37797b","_cell_guid":"99eb4576-f9ea-4d8f-9f15-ab260aaa2eb5","trusted":true},"cell_type":"code","source":"grouped_by_content_df = train_questions_only_df.groupby('content_id')\ncontent_answers_df = grouped_by_content_df.agg(\n    {\n        'answered_correctly': [\n            'mean', \n            'count'\n        ]\n    }\n).copy()\n\ncontent_answers_df.columns = [\n    'mean_que_accuracy', \n    'question_was_asked'\n]\n\ncontent_answers_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"27ddaabc-3b12-4435-86fb-bb6bcbc23834","_cell_guid":"386ca919-96b1-4bec-905e-af997a5107a5","trusted":true},"cell_type":"code","source":"f = []\nfor i in range(len(train)-1):\n    f.append(train['prior_question_elapsed_time'].values[i+1])\nf.append(0)\ntrain['prior_question_elapsed_time']= f\ntrain","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7cb6e799-447b-437b-92b9-4f962d2dc621","_cell_guid":"435af179-651b-4fd4-a17b-8fec1838e1b8","trusted":true},"cell_type":"code","source":"train_questions_only_well = train[train['answered_correctly']!=0]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"212d5cd9-055a-4021-9473-02d00e8e04c1","_cell_guid":"485e375d-3790-4068-89ba-19d1ee34a89c","trusted":true},"cell_type":"code","source":"grouped_by_questions_well = train_questions_only_well.groupby('content_id')\nquest_well_df = grouped_by_questions_well.agg(\n    {\n    'prior_question_elapsed_time':['mean']\n    }).copy()\nquest_well_df.columns = [\n\n    'mean_prior_question_elapsed_time']\n\nquest_well_df[\"mean_prior_question_elapsed_time\"] = quest_well_df[\"mean_prior_question_elapsed_time\"].astype(int)\n\n\nquest_well_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5a69513d-b65a-4670-a3b3-bb763288719a","_cell_guid":"4b20b132-69cf-4863-9cbf-71a597d3dd7b","trusted":true},"cell_type":"code","source":"grouped_by_user_well = train_questions_only_well.groupby('user_id')\nuser_well_df = grouped_by_user_well.agg(\n    {\n    'prior_question_elapsed_time':['mean']\n    }).copy()\nuser_well_df.columns = [\n\n    'mean_prior_question_elapsed_time_user']\nuser_well_df.fillna(max)\n#user_well_df[\"mean_prior_question_elapsed_time_user\"] = user_well_df[\"mean_prior_question_elapsed_time_user\"].astype(int)\n\n\nuser_well_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b5b2d0d8-5ae3-433c-b1c7-0104b263d8ac","_cell_guid":"d9d822c4-df46-4886-98c4-cf6cc4deb760","trusted":true},"cell_type":"markdown","source":"## Получили фичи 1-2, 5-8. Далее можно улучшить 5-8 и заняться кластеризацией для фичей 4 и 9+"},{"metadata":{"_uuid":"bf2156ac-5809-40d7-a745-6365168fd978","_cell_guid":"cc1224b8-6e78-4190-bab8-744468d4d547","trusted":true},"cell_type":"code","source":"#questions\n#lectures\nquestions","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"702fca63-50a8-40c1-a13e-3f9433fa8b6c","_cell_guid":"f2c45bde-28cd-47a2-bfe5-f0134bc1514c","trusted":true},"cell_type":"code","source":"import re\nimport sys","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fdbaab8c-756c-4ff1-be85-3fc56da242a7","_cell_guid":"0e290385-dd24-42ee-b540-90df9e7e9f33","trusted":true},"cell_type":"code","source":"s = set()\n#dirt_list = questions[\"tags\"]\ndirt_list = questions[\"tags\"].str.split(\" \", expand = True) \nfor merge in dirt_list:\n    ns = set(dirt_list[merge])\n    s.update(ns)\n   \n\ns","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"92cc9011-ebeb-482c-860a-89b8a76b9ff3","_cell_guid":"8ce4c684-8549-4c8a-8900-df57d7abf21b","trusted":true},"cell_type":"code","source":"s = {i for i in s if pd.notna(i)}\ns","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"eb7320ce-fb85-49d2-8e48-359e4bf50136","_cell_guid":"d64477c2-6298-481e-8820-8eee8c52e709","trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"53ce9469-0e8a-4a8e-a09d-7cb62da4e895","_cell_guid":"7baac036-e906-4e2a-8716-e0f197f4a687","trusted":true},"cell_type":"code","source":"tag_df = pd.DataFrame(index=questions.value_counts(), columns=s)\ntag_df = tag_df.fillna(0)\n\ntag_df.index = [i for i in range(13522)]\ntag_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c6c566c1-0b76-47a7-a959-f66dfbf1bbb6","_cell_guid":"b38876ad-7853-4b5c-8327-89c19d085d12","trusted":true},"cell_type":"code","source":"buf = []\ntagg_df = pd.DataFrame(index=questions.value_counts())\ntagg_df.index = [i for i in range(13522)]\n\nf = questions['tags'].dropna()\nfor i in s:\n    buf = []\n    for j in range(len(f.values)):\n        d = f.values[j].split(' ')\n        #print(d)\n        if i in d:\n            buf.append(1)\n        else:\n            buf.append(0)\n    df = pd.DataFrame(buf, columns = [i])\n    #print(df)\n    tagg_df = tagg_df.join(df)\n#tagg_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c51e9064-7eff-43f9-aeb6-15f962f0e600","_cell_guid":"489078f4-c0d2-4105-bdae-9112de669c45","trusted":true},"cell_type":"code","source":"tagg_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"02411c7a-84f3-4366-8a95-86b39df08128","_cell_guid":"39848974-e160-445a-ab29-a92448accb5b","trusted":true},"cell_type":"markdown","source":"## Фича 4 (связь вопросов с лекциями)"},{"metadata":{"_uuid":"efedc090-0a4f-40aa-b258-3dc7ea56f459","_cell_guid":"86c834c8-2fbf-4628-b92d-1cc5af3717f8","trusted":true},"cell_type":"code","source":"grouped_by_container_df = train.groupby('task_container_id')\ncontainer_df = grouped_by_container_df.agg(\n    {'answered_correctly': [\n            'mean', \n            'count'],\n    'content_type_id':['sum']\n    }).copy()\ncontainer_df.columns = [\n    'mean_container_acc', \n    'container_unit_count',\n    'lecs_in_container']\ncontainer_df[\"lecs_in_container\"] = container_df[\"lecs_in_container\"].astype(int)\ncontainer_df[\"lecs_to_qs\"] = container_df[\"lecs_in_container\"]/container_df[\"container_unit_count\"]\n\ncontainer_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"abda4112-ec11-4477-a7c3-ef790a917590","_cell_guid":"93437519-350d-4283-b874-e70dd8239f3a","trusted":true},"cell_type":"code","source":"target = 'answered_correctly'\ntrain = train[train[target] != -1] #delet all lections\n#merge all features in one df\ntrain = train.merge(user_ans_df, how='left', on='user_id')\ntrain = train.merge(content_answers_df, how='left', on='content_id')\ntrain = train.merge(container_df, how='left', on='task_container_id')\n#train = train.merge(user_ans_time_df, how='left', on='user_id')\n#train = train.merge(que_ans_time_df, how='left', on='content_id')\ntrain = train.merge(quest_well_df, how='left', on='content_id')\ntrain = train.merge(user_well_df, how='left', on='user_id')\n\ntrain","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8dd46dab-c5aa-4179-9fe7-b5e0db3c1373","_cell_guid":"2175b979-3cba-4c13-962d-8a797f97eb41","trusted":true},"cell_type":"markdown","source":"## Получили набор данных, теперь нужно его почистить от пропусков, выбросов, удалить лишнии колонки"},{"metadata":{"_uuid":"7cfa4f39-eee3-406f-a5ce-1f1bcdd5eb34","_cell_guid":"09694dcd-5a28-43f8-a0eb-fcf8a0059881","trusted":true},"cell_type":"code","source":"train.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9c850a8f-6282-4940-be27-83f399921b99","_cell_guid":"4df31b82-9ebf-44c0-a0e8-ffa85cdeffa5","trusted":true},"cell_type":"code","source":"train['prior_question_had_explanation'] = train['prior_question_had_explanation'].fillna(value=False).astype(bool)\ntrain = train.fillna(train.mean())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"57da955b-324c-4e93-b377-d4fd1cacc522","_cell_guid":"18643d16-1e0d-4810-991c-c5d07c5fa807","trusted":true},"cell_type":"code","source":"col_to_drop = ['timestamp','row_id','user_id','content_id','content_type_id','task_container_id', 'user_answer']\ntrain=train.drop(columns = col_to_drop)\n\ntrain","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"56c89e5c-6d4d-4628-9025-94e38456df91","_cell_guid":"78669b38-ddbf-419a-8075-3ba023e214f1","trusted":true},"cell_type":"code","source":"features = ['prior_question_elapsed_time',\n       'prior_question_had_explanation', 'mean_user_accuracy',\n       'questions_answered', 'mean_que_accuracy', 'question_was_asked',\n       'mean_container_acc', 'container_unit_count', 'lecs_in_container',\n       'lecs_to_qs', 'mean_prior_question_elapsed_time',\n       'mean_prior_question_elapsed_time_user']\n\nfeatures","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a815aa9d-b9ac-4b5e-abee-d6838c361ee4","_cell_guid":"41a4b342-2448-4a20-aa9a-26101e66295e","trusted":true},"cell_type":"code","source":"train_df, test_df, y_train, y_test = train_test_split(train[features], train[target], random_state=300, test_size=0.2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9e917cc2-a650-4754-b31d-e3459e3032fb","_cell_guid":"17019123-6817-4600-b2d8-3d14205c9cff","trusted":true},"cell_type":"code","source":"from sklearn.ensemble import RandomForestClassifier\nmodel = RandomForestClassifier(n_estimators = 300, max_depth=5, criterion = \"entropy\",n_jobs = -1)\nmodel.fit(train_df, y_train)\nprint('Forest ROC-AUC score: ', roc_auc_score(y_test.values, model.predict_proba(test_df)[:, 1]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bad68dc9-b77a-4e6a-99ee-d97a702cac5b","_cell_guid":"3f06cbd6-8979-4b31-bcb7-295371d8d575","trusted":true},"cell_type":"code","source":"## SUBMISSION\nenv = riiideducation.make_env()\niter_test = env.iter_test()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"aae1f7d8-ed62-4627-a67f-88aba044e157","_cell_guid":"f6f46f7e-66ae-4e80-8af1-a770a48455b9","trusted":true},"cell_type":"code","source":"for (test_df, sample_prediction_df) in iter_test:\n    # merge\n    \n    test_df = test_df.merge(user_ans_df, how='left', on='user_id')\n    test_df = test_df.merge(content_answers_df, how='left', on='content_id')\n    test_df = test_df.merge(container_df, how='left', on='task_container_id')\n    #test_df = test_df.merge(user_ans_time_df, how='left', on='user_id')\n    #test_df = test_df.merge(que_ans_time_df, how='left', on='content_id')\n    train = train.merge(quest_well_df, how='left', on='content_id')\n    train = train.merge(user_well_df, how='left', on='user_id')\n    \n    # type transformation\n    test_df['prior_question_had_explanation'] = test_df['prior_question_had_explanation'].fillna(value=False).astype(bool)\n    test_df.fillna(df.mean(), inplace = True)\n    \n    # preds\n    test_df['answered_correctly'] = model.predict_proba(test_df[features])[:, 1]\n    cols_to_submission = ['row_id', 'answered_correctly', 'group_num']\n    env.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}