{"cells":[{"metadata":{"id":"H-lRxveP6_Pc"},"cell_type":"markdown","source":"# Riiid! Answer Correctness Prediction\n## Track knowledge states of 1M+ students in the wild\n\nIn this competition, your challenge is to create algorithms for \"Knowledge Tracing,\" the modeling of student knowledge over time. The goal is to accurately predict how students will perform on future interactions. You will pair your machine learning skills using Riiid’s EdNet data.\n\n**Note: Sections of the code are intended to be run seperately as such there are multiple load data cell at the appropriate breakpoints **"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"top\"></a>\n# Table of contents\n\n*  [Load Libraries and Data](#1)\n*  [EDA using questions - train.csv](#2)\n*  [EDA using questions - questions.csv](#3)\n*  [EDA using questions - lectures.csv](#4)\n*  [Model Building - Data Split, Preparation and Hyperparameter tuning](#5)\n*  [Model Building -Training with full data](#6)"},{"metadata":{"id":"6xSBusOlhSev"},"cell_type":"markdown","source":"<a id=\"1\"></a>\n# Load Libraries and Data"},{"metadata":{"id":"FKBE_TzNhcUk","trusted":true},"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression\nimport numpy as np\nimport pandas as pd\nfrom sklearn import preprocessing\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import roc_auc_score\nfrom sklearn.metrics import accuracy_score\nimport matplotlib.pyplot as plt\n!pip install seaborn --upgrade\nimport seaborn as sns\nimport random\nimport math\nfrom sklearn.model_selection import train_test_split\n\nSEED = 299458792","execution_count":null,"outputs":[]},{"metadata":{"id":"Wg4rAk9nvH-x"},"cell_type":"markdown","source":"## train.csv"},{"metadata":{"id":"-tCQ_pVMvRWt"},"cell_type":"markdown","source":"row_id: (int64) ID code for the row.\n\ntimestamp: (int64) the time in milliseconds between this user interaction and the first event completion from that user.\n\nuser_id: (int32) ID code for the user.\n\ncontent_id: (int16) ID code for the user interaction\n\ncontent_type_id: (int8) 0 if the event was a question being posed to the user, 1 if the event was the user watching a lecture.\n\ntask_container_id: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a task_container_id.\n\nuser_answer: (int8) the user's answer to the question, if any. Read -1 as null, for lectures.\n\nanswered_correctly: (int8) if the user responded correctly. Read -1 as null, for lectures.\n\nprior_question_elapsed_time: (float32) The average time in milliseconds it took a user to answer each question in the previous question bundle, ignoring any lectures in between. Is null for a user's first question bundle or lecture. Note that the time is the average time a user took to solve each question in the previous bundle.\n\nprior_question_had_explanation: (bool) Whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle, ignoring any lectures in between. The value is shared across a single question bundle, and is null for a user's first question bundle or lecture. Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback"},{"metadata":{"id":"_jMfwHEchrUO","trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('../input/riiid-test-answer-prediction/train.csv', low_memory=False, nrows=10**5, \n                       dtype={'row_id': 'int64', 'timestamp': 'int64', 'user_id': 'int32', 'content_id': 'int16', 'content_type_id': 'int8',\n                              'task_container_id': 'int16', 'user_answer': 'int8', 'answered_correctly': 'int8', 'prior_question_elapsed_time': 'float32', \n                             'prior_question_had_explanation': 'boolean',\n                             }\n                      )\n\n#train_df = dt.fread(\"../input/riiid-test-answer-prediction/train.csv\").to_pandas()","execution_count":null,"outputs":[]},{"metadata":{"id":"FXK8PF1xvMWc","outputId":"aefe035c-26dc-4752-a262-611430b4401b","trusted":true},"cell_type":"code","source":"print(\"Train size:\", train_df.shape)","execution_count":null,"outputs":[]},{"metadata":{"id":"nQ2x-C41v-fl","outputId":"a559825c-0b80-406d-e3d7-ee97e506382c","trusted":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"eVoJFU1hvUOb"},"cell_type":"markdown","source":"## question.csv"},{"metadata":{"id":"oLxZvk7PvXnk"},"cell_type":"markdown","source":"questions.csv: metadata for the questions posed to users.\n\nquestion_id: foreign key for the train/test content_id column, when the content type is question (0).\n\nbundle_id: code for which questions are served together.\n\ncorrect_answer: the answer to the question. Can be compared with the train user_answer column to check if the user was right.\n\npart: top level category code for the question.\n\ntags: one or more detailed tag codes for the question. The meaning of the tags will not be provided, but these codes are sufficient for clustering the questions together."},{"metadata":{"id":"RKBXHFsbvN9J","trusted":true},"cell_type":"code","source":"questions = pd.read_csv('../input/riiid-test-answer-prediction/questions.csv')\nquestions['tags'] = questions['tags'].astype('category') \nquestions[['question_id', 'bundle_id','correct_answer','part']] = questions[['question_id', 'bundle_id','correct_answer','part']].apply(pd.to_numeric, downcast='unsigned')","execution_count":null,"outputs":[]},{"metadata":{"id":"6gm9n3devcbT","outputId":"37d6bff8-6bc8-481f-8211-bed1e5d21595","trusted":true},"cell_type":"code","source":"print(\"Questions:\", questions.shape)","execution_count":null,"outputs":[]},{"metadata":{"id":"W0ngNofmwAXs","outputId":"ce0e3308-6bb9-4a79-8cfe-e4fcb521394b","trusted":true},"cell_type":"code","source":"questions.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"cohi-sBFveX-"},"cell_type":"markdown","source":"## lectures.csv"},{"metadata":{"id":"xifMQhduvhOc"},"cell_type":"markdown","source":"lectures.csv: metadata for the lectures watched by users as they progress in their education.\n\nlecture_id: foreign key for the train/test content_id column, when the content type is lecture (1).\n\npart: top level category code for the lecture.\n\ntag: one tag codes for the lecture. The meaning of the tags will not be provided, but these codes are sufficient for clustering the lectures together.\n\ntype_of: brief description of the core purpose of the lecture"},{"metadata":{"id":"l6z10ryvvdfj","trusted":true},"cell_type":"code","source":"lectures = pd.read_csv('../input/riiid-test-answer-prediction/lectures.csv')\nlectures['type_of'] = lectures['type_of'].astype('category') \nlectures[['lecture_id', 'tag','part']] = lectures[['lecture_id', 'tag','part']].apply(pd.to_numeric, downcast='unsigned')","execution_count":null,"outputs":[]},{"metadata":{"id":"ZxLAuIAkvlJZ","outputId":"abc83533-4c61-4e57-b80e-61a8d677bd02","trusted":true},"cell_type":"code","source":"print(\"Lectures:\", lectures.shape)","execution_count":null,"outputs":[]},{"metadata":{"id":"bVj_rUErwCbH","outputId":"bc704769-f6f8-45c6-b9d0-93a741ec9103","trusted":true},"cell_type":"code","source":"lectures.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"-ZPGea9avn5N"},"cell_type":"markdown","source":"<a id=\"2\"></a>\n# EDA using questions - train.csv"},{"metadata":{"id":"MDftiiLyvmlU","outputId":"f26cd088-1a25-4b53-da02-cc693cec3fa5","trusted":true},"cell_type":"code","source":"train_df.describe()","execution_count":null,"outputs":[]},{"metadata":{"id":"b_kB7MgBvwHL","outputId":"7fdf1397-d014-4548-875a-eb127244c0e8","trusted":true},"cell_type":"code","source":"train_df.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"id":"WKVU-N-yvxD_","outputId":"a2a4282b-eb81-4485-aa2e-4a9b0d20a658","trusted":true},"cell_type":"code","source":"print(train_df.dtypes)","execution_count":null,"outputs":[]},{"metadata":{"id":"CZVBdVlOwv9x","outputId":"7d4261a9-0681-4939-8bdd-b9a71bddf1a9","trusted":true},"cell_type":"code","source":"for col in train_df.columns:\n  print('Number of Unique variables in ' + col+':',len(train_df[col].unique()))","execution_count":null,"outputs":[]},{"metadata":{"id":"Q2YzIigWICMu"},"cell_type":"markdown","source":"## Lets ask some simple questions for students who have answered atleast a 100 questions\n1. Who are the top 5 students?\n2. Who are the bottom 5 worst student?\n3. Who are the top 5 fastest students to answer question?\n4. Who are the slowest 5 students to answer questions? What metric was used? Does the distribution have an effect on the answer?\n5. What is the performance distirbution of the class based on answer correctness?\n6.What is the performance distirbution of the class based on speed correctness?\n"},{"metadata":{"id":"WE2R0UBDEO-c","outputId":"6e8a108b-2f17-491b-d8dd-067722afdf17","trusted":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"V_ufT0y4LWTN","trusted":true},"cell_type":"code","source":"train_user_grouped=train_df.groupby(['user_id']).apply(lambda x: pd.Series({\n      'total_answered_correctly'       : x['answered_correctly'].sum(),\n      'total_answered'       : x['answered_correctly'].count(),\n      'fraction_answered_correctly'      : x['answered_correctly'].sum()/x['answered_correctly'].count(),\n      'total_lectures' : x['content_type_id'].sum(),\n      'median_time_to_answer' : x['prior_question_elapsed_time'].median(),\n      'std_time_to_answer' : x['prior_question_elapsed_time'].std()}))","execution_count":null,"outputs":[]},{"metadata":{"id":"N60XG-P6Q0xV","trusted":true},"cell_type":"code","source":"train_user_grouped=train_user_grouped.loc[train_user_grouped.loc[:,'total_answered']>=100,:]","execution_count":null,"outputs":[]},{"metadata":{"id":"b6-iXHOiPXrt"},"cell_type":"markdown","source":"### 1. Who are the top 5 students?"},{"metadata":{"id":"iGYpeTNBRKrI","outputId":"fabe2e37-fb80-4829-ca85-ed70b5168e86","trusted":true},"cell_type":"code","source":"train_user_grouped.sort_values(by=['fraction_answered_correctly'],ascending=False).head(5)","execution_count":null,"outputs":[]},{"metadata":{"id":"53bV45lTTkBK"},"cell_type":"markdown","source":"### 2. Who are the bottom 5 worst student?"},{"metadata":{"id":"LBQrWiuRTyfq","outputId":"c04593bc-7c57-4c1b-e0a9-b218925edba0","trusted":true},"cell_type":"code","source":"train_user_grouped.sort_values(by=['fraction_answered_correctly'],ascending=True).head(5)","execution_count":null,"outputs":[]},{"metadata":{"id":"bpqk5XsITr2i"},"cell_type":"markdown","source":"### 3. Who are the top 5 fastest students to answer question?"},{"metadata":{"id":"9U2s-sstT1rh","outputId":"da79203d-3c29-4c66-c518-9470431883fa","trusted":true},"cell_type":"code","source":"train_user_grouped.sort_values(by=['median_time_to_answer'],ascending=True).head(5)","execution_count":null,"outputs":[]},{"metadata":{"id":"9Wimg6G0TuqT"},"cell_type":"markdown","source":"### 4. Who are the slowest 5 students to answer questions? What metric was used? Does the distribution have an effect on the answer?"},{"metadata":{"id":"FEgsjPGTT77-","outputId":"a1e9b0bb-93dc-486b-fd0f-ede595db4642","trusted":true},"cell_type":"code","source":"train_user_grouped.sort_values(by=['median_time_to_answer'],ascending=False).head(5)","execution_count":null,"outputs":[]},{"metadata":{"id":"mJkKc2V7mvGm","outputId":"6af02767-f43e-4e63-e59d-2d48aad2c4b0","trusted":true},"cell_type":"code","source":"def boxplot_sorted(df, by, column, ax, rot=90):\n    # use dict comprehension to create new dataframe from the iterable groupby object\n    # each group name becomes a column in the new dataframe\n    df2 = pd.DataFrame({col:vals[column] for col, vals in df.groupby(by)})\n    # find and sort the median values in this new dataframe\n    meds = df2.median().sort_values()\n    # use the columns in the dataframe, ordered sorted by median value\n    # return axes so changes can be made outside the function\n    return df2[meds.index].boxplot(rot=rot,ax=ax,return_type=\"axes\")\n\nfig, ax = plt.subplots(figsize=(50,8))\nboxplot_sorted(train_df, by = ['user_id'], column = 'prior_question_elapsed_time', ax=ax)\nplt.xlabel('user_id')\nplt.ylabel('prior_question_elapsed_time')\nplt.title('Boxplots')\nplt.suptitle('')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"Y8EVL17rUDLk"},"cell_type":"markdown","source":"Yes, the distribution does have an signifhcant effect on the answer because the standard deviation is comparable to the median across all the students."},{"metadata":{"id":"DwP2FDibTwzw"},"cell_type":"markdown","source":"### 5. What is the performance distirbution of the class based on answer correctness?"},{"metadata":{"id":"8idUa939Uw1S","outputId":"28b54a80-f2ed-4801-f2f5-c751b5831f33","trusted":true},"cell_type":"code","source":"sns.histplot(train_user_grouped['fraction_answered_correctly'])","execution_count":null,"outputs":[]},{"metadata":{"id":"J-dFwao6UNnz"},"cell_type":"markdown","source":"### 6. What is the performance distirbution of the class based on speed correctness?"},{"metadata":{"id":"oIXmDCLXVU8j","outputId":"29cb32d0-3884-474e-aebe-b232d8d3d759","trusted":true},"cell_type":"code","source":"sns.histplot(train_user_grouped['median_time_to_answer'])","execution_count":null,"outputs":[]},{"metadata":{"id":"HfGv2DTNvzXt"},"cell_type":"markdown","source":"<a id=\"3\"></a>\n# EDA using questions - questions.csv"},{"metadata":{"id":"d8mAXBjMvyHU","outputId":"b284d37d-7c8e-46a5-9633-77ee9a37117b","trusted":true},"cell_type":"code","source":"questions.describe()","execution_count":null,"outputs":[]},{"metadata":{"id":"Jp2astt319oL","outputId":"7fb06d6e-3aaa-4eb3-9360-4913e3a71732","trusted":true},"cell_type":"code","source":"questions.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"id":"M697rFmX2Chy","outputId":"c2d1806e-f66b-4f77-9406-272c5efa4f79","trusted":true},"cell_type":"code","source":"print(questions.dtypes)","execution_count":null,"outputs":[]},{"metadata":{"id":"oSEAMe_O2Knc","outputId":"7cdf0c36-d0b1-47e6-fca7-e8c6c8384eca","trusted":true},"cell_type":"code","source":"for col in questions.columns:\n  print('Number of Unique variables in ' + col+':',len(questions[col].unique()))","execution_count":null,"outputs":[]},{"metadata":{"id":"Z-17xZnZv4VF","outputId":"d32081b8-3189-4770-fe12-cec3a0872e72","trusted":true},"cell_type":"code","source":"questions.tail(100)","execution_count":null,"outputs":[]},{"metadata":{"id":"ULXO-B2XlGA9","outputId":"b89d4e91-cee7-4e40-f86f-516e1680fbf8","trusted":true},"cell_type":"code","source":"sns.pairplot(questions.iloc[:,:-2])","execution_count":null,"outputs":[]},{"metadata":{"id":"r5GRt4ti2He2"},"cell_type":"markdown","source":"## Lets ask some simple questions about questions.csv dataset\n\n1. What is the distribution of questions per bundle? \n2. What is the distribution of questions per part? Which part has the most number of questions? Which part has the least number of questions?\n3. What is the distribution of questions per tags? Which tag has the most number of questions? \n"},{"metadata":{"id":"VQp9F68QnTb9"},"cell_type":"markdown","source":"### 1.  What is the distribution of questions per bundle?"},{"metadata":{"id":"usCknfN1kh6U","trusted":true},"cell_type":"code","source":"questions_bundle_grouped=questions.groupby(['bundle_id']).apply(lambda x: pd.Series({\n      'total_questions'       : x['question_id'].count()}))","execution_count":null,"outputs":[]},{"metadata":{"id":"asAuFEyTnlVg","outputId":"f547ebba-c246-4435-de8a-8768c2e3cedd","trusted":true},"cell_type":"code","source":"sns.histplot(questions_bundle_grouped['total_questions'])","execution_count":null,"outputs":[]},{"metadata":{"id":"-qBgtaKboahb"},"cell_type":"markdown","source":"### 2. What is the distribution of questions per part? Which part has the most number of questions? Which part has the least number of questions?"},{"metadata":{"id":"NUNbLovxodY9","trusted":true},"cell_type":"code","source":"questions_part_grouped=questions.groupby(['part']).apply(lambda x: pd.Series({\n      'total_questions'       : x['question_id'].count()}))","execution_count":null,"outputs":[]},{"metadata":{"id":"r-28n0j2oeTH","outputId":"a8560f8a-ab26-44ec-8fd6-ff1cf4cc3c79","trusted":true},"cell_type":"code","source":"questions_part_grouped","execution_count":null,"outputs":[]},{"metadata":{"id":"ZLaZgZBcpHf2"},"cell_type":"markdown","source":"### 3. What is the distribution of questions per tags? "},{"metadata":{"id":"8d6UWR02pPzU","trusted":true},"cell_type":"code","source":"questions_tags_grouped=questions.groupby(['tags']).apply(lambda x: pd.Series({\n      'total_questions'       : x['question_id'].count()}))","execution_count":null,"outputs":[]},{"metadata":{"id":"GbeGnOJcpQoV","outputId":"0843e994-b6e9-4c67-e0fc-1048359e3dfe","trusted":true},"cell_type":"code","source":"sns.histplot(questions_tags_grouped['total_questions'])","execution_count":null,"outputs":[]},{"metadata":{"id":"T2dlNyQ-qefe","outputId":"643831ca-9dba-4d51-a5fd-1099f62f8b87","trusted":true},"cell_type":"code","source":"questions_tags_grouped.sort_values(by=['total_questions'],ascending=False).head(15)","execution_count":null,"outputs":[]},{"metadata":{"id":"dhrGJNqJ2UKl"},"cell_type":"markdown","source":"<a id=\"4\"></a>\n# EDA using questions - lectures.csv"},{"metadata":{"id":"CqdWmbaU2UKl","outputId":"38519719-0561-4be9-9a81-4aa704224b7b","trusted":true},"cell_type":"code","source":"lectures.describe()","execution_count":null,"outputs":[]},{"metadata":{"id":"73YLSjdh2UKp","outputId":"b8672772-9985-427c-f644-503f475a8b70","trusted":true},"cell_type":"code","source":"lectures.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"id":"tefQizaw2UKq","outputId":"3d650df7-895f-418f-91e1-ffe519bcf455","trusted":true},"cell_type":"code","source":"print(lectures.dtypes)","execution_count":null,"outputs":[]},{"metadata":{"id":"E9iiFcxz2UKs","outputId":"7df5abaf-56a1-47c4-ba6a-ac821d762f71","trusted":true},"cell_type":"code","source":"for col in lectures.columns:\n  print('Number of Unique variables in ' + col+':',len(lectures[col].unique()))","execution_count":null,"outputs":[]},{"metadata":{"id":"tBZJ7Mtlq-Sh"},"cell_type":"markdown","source":"## Lets ask some simple questions about lectures.csv dataset\n\n1. What is the distribution of lectures per tag? \n2. What is the distribution of lectures per part? Which part has the most number of questions? Which part has the least number of questions?\n3. What is the distribution of questions per type_of? Which type has the most number of lectures? \n"},{"metadata":{"id":"TK66XTKTq-Si"},"cell_type":"markdown","source":"### 1. What is the distribution of lectures per tag? \n"},{"metadata":{"id":"8U9ZZE6Yq-Si","trusted":true},"cell_type":"code","source":"lectures_tag_grouped=lectures.groupby(['tag']).apply(lambda x: pd.Series({\n      'total_lectures'       : x['lecture_id'].count()}))","execution_count":null,"outputs":[]},{"metadata":{"id":"_RuLAoWaq-Sk","outputId":"8c3a8073-c2bd-4440-aceb-d627cc1ac7be","trusted":true},"cell_type":"code","source":"sns.histplot(lectures_tag_grouped['total_lectures'])","execution_count":null,"outputs":[]},{"metadata":{"id":"rdtJcOADsVWF","outputId":"c30b3a7a-3ac0-4e5b-e546-4ac7231ae676","trusted":true},"cell_type":"code","source":"lectures_tag_grouped.sort_values(by=['total_lectures'],ascending=False).head(15)","execution_count":null,"outputs":[]},{"metadata":{"id":"p1r-fwcpq-Sn"},"cell_type":"markdown","source":"### 2. What is the distribution of lectures per part? Which part has the most number of questions? Which part has the least number of questions?"},{"metadata":{"id":"NGVseXsVq-Sn","trusted":true},"cell_type":"code","source":"lectures_part_grouped=lectures.groupby(['part']).apply(lambda x: pd.Series({\n      'total_lectures'       : x['lecture_id'].count()}))","execution_count":null,"outputs":[]},{"metadata":{"id":"Zlt2ETg_q-Sp","outputId":"95412c73-ee7a-4efe-ba74-7de8fe93d76c","trusted":true},"cell_type":"code","source":"lectures_part_grouped","execution_count":null,"outputs":[]},{"metadata":{"id":"2-P_l7vaq-Sq"},"cell_type":"markdown","source":"### 3. What is the distribution of questions per type_of? Which type has the most number of lectures? \n"},{"metadata":{"id":"laLteZSqq-Sr","trusted":true},"cell_type":"code","source":"lectures_type_grouped=lectures.groupby(['type_of']).apply(lambda x: pd.Series({\n      'total_lectures'       : x['lecture_id'].count()}))","execution_count":null,"outputs":[]},{"metadata":{"id":"-TUtIQRwq-Su","outputId":"e859b304-1319-4ad9-8aa5-73205cf9d5cb","trusted":true},"cell_type":"code","source":"sns.histplot(lectures_type_grouped['total_lectures'])","execution_count":null,"outputs":[]},{"metadata":{"id":"UF1hIlLzq-Sw","outputId":"e47c37b3-027e-4f0d-ed22-ff3431543abb","trusted":true},"cell_type":"code","source":"lectures_type_grouped.sort_values(by=['total_lectures'],ascending=False).head(15)","execution_count":null,"outputs":[]},{"metadata":{"id":"0W9L47zQLPDI"},"cell_type":"markdown","source":"<a id=\"5\"></a>\n# Model Building - Data Split, Preparation and Hyperparameter tuning"},{"metadata":{"id":"8e4TTVDeMJw9"},"cell_type":"markdown","source":"## Load dataset for Model building"},{"metadata":{"id":"je5IklZNML3G"},"cell_type":"markdown","source":"row_id: (int64) ID code for the row.\n\ntimestamp: (int64) the time in milliseconds between this user interaction and the first event completion from that user.\n\nuser_id: (int32) ID code for the user.\n\ncontent_id: (int16) ID code for the user interaction\n\ncontent_type_id: (int8) 0 if the event was a question being posed to the user, 1 if the event was the user watching a lecture.\n\ntask_container_id: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a task_container_id.\n\nuser_answer: (int8) the user's answer to the question, if any. Read -1 as null, for lectures.\n\nanswered_correctly: (int8) if the user responded correctly. Read -1 as null, for lectures.\n\nprior_question_elapsed_time: (float32) The average time in milliseconds it took a user to answer each question in the previous question bundle, ignoring any lectures in between. Is null for a user's first question bundle or lecture. Note that the time is the average time a user took to solve each question in the previous bundle.\n\nprior_question_had_explanation: (bool) Whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle, ignoring any lectures in between. The value is shared across a single question bundle, and is null for a user's first question bundle or lecture. Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback"},{"metadata":{"id":"zTdBWK_xML3G","trusted":true},"cell_type":"code","source":"#columns for analysis, only select columns needed to prevent out of memory\ncols_2_read = ['timestamp','content_id','user_id','content_type_id','answered_correctly','prior_question_elapsed_time','prior_question_had_explanation']#,'row_id','task_container_id','user_answer','prior_question_had_explanation']\n\n#column type to minimize memory usage\nall_dtype_dic = {'row_id': 'uint64', 'timestamp': 'uint64', 'user_id': 'uint32', 'content_id': 'uint16', 'content_type_id': 'uint8',\n                              'task_container_id': 'uint16', 'user_answer': pd.CategoricalDtype([-1,0,1,2,3]), 'answered_correctly': 'int8', 'prior_question_elapsed_time': 'float32', \n                             'prior_question_had_explanation': 'boolean',\n                             }\n#dictionary to look up column type based on column selected\ncols_2_read_dtype_dic = {k: all_dtype_dic[k] for k in all_dtype_dic.keys() & cols_2_read}\ntrain_df = pd.DataFrame()\n\n#read dataset in chunks to prevent out-of-memory\nfor chunk in pd.read_csv('../input/riiid-test-answer-prediction/train.csv', low_memory=False,nrows=10**7,\n                       dtype=cols_2_read_dtype_dic,usecols = cols_2_read,chunksize=10**5 \n                      ):\n    train_df = pd.concat([train_df, chunk], ignore_index=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above code allows you to add only the columns needed for analysis."},{"metadata":{"id":"k1kOhh3O8lXC","outputId":"14aa5902-1cd5-4602-d48a-188ce6554127","trusted":true},"cell_type":"code","source":"train_df['user_id'].describe()","execution_count":null,"outputs":[]},{"metadata":{"id":"n-jSO6Z-Dt9q","outputId":"592a8d9d-08b9-400b-db64-486aea120241","trusted":true},"cell_type":"code","source":"train_df.describe()","execution_count":null,"outputs":[]},{"metadata":{"id":"ULSQD0Ww7FoP"},"cell_type":"markdown","source":"## Data split"},{"metadata":{"id":"QFGm5hD47FoP"},"cell_type":"markdown","source":"**This time we are going to split data into 3 parts: train (70%), tune (15%), and test (15%). The tune set is used to perform model selection and feature selection (more in latter lessons).**"},{"metadata":{"id":"IjHLPc987FoQ","trusted":true},"cell_type":"code","source":"# data split to have 70% of train, 30% of tune + test\ntrain_df, validation_test_df = train_test_split(train_df,\n                                          test_size=0.3,\n                                          random_state=SEED,\n                                          shuffle=True,\n                                          stratify=None)\n","execution_count":null,"outputs":[]},{"metadata":{"id":"Eo3qJLVe7FoU","trusted":true},"cell_type":"code","source":"# further evenly split between tune and test\nvalidation_df, test_df = train_test_split(validation_test_df,\n                                    test_size=0.5,\n                                    random_state=SEED,\n                                    shuffle=True,\n                                    stratify=None)","execution_count":null,"outputs":[]},{"metadata":{"id":"uMVKld20eWSi","trusted":true},"cell_type":"code","source":"#reset index of created dataframes\ntrain_df=train_df.reset_index(drop=True)\ntest_df=test_df.reset_index(drop=True)\nvalidation_df=validation_df.reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{"id":"T6uHqP6ydp-n","outputId":"26e793ff-5bdd-4dbe-d04b-7e3507e4bd16","trusted":true},"cell_type":"code","source":"#check shape\nprint(train_df.shape)\nprint(validation_df.shape)\nprint(test_df.shape)","execution_count":null,"outputs":[]},{"metadata":{"id":"84OfOcmIwBXt"},"cell_type":"markdown","source":"## Perform a full join of train, questions, and lectures"},{"metadata":{"id":"DPbseg3_P4YF","trusted":true},"cell_type":"code","source":"questions = pd.read_csv('../input/riiid-test-answer-prediction/questions.csv')\nquestions['tags'] = questions['tags'].astype('category') \nquestions[['question_id', 'bundle_id','correct_answer','part']] = questions[['question_id', 'bundle_id','correct_answer','part']].apply(pd.to_numeric, downcast='unsigned')\n\nlectures = pd.read_csv('../input/riiid-test-answer-prediction/lectures.csv')\nlectures['type_of'] = lectures['type_of'].astype('category') \nlectures[['lecture_id', 'tag','part']] = lectures[['lecture_id', 'tag','part']].apply(pd.to_numeric, downcast='unsigned')","execution_count":null,"outputs":[]},{"metadata":{"id":"Zpq0r0vZxbYe","trusted":true},"cell_type":"code","source":"def concat_questions_lectures(train_df,questions,lectures):\n   ##questions\n   train_questions = train_df.loc[train_df.loc[:,'content_type_id']==0,:]\n   train_questions=train_questions.set_index('content_id')\n   questions = questions.rename(columns={\"question_id\": \"content_id\",'tags':'question_tags'})\n   questions = questions.set_index('content_id') \n   train_questions=train_questions.join(questions, how='left')\n   #lectures\n   train_lectures = train_df.loc[train_df.loc[:,'content_type_id']==1,:]\n   train_lectures=train_lectures.set_index('content_id')\n   lectures = lectures.rename(columns={\"lecture_id\": \"content_id\",'tag':'lecture_tag'})\n   lectures = lectures.set_index('content_id') \n   train_lectures=train_lectures.join(lectures, how='left')\n   #lectures and questions\n   train_questions_lectures= pd.concat([train_questions,train_lectures], axis=0)\n   train_questions_lectures.reset_index(inplace=True)\n   train_questions_lectures = train_questions_lectures.rename(columns = {'index':'content_id'})\n   return train_questions_lectures\n\ntrain_df = concat_questions_lectures(train_df,questions,lectures)\nvalidation_df = concat_questions_lectures(validation_df,questions,lectures)\ntest_df = concat_questions_lectures(test_df,questions,lectures)","execution_count":null,"outputs":[]},{"metadata":{"id":"tGHNCrE9wJIG"},"cell_type":"markdown","source":"## Convert all to Numeric"},{"metadata":{"id":"CxNWa0Eje9m-","outputId":"be06d8e0-57a9-4cc9-acd9-e00db68d9814","trusted":true},"cell_type":"code","source":"train_df.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"id":"eNGZF05XBr4A","outputId":"411377bd-df0a-4731-fa93-6646d69fb001","trusted":true},"cell_type":"code","source":"train_df.dtypes","execution_count":null,"outputs":[]},{"metadata":{"id":"zcVyMH3QfbCP","trusted":true},"cell_type":"code","source":"train_df[['type_of','question_tags','lecture_tag']] = train_df[['type_of','question_tags','lecture_tag']].astype('category') \ntrain_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']] = train_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']].apply(pd.to_numeric, downcast='unsigned')\n\nvalidation_df[['type_of','question_tags','lecture_tag']] = validation_df[['type_of','question_tags','lecture_tag']].astype('category') \nvalidation_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']] = validation_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']].apply(pd.to_numeric, downcast='unsigned')\n\ntest_df[['type_of','question_tags','lecture_tag']] = train_df[['type_of','question_tags','lecture_tag']].astype('category') \ntest_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']] = test_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']].apply(pd.to_numeric, downcast='unsigned')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Fill NaN values"},{"metadata":{"id":"tC-SVxYckRqG","trusted":true},"cell_type":"code","source":"##fill prior_question_elapsed_time\ntrain_df['prior_question_elapsed_time'] = train_df['prior_question_elapsed_time'].fillna(0)\nvalidation_df['prior_question_elapsed_time'] = validation_df['prior_question_elapsed_time'].fillna(0)\ntest_df['prior_question_elapsed_time'] = test_df['prior_question_elapsed_time'].fillna(0)\n\n##prior_question_had_explanation\ntrain_df['prior_question_had_explanation'] = train_df['prior_question_had_explanation'].fillna(False)\nvalidation_df['prior_question_had_explanation'] = validation_df['prior_question_had_explanation'].fillna(False)\ntest_df['prior_question_had_explanation'] = test_df['prior_question_had_explanation'].fillna(False)\n\n##fill rows for questions which were lectures\ntrain_df['question_tags']=train_df['question_tags'].cat.add_categories(-1)\ntrain_df[['bundle_id','correct_answer','question_tags']] = train_df[['bundle_id','correct_answer','question_tags']].fillna(-1)\nvalidation_df['question_tags']=validation_df['question_tags'].cat.add_categories(-1)\nvalidation_df[['bundle_id','correct_answer','question_tags']] = validation_df[['bundle_id','correct_answer','question_tags']].fillna(-1)\ntest_df['question_tags']=test_df['question_tags'].cat.add_categories(-1)\ntest_df[['bundle_id','correct_answer','question_tags']] = test_df[['bundle_id','correct_answer','question_tags']].fillna(-1)\n\n##fill rows for lectures which were questions\ntrain_df['lecture_tag']=train_df['lecture_tag'].cat.add_categories(-1)\ntrain_df[['lecture_tag']] = train_df[['lecture_tag']].fillna(-1)\nvalidation_df['lecture_tag']=validation_df['lecture_tag'].cat.add_categories(-1)\nvalidation_df[['lecture_tag']] = validation_df[['lecture_tag']].fillna(-1)\ntest_df['lecture_tag']=test_df['lecture_tag'].cat.add_categories(-1)\ntest_df[['lecture_tag']] = test_df[['lecture_tag']].fillna(-1)\n\n##fill type_of as questions which questions\ntrain_df['type_of']=train_df['lecture_tag'].cat.add_categories('question')\ntrain_df[['type_of']] = train_df[['type_of']].fillna('question')\nvalidation_df['type_of']=validation_df['lecture_tag'].cat.add_categories('question')\nvalidation_df[['type_of']] = validation_df[['type_of']].fillna('question')\ntest_df['type_of']=test_df['lecture_tag'].cat.add_categories('question')\ntest_df[['type_of']] = test_df[['type_of']].fillna('question')","execution_count":null,"outputs":[]},{"metadata":{"id":"GU4_SooDk7mp","outputId":"a88901a0-158b-4e7a-d28f-462d4900fb45","trusted":true},"cell_type":"code","source":"train_df.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"id":"sfM9WNrvEhG-","outputId":"32c53dc4-0aa5-4b02-f533-f83ba3830999","trusted":true},"cell_type":"code","source":"train_df.dtypes","execution_count":null,"outputs":[]},{"metadata":{"id":"siv5uobSNdDM"},"cell_type":"markdown","source":"## Feature Engineering"},{"metadata":{"id":"7VXadnGxNfnV"},"cell_type":"markdown","source":"### Calculate Lag (difference in time between two consecutive interactions for each user)"},{"metadata":{"id":"iCVCeB2fNcXq","trusted":true},"cell_type":"code","source":"train_df['user_interaction_lag'] = train_df.sort_values(['user_id','timestamp']).groupby('user_id')['timestamp'].diff()\nvalidation_df['user_interaction_lag'] = validation_df.sort_values(['user_id','timestamp']).groupby('user_id')['timestamp'].diff()\ntest_df['user_interaction_lag'] = test_df.sort_values(['user_id','timestamp']).groupby('user_id')['timestamp'].diff()","execution_count":null,"outputs":[]},{"metadata":{"id":"AB995ZAd22dv"},"cell_type":"markdown","source":"### User statistics"},{"metadata":{"id":"wI9ZIHZQ2R2l","trusted":true},"cell_type":"code","source":"def user_answer_stats(df):\n  questions_only_df = df[df['answered_correctly']!=-1]\n  questions_only_df = questions_only_df[questions_only_df['user_id'].notna()]\n  grouped_by_user_df = questions_only_df.groupby('user_id')\n  user_answers_df = grouped_by_user_df.agg({'answered_correctly': ['mean', 'count', 'std', 'median', 'skew']}).copy()\n  user_answers_df.columns = ['mean_user_accuracy', 'questions_answered', 'std_user_accuracy', 'median_user_accuracy', 'skew_user_accuracy']\n  user_answers_df.reset_index(inplace=True)\n  user_answers_df = user_answers_df.rename(columns = {'index':'user_id'})\n  return user_answers_df\n\ntrain_df = train_df.merge(user_answer_stats(train_df), how='left', on='user_id')\nvalidation_df = validation_df.merge(user_answer_stats(validation_df), how='left', on='user_id')\ntest_df = test_df.merge(user_answer_stats(test_df), how='left', on='user_id')","execution_count":null,"outputs":[]},{"metadata":{"id":"tKSTekdH27H9"},"cell_type":"markdown","source":"### Content Statistics"},{"metadata":{"id":"N0utP15-mwUj","trusted":true},"cell_type":"code","source":"def content_answer_stats(df):\n  questions_only_df = df[df['answered_correctly']!=-1]\n  questions_only_df = questions_only_df[questions_only_df['content_id'].notna()]\n  grouped_by_content_df = questions_only_df.groupby('content_id')\n  content_answers_df = grouped_by_content_df.agg({'answered_correctly': ['mean', 'count', 'std', 'median', 'skew']}).copy()\n  content_answers_df.columns = ['mean_accuracy', 'questions_asked', 'std_accuracy', 'median_accuracy', 'skew_accuracy']\n  content_answers_df.reset_index(inplace=True)\n  content_answers_df = content_answers_df.rename(columns = {'index':'content_id'})\n  return content_answers_df\n\ntrain_df = train_df.merge(content_answer_stats(train_df), how='left', on='content_id')\nvalidation_df = validation_df.merge(content_answer_stats(validation_df), how='left', on='content_id')\ntest_df = test_df.merge(content_answer_stats(test_df), how='left', on='content_id')","execution_count":null,"outputs":[]},{"metadata":{"id":"FGcqH7zyTW02","outputId":"113325cd-d04f-47b3-ecbf-459da1c1d254","trusted":true},"cell_type":"code","source":"train_df.info()","execution_count":null,"outputs":[]},{"metadata":{"id":"YkG6GOx-XKvD"},"cell_type":"markdown","source":"## Features Selection"},{"metadata":{"id":"CAyS8IDdXSf9","trusted":true},"cell_type":"code","source":"features = [\n    'mean_user_accuracy', \n    'questions_answered',\n    'std_user_accuracy', \n    'median_user_accuracy',\n    'skew_user_accuracy',\n    'mean_accuracy', \n    'questions_asked',\n    'std_accuracy', \n    'median_accuracy',\n    'prior_question_elapsed_time', \n    'prior_question_had_explanation',\n    'skew_accuracy',\n]\n\ntarget = 'answered_correctly'","execution_count":null,"outputs":[]},{"metadata":{"id":"05kp4kAIXxq5"},"cell_type":"markdown","source":"## Final Train, Validation, and Test data"},{"metadata":{"id":"n_71yQajXwUP","trusted":true},"cell_type":"code","source":"train_df = train_df[train_df['answered_correctly']!=-1][features + [target]]\nvalidation_df = validation_df[validation_df['answered_correctly']!=-1][features + [target]]\ntest_df = test_df[test_df['answered_correctly']!=-1][features + [target]]","execution_count":null,"outputs":[]},{"metadata":{"id":"wPElhZAJaUmC","outputId":"966b3847-563a-469c-a577-8563bf549d1f","trusted":true},"cell_type":"code","source":"train_df.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"id":"hZJ9rpx0aJ-4","trusted":true},"cell_type":"code","source":"train_df = train_df.replace([np.inf, -np.inf], np.nan)\ntrain_df = train_df.fillna(0.5)\n\nvalidation_df = validation_df.replace([np.inf, -np.inf], np.nan)\nvalidation_df = validation_df.fillna(0.5)\n\ntest_df = test_df.replace([np.inf, -np.inf], np.nan)\ntest_df = test_df.fillna(0.5)\n\ntrain_df['prior_question_had_explanation'] = train_df['prior_question_had_explanation'].astype('bool')\nvalidation_df['prior_question_had_explanation'] = validation_df['prior_question_had_explanation'].astype('bool')\ntest_df['prior_question_had_explanation'] = test_df['prior_question_had_explanation'].astype('bool')","execution_count":null,"outputs":[]},{"metadata":{"id":"oAd56niyj9-x","outputId":"0b927d70-ce48-42fb-b3ad-3a4ee76c3c04","trusted":true},"cell_type":"code","source":"train_df.dtypes","execution_count":null,"outputs":[]},{"metadata":{"id":"W73HtzlrYgtc"},"cell_type":"markdown","source":"## LightGBM model Creation and Hyperparamter tuning"},{"metadata":{"id":"mGM_zv4gZaGB","outputId":"e09ebbbc-c84a-4032-e874-f7f23a003f20","trusted":true},"cell_type":"code","source":"import lightgbm as lgb\nimport optuna","execution_count":null,"outputs":[]},{"metadata":{"id":"dgMrG3QoYMK8","outputId":"97913c61-98ca-4804-c1a0-e41b843a4bc6","trusted":true},"cell_type":"code","source":"def create_model(trial):\n    num_leaves = trial.suggest_int(\"num_leaves\", 2, 31)\n    n_estimators = trial.suggest_int(\"n_estimators\", 50, 300)\n    max_depth = trial.suggest_int('max_depth', 3, 8)\n    min_child_samples = trial.suggest_int('min_child_samples', 100, 1200)\n    learning_rate = trial.suggest_uniform('learning_rate', 0.0001, 0.99)\n    min_data_in_leaf = trial.suggest_int('min_data_in_leaf', 5, 90)\n    bagging_fraction = trial.suggest_uniform('bagging_fraction', 0.0001, 1.0)\n    feature_fraction = trial.suggest_uniform('feature_fraction', 0.0001, 1.0)\n    model = lgb.LGBMClassifier(\n        num_leaves=num_leaves,\n        n_estimators=n_estimators, \n        max_depth=max_depth, \n        min_child_samples=min_child_samples, \n        min_data_in_leaf=min_data_in_leaf,\n        learning_rate=learning_rate,\n        feature_fraction=feature_fraction,\n        random_state=666\n)\n    return model\n\ndef objective(trial):\n    model = create_model(trial)\n    model.fit(train_df[features], train_df[target])\n    score = roc_auc_score(test_df[target].values, model.predict_proba(test_df[features])[:,1])\n    return score\n\n# # uncomment to use optuna\n# study = optuna.create_study(direction=\"maximize\")\n# study.optimize(objective, n_trials=70)\n# params = study.best_params\n# params['random_state'] = 666\n\nparams = {'num_leaves': 6, 'n_estimators': 242, 'max_depth': 3, \n          'min_child_samples': 352, 'learning_rate': 0.1954705382751018, 'min_data_in_leaf': 72, \n          'bagging_fraction': 0.2654709578619099, 'feature_fraction': 0.4901470199766588} #0.7714113571421728 0.7717271921771559  0.7711878055353522\n\nmodel = lgb.LGBMClassifier(**params)\nmodel.fit(train_df[features], train_df[target])","execution_count":null,"outputs":[]},{"metadata":{"id":"NSjicyMBuxVk","outputId":"fadb8b62-1aed-4740-d8c2-f551867e8294","trusted":true},"cell_type":"code","source":"print('LGB score: ', roc_auc_score(validation_df[target].values, model.predict_proba(validation_df[features])[:,1]))","execution_count":null,"outputs":[]},{"metadata":{"id":"N5JSUcapYfgs","outputId":"f398ebf6-db42-4655-ff72-9379f45d9f64","trusted":true},"cell_type":"code","source":"print('LGB score: ', roc_auc_score(test_df[target].values, model.predict_proba(test_df[features])[:,1]))","execution_count":null,"outputs":[]},{"metadata":{"id":"943ckzKaUzfH"},"cell_type":"markdown","source":"<a id=\"6\"></a>\n# Model Building -Training with full data"},{"metadata":{},"cell_type":"markdown","source":"In this section the the model tuned in the previous step is trained using the full dataset. The relevant feature engineering tables are saved to csv. The model is saved to txt. The output will be loaded into a seperate notebook in preparation for Kaggle submission."},{"metadata":{},"cell_type":"markdown","source":"## Load data"},{"metadata":{"id":"H68Yy5F8U5fK","trusted":true},"cell_type":"code","source":"cols_2_read = ['timestamp','content_id','user_id','content_type_id','answered_correctly','prior_question_elapsed_time','prior_question_had_explanation']#,'row_id','task_container_id','user_answer','prior_question_had_explanation']\nall_dtype_dic = {'row_id': 'uint64', 'timestamp': 'uint64', 'user_id': 'uint32', 'content_id': 'uint16', 'content_type_id': 'uint8',\n                              'task_container_id': 'uint16', 'user_answer': pd.CategoricalDtype([-1,0,1,2,3]), 'answered_correctly': 'int8', 'prior_question_elapsed_time': 'float32', \n                             'prior_question_had_explanation': 'boolean',\n                             }\ncols_2_read_dtype_dic = {k: all_dtype_dic[k] for k in all_dtype_dic.keys() & cols_2_read}\ntrain_df = pd.DataFrame()\nfor chunk in pd.read_csv('train.csv', low_memory=False,nrows=10**7,\n                       dtype=cols_2_read_dtype_dic,usecols = cols_2_read,chunksize=10**5 \n                      ):\n    train_df = pd.concat([train_df, chunk], ignore_index=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data preparation"},{"metadata":{"id":"1LltujMgVQtr","trusted":true},"cell_type":"code","source":"#add lectures and questions\ntrain_df = concat_questions_lectures(train_df,questions,lectures)","execution_count":null,"outputs":[]},{"metadata":{"id":"DptbaOWXVV6M","trusted":true},"cell_type":"code","source":"#change to numeric\ntrain_df[['type_of','question_tags','lecture_tag']] = train_df[['type_of','question_tags','lecture_tag']].astype('category') \ntrain_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']] = train_df[['timestamp', 'user_id','content_id','content_type_id','answered_correctly','bundle_id','correct_answer','part']].apply(pd.to_numeric, downcast='unsigned')","execution_count":null,"outputs":[]},{"metadata":{"id":"6SOtJrKsVYVI","trusted":true},"cell_type":"code","source":"#Feature engineering\n\n#user_interaction_lag\ntrain_df['user_interaction_lag'] = train_df.sort_values(['user_id','timestamp']).groupby('user_id')['timestamp'].diff()\n\n#content_answer_stats\ntrain_content_answer_stats_df = content_answer_stats(train_df)\n#train_content_answer_stats_df.to_csv('train_content_answer_stats_df.csv') # uncomment to save output to use for feature engineering of test dataset for competition submissions\ntrain_df = train_df.merge(train_content_answer_stats_df, how='left', on='content_id')\n\n#user answer stats\ntrain_user_answer_stats_df = user_answer_stats(train_df)\n#train_user_answer_stats_df.to_csv('train_user_answer_stats_df.csv') # uncomment to save output to use for feature engineering of test dataset for competition submissions\ntrain_df = train_df.merge(train_user_answer_stats_df, how='left', on='user_id')","execution_count":null,"outputs":[]},{"metadata":{"id":"bM2BLEprXXLf","trusted":true},"cell_type":"code","source":"train_df = train_df[train_df['answered_correctly']!=-1][features + [target]]","execution_count":null,"outputs":[]},{"metadata":{"id":"GcAA1g2dXLYu","outputId":"f6c14203-a5df-4f2d-b69e-95e19435df8d","trusted":true},"cell_type":"code","source":"train_df.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"id":"ecgg92g9XOiz","outputId":"54f03d44-f698-4da3-922a-b24cd4f2ccad","trusted":true},"cell_type":"code","source":"train_df.dtypes","execution_count":null,"outputs":[]},{"metadata":{"id":"ctAsKvAkWxIr","trusted":true},"cell_type":"code","source":"#deal with the nulls\ntrain_df = train_df.replace([np.inf, -np.inf], np.nan)\ntrain_df = train_df.fillna(0.5)\ntrain_df['prior_question_had_explanation'] = train_df['prior_question_had_explanation'].astype('bool')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train full model and save"},{"metadata":{"id":"stDBWNgaU4ok","outputId":"0b9d0fca-5483-4b53-8db7-5659dedad434","trusted":true},"cell_type":"code","source":"params = {'num_leaves': 6, 'n_estimators': 242, 'max_depth': 3, \n          'min_child_samples': 352, 'learning_rate': 0.1954705382751018, 'min_data_in_leaf': 72, \n          'bagging_fraction': 0.2654709578619099, 'feature_fraction': 0.4901470199766588}\n\nmodel = lgb.LGBMClassifier(**params)\nmodel.fit(train_df[features], train_df[target])\n#model.booster_.save_model('model.txt') # uncomment to save output to use for feature engineering of test dataset for competition submissions","execution_count":null,"outputs":[]},{"metadata":{"id":"75Wze0EdR645","trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}