First of all, I'd like to thank the organizers and Kaggle for offering this wonderful competition. The amount of data and the inference API made this competition more challenging, interesting, and practical.
\nI tried several models including LightGBM, CatBoost, and SAKT. In December, I decided to use only LGBM models mainly because of my limited GPU resource. (I bet some teams used only Kaggle GPUs and made better models.)
\nMy final model is a stack of LGBM models.
\nThe first level of the stack is an LGBM model with typical features such as timestamp lag, rolling mean accuracy, and whether a user saw the same quiz before. I also created a custom quiz ranking.
\nThe level 0 model itself trained with half of the data scored the public LB 0.791.
\nIn addition to the same features as level 0, I added
\nThe level 1 model trained with the other half of the data scored the public LB 0.798. I created two level 1 models changing the number of leaves 256 and 500. Averaging them gave me LB 0.799.
\nThe private score was 0.8000.
\nAs I couldn't optimize the memory usage, there are several things I couldn't include in the solution.
\nFor example, the quiz history for each user such as has_see_this_quiz_before, has_answered_this_quiz_before, how_long_has_past_since_last_saw_this_quiz. To save memory, I was using 'content_id' mod p, where I took p=409. Of course, this introduces some noise. There were discussions to use Sqlite3 but I didn't give it a try. In my final model, I didn't use mod p but instead, I only keep track with saw_this_quiz_before and how_long_has_past_since_last_saw_this_quiz features. My notebook used almost all 16GB of memory. If I added one more model for an ensemble, it gave me a memory error.
\nI like this challenging competition. It not only requires data science skills but also engineering skills to make a model work at inference. I look forward to learning new things from your solutions! If you have questions or comments, please leave a comment below!
\nThank you for reading!
", "messages": [ { "id": "1143517", "postDate": "01/08/2021 00:02:32", "content": "First of all, I'd like to thank the organizers and Kaggle for offering this wonderful competition. The amount of data and the inference API made this competition more challenging, interesting, and practical.
\nI tried several models including LightGBM, CatBoost, and SAKT. In December, I decided to use only LGBM models mainly because of my limited GPU resource. (I bet some teams used only Kaggle GPUs and made better models.)
\nMy final model is a stack of LGBM models.
\nThe first level of the stack is an LGBM model with typical features such as timestamp lag, rolling mean accuracy, and whether a user saw the same quiz before. I also created a custom quiz ranking.
\nThe level 0 model itself trained with half of the data scored the public LB 0.791.
\nIn addition to the same features as level 0, I added
\nThe level 1 model trained with the other half of the data scored the public LB 0.798. I created two level 1 models changing the number of leaves 256 and 500. Averaging them gave me LB 0.799.
\nThe private score was 0.8000.
\nAs I couldn't optimize the memory usage, there are several things I couldn't include in the solution.
\nFor example, the quiz history for each user such as has_see_this_quiz_before, has_answered_this_quiz_before, how_long_has_past_since_last_saw_this_quiz. To save memory, I was using 'content_id' mod p, where I took p=409. Of course, this introduces some noise. There were discussions to use Sqlite3 but I didn't give it a try. In my final model, I didn't use mod p but instead, I only keep track with saw_this_quiz_before and how_long_has_past_since_last_saw_this_quiz features. My notebook used almost all 16GB of memory. If I added one more model for an ensemble, it gave me a memory error.
\nI like this challenging competition. It not only requires data science skills but also engineering skills to make a model work at inference. I look forward to learning new things from your solutions! If you have questions or comments, please leave a comment below!
\nThank you for reading!
", "rawMarkdown": "# Good competition!\n\nFirst of all, I'd like to thank the organizers and Kaggle for offering this wonderful competition. The amount of data and the inference API made this competition more challenging, interesting, and practical.\n\n# LGBM only solution\nI tried several models including LightGBM, CatBoost, and SAKT. In December, I decided to use only LGBM models mainly because of my limited GPU resource. (I bet some teams used only Kaggle GPUs and made better models.)\n\n# Stack\nMy final model is a stack of LGBM models.\n\n## Level 0\nThe first level of the stack is an LGBM model with typical features such as timestamp lag, rolling mean accuracy, and whether a user saw the same quiz before. I also created a custom quiz ranking.\n\nThe level 0 model itself trained with half of the data scored the public LB 0.791.\n\n## Level 1\nIn addition to the same features as level 0, I added\n\n* level 0 lgbm prediction\n* previous level 0 lgbm prediction error\n* previous 10 records of these features + some features\n\nThe level 1 model trained with the other half of the data scored the public LB 0.798. I created two level 1 models changing the number of leaves 256 and 500. Averaging them gave me LB 0.799.\n\nThe private score was 0.8000.\n\n## Wish I could do\nAs I couldn't optimize the memory usage, there are several things I couldn't include in the solution.\n\nFor example, the quiz history for each user such as has_see_this_quiz_before, has_answered_this_quiz_before, how_long_has_past_since_last_saw_this_quiz. To save memory, I was using 'content_id' mod p, where I took p=409. Of course, this introduces some noise. There were discussions to use Sqlite3 but I didn't give it a try. In my final model, I didn't use mod p but instead, I only keep track with saw_this_quiz_before and how_long_has_past_since_last_saw_this_quiz features. My notebook used almost all 16GB of memory. If I added one more model for an ensemble, it gave me a memory error.\n\n\n# Finally\nI like this challenging competition. It not only requires data science skills but also engineering skills to make a model work at inference. I look forward to learning new things from your solutions! If you have questions or comments, please leave a comment below!\n\n**Thank you for reading!**", "votes": null }, { "id": "1143525", "postDate": "01/08/2021 00:08:38", "content": "Thank you for sharing and congrats for solo silver. May I ask what you mean by 'custom quiz ranking' ?
", "rawMarkdown": "Thank you for sharing and congrats for solo silver. May I ask what you mean by 'custom quiz ranking' ?", "votes": null }, { "id": "1143526", "postDate": "01/08/2021 00:09:31", "content": "Congratulations for the strong finish! The main challenge in this competition was to use the 16GB of RAM efficiently and getting 0.80 LB with just LGBM is interesting. Looking forward to read your code :)
", "rawMarkdown": "Congratulations for the strong finish! The main challenge in this competition was to use the 16GB of RAM efficiently and getting 0.80 LB with just LGBM is interesting. Looking forward to read your code :)", "votes": null }, { "id": "1143529", "postDate": "01/08/2021 00:11:04", "content": "It's amazing how you were able to achieve such a decent score even after facing all the computational resources. Good Job!
", "rawMarkdown": "It's amazing how you were able to achieve such a decent score even after facing all the computational resources. Good Job!", "votes": null }, { "id": "1143536", "postDate": "01/08/2021 00:15:38", "content": "What is a a quiz? A grouping of questions based on batch_id?
", "rawMarkdown": "What is a a quiz? A grouping of questions based on batch_id?", "votes": null }, { "id": "1143543", "postDate": "01/08/2021 00:18:54", "content": "Congrats on silver medal and thanks for sharing solution. Good score with LGBM
", "rawMarkdown": "Congrats on silver medal and thanks for sharing solution. Good score with LGBM", "votes": null }, { "id": "1143574", "postDate": "01/08/2021 00:37:15", "content": "@sishihara Thank you for your comment. Let me explain what I mean by custom quiz ranking. By a quiz, I meant a question.
\nFor each user the first few quizzes are for assessment. Thus I thought that how to answer the first 20 quizzes might classify students levels. (Here 20 is just my arbitrary choice.) So, students were classified with 2**20 classes. Then for each class, I give a score using its binary (0 for incorrect and 1 for correct and each class is represented by a series of 0 and 1).
\nNow, I take the average of students' rank defined above for each quiz. This is the custom quiz ranking.
", "rawMarkdown": "sishihara Thank you for your comment. Let me explain what I mean by custom quiz ranking. By a quiz, I meant a question.\n\nFor each user the first few quizzes are for assessment. Thus I thought that how to answer the first 20 quizzes might classify students levels. (Here 20 is just my arbitrary choice.) So, students were classified with 2**20 classes. Then for each class, I give a score using its binary (0 for incorrect and 1 for correct and each class is represented by a series of 0 and 1).\n\nNow, I take the average of students' rank defined above for each quiz. This is the custom quiz ranking.", "votes": null }, { "id": "1143592", "postDate": "01/08/2021 00:51:01", "content": "@yutsumura I understand. Thank you for the additional explanation!
", "rawMarkdown": "yutsumura I understand. Thank you for the additional explanation!", "votes": null }, { "id": "1143596", "postDate": "01/08/2021 00:53:03", "content": "This is cool!! Riiid uses 30 questions by default.
", "rawMarkdown": "This is cool!! Riiid uses 30 questions by default.", "votes": null }, { "id": "1143597", "postDate": "01/08/2021 00:54:15", "content": "I managed to get a 0.796 with just a single LGBM. ~500 features
", "rawMarkdown": "I managed to get a 0.796 with just a single LGBM. ~500 features", "votes": null }, { "id": "1143599", "postDate": "01/08/2021 00:54:27", "content": "I am curious if you can explain more about \"previous 10 records of these features\", So you mean to say after creating the features, so did a tail(10) after grouping by the user_id's? Nice idea on using a wide lgbm!
\nWe also have a single gbm with no hyper-tuner params having ~39 features and .792 on LB.
", "rawMarkdown": "I am curious if you can explain more about \"previous 10 records of these features\", So you mean to say after creating the features, so did a tail(10) after grouping by the user_id's? Nice idea on using a wide lgbm!\n\nWe also have a single gbm with no hyper-tuner params having ~39 features and .792 on LB.", "votes": null }, { "id": "1143612", "postDate": "01/08/2021 01:11:30", "content": "I'd be interested to hear how you managed 500 features with the 16GB limitiation!
", "rawMarkdown": "I'd be interested to hear how you managed 500 features with the 16GB limitiation!", "votes": null }, { "id": "1143655", "postDate": "01/08/2021 01:46:07", "content": "Well training was all done locally, and that took like 60+GB. For my prediction notebook, I had about 10 dicts that I used to generate features from. Most were very small, just keeping track of counts and whatnot, but the largest was the ~3GB user history dict which was a dict of users with an 8xN int32 array with their history of content_id/answers/part/etc. With everything loaded in memory I think I was only using about 8GB when starting to predict. I used numba to generate features and update the dicts quickly when submitting… I think it still took 2-3 hours to run.
I‘ve thought about this idea before but abandon at storing stage. Could you please show your way storing big size dict(). In my method it is very slow and get my PC stuck.
", "rawMarkdown": "I‘ve thought about this idea before but abandon at storing stage. Could you please show your way storing big size dict(). In my method it is very slow and get my PC stuck.", "votes": null }, { "id": "1145103", "postDate": "01/08/2021 22:17:27", "content": "@southsakura This is relevant piece of code from my pipeline for storing the user history dictionary.
\n[Edit] Formatting here looks awful, sorry.
\ntrain['timestamp'] = train['timestamp'] / 1000\nu_history = train.groupby('user_id').apply(lambda x: [np.array(x['timestamp'], dtype='i4'), \n np.array(x['task_container_id'], dtype='i4'), \n np.array(x['answered_correctly'], dtype='i4'), \n np.array(x['user_answer'], dtype='i4'),\n np.array(x['content_id'], dtype='i4'),\n np.array(x['part'], dtype='i4'),\n np.array(x['prior_question_had_explanation'].fillna(False), dtype='i4'),\n np.array(x['bundle_id'], dtype='i4'),]).apply(pd.Series)\nu_history.columns = ['timestamp_history', 'task_container_history','answered_correctly_history','user_answer_history', 'user_content_history', 'user_part_history', 'explanation_history', 'bundle_history']\nu_history = u_history.reset_index()\nu_history_dict = {}\nfor row in tqdm(u_history.values):\n u_history_dict[row[0]] = np.vstack([np.array(row[1]), \n np.array(row[2]), \n np.array(row[3]),\n np.array(row[4]), \n np.array(row[5]), \n np.array(row[6]),\n np.array(row[7]),\n np.array(row[8])])\n\nwith open(f\"./data/u_history_dict_{n}.pkl\", \"wb\") as f:\n pkl.dump(u_history_dict, f)\nThank you for sharing and congrats for your silver medal. How do you calculate 'previous level 0 lgbm prediction error' ?
", "rawMarkdown": "Thank you for sharing and congrats for your silver medal. How do you calculate 'previous level 0 lgbm prediction error' ?", "votes": null }, { "id": "1147878", "postDate": "01/10/2021 19:16:14", "content": "For each user and for each selected feature, I simply keep last 10 records as a list. I didn't use pandas groupby at inference.
", "rawMarkdown": "For each user and for each selected feature, I simply keep last 10 records as a list. I didn't use pandas groupby at inference.", "votes": null }, { "id": "1147879", "postDate": "01/10/2021 19:18:55", "content": "I defined it as
\n'previous level0 lgbm prediction error' = 'answered_correctly' - 'lgbm0_prediction'.
\nI also tried its absolute value version but that performed worse.
", "rawMarkdown": "I defined it as\n\n'previous level0 lgbm prediction error' = 'answered_correctly' - 'lgbm0_prediction'.\n\nI also tried its absolute value version but that performed worse.", "votes": null }, { "id": "1147893", "postDate": "01/10/2021 19:39:48", "content": "Excuse my insistence but I can't see how you calculate this feature for inference. Is it the error of the previous interaction?
", "rawMarkdown": "Excuse my insistence but I can't see how you calculate this feature for inference. Is it the error of the previous interaction?", "votes": null } ], "comments": [ { "id": 1143525, "author_name": "sishihara", "author_url": "", "post_date": "01/08/2021 00:08:38", "content": "Thank you for sharing and congrats for solo silver. May I ask what you mean by 'custom quiz ranking' ?
", "votes": null, "replies": [ { "id": 1143536, "author_name": "npa02012", "author_url": "", "post_date": "01/08/2021 00:15:38", "content": "What is a a quiz? A grouping of questions based on batch_id?
", "votes": null, "replies": [] }, { "id": 1143574, "author_name": "yutsumura", "author_url": "", "post_date": "01/08/2021 00:37:15", "content": "@sishihara Thank you for your comment. Let me explain what I mean by custom quiz ranking. By a quiz, I meant a question.
\nFor each user the first few quizzes are for assessment. Thus I thought that how to answer the first 20 quizzes might classify students levels. (Here 20 is just my arbitrary choice.) So, students were classified with 2**20 classes. Then for each class, I give a score using its binary (0 for incorrect and 1 for correct and each class is represented by a series of 0 and 1).
\nNow, I take the average of students' rank defined above for each quiz. This is the custom quiz ranking.
", "votes": null, "replies": [] }, { "id": 1143592, "author_name": "sishihara", "author_url": "", "post_date": "01/08/2021 00:51:01", "content": "@yutsumura I understand. Thank you for the additional explanation!
", "votes": null, "replies": [] }, { "id": 1143596, "author_name": "adityaecdrid", "author_url": "", "post_date": "01/08/2021 00:53:03", "content": "This is cool!! Riiid uses 30 questions by default.
", "votes": null, "replies": [] } ] }, { "id": 1143526, "author_name": "amiiiney", "author_url": "", "post_date": "01/08/2021 00:09:31", "content": "Congratulations for the strong finish! The main challenge in this competition was to use the 16GB of RAM efficiently and getting 0.80 LB with just LGBM is interesting. Looking forward to read your code :)
", "votes": null, "replies": [] }, { "id": 1143529, "author_name": "amoiza1", "author_url": "", "post_date": "01/08/2021 00:11:04", "content": "It's amazing how you were able to achieve such a decent score even after facing all the computational resources. Good Job!
", "votes": null, "replies": [] }, { "id": 1143543, "author_name": "duykhanh99", "author_url": "", "post_date": "01/08/2021 00:18:54", "content": "Congrats on silver medal and thanks for sharing solution. Good score with LGBM
", "votes": null, "replies": [] }, { "id": 1143597, "author_name": "brandenkmurray", "author_url": "", "post_date": "01/08/2021 00:54:15", "content": "I managed to get a 0.796 with just a single LGBM. ~500 features
", "votes": null, "replies": [ { "id": 1143612, "author_name": "npa02012", "author_url": "", "post_date": "01/08/2021 01:11:30", "content": "I'd be interested to hear how you managed 500 features with the 16GB limitiation!
", "votes": null, "replies": [] }, { "id": 1143655, "author_name": "brandenkmurray", "author_url": "", "post_date": "01/08/2021 01:46:07", "content": "Well training was all done locally, and that took like 60+GB. For my prediction notebook, I had about 10 dicts that I used to generate features from. Most were very small, just keeping track of counts and whatnot, but the largest was the ~3GB user history dict which was a dict of users with an 8xN int32 array with their history of content_id/answers/part/etc. With everything loaded in memory I think I was only using about 8GB when starting to predict. I used numba to generate features and update the dicts quickly when submitting… I think it still took 2-3 hours to run.
I‘ve thought about this idea before but abandon at storing stage. Could you please show your way storing big size dict(). In my method it is very slow and get my PC stuck.
", "votes": null, "replies": [] }, { "id": 1145103, "author_name": "brandenkmurray", "author_url": "", "post_date": "01/08/2021 22:17:27", "content": "@southsakura This is relevant piece of code from my pipeline for storing the user history dictionary.
\n[Edit] Formatting here looks awful, sorry.
\ntrain['timestamp'] = train['timestamp'] / 1000\nu_history = train.groupby('user_id').apply(lambda x: [np.array(x['timestamp'], dtype='i4'), \n np.array(x['task_container_id'], dtype='i4'), \n np.array(x['answered_correctly'], dtype='i4'), \n np.array(x['user_answer'], dtype='i4'),\n np.array(x['content_id'], dtype='i4'),\n np.array(x['part'], dtype='i4'),\n np.array(x['prior_question_had_explanation'].fillna(False), dtype='i4'),\n np.array(x['bundle_id'], dtype='i4'),]).apply(pd.Series)\nu_history.columns = ['timestamp_history', 'task_container_history','answered_correctly_history','user_answer_history', 'user_content_history', 'user_part_history', 'explanation_history', 'bundle_history']\nu_history = u_history.reset_index()\nu_history_dict = {}\nfor row in tqdm(u_history.values):\n u_history_dict[row[0]] = np.vstack([np.array(row[1]), \n np.array(row[2]), \n np.array(row[3]),\n np.array(row[4]), \n np.array(row[5]), \n np.array(row[6]),\n np.array(row[7]),\n np.array(row[8])])\n\nwith open(f\"./data/u_history_dict_{n}.pkl\", \"wb\") as f:\n pkl.dump(u_history_dict, f)\nI am curious if you can explain more about \"previous 10 records of these features\", So you mean to say after creating the features, so did a tail(10) after grouping by the user_id's? Nice idea on using a wide lgbm!
\nWe also have a single gbm with no hyper-tuner params having ~39 features and .792 on LB.
", "votes": null, "replies": [ { "id": 1147878, "author_name": "yutsumura", "author_url": "", "post_date": "01/10/2021 19:16:14", "content": "For each user and for each selected feature, I simply keep last 10 records as a list. I didn't use pandas groupby at inference.
", "votes": null, "replies": [] } ] }, { "id": 1147577, "author_name": "maherelouahabi", "author_url": "", "post_date": "01/10/2021 15:35:54", "content": "Thank you for sharing and congrats for your silver medal. How do you calculate 'previous level 0 lgbm prediction error' ?
", "votes": null, "replies": [ { "id": 1147879, "author_name": "yutsumura", "author_url": "", "post_date": "01/10/2021 19:18:55", "content": "I defined it as
\n'previous level0 lgbm prediction error' = 'answered_correctly' - 'lgbm0_prediction'.
\nI also tried its absolute value version but that performed worse.
", "votes": null, "replies": [] }, { "id": 1147893, "author_name": "maherelouahabi", "author_url": "", "post_date": "01/10/2021 19:39:48", "content": "Excuse my insistence but I can't see how you calculate this feature for inference. Is it the error of the previous interaction?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1143517": "# Good competition!\n\nFirst of all, I'd like to thank the organizers and Kaggle for offering this wonderful competition. The amount of data and the inference API made this competition more challenging, interesting, and practical.\n\n# LGBM only solution\nI tried several models including LightGBM, CatBoost, and SAKT. In December, I decided to use only LGBM models mainly because of my limited GPU resource. (I bet some teams used only Kaggle GPUs and made better models.)\n\n# Stack\nMy final model is a stack of LGBM models.\n\n## Level 0\nThe first level of the stack is an LGBM model with typical features such as timestamp lag, rolling mean accuracy, and whether a user saw the same quiz before. I also created a custom quiz ranking.\n\nThe level 0 model itself trained with half of the data scored the public LB 0.791.\n\n## Level 1\nIn addition to the same features as level 0, I added\n\n* level 0 lgbm prediction\n* previous level 0 lgbm prediction error\n* previous 10 records of these features + some features\n\nThe level 1 model trained with the other half of the data scored the public LB 0.798. I created two level 1 models changing the number of leaves 256 and 500. Averaging them gave me LB 0.799.\n\nThe private score was 0.8000.\n\n## Wish I could do\nAs I couldn't optimize the memory usage, there are several things I couldn't include in the solution.\n\nFor example, the quiz history for each user such as has_see_this_quiz_before, has_answered_this_quiz_before, how_long_has_past_since_last_saw_this_quiz. To save memory, I was using 'content_id' mod p, where I took p=409. Of course, this introduces some noise. There were discussions to use Sqlite3 but I didn't give it a try. In my final model, I didn't use mod p but instead, I only keep track with saw_this_quiz_before and how_long_has_past_since_last_saw_this_quiz features. My notebook used almost all 16GB of memory. If I added one more model for an ensemble, it gave me a memory error.\n\n\n# Finally\nI like this challenging competition. It not only requires data science skills but also engineering skills to make a model work at inference. I look forward to learning new things from your solutions! If you have questions or comments, please leave a comment below!\n\n**Thank you for reading!**", "1143525": "Thank you for sharing and congrats for solo silver. May I ask what you mean by 'custom quiz ranking' ?", "1143526": "Congratulations for the strong finish! The main challenge in this competition was to use the 16GB of RAM efficiently and getting 0.80 LB with just LGBM is interesting. Looking forward to read your code :)", "1143529": "It's amazing how you were able to achieve such a decent score even after facing all the computational resources. Good Job!", "1143536": "What is a a quiz? A grouping of questions based on batch_id?", "1143543": "Congrats on silver medal and thanks for sharing solution. Good score with LGBM", "1143574": "sishihara Thank you for your comment. Let me explain what I mean by custom quiz ranking. By a quiz, I meant a question.\n\nFor each user the first few quizzes are for assessment. Thus I thought that how to answer the first 20 quizzes might classify students levels. (Here 20 is just my arbitrary choice.) So, students were classified with 2**20 classes. Then for each class, I give a score using its binary (0 for incorrect and 1 for correct and each class is represented by a series of 0 and 1).\n\nNow, I take the average of students' rank defined above for each quiz. This is the custom quiz ranking.", "1143592": "yutsumura I understand. Thank you for the additional explanation!", "1143596": "This is cool!! Riiid uses 30 questions by default.", "1143597": "I managed to get a 0.796 with just a single LGBM. ~500 features", "1143599": "I am curious if you can explain more about \"previous 10 records of these features\", So you mean to say after creating the features, so did a tail(10) after grouping by the user_id's? Nice idea on using a wide lgbm!\n\nWe also have a single gbm with no hyper-tuner params having ~39 features and .792 on LB.", "1143612": "I'd be interested to hear how you managed 500 features with the 16GB limitiation!", "1143655": "Well training was all done locally, and that took like 60+GB. For my prediction notebook, I had about 10 dicts that I used to generate features from. Most were very small, just keeping track of counts and whatnot, but the largest was the ~3GB user history dict which was a dict of users with an 8xN int32 array with their history of content_id/answers/part/etc. With everything loaded in memory I think I was only using about 8GB when starting to predict. I used `numba` to generate features and update the dicts quickly when submitting... I think it still took 2-3 hours to run.", "1143977": "I‘ve thought about this idea before but abandon at storing stage. Could you please show your way storing big size dict(). In my method it is very slow and get my PC stuck.", "1145103": "southsakura This is relevant piece of code from my pipeline for storing the user history dictionary.\n\n[Edit] Formatting here looks awful, sorry. \n\n```\ntrain['timestamp'] = train['timestamp'] / 1000\nu_history = train.groupby('user_id').apply(lambda x: [np.array(x['timestamp'], dtype='i4'), \n np.array(x['task_container_id'], dtype='i4'), \n np.array(x['answered_correctly'], dtype='i4'), \n np.array(x['user_answer'], dtype='i4'),\n np.array(x['content_id'], dtype='i4'),\n np.array(x['part'], dtype='i4'),\n np.array(x['prior_question_had_explanation'].fillna(False), dtype='i4'),\n np.array(x['bundle_id'], dtype='i4'),]).apply(pd.Series)\nu_history.columns = ['timestamp_history', 'task_container_history','answered_correctly_history','user_answer_history', 'user_content_history', 'user_part_history', 'explanation_history', 'bundle_history']\nu_history = u_history.reset_index()\nu_history_dict = {}\nfor row in tqdm(u_history.values):\n u_history_dict[row[0]] = np.vstack([np.array(row[1]), \n np.array(row[2]), \n np.array(row[3]),\n np.array(row[4]), \n np.array(row[5]), \n np.array(row[6]),\n np.array(row[7]),\n np.array(row[8])])\n \nwith open(f\"./data/u_history_dict_{n}.pkl\", \"wb\") as f:\n pkl.dump(u_history_dict, f)\n```", "1147577": "Thank you for sharing and congrats for your silver medal. How do you calculate 'previous level 0 lgbm prediction error' ?", "1147878": "For each user and for each selected feature, I simply keep last 10 records as a list. I didn't use pandas groupby at inference.", "1147879": "I defined it as\n\n'previous level0 lgbm prediction error' = 'answered_correctly' - 'lgbm0_prediction'.\n\nI also tried its absolute value version but that performed worse.", "1147893": "Excuse my insistence but I can't see how you calculate this feature for inference. Is it the error of the previous interaction?" }, "source": "meta" }