{ "id": 392605, "title": "Which features do you think are the most important?", "url": "/competitions/predict-student-performance-from-game-play/discussion/392605", "author_name": "", "post_date": "2023-03-06T05:36:21.782800900Z", "votes": 2, "comment_count": 22, "views": 0, "content": "

I am new to Kaggle, and from the discussion section I realized that it's very important to choose the proper features for this problem. Then which features do you think are the most important ones that can yield good performance?

", "messages": [ { "id": "2170584", "postDate": "03/06/2023 05:36:21", "content": "

I am new to Kaggle, and from the discussion section I realized that it's very important to choose the proper features for this problem. Then which features do you think are the most important ones that can yield good performance?

", "rawMarkdown": "I am new to Kaggle, and from the discussion section I realized that it's very important to choose the proper features for this problem. Then which features do you think are the most important ones that can yield good performance?", "votes": null }, { "id": "2170812", "postDate": "03/06/2023 09:53:21", "content": "

Hi there,
\nevent_name is the most important feature. I got a lb 0.673 with only putting one-hot of event_name's features into LightGBM.
\nBUT, if you wanna got a better classifier, you should generate more features, maybe which seem not so important. The important features could not contain full information of the game, so we can find more features to complement it.
\nNow I'm waiting the result of 293, 443, 522 features models, which got a cv of 0.690.

", "rawMarkdown": "Hi there, \n`event_name` is the most important feature. I got a lb 0.673 with only putting one-hot of `event_name`'s features into LightGBM.\nBUT, if you wanna got a better classifier, you should generate more features, maybe which seem not so important. The important features could not contain full information of the game, so we can find more features to complement it.\nNow I'm waiting the result of 293, 443, 522 features models, which got a cv of 0.690.", "votes": null }, { "id": "2170825", "postDate": "03/06/2023 10:11:49", "content": "

BTW, you can get important features of model from LightGBM.feature_importance() or XGBClassifier.feature_importances_.

", "rawMarkdown": "BTW, you can get important features of model from `LightGBM.feature_importance()` or `XGBClassifier.feature_importances_`.", "votes": null }, { "id": "2172240", "postDate": "03/07/2023 11:46:32", "content": "

Cool! Thank you for your advice! I am going to try LightGBM!

", "rawMarkdown": "Cool! Thank you for your advice! I am going to try LightGBM!", "votes": null }, { "id": "2172253", "postDate": "03/07/2023 11:55:24", "content": "

i think it's important to generate great variety of features to get a sense of what is important.

", "rawMarkdown": "i think it's important to generate great variety of features to get a sense of what is important.", "votes": null }, { "id": "2172376", "postDate": "03/07/2023 13:41:28", "content": "

Yeah, I generate features from df.groupby(['session_id','level_group])

", "rawMarkdown": "Yeah, I generate features from df.groupby(['session_id','level_group])", "votes": null }, { "id": "2172382", "postDate": "03/07/2023 13:46:27", "content": "

yes you can also try to create specific features for level groups

", "rawMarkdown": "yes you can also try to create specific features for level groups", "votes": null }, { "id": "2172925", "postDate": "03/07/2023 23:41:08", "content": "

Another question, I noticed that you use a great number of features, on what device did you run your program, and how long does it take?

", "rawMarkdown": "Another question, I noticed that you use a great number of features, on what device did you run your program, and how long does it take?", "votes": null }, { "id": "2172996", "postDate": "03/08/2023 02:09:36", "content": "

I just train these features with 30mins using GPU P100 in kaggle, but scoring takes 2hrs more.

", "rawMarkdown": "I just train these features with 30mins using GPU P100 in kaggle, but scoring takes 2hrs more.", "votes": null }, { "id": "2173303", "postDate": "03/08/2023 09:38:20", "content": "

Time, hover_time, past answers

", "rawMarkdown": "Time, hover_time, past answers", "votes": null }, { "id": "2173945", "postDate": "03/08/2023 18:15:44", "content": "

How to use post answers as features?

", "rawMarkdown": "How to use post answers as features?", "votes": null }, { "id": "2174207", "postDate": "03/09/2023 00:12:19", "content": "

Cool, may I ask you how do you get so many feature then?

", "rawMarkdown": "Cool, may I ask you how do you get so many feature then?", "votes": null }, { "id": "2174400", "postDate": "03/09/2023 05:21:23", "content": "

Hey Dennis, would you mind clarifying what you mean by \"post answers\"?

", "rawMarkdown": "Hey Dennis, would you mind clarifying what you mean by \"post answers\"?", "votes": null }, { "id": "2174507", "postDate": "03/09/2023 07:11:23", "content": "

Combine two or more features to renew features. In another word, find relationship between existing features and use this relationship as an new feature.

", "rawMarkdown": "Combine two or more features to renew features. In another word, find relationship between existing features and use this relationship as an new feature.", "votes": null }, { "id": "2174758", "postDate": "03/09/2023 11:03:02", "content": "

Add previous answers that your model generated as features. Like when building a model for q2 use what your model for q1 predicted. For q3 use predictions for q1 and q2, and so on.

", "rawMarkdown": "Add previous answers that your model generated as features. Like when building a model for q2 use what your model for q1 predicted. For q3 use predictions for q1 and q2, and so on.", "votes": null }, { "id": "2174761", "postDate": "03/09/2023 11:03:43", "content": "

Sorry, past, not post 😂

", "rawMarkdown": "Sorry, past, not post 😂", "votes": null }, { "id": "2174886", "postDate": "03/09/2023 13:38:48", "content": "

Thanks, that seems to be a popular method for generating features！

", "rawMarkdown": "Thanks, that seems to be a popular method for generating features！", "votes": null }, { "id": "2174889", "postDate": "03/09/2023 13:40:39", "content": "

Wonderful Dennis, before I tried that on PyTorch Feed Forward NN and LB improved 0.001

", "rawMarkdown": "Wonderful Dennis, before I tried that on PyTorch Feed Forward NN and LB improved 0.001", "votes": null }, { "id": "2174968", "postDate": "03/09/2023 14:44:18", "content": "

Then how many features did you use in the end?

", "rawMarkdown": "Then how many features did you use in the end?", "votes": null }, { "id": "2175610", "postDate": "03/10/2023 02:24:30", "content": "

My best lb 0.695 use 747 1315 1603 features with cv 0.694. However, when I continue to increase features, it got not much improved for cv. Anyway, my best cv is 0.695 with 1792 3570 4338 features. But it's no use and you cann't infer so much features in 9 hrs.

", "rawMarkdown": "My best lb 0.695 use 747 1315 1603 features with cv 0.694. However, when I continue to increase features, it got not much improved for cv. Anyway, my best cv is 0.695 with 1792 3570 4338 features. But it's no use and you cann't infer so much features in 9 hrs.", "votes": null }, { "id": "2175618", "postDate": "03/10/2023 02:30:14", "content": "

By the way, @pourchot's notebook is a good example for data processing.

", "rawMarkdown": "By the way, @pourchot's [notebook](https://www.kaggle.com/code/pourchot/simple-xgb) is a good example for data processing.", "votes": null }, { "id": "2175957", "postDate": "03/10/2023 09:25:11", "content": "

Hey Dennis, would you mind sharing how many the CV and LB improve after adding the past answers features ? I added these features and find that these features improve a lot in my CV (even got 0.696 in CV) but do not bring too much improve in LB (without 0.001). It make me confused since I think the CV is correlated with the LB in this competiion when I generated other features before.

", "rawMarkdown": "Hey Dennis, would you mind sharing how many the CV and LB improve after adding the past answers features ? I added these features and find that these features improve a lot in my CV (even got 0.696 in CV) but do not bring too much improve in LB (without 0.001). It make me confused since I think the CV is correlated with the LB in this competiion when I generated other features before.", "votes": null }, { "id": "2176901", "postDate": "03/11/2023 02:59:18", "content": "

Nice! Thank you!

", "rawMarkdown": "Nice! Thank you!", "votes": null } ], "comments": [ { "id": 2170812, "author_name": "mengvision", "author_url": "", "post_date": "03/06/2023 09:53:21", "content": "

Hi there,
\nevent_name is the most important feature. I got a lb 0.673 with only putting one-hot of event_name's features into LightGBM.
\nBUT, if you wanna got a better classifier, you should generate more features, maybe which seem not so important. The important features could not contain full information of the game, so we can find more features to complement it.
\nNow I'm waiting the result of 293, 443, 522 features models, which got a cv of 0.690.

", "votes": null, "replies": [ { "id": 2170825, "author_name": "mengvision", "author_url": "", "post_date": "03/06/2023 10:11:49", "content": "

BTW, you can get important features of model from LightGBM.feature_importance() or XGBClassifier.feature_importances_.

", "votes": null, "replies": [ { "id": 2172240, "author_name": "harrisonliang", "author_url": "", "post_date": "03/07/2023 11:46:32", "content": "

Cool! Thank you for your advice! I am going to try LightGBM!

", "votes": null, "replies": [] }, { "id": 2172925, "author_name": "harrisonliang", "author_url": "", "post_date": "03/07/2023 23:41:08", "content": "

Another question, I noticed that you use a great number of features, on what device did you run your program, and how long does it take?

", "votes": null, "replies": [ { "id": 2172996, "author_name": "mengvision", "author_url": "", "post_date": "03/08/2023 02:09:36", "content": "

I just train these features with 30mins using GPU P100 in kaggle, but scoring takes 2hrs more.

", "votes": null, "replies": [ { "id": 2174207, "author_name": "harrisonliang", "author_url": "", "post_date": "03/09/2023 00:12:19", "content": "

Cool, may I ask you how do you get so many feature then?

", "votes": null, "replies": [ { "id": 2174507, "author_name": "mengvision", "author_url": "", "post_date": "03/09/2023 07:11:23", "content": "

Combine two or more features to renew features. In another word, find relationship between existing features and use this relationship as an new feature.

", "votes": null, "replies": [ { "id": 2174886, "author_name": "harrisonliang", "author_url": "", "post_date": "03/09/2023 13:38:48", "content": "

Thanks, that seems to be a popular method for generating features！

", "votes": null, "replies": [] }, { "id": 2174968, "author_name": "harrisonliang", "author_url": "", "post_date": "03/09/2023 14:44:18", "content": "

Then how many features did you use in the end?

", "votes": null, "replies": [ { "id": 2175610, "author_name": "mengvision", "author_url": "", "post_date": "03/10/2023 02:24:30", "content": "

My best lb 0.695 use 747 1315 1603 features with cv 0.694. However, when I continue to increase features, it got not much improved for cv. Anyway, my best cv is 0.695 with 1792 3570 4338 features. But it's no use and you cann't infer so much features in 9 hrs.

", "votes": null, "replies": [ { "id": 2175618, "author_name": "mengvision", "author_url": "", "post_date": "03/10/2023 02:30:14", "content": "

By the way, @pourchot's notebook is a good example for data processing.

", "votes": null, "replies": [ { "id": 2176901, "author_name": "harrisonliang", "author_url": "", "post_date": "03/11/2023 02:59:18", "content": "

Nice! Thank you!

", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ] } ] } ] }, { "id": 2172253, "author_name": "simonveitner", "author_url": "", "post_date": "03/07/2023 11:55:24", "content": "

i think it's important to generate great variety of features to get a sense of what is important.

", "votes": null, "replies": [ { "id": 2172376, "author_name": "harrisonliang", "author_url": "", "post_date": "03/07/2023 13:41:28", "content": "

Yeah, I generate features from df.groupby(['session_id','level_group])

", "votes": null, "replies": [ { "id": 2172382, "author_name": "simonveitner", "author_url": "", "post_date": "03/07/2023 13:46:27", "content": "

yes you can also try to create specific features for level groups

", "votes": null, "replies": [] } ] } ] }, { "id": 2173303, "author_name": "sakvaua", "author_url": "", "post_date": "03/08/2023 09:38:20", "content": "

Time, hover_time, past answers

", "votes": null, "replies": [ { "id": 2173945, "author_name": "mengvision", "author_url": "", "post_date": "03/08/2023 18:15:44", "content": "

How to use post answers as features?

", "votes": null, "replies": [] }, { "id": 2174400, "author_name": "hoangnguyen719", "author_url": "", "post_date": "03/09/2023 05:21:23", "content": "

Hey Dennis, would you mind clarifying what you mean by \"post answers\"?

", "votes": null, "replies": [ { "id": 2174758, "author_name": "sakvaua", "author_url": "", "post_date": "03/09/2023 11:03:02", "content": "

Add previous answers that your model generated as features. Like when building a model for q2 use what your model for q1 predicted. For q3 use predictions for q1 and q2, and so on.

", "votes": null, "replies": [ { "id": 2174889, "author_name": "harrisonliang", "author_url": "", "post_date": "03/09/2023 13:40:39", "content": "

Wonderful Dennis, before I tried that on PyTorch Feed Forward NN and LB improved 0.001

", "votes": null, "replies": [] } ] }, { "id": 2174761, "author_name": "sakvaua", "author_url": "", "post_date": "03/09/2023 11:03:43", "content": "

Sorry, past, not post 😂

", "votes": null, "replies": [ { "id": 2175957, "author_name": "dannielscut", "author_url": "", "post_date": "03/10/2023 09:25:11", "content": "

Hey Dennis, would you mind sharing how many the CV and LB improve after adding the past answers features ? I added these features and find that these features improve a lot in my CV (even got 0.696 in CV) but do not bring too much improve in LB (without 0.001). It make me confused since I think the CV is correlated with the LB in this competiion when I generated other features before.

", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "2170584": "I am new to Kaggle, and from the discussion section I realized that it's very important to choose the proper features for this problem. Then which features do you think are the most important ones that can yield good performance?", "2170812": "Hi there, \n`event_name` is the most important feature. I got a lb 0.673 with only putting one-hot of `event_name`'s features into LightGBM.\nBUT, if you wanna got a better classifier, you should generate more features, maybe which seem not so important. The important features could not contain full information of the game, so we can find more features to complement it.\nNow I'm waiting the result of 293, 443, 522 features models, which got a cv of 0.690.", "2170825": "BTW, you can get important features of model from `LightGBM.feature_importance()` or `XGBClassifier.feature_importances_`.", "2172240": "Cool! Thank you for your advice! I am going to try LightGBM!", "2172253": "i think it's important to generate great variety of features to get a sense of what is important.", "2172376": "Yeah, I generate features from df.groupby(['session_id','level_group])", "2172382": "yes you can also try to create specific features for level groups", "2172925": "Another question, I noticed that you use a great number of features, on what device did you run your program, and how long does it take?", "2172996": "I just train these features with 30mins using GPU P100 in kaggle, but scoring takes 2hrs more.", "2173303": "Time, hover_time, past answers", "2173945": "How to use post answers as features?", "2174207": "Cool, may I ask you how do you get so many feature then?", "2174400": "Hey Dennis, would you mind clarifying what you mean by \"post answers\"?", "2174507": "Combine two or more features to renew features. In another word, find relationship between existing features and use this relationship as an new feature.", "2174758": "Add previous answers that your model generated as features. Like when building a model for q2 use what your model for q1 predicted. For q3 use predictions for q1 and q2, and so on.", "2174761": "Sorry, past, not post 😂", "2174886": "Thanks, that seems to be a popular method for generating features！", "2174889": "Wonderful Dennis, before I tried that on PyTorch Feed Forward NN and LB improved 0.001", "2174968": "Then how many features did you use in the end?", "2175610": "My best lb 0.695 use 747 1315 1603 features with cv 0.694. However, when I continue to increase features, it got not much improved for cv. Anyway, my best cv is 0.695 with 1792 3570 4338 features. But it's no use and you cann't infer so much features in 9 hrs.", "2175618": "By the way, @pourchot's [notebook](https://www.kaggle.com/code/pourchot/simple-xgb) is a good example for data processing.", "2175957": "Hey Dennis, would you mind sharing how many the CV and LB improve after adding the past answers features ? I added these features and find that these features improve a lot in my CV (even got 0.696 in CV) but do not bring too much improve in LB (without 0.001). It make me confused since I think the CV is correlated with the LB in this competiion when I generated other features before.", "2176901": "Nice! Thank you!" }, "source": "meta" }