{ "id": 330381, "title": "How to do feature engineering and adjust parameters?", "url": "/competitions/amex-default-prediction/discussion/330381", "author_name": "", "post_date": "2022-06-12T02:44:13.932499900Z", "votes": 3, "comment_count": 6, "views": 0, "content": "

Hey guy, Can you share some insights about feature engineering and adjust parameters in your train models?
\nl saw some tips about feature engineering like:
\nnum features will be enlarge to mean, std, min, max, lastโ€ฆ
\ncat features will be enlarge to unique, count, lastโ€ฆ

\n

Do you have some margin ways to deal will feature engineering?

\n

Do you select model's parameters by your experience or your train?

\n

l hope your guys can share some insights and thanks a lot~~~

", "messages": [ { "id": "1817905", "postDate": "06/12/2022 02:44:13", "content": "

Hey guy, Can you share some insights about feature engineering and adjust parameters in your train models?
\nl saw some tips about feature engineering like:
\nnum features will be enlarge to mean, std, min, max, lastโ€ฆ
\ncat features will be enlarge to unique, count, lastโ€ฆ

\n

Do you have some margin ways to deal will feature engineering?

\n

Do you select model's parameters by your experience or your train?

\n

l hope your guys can share some insights and thanks a lot~~~

", "rawMarkdown": "Hey guy, Can you share some insights about feature engineering and adjust parameters in your train models?\nl saw some tips about feature engineering like:\nnum features will be enlarge to mean, std, min, max, last...\ncat features will be enlarge to unique, count, last...\n\nDo you have some margin ways to deal will feature engineering? \n\nDo you select model's parameters by your experience or your train?\n\nl hope your guys can share some insights and thanks a lot~~~", "votes": null }, { "id": "1817957", "postDate": "06/12/2022 04:26:20", "content": "

Most efficient way to find the right parameters for your model is Hyperparameter Tuning, but they can take quite a lot of time find them for you. So you might want to explore different algorithms to do so. For sklearn you can find bunch of them here

\n

Feature engineering is quite subjective to the task you are working with, but you could do something like:

\n", "rawMarkdown": "Most efficient way to find the right parameters for your model is *Hyperparameter Tuning*, but they can take quite a lot of time find them for you. So you might want to explore different algorithms to do so. For sklearn you can find bunch of them [here](https://scikit-learn.org/stable/modules/grid_search.html#tuning-the-hyper-parameters-of-an-estimator)\n\nFeature engineering is quite subjective to the task you are working with, but you could do something like:\n- Imputation: Where you fill the missing values in your dataset. There are lot's of methods to it!\n- Categorical Encoding: This is also important because you can't just *Hot-Encode* or *Label-Encode* everything. (Read more [here](https://datascience.stackexchange.com/questions/9443/when-to-use-one-hot-encoding-vs-labelencoder-vs-dictvectorizor))\n- Handling outliers\n- Creating new features (PCA can help with this)\n- etc.\nI learned a lot of *Feature Engineering* techniques by reading people's notebooks.", "votes": null }, { "id": "1818026", "postDate": "06/12/2022 06:32:11", "content": "

Hi, how to use PCA to create new features?

", "rawMarkdown": "Hi, how to use PCA to create new features?", "votes": null }, { "id": "1818087", "postDate": "06/12/2022 08:22:20", "content": "

This is good question! And l saw some answers from here
\n(from Keivan loch Hagh share~).

\n

Using PCA because of curse of dimensionality by one-hot Encoding.

\n

So, when you use one-hot Encoding or your dataset have too much features, maybe you can take PCA in your consideration.

", "rawMarkdown": "This is good question! And l saw some answers from [here](https://datascience.stackexchange.com/questions/9443/when-to-use-one-hot-encoding-vs-labelencoder-vs-dictvectorizor/40908#40908?newreg=76dcabdedb1e4ac1beac4fa7e989f43)\n(from *Keivan loch Hagh* share~).\n\nUsing PCA because of curse of dimensionality by *one-hot Encoding.*\n\nSo, when you use *one-hot Encoding* or your dataset have too much features, maybe you can take PCA in your consideration.", "votes": null }, { "id": "1818089", "postDate": "06/12/2022 08:24:37", "content": "

l learn much in your sharing~ Thanks a lot๐Ÿ‘

\n

Is tree models such as Gradient Boosted Decision Trees and Random Forests have good functions to deal with original categorical features? And Do l need to use one-hot Encoding to deal with categorical features?

", "rawMarkdown": "l learn much in your sharing~ Thanks a lot๐Ÿ‘\n\nIs tree models such as *Gradient Boosted Decision Trees* and *Random Forests* have good functions to deal with original categorical features? And Do l need to use *one-hot Encoding* to deal with categorical features?", "votes": null }, { "id": "1818241", "postDate": "06/12/2022 12:23:59", "content": "

I have to say that I haven't modified any hyperparameters.
\nRegardless of the model, the default parameters are good enough~ (Just for now) ๐Ÿ˜†

", "rawMarkdown": "I have to say that I haven't modified any hyperparameters.\nRegardless of the model, the default parameters are good enough~ (Just for now) ๐Ÿ˜†", "votes": null }, { "id": "1818617", "postDate": "06/13/2022 01:14:41", "content": "

thanks a lot~
\nAnd what about feature engineering?๐Ÿ‘€

", "rawMarkdown": "thanks a lot~\nAnd what about feature engineering?๐Ÿ‘€", "votes": null } ], "comments": [ { "id": 1817957, "author_name": "keivanipchihagh", "author_url": "", "post_date": "06/12/2022 04:26:20", "content": "

Most efficient way to find the right parameters for your model is Hyperparameter Tuning, but they can take quite a lot of time find them for you. So you might want to explore different algorithms to do so. For sklearn you can find bunch of them here

\n

Feature engineering is quite subjective to the task you are working with, but you could do something like:

\n", "votes": null, "replies": [ { "id": 1818026, "author_name": "denisplaj", "author_url": "", "post_date": "06/12/2022 06:32:11", "content": "

Hi, how to use PCA to create new features?

", "votes": null, "replies": [] }, { "id": 1818087, "author_name": "shanggangli", "author_url": "", "post_date": "06/12/2022 08:22:20", "content": "

This is good question! And l saw some answers from here
\n(from Keivan loch Hagh share~).

\n

Using PCA because of curse of dimensionality by one-hot Encoding.

\n

So, when you use one-hot Encoding or your dataset have too much features, maybe you can take PCA in your consideration.

", "votes": null, "replies": [] }, { "id": 1818089, "author_name": "shanggangli", "author_url": "", "post_date": "06/12/2022 08:24:37", "content": "

l learn much in your sharing~ Thanks a lot๐Ÿ‘

\n

Is tree models such as Gradient Boosted Decision Trees and Random Forests have good functions to deal with original categorical features? And Do l need to use one-hot Encoding to deal with categorical features?

", "votes": null, "replies": [] } ] }, { "id": 1818241, "author_name": "xiaowangiiiii", "author_url": "", "post_date": "06/12/2022 12:23:59", "content": "

I have to say that I haven't modified any hyperparameters.
\nRegardless of the model, the default parameters are good enough~ (Just for now) ๐Ÿ˜†

", "votes": null, "replies": [ { "id": 1818617, "author_name": "shanggangli", "author_url": "", "post_date": "06/13/2022 01:14:41", "content": "

thanks a lot~
\nAnd what about feature engineering?๐Ÿ‘€

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1817905": "Hey guy, Can you share some insights about feature engineering and adjust parameters in your train models?\nl saw some tips about feature engineering like:\nnum features will be enlarge to mean, std, min, max, last...\ncat features will be enlarge to unique, count, last...\n\nDo you have some margin ways to deal will feature engineering? \n\nDo you select model's parameters by your experience or your train?\n\nl hope your guys can share some insights and thanks a lot~~~", "1817957": "Most efficient way to find the right parameters for your model is *Hyperparameter Tuning*, but they can take quite a lot of time find them for you. So you might want to explore different algorithms to do so. For sklearn you can find bunch of them [here](https://scikit-learn.org/stable/modules/grid_search.html#tuning-the-hyper-parameters-of-an-estimator)\n\nFeature engineering is quite subjective to the task you are working with, but you could do something like:\n- Imputation: Where you fill the missing values in your dataset. There are lot's of methods to it!\n- Categorical Encoding: This is also important because you can't just *Hot-Encode* or *Label-Encode* everything. (Read more [here](https://datascience.stackexchange.com/questions/9443/when-to-use-one-hot-encoding-vs-labelencoder-vs-dictvectorizor))\n- Handling outliers\n- Creating new features (PCA can help with this)\n- etc.\nI learned a lot of *Feature Engineering* techniques by reading people's notebooks.", "1818026": "Hi, how to use PCA to create new features?", "1818087": "This is good question! And l saw some answers from [here](https://datascience.stackexchange.com/questions/9443/when-to-use-one-hot-encoding-vs-labelencoder-vs-dictvectorizor/40908#40908?newreg=76dcabdedb1e4ac1beac4fa7e989f43)\n(from *Keivan loch Hagh* share~).\n\nUsing PCA because of curse of dimensionality by *one-hot Encoding.*\n\nSo, when you use *one-hot Encoding* or your dataset have too much features, maybe you can take PCA in your consideration.", "1818089": "l learn much in your sharing~ Thanks a lot๐Ÿ‘\n\nIs tree models such as *Gradient Boosted Decision Trees* and *Random Forests* have good functions to deal with original categorical features? And Do l need to use *one-hot Encoding* to deal with categorical features?", "1818241": "I have to say that I haven't modified any hyperparameters.\nRegardless of the model, the default parameters are good enough~ (Just for now) ๐Ÿ˜†", "1818617": "thanks a lot~\nAnd what about feature engineering?๐Ÿ‘€" }, "source": "meta" }