{ "id": 55521, "title": "Sample categorical feature encoding methods", "url": "/competitions/avito-demand-prediction/discussion/55521", "author_name": "", "post_date": "2018-04-27T20:30:55.543182200Z", "votes": 18, "comment_count": 15, "views": 0, "content": "

Hi, mates:

\n\n

I know some categorical feature encoding methods as following, these maybe help beginers get a higher score:

\n\n\n\n

I used factorize and frequency encoding like following code in this competition:

\n\n
for c in tqdm(categorical_columns):\n    daset[c+'_freq'] = daset[c].map(daset.groupby(c).size() / daset.shape[0])\n    indexer = pd.factorize(daset[c], sort=True)[1]\n    daset[c] = indexer.get_indexer(daset[c])\n
\n\n

Happy kaggle.

\n\n

Would you please share other trick method for categorical feature engineering ?

\n\n

Thanks

", "messages": [ { "id": "320194", "postDate": "04/27/2018 20:30:55", "content": "

Hi, mates:

\n\n

I know some categorical feature encoding methods as following, these maybe help beginers get a higher score:

\n\n\n\n

I used factorize and frequency encoding like following code in this competition:

\n\n
for c in tqdm(categorical_columns):\n    daset[c+'_freq'] = daset[c].map(daset.groupby(c).size() / daset.shape[0])\n    indexer = pd.factorize(daset[c], sort=True)[1]\n    daset[c] = indexer.get_indexer(daset[c])\n
\n\n

Happy kaggle.

\n\n

Would you please share other trick method for categorical feature engineering ?

\n\n

Thanks

", "rawMarkdown": "Hi, mates:\n\nI know some categorical feature encoding methods as following, these maybe help beginers get a higher score:\n\n- one-hot (benefit for linear models or NN)\n- label-encode\n- mean-encoding (be careful , easy over fitting if you have not a right validation strategy)\n- factorize-encoding\n- frequency-encoding\n\nI used factorize and frequency encoding like following code in this competition:\n\n for c in tqdm(categorical_columns):\n daset[c+'_freq'] = daset[c].map(daset.groupby(c).size() / daset.shape[0])\n indexer = pd.factorize(daset[c], sort=True)[1]\n daset[c] = indexer.get_indexer(daset[c])\n\nHappy kaggle.\n\nWould you please share other trick method for categorical feature engineering ?\n\nThanks", "votes": null }, { "id": "320203", "postDate": "04/27/2018 21:12:53", "content": "

For mean encoding that reduces risk of overfitting, I recommend Oliver's Target Encoding kernel from the Porto Seguro competition.

", "rawMarkdown": "For mean encoding that reduces risk of overfitting, I recommend [Oliver's Target Encoding kernel](https://www.kaggle.com/ogrellier/python-target-encoding-for-categorical-features) from the Porto Seguro competition.", "votes": null }, { "id": "320211", "postDate": "04/27/2018 21:39:25", "content": "

What's the difference between factorize encoding and label encoding?

", "rawMarkdown": "What's the difference between factorize encoding and label encoding?", "votes": null }, { "id": "324736", "postDate": "05/07/2018 23:05:29", "content": "

Upon further reflection, for LightGBM, it looks like simply using the built-in categorical encoding is outperforming any kind of categorical encoding I can personally do.

", "rawMarkdown": "Upon further reflection, for LightGBM, it looks like simply using the [built-in categorical encoding](https://github.com/Microsoft/LightGBM/blob/master/docs/Features.rst#optimal-split-for-categorical-features) is outperforming any kind of categorical encoding I can personally do.", "votes": null }, { "id": "324895", "postDate": "05/08/2018 01:23:07", "content": "

Factorize can catch some frequency feature of categorical feature,but label encoding just assign a number to category

", "rawMarkdown": "Factorize can catch some frequency feature of categorical feature,but label encoding just assign a number to category", "votes": null }, { "id": "324908", "postDate": "05/08/2018 01:29:05", "content": "

Yes, thank you Peter. Let me know more clear about how to handle categorical features using target encoding

", "rawMarkdown": "Yes, thank you Peter. Let me know more clear about how to handle categorical features using target encoding", "votes": null }, { "id": "324909", "postDate": "05/08/2018 01:30:51", "content": "

Thank you Peter let me know lightgbm how to handle categorical features

", "rawMarkdown": "Thank you Peter let me know lightgbm how to handle categorical features", "votes": null }, { "id": "324912", "postDate": "05/08/2018 01:32:12", "content": "

Maybe some important lightgbm parameters will tuning when we pass categorical features to it

", "rawMarkdown": "Maybe some important lightgbm parameters will tuning when we pass categorical features to it", "votes": null }, { "id": "324914", "postDate": "05/08/2018 01:34:08", "content": "

BTW. we should take care about overfitting when we working on date series data.

", "rawMarkdown": "BTW. we should take care about overfitting when we working on date series data.", "votes": null }, { "id": "326500", "postDate": "05/09/2018 20:23:23", "content": "

Other option is Cat2Vec

", "rawMarkdown": "Other option is [Cat2Vec][1]\n\n\n [1]: https://openreview.net/pdf?id=HyNxRZ9xg", "votes": null }, { "id": "326799", "postDate": "05/10/2018 11:06:55", "content": "

I also implemented baysian encoding.

\n\n

See paper here

", "rawMarkdown": "I also implemented baysian encoding. \n\n[See paper here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/225952/7441/high%20cardinality%20categoricals.pdf", "votes": null }, { "id": "327026", "postDate": "05/10/2018 17:40:55", "content": "

Thanks @Dieter

", "rawMarkdown": "Thanks @Dieter", "votes": null }, { "id": "327027", "postDate": "05/10/2018 17:41:25", "content": "

Yes, cat2vec is very powfull in my model.

", "rawMarkdown": "Yes, cat2vec is very powfull in my model.", "votes": null }, { "id": "524744", "postDate": "04/29/2019 13:03:37", "content": "

Thanks I thaught there is method only like le or ohe.

", "rawMarkdown": "Thanks I thaught there is method only like le or ohe.", "votes": null }, { "id": "651645", "postDate": "10/17/2019 18:27:10", "content": "

Thanks for sharing.

", "rawMarkdown": "Thanks for sharing.", "votes": null }, { "id": "793449", "postDate": "04/01/2020 03:18:24", "content": "

Thanks for sharing!

", "rawMarkdown": "Thanks for sharing!", "votes": null } ], "comments": [ { "id": 320203, "author_name": "peterhurford", "author_url": "", "post_date": "04/27/2018 21:12:53", "content": "

For mean encoding that reduces risk of overfitting, I recommend Oliver's Target Encoding kernel from the Porto Seguro competition.

", "votes": null, "replies": [ { "id": 324908, "author_name": "classtag", "author_url": "", "post_date": "05/08/2018 01:29:05", "content": "

Yes, thank you Peter. Let me know more clear about how to handle categorical features using target encoding

", "votes": null, "replies": [] }, { "id": 324914, "author_name": "classtag", "author_url": "", "post_date": "05/08/2018 01:34:08", "content": "

BTW. we should take care about overfitting when we working on date series data.

", "votes": null, "replies": [] } ] }, { "id": 320211, "author_name": "peterhurford", "author_url": "", "post_date": "04/27/2018 21:39:25", "content": "

What's the difference between factorize encoding and label encoding?

", "votes": null, "replies": [ { "id": 324895, "author_name": "classtag", "author_url": "", "post_date": "05/08/2018 01:23:07", "content": "

Factorize can catch some frequency feature of categorical feature,but label encoding just assign a number to category

", "votes": null, "replies": [] } ] }, { "id": 324736, "author_name": "peterhurford", "author_url": "", "post_date": "05/07/2018 23:05:29", "content": "

Upon further reflection, for LightGBM, it looks like simply using the built-in categorical encoding is outperforming any kind of categorical encoding I can personally do.

", "votes": null, "replies": [ { "id": 324909, "author_name": "classtag", "author_url": "", "post_date": "05/08/2018 01:30:51", "content": "

Thank you Peter let me know lightgbm how to handle categorical features

", "votes": null, "replies": [] }, { "id": 324912, "author_name": "classtag", "author_url": "", "post_date": "05/08/2018 01:32:12", "content": "

Maybe some important lightgbm parameters will tuning when we pass categorical features to it

", "votes": null, "replies": [] } ] }, { "id": 326500, "author_name": "legorreta", "author_url": "", "post_date": "05/09/2018 20:23:23", "content": "

Other option is Cat2Vec

", "votes": null, "replies": [ { "id": 327027, "author_name": "classtag", "author_url": "", "post_date": "05/10/2018 17:41:25", "content": "

Yes, cat2vec is very powfull in my model.

", "votes": null, "replies": [] } ] }, { "id": 326799, "author_name": "christofhenkel", "author_url": "", "post_date": "05/10/2018 11:06:55", "content": "

I also implemented baysian encoding.

\n\n

See paper here

", "votes": null, "replies": [ { "id": 327026, "author_name": "classtag", "author_url": "", "post_date": "05/10/2018 17:40:55", "content": "

Thanks @Dieter

", "votes": null, "replies": [] } ] }, { "id": 524744, "author_name": "smohubal", "author_url": "", "post_date": "04/29/2019 13:03:37", "content": "

Thanks I thaught there is method only like le or ohe.

", "votes": null, "replies": [] }, { "id": 651645, "author_name": "prakharm21", "author_url": "", "post_date": "10/17/2019 18:27:10", "content": "

Thanks for sharing.

", "votes": null, "replies": [] }, { "id": 793449, "author_name": "davidhopezhao", "author_url": "", "post_date": "04/01/2020 03:18:24", "content": "

Thanks for sharing!

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "320194": "Hi, mates:\n\nI know some categorical feature encoding methods as following, these maybe help beginers get a higher score:\n\n- one-hot (benefit for linear models or NN)\n- label-encode\n- mean-encoding (be careful , easy over fitting if you have not a right validation strategy)\n- factorize-encoding\n- frequency-encoding\n\nI used factorize and frequency encoding like following code in this competition:\n\n for c in tqdm(categorical_columns):\n daset[c+'_freq'] = daset[c].map(daset.groupby(c).size() / daset.shape[0])\n indexer = pd.factorize(daset[c], sort=True)[1]\n daset[c] = indexer.get_indexer(daset[c])\n\nHappy kaggle.\n\nWould you please share other trick method for categorical feature engineering ?\n\nThanks", "320203": "For mean encoding that reduces risk of overfitting, I recommend [Oliver's Target Encoding kernel](https://www.kaggle.com/ogrellier/python-target-encoding-for-categorical-features) from the Porto Seguro competition.", "320211": "What's the difference between factorize encoding and label encoding?", "324736": "Upon further reflection, for LightGBM, it looks like simply using the [built-in categorical encoding](https://github.com/Microsoft/LightGBM/blob/master/docs/Features.rst#optimal-split-for-categorical-features) is outperforming any kind of categorical encoding I can personally do.", "324895": "Factorize can catch some frequency feature of categorical feature,but label encoding just assign a number to category", "324908": "Yes, thank you Peter. Let me know more clear about how to handle categorical features using target encoding", "324909": "Thank you Peter let me know lightgbm how to handle categorical features", "324912": "Maybe some important lightgbm parameters will tuning when we pass categorical features to it", "324914": "BTW. we should take care about overfitting when we working on date series data.", "326500": "Other option is [Cat2Vec][1]\n\n\n [1]: https://openreview.net/pdf?id=HyNxRZ9xg", "326799": "I also implemented baysian encoding. \n\n[See paper here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/225952/7441/high%20cardinality%20categoricals.pdf", "327026": "Thanks @Dieter", "327027": "Yes, cat2vec is very powfull in my model.", "524744": "Thanks I thaught there is method only like le or ohe.", "651645": "Thanks for sharing.", "793449": "Thanks for sharing!" }, "source": "meta" }