Hi Everyone,
\nGrandMaster Chris Deotte had shared this trick during an interview about their team's RecSys Winning Solution, I believe one of these was taught to him by GrandMaster Giba so thanks to both of them for sharing this.
\nThe interview is over an hour so I have made two 3 minute clips where Chris spoke about these:
\n\nThis tutorial by Ryan explains really well why we apply encoding and applying smoothing to Target Encoding:
\nWhenever working with categorical variables, we need to process numbers inside or models so we figure out a way to \"encode\" this information. There are various ways of doing this. The \"trick\" here is applying to smooth and creating folds to avoid overfitting.
\nThe NVTabular Package provides a nice function that allows this. You can find it here and leverage the same on GPUs without having to implement it :)
\nThe trick is to create k-random folds and apply smoothing using mean of the values or another method.
\nIn the video above, we discussed the RecSys competition which had a somewhat similar split in data containing authors and readers in different columns. Visualised below:
\nEx: Table A has
\n| UserID | ColA |..... | Col Z|
Table B has:
\n| UserID | Col A1 | ... | Col Z1|
So here we can apply a Feature Engineering of Count Encoding and merge based on the same userid onto TableB.
\nChris has already shared a trick to reduce memory usage here
\nOn top of the same, in this competition case, for example-we can build features in one table of articles and customers and merge them together or onto transactions.
Once again, Thanks again Grandmaster Chris and Giba for sharing this with us, I have merely spent time understanding and then re-sharing it here.
\nI was quite new to these tricks and I didn't see them mentioned in this competition so I decided to post these.
\nHope this helps! :)
", "messages": [ { "id": "1716040", "postDate": "03/08/2022 15:45:32", "content": "Hi Everyone,
\nGrandMaster Chris Deotte had shared this trick during an interview about their team's RecSys Winning Solution, I believe one of these was taught to him by GrandMaster Giba so thanks to both of them for sharing this.
\nThe interview is over an hour so I have made two 3 minute clips where Chris spoke about these:
\n\nThis tutorial by Ryan explains really well why we apply encoding and applying smoothing to Target Encoding:
\nWhenever working with categorical variables, we need to process numbers inside or models so we figure out a way to \"encode\" this information. There are various ways of doing this. The \"trick\" here is applying to smooth and creating folds to avoid overfitting.
\nThe NVTabular Package provides a nice function that allows this. You can find it here and leverage the same on GPUs without having to implement it :)
\nThe trick is to create k-random folds and apply smoothing using mean of the values or another method.
\nIn the video above, we discussed the RecSys competition which had a somewhat similar split in data containing authors and readers in different columns. Visualised below:
\nEx: Table A has
\n| UserID | ColA |..... | Col Z|
Table B has:
\n| UserID | Col A1 | ... | Col Z1|
So here we can apply a Feature Engineering of Count Encoding and merge based on the same userid onto TableB.
\nChris has already shared a trick to reduce memory usage here
\nOn top of the same, in this competition case, for example-we can build features in one table of articles and customers and merge them together or onto transactions.
Once again, Thanks again Grandmaster Chris and Giba for sharing this with us, I have merely spent time understanding and then re-sharing it here.
\nI was quite new to these tricks and I didn't see them mentioned in this competition so I decided to post these.
\nHope this helps! :)
", "rawMarkdown": "Hi Everyone, \n\nGrandMaster [Chris Deotte](http://kaggle.com/cdeotte) had shared this trick during an interview about their team's RecSys Winning Solution, I believe one of these was taught to him by GrandMaster [Giba](http://kaggle.com/titericz) so thanks to both of them for sharing this. \n\nThe interview is over an hour so I have made two 3 minute clips where Chris spoke about these:\n\n- Target Encoding Trick [Video](https://youtu.be/OjxsAL45Lgg)\n- \"Reverse Target & Count Encoding\" Trick [Video](https://youtu.be/rX5Kj7iW9Ew)\n\n### Target Encoding Trick:\n\n[This tutorial](https://www.kaggle.com/ryanholbrook/target-encoding) by Ryan explains really well why we apply encoding and applying smoothing to Target Encoding:\n\nWhenever working with categorical variables, we need to process numbers inside or models so we figure out a way to \"encode\" this information. There are various ways of doing this. The \"trick\" here is applying to smooth and creating folds to avoid overfitting. \n\nThe NVTabular Package provides a nice function that allows this. You can find it [here](https://nvidia-merlin.github.io/NVTabular/v0.7.1/api/ops/targetencoding.html) and leverage the same on GPUs without having to implement it :)\n\nThe trick is to create k-random folds and apply smoothing using mean of the values or another method.\n\n### \"Reverse Target and Count Encoding\":\n\nIn the video above, we discussed the RecSys competition which had a somewhat similar split in data containing authors and readers in different columns. Visualised below:\n\nEx: Table A has \n\n` | UserID | ColA |..... | Col Z|`\n\nTable B has: \n\n` | UserID | Col A1 | ... | Col Z1|`\n\nSo here we can apply a Feature Engineering of Count Encoding and merge based on the same userid onto TableB. \n\n### Why is this powerful?\n\nChris has already shared a trick to reduce memory usage [here](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308635)\n\nOn top of the same, in this competition case, for example-we can build features in one table of `articles` and `customers` and merge them together or onto `transactions.`\n\nOnce again, Thanks again Grandmaster Chris and Giba for sharing this with us, I have merely spent time understanding and then re-sharing it here. \n\nI was quite new to these tricks and I didn't see them mentioned in this competition so I decided to post these.\n\nHope this helps! :)", "votes": null }, { "id": "1716141", "postDate": "03/08/2022 17:21:41", "content": "Hi @init27, thank you for sharing.
", "rawMarkdown": "Hi @init27, thank you for sharing.", "votes": null }, { "id": "1716379", "postDate": "03/09/2022 00:15:32", "content": "Thanks for sharing Sanyam. I will say that my current solution uses \"TE\" and/or \"reverse TE\" in the following sense. For each customer and/or item, we can create new features by using groupby('customer_id') or groupby('article_id') and then aggregating some statistic. Using this information we can either add new candidates to recommend or rerank recommendations that were found using a previous algorithm.
Remember the general approach to this competition is (1) find 12 or 24 or 36 candidates to recommend each customer (2) rerank these recommendations arranging the most likely (to be purchased) first and least likely last. So there are two steps (1) find candidates (2) rearrange those candidates
", "rawMarkdown": "Thanks for sharing Sanyam. I will say that my current solution uses \"TE\" and/or \"reverse TE\" in the following sense. For each customer and/or item, we can create new features by using `groupby('customer_id')` or `groupby('article_id')` and then aggregating some statistic. Using this information we can either add new candidates to recommend or rerank recommendations that were found using a previous algorithm.\n\nRemember the general approach to this competition is (1) find 12 or 24 or 36 candidates to recommend each customer (2) rerank these recommendations arranging the most likely (to be purchased) first and least likely last. So there are two steps (1) find candidates (2) rearrange those candidates", "votes": null }, { "id": "1716699", "postDate": "03/09/2022 09:58:47", "content": "@cdeotte Hi Chris, how do we ensure the ranking as there are multiple entries of same article which also indicate a return of these articles.
\nor simply put
\nhow do we apply LTR(learning to rank) technique here
There are many ways to rerank. We can use heuristics or we can train a reranking model. The model/heuristic can be general or specific to the customer and/or item.
\nHere is one heuristic example. Imagine that we are simply recommending every customer the top 24 most popular items. Next we can loop through every customer. If the customer gender is male, we can move the female items lower in the recommendation rank (for that specific customer). If the customer gender is female, we can move the male items lower in the recommendation rank.
\nWe begin with 24 candidates per customer. Then we re-ranked them. Finally we keep only the top 12 and make a submission to Kaggle. Using just top 12 gets one score, but using top 24 then reranking then picking top 12 afterward gets a better score.
", "rawMarkdown": "There are many ways to rerank. We can use heuristics or we can train a reranking model. The model/heuristic can be general or specific to the customer and/or item.\n\nHere is one heuristic example. Imagine that we are simply recommending every customer the top 24 most popular items. Next we can loop through every customer. If the customer gender is male, we can move the female items lower in the recommendation rank (for that specific customer). If the customer gender is female, we can move the male items lower in the recommendation rank. \n\nWe begin with 24 candidates per customer. Then we re-ranked them. Finally we keep only the top 12 and make a submission to Kaggle. Using just top 12 gets one score, but using top 24 then reranking then picking top 12 afterward gets a better score.", "votes": null }, { "id": "1716956", "postDate": "03/09/2022 14:35:31", "content": "Bala, please thank the Grandmasters that taught me these tricks :)
", "rawMarkdown": "Bala, please thank the Grandmasters that taught me these tricks :)", "votes": null }, { "id": "1716981", "postDate": "03/09/2022 14:55:07", "content": "Pawel describes the procedure in more detail here
", "rawMarkdown": "Pawel describes the procedure in more detail [here][1]\n\n[1]: https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/307288", "votes": null }, { "id": "1716999", "postDate": "03/09/2022 15:17:58", "content": "Thanks alot @cdeotte. will try to implement them.
", "rawMarkdown": "Thanks alot @cdeotte. will try to implement them.", "votes": null }, { "id": "1717515", "postDate": "03/10/2022 01:54:07", "content": "(In response to top comment)
\nThank you for the reply, Chris!
\nI will share my embarassing approach to understanding, incase its helpful for anyone else-I had asked the similar question to Chris around how this is applied in the Memory trick thread but then on paper after drawing and it didn't make sense to me still.
\nI was still thinking if we merge all tables, maybe the trick won't apply here because then we have 1 final df.
After wasting silly hours and then finally firing up a jupyter nb, I realised if we merge rows, there always isn't a 1:1 mapping and we can use groupby().
Lesson learned: Code first and draw on paper when stuck not the other way around to save time
", "rawMarkdown": "(In response to top comment) \n\nThank you for the reply, Chris! \n\nI will share my embarassing approach to understanding, incase its helpful for anyone else-I had asked the similar question to Chris around how this is applied in the Memory trick thread but then on paper after drawing and it didn't make sense to me still. \n\nI was still thinking if we merge all tables, maybe the trick won't apply here because then we have 1 final `df`. \n\nAfter wasting silly hours and then finally firing up a jupyter nb, I realised if we merge rows, there always isn't a 1:1 mapping and we can use `groupby()`. \n\nLesson learned: Code first and draw on paper when stuck not the other way around to save time", "votes": null }, { "id": "1721578", "postDate": "03/13/2022 19:57:47", "content": "@cdeotte I just want to make sure I undestand it correctly. If I want to make target encoding for each of the user combined with each of the products as a dynamic feature for ranking, without smoothing the target encoding would be number of times article appeared in a customer baskets and divided by the number of other articles that ever appeared in the customer basket right (in specified time range)? thank you for sharing your knowledge
", "rawMarkdown": "cdeotte I just want to make sure I undestand it correctly. If I want to make target encoding for each of the user combined with each of the products as a dynamic feature for ranking, without smoothing the target encoding would be number of times article appeared in a customer baskets and divided by the number of other articles that ever appeared in the customer basket right (in specified time range)? thank you for sharing your knowledge", "votes": null } ], "comments": [ { "id": 1716141, "author_name": "balabaskar", "author_url": "", "post_date": "03/08/2022 17:21:41", "content": "Hi @init27, thank you for sharing.
", "votes": null, "replies": [ { "id": 1716956, "author_name": "init27", "author_url": "", "post_date": "03/09/2022 14:35:31", "content": "Bala, please thank the Grandmasters that taught me these tricks :)
", "votes": null, "replies": [] } ] }, { "id": 1716379, "author_name": "cdeotte", "author_url": "", "post_date": "03/09/2022 00:15:32", "content": "Thanks for sharing Sanyam. I will say that my current solution uses \"TE\" and/or \"reverse TE\" in the following sense. For each customer and/or item, we can create new features by using groupby('customer_id') or groupby('article_id') and then aggregating some statistic. Using this information we can either add new candidates to recommend or rerank recommendations that were found using a previous algorithm.
Remember the general approach to this competition is (1) find 12 or 24 or 36 candidates to recommend each customer (2) rerank these recommendations arranging the most likely (to be purchased) first and least likely last. So there are two steps (1) find candidates (2) rearrange those candidates
", "votes": null, "replies": [ { "id": 1716699, "author_name": "vjagannath786", "author_url": "", "post_date": "03/09/2022 09:58:47", "content": "@cdeotte Hi Chris, how do we ensure the ranking as there are multiple entries of same article which also indicate a return of these articles.
\nor simply put
\nhow do we apply LTR(learning to rank) technique here
There are many ways to rerank. We can use heuristics or we can train a reranking model. The model/heuristic can be general or specific to the customer and/or item.
\nHere is one heuristic example. Imagine that we are simply recommending every customer the top 24 most popular items. Next we can loop through every customer. If the customer gender is male, we can move the female items lower in the recommendation rank (for that specific customer). If the customer gender is female, we can move the male items lower in the recommendation rank.
\nWe begin with 24 candidates per customer. Then we re-ranked them. Finally we keep only the top 12 and make a submission to Kaggle. Using just top 12 gets one score, but using top 24 then reranking then picking top 12 afterward gets a better score.
", "votes": null, "replies": [] }, { "id": 1716981, "author_name": "cdeotte", "author_url": "", "post_date": "03/09/2022 14:55:07", "content": "Pawel describes the procedure in more detail here
", "votes": null, "replies": [] }, { "id": 1716999, "author_name": "vjagannath786", "author_url": "", "post_date": "03/09/2022 15:17:58", "content": "Thanks alot @cdeotte. will try to implement them.
", "votes": null, "replies": [] }, { "id": 1717515, "author_name": "init27", "author_url": "", "post_date": "03/10/2022 01:54:07", "content": "(In response to top comment)
\nThank you for the reply, Chris!
\nI will share my embarassing approach to understanding, incase its helpful for anyone else-I had asked the similar question to Chris around how this is applied in the Memory trick thread but then on paper after drawing and it didn't make sense to me still.
\nI was still thinking if we merge all tables, maybe the trick won't apply here because then we have 1 final df.
After wasting silly hours and then finally firing up a jupyter nb, I realised if we merge rows, there always isn't a 1:1 mapping and we can use groupby().
Lesson learned: Code first and draw on paper when stuck not the other way around to save time
", "votes": null, "replies": [] }, { "id": 1721578, "author_name": "duuuscha", "author_url": "", "post_date": "03/13/2022 19:57:47", "content": "@cdeotte I just want to make sure I undestand it correctly. If I want to make target encoding for each of the user combined with each of the products as a dynamic feature for ranking, without smoothing the target encoding would be number of times article appeared in a customer baskets and divided by the number of other articles that ever appeared in the customer basket right (in specified time range)? thank you for sharing your knowledge
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1716040": "Hi Everyone, \n\nGrandMaster [Chris Deotte](http://kaggle.com/cdeotte) had shared this trick during an interview about their team's RecSys Winning Solution, I believe one of these was taught to him by GrandMaster [Giba](http://kaggle.com/titericz) so thanks to both of them for sharing this. \n\nThe interview is over an hour so I have made two 3 minute clips where Chris spoke about these:\n\n- Target Encoding Trick [Video](https://youtu.be/OjxsAL45Lgg)\n- \"Reverse Target & Count Encoding\" Trick [Video](https://youtu.be/rX5Kj7iW9Ew)\n\n### Target Encoding Trick:\n\n[This tutorial](https://www.kaggle.com/ryanholbrook/target-encoding) by Ryan explains really well why we apply encoding and applying smoothing to Target Encoding:\n\nWhenever working with categorical variables, we need to process numbers inside or models so we figure out a way to \"encode\" this information. There are various ways of doing this. The \"trick\" here is applying to smooth and creating folds to avoid overfitting. \n\nThe NVTabular Package provides a nice function that allows this. You can find it [here](https://nvidia-merlin.github.io/NVTabular/v0.7.1/api/ops/targetencoding.html) and leverage the same on GPUs without having to implement it :)\n\nThe trick is to create k-random folds and apply smoothing using mean of the values or another method.\n\n### \"Reverse Target and Count Encoding\":\n\nIn the video above, we discussed the RecSys competition which had a somewhat similar split in data containing authors and readers in different columns. Visualised below:\n\nEx: Table A has \n\n` | UserID | ColA |..... | Col Z|`\n\nTable B has: \n\n` | UserID | Col A1 | ... | Col Z1|`\n\nSo here we can apply a Feature Engineering of Count Encoding and merge based on the same userid onto TableB. \n\n### Why is this powerful?\n\nChris has already shared a trick to reduce memory usage [here](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308635)\n\nOn top of the same, in this competition case, for example-we can build features in one table of `articles` and `customers` and merge them together or onto `transactions.`\n\nOnce again, Thanks again Grandmaster Chris and Giba for sharing this with us, I have merely spent time understanding and then re-sharing it here. \n\nI was quite new to these tricks and I didn't see them mentioned in this competition so I decided to post these.\n\nHope this helps! :)", "1716141": "Hi @init27, thank you for sharing.", "1716379": "Thanks for sharing Sanyam. I will say that my current solution uses \"TE\" and/or \"reverse TE\" in the following sense. For each customer and/or item, we can create new features by using `groupby('customer_id')` or `groupby('article_id')` and then aggregating some statistic. Using this information we can either add new candidates to recommend or rerank recommendations that were found using a previous algorithm.\n\nRemember the general approach to this competition is (1) find 12 or 24 or 36 candidates to recommend each customer (2) rerank these recommendations arranging the most likely (to be purchased) first and least likely last. So there are two steps (1) find candidates (2) rearrange those candidates", "1716699": "cdeotte Hi Chris, how do we ensure the ranking as there are multiple entries of same article which also indicate a return of these articles.\nor simply put \nhow do we apply LTR(learning to rank) technique here", "1716936": "There are many ways to rerank. We can use heuristics or we can train a reranking model. The model/heuristic can be general or specific to the customer and/or item.\n\nHere is one heuristic example. Imagine that we are simply recommending every customer the top 24 most popular items. Next we can loop through every customer. If the customer gender is male, we can move the female items lower in the recommendation rank (for that specific customer). If the customer gender is female, we can move the male items lower in the recommendation rank. \n\nWe begin with 24 candidates per customer. Then we re-ranked them. Finally we keep only the top 12 and make a submission to Kaggle. Using just top 12 gets one score, but using top 24 then reranking then picking top 12 afterward gets a better score.", "1716956": "Bala, please thank the Grandmasters that taught me these tricks :)", "1716981": "Pawel describes the procedure in more detail [here][1]\n\n[1]: https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/307288", "1716999": "Thanks alot @cdeotte. will try to implement them.", "1717515": "(In response to top comment) \n\nThank you for the reply, Chris! \n\nI will share my embarassing approach to understanding, incase its helpful for anyone else-I had asked the similar question to Chris around how this is applied in the Memory trick thread but then on paper after drawing and it didn't make sense to me still. \n\nI was still thinking if we merge all tables, maybe the trick won't apply here because then we have 1 final `df`. \n\nAfter wasting silly hours and then finally firing up a jupyter nb, I realised if we merge rows, there always isn't a 1:1 mapping and we can use `groupby()`. \n\nLesson learned: Code first and draw on paper when stuck not the other way around to save time", "1721578": "cdeotte I just want to make sure I undestand it correctly. If I want to make target encoding for each of the user combined with each of the products as a dynamic feature for ranking, without smoothing the target encoding would be number of times article appeared in a customer baskets and divided by the number of other articles that ever appeared in the customer basket right (in specified time range)? thank you for sharing your knowledge" }, "source": "meta" }