\n
![\"arch.png\"](\"https://i.postimg.cc/CMWhdC2L/arch.png\")
Thanks to H&M and Kaggle Team for organizing such a great recommendation competition.
\nThanks @paweljankiewicz, @titericz, @cdeotte and many Kagglers for their great discussions and notebooks.
\nI hope to get a solo gold metal this time, so I decided to do this competition by myself in the beginning.
I followed the same pipeline as @paweljankiewicz suggested in Addressing common questions and what the competition is really about . First, generate candidates using different recall models for each customers. Second, build a ranking model that ranks the candidates within customer.
\n
I mainly followed How To Setup Local CV, I created 5 folds and used 3 folds as training data. I also tried more training data, but it didn’t increase the local CV and LB.
\nI developed 4 recall models in total.
\nI used a gating network to make sure that the user tower can learn well by using different experts for recent active customers and non active customers.
\n![\"user-tower.png\"](\"https://i.postimg.cc/j54HCdxR/user-tower.png\")
\n![\"item-tower.png\"](\"https://i.postimg.cc/RVNtxTHS/item-tower.png\")
\n![\"loss.png\"](\"https://i.postimg.cc/wMP7jSm6/loss.png\")
I mainly used lightgbm with lambda-rank objective. I also implement DCN model described in https://arxiv.org/abs/2008.13535. But I didn’t have enough time to tune it. The score of lightgbm is much better.
\nTwo Tower MMoE can also generate candidates using user demographic features for customers without purchase log.
\nBesides product meta features, I also extracted the following features from text and image.
\nbest local CV is 0.039, LB is 0.0349.
\nMy final submission combined my previous submissions by
\nh-m-ensembling-how-to
Thanks to H&M and Kaggle Team for organizing such a great recommendation competition.
\nThanks @paweljankiewicz, @titericz, @cdeotte and many Kagglers for their great discussions and notebooks.
\nI hope to get a solo gold metal this time, so I decided to do this competition by myself in the beginning.
I followed the same pipeline as @paweljankiewicz suggested in Addressing common questions and what the competition is really about . First, generate candidates using different recall models for each customers. Second, build a ranking model that ranks the candidates within customer.
\n
I mainly followed How To Setup Local CV, I created 5 folds and used 3 folds as training data. I also tried more training data, but it didn’t increase the local CV and LB.
\nI developed 4 recall models in total.
\nI used a gating network to make sure that the user tower can learn well by using different experts for recent active customers and non active customers.
\n
\n
\n
I mainly used lightgbm with lambda-rank objective. I also implement DCN model described in https://arxiv.org/abs/2008.13535. But I didn’t have enough time to tune it. The score of lightgbm is much better.
\nTwo Tower MMoE can also generate candidates using user demographic features for customers without purchase log.
\nBesides product meta features, I also extracted the following features from text and image.
\nbest local CV is 0.039, LB is 0.0349.
\nMy final submission combined my previous submissions by
\nh-m-ensembling-how-to
Congratulations to my former colleague for winning solo gold and taking a big step forward on the path to becoming a GM!
", "rawMarkdown": "Congratulations to my former colleague for winning solo gold and taking a big step forward on the path to becoming a GM!", "votes": null }, { "id": "1783116", "postDate": "05/10/2022 06:18:05", "content": "Congratulations! Great work!
", "rawMarkdown": "Congratulations! Great work!", "votes": null }, { "id": "1783140", "postDate": "05/10/2022 06:43:14", "content": "Congratulations！
", "rawMarkdown": "Congratulations！", "votes": null }, { "id": "1783239", "postDate": "05/10/2022 08:07:49", "content": "Amazing! Congratulations!
", "rawMarkdown": "Amazing! Congratulations!", "votes": null }, { "id": "1783380", "postDate": "05/10/2022 10:55:48", "content": "Thanks a lot for sharing this with us! Great explanation!
", "rawMarkdown": "Thanks a lot for sharing this with us! Great explanation!", "votes": null }, { "id": "1783383", "postDate": "05/10/2022 11:00:20", "content": "Congrats!Two Tower MMoE recall should have much fun
", "rawMarkdown": "Congrats!Two Tower MMoE recall should have much fun", "votes": null }, { "id": "1783427", "postDate": "05/10/2022 11:50:26", "content": "Thanks! But honestly it is really challenge to train a NN model with so many cold users.
", "rawMarkdown": "Thanks! But honestly it is really challenge to train a NN model with so many cold users.", "votes": null }, { "id": "1783460", "postDate": "05/10/2022 12:17:13", "content": "Congrats! your tow tower MMoE model is very interesting. to get greater user representation your gated NN learn from users' profile to control the \"importance\" between user static profile and user history behavior sequence. btw，how is LB or oof score of this mmoe model?
", "rawMarkdown": "Congrats! your tow tower MMoE model is very interesting. to get greater user representation your gated NN learn from users' profile to control the \"importance\" between user static profile and user history behavior sequence. btw，how is LB or oof score of this mmoe model?", "votes": null }, { "id": "1783487", "postDate": "05/10/2022 12:32:44", "content": "\n\n\nhow is LB or oof score of this mmoe model?
\n
I didn't submit it. Actually I mainly focused on improving the recall of this model instead of mAP. I only calculated mAP@12 local as a reference, it is around 0.025 for last week.
", "rawMarkdown": "biubiug \n\n> how is LB or oof score of this mmoe model?\n\nI didn't submit it. Actually I mainly focused on improving the recall of this model instead of mAP. I only calculated mAP@12 local as a reference, it is around 0.025 for last week.", "votes": null }, { "id": "1785485", "postDate": "05/12/2022 06:14:42", "content": "@hongweizhang congratulations! and thanks for the detailed write up :)
", "rawMarkdown": "hongweizhang congratulations! and thanks for the detailed write up :)", "votes": null }, { "id": "1785693", "postDate": "05/12/2022 10:23:04", "content": "Thanks a lot for sharing! Great explanation
", "rawMarkdown": "Thanks a lot for sharing! Great explanation", "votes": null }, { "id": "1794544", "postDate": "05/18/2022 23:15:48", "content": "@hongweizhang Hi, the design of MMoE tower is very interesting. Can you give more details? What's the input of the gate? How to get time embedding? Is it one feature you designed?
", "rawMarkdown": "hongweizhang Hi, the design of MMoE tower is very interesting. Can you give more details? What's the input of the gate? How to get time embedding? Is it one feature you designed?", "votes": null } ], "comments": [ { "id": 1783072, "author_name": "haqishen", "author_url": "", "post_date": "05/10/2022 05:20:04", "content": "Congratulations to my former colleague for winning solo gold and taking a big step forward on the path to becoming a GM!
", "votes": null, "replies": [] }, { "id": 1783116, "author_name": "alexkim2", "author_url": "", "post_date": "05/10/2022 06:18:05", "content": "Congratulations! Great work!
", "votes": null, "replies": [] }, { "id": 1783140, "author_name": "xuxiaodong", "author_url": "", "post_date": "05/10/2022 06:43:14", "content": "Congratulations！
", "votes": null, "replies": [] }, { "id": 1783239, "author_name": "pakersmuch", "author_url": "", "post_date": "05/10/2022 08:07:49", "content": "Amazing! Congratulations!
", "votes": null, "replies": [] }, { "id": 1783380, "author_name": "loicge", "author_url": "", "post_date": "05/10/2022 10:55:48", "content": "Thanks a lot for sharing this with us! Great explanation!
", "votes": null, "replies": [] }, { "id": 1783383, "author_name": "senkin13", "author_url": "", "post_date": "05/10/2022 11:00:20", "content": "Congrats!Two Tower MMoE recall should have much fun
", "votes": null, "replies": [ { "id": 1783427, "author_name": "hongweizhang", "author_url": "", "post_date": "05/10/2022 11:50:26", "content": "Thanks! But honestly it is really challenge to train a NN model with so many cold users.
", "votes": null, "replies": [] } ] }, { "id": 1783460, "author_name": "biubiug", "author_url": "", "post_date": "05/10/2022 12:17:13", "content": "Congrats! your tow tower MMoE model is very interesting. to get greater user representation your gated NN learn from users' profile to control the \"importance\" between user static profile and user history behavior sequence. btw，how is LB or oof score of this mmoe model?
", "votes": null, "replies": [ { "id": 1783487, "author_name": "hongweizhang", "author_url": "", "post_date": "05/10/2022 12:32:44", "content": "\n\n\nhow is LB or oof score of this mmoe model?
\n
I didn't submit it. Actually I mainly focused on improving the recall of this model instead of mAP. I only calculated mAP@12 local as a reference, it is around 0.025 for last week.
", "votes": null, "replies": [] } ] }, { "id": 1785485, "author_name": "lachlangillian", "author_url": "", "post_date": "05/12/2022 06:14:42", "content": "@hongweizhang congratulations! and thanks for the detailed write up :)
", "votes": null, "replies": [] }, { "id": 1785693, "author_name": "abhishekwani", "author_url": "", "post_date": "05/12/2022 10:23:04", "content": "Thanks a lot for sharing! Great explanation
", "votes": null, "replies": [] }, { "id": 1794544, "author_name": "laimc01", "author_url": "", "post_date": "05/18/2022 23:15:48", "content": "@hongweizhang Hi, the design of MMoE tower is very interesting. Can you give more details? What's the input of the gate? How to get time embedding? Is it one feature you designed?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1783066": "Thanks to H&M and Kaggle Team for organizing such a great recommendation competition.\nThanks @paweljankiewicz, @titericz, @cdeotte and many Kagglers for their great discussions and notebooks.\nI hope to get a solo gold metal this time, so I decided to do this competition by myself in the beginning.\n\n# Overview\nI followed the same pipeline as @paweljankiewicz suggested in [Addressing common questions and what the competition is really about](kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/discussion/307288) . First, generate candidates using different recall models for each customers. Second, build a ranking model that ranks the candidates within customer.\n[](https://postimg.cc/PNQsRDhB)\n\n# CV setup\nI mainly followed [How To Setup Local CV](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/discussion/308919), I created 5 folds and used 3 folds as training data. I also tried more training data, but it didn’t increase the local CV and LB.\n\n# Recall model\nI developed 4 recall models in total.\n* Item2item CF: the relationship between the pair of items (the user who bought Y, also bought Z).\n* repurchase: latest 20 purchased products by each customers.\n* popular: popular ranking in last week.\n* Two Tower MMoE: I mainly focused on improving the performance of this model. Because it can generate candidates of arbitrary length for all users, and the user/item embeddings (similarity) can also be used as an important feature for ranking model.\n\nI used a gating network to make sure that the user tower can learn well by using different experts for recent active customers and non active customers.\n[](https://postimg.cc/phyh6v5S)\n[](https://postimg.cc/crGCQwh2)\n[](https://postimg.cc/kRQnjYsZ)\n\n# Ranking model\nI mainly used lightgbm with lambda-rank objective. I also implement DCN model described in https://arxiv.org/abs/2008.13535. But I didn’t have enough time to tune it. The score of lightgbm is much better.\n\n# Cold-start users\nTwo Tower MMoE can also generate candidates using user demographic features for customers without purchase log.\n\n# Cold-start items\nBesides product meta features, I also extracted the following features from text and image.\n* Text: Extract TF-IDF features from product description, cluster products using SVD + K-means. Then use cluster id as feature.\n* Image: Extract image vectors using pre-trained tf_efficientnet_b3_ns backbone, cluster products using PCA + K-means. Then use cluster id as feature.\n\n# Submission\nbest local CV is 0.039, LB is 0.0349.\nMy final submission combined my previous submissions by\n[h-m-ensembling-how-to](https://www.kaggle.com/code/titericz/h-m-ensembling-how-to)\n\n# to be updated", "1783072": "Congratulations to my former colleague for winning solo gold and taking a big step forward on the path to becoming a GM!", "1783116": "Congratulations! Great work!", "1783140": "Congratulations！", "1783239": "Amazing! Congratulations!", "1783380": "Thanks a lot for sharing this with us! Great explanation!", "1783383": "Congrats!Two Tower MMoE recall should have much fun", "1783427": "Thanks! But honestly it is really challenge to train a NN model with so many cold users.", "1783460": "Congrats! your tow tower MMoE model is very interesting. to get greater user representation your gated NN learn from users' profile to control the \"importance\" between user static profile and user history behavior sequence. btw，how is LB or oof score of this mmoe model?", "1783487": "biubiug \n\n> how is LB or oof score of this mmoe model?\n\nI didn't submit it. Actually I mainly focused on improving the recall of this model instead of mAP. I only calculated mAP@12 local as a reference, it is around 0.025 for last week.", "1785485": "hongweizhang congratulations! and thanks for the detailed write up :)", "1785693": "Thanks a lot for sharing! Great explanation", "1794544": "hongweizhang Hi, the design of MMoE tower is very interesting. Can you give more details? What's the input of the gate? How to get time embedding? Is it one feature you designed?" }, "source": "meta" }