Before going on to our solution, I would like to thank Kaggle staff and H&M team, alongside with all competitors on the leaderboard for competing with us in this competition. And of course, special thanks to @A.Sato, @taksai, and @tomo20180402 who are my teammates, for their amazing effort. We are honored to win the gold medal!
\nThe overall strategy of our solution is based on four steps.
\nWe trained three LightGBM ranker models with different candidate selection methodologies, and finally used an ensemble technique to boost our score.
\nThe key idea of our solution was utilizing item2vec embeddings, which is an algorithm inspired by word2vec.
\nArticle embeddings were retrieved using item2vec embeddings as they are, and customer embeddings were created by taking an average among all dimensions of all past purchased articles for each customer.
In the candidate selection phase, we tried to sample positive and negative samples for each customer. The candidate selection methods were different for each of the three models, and the diversity of candidates between our models leveraged our scores after ensembling.
\nThe ensemble method for Model1 was a concatenation of 4 strategies. Past 14 weeks were used to train the model.
\nAside from recently purchased items (and items with the same product code of recently purchased items), my main model solely relies on the local popularity among the nearest 10,000 customers. 1.I obtained user's latent expression by getting the mean of the item2vec embedded expression (32 dimentions) of the items that the customer purchased in the past, 2.then selected 10,000 nearest customers for EACH customer based on the cosine similarity of user latent expression, 3.then picked top 300 last week popular items among 10,000 nearest customers as candidates.
\n
\nAs for single model, this approach got the best score among three models(Private LB 330-ish).
Firstly, I created 4 types of clusters for each customer based on the techniques bellow.
\n(Article-based embeddings were averaged by each dimension of past bought articles to create customer embeddings)
\nAfter the clustering process, each customer will belong to 4 different clusters.
\nFor each cluster, I created an article ranking for the past 30 days, and made a weighted overall ranking depending on which clusters each customer belonged to.
\nRecent bought article_ids and product_codes were weighted heavier according to the customer.\nFrom the overall ranking, Model3 took the top 350 articles per user.\n
For each of the candidates, we created features as bellow.
\nAll three models were trained by LightGBMRaker.
\nModel1 was trained with 14 weeks,and model2/3 were only trained by using candidates from the last 1 week.
Ensembling was simply based by using the methodology used in many of the public notebooks (with a slight change).
\ndef cust_blend(dt, W = [1,1,1], base=3): # base option is added\n\n #Create a list of all model predictions\n REC = []\n REC.append(dt['model1'].split())\n REC.append(dt['model2'].split())\n REC.append(dt['model3'].split())\n\n\n res = {}\n for M in range(len(REC)):\n if type(REC[M]) == list:\n for n, v in enumerate(REC[M]):\n if v in res:\n res[v] += (W[M]/(n+base))\n else:\n res[v] = (W[M]/(n+base))\n\n res = list(dict(sorted(res.items(), key=lambda item: -item[1])).keys())\n\n return ' '.join(res[:12])\nBefore going on to our solution, I would like to thank Kaggle staff and H&M team, alongside with all competitors on the leaderboard for competing with us in this competition. And of course, special thanks to @A.Sato, @taksai, and @tomo20180402 who are my teammates, for their amazing effort. We are honored to win the gold medal!
\nThe overall strategy of our solution is based on four steps.
\nWe trained three LightGBM ranker models with different candidate selection methodologies, and finally used an ensemble technique to boost our score.
\nThe key idea of our solution was utilizing item2vec embeddings, which is an algorithm inspired by word2vec.
\nArticle embeddings were retrieved using item2vec embeddings as they are, and customer embeddings were created by taking an average among all dimensions of all past purchased articles for each customer.
In the candidate selection phase, we tried to sample positive and negative samples for each customer. The candidate selection methods were different for each of the three models, and the diversity of candidates between our models leveraged our scores after ensembling.
\nThe ensemble method for Model1 was a concatenation of 4 strategies. Past 14 weeks were used to train the model.
\nAside from recently purchased items (and items with the same product code of recently purchased items), my main model solely relies on the local popularity among the nearest 10,000 customers. 1.I obtained user's latent expression by getting the mean of the item2vec embedded expression (32 dimentions) of the items that the customer purchased in the past, 2.then selected 10,000 nearest customers for EACH customer based on the cosine similarity of user latent expression, 3.then picked top 300 last week popular items among 10,000 nearest customers as candidates.
\n
\nAs for single model, this approach got the best score among three models(Private LB 330-ish).
Firstly, I created 4 types of clusters for each customer based on the techniques bellow.
\n(Article-based embeddings were averaged by each dimension of past bought articles to create customer embeddings)
\nAfter the clustering process, each customer will belong to 4 different clusters.
\nFor each cluster, I created an article ranking for the past 30 days, and made a weighted overall ranking depending on which clusters each customer belonged to.
\nRecent bought article_ids and product_codes were weighted heavier according to the customer.\nFrom the overall ranking, Model3 took the top 350 articles per user.\n
For each of the candidates, we created features as bellow.
\nAll three models were trained by LightGBMRaker.
\nModel1 was trained with 14 weeks,and model2/3 were only trained by using candidates from the last 1 week.
Ensembling was simply based by using the methodology used in many of the public notebooks (with a slight change).
\ndef cust_blend(dt, W = [1,1,1], base=3): # base option is added\n\n #Create a list of all model predictions\n REC = []\n REC.append(dt['model1'].split())\n REC.append(dt['model2'].split())\n REC.append(dt['model3'].split())\n\n\n res = {}\n for M in range(len(REC)):\n if type(REC[M]) == list:\n for n, v in enumerate(REC[M]):\n if v in res:\n res[v] += (W[M]/(n+base))\n else:\n res[v] = (W[M]/(n+base))\n\n res = list(dict(sorted(res.items(), key=lambda item: -item[1])).keys())\n\n return ' '.join(res[:12])\n\nAfter the clustering process, each customer will belong to 4 different clusters.
\nFor each cluster, I created an article ranking for the past 30 days, and made a weighted overall ranking depending on which clusters each customer belonged to.
\nRecent bought article_ids and product_codes were weighted heavier according to the customer.\nFrom the overall ranking, Model3 took the top 350 articles per user.\n
Awesome result! For the second model could I ask what kind of library did you use to obtain item2vec embedded expression for the items that the customer purchased in the past? The idea seems so simple and awesome! :)
", "rawMarkdown": "Awesome result! For the second model could I ask what kind of library did you use to obtain item2vec embedded expression for the items that the customer purchased in the past? The idea seems so simple and awesome! :)", "votes": null }, { "id": "1784148", "postDate": "05/11/2022 01:08:59", "content": "Thank you!
\nThe library we used is called gensim
Congr @tomotomo5, keep growing.
", "rawMarkdown": "Congr @tomotomo5, keep growing.", "votes": null }, { "id": "1785610", "postDate": "05/12/2022 08:58:49", "content": "Thank you @muhammedtausif
", "rawMarkdown": "Thank you @muhammedtausif", "votes": null } ], "comments": [ { "id": 1783663, "author_name": "simonelis", "author_url": "", "post_date": "05/10/2022 15:03:10", "content": "Awesome result! For the second model could I ask what kind of library did you use to obtain item2vec embedded expression for the items that the customer purchased in the past? The idea seems so simple and awesome! :)
", "votes": null, "replies": [ { "id": 1784148, "author_name": "tomotomo5", "author_url": "", "post_date": "05/11/2022 01:08:59", "content": "Thank you!
\nThe library we used is called gensim
Congr @tomotomo5, keep growing.
", "votes": null, "replies": [ { "id": 1785610, "author_name": "tomotomo5", "author_url": "", "post_date": "05/12/2022 08:58:49", "content": "Thank you @muhammedtausif
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1783606": "Before going on to our solution, I would like to thank Kaggle staff and H&M team, alongside with all competitors on the leaderboard for competing with us in this competition. And of course, special thanks to @A.Sato, @taksai, and @tomo20180402 who are my teammates, for their amazing effort. We are honored to win the gold medal!\n \n# Overall Strategy\nThe overall strategy of our solution is based on four steps.\n\n1. Candidate Selection\n2. Feature Engineering\n3. Ranking\n4. Ensembling\n\n# Key Points\nWe trained three LightGBM ranker models with different candidate selection methodologies, and finally used an ensemble technique to boost our score.\nThe key idea of our solution was utilizing item2vec embeddings, which is an algorithm inspired by word2vec.\nArticle embeddings were retrieved using item2vec embeddings as they are, and customer embeddings were created by taking an average among all dimensions of all past purchased articles for each customer.\n \n# Step1: Candidate(Article) Selection\nIn the candidate selection phase, we tried to sample positive and negative samples for each customer. The candidate selection methods were different for each of the three models, and the diversity of candidates between our models leveraged our scores after ensembling.\n \n## Model1\nThe ensemble method for Model1 was a concatenation of 4 strategies. Past 14 weeks were used to train the model.\n 1. recent 50 purchased articles for each customer\n 2. top 20 products with the same product code of each article in 1.\n 3. last week top 100 articles of all customers for each sales_channel_id group\n 4. last week top 30 of all customers for each [sales_channel_id, index_group_no] group\n \n## Model2\nAside from recently purchased items (and items with the same product code of recently purchased items), my main model solely relies on the local popularity among the nearest 10,000 customers. 1.I obtained user's latent expression by getting the mean of the item2vec embedded expression (32 dimentions) of the items that the customer purchased in the past, 2.then selected 10,000 nearest customers for EACH customer based on the cosine similarity of user latent expression, 3.then picked top 300 last week popular items among 10,000 nearest customers as candidates. \n\nAfter the clustering process, each customer will belong to 4 different clusters.
\nFor each cluster, I created an article ranking for the past 30 days, and made a weighted overall ranking depending on which clusters each customer belonged to.
\nRecent bought article_ids and product_codes were weighted heavier according to the customer.\nFrom the overall ranking, Model3 took the top 350 articles per user.\n