{ "id": 382772, "title": "116th place write up", "url": "/competitions/otto-recommender-system/writeups/hidehisa-arai-116th-place-write-up", "author_name": "", "post_date": "2023-02-01T00:16:39.983Z", "votes": 36, "comment_count": 6, "views": 0, "content": "

First of all, I want to say thank you to Otto to hold such an interesting competition. Since we only have the interaction information, the task was innevitably hard to solve, but I learned a lot of things from that quite challenging task. I'm also involved in recommender systems at work, so I'm sure this experience will help me.

\n

Handling a large number of sessions (users) was also a hard point of this competition. I'm sure many people suffered from this. Maybe some of you started using cudf or polars instead of the famous pandas. I used BigQuery for joinning large tables, and this was the first time for me to use BQ for Kaggle competition.

\n

We've seen some sad things happened. Some kaggle GMs and Masters seemed to have been cheating, and it turned out that they have likely been doing so for years. I hope Kaggle will take appropriate actions for this problem.

\n

Anyway, here's what I did to achieve this place in this competition.

\n

Overview

\n

I followed the standard candidate re-rank pipeline. I didn't have time to prepare ensembles, so my score comes from single XGBoost Ranker model.

\n

In the first step, I split the train.jsonl into train data (which I call train split afterwards) and validation data (validation split). This was done using the code from host's repository.

\n

Then, I randomly truncate each session in train split to make labels for it.

\n

Then, I made three types of co-visitation matrices which was introduced in this notebook. To avoid leak, I made them in three ways. First ones are for training, so I used truncated version of train split and test data (test.jsonl; I'll call this test split) to make three co-visitation matrices. Second ones are for validation, so I used nontrucated version of train split and test split. Third ones are for inference, so I used the whole data.

\n

I also trained two Word2Vec model in three ways (the same way I used for making co-visitation matrices). One is trained with aid sequences and the other is trained with aid_type sequences (I concatenated aid and type. For example, if there is a record that with aid=0 and type='carts', then aid_carts will be 0_carts).

\n

Then I generate candidates. For each session (user), I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items. For each session, there are average 193 aids (max. 834 items), and candidate recalls are

\n\n

for the validation split.

\n

I sampled around 200,000 sessions for each targets. Then I created around 50 features. With these features, I trained XGBoost Rankers for each target. Finally I rerank the candidates with this model. The recalls@20 for each target is

\n\n

This gave me 0.586 on public LB.

\n

Features

\n

I used 53 features in total.

\n

Rank features from public notebook

\n

The ranking of this notebook was pretty good. I used the rank of each items in the result of the notebook. In the notebook, there are two types of list (list for clicks target and list for buys target), so I used aid rank from both list. These are called clicks_rank and buys_rank respectively.

\n

history features

\n

Some candidate items come from session's history, so I made some history features. I made some group inside each session. It is said that what is called a session is in fact a user in general meaning, so I used a simple method to detect session (general meaning) and called them groups (I used a method similar to the way introduced in this discussion.)

\n

For each aid I generated 16 history features.

\n\n

co-visitation matrices' weights features

\n\n

popularity features

\n\n

cosine similarity features

\n\n

XGBoost Ranker

\n

I used the following parameters to train my XGBoost model

\n
params = {\n    : ,\n    : ,\n    : ,\n    : ,\n    : ,\n    : ,\n    : ,\n}\n
\n

I put aside 20,000 sessions at random to use them for early stopping.

\n

Here's the F-score plot for carts model

\n

\"\"

\n

and here's the one for orders model

\n

\"\"

\n

and here's the one for clicks model

\n

\"\"

", "messages": [ { "id": "2124335", "postDate": "02/01/2023 00:16:18", "content": "

First of all, I want to say thank you to Otto to hold such an interesting competition. Since we only have the interaction information, the task was innevitably hard to solve, but I learned a lot of things from that quite challenging task. I'm also involved in recommender systems at work, so I'm sure this experience will help me.

\n

Handling a large number of sessions (users) was also a hard point of this competition. I'm sure many people suffered from this. Maybe some of you started using cudf or polars instead of the famous pandas. I used BigQuery for joinning large tables, and this was the first time for me to use BQ for Kaggle competition.

\n

We've seen some sad things happened. Some kaggle GMs and Masters seemed to have been cheating, and it turned out that they have likely been doing so for years. I hope Kaggle will take appropriate actions for this problem.

\n

Anyway, here's what I did to achieve this place in this competition.

\n

Overview

\n

I followed the standard candidate re-rank pipeline. I didn't have time to prepare ensembles, so my score comes from single XGBoost Ranker model.

\n

In the first step, I split the train.jsonl into train data (which I call train split afterwards) and validation data (validation split). This was done using the code from host's repository.

\n

Then, I randomly truncate each session in train split to make labels for it.

\n

Then, I made three types of co-visitation matrices which was introduced in this notebook. To avoid leak, I made them in three ways. First ones are for training, so I used truncated version of train split and test data (test.jsonl; I'll call this test split) to make three co-visitation matrices. Second ones are for validation, so I used nontrucated version of train split and test split. Third ones are for inference, so I used the whole data.

\n

I also trained two Word2Vec model in three ways (the same way I used for making co-visitation matrices). One is trained with aid sequences and the other is trained with aid_type sequences (I concatenated aid and type. For example, if there is a record that with aid=0 and type='carts', then aid_carts will be 0_carts).

\n

Then I generate candidates. For each session (user), I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items. For each session, there are average 193 aids (max. 834 items), and candidate recalls are

\n\n

for the validation split.

\n

I sampled around 200,000 sessions for each targets. Then I created around 50 features. With these features, I trained XGBoost Rankers for each target. Finally I rerank the candidates with this model. The recalls@20 for each target is

\n\n

This gave me 0.586 on public LB.

\n

Features

\n

I used 53 features in total.

\n

Rank features from public notebook

\n

The ranking of this notebook was pretty good. I used the rank of each items in the result of the notebook. In the notebook, there are two types of list (list for clicks target and list for buys target), so I used aid rank from both list. These are called clicks_rank and buys_rank respectively.

\n

history features

\n

Some candidate items come from session's history, so I made some history features. I made some group inside each session. It is said that what is called a session is in fact a user in general meaning, so I used a simple method to detect session (general meaning) and called them groups (I used a method similar to the way introduced in this discussion.)

\n

For each aid I generated 16 history features.

\n\n

co-visitation matrices' weights features

\n\n

popularity features

\n\n

cosine similarity features

\n\n

XGBoost Ranker

\n

I used the following parameters to train my XGBoost model

\n
params = {\n    : ,\n    : ,\n    : ,\n    : ,\n    : ,\n    : ,\n    : ,\n}\n
\n

I put aside 20,000 sessions at random to use them for early stopping.

\n

Here's the F-score plot for carts model

\n

\"\"

\n

and here's the one for orders model

\n

\"\"

\n

and here's the one for clicks model

\n

\"\"

", "rawMarkdown": "First of all, I want to say thank you to Otto to hold such an interesting competition. Since we only have the interaction information, the task was innevitably hard to solve, but I learned a lot of things from that quite challenging task. I'm also involved in recommender systems at work, so I'm sure this experience will help me.\n\nHandling a large number of sessions (users) was also a hard point of this competition. I'm sure many people suffered from this. Maybe some of you started using cudf or polars instead of the famous pandas. I used BigQuery for joinning large tables, and this was the first time for me to use BQ for Kaggle competition.\n\nWe've seen some sad things happened. Some kaggle GMs and Masters seemed to have been cheating, and it turned out that they have likely been doing so for years. I hope Kaggle will take appropriate actions for this problem.\n\nAnyway, here's what I did to achieve this place in this competition.\n\n## Overview\n\nI followed the standard candidate re-rank pipeline. I didn't have time to prepare ensembles, so my score comes from single XGBoost Ranker model.\n\nIn the first step, I split the `train.jsonl` into train data (which I call `train split` afterwards) and validation data (`validation split`). This was done using the code from [host's repository](https://github.com/otto-de/recsys-dataset/blob/main/src/testset.py).\n\nThen, I randomly truncate each session in `train split` to make labels for it.\n\nThen, I made three types of co-visitation matrices which was introduced in [this notebook](https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575). To avoid leak, I made them in three ways. First ones are for training, so I used truncated version of `train split` and test data (`test.jsonl`; I'll call this `test split`) to make three co-visitation matrices. Second ones are for validation, so I used nontrucated version of `train split` and `test split`. Third ones are for inference, so I used the whole data.\n\nI also trained two Word2Vec model in three ways (the same way I used for making co-visitation matrices). One is trained with `aid` sequences and the other is trained with `aid_type` sequences (I concatenated `aid` and `type`. For example, if there is a record that with `aid=0` and `type='carts'`, then `aid_carts` will be `0_carts`).\n\nThen I generate candidates. For each session (user), I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items. For each session, there are average 193 aids (max. 834 items), and candidate recalls are\n\n* clicks: 0.663\n* carts: 0.541\n* orders: 0.726\n\nfor the `validation split`.\n\nI sampled around 200,000 sessions for each targets. Then I created around 50 features. With these features, I trained XGBoost Rankers for each target. Finally I rerank the candidates with this model. The recalls@20 for each target is\n\n* clicks: 0.528\n* carts: 0.422\n* orders: 0.657\n* overall: 0.574\n\nThis gave me 0.586 on public LB.\n\n## Features\n\nI used 53 features in total. \n\n### Rank features from public notebook\n\nThe ranking of [this notebook](https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575) was pretty good. I used the rank of each items in the result of the notebook. In the notebook, there are two types of list (list for clicks target and list for buys target), so I used aid rank from both list. These are called `clicks_rank` and `buys_rank` respectively.\n\n### history features\n\nSome candidate items come from session's history, so I made some history features. I made some group inside each session. It is said that what is called a `session` is in fact a `user` in general meaning, so I used a simple method to detect `session` (general meaning) and called them `groups` (I used a method similar to the way introduced in [this discussion](https://www.kaggle.com/competitions/otto-recommender-system/discussion/366138).)\n\nFor each `aid` I generated 16 history features.\n\n* clicks count\n* orders count\n* carts count\n* events count \n - (clicks_count + orders_count + carts_count)\n* clicks group count\n - the number of groups which contains 'carts' event of the item\n* orders group count\n* carts group count\n* events group count\n* time elapsed from last clicks\n* time elapsed from last orders\n* time elapsed from last carts\n* time elapsed from last events\n* events count from last clicks\n - the number of actions taken to the other items after the user took 'clicks' action to the item\n* events count from last orders\n* events count from last carts\n* events count from last events\n\n### co-visitation matrices' weights features\n\n* buy2buy covisitation matrix weight of the item to user's last item\n* buy2buy covisitation matrix weight of the item to the most frequently actioned item in the user's history\n* carts_orders covisitation matrix weight of the item to user's last item\n* carts_orders covisitation matrix weight of the item to the most frequently actioned item in the user's history\n* clicks covisitation matrix weight of the item to user's last item\n* clicks covisitation matrix weight of the item to the most frequently actioned item in the user's history\n\n### popularity features\n\n* global carts count (counted in three ways to avoid leak)\n* global orders count\n* global clicks count\n* number of sessions who clicked the item\n* number of sessions who put the item into cart\n* number of sessions who ordered the item\n\n### cosine similarity features\n\n* items word2vec vector similarity to the last item of the session\n* items word2vec vector similarity to the last item of the session\n* aid_type word2vec vector similarity to the last aid_type of the session\n - concatenate the candidate aid and actions (clicks, carts, orders)\n* aid_type word2vec vector similarity to the mean of aid_types in the last group of the session\n - concatenate the candidate aid and actions (clicks, carts, orders)\n* aid_type word2vec vector similarity to the mean of unique aid_types in the last group of the session\n* max aid_type word2vec vector similarity to the aid_types in the last group of the session\n* aid_type word2vec vector similarity to the mean of aid_types in the session\n* aid_type word2vec vector similarity to the mean of unique aid_types in the session\n* max aid_type word2vec vector similarity ot the aid_types in the session\n\n## XGBoost Ranker\n\nI used the following parameters to train my XGBoost model\n\n```python\nparams = {\n \"verbosity\": 1,\n \"nthread\": 24,\n \"eta\": 0.05,\n \"subsample\": 0.6,\n \"colsample_bytree\": 0.8,\n \"tree_method\": \"gpu_hist\",\n \"objective\": \"rank:pairwise\",\n}\n```\n\nI put aside 20,000 sessions at random to use them for early stopping.\n\nHere's the F-score plot for carts model\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489975%2F748998b852c018b77c0079b3f42a8a75%2Fcarts.png?generation=1675210503694228&alt=media)\n\nand here's the one for orders model\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489975%2F062ed3117586218146f4ed9599c67a16%2Forders.png?generation=1675210547893838&alt=media)\n\nand here's the one for clicks model\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489975%2F6cefdba258d07289198de30823cc331b%2Fclicks.png?generation=1675210576351554&alt=media)", "votes": null }, { "id": "2124378", "postDate": "02/01/2023 00:58:07", "content": "

\"Rank features from public notebook\": can this be considered as stacking?

\n

Congrats!

", "rawMarkdown": "\"Rank features from public notebook\": can this be considered as stacking?\n\nCongrats!", "votes": null }, { "id": "2124543", "postDate": "02/01/2023 04:17:12", "content": "

Congrats @hidehisaarai1213, thanks for sharing! From what you said \"For each session, there are average 193 aids (max. 834 items)\", I wonder whether you use all aids for each session or have a strategy to truncate the session has more than 193 aids?

", "rawMarkdown": "Congrats @hidehisaarai1213, thanks for sharing! From what you said \"For each session, there are average 193 aids (max. 834 items)\", I wonder whether you use all aids for each session or have a strategy to truncate the session has more than 193 aids?", "votes": null }, { "id": "2124988", "postDate": "02/01/2023 11:00:17", "content": "

Thank you for your report!

\n", "rawMarkdown": "Thank you for your report!\n- You used top50 candidates to generate inputs for the Ranker, right?\n- I see you only used co-visitation matrices comparing candidate to a single aid. You used BigQuery and still considered it would take too much time to calculate sum/mean weights for candidate in comparison to 5/10/15 last aids from session?", "votes": null }, { "id": "2125036", "postDate": "02/01/2023 11:31:15", "content": "
\n

For each session, there are average 193 aids (max. 834 items)

\n
\n

This is about the number of candidates for each session!

", "rawMarkdown": "> For each session, there are average 193 aids (max. 834 items)\n\nThis is about the number of candidates for each session!", "votes": null }, { "id": "2125042", "postDate": "02/01/2023 11:36:43", "content": "
\n

You used top50 candidates to generate inputs for the Ranker, right?

\n
\n

It isn't. Here's the way I generated the candidates.

\n
 I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items.\n
\n

The number of candidates varies between sessions (avg: 193, max: 834).

\n
\n

I see you only used co-visitation matrices comparing candidate to a single aid. You used BigQuery and still considered it would take too much time to calculate sum/mean weights for candidate in comparison to 5/10/15 last aids from session?

\n
\n

Calculating Co-visitation matrices was done on my local PC (with cudf, which was the way Chris introduced). I used BigQuery for merging features to the candidate pairs.

", "rawMarkdown": "> You used top50 candidates to generate inputs for the Ranker, right?\n\nIt isn't. Here's the way I generated the candidates.\n\n```\n I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items.\n```\n\nThe number of candidates varies between sessions (avg: 193, max: 834).\n\n> I see you only used co-visitation matrices comparing candidate to a single aid. You used BigQuery and still considered it would take too much time to calculate sum/mean weights for candidate in comparison to 5/10/15 last aids from session?\n\nCalculating Co-visitation matrices was done on my local PC (with cudf, which was the way Chris introduced). I used BigQuery for merging features to the candidate pairs.", "votes": null }, { "id": "2125521", "postDate": "02/01/2023 17:32:03", "content": "

Thank you, understood it about number of features.

\n

Applying matrix to all the pairs of history aids and candidates takes more time and memory than calculating the matrix itself.

", "rawMarkdown": "Thank you, understood it about number of features.\n\nApplying matrix to all the pairs of history aids and candidates takes more time and memory than calculating the matrix itself.", "votes": null } ], "comments": [ { "id": 2124378, "author_name": "hinepo", "author_url": "", "post_date": "02/01/2023 00:58:07", "content": "

\"Rank features from public notebook\": can this be considered as stacking?

\n

Congrats!

", "votes": null, "replies": [] }, { "id": 2124543, "author_name": "huyduong7101", "author_url": "", "post_date": "02/01/2023 04:17:12", "content": "

Congrats @hidehisaarai1213, thanks for sharing! From what you said \"For each session, there are average 193 aids (max. 834 items)\", I wonder whether you use all aids for each session or have a strategy to truncate the session has more than 193 aids?

", "votes": null, "replies": [ { "id": 2125036, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "02/01/2023 11:31:15", "content": "
\n

For each session, there are average 193 aids (max. 834 items)

\n
\n

This is about the number of candidates for each session!

", "votes": null, "replies": [] } ] }, { "id": 2124988, "author_name": "artemfedorov", "author_url": "", "post_date": "02/01/2023 11:00:17", "content": "

Thank you for your report!

\n", "votes": null, "replies": [ { "id": 2125042, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "02/01/2023 11:36:43", "content": "
\n

You used top50 candidates to generate inputs for the Ranker, right?

\n
\n

It isn't. Here's the way I generated the candidates.

\n
 I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items.\n
\n

The number of candidates varies between sessions (avg: 193, max: 834).

\n
\n

I see you only used co-visitation matrices comparing candidate to a single aid. You used BigQuery and still considered it would take too much time to calculate sum/mean weights for candidate in comparison to 5/10/15 last aids from session?

\n
\n

Calculating Co-visitation matrices was done on my local PC (with cudf, which was the way Chris introduced). I used BigQuery for merging features to the candidate pairs.

", "votes": null, "replies": [ { "id": 2125521, "author_name": "artemfedorov", "author_url": "", "post_date": "02/01/2023 17:32:03", "content": "

Thank you, understood it about number of features.

\n

Applying matrix to all the pairs of history aids and candidates takes more time and memory than calculating the matrix itself.

", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2124335": "First of all, I want to say thank you to Otto to hold such an interesting competition. Since we only have the interaction information, the task was innevitably hard to solve, but I learned a lot of things from that quite challenging task. I'm also involved in recommender systems at work, so I'm sure this experience will help me.\n\nHandling a large number of sessions (users) was also a hard point of this competition. I'm sure many people suffered from this. Maybe some of you started using cudf or polars instead of the famous pandas. I used BigQuery for joinning large tables, and this was the first time for me to use BQ for Kaggle competition.\n\nWe've seen some sad things happened. Some kaggle GMs and Masters seemed to have been cheating, and it turned out that they have likely been doing so for years. I hope Kaggle will take appropriate actions for this problem.\n\nAnyway, here's what I did to achieve this place in this competition.\n\n## Overview\n\nI followed the standard candidate re-rank pipeline. I didn't have time to prepare ensembles, so my score comes from single XGBoost Ranker model.\n\nIn the first step, I split the `train.jsonl` into train data (which I call `train split` afterwards) and validation data (`validation split`). This was done using the code from [host's repository](https://github.com/otto-de/recsys-dataset/blob/main/src/testset.py).\n\nThen, I randomly truncate each session in `train split` to make labels for it.\n\nThen, I made three types of co-visitation matrices which was introduced in [this notebook](https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575). To avoid leak, I made them in three ways. First ones are for training, so I used truncated version of `train split` and test data (`test.jsonl`; I'll call this `test split`) to make three co-visitation matrices. Second ones are for validation, so I used nontrucated version of `train split` and `test split`. Third ones are for inference, so I used the whole data.\n\nI also trained two Word2Vec model in three ways (the same way I used for making co-visitation matrices). One is trained with `aid` sequences and the other is trained with `aid_type` sequences (I concatenated `aid` and `type`. For example, if there is a record that with `aid=0` and `type='carts'`, then `aid_carts` will be `0_carts`).\n\nThen I generate candidates. For each session (user), I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items. For each session, there are average 193 aids (max. 834 items), and candidate recalls are\n\n* clicks: 0.663\n* carts: 0.541\n* orders: 0.726\n\nfor the `validation split`.\n\nI sampled around 200,000 sessions for each targets. Then I created around 50 features. With these features, I trained XGBoost Rankers for each target. Finally I rerank the candidates with this model. The recalls@20 for each target is\n\n* clicks: 0.528\n* carts: 0.422\n* orders: 0.657\n* overall: 0.574\n\nThis gave me 0.586 on public LB.\n\n## Features\n\nI used 53 features in total. \n\n### Rank features from public notebook\n\nThe ranking of [this notebook](https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575) was pretty good. I used the rank of each items in the result of the notebook. In the notebook, there are two types of list (list for clicks target and list for buys target), so I used aid rank from both list. These are called `clicks_rank` and `buys_rank` respectively.\n\n### history features\n\nSome candidate items come from session's history, so I made some history features. I made some group inside each session. It is said that what is called a `session` is in fact a `user` in general meaning, so I used a simple method to detect `session` (general meaning) and called them `groups` (I used a method similar to the way introduced in [this discussion](https://www.kaggle.com/competitions/otto-recommender-system/discussion/366138).)\n\nFor each `aid` I generated 16 history features.\n\n* clicks count\n* orders count\n* carts count\n* events count \n - (clicks_count + orders_count + carts_count)\n* clicks group count\n - the number of groups which contains 'carts' event of the item\n* orders group count\n* carts group count\n* events group count\n* time elapsed from last clicks\n* time elapsed from last orders\n* time elapsed from last carts\n* time elapsed from last events\n* events count from last clicks\n - the number of actions taken to the other items after the user took 'clicks' action to the item\n* events count from last orders\n* events count from last carts\n* events count from last events\n\n### co-visitation matrices' weights features\n\n* buy2buy covisitation matrix weight of the item to user's last item\n* buy2buy covisitation matrix weight of the item to the most frequently actioned item in the user's history\n* carts_orders covisitation matrix weight of the item to user's last item\n* carts_orders covisitation matrix weight of the item to the most frequently actioned item in the user's history\n* clicks covisitation matrix weight of the item to user's last item\n* clicks covisitation matrix weight of the item to the most frequently actioned item in the user's history\n\n### popularity features\n\n* global carts count (counted in three ways to avoid leak)\n* global orders count\n* global clicks count\n* number of sessions who clicked the item\n* number of sessions who put the item into cart\n* number of sessions who ordered the item\n\n### cosine similarity features\n\n* items word2vec vector similarity to the last item of the session\n* items word2vec vector similarity to the last item of the session\n* aid_type word2vec vector similarity to the last aid_type of the session\n - concatenate the candidate aid and actions (clicks, carts, orders)\n* aid_type word2vec vector similarity to the mean of aid_types in the last group of the session\n - concatenate the candidate aid and actions (clicks, carts, orders)\n* aid_type word2vec vector similarity to the mean of unique aid_types in the last group of the session\n* max aid_type word2vec vector similarity to the aid_types in the last group of the session\n* aid_type word2vec vector similarity to the mean of aid_types in the session\n* aid_type word2vec vector similarity to the mean of unique aid_types in the session\n* max aid_type word2vec vector similarity ot the aid_types in the session\n\n## XGBoost Ranker\n\nI used the following parameters to train my XGBoost model\n\n```python\nparams = {\n \"verbosity\": 1,\n \"nthread\": 24,\n \"eta\": 0.05,\n \"subsample\": 0.6,\n \"colsample_bytree\": 0.8,\n \"tree_method\": \"gpu_hist\",\n \"objective\": \"rank:pairwise\",\n}\n```\n\nI put aside 20,000 sessions at random to use them for early stopping.\n\nHere's the F-score plot for carts model\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489975%2F748998b852c018b77c0079b3f42a8a75%2Fcarts.png?generation=1675210503694228&alt=media)\n\nand here's the one for orders model\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489975%2F062ed3117586218146f4ed9599c67a16%2Forders.png?generation=1675210547893838&alt=media)\n\nand here's the one for clicks model\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489975%2F6cefdba258d07289198de30823cc331b%2Fclicks.png?generation=1675210576351554&alt=media)", "2124378": "\"Rank features from public notebook\": can this be considered as stacking?\n\nCongrats!", "2124543": "Congrats @hidehisaarai1213, thanks for sharing! From what you said \"For each session, there are average 193 aids (max. 834 items)\", I wonder whether you use all aids for each session or have a strategy to truncate the session has more than 193 aids?", "2124988": "Thank you for your report!\n- You used top50 candidates to generate inputs for the Ranker, right?\n- I see you only used co-visitation matrices comparing candidate to a single aid. You used BigQuery and still considered it would take too much time to calculate sum/mean weights for candidate in comparison to 5/10/15 last aids from session?", "2125036": "> For each session, there are average 193 aids (max. 834 items)\n\nThis is about the number of candidates for each session!", "2125042": "> You used top50 candidates to generate inputs for the Ranker, right?\n\nIt isn't. Here's the way I generated the candidates.\n\n```\n I use the whole aids in the history, top 50 most co-occurring items to the last item (for each co-visitation matrix), top 50 most similar items to the last item (using word2vec vector), top 50 most similar (co-occuring) items to session's most frequent item, top 50 most similar (co-occuring) items to session's most frequent clicks (carts, orders), and most popular items.\n```\n\nThe number of candidates varies between sessions (avg: 193, max: 834).\n\n> I see you only used co-visitation matrices comparing candidate to a single aid. You used BigQuery and still considered it would take too much time to calculate sum/mean weights for candidate in comparison to 5/10/15 last aids from session?\n\nCalculating Co-visitation matrices was done on my local PC (with cudf, which was the way Chris introduced). I used BigQuery for merging features to the candidate pairs.", "2125521": "Thank you, understood it about number of features.\n\nApplying matrix to all the pairs of history aids and candidates takes more time and memory than calculating the matrix itself." }, "source": "meta" }