Hello guys, thanks for the competition. I read some articles and I understood we need to use candidates to improve recall and construct user-item features to create the model.
\nSo, let's create an example to simplify my question. Let S = ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'D' ] a session where the letters are product ids. Also, let 'O = {B, C}' the purchased products from the same session S. In this example 'A' was the first clicked in 'D` was the last.
Ok, let's generate the candidates. I'm going to use two ideas from this article:
\nSo, from GenerationIdea1 we will add to Candidates set the products {B,C} as they were previously purchased.
\nAnd, from GenerationIdea2 let's suppose the model outputted the products {F} (yeah, I think the model can output a product that was not on the clicks, do you agree?). Therefore, our final candidate set will be Candidates = {B, C, F}.
Now we created the candidates, let's think about some user-item features. For simplification, let's just use:
\nFinally, we will have a table like this:
\n| ItemId | \nsession_length | \nn_times_clicked | \nis_first | \nis_last | \n
|---|---|---|---|---|
| B | \n8 | \n2 | \n0 | \n0 | \n
| C | \n8 | \n2 | \n0 | \n0 | \n
| F | \n8 | \n0 | \n0 | \n0 | \n
Ok, now I have some question:
\n1: Is my idea correct?
\n2: If my idea is correct, how the product item F could be useful for the model if it was not clicked? Probably the model will learn that F is not useful.
\n3: If we want to output 10 items, and our model can output just 3 items. How can we predict the last 7 (10 - 3) items if we are input just 3 items (F, B, and C in this case)
Thank you!
", "messages": [ { "id": "2062291", "postDate": "12/12/2022 00:30:14", "content": "Hello guys, thanks for the competition. I read some articles and I understood we need to use candidates to improve recall and construct user-item features to create the model.
\nSo, let's create an example to simplify my question. Let S = ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'D' ] a session where the letters are product ids. Also, let 'O = {B, C}' the purchased products from the same session S. In this example 'A' was the first clicked in 'D` was the last.
Ok, let's generate the candidates. I'm going to use two ideas from this article:
\nSo, from GenerationIdea1 we will add to Candidates set the products {B,C} as they were previously purchased.
\nAnd, from GenerationIdea2 let's suppose the model outputted the products {F} (yeah, I think the model can output a product that was not on the clicks, do you agree?). Therefore, our final candidate set will be Candidates = {B, C, F}.
Now we created the candidates, let's think about some user-item features. For simplification, let's just use:
\nFinally, we will have a table like this:
\n| ItemId | \nsession_length | \nn_times_clicked | \nis_first | \nis_last | \n
|---|---|---|---|---|
| B | \n8 | \n2 | \n0 | \n0 | \n
| C | \n8 | \n2 | \n0 | \n0 | \n
| F | \n8 | \n0 | \n0 | \n0 | \n
Ok, now I have some question:
\n1: Is my idea correct?
\n2: If my idea is correct, how the product item F could be useful for the model if it was not clicked? Probably the model will learn that F is not useful.
\n3: If we want to output 10 items, and our model can output just 3 items. How can we predict the last 7 (10 - 3) items if we are input just 3 items (F, B, and C in this case)
Thank you!
", "rawMarkdown": "Hello guys, thanks for the competition. I read some articles and I understood we need to use candidates to improve recall and construct user-item features to create the model. \n\nSo, let's create an example to simplify my question. Let `S = ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'D' ]` a session where the letters are product ids. Also, let 'O = {B, C}' the purchased products from the same session `S`. In this example 'A' was the first clicked in 'D` was the last. \n\nOk, let's generate the candidates. I'm going to use two ideas from this [article](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/discussion/324070):\n- Repurchase products. (we will call **GenerationIdea1**)\n- Item collaborative filtering. (we will call **GenerationIdea2**)\n\nSo, from **GenerationIdea1** we will add to `Candidates` set the products `{B,C}` as they were previously purchased. \nAnd, from **GenerationIdea2** let's suppose the model outputted the products `{F}` (*yeah, I think the model can output a product that was not on the clicks, do you agree?*). Therefore, our final candidate set will be `Candidates = {B, C, F}`. \n\nNow we created the candidates, let's think about some user-item features. For simplification, let's just use:\n- Session length\n- Number of times the item was clicked. \n- If it's the first item clicked\n- if it's the last item clicked\n\nFinally, we will have a table like this:\n\n| ItemId | session_length| n_times_clicked | is_first| is_last |\n| --- | --- | --- | --- | --- |\n| B | 8 | 2 | 0 | 0 | \n| C | 8 | 2 | 0 | 0 |\n| F | 8 | 0 | 0 | 0 |\n\nOk, now I have some question:\n1: Is my idea correct? \n2: If my idea is correct, how the product item `F` could be useful for the model if it was not clicked? Probably the model will learn that `F` is not useful. \n3: If we want to output 10 items, and our model can output just 3 items. How can we predict the last 7 (10 - 3) items if we are input just 3 items (`F`, `B`, and `C` in this case)\n\n\nThank you!", "votes": null }, { "id": "2063123", "postDate": "12/12/2022 16:41:20", "content": "Hi @kevintakano ! Yes, your general idea is correct. To summarize, we can split the procedure into a few steps:
\nK items for each session S. In this phase, your goal is to restrict the search space to a limited number of relevant items (= candidates).N recommendations from the re-ranking model.Obviously, K>=N. To answer your questions:
F could be meaningful because we are not bound to recommend something inside the same session S, but instead the next item that we have to predict can be anything (even if this notebook seems to show the opposite, especially for carts/orders)Hope I was clear enough😊
", "rawMarkdown": "Hi @kevintakano ! Yes, your general idea is correct. To summarize, we can split the procedure into a few steps:\n1. Candidate Generation: extract `K` items for each session `S`. In this phase, your goal is to restrict the search space to a limited number of relevant items (= *candidates*).\n2. Re-Ranking: use a re-ranking model to order the candidates from the most to the least relevant. Usually, this kind of model needs a set of features like a classical regression/classification problem, as you pointed out. \n3. Predictions: extract `N` recommendations from the re-ranking model.\n\nObviously, `K`>=`N`. To answer your questions:\n1. Yes, your idea is correct.\n2. The product `F` could be meaningful because we are not bound to recommend something inside the same session `S`, but instead the next item that we have to predict can be anything (even if [this notebook](https://www.kaggle.com/code/sbunzini/carts-and-orders-follow-clicks) seems to show the opposite, especially for *carts/orders*)\n3. In general, you can consider multiple techniques to fill in the missing 7 items. For example, you can add the *most popular items* to your recommendations to reach the number of elements you need. This applies to both *candidate generation models* and the *re-ranker*.\n\nHope I was clear enough😊", "votes": null }, { "id": "2063144", "postDate": "12/12/2022 16:59:43", "content": "Hi Kevin your idea is correct but you are missing the target column in your dataframe. The test data that we download from Kaggle is the first half of user activity and we need to predict the second half (i.e. the Kaggle leaderboard LB). Therefore your full session would be below and the last 5 activity is hidden from us.
\nS = ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'D' || 'A', 'B', 'G', 'F', 'A']
\nIn total the user interacted with 13 items. Kaggle test data will include the first 8 and we need to predict the last 5. Therefore your data frame will include 3 columns of targets (using the ['A', 'B', 'G', 'F', 'A'])
\n| ItemId | \nsession_length | \nn_times_clicked | \nis_first | \nis_last | \nclick | \ncart | \norder | \n
|---|---|---|---|---|---|---|---|
| B | \n8 | \n2 | \n0 | \n0 | \n1 | \n0 | \n0 | \n
| C | \n8 | \n2 | \n0 | \n0 | \n0 | \n0 | \n0 | \n
| F | \n8 | \n0 | \n0 | \n0 | \n0 | \n0 | \n1 | \n
The X_train = df[:,1:5] and y_train = df[:,5:] when you train your GBT ranker model
In order to train a model, we need to split Kaggle's train data (with is 4 weeks) into 3 datasets. Read Radek's post here to learn how to split. The first 3 weeks are \"new train\". The first half of week 4 is \"new test\" and second half of week 4 is \"new LB (leaderboard)\". Then we train our model using the first two datasets as features and the third dataset as targets. Then we infer our model using Kaggle train, Kaggle test to predict Kaggle LB.
", "rawMarkdown": "Hi Kevin your idea is correct but you are missing the target column in your dataframe. The test data that we download from Kaggle is the first half of user activity and we need to predict the second half (i.e. the Kaggle leaderboard LB). Therefore your full session would be below and the last 5 activity is hidden from us.\n\nS = ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'D' || 'A', 'B', 'G', 'F', 'A']\n\nIn total the user interacted with 13 items. Kaggle test data will include the first 8 and we need to predict the last 5. Therefore your data frame will include 3 columns of targets (using the ['A', 'B', 'G', 'F', 'A'])\n\n| ItemId | session_length | n_times_clicked | is_first | is_last | click | cart | order |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| B | 8 | 2 | 0 | 0 | 1 | 0 | 0 |\n| C | 8 | 2 | 0 | 0 | 0 | 0 | 0 |\n| F | 8 | 0 | 0 | 0 | 0 | 0 | 1 |\n\nThe `X_train = df[:,1:5]` and `y_train = df[:,5:]` when you train your GBT ranker model\n\nIn order to train a model, we need to split Kaggle's train data (with is 4 weeks) into 3 datasets. Read Radek's post [here][1] to learn how to split. The first 3 weeks are \"new train\". The first half of week 4 is \"new test\" and second half of week 4 is \"new LB (leaderboard)\". Then we train our model using the first two datasets as features and the third dataset as targets. Then we infer our model using Kaggle train, Kaggle test to predict Kaggle LB.\n\n[1]: https://www.kaggle.com/competitions/otto-recommender-system/discussion/364991", "votes": null }, { "id": "2063473", "postDate": "12/13/2022 00:34:41", "content": "If I understand correctly, we use the 3 datasets like follows:
\nFor Validation: use the 1st dataset as the feature and 2nd dataset as the target. Then we inference using the 1st 2 datasets as the feature and validation on the 3rd dataset to get the CV score.
\nFor Submission: use the 1st 2 datasets as the feature and the 3rd dataset as the target to train the model and inference using the kaggle test set.
\nCorrect me if I'm wrong. Thanks!
\n(I'm busy with work recently, didn't got time to start, just keep reading the discussions to keep updated)
Hi @kevintakano ! Yes, your general idea is correct. To summarize, we can split the procedure into a few steps:
\nK items for each session S. In this phase, your goal is to restrict the search space to a limited number of relevant items (= candidates).N recommendations from the re-ranking model.Obviously, K>=N. To answer your questions:
F could be meaningful because we are not bound to recommend something inside the same session S, but instead the next item that we have to predict can be anything (even if this notebook seems to show the opposite, especially for carts/orders)Hope I was clear enough😊
", "votes": null, "replies": [] }, { "id": 2063144, "author_name": "cdeotte", "author_url": "", "post_date": "12/12/2022 16:59:43", "content": "Hi Kevin your idea is correct but you are missing the target column in your dataframe. The test data that we download from Kaggle is the first half of user activity and we need to predict the second half (i.e. the Kaggle leaderboard LB). Therefore your full session would be below and the last 5 activity is hidden from us.
\nS = ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'D' || 'A', 'B', 'G', 'F', 'A']
\nIn total the user interacted with 13 items. Kaggle test data will include the first 8 and we need to predict the last 5. Therefore your data frame will include 3 columns of targets (using the ['A', 'B', 'G', 'F', 'A'])
\n| ItemId | \nsession_length | \nn_times_clicked | \nis_first | \nis_last | \nclick | \ncart | \norder | \n
|---|---|---|---|---|---|---|---|
| B | \n8 | \n2 | \n0 | \n0 | \n1 | \n0 | \n0 | \n
| C | \n8 | \n2 | \n0 | \n0 | \n0 | \n0 | \n0 | \n
| F | \n8 | \n0 | \n0 | \n0 | \n0 | \n0 | \n1 | \n
The X_train = df[:,1:5] and y_train = df[:,5:] when you train your GBT ranker model
In order to train a model, we need to split Kaggle's train data (with is 4 weeks) into 3 datasets. Read Radek's post here to learn how to split. The first 3 weeks are \"new train\". The first half of week 4 is \"new test\" and second half of week 4 is \"new LB (leaderboard)\". Then we train our model using the first two datasets as features and the third dataset as targets. Then we infer our model using Kaggle train, Kaggle test to predict Kaggle LB.
", "votes": null, "replies": [ { "id": 2063473, "author_name": "wuwenmin", "author_url": "", "post_date": "12/13/2022 00:34:41", "content": "If I understand correctly, we use the 3 datasets like follows:
\nFor Validation: use the 1st dataset as the feature and 2nd dataset as the target. Then we inference using the 1st 2 datasets as the feature and validation on the 3rd dataset to get the CV score.
\nFor Submission: use the 1st 2 datasets as the feature and the 3rd dataset as the target to train the model and inference using the kaggle test set.
\nCorrect me if I'm wrong. Thanks!
\n(I'm busy with work recently, didn't got time to start, just keep reading the discussions to keep updated)