In my experiment, I use past event, co-visit matrix to generate candidates, each session has about 52 candidates in average, and here is the CV result of my recall method:
\nclicks recall = \ncarts recall = \norders recall = \n=============\nOverall Recall = \n=============\nthen I create 60 features and bulid my ranking model, it got 0.581 in LB.
\nWhat about your results?
In my experiment, I use past event, co-visit matrix to generate candidates, each session has about 52 candidates in average, and here is the CV result of my recall method:
\nclicks recall = \ncarts recall = \norders recall = \n=============\nOverall Recall = \n=============\nthen I create 60 features and bulid my ranking model, it got 0.581 in LB.
\nWhat about your results?
i create 100+ feature, but the rank model doesn't perform well, the good feature i don't find, but some feature i think it's good run too slow,how many time do you generate features?
", "rawMarkdown": "i create 100+ feature, but the rank model doesn't perform well, the good feature i don't find, but some feature i think it's good run too slow,how many time do you generate features?" }, { "id": 2077768, "postDate": "2022-12-27T21:29:12.297Z", "content": "Did you build for each type ( clicks/carts/orders) one specific classifier for each ?
\nAnother question is about a classifier I trained ( lightgbm ) using only clicks from co-visitation matrix of clicks, the model converge, but when computing the recall, I got a worser one.
", "rawMarkdown": "Did you build for each type ( clicks/carts/orders) one specific classifier for each ?\n\nAnother question is about a classifier I trained ( lightgbm ) using only clicks from co-visitation matrix of clicks, the model converge, but when computing the recall, I got a worser one.\n", "replies": [ { "id": 2078340, "postDate": "2022-12-28T08:58:42.343Z", "content": "Ok thanks.
\nWhat about the AUC of your tree ? can you give us an estimation of the AUC at the last iteration ?
", "rawMarkdown": "Ok thanks.\n\nWhat about the AUC of your tree ? can you give us an estimation of the AUC at the last iteration ?" }, { "id": 2078682, "postDate": "2022-12-28T14:32:37.303Z", "content": "Sorry, I hadn't use the AUC as my eval metric, while my ndcg@20 is around 0.88, binary log loss is about 0.3
", "rawMarkdown": "Sorry, I hadn't use the AUC as my eval metric, while my ndcg@20 is around 0.88, binary log loss is about 0.3", "votes": 1 }, { "id": 2080789, "postDate": "2022-12-30T13:38:58.807Z", "content": "Awesome, did you downsample negative candidates ??
", "rawMarkdown": "Awesome, did you downsample negative candidates ??" }, { "id": 2082791, "postDate": "2023-01-02T00:12:19.503Z", "content": "Also, my binary log loss is under 0.3, but still my validation score is not good
", "rawMarkdown": "Also, my binary log loss is under 0.3, but still my validation score is not good\n" }, { "id": 2082966, "postDate": "2023-01-02T06:59:16.740Z", "content": "Yes, I downsample negative candidates, and the binary log loss is depend on the positive/negative rate of the samples
", "rawMarkdown": "Yes, I downsample negative candidates, and the binary log loss is depend on the positive/negative rate of the samples" }, { "id": 2086698, "postDate": "2023-01-05T00:17:22.320Z", "content": "Ok ! I have one last question.
\nFor the candidates generation, did you use both train+test to generate the co-visitation matrix ? or did you generate one for each ?
Good job, I have a question, did you use the co visitation matrices to make the features for the rank model?
", "rawMarkdown": "Good job, I have a question, did you use the co visitation matrices to make the features for the rank model?", "replies": [ { "id": 2077397, "postDate": "2022-12-27T14:38:51.283Z", "content": "Not yet, but I use the rank of co-visit candidates, it helps
", "rawMarkdown": "Not yet, but I use the rank of co-visit candidates, it helps", "replies": [ { "id": 2082992, "postDate": "2023-01-02T07:16:30.090Z", "content": "Thank you! Can I join your team? I have been confused by some problems for a long time, and I hope to get help from you.
", "rawMarkdown": "Thank you! Can I join your team? I have been confused by some problems for a long time, and I hope to get help from you." } ] } ] }, { "id": 2076582, "postDate": "2022-12-26T16:05:55.443Z", "content": "What the objective and eval_metric did you use for the ranking model? I found some people said the rank model's eval_metric doesn't correlate well with the recall20
I use binary classifier
", "rawMarkdown": "I use binary classifier", "replies": [ { "id": 2077187, "postDate": "2022-12-27T10:20:34.247Z", "content": "Thanks, do you willing to share the strategy to get high recall click candidates? I can only get recall = 0.584 for 200 candidates using the strategy in public notebooks.
", "rawMarkdown": "Thanks, do you willing to share the strategy to get high recall click candidates? I can only get recall = 0.584 for 200 candidates using the strategy in public notebooks." }, { "id": 2077311, "postDate": "2022-12-27T12:55:47.333Z", "content": "I also use the strategy in public notebooks, with some modification, the best public notebooks can get 0.576 in LB with just 20 candidates, I think 200+ candidates can easily get recall over 0.6+, maybe there are something wrong with your code?
", "rawMarkdown": "I also use the strategy in public notebooks, with some modification, the best public notebooks can get 0.576 in LB with just 20 candidates, I think 200+ candidates can easily get recall over 0.6+, maybe there are something wrong with your code?" }, { "id": 2077459, "postDate": "2022-12-27T15:41:48.863Z", "content": "I mean the recall for clicks. The recall scores of 200 candidates for each type are:
\nclicks recall = 0.58486\ncarts recall = 0.49270\norders recall = 0.69467\noverall recall = 0.62310\nThanks for your reply. After modifying some parts of the public notebook, my clicks recall@200 increased to 0.6155
", "rawMarkdown": "Thanks for your reply. After modifying some parts of the public notebook, my clicks recall@200 increased to 0.6155" } ] } ] }, { "id": 2077292, "postDate": "2022-12-27T12:36:51.003Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2082884, "author_name": "wcq_glhf", "author_url": "", "post_date": "2023-01-02T04:20:45.380000", "content": "i create 100+ feature, but the rank model doesn't perform well, the good feature i don't find, but some feature i think it's good run too slow,how many time do you generate features?
", "votes": 0, "replies": [] }, { "id": 2077768, "author_name": "Rayan-aay", "author_url": "", "post_date": "2022-12-27T21:29:12.297000", "content": "Did you build for each type ( clicks/carts/orders) one specific classifier for each ?
\nAnother question is about a classifier I trained ( lightgbm ) using only clicks from co-visitation matrix of clicks, the model converge, but when computing the recall, I got a worser one.
", "votes": 0, "replies": [ { "id": 2078340, "author_name": "Lukan", "author_url": "", "post_date": "2022-12-28T08:58:42.343000", "content": "Ok thanks.
\nWhat about the AUC of your tree ? can you give us an estimation of the AUC at the last iteration ?
", "votes": 0, "replies": [] }, { "id": 2078682, "author_name": "Lukan", "author_url": "", "post_date": "2022-12-28T14:32:37.303000", "content": "Sorry, I hadn't use the AUC as my eval metric, while my ndcg@20 is around 0.88, binary log loss is about 0.3
", "votes": 1, "replies": [] }, { "id": 2080789, "author_name": "Rayan-aay", "author_url": "", "post_date": "2022-12-30T13:38:58.807000", "content": "Awesome, did you downsample negative candidates ??
", "votes": 0, "replies": [] }, { "id": 2082791, "author_name": "Rayan-aay", "author_url": "", "post_date": "2023-01-02T00:12:19.503000", "content": "Also, my binary log loss is under 0.3, but still my validation score is not good
", "votes": 0, "replies": [] }, { "id": 2082966, "author_name": "Lukan", "author_url": "", "post_date": "2023-01-02T06:59:16.740000", "content": "Yes, I downsample negative candidates, and the binary log loss is depend on the positive/negative rate of the samples
", "votes": 0, "replies": [] }, { "id": 2086698, "author_name": "Rayan-aay", "author_url": "", "post_date": "2023-01-05T00:17:22.320000", "content": "Ok ! I have one last question.
\nFor the candidates generation, did you use both train+test to generate the co-visitation matrix ? or did you generate one for each ?
Good job, I have a question, did you use the co visitation matrices to make the features for the rank model?
", "votes": 0, "replies": [ { "id": 2077397, "author_name": "Lukan", "author_url": "", "post_date": "2022-12-27T14:38:51.283000", "content": "Not yet, but I use the rank of co-visit candidates, it helps
", "votes": 0, "replies": [ { "id": 2082992, "author_name": "Dong", "author_url": "", "post_date": "2023-01-02T07:16:30.090000", "content": "Thank you! Can I join your team? I have been confused by some problems for a long time, and I hope to get help from you.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2076582, "author_name": "william.wu", "author_url": "", "post_date": "2022-12-26T16:05:55.443000", "content": "What the objective and eval_metric did you use for the ranking model? I found some people said the rank model's eval_metric doesn't correlate well with the recall20
I use binary classifier
", "votes": 0, "replies": [ { "id": 2077187, "author_name": "william.wu", "author_url": "", "post_date": "2022-12-27T10:20:34.247000", "content": "Thanks, do you willing to share the strategy to get high recall click candidates? I can only get recall = 0.584 for 200 candidates using the strategy in public notebooks.
", "votes": 0, "replies": [] }, { "id": 2077311, "author_name": "Lukan", "author_url": "", "post_date": "2022-12-27T12:55:47.333000", "content": "I also use the strategy in public notebooks, with some modification, the best public notebooks can get 0.576 in LB with just 20 candidates, I think 200+ candidates can easily get recall over 0.6+, maybe there are something wrong with your code?
", "votes": 0, "replies": [] }, { "id": 2077459, "author_name": "william.wu", "author_url": "", "post_date": "2022-12-27T15:41:48.863000", "content": "I mean the recall for clicks. The recall scores of 200 candidates for each type are:
\nclicks recall = 0.58486\ncarts recall = 0.49270\norders recall = 0.69467\noverall recall = 0.62310\nThanks for your reply. After modifying some parts of the public notebook, my clicks recall@200 increased to 0.6155
", "votes": 0, "replies": [] } ] } ] }, { "id": 2077292, "author_name": "", "author_url": "", "post_date": "2022-12-27T12:36:51.003000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2076552": "In my experiment, I use past event, co-visit matrix to generate candidates, each session has about 52 candidates in average, and here is the CV result of my recall method:\n```python\nclicks recall = 0.6101419852876675\ncarts recall = 0.4724449332329544\norders recall = 0.6862270709185676\n=============\nOverall Recall = 0.6144839210497937\n=============\n```\nthen I create 60 features and bulid my ranking model, it got 0.581 in LB.\nWhat about your results?", "2082884": "i create 100+ feature, but the rank model doesn't perform well, the good feature i don't find, but some feature i think it's good run too slow,how many time do you generate features?", "2077768": "Did you build for each type ( clicks/carts/orders) one specific classifier for each ?\n\nAnother question is about a classifier I trained ( lightgbm ) using only clicks from co-visitation matrix of clicks, the model converge, but when computing the recall, I got a worser one.\n", "2077371": "Good job, I have a question, did you use the co visitation matrices to make the features for the rank model?", "2076582": "What the `objective` and `eval_metric` did you use for the ranking model? I found some people said the rank model's `eval_metric` doesn't correlate well with the recall20", "2077292": "" } }