I have been working so hard in this challenge for the past two weeks ,but unfortunately my ranker model can’t beat the heuristic approach shared in public notebooks.
\nMy pipeline is for each target :
\n 1. Generate up to 50/200 aids with co-visitation matrices .
\n 2. I used suggests function to generate candidates . =>Top 20
\n 3. I train on half of the session of @cdeotte data ,for train (%2==0) and validate (%2==1)
\n 4. I create user features and item features(using train + test) (around 90 features)
\n 5. before merging with the true labels , under sample negatives by 15%
\n 6. after merging my candidate data with ground truth i got :
\n\n\n
\n- 2633153 negatives & 68725 positives for clicks
\n- 2684244 negatives & 17634 positives for carts
\n- 2686587 negatives & 15291 positives for orders
\n\n
\n- I train LGBMRanker for
\n20 rounds
\n My CV is :
\n clicks recall = 0.5231477262371275
\n carts recall = 0.40543254663219636
\n orders recall = 0.6483631646548363
\n Overall Recall = 0.5629624354062734
I tried to generate more candidates with more co-vis matrices but CV always decrease , top 20 give best CV .
\nAlso I tried to pass GC_ranking as feature but CV decrease.
\nAnything is wrong with my pipeline ?
I have been working so hard in this challenge for the past two weeks ,but unfortunately my ranker model can’t beat the heuristic approach shared in public notebooks.
\nMy pipeline is for each target :
\n 1. Generate up to 50/200 aids with co-visitation matrices .
\n 2. I used suggests function to generate candidates . =>Top 20
\n 3. I train on half of the session of @cdeotte data ,for train (%2==0) and validate (%2==1)
\n 4. I create user features and item features(using train + test) (around 90 features)
\n 5. before merging with the true labels , under sample negatives by 15%
\n 6. after merging my candidate data with ground truth i got :
\n\n\n
\n- 2633153 negatives & 68725 positives for clicks
\n- 2684244 negatives & 17634 positives for carts
\n- 2686587 negatives & 15291 positives for orders
\n\n
\n- I train LGBMRanker for
\n20 rounds
\n My CV is :
\n clicks recall = 0.5231477262371275
\n carts recall = 0.40543254663219636
\n orders recall = 0.6483631646548363
\n Overall Recall = 0.5629624354062734
I tried to generate more candidates with more co-vis matrices but CV always decrease , top 20 give best CV .
\nAlso I tried to pass GC_ranking as feature but CV decrease.
\nAnything is wrong with my pipeline ?
HI, I also experimented this. From my point of view, here's what you can try ( that works for me ):
\nAlso, your recall can be improved by mixing the predictions of your ranker and the heuristic approach, simply because co-visitation matrices are much powerful in some cases.
", "rawMarkdown": "HI, I also experimented this. From my point of view, here's what you can try ( that works for me ):\n\n- Create similarity scores variables between users|candidates, and candidates | last item seen by the user.\n- Increase the learning rate ( it gave me the little boost I needed to reach 0.6521).\n\n\n\nAlso, your recall can be improved by mixing the predictions of your ranker and the heuristic approach, simply because co-visitation matrices are much powerful in some cases.", "votes": null }, { "id": "2110789", "postDate": "01/22/2023 12:33:30", "content": "thank you 😊
", "rawMarkdown": "thank you 😊", "votes": null } ], "comments": [ { "id": 2110745, "author_name": "rayanaay", "author_url": "", "post_date": "01/22/2023 11:52:19", "content": "HI, I also experimented this. From my point of view, here's what you can try ( that works for me ):
\nAlso, your recall can be improved by mixing the predictions of your ranker and the heuristic approach, simply because co-visitation matrices are much powerful in some cases.
", "votes": null, "replies": [ { "id": 2110789, "author_name": "ihebch", "author_url": "", "post_date": "01/22/2023 12:33:30", "content": "thank you 😊
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2110039": "I have been working so hard in this challenge for the past two weeks ,but unfortunately my ranker model can’t beat the heuristic approach shared in public notebooks.\nMy pipeline is for each target :\n 1. Generate up to 50/200 aids with co-visitation matrices .\n 2. I used suggests function to generate candidates . =>Top 20\n 3. I train on half of the session of @cdeotte data ,for train (`%2==0`) and validate (`%2==1`)\n 4. I create user features and item features(using train + test) (around 90 features)\n 5. before merging with the true labels , under sample negatives by 15%\n 6. after merging my candidate data with ground truth i got :\n > * 2633153 negatives & 68725 positives for clicks\n > * 2684244 negatives & 17634 positives for carts\n > * 2686587 negatives & 15291 positives for orders\n4. I train LGBMRanker for `20 rounds`\nMy CV is :\nclicks recall = 0.5231477262371275\ncarts recall = 0.40543254663219636\norders recall = 0.6483631646548363\n**Overall Recall = 0.5629624354062734**\n\nI tried to generate more candidates with more co-vis matrices but CV always decrease , top 20 give best CV .\nAlso I tried to pass GC_ranking as feature but CV decrease.\nAnything is wrong with my pipeline ?", "2110745": "HI, I also experimented this. From my point of view, here's what you can try ( that works for me ):\n\n- Create similarity scores variables between users|candidates, and candidates | last item seen by the user.\n- Increase the learning rate ( it gave me the little boost I needed to reach 0.6521).\n\n\n\nAlso, your recall can be improved by mixing the predictions of your ranker and the heuristic approach, simply because co-visitation matrices are much powerful in some cases.", "2110789": "thank you 😊" }, "source": "meta" }