I found some participants are using rank as the objective for the reranking model, e.g. lambdarank, while some participants are treating the task as classification.
Currently, I'm using the same hyperparameters as here.
\nJust wanna know that, which one is better based on your experiments:)
", "messages": [ { "id": "2099139", "postDate": "01/14/2023 07:45:26", "content": "I found some participants are using rank as the objective for the reranking model, e.g. lambdarank, while some participants are treating the task as classification.
Currently, I'm using the same hyperparameters as here.
\nJust wanna know that, which one is better based on your experiments:)
", "rawMarkdown": "I found some participants are using `rank` as the objective for the reranking model, e.g. `lambdarank`, while some participants are treating the task as classification.\n\nCurrently, I'm using the same hyperparameters as [here](https://www.kaggle.com/code/alberteinsten/cudf-pandas-proof-of-concept-lgbm-ranker). \n\nJust wanna know that, which one is better based on your experiments:)", "votes": null }, { "id": "2099601", "postDate": "01/14/2023 14:51:04", "content": "I don't know about classification, but I have experimented regression + sorting vs ranking + sorting. (LGBM)
\nThe difference in LB was 0.52 vs 0.523, where LGBMRanker had slightly better performance.
\nEver since then, I just used ranker no brainer. Can't do much as a noob black boxing all the methods haha… 0 intuition used.
Thanks for sharing @seholee , I'm also trying classification now
", "rawMarkdown": "Thanks for sharing @seholee , I'm also trying classification now", "votes": null }, { "id": "2099908", "postDate": "01/14/2023 19:02:24", "content": "Great ! you used only 20 estimators ? did you notice a decreased in the score when increasing the number of estimators ?
", "rawMarkdown": "Great ! you used only 20 estimators ? did you notice a decreased in the score when increasing the number of estimators ?", "votes": null }, { "id": "2100633", "postDate": "01/15/2023 10:12:20", "content": "Currently, keep increasing both CV & LB score. I'm experimenting with 100 estimators now:)
", "rawMarkdown": "Currently, keep increasing both CV & LB score. I'm experimenting with 100 estimators now:)", "votes": null }, { "id": "2106555", "postDate": "01/19/2023 08:13:39", "content": "i use CatBoostRanker, LGBMClassifier
\ngiven the same feature set, CatBoostRanker give higher LB
but it will depend on what kind of user-item features you have
", "rawMarkdown": "i use CatBoostRanker, LGBMClassifier\ngiven the same feature set, CatBoostRanker give higher LB\n1. CatBoostRanker CV: 0.5681 LB: 0.579\n2. LGBMClassifier CV: 0.5667 LB: 0.578\n\nbut it will depend on what kind of user-item features you have", "votes": null }, { "id": "2106754", "postDate": "01/19/2023 10:39:17", "content": "I think it depends on your hyperparameters of each model
", "rawMarkdown": "I think it depends on your hyperparameters of each model", "votes": null } ], "comments": [ { "id": 2099601, "author_name": "seholee", "author_url": "", "post_date": "01/14/2023 14:51:04", "content": "I don't know about classification, but I have experimented regression + sorting vs ranking + sorting. (LGBM)
\nThe difference in LB was 0.52 vs 0.523, where LGBMRanker had slightly better performance.
\nEver since then, I just used ranker no brainer. Can't do much as a noob black boxing all the methods haha… 0 intuition used.
Thanks for sharing @seholee , I'm also trying classification now
", "votes": null, "replies": [] } ] }, { "id": 2099908, "author_name": "rayanaay", "author_url": "", "post_date": "01/14/2023 19:02:24", "content": "Great ! you used only 20 estimators ? did you notice a decreased in the score when increasing the number of estimators ?
", "votes": null, "replies": [ { "id": 2100633, "author_name": "wuwenmin", "author_url": "", "post_date": "01/15/2023 10:12:20", "content": "Currently, keep increasing both CV & LB score. I'm experimenting with 100 estimators now:)
", "votes": null, "replies": [] } ] }, { "id": 2106555, "author_name": "ajisamudra", "author_url": "", "post_date": "01/19/2023 08:13:39", "content": "i use CatBoostRanker, LGBMClassifier
\ngiven the same feature set, CatBoostRanker give higher LB
but it will depend on what kind of user-item features you have
", "votes": null, "replies": [ { "id": 2106754, "author_name": "theextremelycuteone", "author_url": "", "post_date": "01/19/2023 10:39:17", "content": "I think it depends on your hyperparameters of each model
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2099139": "I found some participants are using `rank` as the objective for the reranking model, e.g. `lambdarank`, while some participants are treating the task as classification.\n\nCurrently, I'm using the same hyperparameters as [here](https://www.kaggle.com/code/alberteinsten/cudf-pandas-proof-of-concept-lgbm-ranker). \n\nJust wanna know that, which one is better based on your experiments:)", "2099601": "I don't know about classification, but I have experimented regression + sorting vs ranking + sorting. (LGBM)\nThe difference in LB was 0.52 vs 0.523, where LGBMRanker had slightly better performance.\nEver since then, I just used ranker no brainer. Can't do much as a noob black boxing all the methods haha... 0 intuition used.", "2099646": "Thanks for sharing @seholee , I'm also trying classification now", "2099908": "Great ! you used only 20 estimators ? did you notice a decreased in the score when increasing the number of estimators ?", "2100633": "Currently, keep increasing both CV & LB score. I'm experimenting with 100 estimators now:)", "2106555": "i use CatBoostRanker, LGBMClassifier\ngiven the same feature set, CatBoostRanker give higher LB\n1. CatBoostRanker CV: 0.5681 LB: 0.579\n2. LGBMClassifier CV: 0.5667 LB: 0.578\n\nbut it will depend on what kind of user-item features you have", "2106754": "I think it depends on your hyperparameters of each model" }, "source": "meta" }