Hey. I am wondering if anyone else has tried boosted trees on this and if they have been having success?
\n\nI haven't used boosted trees much, but I am finding them to be rather second-rate in approaching this problem. Not sure if I just don't know how to use 'em, or if its just the wrong problem in which to apply them.
", "messages": [ { "id": "92926", "postDate": "09/18/2015 19:39:03", "content": "Hey. I am wondering if anyone else has tried boosted trees on this and if they have been having success?
\n\nI haven't used boosted trees much, but I am finding them to be rather second-rate in approaching this problem. Not sure if I just don't know how to use 'em, or if its just the wrong problem in which to apply them.
", "rawMarkdown": "Hey. I am wondering if anyone else has tried boosted trees on this and if they have been having success? \r\n\r\nI haven't used boosted trees much, but I am finding them to be rather second-rate in approaching this problem. Not sure if I just don't know how to use 'em, or if its just the wrong problem in which to apply them.", "votes": null }, { "id": "92928", "postDate": "09/18/2015 20:42:22", "content": "Yes. My team is using boosting, and of course, xgboost.
", "rawMarkdown": "Yes. My team is using boosting, and of course, xgboost.", "votes": null }, { "id": "92929", "postDate": "09/18/2015 20:53:26", "content": "[quote=Gabriel Barello;92926]\nI am finding them to be rather second-rate in approaching this problem. \n[/quote]
\n\nAnd... what is first-rate for you?
", "rawMarkdown": "[quote=Gabriel Barello;92926]\r\nI am finding them to be rather second-rate in approaching this problem. \r\n[/quote]\r\n\r\nAnd... what is first-rate for you?", "votes": null }, { "id": "92931", "postDate": "09/18/2015 21:00:53", "content": "I am having good luck with a random forest.
", "rawMarkdown": "I am having good luck with a random forest.", "votes": null }, { "id": "92939", "postDate": "09/18/2015 21:57:13", "content": "Among all the competitions I participated, xgboost outperformed random forest all the time.
\n\nYou should probably give it a shot.
", "rawMarkdown": "Among all the competitions I participated, xgboost outperformed random forest all the time.\r\n\r\nYou should probably give it a shot.", "votes": null }, { "id": "92940", "postDate": "09/18/2015 22:19:42", "content": "I don't mean to be overly naive, but hey I guess I am so theres no helping it. Now that I've admitted that: do you have any suggestions about what number of features are reasonable to feed a boosted tree? Or is it just as many as my computer can handle?
", "rawMarkdown": "I don't mean to be overly naive, but hey I guess I am so theres no helping it. Now that I've admitted that: do you have any suggestions about what number of features are reasonable to feed a boosted tree? Or is it just as many as my computer can handle?", "votes": null }, { "id": "92991", "postDate": "09/19/2015 16:03:02", "content": "Xgboost and RF are performing about the same for me, with an ensemble of xgboost and RF doing the best so far. I used 100 features initially and then 50, with similar results. Unfortunately for me, those features don't seem to be quite the best ones:)
", "rawMarkdown": "Xgboost and RF are performing about the same for me, with an ensemble of xgboost and RF doing the best so far. I used 100 features initially and then 50, with similar results. Unfortunately for me, those features don't seem to be quite the best ones:)", "votes": null }, { "id": "95836", "postDate": "10/12/2015 15:41:26", "content": "I've adopted http://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_selection with xgboost in grid search cv so that I don't really have to pick the number. The catch was of course that It takes (cpu) time and probably (memory) space.
", "rawMarkdown": "I've adopted http://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_selection with xgboost in grid search cv so that I don't really have to pick the number. The catch was of course that It takes (cpu) time and probably (memory) space.", "votes": null } ], "comments": [ { "id": 92928, "author_name": "xiaozhouwang", "author_url": "", "post_date": "09/18/2015 20:42:22", "content": "Yes. My team is using boosting, and of course, xgboost.
", "votes": null, "replies": [] }, { "id": 92929, "author_name": "carloshuertas", "author_url": "", "post_date": "09/18/2015 20:53:26", "content": "[quote=Gabriel Barello;92926]\nI am finding them to be rather second-rate in approaching this problem. \n[/quote]
\n\nAnd... what is first-rate for you?
", "votes": null, "replies": [] }, { "id": 92931, "author_name": "gbrello", "author_url": "", "post_date": "09/18/2015 21:00:53", "content": "I am having good luck with a random forest.
", "votes": null, "replies": [] }, { "id": 92939, "author_name": "xiaozhouwang", "author_url": "", "post_date": "09/18/2015 21:57:13", "content": "Among all the competitions I participated, xgboost outperformed random forest all the time.
\n\nYou should probably give it a shot.
", "votes": null, "replies": [] }, { "id": 92940, "author_name": "gbrello", "author_url": "", "post_date": "09/18/2015 22:19:42", "content": "I don't mean to be overly naive, but hey I guess I am so theres no helping it. Now that I've admitted that: do you have any suggestions about what number of features are reasonable to feed a boosted tree? Or is it just as many as my computer can handle?
", "votes": null, "replies": [] }, { "id": 92991, "author_name": "jpmiller", "author_url": "", "post_date": "09/19/2015 16:03:02", "content": "Xgboost and RF are performing about the same for me, with an ensemble of xgboost and RF doing the best so far. I used 100 features initially and then 50, with similar results. Unfortunately for me, those features don't seem to be quite the best ones:)
", "votes": null, "replies": [] }, { "id": 95836, "author_name": "tmjiang", "author_url": "", "post_date": "10/12/2015 15:41:26", "content": "I've adopted http://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_selection with xgboost in grid search cv so that I don't really have to pick the number. The catch was of course that It takes (cpu) time and probably (memory) space.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "92926": "Hey. I am wondering if anyone else has tried boosted trees on this and if they have been having success? \r\n\r\nI haven't used boosted trees much, but I am finding them to be rather second-rate in approaching this problem. Not sure if I just don't know how to use 'em, or if its just the wrong problem in which to apply them.", "92928": "Yes. My team is using boosting, and of course, xgboost.", "92929": "[quote=Gabriel Barello;92926]\r\nI am finding them to be rather second-rate in approaching this problem. \r\n[/quote]\r\n\r\nAnd... what is first-rate for you?", "92931": "I am having good luck with a random forest.", "92939": "Among all the competitions I participated, xgboost outperformed random forest all the time.\r\n\r\nYou should probably give it a shot.", "92940": "I don't mean to be overly naive, but hey I guess I am so theres no helping it. Now that I've admitted that: do you have any suggestions about what number of features are reasonable to feed a boosted tree? Or is it just as many as my computer can handle?", "92991": "Xgboost and RF are performing about the same for me, with an ensemble of xgboost and RF doing the best so far. I used 100 features initially and then 50, with similar results. Unfortunately for me, those features don't seem to be quite the best ones:)", "95836": "I've adopted http://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_selection with xgboost in grid search cv so that I don't really have to pick the number. The catch was of course that It takes (cpu) time and probably (memory) space." }, "source": "meta" }