Now that we're in the \"endgame\" of this competition now, people will start to ensemble/stack their models. Just pure blending might not go a long way, but perhaps these few methods can.
\nStackNet.Let me know if you have anything else to share pertaining this topic!
\nEdit: pertaining to the suggestion of @ajaychaudhari , stacking won't ideally take much time depending on what type of model you use - in the public kernels stacking doesn't take much time AFAIK.
\nHowever if you are using heavier models which take > 2.5 hrs to run, then stacking might not be ideal.
", "messages": [ { "id": "1020337", "postDate": "09/21/2020 05:28:19", "content": "Now that we're in the \"endgame\" of this competition now, people will start to ensemble/stack their models. Just pure blending might not go a long way, but perhaps these few methods can.
\nStackNet.Let me know if you have anything else to share pertaining this topic!
\nEdit: pertaining to the suggestion of @ajaychaudhari , stacking won't ideally take much time depending on what type of model you use - in the public kernels stacking doesn't take much time AFAIK.
\nHowever if you are using heavier models which take > 2.5 hrs to run, then stacking might not be ideal.
", "rawMarkdown": "Now that we're in the \"endgame\" of this competition now, people will start to ensemble/stack their models. Just pure blending might not go a long way, but perhaps these few methods can.\n\n+ https://www.kaggle.com/c/ashrae-energy-prediction/discussion/122369 scipy.optimize's lsq_linear\n+ https://github.com/h2oai/pystacknet PyStackNet, a Python implementation of GM Kazanova's library `StackNet`.\n+ https://www.kaggle.com/khoongweihao/post-processing-technique-c-f-1st-place-jigsaw Again, this might be unstable but it might work if you're lucky and have thoroughly checked the coefficient.\n\n**Let me know if you have anything else to share pertaining this topic!**\n\nEdit: pertaining to the suggestion of @ajaychaudhari , stacking won't ideally take much time depending on what type of model you use - in the public kernels stacking doesn't take much time AFAIK.\n\nHowever if you are using heavier models which take > 2.5 hrs to run, then stacking might not be ideal.", "votes": null }, { "id": "1020344", "postDate": "09/21/2020 05:38:09", "content": "I tried out stacking but notebook ran out of time. Any results you want to share?
", "rawMarkdown": "I tried out stacking but notebook ran out of time. Any results you want to share?", "votes": null }, { "id": "1021415", "postDate": "09/21/2020 20:16:25", "content": "Thank you for sharing.
\nIn my previous competition, I used https://www.kaggle.com/mekhdigakhramanian/post-processing-v2 with public submission outputs and 3 my own kernels but it was overfitted, be careful
Similar to what you have mentioned in point-1, I tried blending models by finding optimal weights using OOF predictions. Blending gives a slight boost to the CV.
\nStackNet might be tricky to implement here given that most of the conventional models are not doing well in this problem and its also going to be a challenge to find a lot of uncorrelated models. Any thoughts on this?
", "rawMarkdown": "Similar to what you have mentioned in point-1, I tried blending models by finding optimal weights using OOF predictions. Blending gives a slight boost to the CV. \n\nStackNet might be tricky to implement here given that most of the conventional models are not doing well in this problem and its also going to be a challenge to find a lot of uncorrelated models. Any thoughts on this?", "votes": null }, { "id": "1023180", "postDate": "09/23/2020 02:24:04", "content": "\n\nmost of the conventional models are not doing well in this problem
\n
Since most people are using NNs or a variant, I doubt they'd be much interested in using StackNet. LightGBM in a public kernel was pretty decent, but haven't seen much of LGB after that.
\nUncorrelated models too might be a bit of a challenge, so you'll need to run a lot of (computationally expensive) experiments in order to get the best possible models from your experiments.
", "rawMarkdown": "> most of the conventional models are not doing well in this problem\n\nSince most people are using NNs or a variant, I doubt they'd be much interested in using StackNet. LightGBM in a public kernel was pretty decent, but haven't seen much of LGB after that.\n\nUncorrelated models too might be a bit of a challenge, so you'll need to run a lot of (computationally expensive) experiments in order to get the best possible models from your experiments.", "votes": null }, { "id": "1025196", "postDate": "09/24/2020 11:57:24", "content": "informative , Thanks for sharing @nxrprime
", "rawMarkdown": "informative , Thanks for sharing @nxrprime", "votes": null }, { "id": "1026959", "postDate": "09/25/2020 17:28:00", "content": "Thanks for sharing @nxrprime
", "rawMarkdown": "Thanks for sharing @nxrprime", "votes": null } ], "comments": [ { "id": 1020344, "author_name": "ajay19", "author_url": "", "post_date": "09/21/2020 05:38:09", "content": "I tried out stacking but notebook ran out of time. Any results you want to share?
", "votes": null, "replies": [] }, { "id": 1021415, "author_name": "", "author_url": "", "post_date": "09/21/2020 20:16:25", "content": "Thank you for sharing.
\nIn my previous competition, I used https://www.kaggle.com/mekhdigakhramanian/post-processing-v2 with public submission outputs and 3 my own kernels but it was overfitted, be careful
Similar to what you have mentioned in point-1, I tried blending models by finding optimal weights using OOF predictions. Blending gives a slight boost to the CV.
\nStackNet might be tricky to implement here given that most of the conventional models are not doing well in this problem and its also going to be a challenge to find a lot of uncorrelated models. Any thoughts on this?
", "votes": null, "replies": [ { "id": 1023180, "author_name": "nxrprime", "author_url": "", "post_date": "09/23/2020 02:24:04", "content": "\n\nmost of the conventional models are not doing well in this problem
\n
Since most people are using NNs or a variant, I doubt they'd be much interested in using StackNet. LightGBM in a public kernel was pretty decent, but haven't seen much of LGB after that.
\nUncorrelated models too might be a bit of a challenge, so you'll need to run a lot of (computationally expensive) experiments in order to get the best possible models from your experiments.
", "votes": null, "replies": [] } ] }, { "id": 1026959, "author_name": "vineeth1999", "author_url": "", "post_date": "09/25/2020 17:28:00", "content": "Thanks for sharing @nxrprime
", "votes": null, "replies": [] }, { "id": 1025196, "author_name": "pinakimishrads", "author_url": "", "post_date": "09/24/2020 11:57:24", "content": "informative , Thanks for sharing @nxrprime
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1020337": "Now that we're in the \"endgame\" of this competition now, people will start to ensemble/stack their models. Just pure blending might not go a long way, but perhaps these few methods can.\n\n+ https://www.kaggle.com/c/ashrae-energy-prediction/discussion/122369 scipy.optimize's lsq_linear\n+ https://github.com/h2oai/pystacknet PyStackNet, a Python implementation of GM Kazanova's library `StackNet`.\n+ https://www.kaggle.com/khoongweihao/post-processing-technique-c-f-1st-place-jigsaw Again, this might be unstable but it might work if you're lucky and have thoroughly checked the coefficient.\n\n**Let me know if you have anything else to share pertaining this topic!**\n\nEdit: pertaining to the suggestion of @ajaychaudhari , stacking won't ideally take much time depending on what type of model you use - in the public kernels stacking doesn't take much time AFAIK.\n\nHowever if you are using heavier models which take > 2.5 hrs to run, then stacking might not be ideal.", "1020344": "I tried out stacking but notebook ran out of time. Any results you want to share?", "1021415": "Thank you for sharing.\nIn my previous competition, I used https://www.kaggle.com/mekhdigakhramanian/post-processing-v2 with public submission outputs and 3 my own kernels but it was overfitted, be careful", "1022832": "Similar to what you have mentioned in point-1, I tried blending models by finding optimal weights using OOF predictions. Blending gives a slight boost to the CV. \n\nStackNet might be tricky to implement here given that most of the conventional models are not doing well in this problem and its also going to be a challenge to find a lot of uncorrelated models. Any thoughts on this?", "1023180": "> most of the conventional models are not doing well in this problem\n\nSince most people are using NNs or a variant, I doubt they'd be much interested in using StackNet. LightGBM in a public kernel was pretty decent, but haven't seen much of LGB after that.\n\nUncorrelated models too might be a bit of a challenge, so you'll need to run a lot of (computationally expensive) experiments in order to get the best possible models from your experiments.", "1025196": "informative , Thanks for sharing @nxrprime", "1026959": "Thanks for sharing @nxrprime" }, "source": "meta" }