{
  "id": 25194,
  "title": "Xgb stacking. Does it work?",
  "url": "/competitions/outbrain-click-prediction/discussion/25194",
  "author_name": "",
  "post_date": "2016-11-07T17:14:03.500Z",
  "votes": 1,
  "comment_count": 8,
  "views": 1088,
  "content": "<p>Hey guys, I've been using xgb stacking for my entire kaggle life but this time it is overfitting like a hell. Did it work for you? Thanks!</p>",
  "messages": [
    {
      "id": "143231",
      "postDate": "11/07/2016 17:14:03",
      "content": "<p>Hey guys, I've been using xgb stacking for my entire kaggle life but this time it is overfitting like a hell. Did it work for you? Thanks!</p>",
      "rawMarkdown": "Hey guys, I've been using xgb stacking for my entire kaggle life but this time it is overfitting like a hell. Did it work for you? Thanks!",
      "votes": null
    },
    {
      "id": "143242",
      "postDate": "11/07/2016 18:10:49",
      "content": "<p>@rcarson, I&#8217;ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\nXgb worked for 'present' data, but overfited for 'future' data.</p>\n\n<p>So using xgb for 'present' data may work even for stacking.</p>",
      "rawMarkdown": "rcarson, I’ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\r\nXgb worked for 'present' data, but overfited for 'future' data.\r\n\r\nSo using xgb for 'present' data may work even for stacking.",
      "votes": null
    },
    {
      "id": "143250",
      "postDate": "11/07/2016 18:42:32",
      "content": "<p>@CuteChibiko, I hope I could upvote you 10x times! </p>\n\n<p>Yeah, it makes perfect sense and i should be careful about time split in stacking too. Meanwhile I highly recommend LightGBM which uses 30% memory of python version xgb, faster and gets equally good scores.</p>",
      "rawMarkdown": "CuteChibiko, I hope I could upvote you 10x times! \r\n\r\nYeah, it makes perfect sense and i should be careful about time split in stacking too. Meanwhile I highly recommend LightGBM which uses 30% memory of python version xgb, faster and gets equally good scores.",
      "votes": null
    },
    {
      "id": "143448",
      "postDate": "11/08/2016 21:02:53",
      "content": "<p>@rcarson \nTBH python is memory inefficient, so you should compare c++ xgb with c++ lightgbm. But anyway it's good to know about lightgbm.  </p>",
      "rawMarkdown": "rcarson \r\nTBH python is memory inefficient, so you should compare c++ xgb with c++ lightgbm. But anyway it's good to know about lightgbm.",
      "votes": null
    },
    {
      "id": "143565",
      "postDate": "11/09/2016 16:28:48",
      "content": "<p>[quote=CuteChibiko;143242]</p>\n\n<p>@rcarson, I&#8217;ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\nXgb worked for 'present' data, but overfited for 'future' data.</p>\n\n<p>So using xgb for 'present' data may work even for stacking.</p>\n\n<p>[/quote]</p>\n\n<p>Could you please clear me what you mean by 'present' and 'future'?</p>",
      "rawMarkdown": "[quote=CuteChibiko;143242]\r\n\r\n@rcarson, I’ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\r\nXgb worked for 'present' data, but overfited for 'future' data.\r\n\r\nSo using xgb for 'present' data may work even for stacking.\r\n\r\n[/quote]\r\n\r\nCould you please clear me what you mean by 'present' and 'future'?",
      "votes": null
    },
    {
      "id": "143578",
      "postDate": "11/09/2016 17:40:51",
      "content": "<p>@shappy, I quoted <a href=\"https://www.kaggle.com/c/outbrain-click-prediction/forums/t/24255/cv-vs-lb/138818#post138818\">this Eric's comment</a>.</p>",
      "rawMarkdown": "shappy, I quoted [this Eric's comment](https://www.kaggle.com/c/outbrain-click-prediction/forums/t/24255/cv-vs-lb/138818#post138818).",
      "votes": null
    },
    {
      "id": "143717",
      "postDate": "11/10/2016 10:24:36",
      "content": "<p>@rcarson\nwhat is xgb stacking? it is used stacking related method to process xgb result? or the method is integrated in the xgb? thanks</p>",
      "rawMarkdown": "rcarson\r\nwhat is xgb stacking? it is used stacking related method to process xgb result? or the method is integrated in the xgb? thanks",
      "votes": null
    },
    {
      "id": "143794",
      "postDate": "11/10/2016 17:59:07",
      "content": "<p>I mean using xgb as the stacking classifier to process other models' results like ffm, ftrl and so on.</p>",
      "rawMarkdown": "I mean using xgb as the stacking classifier to process other models' results like ffm, ftrl and so on.",
      "votes": null
    },
    {
      "id": "151153",
      "postDate": "12/19/2016 03:52:23",
      "content": "<p>@rcarson Why is it that you are using only present data from train set for stacking (and not future) ? Also why would it over fit on the the future data ?</p>",
      "rawMarkdown": "rcarson Why is it that you are using only present data from train set for stacking (and not future) ? Also why would it over fit on the the future data ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 143242,
      "author_name": "its7171",
      "author_url": "",
      "post_date": "11/07/2016 18:10:49",
      "content": "<p>@rcarson, I&#8217;ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\nXgb worked for 'present' data, but overfited for 'future' data.</p>\n\n<p>So using xgb for 'present' data may work even for stacking.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 143250,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "11/07/2016 18:42:32",
      "content": "<p>@CuteChibiko, I hope I could upvote you 10x times! </p>\n\n<p>Yeah, it makes perfect sense and i should be careful about time split in stacking too. Meanwhile I highly recommend LightGBM which uses 30% memory of python version xgb, faster and gets equally good scores.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 143448,
      "author_name": "adamszalucha",
      "author_url": "",
      "post_date": "11/08/2016 21:02:53",
      "content": "<p>@rcarson \nTBH python is memory inefficient, so you should compare c++ xgb with c++ lightgbm. But anyway it's good to know about lightgbm.  </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 143565,
      "author_name": "azsefi",
      "author_url": "",
      "post_date": "11/09/2016 16:28:48",
      "content": "<p>[quote=CuteChibiko;143242]</p>\n\n<p>@rcarson, I&#8217;ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\nXgb worked for 'present' data, but overfited for 'future' data.</p>\n\n<p>So using xgb for 'present' data may work even for stacking.</p>\n\n<p>[/quote]</p>\n\n<p>Could you please clear me what you mean by 'present' and 'future'?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 143578,
      "author_name": "its7171",
      "author_url": "",
      "post_date": "11/09/2016 17:40:51",
      "content": "<p>@shappy, I quoted <a href=\"https://www.kaggle.com/c/outbrain-click-prediction/forums/t/24255/cv-vs-lb/138818#post138818\">this Eric's comment</a>.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 143717,
      "author_name": "flightrush",
      "author_url": "",
      "post_date": "11/10/2016 10:24:36",
      "content": "<p>@rcarson\nwhat is xgb stacking? it is used stacking related method to process xgb result? or the method is integrated in the xgb? thanks</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 143794,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "11/10/2016 17:59:07",
      "content": "<p>I mean using xgb as the stacking classifier to process other models' results like ffm, ftrl and so on.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151153,
      "author_name": "keerath",
      "author_url": "",
      "post_date": "12/19/2016 03:52:23",
      "content": "<p>@rcarson Why is it that you are using only present data from train set for stacking (and not future) ? Also why would it over fit on the the future data ?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "143231": "Hey guys, I've been using xgb stacking for my entire kaggle life but this time it is overfitting like a hell. Did it work for you? Thanks!",
    "143242": "rcarson, I’ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\r\nXgb worked for 'present' data, but overfited for 'future' data.\r\n\r\nSo using xgb for 'present' data may work even for stacking.",
    "143250": "CuteChibiko, I hope I could upvote you 10x times! \r\n\r\nYeah, it makes perfect sense and i should be careful about time split in stacking too. Meanwhile I highly recommend LightGBM which uses 30% memory of python version xgb, faster and gets equally good scores.",
    "143448": "rcarson \r\nTBH python is memory inefficient, so you should compare c++ xgb with c++ lightgbm. But anyway it's good to know about lightgbm.",
    "143565": "[quote=CuteChibiko;143242]\r\n\r\n@rcarson, I’ve not tried stacking yet, but I made 2 xgb models for 'present' data and 'future' data.\r\nXgb worked for 'present' data, but overfited for 'future' data.\r\n\r\nSo using xgb for 'present' data may work even for stacking.\r\n\r\n[/quote]\r\n\r\nCould you please clear me what you mean by 'present' and 'future'?",
    "143578": "shappy, I quoted [this Eric's comment](https://www.kaggle.com/c/outbrain-click-prediction/forums/t/24255/cv-vs-lb/138818#post138818).",
    "143717": "rcarson\r\nwhat is xgb stacking? it is used stacking related method to process xgb result? or the method is integrated in the xgb? thanks",
    "143794": "I mean using xgb as the stacking classifier to process other models' results like ffm, ftrl and so on.",
    "151153": "rcarson Why is it that you are using only present data from train set for stacking (and not future) ? Also why would it over fit on the the future data ?"
  },
  "source": "meta"
}