{
  "id": 27784,
  "title": "Ensemble Techniques for this competition",
  "url": "/competitions/outbrain-click-prediction/discussion/27784",
  "author_name": "",
  "post_date": "2017-01-16T12:11:54.857Z",
  "votes": 1,
  "comment_count": 5,
  "views": 456,
  "content": "<p>I have couple of models, not many, but simple average based ensemble is not working. I am requesting leaders to share their ideas for ensembling. Or leaders are achieving scores in the range 0.69 by using single model.\nAny comments will help</p>",
  "messages": [
    {
      "id": "156455",
      "postDate": "01/16/2017 12:11:54",
      "content": "<p>I have couple of models, not many, but simple average based ensemble is not working. I am requesting leaders to share their ideas for ensembling. Or leaders are achieving scores in the range 0.69 by using single model.\nAny comments will help</p>",
      "rawMarkdown": "I have couple of models, not many, but simple average based ensemble is not working. I am requesting leaders to share their ideas for ensembling. Or leaders are achieving scores in the range 0.69 by using single model.\r\nAny comments will help",
      "votes": null
    },
    {
      "id": "156582",
      "postDate": "01/17/2017 04:50:46",
      "content": "<p>Try Rank averaging, see explanations in here: <a href=\"http://mlwave.com/kaggle-ensembling-guide/\">http://mlwave.com/kaggle-ensembling-guide/ </a>in the \"<em>Rank averaging</em>\" section, it should do better.</p>",
      "rawMarkdown": "Try Rank averaging, see explanations in here: [http://mlwave.com/kaggle-ensembling-guide/ ][1]in the \"*Rank averaging*\" section, it should do better.\r\n\r\n\r\n  [1]: http://mlwave.com/kaggle-ensembling-guide/",
      "votes": null
    },
    {
      "id": "156584",
      "postDate": "01/17/2017 04:54:09",
      "content": "<p>Thanks Yehoshapat.\nWill try this technique.</p>",
      "rawMarkdown": "Thanks Yehoshapat.\r\nWill try this technique.",
      "votes": null
    },
    {
      "id": "156630",
      "postDate": "01/17/2017 12:16:19",
      "content": "<p>We're currently using a random forest with the scores of different models as features for our ensemble.\nMaybe simple averaging could work if all scores are normalised beforehand. \nAnyway, good luck! </p>",
      "rawMarkdown": "We're currently using a random forest with the scores of different models as features for our ensemble.\r\nMaybe simple averaging could work if all scores are normalised beforehand. \r\nAnyway, good luck!",
      "votes": null
    },
    {
      "id": "156923",
      "postDate": "01/18/2017 16:07:41",
      "content": "<p>[quote=OlivierJeunen;156630]</p>\n\n<p>We're currently using a random forest with the scores of different models as features for our ensemble.\nMaybe simple averaging could work if all scores are normalised beforehand. \nAnyway, good luck! </p>\n\n<p>[/quote]\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?</p>",
      "rawMarkdown": "[quote=OlivierJeunen;156630]\r\n\r\nWe're currently using a random forest with the scores of different models as features for our ensemble.\r\nMaybe simple averaging could work if all scores are normalised beforehand. \r\nAnyway, good luck! \r\n\r\n[/quote]\r\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?",
      "votes": null
    },
    {
      "id": "156959",
      "postDate": "01/18/2017 19:01:11",
      "content": "<p>[quote=Avishek;156923]</p>\n\n<p>[quote=OlivierJeunen;156630]</p>\n\n<p>We're currently using a random forest with the scores of different models as features for our ensemble.\nMaybe simple averaging could work if all scores are normalised beforehand. \nAnyway, good luck! </p>\n\n<p>[/quote]\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?</p>\n\n<p>[/quote]</p>\n\n<p>Bad word choice on my part: by scores I meant the probabilities you get from your model. In case of a linear model, that would indeed be wX + b. And that's indeed why I suggested normalising them first.</p>",
      "rawMarkdown": "[quote=Avishek;156923]\r\n\r\n[quote=OlivierJeunen;156630]\r\n\r\nWe're currently using a random forest with the scores of different models as features for our ensemble.\r\nMaybe simple averaging could work if all scores are normalised beforehand. \r\nAnyway, good luck! \r\n\r\n[/quote]\r\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?\r\n\r\n[/quote]\r\n\r\n\r\nBad word choice on my part: by scores I meant the probabilities you get from your model. In case of a linear model, that would indeed be wX + b. And that's indeed why I suggested normalising them first.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 156582,
      "author_name": "yoshscl",
      "author_url": "",
      "post_date": "01/17/2017 04:50:46",
      "content": "<p>Try Rank averaging, see explanations in here: <a href=\"http://mlwave.com/kaggle-ensembling-guide/\">http://mlwave.com/kaggle-ensembling-guide/ </a>in the \"<em>Rank averaging</em>\" section, it should do better.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 156584,
      "author_name": "adityakumarsinha",
      "author_url": "",
      "post_date": "01/17/2017 04:54:09",
      "content": "<p>Thanks Yehoshapat.\nWill try this technique.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 156630,
      "author_name": "olivierjeunen",
      "author_url": "",
      "post_date": "01/17/2017 12:16:19",
      "content": "<p>We're currently using a random forest with the scores of different models as features for our ensemble.\nMaybe simple averaging could work if all scores are normalised beforehand. \nAnyway, good luck! </p>",
      "votes": null,
      "replies": [
        {
          "id": 156923,
          "author_name": "avishek",
          "author_url": "",
          "post_date": "01/18/2017 16:07:41",
          "content": "<p>[quote=OlivierJeunen;156630]</p>\n\n<p>We're currently using a random forest with the scores of different models as features for our ensemble.\nMaybe simple averaging could work if all scores are normalised beforehand. \nAnyway, good luck! </p>\n\n<p>[/quote]\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?</p>",
          "votes": null,
          "replies": [
            {
              "id": 156959,
              "author_name": "olivierjeunen",
              "author_url": "",
              "post_date": "01/18/2017 19:01:11",
              "content": "<p>[quote=Avishek;156923]</p>\n\n<p>[quote=OlivierJeunen;156630]</p>\n\n<p>We're currently using a random forest with the scores of different models as features for our ensemble.\nMaybe simple averaging could work if all scores are normalised beforehand. \nAnyway, good luck! </p>\n\n<p>[/quote]\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?</p>\n\n<p>[/quote]</p>\n\n<p>Bad word choice on my part: by scores I meant the probabilities you get from your model. In case of a linear model, that would indeed be wX + b. And that's indeed why I suggested normalising them first.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "156455": "I have couple of models, not many, but simple average based ensemble is not working. I am requesting leaders to share their ideas for ensembling. Or leaders are achieving scores in the range 0.69 by using single model.\r\nAny comments will help",
    "156582": "Try Rank averaging, see explanations in here: [http://mlwave.com/kaggle-ensembling-guide/ ][1]in the \"*Rank averaging*\" section, it should do better.\r\n\r\n\r\n  [1]: http://mlwave.com/kaggle-ensembling-guide/",
    "156584": "Thanks Yehoshapat.\r\nWill try this technique.",
    "156630": "We're currently using a random forest with the scores of different models as features for our ensemble.\r\nMaybe simple averaging could work if all scores are normalised beforehand. \r\nAnyway, good luck!",
    "156923": "[quote=OlivierJeunen;156630]\r\n\r\nWe're currently using a random forest with the scores of different models as features for our ensemble.\r\nMaybe simple averaging could work if all scores are normalised beforehand. \r\nAnyway, good luck! \r\n\r\n[/quote]\r\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?",
    "156959": "[quote=Avishek;156923]\r\n\r\n[quote=OlivierJeunen;156630]\r\n\r\nWe're currently using a random forest with the scores of different models as features for our ensemble.\r\nMaybe simple averaging could work if all scores are normalised beforehand. \r\nAnyway, good luck! \r\n\r\n[/quote]\r\n@Olivier - By scores I guess you mean, wX+b for a linear model for example that is before applying the sigmoid transform? I guess you suggest normalizing the scores modelwise to account for the difference in mean and dispersion of predictions produced by different models?\r\n\r\n[/quote]\r\n\r\n\r\nBad word choice on my part: by scores I meant the probabilities you get from your model. In case of a linear model, that would indeed be wX + b. And that's indeed why I suggested normalising them first."
  },
  "source": "meta"
}