{
  "id": 77330,
  "title": "How much boost did you get from ensemble?",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/77330",
  "author_name": "",
  "post_date": "2019-01-11T15:33:03.059776900Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Our best single model scores at [0.614 (public), 0.540 (private)]. Our final model is an ensemble of 3 models, which gives us a boost to [0.634 (public), 0.557 (private)]. That is approximate ~0.02 public and 0.017 private. </p>\n\n<p>We used the model predicted probability to ensemble. We tried 2 different strategies:</p>\n\n<ol>\n<li>Train a linear model for each class, then ensemble everything together. This means we trained 28 linear models. (This gives us the best results on both public and private leaderboard.)</li>\n<li>Do simple weighted average. Since we only put 3 models into the ensemble, we applied weights based experience.</li>\n</ol>\n\n<p>Actually the 2nd strategy is not bad at all, it scores [0.622 (public), 0.554(private)]. </p>\n\n<p>Please post your results if you would like to share. </p>",
  "messages": [
    {
      "id": "454403",
      "postDate": "01/11/2019 15:33:03",
      "content": "<p>Our best single model scores at [0.614 (public), 0.540 (private)]. Our final model is an ensemble of 3 models, which gives us a boost to [0.634 (public), 0.557 (private)]. That is approximate ~0.02 public and 0.017 private. </p>\n\n<p>We used the model predicted probability to ensemble. We tried 2 different strategies:</p>\n\n<ol>\n<li>Train a linear model for each class, then ensemble everything together. This means we trained 28 linear models. (This gives us the best results on both public and private leaderboard.)</li>\n<li>Do simple weighted average. Since we only put 3 models into the ensemble, we applied weights based experience.</li>\n</ol>\n\n<p>Actually the 2nd strategy is not bad at all, it scores [0.622 (public), 0.554(private)]. </p>\n\n<p>Please post your results if you would like to share. </p>",
      "rawMarkdown": "Our best single model scores at [0.614 (public), 0.540 (private)]. Our final model is an ensemble of 3 models, which gives us a boost to [0.634 (public), 0.557 (private)]. That is approximate ~0.02 public and 0.017 private. \n\nWe used the model predicted probability to ensemble. We tried 2 different strategies:\n\n1. Train a linear model for each class, then ensemble everything together. This means we trained 28 linear models. (This gives us the best results on both public and private leaderboard.)\n2. Do simple weighted average. Since we only put 3 models into the ensemble, we applied weights based experience.\n\nActually the 2nd strategy is not bad at all, it scores [0.622 (public), 0.554(private)]. \n\nPlease post your results if you would like to share.",
      "votes": null
    },
    {
      "id": "454686",
      "postDate": "01/12/2019 02:05:06",
      "content": "<p>our best single model scores at[0.562(private), but 0.606(public)], not choosed, and best single model on public[0.545(private), 0.612(public)]. ensemble just [0.557(private), public(0.639)].</p>",
      "rawMarkdown": "our best single model scores at[0.562(private), but 0.606(public)], not choosed, and best single model on public[0.545(private), 0.612(public)]. ensemble just [0.557(private), public(0.639)].",
      "votes": null
    },
    {
      "id": "456785",
      "postDate": "01/16/2019 14:46:30",
      "content": "<p>Could you please explain what do you mean by linear model - a binary classifier? Also for 1, did you just average all 28 models' output probabilities and use that to calculate thresholds (assuming you have different thresholds for different classes)? TIA.</p>",
      "rawMarkdown": "Could you please explain what do you mean by linear model - a binary classifier? Also for 1, did you just average all 28 models' output probabilities and use that to calculate thresholds (assuming you have different thresholds for different classes)? TIA.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 454686,
      "author_name": "garybios",
      "author_url": "",
      "post_date": "01/12/2019 02:05:06",
      "content": "<p>our best single model scores at[0.562(private), but 0.606(public)], not choosed, and best single model on public[0.545(private), 0.612(public)]. ensemble just [0.557(private), public(0.639)].</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 456785,
      "author_name": "avipartho",
      "author_url": "",
      "post_date": "01/16/2019 14:46:30",
      "content": "<p>Could you please explain what do you mean by linear model - a binary classifier? Also for 1, did you just average all 28 models' output probabilities and use that to calculate thresholds (assuming you have different thresholds for different classes)? TIA.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "454403": "Our best single model scores at [0.614 (public), 0.540 (private)]. Our final model is an ensemble of 3 models, which gives us a boost to [0.634 (public), 0.557 (private)]. That is approximate ~0.02 public and 0.017 private. \n\nWe used the model predicted probability to ensemble. We tried 2 different strategies:\n\n1. Train a linear model for each class, then ensemble everything together. This means we trained 28 linear models. (This gives us the best results on both public and private leaderboard.)\n2. Do simple weighted average. Since we only put 3 models into the ensemble, we applied weights based experience.\n\nActually the 2nd strategy is not bad at all, it scores [0.622 (public), 0.554(private)]. \n\nPlease post your results if you would like to share.",
    "454686": "our best single model scores at[0.562(private), but 0.606(public)], not choosed, and best single model on public[0.545(private), 0.612(public)]. ensemble just [0.557(private), public(0.639)].",
    "456785": "Could you please explain what do you mean by linear model - a binary classifier? Also for 1, did you just average all 28 models' output probabilities and use that to calculate thresholds (assuming you have different thresholds for different classes)? TIA."
  },
  "source": "meta"
}