{
  "id": 173365,
  "title": "People Started Ensembling?",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/173365",
  "author_name": "",
  "post_date": "2020-08-09T00:26:40.129450100Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I'm kinda new new into competitions, was hanging around similar spots in LB lately with my basic blends but past 1-2 days I dropped like 50+ places.</p>\n<p>Is it due to people start ensembling and overfitting to public test set or am I missing something? Can't stop myself to submit and waste one of my submissions with overfit submission now :)</p>\n<p>What are your final week approaches?</p>",
  "messages": [
    {
      "id": "963355",
      "postDate": "08/09/2020 00:26:40",
      "content": "<p>I'm kinda new new into competitions, was hanging around similar spots in LB lately with my basic blends but past 1-2 days I dropped like 50+ places.</p>\n<p>Is it due to people start ensembling and overfitting to public test set or am I missing something? Can't stop myself to submit and waste one of my submissions with overfit submission now :)</p>\n<p>What are your final week approaches?</p>",
      "rawMarkdown": "I'm kinda new new into competitions, was hanging around similar spots in LB lately with my basic blends but past 1-2 days I dropped like 50+ places.\n\nIs it due to people start ensembling and overfitting to public test set or am I missing something? Can't stop myself to submit and waste one of my submissions with overfit submission now :)\n\nWhat are your final week approaches?",
      "votes": null
    },
    {
      "id": "963632",
      "postDate": "08/09/2020 07:25:21",
      "content": "<p>Great words.</p>",
      "rawMarkdown": "Great words.",
      "votes": null
    },
    {
      "id": "964451",
      "postDate": "08/09/2020 22:36:11",
      "content": "<p>It's not blending anymore, it has became \"Nested Blending\". We may see 0.98 public notebooks soon lol</p>",
      "rawMarkdown": "It's not blending anymore, it has became \"Nested Blending\". We may see 0.98 public notebooks soon lol",
      "votes": null
    },
    {
      "id": "964556",
      "postDate": "08/10/2020 02:16:18",
      "content": "<p>I wouldn't worry too much for blending kernels.<br>\nTaking highest score submission on the public LB is creating a huge bias toward the public only:</p>\n<ul>\n<li>The base notebooks have in themselves some amount of overfitting due to the HP they use. Those that were used for blending have a high selection bias based on public LB.</li>\n<li>Then ensemble HP or techniques were used on them once again by taking the public LB as deciding method.</li>\n<li>Now we are seeing ensembles of scores from ensemble… once again with the same strategy to select the HP.</li>\n</ul>\n<p>Either the public LB is a perfect representation of private and in this case it's ok or some small differences can make the above very inefficient.</p>\n<p>It should be <strong>OBVIOUS</strong> considering the low amount of malignant that enough people poking the public LB by adjusting hyperparameters and what to put/drop from the ensemble will end up converging to an overfit ensemble in the end.</p>\n<p>On the other hand it proves (once again) ensembling gives good boost over single models in the 0.94-0.95 region. So this last week should be dedicated to make your ensemble diverse and even explore multi layer ensembling.</p>",
      "rawMarkdown": "I wouldn't worry too much for blending kernels.\nTaking highest score submission on the public LB is creating a huge bias toward the public only:\n* The base notebooks have in themselves some amount of overfitting due to the HP they use. Those that were used for blending have a high selection bias based on public LB.\n* Then ensemble HP or techniques were used on them once again by taking the public LB as deciding method.\n* Now we are seeing ensembles of scores from ensemble... once again with the same strategy to select the HP.\n\nEither the public LB is a perfect representation of private and in this case it's ok or some small differences can make the above very inefficient.\n\nIt should be **OBVIOUS** considering the low amount of malignant that enough people poking the public LB by adjusting hyperparameters and what to put/drop from the ensemble will end up converging to an overfit ensemble in the end.\n\nOn the other hand it proves (once again) ensembling gives good boost over single models in the 0.94-0.95 region. So this last week should be dedicated to make your ensemble diverse and even explore multi layer ensembling.",
      "votes": null
    },
    {
      "id": "964567",
      "postDate": "08/10/2020 02:35:01",
      "content": "<p>let me do it xD</p>",
      "rawMarkdown": "let me do it xD",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 964556,
      "author_name": "arroqc",
      "author_url": "",
      "post_date": "08/10/2020 02:16:18",
      "content": "<p>I wouldn't worry too much for blending kernels.<br>\nTaking highest score submission on the public LB is creating a huge bias toward the public only:</p>\n<ul>\n<li>The base notebooks have in themselves some amount of overfitting due to the HP they use. Those that were used for blending have a high selection bias based on public LB.</li>\n<li>Then ensemble HP or techniques were used on them once again by taking the public LB as deciding method.</li>\n<li>Now we are seeing ensembles of scores from ensemble… once again with the same strategy to select the HP.</li>\n</ul>\n<p>Either the public LB is a perfect representation of private and in this case it's ok or some small differences can make the above very inefficient.</p>\n<p>It should be <strong>OBVIOUS</strong> considering the low amount of malignant that enough people poking the public LB by adjusting hyperparameters and what to put/drop from the ensemble will end up converging to an overfit ensemble in the end.</p>\n<p>On the other hand it proves (once again) ensembling gives good boost over single models in the 0.94-0.95 region. So this last week should be dedicated to make your ensemble diverse and even explore multi layer ensembling.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 963632,
      "author_name": "",
      "author_url": "",
      "post_date": "08/09/2020 07:25:21",
      "content": "<p>Great words.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 964451,
      "author_name": "mahmoodhoseini",
      "author_url": "",
      "post_date": "08/09/2020 22:36:11",
      "content": "<p>It's not blending anymore, it has became \"Nested Blending\". We may see 0.98 public notebooks soon lol</p>",
      "votes": null,
      "replies": [
        {
          "id": 964567,
          "author_name": "yash612",
          "author_url": "",
          "post_date": "08/10/2020 02:35:01",
          "content": "<p>let me do it xD</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "963355": "I'm kinda new new into competitions, was hanging around similar spots in LB lately with my basic blends but past 1-2 days I dropped like 50+ places.\n\nIs it due to people start ensembling and overfitting to public test set or am I missing something? Can't stop myself to submit and waste one of my submissions with overfit submission now :)\n\nWhat are your final week approaches?",
    "963632": "Great words.",
    "964451": "It's not blending anymore, it has became \"Nested Blending\". We may see 0.98 public notebooks soon lol",
    "964556": "I wouldn't worry too much for blending kernels.\nTaking highest score submission on the public LB is creating a huge bias toward the public only:\n* The base notebooks have in themselves some amount of overfitting due to the HP they use. Those that were used for blending have a high selection bias based on public LB.\n* Then ensemble HP or techniques were used on them once again by taking the public LB as deciding method.\n* Now we are seeing ensembles of scores from ensemble... once again with the same strategy to select the HP.\n\nEither the public LB is a perfect representation of private and in this case it's ok or some small differences can make the above very inefficient.\n\nIt should be **OBVIOUS** considering the low amount of malignant that enough people poking the public LB by adjusting hyperparameters and what to put/drop from the ensemble will end up converging to an overfit ensemble in the end.\n\nOn the other hand it proves (once again) ensembling gives good boost over single models in the 0.94-0.95 region. So this last week should be dedicated to make your ensemble diverse and even explore multi layer ensembling.",
    "964567": "let me do it xD"
  },
  "source": "meta"
}