{
  "id": 133321,
  "title": "strange!!! ensemble of lb0.9828 and lb0.9832 is lb0.9795",
  "url": "/competitions/bengaliai-cv19/discussion/133321",
  "author_name": "",
  "post_date": "2020-03-02T03:33:16.512390900Z",
  "votes": 6,
  "comment_count": 9,
  "views": 0,
  "content": "<p>2/8-single folder 0:\ngrapheme_root=0.9890079448705619\nconsonant_diacritic=0.9949021689661894\nvowel_diacritic=0.9933267294151111\ncv: 0.991561 \nlb: 0.9829</p>\n\n<p>2/8-single folder 1:\ngrapheme_root=0.9879757112866114\nconsonant_diacritic=0.994370991717172\nvowel_diacritic=0.9940929613980307\ncv:0.99110384\nlb:0.9832</p>\n\n<p>when ensemble, lb is 0.9795.\nensemble: out = np.argmax((softmax(output1) + softmax(output2)))</p>\n\n<p>It's strange! Hi, guys,  any suggestions? 😄 </p>",
  "messages": [
    {
      "id": "761025",
      "postDate": "03/02/2020 03:33:16",
      "content": "<p>2/8-single folder 0:\ngrapheme_root=0.9890079448705619\nconsonant_diacritic=0.9949021689661894\nvowel_diacritic=0.9933267294151111\ncv: 0.991561 \nlb: 0.9829</p>\n\n<p>2/8-single folder 1:\ngrapheme_root=0.9879757112866114\nconsonant_diacritic=0.994370991717172\nvowel_diacritic=0.9940929613980307\ncv:0.99110384\nlb:0.9832</p>\n\n<p>when ensemble, lb is 0.9795.\nensemble: out = np.argmax((softmax(output1) + softmax(output2)))</p>\n\n<p>It's strange! Hi, guys,  any suggestions? 😄 </p>",
      "rawMarkdown": "2/8-single folder 0:\ngrapheme_root=0.9890079448705619\nconsonant_diacritic=0.9949021689661894\nvowel_diacritic=0.9933267294151111\ncv: 0.991561 \nlb: 0.9829\n\n2/8-single folder 1:\ngrapheme_root=0.9879757112866114\nconsonant_diacritic=0.994370991717172\nvowel_diacritic=0.9940929613980307\ncv:0.99110384\nlb:0.9832\n\nwhen ensemble, lb is 0.9795.\nensemble: out = np.argmax((softmax(output1) + softmax(output2)))\n\nIt's strange! Hi, guys,  any suggestions? 😄",
      "votes": null
    },
    {
      "id": "761068",
      "postDate": "03/02/2020 05:03:01",
      "content": "<p>Can you please try: \n<code>np.argmax(np.mean(np.array([output1, output2]), axis=0), axis=1)</code></p>",
      "rawMarkdown": "Can you please try: \n`np.argmax(np.mean(np.array([output1, output2]), axis=0), axis=1)`",
      "votes": null
    },
    {
      "id": "761084",
      "postDate": "03/02/2020 05:38:19",
      "content": "<p>thanks, but argmax((a+b)/2)=argmax(a+b). right?</p>",
      "rawMarkdown": "thanks, but argmax((a+b)/2)=argmax(a+b). right?",
      "votes": null
    },
    {
      "id": "761147",
      "postDate": "03/02/2020 07:32:31",
      "content": "<p>× out = np.argmax((softmax(output1) + softmax(output2))) \n√ out = np.argmax(softmax(output1 + output2)) </p>\n\n<p>Have a try on this？</p>",
      "rawMarkdown": "× out = np.argmax((softmax(output1) + softmax(output2))) \n√ out = np.argmax(softmax(output1 + output2)) \n\nHave a try on this？",
      "votes": null
    },
    {
      "id": "761222",
      "postDate": "03/02/2020 09:32:08",
      "content": "<p>Unfortunately， it's 0.9793. Thanks anyway.</p>",
      "rawMarkdown": "Unfortunately， it's 0.9793. Thanks anyway.",
      "votes": null
    },
    {
      "id": "761231",
      "postDate": "03/02/2020 09:43:48",
      "content": "<p>Yeah, dont overfit.</p>",
      "rawMarkdown": "Yeah, dont overfit.",
      "votes": null
    },
    {
      "id": "761348",
      "postDate": "03/02/2020 12:14:10",
      "content": "<p>try <code>np.argmax(np.max(out1, out2))</code>, in my experiment,<code>np.max(out1, out2)</code> is always better than <code>np.mean(out1, out2)</code>, maybe you can try to ensemble more models. BTW, a low public LB does not mean a low private LB.</p>",
      "rawMarkdown": "try `np.argmax(np.max(out1, out2))`, in my experiment,` np.max(out1, out2)` is always better than `np.mean(out1, out2)`, maybe you can try to ensemble more models. BTW, a low public LB does not mean a low private LB.",
      "votes": null
    },
    {
      "id": "761649",
      "postDate": "03/02/2020 20:20:00",
      "content": "<p>well if your 2 first folds disagree so much it's kind of good news (you've got two good but different models), you can probably get a good boost when using a third fold, I would personally train a third one and try this again!</p>",
      "rawMarkdown": "well if your 2 first folds disagree so much it's kind of good news (you've got two good but different models), you can probably get a good boost when using a third fold, I would personally train a third one and try this again!",
      "votes": null
    },
    {
      "id": "761993",
      "postDate": "03/03/2020 05:26:00",
      "content": "<p>yes, it's better for 0.9799, but still smaller than before emsenble. thanks👍 </p>",
      "rawMarkdown": "yes, it's better for 0.9799, but still smaller than before emsenble. thanks👍",
      "votes": null
    },
    {
      "id": "769456",
      "postDate": "03/11/2020 23:31:34",
      "content": "<p>Hey! Have you found why this was happening? I'm currently having this issue when ensembling 2 models.. </p>",
      "rawMarkdown": "Hey! Have you found why this was happening? I'm currently having this issue when ensembling 2 models..",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 761068,
      "author_name": "aroraaman",
      "author_url": "",
      "post_date": "03/02/2020 05:03:01",
      "content": "<p>Can you please try: \n<code>np.argmax(np.mean(np.array([output1, output2]), axis=0), axis=1)</code></p>",
      "votes": null,
      "replies": [
        {
          "id": 761084,
          "author_name": "h030162",
          "author_url": "",
          "post_date": "03/02/2020 05:38:19",
          "content": "<p>thanks, but argmax((a+b)/2)=argmax(a+b). right?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 761147,
      "author_name": "haqishen",
      "author_url": "",
      "post_date": "03/02/2020 07:32:31",
      "content": "<p>× out = np.argmax((softmax(output1) + softmax(output2))) \n√ out = np.argmax(softmax(output1 + output2)) </p>\n\n<p>Have a try on this？</p>",
      "votes": null,
      "replies": [
        {
          "id": 761222,
          "author_name": "h030162",
          "author_url": "",
          "post_date": "03/02/2020 09:32:08",
          "content": "<p>Unfortunately， it's 0.9793. Thanks anyway.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 761231,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "03/02/2020 09:43:48",
      "content": "<p>Yeah, dont overfit.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 761348,
      "author_name": "welkinfeng",
      "author_url": "",
      "post_date": "03/02/2020 12:14:10",
      "content": "<p>try <code>np.argmax(np.max(out1, out2))</code>, in my experiment,<code>np.max(out1, out2)</code> is always better than <code>np.mean(out1, out2)</code>, maybe you can try to ensemble more models. BTW, a low public LB does not mean a low private LB.</p>",
      "votes": null,
      "replies": [
        {
          "id": 761993,
          "author_name": "h030162",
          "author_url": "",
          "post_date": "03/03/2020 05:26:00",
          "content": "<p>yes, it's better for 0.9799, but still smaller than before emsenble. thanks👍 </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 761649,
      "author_name": "optimo",
      "author_url": "",
      "post_date": "03/02/2020 20:20:00",
      "content": "<p>well if your 2 first folds disagree so much it's kind of good news (you've got two good but different models), you can probably get a good boost when using a third fold, I would personally train a third one and try this again!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 769456,
      "author_name": "yannmajewski",
      "author_url": "",
      "post_date": "03/11/2020 23:31:34",
      "content": "<p>Hey! Have you found why this was happening? I'm currently having this issue when ensembling 2 models.. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "761025": "2/8-single folder 0:\ngrapheme_root=0.9890079448705619\nconsonant_diacritic=0.9949021689661894\nvowel_diacritic=0.9933267294151111\ncv: 0.991561 \nlb: 0.9829\n\n2/8-single folder 1:\ngrapheme_root=0.9879757112866114\nconsonant_diacritic=0.994370991717172\nvowel_diacritic=0.9940929613980307\ncv:0.99110384\nlb:0.9832\n\nwhen ensemble, lb is 0.9795.\nensemble: out = np.argmax((softmax(output1) + softmax(output2)))\n\nIt's strange! Hi, guys,  any suggestions? 😄",
    "761068": "Can you please try: \n`np.argmax(np.mean(np.array([output1, output2]), axis=0), axis=1)`",
    "761084": "thanks, but argmax((a+b)/2)=argmax(a+b). right?",
    "761147": "× out = np.argmax((softmax(output1) + softmax(output2))) \n√ out = np.argmax(softmax(output1 + output2)) \n\nHave a try on this？",
    "761222": "Unfortunately， it's 0.9793. Thanks anyway.",
    "761231": "Yeah, dont overfit.",
    "761348": "try `np.argmax(np.max(out1, out2))`, in my experiment,` np.max(out1, out2)` is always better than `np.mean(out1, out2)`, maybe you can try to ensemble more models. BTW, a low public LB does not mean a low private LB.",
    "761649": "well if your 2 first folds disagree so much it's kind of good news (you've got two good but different models), you can probably get a good boost when using a third fold, I would personally train a third one and try this again!",
    "761993": "yes, it's better for 0.9799, but still smaller than before emsenble. thanks👍",
    "769456": "Hey! Have you found why this was happening? I'm currently having this issue when ensembling 2 models.."
  },
  "source": "meta"
}