{
  "id": 160903,
  "title": "59th Place Solution (And My First Silver Medal)",
  "url": "/competitions/jigsaw-multilingual-toxic-comment-classification/writeups/ensemble-59th-place-solution-and-my-first-silver-m",
  "author_name": "",
  "post_date": "2020-06-23T04:40:14.670614400Z",
  "votes": 25,
  "comment_count": 17,
  "views": 0,
  "content": "<h1>Overview</h1>\n\n<p>My two best submissions were from a stacking pipeline. I've trained several models on Kaggle and Colab, which I later stacked their predictions using various stacking techniques (not just the usual mean, median, minmax) to achieve higher LB scores. </p>\n\n<h2>The Two Best Submissions</h2>\n\n<ul>\n<li>Stacking &gt; 12 models (used mean of the predictions of 12 submission files, from 9 single models and 3 blends) <strong>(Private LB: 0.9469, Public LB: 0.9485)</strong></li>\n<li>Blend of stacked predictions from above that gave 0.9485 on the public LB with <code>ExtraTreesClassifier</code> predictions trained on the validation data with <a href=\"https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse/notebook\">https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse/notebook</a> <strong>(Private LB: 0.9471, Public LB: 0.9485)</strong></li>\n</ul>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1738095%2F78d1c6c510639908e3d10b481e1ec8ea%2Fbest_subs.png?generation=1592885422854659&amp;alt=media\" alt=\"\"></p>\n\n<h1>Models Used For Stacking</h1>\n\n<ul>\n<li>XLM-RoBERTa</li>\n<li>XLM-RoBERTa Large</li>\n<li>XLM-RoBERTa Large + MLM training</li>\n<li>ExtraTreesClassifier</li>\n</ul>\n\n<p>P.S. Might have missed out one or two, will update here should I find any missing.</p>\n\n<h1>How Models Were Trained</h1>\n\n<ul>\n<li>For the MLM model (third above), I trained it on Kaggle </li>\n<li>For the XLM-RoBERTa models, I trained them on Google Colab TPU</li>\n</ul>\n\n<h1>Notebooks With Models Referenced Above</h1>\n\n<p>A big thanks to all whom contributed to the following notebooks, you have taught me quite a lot! I'm still relatively new to NLP, and your sharing has made me more comfortable and interested in the domain! :)</p>\n\n<ul>\n<li><a href=\"https://www.kaggle.com/riblidezso/train-from-mlm-finetuned-xlm-roberta-large\">https://www.kaggle.com/riblidezso/train-from-mlm-finetuned-xlm-roberta-large</a> </li>\n<li><a href=\"https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse\">https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse</a></li>\n<li><a href=\"https://www.kaggle.com/hamditarek/ensemble\">https://www.kaggle.com/hamditarek/ensemble</a></li>\n<li><a href=\"https://www.kaggle.com/sai11fkaneko/data-leak\">https://www.kaggle.com/sai11fkaneko/data-leak</a></li>\n<li><a href=\"https://www.kaggle.com/shonenkov/tpu-inference-super-fast-xlmroberta\">https://www.kaggle.com/shonenkov/tpu-inference-super-fast-xlmroberta</a></li>\n<li><a href=\"https://www.kaggle.com/shonenkov/tpu-training-super-fast-xlmroberta\">https://www.kaggle.com/shonenkov/tpu-training-super-fast-xlmroberta</a></li>\n<li><a href=\"https://www.kaggle.com/aiaiooas/parcor-regularised-classification\">https://www.kaggle.com/aiaiooas/parcor-regularised-classification</a></li>\n<li><a href=\"https://www.kaggle.com/shahules/fine-tune-xlm-kfold-cv-0-93-lb\">https://www.kaggle.com/shahules/fine-tune-xlm-kfold-cv-0-93-lb</a></li>\n</ul>\n\n<h1>Notebooks Used For Submission</h1>\n\n<ul>\n<li><a href=\"https://www.kaggle.com/khoongweihao/ensemble-ii-the-dark-side-of-stacking\">https://www.kaggle.com/khoongweihao/ensemble-ii-the-dark-side-of-stacking</a> <strong>(Mine)</strong></li>\n<li><a href=\"https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse\">https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse</a> <strong>(Public)</strong></li>\n</ul>\n\n<h1>Ensemble Methods That Did Not Work</h1>\n\n<ul>\n<li>Bi-weighted correlations stacking (something that stemmed from my personal research, which worked on other regression and classification problems and datasets, just not this one sadly. It actually worked at the start, but in the last month of the competition I could not find a suitable configuration due to the lack of submissions. See my notebook above for details)</li>\n<li>Power-weighted stacking/blend (intention was to penalize the classification probabilities, idea came from the winning solution of the previous Jigsaw competition. Sadly this did not work out too)</li>\n<li>Distance-weighted stacking (main idea is to give higher weights to predictions with lower distances such as correlation/euclidean distance. This did not work too)</li>\n</ul>",
  "messages": [
    {
      "id": "897759",
      "postDate": "06/23/2020 04:40:14",
      "content": "<h1>Overview</h1>\n\n<p>My two best submissions were from a stacking pipeline. I've trained several models on Kaggle and Colab, which I later stacked their predictions using various stacking techniques (not just the usual mean, median, minmax) to achieve higher LB scores. </p>\n\n<h2>The Two Best Submissions</h2>\n\n<ul>\n<li>Stacking &gt; 12 models (used mean of the predictions of 12 submission files, from 9 single models and 3 blends) <strong>(Private LB: 0.9469, Public LB: 0.9485)</strong></li>\n<li>Blend of stacked predictions from above that gave 0.9485 on the public LB with <code>ExtraTreesClassifier</code> predictions trained on the validation data with <a href=\"https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse/notebook\">https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse/notebook</a> <strong>(Private LB: 0.9471, Public LB: 0.9485)</strong></li>\n</ul>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1738095%2F78d1c6c510639908e3d10b481e1ec8ea%2Fbest_subs.png?generation=1592885422854659&amp;alt=media\" alt=\"\"></p>\n\n<h1>Models Used For Stacking</h1>\n\n<ul>\n<li>XLM-RoBERTa</li>\n<li>XLM-RoBERTa Large</li>\n<li>XLM-RoBERTa Large + MLM training</li>\n<li>ExtraTreesClassifier</li>\n</ul>\n\n<p>P.S. Might have missed out one or two, will update here should I find any missing.</p>\n\n<h1>How Models Were Trained</h1>\n\n<ul>\n<li>For the MLM model (third above), I trained it on Kaggle </li>\n<li>For the XLM-RoBERTa models, I trained them on Google Colab TPU</li>\n</ul>\n\n<h1>Notebooks With Models Referenced Above</h1>\n\n<p>A big thanks to all whom contributed to the following notebooks, you have taught me quite a lot! I'm still relatively new to NLP, and your sharing has made me more comfortable and interested in the domain! :)</p>\n\n<ul>\n<li><a href=\"https://www.kaggle.com/riblidezso/train-from-mlm-finetuned-xlm-roberta-large\">https://www.kaggle.com/riblidezso/train-from-mlm-finetuned-xlm-roberta-large</a> </li>\n<li><a href=\"https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse\">https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse</a></li>\n<li><a href=\"https://www.kaggle.com/hamditarek/ensemble\">https://www.kaggle.com/hamditarek/ensemble</a></li>\n<li><a href=\"https://www.kaggle.com/sai11fkaneko/data-leak\">https://www.kaggle.com/sai11fkaneko/data-leak</a></li>\n<li><a href=\"https://www.kaggle.com/shonenkov/tpu-inference-super-fast-xlmroberta\">https://www.kaggle.com/shonenkov/tpu-inference-super-fast-xlmroberta</a></li>\n<li><a href=\"https://www.kaggle.com/shonenkov/tpu-training-super-fast-xlmroberta\">https://www.kaggle.com/shonenkov/tpu-training-super-fast-xlmroberta</a></li>\n<li><a href=\"https://www.kaggle.com/aiaiooas/parcor-regularised-classification\">https://www.kaggle.com/aiaiooas/parcor-regularised-classification</a></li>\n<li><a href=\"https://www.kaggle.com/shahules/fine-tune-xlm-kfold-cv-0-93-lb\">https://www.kaggle.com/shahules/fine-tune-xlm-kfold-cv-0-93-lb</a></li>\n</ul>\n\n<h1>Notebooks Used For Submission</h1>\n\n<ul>\n<li><a href=\"https://www.kaggle.com/khoongweihao/ensemble-ii-the-dark-side-of-stacking\">https://www.kaggle.com/khoongweihao/ensemble-ii-the-dark-side-of-stacking</a> <strong>(Mine)</strong></li>\n<li><a href=\"https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse\">https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse</a> <strong>(Public)</strong></li>\n</ul>\n\n<h1>Ensemble Methods That Did Not Work</h1>\n\n<ul>\n<li>Bi-weighted correlations stacking (something that stemmed from my personal research, which worked on other regression and classification problems and datasets, just not this one sadly. It actually worked at the start, but in the last month of the competition I could not find a suitable configuration due to the lack of submissions. See my notebook above for details)</li>\n<li>Power-weighted stacking/blend (intention was to penalize the classification probabilities, idea came from the winning solution of the previous Jigsaw competition. Sadly this did not work out too)</li>\n<li>Distance-weighted stacking (main idea is to give higher weights to predictions with lower distances such as correlation/euclidean distance. This did not work too)</li>\n</ul>",
      "rawMarkdown": "# Overview\n\nMy two best submissions were from a stacking pipeline. I've trained several models on Kaggle and Colab, which I later stacked their predictions using various stacking techniques (not just the usual mean, median, minmax) to achieve higher LB scores. \n\n## The Two Best Submissions\n\n- Stacking &gt; 12 models (used mean of the predictions of 12 submission files, from 9 single models and 3 blends) **(Private LB: 0.9469, Public LB: 0.9485)**\n- Blend of stacked predictions from above that gave 0.9485 on the public LB with `ExtraTreesClassifier` predictions trained on the validation data with https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse/notebook **(Private LB: 0.9471, Public LB: 0.9485)**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1738095%2F78d1c6c510639908e3d10b481e1ec8ea%2Fbest_subs.png?generation=1592885422854659&amp;alt=media)\n\n# Models Used For Stacking\n\n- XLM-RoBERTa\n- XLM-RoBERTa Large\n- XLM-RoBERTa Large + MLM training\n- ExtraTreesClassifier\n\nP.S. Might have missed out one or two, will update here should I find any missing.\n\n# How Models Were Trained\n\n- For the MLM model (third above), I trained it on Kaggle \n- For the XLM-RoBERTa models, I trained them on Google Colab TPU\n\n# Notebooks With Models Referenced Above\n\nA big thanks to all whom contributed to the following notebooks, you have taught me quite a lot! I'm still relatively new to NLP, and your sharing has made me more comfortable and interested in the domain! :)\n\n- https://www.kaggle.com/riblidezso/train-from-mlm-finetuned-xlm-roberta-large \n- https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse\n- https://www.kaggle.com/hamditarek/ensemble\n- https://www.kaggle.com/sai11fkaneko/data-leak\n- https://www.kaggle.com/shonenkov/tpu-inference-super-fast-xlmroberta\n- https://www.kaggle.com/shonenkov/tpu-training-super-fast-xlmroberta\n- https://www.kaggle.com/aiaiooas/parcor-regularised-classification\n- https://www.kaggle.com/shahules/fine-tune-xlm-kfold-cv-0-93-lb\n\n# Notebooks Used For Submission\n\n- https://www.kaggle.com/khoongweihao/ensemble-ii-the-dark-side-of-stacking **(Mine)**\n- https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse **(Public)**\n\n# Ensemble Methods That Did Not Work\n\n- Bi-weighted correlations stacking (something that stemmed from my personal research, which worked on other regression and classification problems and datasets, just not this one sadly. It actually worked at the start, but in the last month of the competition I could not find a suitable configuration due to the lack of submissions. See my notebook above for details)\n- Power-weighted stacking/blend (intention was to penalize the classification probabilities, idea came from the winning solution of the previous Jigsaw competition. Sadly this did not work out too)\n- Distance-weighted stacking (main idea is to give higher weights to predictions with lower distances such as correlation/euclidean distance. This did not work too)",
      "votes": null
    },
    {
      "id": "897761",
      "postDate": "06/23/2020 04:42:10",
      "content": "<p>Weli done!</p>",
      "rawMarkdown": "Weli done!",
      "votes": null
    },
    {
      "id": "897764",
      "postDate": "06/23/2020 04:43:28",
      "content": "<p><a href=\"/haythemtellili5\">@haythemtellili5</a> Thank you bro! Congratulations to you and your team too!</p>",
      "rawMarkdown": "haythemtellili5 Thank you bro! Congratulations to you and your team too!",
      "votes": null
    },
    {
      "id": "897769",
      "postDate": "06/23/2020 04:52:54",
      "content": "<p>Good work Wei Hao. Congrats.! Dr.Patrick</p>",
      "rawMarkdown": "Good work Wei Hao. Congrats.! Dr.Patrick",
      "votes": null
    },
    {
      "id": "897779",
      "postDate": "06/23/2020 05:01:59",
      "content": "<p><a href=\"/drpatrickchan\">@drpatrickchan</a> Thank you! And congratulations to your team as well! Awesome 3rd :)</p>",
      "rawMarkdown": "drpatrickchan Thank you! And congratulations to your team as well! Awesome 3rd :)",
      "votes": null
    },
    {
      "id": "897794",
      "postDate": "06/23/2020 05:24:03",
      "content": "<p>congratz Wei!</p>",
      "rawMarkdown": "congratz Wei!",
      "votes": null
    },
    {
      "id": "897799",
      "postDate": "06/23/2020 05:26:52",
      "content": "<p><a href=\"/frtgnn\">@frtgnn</a> Thank you! Congratulations to your team too! 👍 😄 </p>",
      "rawMarkdown": "frtgnn Thank you! Congratulations to your team too! 👍 😄",
      "votes": null
    },
    {
      "id": "898894",
      "postDate": "06/23/2020 20:20:11",
      "content": "<p>Congratulations! Well deserved!</p>",
      "rawMarkdown": "Congratulations! Well deserved!",
      "votes": null
    },
    {
      "id": "899059",
      "postDate": "06/24/2020 00:03:37",
      "content": "<p><a href=\"/piyushmishra1999\">@piyushmishra1999</a> Thank you!</p>",
      "rawMarkdown": "piyushmishra1999 Thank you!",
      "votes": null
    },
    {
      "id": "900816",
      "postDate": "06/25/2020 04:46:12",
      "content": "<p>good job </p>",
      "rawMarkdown": "good job",
      "votes": null
    },
    {
      "id": "900834",
      "postDate": "06/25/2020 05:03:37",
      "content": "<p><a href=\"/mdselimreza\">@mdselimreza</a> Thank you mate</p>",
      "rawMarkdown": "mdselimreza Thank you mate",
      "votes": null
    },
    {
      "id": "900842",
      "postDate": "06/25/2020 05:13:39",
      "content": "<p>Well done. Congratulations!</p>",
      "rawMarkdown": "Well done. Congratulations!",
      "votes": null
    },
    {
      "id": "900976",
      "postDate": "06/25/2020 07:22:37",
      "content": "<p><a href=\"/cdeotte\">@cdeotte</a> Thank you! And congrats as well 👍 </p>",
      "rawMarkdown": "cdeotte Thank you! And congrats as well 👍",
      "votes": null
    },
    {
      "id": "902166",
      "postDate": "06/26/2020 00:50:01",
      "content": "<p>Great!</p>",
      "rawMarkdown": "Great!",
      "votes": null
    },
    {
      "id": "902526",
      "postDate": "06/26/2020 07:46:42",
      "content": "<p><a href=\"/rashidulhasanhridoy\">@rashidulhasanhridoy</a> Thank you!</p>",
      "rawMarkdown": "rashidulhasanhridoy Thank you!",
      "votes": null
    },
    {
      "id": "904340",
      "postDate": "06/27/2020 14:51:43",
      "content": "<p>Welcome Buddy!</p>",
      "rawMarkdown": "Welcome Buddy!",
      "votes": null
    },
    {
      "id": "904427",
      "postDate": "06/27/2020 16:03:11",
      "content": "<p>congratulations on your success</p>",
      "rawMarkdown": "congratulations on your success",
      "votes": null
    },
    {
      "id": "904458",
      "postDate": "06/27/2020 16:30:25",
      "content": "<p><a href=\"/kamruzzamanselim\">@kamruzzamanselim</a> Thank you!</p>",
      "rawMarkdown": "kamruzzamanselim Thank you!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 897761,
      "author_name": "haythemtellili5",
      "author_url": "",
      "post_date": "06/23/2020 04:42:10",
      "content": "<p>Weli done!</p>",
      "votes": null,
      "replies": [
        {
          "id": 897764,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/23/2020 04:43:28",
          "content": "<p><a href=\"/haythemtellili5\">@haythemtellili5</a> Thank you bro! Congratulations to you and your team too!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 897769,
      "author_name": "drpatrickchan",
      "author_url": "",
      "post_date": "06/23/2020 04:52:54",
      "content": "<p>Good work Wei Hao. Congrats.! Dr.Patrick</p>",
      "votes": null,
      "replies": [
        {
          "id": 897779,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/23/2020 05:01:59",
          "content": "<p><a href=\"/drpatrickchan\">@drpatrickchan</a> Thank you! And congratulations to your team as well! Awesome 3rd :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 897794,
      "author_name": "frtgnn",
      "author_url": "",
      "post_date": "06/23/2020 05:24:03",
      "content": "<p>congratz Wei!</p>",
      "votes": null,
      "replies": [
        {
          "id": 897799,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/23/2020 05:26:52",
          "content": "<p><a href=\"/frtgnn\">@frtgnn</a> Thank you! Congratulations to your team too! 👍 😄 </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 898894,
      "author_name": "piyushmishra1999",
      "author_url": "",
      "post_date": "06/23/2020 20:20:11",
      "content": "<p>Congratulations! Well deserved!</p>",
      "votes": null,
      "replies": [
        {
          "id": 899059,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/24/2020 00:03:37",
          "content": "<p><a href=\"/piyushmishra1999\">@piyushmishra1999</a> Thank you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 900816,
      "author_name": "mdselimreza",
      "author_url": "",
      "post_date": "06/25/2020 04:46:12",
      "content": "<p>good job </p>",
      "votes": null,
      "replies": [
        {
          "id": 900834,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/25/2020 05:03:37",
          "content": "<p><a href=\"/mdselimreza\">@mdselimreza</a> Thank you mate</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 904340,
          "author_name": "mdselimreza",
          "author_url": "",
          "post_date": "06/27/2020 14:51:43",
          "content": "<p>Welcome Buddy!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 900842,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "06/25/2020 05:13:39",
      "content": "<p>Well done. Congratulations!</p>",
      "votes": null,
      "replies": [
        {
          "id": 900976,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/25/2020 07:22:37",
          "content": "<p><a href=\"/cdeotte\">@cdeotte</a> Thank you! And congrats as well 👍 </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 902166,
      "author_name": "rashidulhasanhridoy",
      "author_url": "",
      "post_date": "06/26/2020 00:50:01",
      "content": "<p>Great!</p>",
      "votes": null,
      "replies": [
        {
          "id": 902526,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/26/2020 07:46:42",
          "content": "<p><a href=\"/rashidulhasanhridoy\">@rashidulhasanhridoy</a> Thank you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 904427,
      "author_name": "kamruzzamanselim",
      "author_url": "",
      "post_date": "06/27/2020 16:03:11",
      "content": "<p>congratulations on your success</p>",
      "votes": null,
      "replies": [
        {
          "id": 904458,
          "author_name": "khoongweihao",
          "author_url": "",
          "post_date": "06/27/2020 16:30:25",
          "content": "<p><a href=\"/kamruzzamanselim\">@kamruzzamanselim</a> Thank you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "897759": "# Overview\n\nMy two best submissions were from a stacking pipeline. I've trained several models on Kaggle and Colab, which I later stacked their predictions using various stacking techniques (not just the usual mean, median, minmax) to achieve higher LB scores. \n\n## The Two Best Submissions\n\n- Stacking &gt; 12 models (used mean of the predictions of 12 submission files, from 9 single models and 3 blends) **(Private LB: 0.9469, Public LB: 0.9485)**\n- Blend of stacked predictions from above that gave 0.9485 on the public LB with `ExtraTreesClassifier` predictions trained on the validation data with https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse/notebook **(Private LB: 0.9471, Public LB: 0.9485)**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1738095%2F78d1c6c510639908e3d10b481e1ec8ea%2Fbest_subs.png?generation=1592885422854659&amp;alt=media)\n\n# Models Used For Stacking\n\n- XLM-RoBERTa\n- XLM-RoBERTa Large\n- XLM-RoBERTa Large + MLM training\n- ExtraTreesClassifier\n\nP.S. Might have missed out one or two, will update here should I find any missing.\n\n# How Models Were Trained\n\n- For the MLM model (third above), I trained it on Kaggle \n- For the XLM-RoBERTa models, I trained them on Google Colab TPU\n\n# Notebooks With Models Referenced Above\n\nA big thanks to all whom contributed to the following notebooks, you have taught me quite a lot! I'm still relatively new to NLP, and your sharing has made me more comfortable and interested in the domain! :)\n\n- https://www.kaggle.com/riblidezso/train-from-mlm-finetuned-xlm-roberta-large \n- https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse\n- https://www.kaggle.com/hamditarek/ensemble\n- https://www.kaggle.com/sai11fkaneko/data-leak\n- https://www.kaggle.com/shonenkov/tpu-inference-super-fast-xlmroberta\n- https://www.kaggle.com/shonenkov/tpu-training-super-fast-xlmroberta\n- https://www.kaggle.com/aiaiooas/parcor-regularised-classification\n- https://www.kaggle.com/shahules/fine-tune-xlm-kfold-cv-0-93-lb\n\n# Notebooks Used For Submission\n\n- https://www.kaggle.com/khoongweihao/ensemble-ii-the-dark-side-of-stacking **(Mine)**\n- https://www.kaggle.com/jazivxt/howling-with-wolf-on-l-genpresse **(Public)**\n\n# Ensemble Methods That Did Not Work\n\n- Bi-weighted correlations stacking (something that stemmed from my personal research, which worked on other regression and classification problems and datasets, just not this one sadly. It actually worked at the start, but in the last month of the competition I could not find a suitable configuration due to the lack of submissions. See my notebook above for details)\n- Power-weighted stacking/blend (intention was to penalize the classification probabilities, idea came from the winning solution of the previous Jigsaw competition. Sadly this did not work out too)\n- Distance-weighted stacking (main idea is to give higher weights to predictions with lower distances such as correlation/euclidean distance. This did not work too)",
    "897761": "Weli done!",
    "897764": "haythemtellili5 Thank you bro! Congratulations to you and your team too!",
    "897769": "Good work Wei Hao. Congrats.! Dr.Patrick",
    "897779": "drpatrickchan Thank you! And congratulations to your team as well! Awesome 3rd :)",
    "897794": "congratz Wei!",
    "897799": "frtgnn Thank you! Congratulations to your team too! 👍 😄",
    "898894": "Congratulations! Well deserved!",
    "899059": "piyushmishra1999 Thank you!",
    "900816": "good job",
    "900834": "mdselimreza Thank you mate",
    "900842": "Well done. Congratulations!",
    "900976": "cdeotte Thank you! And congrats as well 👍",
    "902166": "Great!",
    "902526": "rashidulhasanhridoy Thank you!",
    "904340": "Welcome Buddy!",
    "904427": "congratulations on your success",
    "904458": "kamruzzamanselim Thank you!"
  },
  "source": "meta"
}