{
  "id": 80495,
  "title": "3rd place kernel",
  "url": "/competitions/quora-insincere-questions-classification/writeups/guanshuo-xu-3rd-place-kernel",
  "author_name": "",
  "post_date": "2019-02-14T03:57:01.920Z",
  "votes": 124,
  "comment_count": 47,
  "views": 0,
  "content": "<p>Hi,\nI have published the 3rd place kernel.\n<a href=\"https://www.kaggle.com/wowfattie/3rd-place\">https://www.kaggle.com/wowfattie/3rd-place</a></p>\n\n<p>I used a lot of others works. The key factors of my method are:\n- Spacy tokenizer\n- No truncation of tokens\n- Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\n- 2 layer of globalmaxpooling\n- checkpoint ensemble\n- Local solid CV to tune all the hyperparameters</p>\n\n<p>Questions, advises, suggestions are all welcome.</p>\n\n<p>EDIT: I forgot to mention that all the punctuations are included.  \"if token.pos_ is not \"PUNCT\" has no actual effect</p>",
  "messages": [
    {
      "id": "471106",
      "postDate": "02/14/2019 03:17:57",
      "content": "<p>Hi,\nI have published the 3rd place kernel.\n<a href=\"https://www.kaggle.com/wowfattie/3rd-place\">https://www.kaggle.com/wowfattie/3rd-place</a></p>\n\n<p>I used a lot of others works. The key factors of my method are:\n- Spacy tokenizer\n- No truncation of tokens\n- Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\n- 2 layer of globalmaxpooling\n- checkpoint ensemble\n- Local solid CV to tune all the hyperparameters</p>\n\n<p>Questions, advises, suggestions are all welcome.</p>\n\n<p>EDIT: I forgot to mention that all the punctuations are included.  \"if token.pos_ is not \"PUNCT\" has no actual effect</p>",
      "rawMarkdown": "Hi,\nI have published the 3rd place kernel.\nhttps://www.kaggle.com/wowfattie/3rd-place\n\nI used a lot of others works. The key factors of my method are:\n- Spacy tokenizer\n- No truncation of tokens\n- Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\n- 2 layer of globalmaxpooling\n- checkpoint ensemble\n- Local solid CV to tune all the hyperparameters\n\nQuestions, advises, suggestions are all welcome.\n\nEDIT: I forgot to mention that all the punctuations are included.  \"if token.pos_ is not \"PUNCT\" has no actual effect",
      "votes": null
    },
    {
      "id": "471107",
      "postDate": "02/14/2019 03:28:30",
      "content": "<p>Congrats! does Spacy tokenizer perform considerably better than keras or nltk tokenizer for this data? </p>",
      "rawMarkdown": "Congrats! does Spacy tokenizer perform considerably better than keras or nltk tokenizer for this data?",
      "votes": null
    },
    {
      "id": "471110",
      "postDate": "02/14/2019 03:34:07",
      "content": "<p>I did not try keras tokenizer, so no direct comparison. But I did read the code of keras tokenizer, and there's nothing fancy of it. So I believed spacy tokenzier should be at least as good as keras's if not a lot better</p>",
      "rawMarkdown": "I did not try keras tokenizer, so no direct comparison. But I did read the code of keras tokenizer, and there's nothing fancy of it. So I believed spacy tokenzier should be at least as good as keras's if not a lot better",
      "votes": null
    },
    {
      "id": "471115",
      "postDate": "02/14/2019 03:38:08",
      "content": "<p>Wow, amazing Kernel! Your solution is simple and effective!\nI think you only use few data cleaning (unlike those{aren't: are not ......}) Is that the key promotion in this competition? Is that means heavily data cleaning do will hurt performance?\nThanks!</p>",
      "rawMarkdown": "Wow, amazing Kernel! Your solution is simple and effective!\nI think you only use few data cleaning (unlike those{aren't: are not ......}) Is that the key promotion in this competition? Is that means heavily data cleaning do will hurt performance?\nThanks!",
      "votes": null
    },
    {
      "id": "471118",
      "postDate": "02/14/2019 03:41:14",
      "content": "<p>split puncts then keras tokenizer will be better than Spacy tokenizer</p>",
      "rawMarkdown": "split puncts then keras tokenizer will be better than Spacy tokenizer",
      "votes": null
    },
    {
      "id": "471120",
      "postDate": "02/14/2019 03:46:20",
      "content": "<p>hmm.. Depends on how to define data cleaning. Note that I did use stemmers, spell correcters and so on to process. Your example is more like manual cleaning and may not cover as much. And I don't think fixings like changing \"aren't\" to \"are not\" relate with our target.</p>",
      "rawMarkdown": "hmm.. Depends on how to define data cleaning. Note that I did use stemmers, spell correcters and so on to process. Your example is more like manual cleaning and may not cover as much. And I don't think fixings like changing \"aren't\" to \"are not\" relate with our target.",
      "votes": null
    },
    {
      "id": "471121",
      "postDate": "02/14/2019 03:47:32",
      "content": "<p>I guess the key takeaway is on the stemmer, lemmatizer, spell correcter, etc. to find word vectors</p>",
      "rawMarkdown": "I guess the key takeaway is on the stemmer, lemmatizer, spell correcter, etc. to find word vectors",
      "votes": null
    },
    {
      "id": "471123",
      "postDate": "02/14/2019 03:49:24",
      "content": "<p>Congrats! I try your ways to find word vectors but no improvement in golve*0.64:params*0.36 embed weights.</p>",
      "rawMarkdown": "Congrats! I try your ways to find word vectors but no improvement in golve*0.64:params*0.36 embed weights.",
      "votes": null
    },
    {
      "id": "471124",
      "postDate": "02/14/2019 03:49:34",
      "content": "<p><a href=\"/canming\">@canming</a> Do you have any idea why? I don't see anything keras can do but spacy cannot</p>",
      "rawMarkdown": "canming Do you have any idea why? I don't see anything keras can do but spacy cannot",
      "votes": null
    },
    {
      "id": "471125",
      "postDate": "02/14/2019 03:50:15",
      "content": "<p>Thank you for sharing an amazing solution!</p>\n\n<p>I looked at the code, are you not using AttentionWeightedAverage?</p>",
      "rawMarkdown": "Thank you for sharing an amazing solution!\n\nI looked at the code, are you not using AttentionWeightedAverage?",
      "votes": null
    },
    {
      "id": "471127",
      "postDate": "02/14/2019 03:52:47",
      "content": "<p>No, I'm not. I forgot to delete that part</p>",
      "rawMarkdown": "No, I'm not. I forgot to delete that part",
      "votes": null
    },
    {
      "id": "471128",
      "postDate": "02/14/2019 03:55:03",
      "content": "<p>em...I don't transfrom the oov words to find word vectors. :)</p>",
      "rawMarkdown": "em...I don't transfrom the oov words to find word vectors. :)",
      "votes": null
    },
    {
      "id": "471133",
      "postDate": "02/14/2019 04:05:01",
      "content": "<p>Thanks for your solution!</p>",
      "rawMarkdown": "Thanks for your solution!",
      "votes": null
    },
    {
      "id": "471134",
      "postDate": "02/14/2019 04:10:01",
      "content": "<p>Congrats! We also used Spacy. Originally, it's better than keras tokenizer.</p>",
      "rawMarkdown": "Congrats! We also used Spacy. Originally, it's better than keras tokenizer.",
      "votes": null
    },
    {
      "id": "471153",
      "postDate": "02/14/2019 04:49:00",
      "content": "<p>Congrats! nice kernel.  </p>",
      "rawMarkdown": "Congrats! nice kernel.",
      "votes": null
    },
    {
      "id": "471188",
      "postDate": "02/14/2019 06:11:07",
      "content": "<p>Congratulations, I have learned a lot in this competition. Any post-processing on the results?</p>",
      "rawMarkdown": "Congratulations, I have learned a lot in this competition. Any post-processing on the results?",
      "votes": null
    },
    {
      "id": "471223",
      "postDate": "02/14/2019 06:51:44",
      "content": "<p>Congrats!! Thanks for your share</p>",
      "rawMarkdown": "Congrats!! Thanks for your share",
      "votes": null
    },
    {
      "id": "471227",
      "postDate": "02/14/2019 06:55:24",
      "content": "<p>Thanks for your sharing! You did a lot of work on tokens and your embedding matrix must be better than the public kernel.\nI wonder how much improvement does these three factors \"Spacy tokenizer\", \"Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\" and \"checkpoint ensemble\" gives respectively?</p>",
      "rawMarkdown": "Thanks for your sharing! You did a lot of work on tokens and your embedding matrix must be better than the public kernel.\nI wonder how much improvement does these three factors \"Spacy tokenizer\", \"Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\" and \"checkpoint ensemble\" gives respectively?",
      "votes": null
    },
    {
      "id": "471234",
      "postDate": "02/14/2019 07:11:14",
      "content": "<p>Congrats!  Same ideas like \nSpacy tokenizer/No truncation of tokens/Try stemmer, lemmatizer, spell correcter, etc. to find word vectors</p>",
      "rawMarkdown": "Congrats!  Same ideas like \nSpacy tokenizer/No truncation of tokens/Try stemmer, lemmatizer, spell correcter, etc. to find word vectors",
      "votes": null
    },
    {
      "id": "471294",
      "postDate": "02/14/2019 09:06:37",
      "content": "<p>Thank you so much for sharing your great kernel and congratulations!</p>\n\n<p>Would you share local CV part of code if possible? thanks!</p>",
      "rawMarkdown": "Thank you so much for sharing your great kernel and congratulations!\n\nWould you share local CV part of code if possible? thanks!",
      "votes": null
    },
    {
      "id": "471333",
      "postDate": "02/14/2019 10:03:58",
      "content": "<p>Thank you for sharing! And Congrats! </p>",
      "rawMarkdown": "Thank you for sharing! And Congrats!",
      "votes": null
    },
    {
      "id": "471336",
      "postDate": "02/14/2019 10:07:33",
      "content": "<p>Congrats!!\nThe work you are focusing on is exactly what we lack. Learn from you.</p>",
      "rawMarkdown": "Congrats!!\nThe work you are focusing on is exactly what we lack. Learn from you.",
      "votes": null
    },
    {
      "id": "471343",
      "postDate": "02/14/2019 10:22:48",
      "content": "<p>Congratulations! It looks like you put a lot of time and thought into getting the word from the embeddings rather than using lots of fancy models - and it really paid off! Did you always expect this to work? or was it something you realised after experimentation with a few ideas?</p>\n\n<p>Also your code is really easy to follow!</p>",
      "rawMarkdown": "Congratulations! It looks like you put a lot of time and thought into getting the word from the embeddings rather than using lots of fancy models - and it really paid off! Did you always expect this to work? or was it something you realised after experimentation with a few ideas?\n\nAlso your code is really easy to follow!",
      "votes": null
    },
    {
      "id": "471386",
      "postDate": "02/14/2019 11:35:11",
      "content": "<p><a href=\"/wowfattie\">@wowfattie</a> congrats and thanks for sharing your solution.</p>",
      "rawMarkdown": "wowfattie congrats and thanks for sharing your solution.",
      "votes": null
    },
    {
      "id": "471436",
      "postDate": "02/14/2019 13:04:30",
      "content": "<p>Congrats! Thank you for sharing great kernel!</p>",
      "rawMarkdown": "Congrats! Thank you for sharing great kernel!",
      "votes": null
    },
    {
      "id": "471448",
      "postDate": "02/14/2019 13:36:47",
      "content": "<p>Yes, I expected this to work before I coded this, and I confirmed the effectiveness by experiments. Finding as many word vectors as possible is one of the keys for this competition because 1) word vectors were obtained by external data which means we train on much larger data (legally), 2) word vectors have attribution to cluster similar words which means better generalization to unseen words.</p>",
      "rawMarkdown": "Yes, I expected this to work before I coded this, and I confirmed the effectiveness by experiments. Finding as many word vectors as possible is one of the keys for this competition because 1) word vectors were obtained by external data which means we train on much larger data (legally), 2) word vectors have attribution to cluster similar words which means better generalization to unseen words.",
      "votes": null
    },
    {
      "id": "471450",
      "postDate": "02/14/2019 13:40:13",
      "content": "<p>Just routing 5fold CV. In the submission kernel I train on all data due to the time limit </p>",
      "rawMarkdown": "Just routing 5fold CV. In the submission kernel I train on all data due to the time limit",
      "votes": null
    },
    {
      "id": "471452",
      "postDate": "02/14/2019 13:43:29",
      "content": "<p>Sorry I'm unable to provide exact numbers about improvement. You can have some try in the kernel?</p>",
      "rawMarkdown": "Sorry I'm unable to provide exact numbers about improvement. You can have some try in the kernel?",
      "votes": null
    },
    {
      "id": "471454",
      "postDate": "02/14/2019 13:44:38",
      "content": "<p>No post-processing after model predictions. Just weighted average of predicted probabilities and threshold</p>",
      "rawMarkdown": "No post-processing after model predictions. Just weighted average of predicted probabilities and threshold",
      "votes": null
    },
    {
      "id": "471478",
      "postDate": "02/14/2019 14:17:07",
      "content": "<p>Congrats\nThanks for your sharing</p>",
      "rawMarkdown": "Congrats\nThanks for your sharing",
      "votes": null
    },
    {
      "id": "471626",
      "postDate": "02/14/2019 17:17:14",
      "content": "<p>Great solution and congratz! We also tried checkpoint ensembling a bit but never really did rigorous experiments specifically in terms of weighting them which seems like a great idea!</p>",
      "rawMarkdown": "Great solution and congratz! We also tried checkpoint ensembling a bit but never really did rigorous experiments specifically in terms of weighting them which seems like a great idea!",
      "votes": null
    },
    {
      "id": "471809",
      "postDate": "02/14/2019 22:49:54",
      "content": "<p>Congrats! Simple is better. and. Trust CV is the key.</p>",
      "rawMarkdown": "Congrats! Simple is better. and. Trust CV is the key.",
      "votes": null
    },
    {
      "id": "471887",
      "postDate": "02/15/2019 03:22:26",
      "content": "<p>Very well done...Congrats!</p>",
      "rawMarkdown": "Very well done...Congrats!",
      "votes": null
    },
    {
      "id": "472493",
      "postDate": "02/16/2019 03:01:49",
      "content": "<p>simple yet effective， thanks for sharing</p>",
      "rawMarkdown": "simple yet effective， thanks for sharing",
      "votes": null
    },
    {
      "id": "472949",
      "postDate": "02/17/2019 00:43:39",
      "content": "<p>Thanks for sharing the solution !! Fantastic Clean looking code !!</p>",
      "rawMarkdown": "Thanks for sharing the solution !! Fantastic Clean looking code !!",
      "votes": null
    },
    {
      "id": "472982",
      "postDate": "02/17/2019 03:01:11",
      "content": "<p>Thanks for your sharing.The code is amazing.</p>",
      "rawMarkdown": "Thanks for your sharing.The code is amazing.",
      "votes": null
    },
    {
      "id": "474555",
      "postDate": "02/19/2019 14:43:33",
      "content": "<p>Thanks for sharing! Amazing kernel. </p>",
      "rawMarkdown": "Thanks for sharing! Amazing kernel.",
      "votes": null
    },
    {
      "id": "475044",
      "postDate": "02/20/2019 06:48:35",
      "content": "<p>大神! Thanks for your shared code.</p>",
      "rawMarkdown": "大神! Thanks for your shared code.",
      "votes": null
    },
    {
      "id": "478116",
      "postDate": "02/25/2019 18:55:56",
      "content": "<p>Thanks for sharing!</p>",
      "rawMarkdown": "Thanks for sharing!",
      "votes": null
    },
    {
      "id": "478352",
      "postDate": "02/26/2019 03:57:03",
      "content": "<p>Thanks for your sharing! It's cool!</p>",
      "rawMarkdown": "Thanks for your sharing! It's cool!",
      "votes": null
    },
    {
      "id": "478679",
      "postDate": "02/26/2019 13:32:56",
      "content": "<p>Thanks for sharing! How did you come up with the method of checkpoint ensemble?</p>",
      "rawMarkdown": "Thanks for sharing! How did you come up with the method of checkpoint ensemble?",
      "votes": null
    },
    {
      "id": "478704",
      "postDate": "02/26/2019 14:01:41",
      "content": "<p>It's a common ensemble method. See <a href=\"http://cs231n.github.io/neural-networks-3/#ensemble\">http://cs231n.github.io/neural-networks-3/#ensemble</a></p>",
      "rawMarkdown": "It's a common ensemble method. See http://cs231n.github.io/neural-networks-3/#ensemble",
      "votes": null
    },
    {
      "id": "481294",
      "postDate": "03/01/2019 08:17:38",
      "content": "<p>Thanks for sharing Guanshuo Xu.</p>",
      "rawMarkdown": "Thanks for sharing Guanshuo Xu.",
      "votes": null
    },
    {
      "id": "483353",
      "postDate": "03/04/2019 13:51:50",
      "content": "<p>Hi Guanshuo \nThanks for sharing a great kernel, however I would like to know how you decided on .15 and .35 for ensembling the prediction. Can you also share the code CV ? You also wrote the class for about attention layer, but you did not use it?</p>",
      "rawMarkdown": "Hi Guanshuo \nThanks for sharing a great kernel, however I would like to know how you decided on .15 and .35 for ensembling the prediction. Can you also share the code CV ? You also wrote the class for about attention layer, but you did not use it?",
      "votes": null
    },
    {
      "id": "485262",
      "postDate": "03/07/2019 06:34:35",
      "content": "<p>Hey Guanshuo,  it would really be great if you can please share the CV code.</p>",
      "rawMarkdown": "Hey Guanshuo,  it would really be great if you can please share the CV code.",
      "votes": null
    },
    {
      "id": "493907",
      "postDate": "03/19/2019 08:26:33",
      "content": "<p>Thanks for your sharing. Nice work.</p>",
      "rawMarkdown": "Thanks for your sharing. Nice work.",
      "votes": null
    },
    {
      "id": "548808",
      "postDate": "06/09/2019 23:53:06",
      "content": "<p>Thanks a ton!</p>",
      "rawMarkdown": "Thanks a ton!",
      "votes": null
    },
    {
      "id": "601758",
      "postDate": "08/18/2019 05:42:21",
      "content": "<p>Congrats! That's quite helpful! Thanks so much for sharing. </p>",
      "rawMarkdown": "Congrats! That's quite helpful! Thanks so much for sharing.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 471107,
      "author_name": "dicksonchin93",
      "author_url": "",
      "post_date": "02/14/2019 03:28:30",
      "content": "<p>Congrats! does Spacy tokenizer perform considerably better than keras or nltk tokenizer for this data? </p>",
      "votes": null,
      "replies": [
        {
          "id": 471110,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 03:34:07",
          "content": "<p>I did not try keras tokenizer, so no direct comparison. But I did read the code of keras tokenizer, and there's nothing fancy of it. So I believed spacy tokenzier should be at least as good as keras's if not a lot better</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 471118,
          "author_name": "canming",
          "author_url": "",
          "post_date": "02/14/2019 03:41:14",
          "content": "<p>split puncts then keras tokenizer will be better than Spacy tokenizer</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 471121,
          "author_name": "dicksonchin93",
          "author_url": "",
          "post_date": "02/14/2019 03:47:32",
          "content": "<p>I guess the key takeaway is on the stemmer, lemmatizer, spell correcter, etc. to find word vectors</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 471124,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 03:49:34",
          "content": "<p><a href=\"/canming\">@canming</a> Do you have any idea why? I don't see anything keras can do but spacy cannot</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 471128,
          "author_name": "canming",
          "author_url": "",
          "post_date": "02/14/2019 03:55:03",
          "content": "<p>em...I don't transfrom the oov words to find word vectors. :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 471115,
      "author_name": "dilapsky",
      "author_url": "",
      "post_date": "02/14/2019 03:38:08",
      "content": "<p>Wow, amazing Kernel! Your solution is simple and effective!\nI think you only use few data cleaning (unlike those{aren't: are not ......}) Is that the key promotion in this competition? Is that means heavily data cleaning do will hurt performance?\nThanks!</p>",
      "votes": null,
      "replies": [
        {
          "id": 471120,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 03:46:20",
          "content": "<p>hmm.. Depends on how to define data cleaning. Note that I did use stemmers, spell correcters and so on to process. Your example is more like manual cleaning and may not cover as much. And I don't think fixings like changing \"aren't\" to \"are not\" relate with our target.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 471123,
      "author_name": "canming",
      "author_url": "",
      "post_date": "02/14/2019 03:49:24",
      "content": "<p>Congrats! I try your ways to find word vectors but no improvement in golve*0.64:params*0.36 embed weights.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471125,
      "author_name": "atsunorifujita",
      "author_url": "",
      "post_date": "02/14/2019 03:50:15",
      "content": "<p>Thank you for sharing an amazing solution!</p>\n\n<p>I looked at the code, are you not using AttentionWeightedAverage?</p>",
      "votes": null,
      "replies": [
        {
          "id": 471127,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 03:52:47",
          "content": "<p>No, I'm not. I forgot to delete that part</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 471133,
      "author_name": "strideradu",
      "author_url": "",
      "post_date": "02/14/2019 04:05:01",
      "content": "<p>Thanks for your solution!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471134,
      "author_name": "laevatein",
      "author_url": "",
      "post_date": "02/14/2019 04:10:01",
      "content": "<p>Congrats! We also used Spacy. Originally, it's better than keras tokenizer.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471153,
      "author_name": "alexyung757",
      "author_url": "",
      "post_date": "02/14/2019 04:49:00",
      "content": "<p>Congrats! nice kernel.  </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471188,
      "author_name": "karthik7395",
      "author_url": "",
      "post_date": "02/14/2019 06:11:07",
      "content": "<p>Congratulations, I have learned a lot in this competition. Any post-processing on the results?</p>",
      "votes": null,
      "replies": [
        {
          "id": 471454,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 13:44:38",
          "content": "<p>No post-processing after model predictions. Just weighted average of predicted probabilities and threshold</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 471223,
      "author_name": "super13579",
      "author_url": "",
      "post_date": "02/14/2019 06:51:44",
      "content": "<p>Congrats!! Thanks for your share</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471227,
      "author_name": "wxytalent",
      "author_url": "",
      "post_date": "02/14/2019 06:55:24",
      "content": "<p>Thanks for your sharing! You did a lot of work on tokens and your embedding matrix must be better than the public kernel.\nI wonder how much improvement does these three factors \"Spacy tokenizer\", \"Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\" and \"checkpoint ensemble\" gives respectively?</p>",
      "votes": null,
      "replies": [
        {
          "id": 471452,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 13:43:29",
          "content": "<p>Sorry I'm unable to provide exact numbers about improvement. You can have some try in the kernel?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 471234,
      "author_name": "baomengjiao",
      "author_url": "",
      "post_date": "02/14/2019 07:11:14",
      "content": "<p>Congrats!  Same ideas like \nSpacy tokenizer/No truncation of tokens/Try stemmer, lemmatizer, spell correcter, etc. to find word vectors</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471294,
      "author_name": "higepon",
      "author_url": "",
      "post_date": "02/14/2019 09:06:37",
      "content": "<p>Thank you so much for sharing your great kernel and congratulations!</p>\n\n<p>Would you share local CV part of code if possible? thanks!</p>",
      "votes": null,
      "replies": [
        {
          "id": 471450,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 13:40:13",
          "content": "<p>Just routing 5fold CV. In the submission kernel I train on all data due to the time limit </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 471333,
      "author_name": "soloway",
      "author_url": "",
      "post_date": "02/14/2019 10:03:58",
      "content": "<p>Thank you for sharing! And Congrats! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471336,
      "author_name": "xiaobai1123q",
      "author_url": "",
      "post_date": "02/14/2019 10:07:33",
      "content": "<p>Congrats!!\nThe work you are focusing on is exactly what we lack. Learn from you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471343,
      "author_name": "hamishdickson",
      "author_url": "",
      "post_date": "02/14/2019 10:22:48",
      "content": "<p>Congratulations! It looks like you put a lot of time and thought into getting the word from the embeddings rather than using lots of fancy models - and it really paid off! Did you always expect this to work? or was it something you realised after experimentation with a few ideas?</p>\n\n<p>Also your code is really easy to follow!</p>",
      "votes": null,
      "replies": [
        {
          "id": 471448,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/14/2019 13:36:47",
          "content": "<p>Yes, I expected this to work before I coded this, and I confirmed the effectiveness by experiments. Finding as many word vectors as possible is one of the keys for this competition because 1) word vectors were obtained by external data which means we train on much larger data (legally), 2) word vectors have attribution to cluster similar words which means better generalization to unseen words.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 471386,
      "author_name": "sheriytm",
      "author_url": "",
      "post_date": "02/14/2019 11:35:11",
      "content": "<p><a href=\"/wowfattie\">@wowfattie</a> congrats and thanks for sharing your solution.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471436,
      "author_name": "donjyarahoi",
      "author_url": "",
      "post_date": "02/14/2019 13:04:30",
      "content": "<p>Congrats! Thank you for sharing great kernel!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471478,
      "author_name": "biedlin",
      "author_url": "",
      "post_date": "02/14/2019 14:17:07",
      "content": "<p>Congrats\nThanks for your sharing</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471626,
      "author_name": "philippsinger",
      "author_url": "",
      "post_date": "02/14/2019 17:17:14",
      "content": "<p>Great solution and congratz! We also tried checkpoint ensembling a bit but never really did rigorous experiments specifically in terms of weighting them which seems like a great idea!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471809,
      "author_name": "jmourad100",
      "author_url": "",
      "post_date": "02/14/2019 22:49:54",
      "content": "<p>Congrats! Simple is better. and. Trust CV is the key.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 471887,
      "author_name": "mkhades",
      "author_url": "",
      "post_date": "02/15/2019 03:22:26",
      "content": "<p>Very well done...Congrats!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 472493,
      "author_name": "zjucor",
      "author_url": "",
      "post_date": "02/16/2019 03:01:49",
      "content": "<p>simple yet effective， thanks for sharing</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 472949,
      "author_name": "viswanathravindran",
      "author_url": "",
      "post_date": "02/17/2019 00:43:39",
      "content": "<p>Thanks for sharing the solution !! Fantastic Clean looking code !!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 472982,
      "author_name": "guoyan",
      "author_url": "",
      "post_date": "02/17/2019 03:01:11",
      "content": "<p>Thanks for your sharing.The code is amazing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 474555,
      "author_name": "amberyesohyes",
      "author_url": "",
      "post_date": "02/19/2019 14:43:33",
      "content": "<p>Thanks for sharing! Amazing kernel. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 475044,
      "author_name": "zhuflower",
      "author_url": "",
      "post_date": "02/20/2019 06:48:35",
      "content": "<p>大神! Thanks for your shared code.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 478116,
      "author_name": "",
      "author_url": "",
      "post_date": "02/25/2019 18:55:56",
      "content": "<p>Thanks for sharing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 478352,
      "author_name": "mnk812",
      "author_url": "",
      "post_date": "02/26/2019 03:57:03",
      "content": "<p>Thanks for your sharing! It's cool!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 478679,
      "author_name": "leonmaogw",
      "author_url": "",
      "post_date": "02/26/2019 13:32:56",
      "content": "<p>Thanks for sharing! How did you come up with the method of checkpoint ensemble?</p>",
      "votes": null,
      "replies": [
        {
          "id": 478704,
          "author_name": "wowfattie",
          "author_url": "",
          "post_date": "02/26/2019 14:01:41",
          "content": "<p>It's a common ensemble method. See <a href=\"http://cs231n.github.io/neural-networks-3/#ensemble\">http://cs231n.github.io/neural-networks-3/#ensemble</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 481294,
      "author_name": "ravitee",
      "author_url": "",
      "post_date": "03/01/2019 08:17:38",
      "content": "<p>Thanks for sharing Guanshuo Xu.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 483353,
      "author_name": "akhileshrai003",
      "author_url": "",
      "post_date": "03/04/2019 13:51:50",
      "content": "<p>Hi Guanshuo \nThanks for sharing a great kernel, however I would like to know how you decided on .15 and .35 for ensembling the prediction. Can you also share the code CV ? You also wrote the class for about attention layer, but you did not use it?</p>",
      "votes": null,
      "replies": [
        {
          "id": 485262,
          "author_name": "vaibhavattri",
          "author_url": "",
          "post_date": "03/07/2019 06:34:35",
          "content": "<p>Hey Guanshuo,  it would really be great if you can please share the CV code.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 493907,
      "author_name": "liaozhq",
      "author_url": "",
      "post_date": "03/19/2019 08:26:33",
      "content": "<p>Thanks for your sharing. Nice work.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 548808,
      "author_name": "",
      "author_url": "",
      "post_date": "06/09/2019 23:53:06",
      "content": "<p>Thanks a ton!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 601758,
      "author_name": "",
      "author_url": "",
      "post_date": "08/18/2019 05:42:21",
      "content": "<p>Congrats! That's quite helpful! Thanks so much for sharing. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "471106": "Hi,\nI have published the 3rd place kernel.\nhttps://www.kaggle.com/wowfattie/3rd-place\n\nI used a lot of others works. The key factors of my method are:\n- Spacy tokenizer\n- No truncation of tokens\n- Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\n- 2 layer of globalmaxpooling\n- checkpoint ensemble\n- Local solid CV to tune all the hyperparameters\n\nQuestions, advises, suggestions are all welcome.\n\nEDIT: I forgot to mention that all the punctuations are included.  \"if token.pos_ is not \"PUNCT\" has no actual effect",
    "471107": "Congrats! does Spacy tokenizer perform considerably better than keras or nltk tokenizer for this data?",
    "471110": "I did not try keras tokenizer, so no direct comparison. But I did read the code of keras tokenizer, and there's nothing fancy of it. So I believed spacy tokenzier should be at least as good as keras's if not a lot better",
    "471115": "Wow, amazing Kernel! Your solution is simple and effective!\nI think you only use few data cleaning (unlike those{aren't: are not ......}) Is that the key promotion in this competition? Is that means heavily data cleaning do will hurt performance?\nThanks!",
    "471118": "split puncts then keras tokenizer will be better than Spacy tokenizer",
    "471120": "hmm.. Depends on how to define data cleaning. Note that I did use stemmers, spell correcters and so on to process. Your example is more like manual cleaning and may not cover as much. And I don't think fixings like changing \"aren't\" to \"are not\" relate with our target.",
    "471121": "I guess the key takeaway is on the stemmer, lemmatizer, spell correcter, etc. to find word vectors",
    "471123": "Congrats! I try your ways to find word vectors but no improvement in golve*0.64:params*0.36 embed weights.",
    "471124": "canming Do you have any idea why? I don't see anything keras can do but spacy cannot",
    "471125": "Thank you for sharing an amazing solution!\n\nI looked at the code, are you not using AttentionWeightedAverage?",
    "471127": "No, I'm not. I forgot to delete that part",
    "471128": "em...I don't transfrom the oov words to find word vectors. :)",
    "471133": "Thanks for your solution!",
    "471134": "Congrats! We also used Spacy. Originally, it's better than keras tokenizer.",
    "471153": "Congrats! nice kernel.",
    "471188": "Congratulations, I have learned a lot in this competition. Any post-processing on the results?",
    "471223": "Congrats!! Thanks for your share",
    "471227": "Thanks for your sharing! You did a lot of work on tokens and your embedding matrix must be better than the public kernel.\nI wonder how much improvement does these three factors \"Spacy tokenizer\", \"Try stemmer, lemmatizer, spell correcter, etc. to find word vectors\" and \"checkpoint ensemble\" gives respectively?",
    "471234": "Congrats!  Same ideas like \nSpacy tokenizer/No truncation of tokens/Try stemmer, lemmatizer, spell correcter, etc. to find word vectors",
    "471294": "Thank you so much for sharing your great kernel and congratulations!\n\nWould you share local CV part of code if possible? thanks!",
    "471333": "Thank you for sharing! And Congrats!",
    "471336": "Congrats!!\nThe work you are focusing on is exactly what we lack. Learn from you.",
    "471343": "Congratulations! It looks like you put a lot of time and thought into getting the word from the embeddings rather than using lots of fancy models - and it really paid off! Did you always expect this to work? or was it something you realised after experimentation with a few ideas?\n\nAlso your code is really easy to follow!",
    "471386": "wowfattie congrats and thanks for sharing your solution.",
    "471436": "Congrats! Thank you for sharing great kernel!",
    "471448": "Yes, I expected this to work before I coded this, and I confirmed the effectiveness by experiments. Finding as many word vectors as possible is one of the keys for this competition because 1) word vectors were obtained by external data which means we train on much larger data (legally), 2) word vectors have attribution to cluster similar words which means better generalization to unseen words.",
    "471450": "Just routing 5fold CV. In the submission kernel I train on all data due to the time limit",
    "471452": "Sorry I'm unable to provide exact numbers about improvement. You can have some try in the kernel?",
    "471454": "No post-processing after model predictions. Just weighted average of predicted probabilities and threshold",
    "471478": "Congrats\nThanks for your sharing",
    "471626": "Great solution and congratz! We also tried checkpoint ensembling a bit but never really did rigorous experiments specifically in terms of weighting them which seems like a great idea!",
    "471809": "Congrats! Simple is better. and. Trust CV is the key.",
    "471887": "Very well done...Congrats!",
    "472493": "simple yet effective， thanks for sharing",
    "472949": "Thanks for sharing the solution !! Fantastic Clean looking code !!",
    "472982": "Thanks for your sharing.The code is amazing.",
    "474555": "Thanks for sharing! Amazing kernel.",
    "475044": "大神! Thanks for your shared code.",
    "478116": "Thanks for sharing!",
    "478352": "Thanks for your sharing! It's cool!",
    "478679": "Thanks for sharing! How did you come up with the method of checkpoint ensemble?",
    "478704": "It's a common ensemble method. See http://cs231n.github.io/neural-networks-3/#ensemble",
    "481294": "Thanks for sharing Guanshuo Xu.",
    "483353": "Hi Guanshuo \nThanks for sharing a great kernel, however I would like to know how you decided on .15 and .35 for ensembling the prediction. Can you also share the code CV ? You also wrote the class for about attention layer, but you did not use it?",
    "485262": "Hey Guanshuo,  it would really be great if you can please share the CV code.",
    "493907": "Thanks for your sharing. Nice work.",
    "548808": "Thanks a ton!",
    "601758": "Congrats! That's quite helpful! Thanks so much for sharing."
  },
  "source": "meta"
}