{
  "id": 76905,
  "title": "speed up your stemmer",
  "url": "/competitions/quora-insincere-questions-classification/discussion/76905",
  "author_name": "",
  "post_date": "2019-01-07T19:12:51.476478700Z",
  "votes": 10,
  "comment_count": 1,
  "views": 0,
  "content": "<p>In the last kernel, which name <a href=\"https://www.kaggle.com/syhens/speed-up-your-preprocessing\">speed up your preprocessing</a>, I use <code>in</code> instead of create text on every loop. And now, I'll help you speed up your stemmer use <code>in</code> again.</p>\n\n<pre><code>stemmer = PorterStemmer()\n\nclass FasterStemmer(object):\n    def __init__(self):\n        self.words = {}\n\n    def stem(self, x):\n        if x in self.words:\n            return self.words[x]\n        t = stemmer.stem(x)\n        self.words[x] = t\n        return t\n\nfaster_stemmer = FasterStemmer()\n</code></pre>\n\n<p>use <code>faster_stemmer.stem()</code> instead of <code>stemmer.stem()</code> can help your stem from <code>4min 33s</code> to <code>10.7 s</code> on whole train set.</p>\n\n<p>Notice: it eat memory. So, do not forget to delete the <code>faster_stemmer</code> once finished your stem.</p>",
  "messages": [
    {
      "id": "451851",
      "postDate": "01/07/2019 19:12:51",
      "content": "<p>In the last kernel, which name <a href=\"https://www.kaggle.com/syhens/speed-up-your-preprocessing\">speed up your preprocessing</a>, I use <code>in</code> instead of create text on every loop. And now, I'll help you speed up your stemmer use <code>in</code> again.</p>\n\n<pre><code>stemmer = PorterStemmer()\n\nclass FasterStemmer(object):\n    def __init__(self):\n        self.words = {}\n\n    def stem(self, x):\n        if x in self.words:\n            return self.words[x]\n        t = stemmer.stem(x)\n        self.words[x] = t\n        return t\n\nfaster_stemmer = FasterStemmer()\n</code></pre>\n\n<p>use <code>faster_stemmer.stem()</code> instead of <code>stemmer.stem()</code> can help your stem from <code>4min 33s</code> to <code>10.7 s</code> on whole train set.</p>\n\n<p>Notice: it eat memory. So, do not forget to delete the <code>faster_stemmer</code> once finished your stem.</p>",
      "rawMarkdown": "In the last kernel, which name [speed up your preprocessing](https://www.kaggle.com/syhens/speed-up-your-preprocessing), I use `in` instead of create text on every loop. And now, I'll help you speed up your stemmer use `in` again.\n\n    stemmer = PorterStemmer()\n\n    class FasterStemmer(object):\n        def __init__(self):\n            self.words = {}\n    \n        def stem(self, x):\n            if x in self.words:\n                return self.words[x]\n            t = stemmer.stem(x)\n            self.words[x] = t\n            return t\n\n    faster_stemmer = FasterStemmer()\n\n\n\nuse `faster_stemmer.stem()` instead of `stemmer.stem()` can help your stem from `4min 33s` to `10.7 s` on whole train set.\n\nNotice: it eat memory. So, do not forget to delete the `faster_stemmer` once finished your stem.",
      "votes": null
    },
    {
      "id": "451861",
      "postDate": "01/07/2019 19:31:16",
      "content": "<p>But I don't use stem _(:з」∠)_</p>",
      "rawMarkdown": "But I don't use stem \\_(:з」∠)\\_",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 451861,
      "author_name": "syhens",
      "author_url": "",
      "post_date": "01/07/2019 19:31:16",
      "content": "<p>But I don't use stem _(:з」∠)_</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "451851": "In the last kernel, which name [speed up your preprocessing](https://www.kaggle.com/syhens/speed-up-your-preprocessing), I use `in` instead of create text on every loop. And now, I'll help you speed up your stemmer use `in` again.\n\n    stemmer = PorterStemmer()\n\n    class FasterStemmer(object):\n        def __init__(self):\n            self.words = {}\n    \n        def stem(self, x):\n            if x in self.words:\n                return self.words[x]\n            t = stemmer.stem(x)\n            self.words[x] = t\n            return t\n\n    faster_stemmer = FasterStemmer()\n\n\n\nuse `faster_stemmer.stem()` instead of `stemmer.stem()` can help your stem from `4min 33s` to `10.7 s` on whole train set.\n\nNotice: it eat memory. So, do not forget to delete the `faster_stemmer` once finished your stem.",
    "451861": "But I don't use stem \\_(:з」∠)\\_"
  },
  "source": "meta"
}