As posted here, it seems that a forum topic for each external data will just fill the forum and make it hard to navigate. Post here all your external data, in a single topic, we can gather all together.
\n\nI do not have any current external data.
", "messages": [ { "id": "91268", "postDate": "09/01/2015 20:27:22", "content": "As posted here, it seems that a forum topic for each external data will just fill the forum and make it hard to navigate. Post here all your external data, in a single topic, we can gather all together.
\n\nI do not have any current external data.
", "rawMarkdown": "As posted [here][1], it seems that a forum topic for each external data will just fill the forum and make it hard to navigate. Post here all your external data, in a single topic, we can gather all together.\r\n\r\nI do not have any current external data.\r\n\r\n\r\n [1]: https://www.kaggle.com/c/dato-native/forums/t/16154/external-dictionaries/91264#post91264", "votes": null }, { "id": "91337", "postDate": "09/02/2015 13:49:19", "content": "I'm using the english stopwords from the TM package in R.
", "rawMarkdown": "I'm using the english [stopwords][1] from the [TM package][2] in R.\r\n\r\n\r\n [1]: http://www.inside-r.org/packages/cran/tm/docs/stopwords\r\n [2]: https://cran.r-project.org/web/packages/tm/index.html", "votes": null }, { "id": "91594", "postDate": "09/04/2015 16:53:55", "content": "I'm use the nltk english stopwords
", "rawMarkdown": "I'm use the [nltk english stopwords][1]\r\n\r\n\r\n [1]: http://www.nltk.org/nltk_data/", "votes": null }, { "id": "93767", "postDate": "10/01/2015 01:35:43", "content": "We use the file-extension-related data from the following sites:
\n\nWeb Files: http://fileinfo.com/filetypes/web
\n\nCommon Files: http://fileinfo.com/filetypes/common
", "rawMarkdown": "We use the file-extension-related data from the following sites:\r\n\r\nWeb Files: http://fileinfo.com/filetypes/web\r\n\r\nCommon Files: http://fileinfo.com/filetypes/common", "votes": null }, { "id": "93850", "postDate": "10/02/2015 03:53:22", "content": "Hey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?
\n\n[quote=Zach;91337]
\n\nI'm using the english stopwords from the TM package in R.
\n\n[/quote]
", "rawMarkdown": "Hey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?\r\n\r\n[quote=Zach;91337]\r\n\r\nI'm using the english [stopwords][1] from the [TM package][2] in R.\r\n\r\n\r\n [1]: http://www.inside-r.org/packages/cran/tm/docs/stopwords\r\n [2]: https://cran.r-project.org/web/packages/tm/index.html\r\n\r\n[/quote]", "votes": null }, { "id": "93896", "postDate": "10/02/2015 15:12:56", "content": "[quote=ask788;93850]
\n\nHey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?
\n\n[/quote]
\n\nThe TM package is way too slow and memory - hungry for this competition. You could try the tmlite package , but the dataset may still be too big.
\n\nI'm manually pre-processing each file, one at a time, and then manually assembling the pre-processed files into a document-term matrix.
\n\nIt's slooooooooooow but doesn't run out of RAM.
", "rawMarkdown": "[quote=ask788;93850]\r\n\r\nHey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?\r\n\r\n[/quote]\r\n\r\nThe TM package is way too slow and memory - hungry for this competition. You could try the [tmlite package][1] , but the dataset may still be too big.\r\n\r\nI'm manually pre-processing each file, one at a time, and then manually assembling the pre-processed files into a document-term matrix.\r\n\r\nIt's slooooooooooow but doesn't run out of RAM.\r\n\r\n\r\n [1]: https://github.com/dselivanov/tmlite", "votes": null }, { "id": "93897", "postDate": "10/02/2015 15:21:24", "content": "I'm using this list of stopwords: http://www.ranks.nl/stopwords ("Default English stopwords list")
", "rawMarkdown": "I'm using this list of stopwords: http://www.ranks.nl/stopwords (\"Default English stopwords list\")", "votes": null }, { "id": "95299", "postDate": "10/06/2015 19:35:54", "content": "I am using sklearn.feature_extraction.stop_words.ENGLISH_STOP_WORDS and nltk.corpus.stopwords.
", "rawMarkdown": "I am using sklearn.feature_extraction.stop_words.ENGLISH_STOP_WORDS and nltk.corpus.stopwords.", "votes": null }, { "id": "95311", "postDate": "10/06/2015 20:46:22", "content": "Already posted, but I am using NLTK stop words as well.
", "rawMarkdown": "Already posted, but I am using NLTK stop words as well.", "votes": null }, { "id": "95497", "postDate": "10/08/2015 13:41:08", "content": "https://www.kaggle.com/c/word2vec-nlp-tutorial/data from paper http://ai.stanford.edu/~ang/papers/acl11-WordVectorsSentimentAnalysis.pdf
\n\nand Newsgroups20 http://qwone.com/~jason/20Newsgroups/ via http://scikit-learn.org/stable/datasets/twenty_newsgroups.html
", "rawMarkdown": "https://www.kaggle.com/c/word2vec-nlp-tutorial/data from paper http://ai.stanford.edu/~ang/papers/acl11-WordVectorsSentimentAnalysis.pdf\r\n\r\nand Newsgroups20 http://qwone.com/~jason/20Newsgroups/ via http://scikit-learn.org/stable/datasets/twenty_newsgroups.html", "votes": null }, { "id": "95610", "postDate": "10/09/2015 19:32:54", "content": "I've found it useful to have a list of common HTML tags: http://www.w3schools.com/tags/
", "rawMarkdown": "I've found it useful to have a list of common HTML tags: http://www.w3schools.com/tags/", "votes": null }, { "id": "95622", "postDate": "10/09/2015 21:48:37", "content": "I used attributes from here: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes
", "rawMarkdown": "I used attributes from here: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes", "votes": null }, { "id": "95698", "postDate": "10/10/2015 17:09:23", "content": "I am using english stopwords from nltk package.
", "rawMarkdown": "I am using english stopwords from nltk package.", "votes": null } ], "comments": [ { "id": 91337, "author_name": "zachmayer", "author_url": "", "post_date": "09/02/2015 13:49:19", "content": "I'm using the english stopwords from the TM package in R.
", "votes": null, "replies": [] }, { "id": 91594, "author_name": "chonglinsun", "author_url": "", "post_date": "09/04/2015 16:53:55", "content": "I'm use the nltk english stopwords
", "votes": null, "replies": [] }, { "id": 93767, "author_name": "senbong", "author_url": "", "post_date": "10/01/2015 01:35:43", "content": "We use the file-extension-related data from the following sites:
\n\nWeb Files: http://fileinfo.com/filetypes/web
\n\nCommon Files: http://fileinfo.com/filetypes/common
", "votes": null, "replies": [] }, { "id": 93850, "author_name": "ask788", "author_url": "", "post_date": "10/02/2015 03:53:22", "content": "Hey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?
\n\n[quote=Zach;91337]
\n\nI'm using the english stopwords from the TM package in R.
\n\n[/quote]
", "votes": null, "replies": [] }, { "id": 93896, "author_name": "zachmayer", "author_url": "", "post_date": "10/02/2015 15:12:56", "content": "[quote=ask788;93850]
\n\nHey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?
\n\n[/quote]
\n\nThe TM package is way too slow and memory - hungry for this competition. You could try the tmlite package , but the dataset may still be too big.
\n\nI'm manually pre-processing each file, one at a time, and then manually assembling the pre-processed files into a document-term matrix.
\n\nIt's slooooooooooow but doesn't run out of RAM.
", "votes": null, "replies": [] }, { "id": 93897, "author_name": "mortehu", "author_url": "", "post_date": "10/02/2015 15:21:24", "content": "I'm using this list of stopwords: http://www.ranks.nl/stopwords ("Default English stopwords list")
", "votes": null, "replies": [] }, { "id": 95299, "author_name": "tobycheese", "author_url": "", "post_date": "10/06/2015 19:35:54", "content": "I am using sklearn.feature_extraction.stop_words.ENGLISH_STOP_WORDS and nltk.corpus.stopwords.
", "votes": null, "replies": [] }, { "id": 95311, "author_name": "carloshuertas", "author_url": "", "post_date": "10/06/2015 20:46:22", "content": "Already posted, but I am using NLTK stop words as well.
", "votes": null, "replies": [] }, { "id": 95497, "author_name": "triskelion", "author_url": "", "post_date": "10/08/2015 13:41:08", "content": "https://www.kaggle.com/c/word2vec-nlp-tutorial/data from paper http://ai.stanford.edu/~ang/papers/acl11-WordVectorsSentimentAnalysis.pdf
\n\nand Newsgroups20 http://qwone.com/~jason/20Newsgroups/ via http://scikit-learn.org/stable/datasets/twenty_newsgroups.html
", "votes": null, "replies": [] }, { "id": 95610, "author_name": "jmaatta", "author_url": "", "post_date": "10/09/2015 19:32:54", "content": "I've found it useful to have a list of common HTML tags: http://www.w3schools.com/tags/
", "votes": null, "replies": [] }, { "id": 95622, "author_name": "davidshinn", "author_url": "", "post_date": "10/09/2015 21:48:37", "content": "I used attributes from here: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes
", "votes": null, "replies": [] }, { "id": 95698, "author_name": "", "author_url": "", "post_date": "10/10/2015 17:09:23", "content": "I am using english stopwords from nltk package.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "91268": "As posted [here][1], it seems that a forum topic for each external data will just fill the forum and make it hard to navigate. Post here all your external data, in a single topic, we can gather all together.\r\n\r\nI do not have any current external data.\r\n\r\n\r\n [1]: https://www.kaggle.com/c/dato-native/forums/t/16154/external-dictionaries/91264#post91264", "91337": "I'm using the english [stopwords][1] from the [TM package][2] in R.\r\n\r\n\r\n [1]: http://www.inside-r.org/packages/cran/tm/docs/stopwords\r\n [2]: https://cran.r-project.org/web/packages/tm/index.html", "91594": "I'm use the [nltk english stopwords][1]\r\n\r\n\r\n [1]: http://www.nltk.org/nltk_data/", "93767": "We use the file-extension-related data from the following sites:\r\n\r\nWeb Files: http://fileinfo.com/filetypes/web\r\n\r\nCommon Files: http://fileinfo.com/filetypes/common", "93850": "Hey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?\r\n\r\n[quote=Zach;91337]\r\n\r\nI'm using the english [stopwords][1] from the [TM package][2] in R.\r\n\r\n\r\n [1]: http://www.inside-r.org/packages/cran/tm/docs/stopwords\r\n [2]: https://cran.r-project.org/web/packages/tm/index.html\r\n\r\n[/quote]", "93896": "[quote=ask788;93850]\r\n\r\nHey Zach, what kind of machine are you using? I'm using tm package in R too on 16GB RAM Mac but have trouble running it on the training set. It's too slow and freezes my computer. How were you able to run it?\r\n\r\n[/quote]\r\n\r\nThe TM package is way too slow and memory - hungry for this competition. You could try the [tmlite package][1] , but the dataset may still be too big.\r\n\r\nI'm manually pre-processing each file, one at a time, and then manually assembling the pre-processed files into a document-term matrix.\r\n\r\nIt's slooooooooooow but doesn't run out of RAM.\r\n\r\n\r\n [1]: https://github.com/dselivanov/tmlite", "93897": "I'm using this list of stopwords: http://www.ranks.nl/stopwords (\"Default English stopwords list\")", "95299": "I am using sklearn.feature_extraction.stop_words.ENGLISH_STOP_WORDS and nltk.corpus.stopwords.", "95311": "Already posted, but I am using NLTK stop words as well.", "95497": "https://www.kaggle.com/c/word2vec-nlp-tutorial/data from paper http://ai.stanford.edu/~ang/papers/acl11-WordVectorsSentimentAnalysis.pdf\r\n\r\nand Newsgroups20 http://qwone.com/~jason/20Newsgroups/ via http://scikit-learn.org/stable/datasets/twenty_newsgroups.html", "95610": "I've found it useful to have a list of common HTML tags: http://www.w3schools.com/tags/", "95622": "I used attributes from here: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes", "95698": "I am using english stopwords from nltk package." }, "source": "meta" }