{
  "id": 1503,
  "title": "Any ideas / suggestions how to approach this problem?",
  "url": "/competitions/kddcup2012-track2/discussion/1503",
  "author_name": "",
  "post_date": "2012-03-08T05:49:46.207Z",
  "votes": null,
  "comment_count": 3,
  "views": 4102,
  "content": "<p>I started off thinking creating a linear regression baseline would be nice... however w. almost 150 million training samples, that is pretty much intractable. What is the general idea you guys have in mind in approaching this problem?</p>",
  "messages": [
    {
      "id": "9161",
      "postDate": "03/08/2012 05:49:46",
      "content": "<p>I started off thinking creating a linear regression baseline would be nice... however w. almost 150 million training samples, that is pretty much intractable. What is the general idea you guys have in mind in approaching this problem?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "9185",
      "postDate": "03/08/2012 20:12:49",
      "content": "<p>Thats a very good question. It is very hard to process a data this large without using tecnologies like hadoop or gpu processing...<br>\r\nunfortunately I couldn,t found any gpu implementation that works on my windows machine...</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "9375",
      "postDate": "03/15/2012 22:56:30",
      "content": "<p>The NTU team authored a tool at the KDD 2010 conference for running regression on datasets much larger than working memory. You might like to give that a shot. It's a modification of liblinear.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "10335",
      "postDate": "04/24/2012 17:22:04",
      "content": "<p>Have you tried the real time version of linear regression? there is no limit for the number of rows.</p>\r\n<p>http://www.amazon.com/Real-Time-Data-Mining-Sayad/dp/0986606049/ref=sr_1_1?ie=UTF8&amp;s=books&amp;qid=1297618703&amp;sr=8-1</p>\r\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 9185,
      "author_name": "leustagos",
      "author_url": "",
      "post_date": "03/08/2012 20:12:49",
      "content": "<p>Thats a very good question. It is very hard to process a data this large without using tecnologies like hadoop or gpu processing...<br>\r\nunfortunately I couldn,t found any gpu implementation that works on my windows machine...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 9375,
      "author_name": "zachpardos",
      "author_url": "",
      "post_date": "03/15/2012 22:56:30",
      "content": "<p>The NTU team authored a tool at the KDD 2010 conference for running regression on datasets much larger than working memory. You might like to give that a shot. It's a modification of liblinear.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 10335,
      "author_name": "",
      "author_url": "",
      "post_date": "04/24/2012 17:22:04",
      "content": "<p>Have you tried the real time version of linear regression? there is no limit for the number of rows.</p>\r\n<p>http://www.amazon.com/Real-Time-Data-Mining-Sayad/dp/0986606049/ref=sr_1_1?ie=UTF8&amp;s=books&amp;qid=1297618703&amp;sr=8-1</p>\r\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "9161": "",
    "9185": "",
    "9375": "",
    "10335": ""
  },
  "source": "meta"
}