{
  "id": 21094,
  "title": "R : readLines ",
  "url": "/competitions/expedia-hotel-recommendations/discussion/21094",
  "author_name": "",
  "post_date": "2016-05-20T13:29:19.587Z",
  "votes": null,
  "comment_count": 2,
  "views": 479,
  "content": "<p>Hi, </p>\n\n<p>Has anyone tried &quot;readLines&quot; in R ?\nSome solutions written with Python are reading the lines and doing it without time-out.</p>\n\n<p>&quot;readLines&quot; did not even reach 2M lines till timed out.\nBelow is my code, can anyone share the idea to read train data in R, please ?</p>\n\n<h1>======================</h1>\n\n<p>f &lt;- file (&quot;../input/train.csv&quot;)\ntotal &lt;- 0</p>\n\n<p>while(1) {\n  line &lt;- readLines(f, n=1)\n  total &lt;- total + 1\n  if((total %% 2000000) == 0) {\n    cat (&quot; read lines = &quot;, total);\n  }\n  if(length(line) == 0) {\n    break\n  } else {\n    line.f &lt;- textConnection (line)\n    line.df &lt;- read.csv(line.f, header=FALSE, sep=&quot;,&quot;)\n  }\n}</p>\n\n<h1>======================</h1>\n\n<p>Best regards, \nKendo</p>",
  "messages": [
    {
      "id": "120772",
      "postDate": "05/20/2016 13:29:19",
      "content": "<p>Hi, </p>\n\n<p>Has anyone tried &quot;readLines&quot; in R ?\nSome solutions written with Python are reading the lines and doing it without time-out.</p>\n\n<p>&quot;readLines&quot; did not even reach 2M lines till timed out.\nBelow is my code, can anyone share the idea to read train data in R, please ?</p>\n\n<h1>======================</h1>\n\n<p>f &lt;- file (&quot;../input/train.csv&quot;)\ntotal &lt;- 0</p>\n\n<p>while(1) {\n  line &lt;- readLines(f, n=1)\n  total &lt;- total + 1\n  if((total %% 2000000) == 0) {\n    cat (&quot; read lines = &quot;, total);\n  }\n  if(length(line) == 0) {\n    break\n  } else {\n    line.f &lt;- textConnection (line)\n    line.df &lt;- read.csv(line.f, header=FALSE, sep=&quot;,&quot;)\n  }\n}</p>\n\n<h1>======================</h1>\n\n<p>Best regards, \nKendo</p>",
      "rawMarkdown": "Hi, \r\n\r\nHas anyone tried \"readLines\" in R ?\r\nSome solutions written with Python are reading the lines and doing it without time-out.\r\n\r\n\"readLines\" did not even reach 2M lines till timed out.\r\nBelow is my code, can anyone share the idea to read train data in R, please ?\r\n\r\n#======================\r\nf <- file (\"../input/train.csv\")\r\ntotal <- 0\r\n\r\nwhile(1) {\r\n  line <- readLines(f, n=1)\r\n  total <- total + 1\r\n  if((total %% 2000000) == 0) {\r\n    cat (\" read lines = \", total);\r\n  }\r\n  if(length(line) == 0) {\r\n    break\r\n  } else {\r\n    line.f <- textConnection (line)\r\n    line.df <- read.csv(line.f, header=FALSE, sep=\",\")\r\n  }\r\n}\r\n#======================\r\n\r\nBest regards, \r\nKendo",
      "votes": null
    },
    {
      "id": "120780",
      "postDate": "05/20/2016 14:27:21",
      "content": "<p>Hello Kendo,</p>\n\n<p>a shameless copy from stackoverflow:\n<a href=\"http://stackoverflow.com/questions/12626637/reading-a-text-file-in-r-line-by-line\">http://stackoverflow.com/questions/12626637/reading-a-text-file-in-r-line-by-line</a></p>\n\n<pre><code>processFile = function(filepath) {\n  con = file(filepath, &quot;r&quot;)\n  while ( TRUE ) {\n    line = readLines(con, n = 1)\n    if ( length(line) == 0 ) {\n      break\n    }\n    print(line)\n  }\n\n  close(con)\n}\n</code></pre>\n\n<p>I hope this gets you the idea.</p>",
      "rawMarkdown": "Hello Kendo,\r\n\r\na shameless copy from stackoverflow:\r\nhttp://stackoverflow.com/questions/12626637/reading-a-text-file-in-r-line-by-line\r\n\r\n    processFile = function(filepath) {\r\n      con = file(filepath, \"r\")\r\n      while ( TRUE ) {\r\n        line = readLines(con, n = 1)\r\n        if ( length(line) == 0 ) {\r\n          break\r\n        }\r\n        print(line)\r\n      }\r\n    \r\n      close(con)\r\n    }\r\n\r\nI hope this gets you the idea.",
      "votes": null
    },
    {
      "id": "120902",
      "postDate": "05/21/2016 15:02:21",
      "content": "<p>Thank you Justfor.</p>",
      "rawMarkdown": "Thank you Justfor.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 120780,
      "author_name": "justfor",
      "author_url": "",
      "post_date": "05/20/2016 14:27:21",
      "content": "<p>Hello Kendo,</p>\n\n<p>a shameless copy from stackoverflow:\n<a href=\"http://stackoverflow.com/questions/12626637/reading-a-text-file-in-r-line-by-line\">http://stackoverflow.com/questions/12626637/reading-a-text-file-in-r-line-by-line</a></p>\n\n<pre><code>processFile = function(filepath) {\n  con = file(filepath, &quot;r&quot;)\n  while ( TRUE ) {\n    line = readLines(con, n = 1)\n    if ( length(line) == 0 ) {\n      break\n    }\n    print(line)\n  }\n\n  close(con)\n}\n</code></pre>\n\n<p>I hope this gets you the idea.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 120902,
      "author_name": "leekendo",
      "author_url": "",
      "post_date": "05/21/2016 15:02:21",
      "content": "<p>Thank you Justfor.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "120772": "Hi, \r\n\r\nHas anyone tried \"readLines\" in R ?\r\nSome solutions written with Python are reading the lines and doing it without time-out.\r\n\r\n\"readLines\" did not even reach 2M lines till timed out.\r\nBelow is my code, can anyone share the idea to read train data in R, please ?\r\n\r\n#======================\r\nf <- file (\"../input/train.csv\")\r\ntotal <- 0\r\n\r\nwhile(1) {\r\n  line <- readLines(f, n=1)\r\n  total <- total + 1\r\n  if((total %% 2000000) == 0) {\r\n    cat (\" read lines = \", total);\r\n  }\r\n  if(length(line) == 0) {\r\n    break\r\n  } else {\r\n    line.f <- textConnection (line)\r\n    line.df <- read.csv(line.f, header=FALSE, sep=\",\")\r\n  }\r\n}\r\n#======================\r\n\r\nBest regards, \r\nKendo",
    "120780": "Hello Kendo,\r\n\r\na shameless copy from stackoverflow:\r\nhttp://stackoverflow.com/questions/12626637/reading-a-text-file-in-r-line-by-line\r\n\r\n    processFile = function(filepath) {\r\n      con = file(filepath, \"r\")\r\n      while ( TRUE ) {\r\n        line = readLines(con, n = 1)\r\n        if ( length(line) == 0 ) {\r\n          break\r\n        }\r\n        print(line)\r\n      }\r\n    \r\n      close(con)\r\n    }\r\n\r\nI hope this gets you the idea.",
    "120902": "Thank you Justfor."
  },
  "source": "meta"
}