Allowed Memory in R and C Stack Usage -

I am trying to read a large file (~ 5GB) in R and the process of data. I can fully read in full 5GB file, but the problem occurs when I apply my processing, I do not have a great understanding on memory basics in R, and I'm hoping some of you understand me better Can help in

Here is an example of what I'm going to do

file = Fred ("file.txt") # file.txt: 5 GB non-existent data T. RT & lt; -strptime (file $ time, "% m /% d /% Y% H:% M:% S" "#) from date column to class = = as.numeric (format (t.str,"% m ") ) # Create a vector from the file column high = ifelse (file $ age> 70,1,0) # Creating a vector from the file's column # There are about ten more lines that work on this file. < / Pre>

The Fred file works well in reading and the first three or four operations run on the 'file' data frame task. However, after running some of them, I got an error It is that which says:

  C stack use is very close to the limit of 19923892  

I am pretty sure that this issue is not a definite command, because I work on small data sets. I have read a heap, but this is not a warning, I fully understand. Es means that R is using an indicator to run through these big vectors , And I have gone out of the indicator space (?). Here read about a similar problem:

One user suggested increasing the stack size in the shell. I tried to look at it further, but I'm not sure how to move forward. Here's a suggestion:

  $ ulimit -s # print default 8192 $ R --slave -e 'cstack_info () ["size"]' size 8388608  

Does anyone help me to understand whether it means, or just a little bit about stack usage in R? Or does anyone know a better way to process this data than stack usage? I'm not sure how to give you copied data to the people.

Edit to add data example:

  PersonID time energy age 1301839 07/24/2013 07: 15:00 0.13 68 1301521 07/24/2013 07:30:00 0.19 68 1301890 07/24/2013 07:45:00 0.10 68 1301890 07/24/2013 08:00:00 0.06 68 1307112 07/24/2013 08:15:00 0.01 68  

Sorry, this is not really an answer, but I do not have enough points to comment on You can try reading and processing the data in part, or see some large memory packages in the Cran Task View You can. You can also read about memory usage