Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
566 views
in Technique[技术] by (71.8m points)

dataframe - Faster way to subset on rows of a data frame in R?

I have been using these 2 methods interchangeably to subset data from a data frame in R.
Method 1
subset_df <- df[which(df$age>5) , ]
Method 2
subset_df <- subset(df, age>5)

I had 2 questions belonging to these.
1. Which one is faster considering I have very large data?
2. This post here Subsetting data frames in R suggests that there is in fact difference between above 2 methods. One of them handles NA accurately. Which one is safe to use then?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

The question asks for a faster way to subset rows of a data frame. The fastest way is with data.table.

set.seed(1)  # for reproducible example
# 1 million rows - big enough?
df <- data.frame(age=sample(1:65,1e6,replace=TRUE),x=rnorm(1e6),y=rpois(1e6,25))

library(microbenchmark)
microbenchmark(result<-df[which(df$age>5),],
               result<-subset(df, age>5), 
               result<-df[df$age>5,],
               times=10)
# Unit: milliseconds
#                               expr       min        lq    median       uq      max neval
#  result <- df[which(df$age > 5), ]  77.01055  80.62678  81.43786 133.7753 145.4756    10
#      result <- subset(df, age > 5) 190.89829 193.04221 197.49973 203.7571 263.7738    10
#         result <- df[df$age > 5, ] 169.85649 171.02084 176.47480 185.9394 191.2803    10

library(data.table)
DT <- as.data.table(df)     # data.table
microbenchmark(DT[age > 5],times=10)
# Unit: milliseconds
#         expr      min       lq  median       uq      max neval
#  DT[age > 5] 29.49726 29.93907 30.1813 30.67168 32.81204    10

So in this simple case data.table is a little more than twice as fast as which(...), and more than 6 times faster than subset(...).


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...