R: Drop columns from data.table, by reference, without having the name

Question

This is almost a duplicate of this. I want to drop columns from a data table, but I want to do it efficiently. I have a list of names of columns that I want to keep. All the answers to the linked question imply doing something akin to

data.table.new <- data.table.old[, my.list]

which at some crucial point will give me a new object, while the old object is still in memory. However, my data.table.old is huge, and hence I prefer to do this via reference, as suggested here

set(data.table.old, j = 'a', value = NULL)

However, as I have a whitelist of columns, and not a blacklist, I would need to iterate through all the column names, checks whether they are in my.list, and then apply set(). Is there any cleaner/other way to doing so?

possible duplicate of Idiom for dropping a single column in a data.table mnel's answer there is easily extended to the drop-more-than-one-column case (with toDrop analogous to dropcols in Jan's answer). — Frank
– Frank, Commented Jul 7, 2015 at 16:55

jangorecki · Accepted Answer · 2015-07-07 12:03:04Z

8

Not sure if you can do by reference ops on data.frame without making it data.table.
Below code should works if you consider to use data.table.

library(data.table)
setDT(data.frame.old)
dropcols <- names(data.frame.old)[!names(data.frame.old) %in% my.list]
data.frame.old[, c(dropcols) := NULL]

answered Jul 7, 2015 at 12:03

jangorecki

16.8k5 gold badges86 silver badges169 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Frank Over a year ago

I think setdiff is better for construction of dropcols

jangorecki Over a year ago

@Frank Very likely, but I personally used to the not in vector subset

Farrel Over a year ago

Why doesn't dropcols := NULL work? What is the c doing?

jangorecki Over a year ago

@Farrel it evaluates dropcols to the character vector. Your code wouldn’t work because it would try to remove column named "dropcols" instead of the column names which dropcols variables stores. It doesn't need to be c(dropcols), it can be (dropcols) which also evaluates the variable.

Collectives™ on Stack Overflow

R: Drop columns from data.table, by reference, without having the name

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related