looping dataframes with different number of columns in r

Question

Maybe is something trivial but I am trying to solve this problem:

I have to data frames, one with 25 and another with 9 columns. Now, what I need to do is to fit polynomial equations where my dependent variable is in the data frame with 25 columns and my independent variable is in the data frame with 9 columns. At the moment I combined the columns together and created a data frame called "my.data", so I am looping over the dependent variables using one independent variable at the time. But, I would like do the functions in the loop 25 * 9 times automatically. Is there any way to do that?

setwd("C:\\......")

 my.data <- read.table("MyData.txt", header = TRUE, sep = "\t")


for(i in seq_along(my.data))
 {

    fit1b <- lm(my.data[ ,i] ~ my.data$V1)
    fit2b <- lm(my.data[ ,i] ~ poly(my.data$V1, 2, raw=TRUE))
    fit3b <- lm(my.data[ ,i] ~ poly(my.data$V1, 3, raw=TRUE))
    poly1 <-capture.output(summary(fit1b))
    poly2 <-capture.output(summary(fit2b))
    poly3 <-capture.output(summary(fit3b))


con = file(description = "MyResults.txt", open="a")
write.table(poly1, file= con, append = TRUE, quote=F, col.names=FALSE, row.names= F)
write.table(poly2, file= con, append = TRUE, quote=F, col.names=FALSE, row.names= F)
write.table(poly3, file= con, append = TRUE, quote=F, col.names=FALSE, row.names= F)
close(con)
 }

mnel · Accepted Answer · 2013-03-28 01:54:26Z

1

This is a perfect opportunity to use mapply and expand.grid

For example.

# some dummy data
xx <- data.frame(replicate(5, runif(50)))
yy <- setNames(data.frame(replicate(3, runif(50))), paste0('Y',1:3))
# all combinations
cs <- expand.grid(list(pred = names(xx), resp = names(yy)), stringsAsFactors= FALSE)

# a function to do the fitting
fitting <- function(pred, resp, dd){
  # fit linear model
  ff <- reformulate(pred, resp)
  lmf <- lm(ff, data =dd)
  # create a formula for poly(,2)
  ff.poly2 <- update(ff, .~poly(.,2, raw=TRUE))
  # and poly(,3)
  ff.poly3 <- update(ff, .~poly(.,3, raw=TRUE))
  # fit these models
  lmp2 <- lm(ff.poly2, data = dd)
  lmp3 <- lm(ff.poly3, data = dd)
  # return a list with these three models
  list(linear = lmf, poly2 = lmp2, poly3 = lmp3)
}

biglist <- mapply('fitting', pred = as.list(cs[['pred']]), 
        resp = as.list(cs[['resp']]),
       MoreArgs = list(dd = cbind(xx,yy)), SIMPLIFY = FALSE)

# give this list meaningful names

names(biglist) <- do.call(paste, c(cs, sep = ':'))

You can then extract things / summarize things using some nested lapply statements

eg summaries of all the linear models

lapply(lapply(biglist, `[[`,'linear'), summary)

of the quadratic models

lapply(lapply(biglist, `[[`,'poly2'), summary)

If you want to extract the information from print(summary(lm)) in a single file, something like

capture.output(lapply(biglist, function(x) lapply(x, summary)), file = 'results.txt')

will create a file called results.txt with all the results printed there.

edited Mar 28, 2013 at 1:54

answered Mar 28, 2013 at 0:23

mnel

116k28 gold badges269 silver badges255 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

david Over a year ago

thank you very much mnel, that worked very well!!! I m very rusty on R.... Thank you very much again!!!! David

david Over a year ago

There is one thing in the code I would like to make it do. To have the output file summarized in the way below, if it is a list it does not come like that, but I am not sure I can do it as a summary.

david · Accepted Answer · 2013-03-28 01:42:12Z

There is one thing I would like to do, to output the summary rather than the list, but I am not sure is possible to then use the writing function you have. Is there any way to obtain that?

Call: lm(formula = My-Y-Lable ~ My-X-Label)

Residuals: Min 1Q Median 3Q Max -0.35445 -0.17420 -0.10931 0.06975 0.60246

Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.7560212 0.0720984 10.49 1.24e-14 *

My-X-Label 0.0072100 0.0006597 10.93 2.68e-15 *

Signif. codes: 0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.2812 on 54 degrees of freedom Multiple R-squared: 0.6887, Adjusted R-squared: 0.6829 F-statistic: 119.5 on 1 and 54 DF, p-value: 2.676e-15

Collectives™ on Stack Overflow

looping dataframes with different number of columns in r

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related