1

I'd like to use foreach to increase the computation speed. What I want to do is to output two results from single foreach loop.

Following is the simplified version of the original idea:

output <- list(matrix_addition = matrix(0, nrow = 2, ncol = 2), process_list = list())

for(i in c(1:10)){
  # value_1 indicates some calculation to get the matrix
  value_1 <- i * 2

  # value_2 indicates some calculation to process_list for each i
  value_2 <- i / 2

  output$matrix_addition <- output$matrix_addition + matrix(value_1, nrow = 2, ncol = 2)
  output$process_list <- append(output$process_list, value_2)
}

The expected output will be like:

$matrix_addition
     [,1] [,2]
[1,]  110  110
[2,]  110  110

and

$process_list
$process_list[[1]]
[1] 0.5

$process_list[[2]]
[1] 1

...

$process_list[[10]]
[1] 5

I tried to use foreach with .combine = "+" for the matrix addition part, but when it comes to multiple output, mapply seems not able to combine with "+". I found several codes online with same operation for both outputs by using .combine = "cbind", .multicombine=TRUE. When it comes to different operation for two outputs, is there any way to do it?

Thanks for the help!

edit:

My foreach code for only matrix addition is

library(foreach)
library(doParallel)

cl <- makeCluster(2)
registerDoParallel(cl)

output <- foreach(i = 1:10, .combine = "+") %dopar% {
  value_1 <- i * 2
  matrix <- matrix(value_1, nrow = 2, ncol = 2)
}

stopCluster(cl)

And my first attempt for the two output is

output <- foreach(i = 1:10) %dopar% {
  if(exists("temp") == FALSE) {
    output <- list(matrix_addition = matrix(0, nrow = 2, ncol = 2), process_list = list())
  }

  value_1 <- i * 2
  value_2 <- i / 2

  output$matrix_addition <- output$matrix_addition + matrix(value_1, nrow = 2, ncol = 2)
  output$process_list <- append(output$process_list, value_2)
  output
}

It seems that the original for loop idea can't directly use in foreach.

Next, I found these codes online, it looks like:

comb <- function(...) {
  mapply("cbind", ..., SIMPLIFY=FALSE)
}

output <- foreach(i = 1:10, .combine="comb", .multicombine = TRUE) %dopar% {
  value_1 <- i * 2
  value_2 <- i / 2

  matrix_addition_part  <- matrix(value_1, nrow = 2, ncol = 2)
  process_list_part     <- value_2
  list(matrix_addition_part, process_list_part)
}

and

comb <- function(x, ...) {
  lapply(seq_along(x),
         function(i){c(x[[i]], lapply(list(...), function(y) y[[i]]))})
}

output <- foreach(i=1:10, .combine="comb", .multicombine=TRUE, .init=list(list(), list())) %dopar% {
  value_1 <- i * 2
  value_2 <- i / 2

  list(matrix(value_1, nrow = 2, ncol = 2), value_2)
}

These two are similar, the idea here is to save all matrix in the matrix addition part, and after foreach, matrix can be then add up. But the problem for me is that each matrix in my code is too big, which requires huge space to store, therefore I can't save all the matrix. I think I need to replace "cbind" in matrix addition part into something like "+", but it doesn't work in mapply.

My guess is that I need to make the function comp contains both "+" and "append", but still can't come up with a proper solution by utilizing apply.

4
  • Can you show your code for foreach Commented Aug 6, 2019 at 3:14
  • I add several foreach attempts of mine above. Commented Aug 6, 2019 at 4:16
  • Please look for "[r] foreach multiple" in SO. This has been answered several times already. You have basically two solutions: either you make a custom .combine function, or you just output a list and combine results afterwards. Commented Aug 6, 2019 at 4:45
  • Thanks for the help, just figure out how to do it. Commented Aug 6, 2019 at 11:56

1 Answer 1

1

Just figure out how to do it:

library(foreach)
library(doParallel)
cl <- makeCluster(2)
registerDoParallel(cl)

comb <- function(List1, List2) {
  output_a <- apply(abind(List1[[1]], List2[[1]], along = 3), 1:2, sum)
  output_b <- c(List1[[2]], List2[[2]])
  return(list(matrix_addition = output_a, process_list = output_b))
}

output <- foreach(i = 1:10, .combine = "comb", .init=list(matrix(0, 2, 2), list())) %dopar% {
  value_1 <- i * 2
  value_2 <- i / 2

  list(matrix_addition = matrix(value_1, nrow = 2, ncol = 2), process_list = value_2)
}

stopCluster(cl)

Here, by customizing the .combine function comb, separate the matrix addition part and process list part, and merge them into a list. Also, the first input (List1 here) for the comp function will be .init or previous loop iteration (i-1) result, and second input (List2 here) will be the current loop iteration (i) result.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.