I'd like to use foreach to increase the computation speed. What I want to do is to output two results from single foreach loop.
Following is the simplified version of the original idea:
output <- list(matrix_addition = matrix(0, nrow = 2, ncol = 2), process_list = list())
for(i in c(1:10)){
# value_1 indicates some calculation to get the matrix
value_1 <- i * 2
# value_2 indicates some calculation to process_list for each i
value_2 <- i / 2
output$matrix_addition <- output$matrix_addition + matrix(value_1, nrow = 2, ncol = 2)
output$process_list <- append(output$process_list, value_2)
}
The expected output will be like:
$matrix_addition
[,1] [,2]
[1,] 110 110
[2,] 110 110
and
$process_list
$process_list[[1]]
[1] 0.5
$process_list[[2]]
[1] 1
...
$process_list[[10]]
[1] 5
I tried to use foreach with .combine = "+" for the matrix addition part, but when it comes to multiple output, mapply seems not able to combine with "+". I found several codes online with same operation for both outputs by using .combine = "cbind", .multicombine=TRUE. When it comes to different operation for two outputs, is there any way to do it?
Thanks for the help!
edit:
My foreach code for only matrix addition is
library(foreach)
library(doParallel)
cl <- makeCluster(2)
registerDoParallel(cl)
output <- foreach(i = 1:10, .combine = "+") %dopar% {
value_1 <- i * 2
matrix <- matrix(value_1, nrow = 2, ncol = 2)
}
stopCluster(cl)
And my first attempt for the two output is
output <- foreach(i = 1:10) %dopar% {
if(exists("temp") == FALSE) {
output <- list(matrix_addition = matrix(0, nrow = 2, ncol = 2), process_list = list())
}
value_1 <- i * 2
value_2 <- i / 2
output$matrix_addition <- output$matrix_addition + matrix(value_1, nrow = 2, ncol = 2)
output$process_list <- append(output$process_list, value_2)
output
}
It seems that the original for loop idea can't directly use in foreach.
Next, I found these codes online, it looks like:
comb <- function(...) {
mapply("cbind", ..., SIMPLIFY=FALSE)
}
output <- foreach(i = 1:10, .combine="comb", .multicombine = TRUE) %dopar% {
value_1 <- i * 2
value_2 <- i / 2
matrix_addition_part <- matrix(value_1, nrow = 2, ncol = 2)
process_list_part <- value_2
list(matrix_addition_part, process_list_part)
}
and
comb <- function(x, ...) {
lapply(seq_along(x),
function(i){c(x[[i]], lapply(list(...), function(y) y[[i]]))})
}
output <- foreach(i=1:10, .combine="comb", .multicombine=TRUE, .init=list(list(), list())) %dopar% {
value_1 <- i * 2
value_2 <- i / 2
list(matrix(value_1, nrow = 2, ncol = 2), value_2)
}
These two are similar, the idea here is to save all matrix in the matrix addition part, and after foreach, matrix can be then add up. But the problem for me is that each matrix in my code is too big, which requires huge space to store, therefore I can't save all the matrix. I think I need to replace "cbind" in matrix addition part into something like "+", but it doesn't work in mapply.
My guess is that I need to make the function comp contains both "+" and "append", but still can't come up with a proper solution by utilizing apply.
foreachforeachattempts of mine above..combinefunction, or you just output a list and combine results afterwards.