exporting function from R package to cluster (parallel processing)

Question

I am trying to use parLapply again (now successfully used it in this context) but this time the function to parallelize contains some functions from the mc2d package (ie. deprt, ppert). I installed and loaded the package, before the parallel part of the code.

I successfully exported my variables to the cluster using cluster export,

clusterExport(C1, "variable")
...

However I tried to export the functions dpert and ppert using

clusterEvalQ(C1,dpert)

and I get the error

object C1 not found

how does it not find C1? is there another way to export R-packages functions to the cluster for parallel processing?

Ankur Chakravarthy · Accepted Answer · 2014-10-19 00:56:01Z

9

Use clusterExport for functions as well...

clusterExport(Cl,c('dpert','variable'))

answered Oct 19, 2014 at 0:56

Ankur Chakravarthy

4084 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Gregory108 · Accepted Answer · 2018-06-14 23:48:55Z

0

Another way I accidentally discovered is to add the name of the package to which the function belongs before the function call. E.g. instead of say syms() or group_indices() you use rlang::syms() or dplyr::group_indices() inside parLapply() (or any other parallel library function).

answered Jun 14, 2018 at 23:48

Gregory108

415 bronze badges

1 Comment

Dan Chaltiel Over a year ago

could you give a full clusterExport call example ?

Collectives™ on Stack Overflow

exporting function from R package to cluster (parallel processing)

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related