1

I've been using the SBM algorithm, implemented in the graph-tool package. I need to process a huge amount of data and need to run it in parallel.

I know that OpenMP is activated by default in this package and used in the specific and compatible algorithms, but the documentation doesn't specify which algorithms.

I've tried openmp_enabled() or openmp_set_num_threads() and also export OMP_NUM_THREADS=16. Everything seems fine but when I check the running processes, it's not paralleled.

Do you have any experience with implementing SBM parallelized?

Improve this question

1 Answer 1

Reset to default
2

Although graph-tool uses OpenMP, not every algorithm is implemented in parallel, simply because this cannot be done in some cases. The SBM inference algorithm implemented in graph-tool is based on MCMC, which cannot be parallelized in general. Because of this, enabling OpenMP will have no effect.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you, dear Tiago. It would be amazing if there was a shortlist of available algorithms that can be parallelized in graph-tools.
@Tiago, I'm confused. Isn't the (multiflip_)mcmc_sweep() function parallelized via OpenMP? When I've used minimize_blockmodel_dl() in the past, I notice my CPU cores oscillating between full utilization (all cores) to single-core while the function runs, indicating that at least some of the computation is parallelized. Wasn't that due to this line?
@StuartBerg That line has to do with the parallel tempering algorithm (which runs in parallel), not the regular MCMC code (which does not)! The function minimize_blockmodel_dl() also has a block merging phase which does run in parallel, but that is a minor part.
@SaeedChoobani For all functions in graph-tool (singular) that are implemented in parallel, this is mentioned in the documentation. If this is not mentioned, then it's not the case.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.