Parallel computing in Julia - running a simple for-loop on multiple cores

Question

For starters, I have to say I'm completely new to parallel computing (and know close to nothing about computer science), so my understanding of what things like "workers" or "processes" actually are is very limited. I do however have a question about running a simple for-loop that presumably has no dependencies between the iterations in parallel.

Let's say I wanted to do the following:

for N in 1:5:20
    println("The N of this iteration in $N")
end

If I simply wanted these messages to appear on screen and the order of appearance didn't matter, how could one achieve this in Julia 0.6, and for future reference in Julia 0.7 (and therefore 1.0)?

Alex338207 · Accepted Answer · 2023-08-02 10:03:32Z

17

Distributed Processing

Start julia with e.g. julia -p 4 if you want to use 4 cpus (or use the function addprocs(4)). In Julia 1.x, you make a parallel loop as following:

using Distributed
@distributed for N in 1:5:20
    println("The N of this iteration in $N")
end

Note that every process have its own variables per default. For any serious work, have a look at the manual https://docs.julialang.org/en/v1/manual/parallel-computing/, in particular the section about SharedArrays.

Another option for distributed computing are the function pmap or the package MPI.jl.

Threads

Since Julia 1.3, you can also use Threads as noted by wueli. Start julia with e.g. julia -t 4 to use 4 threads. Alternatively you can or set the environment variable JULIA_NUM_THREADS before starting julia. For example Linux/Mac OS:

export JULIA_NUM_THREADS=4

In windows, you can use set JULIA_NUM_THREADS 4 in the cmd prompt.

Then in julia:

   Threads.@threads for N = 1::20
       println("N = $N (thread $(Threads.threadid()) of out $(Threads.nthreads()))")
   end

All CPUs are assumed to have access to shared memory in the examples above (e.g. "OpenMP style" parallelism) which is the common case for multi-core CPUs.

edited Aug 2, 2023 at 10:03

answered Jul 21, 2018 at 20:59

Alex338207

1,91512 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Chris Rackauckas Over a year ago

SharedArrays shouldn't be used with too serious work because you want to decrease the amount of communication, but it's a nice tool in areas which don't need the most performance. pmap and @parallel/@distributed both distribute work among different computers: it's just a matter of setting up processes on different computers and having an SSH connection. And the most speed will come from Threads.@threads. If you need to print from threads you need to use Core.print though.

wueli · Accepted Answer · 2019-11-27 12:56:32Z

16

Just to add the example to the answer of Chris. Since the release of julia 1.3 you do this easily with Threads.@threads

Threads.@threads for N in 1:5:20
    println("The number of this iteration is $N")
end

Here you are running only one julia session with multiple threads instead of using Distributed where you run multiple julia sessions.

See, e.g. multithreading blog post for more information.

answered Nov 27, 2019 at 12:56

wueli

1,26915 silver badges24 bronze badges

Collectives™ on Stack Overflow

Parallel computing in Julia - running a simple for-loop on multiple cores

2 Answers 2

Distributed Processing

Threads

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Distributed Processing

Threads

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related