Parallellize a for loop in Java using multi-threading

Question

I am very new to java and I want to parallelize a nested for loop using executor service or using any other method in java. I want to create some fixed number of threads so that CPU is not completely acquired by threads.

    for(SellerNames sellerNames : sellerDataList) {
        for(String selleName : sellerNames) {
        //getSellerAddress(sellerName)
        //parallize this task
        }
    }

size of sellerDataList = 1000 and size of sellerNames = 5000.

Now I want to create 10 threads and assign equal chunk of task to each thread equally. That is for i'th sellerDataList, first thread should get address for 500 names, second thread should get address for next 500 names and so on.
What is the best way to do this job?

You can use a FixedSizeThreadPool and submit one Task for each sellerNames. See docs.oracle.com/javase/8/docs/api/java/util/concurrent/… When you submitted all tasks, issue a shutdown on the Executor , then wait for it to finish with awaitTermination — Fildor
– Fildor, Commented Apr 27, 2017 at 10:59
maybe Collections .stream().parallel() form is an option? — Timothy Truckle
– Timothy Truckle, Commented Apr 27, 2017 at 11:06
I have tried Collections .stream().parallel(). It is limited by number of cores. — Jhutan Debnath
– Jhutan Debnath, Commented Apr 27, 2017 at 11:10
The syntax is wrong. Where you wrote foreach it should be for. — Lew Bloch
– Lew Bloch, Commented Apr 27, 2017 at 11:35
@Jhutan Debnath, "limited by [the] number of cores" isn't actually much of a limitation. "parallize" should be spelled "parallelize". — Lew Bloch
– Lew Bloch, Commented Apr 27, 2017 at 11:38

Tamas Rev · Accepted Answer · 2017-04-27 13:55:02Z

26

There are two ways to make it run parallelly: Streams and Executors.

Using streams

You can use parallel streams and leave the rest to the jvm. In this case you don't have too much control over what happens when. On the other hand your code will be easy to read and maintain:

    sellerDataList.stream().forEach(sellerNames -> {
        Stream<String> stream = StreamSupport.stream(sellerNames.spliterator(), true); // true means use parallel stream
        stream.forEach(sellerName -> {
            getSellerAddress(sellerName);
        });
    });

Using an ExecutorService

Suppose, you want 5 Threads and you want to be able to wait until task completion. Then you can use a fixed thread pool with 5 threads and use Future-s so you can wait until they are done.

    final ExecutorService executor = Executors.newFixedThreadPool(5); // it's just an arbitrary number
    final List<Future<?>> futures = new ArrayList<>();
    for (SellerNames sellerNames : sellerDataList) {
        for (final String sellerName : sellerNames) {
            Future<?> future = executor.submit(() -> {
                getSellerAddress(sellerName);
            });
            futures.add(future);
        }
    }
    try {
        for (Future<?> future : futures) {
            future.get(); // do anything you need, e.g. isDone(), ...
        }
    } catch (InterruptedException | ExecutionException e) {
        e.printStackTrace();
    }

answered Apr 27, 2017 at 13:55

Tamas Rev

7,1745 gold badges37 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Phil Over a year ago

Now in Java 8 you can do sellerNames.parallelStream().forEach(...)

Dhananjay · Accepted Answer · 2021-03-16 05:51:08Z

0

If you are using a parallel stream you can still control the thread by creating your own ForkJoinPool.

List<Long> aList = LongStream.rangeClosed(firstNum, lastNum).boxed()
  .collect(Collectors.toList());

ForkJoinPool customThreadPool = new ForkJoinPool(4);
long actualTotal = customThreadPool.submit(
  () -> aList.parallelStream().reduce(0L, Long::sum)).get();

Here on this site, it is described very well. https://www.baeldung.com/java-8-parallel-streams-custom-threadpool

answered Mar 16, 2021 at 5:51

Dhananjay

33 bronze badges

Collectives™ on Stack Overflow

Parallellize a for loop in Java using multi-threading

2 Answers 2

Using streams

Using an ExecutorService

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Using streams

Using an ExecutorService

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related