Java 8 Parallel processing and Lambda expression

Question

I want to test how much faster Java8 parallel stream so I wrote a program .The program count the number of prime numbers in a given list of numbers. The program counts prime numbers in there ways:

using for loop;
by using lambda expression;
by using lambda expression(parallel stream).

Before executing the program I was expecting that parallel stream version should be faster. But the result is

Total prime no found 664579 in 4237 mili sec ----for loop version
Total prime no found 664579 in 2440 mili sec ----parallel stream
Total prime no found 664579 in 2166 mili sec ----lambda expression

My doubt is why parallel version is slower then lambda version

List<Integer> numbers = new ArrayList<>();
        for (int i = 0; i < 10000000; i++) {
            numbers.add(i);
        }
        Stopwatch stopwatch = Stopwatch.createStarted();
        int counter = 0;
        for (int number : numbers) {
            if (isPrime(number)) {
                counter++;
            }
        }
        stopwatch.stop();
        System.out.println("Total prime no found " + counter + " in " + stopwatch.elapsed(TimeUnit.MILLISECONDS) + " mili sec");

        stopwatch = Stopwatch.createStarted();
        long count1 = numbers.parallelStream().filter(n -> isPrime(n)).count();
        stopwatch.stop();
        System.out.println("Total prime no found " + count1 + " in " + stopwatch.elapsed(TimeUnit.MILLISECONDS) + " mili sec");

        stopwatch = Stopwatch.createStarted();
        long count2 = numbers.stream().filter(n -> isPrime(n)).count();
        System.out.println("Total prime no found " + count2 + " in " + stopwatch.elapsed(TimeUnit.MILLISECONDS) + " mili sec");
        stopwatch.stop();

The above program uses google Guava library to calculate time elapsed.

while doing benchmarking you should also take care of warmup period — Jigar Joshi
– Jigar Joshi, Commented Mar 28, 2014 at 3:53
Total prime no found 664579 in 4178 mili sec --loop Total prime no found 664579 in 2597 mili sec --parallel stream Total prime no found 664579 in 4187 mili sec --lambda expression Total prime no found 664579 in 4198 mili sec --loop Total prime no found 664579 in 4690 mili sec --in parallel stream Total prime no found 664579 in 4195 mili sec --in lambda expression Total prime no found 664579 in 4252 mili sec --loop Total prime no found 664579 in 2455 mili sec --parallel Total prime no found 664579 in 4555 mili sec --lambda expression — Ashok_Pradhan
– Ashok_Pradhan, Commented Mar 28, 2014 at 6:28
Your benchmarking methodology is a bit simplistic. You should use a framework such as jmh. — assylias
– assylias, Commented Mar 28, 2014 at 8:58

nosid · Accepted Answer · 2014-03-29 09:53:34Z

3

Most likely, the problem is that during your test the JIT compiler (re-)compiles code. That means that your comparison isn't fair, because the later tests benefit from compilations caused by the earlier tests.

This is a frequently made mistake of micro benchmarks. There are a lot of articles explaining the problem, e.g. Robust Java benchmarking. If I add some code to warmup the JIT first, the results are expected. My main method looks as follows:

public static void main(String... args) {
    System.out.println("Warmup...");
    for (int i = 0; i < 5000; ++i) {
        run(Demo::testLoop, 5000);
        run(Demo::testStream, 5000);
        run(Demo::testParallel, 5000);
    }
    System.out.println("Benchmark...");
    int bound = 10000000;
    System.out.printf("Loop:     %s\n", run(Demo::testLoop, bound));
    System.out.printf("Stream:   %s\n", run(Demo::testStream, bound));
    System.out.printf("Parallel: %s\n", run(Demo::testParallel, bound));
}

And the output looks as follows:

Loop:     7.06s (664579)
Stream:   7.06s (664579)
Parallel: 3.84s (664579)

If you pass the option -XX:+PrintCompilation to the Java VM, you can see when and where the JIT kicks in, and that almost all compilations now happen during the warmup phase.

Note that parallel streams are not the best solution for this kind of parallelization, because the complexity of isPrime depends on the value. That means, the first half of the sequence requires significantly less work than the second half (and so on).

For reference, here are the remaining methods of my implementation:

public static boolean isPrime(int value) {
    for (int i = 2; i * i <= value; ++i)
        if (value % i == 0) return false;
    return true;
}

public static long testLoop(int bound) {
    long count = 0;
    for (int i = 2; i < bound; ++i)
        if (isPrime(i)) ++count;
    return count;
}

public static long testStream(int bound) {
    return IntStream.range(2, bound).filter(Demo::isPrime).count();
}

public static long testParallel(int bound) {
    return IntStream.range(2, bound).parallel().filter(Demo::isPrime).count();
}

public static String run(IntToLongFunction operation, int bound) {
    long start = System.currentTimeMillis();
    long count = operation.applyAsLong(bound);
    long millis = System.currentTimeMillis() - start;
    return String.format("%4.2fs (%d)", millis / 1000.0, count);
}

edited Mar 29, 2014 at 9:53

answered Mar 28, 2014 at 12:08

nosid

50.3k13 gold badges117 silver badges146 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ashok_Pradhan Over a year ago

Can you elaborate what is JIT warm up ,If I am not wrong you are doing warm up in side main method by that for loop . thanks for your detailed explanation

nosid Over a year ago

I have added a link to an article that explains warmup. You can use the command line option -XX:+PrintCompilation to see what's going on.

edharned · Accepted Answer · 2014-03-28 13:59:13Z

0

It is a well known fact that the F/J framework needs to warm up. The code is written in such a fashion that it only performs well when compiled. You also have to consider the time it takes to create threads. Having a warm up period in a production environment is subjective.

There is a lot of awful code in the framework to overcome this sluggish behavior when first starting.

answered Mar 28, 2014 at 13:59

edharned

1,9041 gold badge19 silver badges20 bronze badges

Collectives™ on Stack Overflow

Java 8 Parallel processing and Lambda expression

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related