Multithreading/Parallelizing each item of for loop JAVA

Question

I am trying to have each thread access a single item of for loop while another thread accesses the next item. I want to do this using multiple threads and number of multiple threads created will be input by the user. I have done this using executorservice and streams. I want to do this using simple threads. Is the below correct? Is there a better way?

Map<String, String> fileMap = new HashMap<>();
fileMap.put("Age", "Age is not remotely associated with it.");
fileMap.put("Gender", "Gender plays a role but not that important.");
fileMap.put("Money", "People do not believe but this is the only factor that matters.");

Runnable myRunnable = new Runnable(){
    public void run(){
        for (Map.Entry<String, String> entry : fileMap.entrySet()) {
            synchronized(this){
                int counter = 0;
                Pattern p = Pattern.compile("not");
                Matcher m = p.matcher(entry.getValue());
                while (m.find()) {
                    counter++;
                }
                System.out.println("File Name: " + entry.getKey());
                System.out.println("Count: " + counter);
                System.out.println(Thread.currentThread().getName());
            }
        }
    }    
};

int n = Integer.parseInt(args[0]);
for (int x=0; x<n; x++)
{
    Thread temp= new Thread(myRunnable, "Thread #" + x);
    temp.start();
    System.out.println("Started Thread:" + x);
}

Also, is it possible to have a thread not to go back to previous item since a previous thread has already computed the value? Any help would be appreciated. Thanks

seems pointless to have threads and then try to block it from running due to synchronized. although what is this in this case? If you want to prevent it from re-processing already found lines, then maybe remove it from the map — Scary Wombat
– Scary Wombat, Commented Dec 1, 2016 at 1:17
It is not clear to me what you mean by simple thread. Java Threads do not have indexes (such as you see in OpenCL/Cuda) so you would need to extend the functionality of threads if you want to give them indexes though your runable COULD hold the index. Your logic in this application is also flawed since the number of threads you are creating won't always match the number of items you are processing. Minimally I would change your for loop from x<n to x < n && x < fileMap.size() — Ralph Ritoch
– Ralph Ritoch, Commented Dec 1, 2016 at 1:25
@RalphRitoch Excuse my English. I meant 'do this using simply threads' and not through ExecutorService or any other package. Just threads along with maybe Maps/ConcurrentHashMaps — Tao
– Tao, Commented Dec 1, 2016 at 21:02

Ralph Ritoch · Accepted Answer · 2016-12-01 02:24:55Z

2

Here is a solution to your problem. This parses the thread name to provide the index and uses final arrays to handle passing data into the threads.

Map<String, String> fileMap = new HashMap<>();
fileMap.put("Age", "Age is not remotely associated with it.");
fileMap.put("Gender", "Gender plays a role but not that important.");
fileMap.put("Money", "People do not believe but this is the only factor that matters.");


final int[] tgSize = new int[]{0};
final Map.Entry[][] entryArr = new Map.Entry[1][];

Runnable myRunnable = new Runnable(){
    public void run(){
        Integer index = Integer.valueOf(Thread.currentThread().getName().substring(8));

        for(int i = index; i < fileMap.size(); i += tgSize[0]) {
            int counter = 0;
            @SuppressWarnings("unchecked")
            Map.Entry<String, String> entry = entryArr[0][i];
            Pattern p = Pattern.compile("not");
            Matcher m = p.matcher(entry.getValue());
            while (m.find()) {
                counter++;
            }
            synchronized(this) {
                System.out.println("File Name: " + entry.getKey());
                System.out.println("Count: " + counter);
                System.out.println(Thread.currentThread().getName());            
            }
        }
    }    
};

int n = Integer.parseInt(args[0]);

tgSize[0] = n < fileMap.size() ? n : fileMap.size();
entryArr[0] = fileMap.entrySet().toArray(new Map.Entry[fileMap.size()]);


for (int x=0; x<n && x < fileMap.size(); x++)
{
    Thread temp= new Thread(myRunnable, "Thread #" + x);
    temp.start();
    System.out.println("Started Thread:" + x);
}

edited Dec 1, 2016 at 2:24

answered Dec 1, 2016 at 2:11

Ralph Ritoch

3,51030 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Tao Over a year ago

Thanks. This works perfectly and what I was looking for. But can you let me know why we need to create Map.Entry[ ][ ]?

Tao Over a year ago

I got it. Its going to the array for later being processed, correct?

Ralph Ritoch Over a year ago

Correct, it is a mirror of how GPU processing works. First you set the data into global memory and then you process the items using the index to ensure that shaders aren't doing the same work. Final arrays are a good way to mimic global (shared) memory in java.

user_3380739 · Accepted Answer · 2023-01-13 05:18:40Z

1

It can be achieved by paralledStream by abacus-common

final Pattern p = Pattern.compile("not");

Stream.of(fileMap).parallel(threadNum).map(entry -> {
    Matcher m = p.matcher(entry.getValue());
    int count = 0;
    while (m.find()) {
        count++;
    }
    return Pair.of(entry.getKey(), count);
}).forEach(entry -> {
    N.println("File Name: " + entry.getKey() + ", Count: " + entry.getValue());
});

If you want to learn how to write multi-thread code by yourself. here is a simple sample:

final int threadNum = 3;
final ExecutorService executorService = Executors.newFixedThreadPool(threadNum);
final Iterator<Entry<String, String>> iter = fileMap.entrySet().iterator();

for (int i = 0; i < threadNum; i++) {
    executorService.execute(new Runnable() {
        @Override
        public void run() {
            Entry<String, String> entry = null;
            while (true) {
                synchronized (iter) {
                    if (iter.hasNext() == false) {
                        break;
                    }
                    entry = iter.next();
                }

                final Matcher m = p.matcher(entry.getValue());
                int count = 0;
                while (m.find()) {
                    count++;
                }

                System.out.println("File Name: " + entry.getKey() + ", Count: " + count + ", thread: " + Thread.currentThread().getName());
            }
        }
    });
}

Declaration： I'm the developer of abacus-common.

edited Jan 13, 2023 at 5:18

answered Dec 1, 2016 at 2:28

user_3380739

1,32416 silver badges15 bronze badges

9 Comments

Ralph Ritoch Over a year ago

Nice library but he did say in the question that he wanted to use "simple" threads. He also didn't define what he means by simple.

Ralph Ritoch Over a year ago

Is your library on maven?

GOXR3PLUS Over a year ago

Hey i really like this library you mentioned!

user_3380739 Over a year ago

@Ralph, I edit the answer based on your comments. and yes, it's on Maven

user_3380739 Over a year ago

@GOXR3PLUS, glad to know you like it

|

Paul Buis · Accepted Answer · 2016-12-01 03:31:15Z

The standard way to parallelize a loop using raw threads iterating over an array is illustrated below using your problem.

import java.util.*;
import java.util.regex.*;

public class MyClass {

public static void main(String[] args) {
    Map<String, String> fileMap = new HashMap<>();
    fileMap.put("Age", "Age is not remotely associated with it.");
    fileMap.put("Gender", "Gender plays a role but not that important.");
    fileMap.put("Money", "People do not believe but this is the only factor that matters.");
    String[] keys = fileMap.keySet().toArray(new String[fileMap.size()]);

    int n = 2; //Integer.parseInt(args[0]);
    for (int x=0; x<n; x++)
    {
        Runnable myRunnable = new MyRunnable(fileMap, keys, x, n);
        Thread temp= new Thread(myRunnable);
        temp.start();
        //System.out.println("Started Thread:" + x);
    }
}

    private static class MyRunnable implements Runnable {
        private Map<String, String> fileMap;
        private String[] keys;
        private int threadID;
        private int threadCount;
        Pattern p = Pattern.compile("not");
        public MyRunnable(Map<String, String> fileMap, String[] keys, int threadID, int threadCount) {
            this.fileMap = fileMap;
            this.keys = keys;
            this.threadID = threadID;
            this.threadCount = threadCount;
        }
        public void run(){
            for (int i=threadID; i<keys.length; i+= threadCount) {
                int counter = 0;
                Matcher m = p.matcher(fileMap.get(keys[i]));
                while (m.find()) {
                    counter++;
                }
                synchronized(MyClass.class){
                    System.out.println("File Name: " + keys[i]);
                    System.out.println("Count: " + counter);
                    System.out.println("ThreadID: " + threadID);
                }
            }
        }    
    }
    }

Thanks. I was trying to do it without creating another class. But the solution is simpler. Thank you

Collectives™ on Stack Overflow

Multithreading/Parallelizing each item of for loop JAVA

3 Answers 3

3 Comments

9 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

9 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related