Data processing using multithreading java

Question

I have two folders which contain several csv files. Folder 1 has several csv files and Folder 2 has several csv files. Based on certain conditions, I am comparing csv files (code for which is present in compareCSV(File f1, File f2)) and writing into a new output file.

public void traverseThroughFiles(){
      //some filename checking code
      //if output is true call below function
      compareCSV(File f1, File f2)
}

Now I wish to use multithreading in java so that I can process multiple files simultaneously. As per my understanding, if we could call function comapareCSV(File f1, File f2) using multiple threads then I should achieve my goal. Number of threads need to be determined by the user. But, consider numberofThreads = 5

Are there any functions/classes which are present in java that can solve my problem?

I have tried to use ThreadPool and I was unable to implement in my case. Also, how can I use ExecutorService in my case?

assylias · Accepted Answer · 2016-05-20 18:40:28Z

1

Unless you have more than one hard drive, parallelising the process won't speed up things since your operation is essentially I/O constrained - it would probably even be detrimental to performance.

Assuming that using more than one thread may improve performance, you can simply do:

ExecutorService executor = Executors.newFixedThreadPool(5);

executor.submit(() -> compareCSV(f1, f2));
executor.submit(() -> compareCSV(f3, f4));
executor.submit(() -> compareCSV(f5, f6));
executor.submit(() -> compareCSV(f7, f8));

and the method will execute in parallel (you will probably want to store the files in some collection and use a loop but you get the idea).

answered May 20, 2016 at 18:40

assylias

330k84 gold badges680 silver badges806 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Globox Over a year ago

Thanks for your help. This is working. Except that executor.shutdown() needs to be called after completion.

Collectives™ on Stack Overflow

Data processing using multithreading java

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related