Java Multithread File Read + Operation + Write

Question

I am trying to write a small program to solve the following problems using multithreading in java. I am struggling to even understand where to start, and am looking for some advice. The desired steps in the process are as follows:

Read in all the text files contained within a given directory
Create a word count for each one of the files read.
Write the count of words as an output to a new file in a different directory.

I have written the wordcount function, and that works fine, but would like to know more about how to multithread this operation so that the files are read, the words counted, and then the output is all written in parallel.

Perhaps using Stream functionality in Java (docs.oracle.com/javase/tutorial/collections/streams/…). Create a List of files contained within the directory, and get a parallelStream of those File objects. Then you can process each file, returning the word count as part of the parallel stream processing. Then you can figure out what to do with the count... if each is written to their own file, that too could be part of the stream processing. I'd start there — EdH
– EdH, Commented Nov 16, 2018 at 22:22

K. M. Fazle Azim Babu · Accepted Answer · 2018-11-16 22:24:33Z

1

Can you share the single threaded version? Conceptually it can be as simple as this (pseudo Java code). countWords and writeOutput are your methods and files is a list of files you already read in.

files.parallelStream()
    .map(file -> new Pair(file, countWords(file)))
    .forEach((file, count) -> writeOutput(file, count));

answered Nov 16, 2018 at 22:24

K. M. Fazle Azim Babu

1956 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

dorony · Accepted Answer · 2018-11-16 23:13:12Z

0

I like your enthusiastic in learning multi-threaded programming!
Beside parallel your task between files, you can also parallel and optimize the actual countWords task.

This is a classic case for using Fork/Join. A rather advance level of multi-threaded programming but a very satisfying one :)

you can search the web for examples similar to your own, here is a one to start with: https://www.baeldung.com/java-fork-join

answered Nov 16, 2018 at 23:13

dorony

1,0821 gold badge16 silver badges32 bronze badges

Collectives™ on Stack Overflow

Java Multithread File Read + Operation + Write

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related