Java create unique thread per database row

Question

I have the requirement to poll the database every x seconds and pull out records that need to be processed. I will start a thread for each of these rows. The next time I poll the threads may not have completed yet. I don't want to start another thread for the same ID as it is still processing.

A HashMap<String, Callable> seems appropriate for storing the threads and ensuring that only one thread per ID will exist . What I can't figure out is how do I remove the thread after the thread is done?

My understanding is that if I wait on the thread then it will be blocking the main thread and the polling will not continue.

I have tried the following code and it works, but the HashMap will continue to grow as more tasks come in. There is a possibility that the row could be reprocessed later in the day.

HashMap<String, Callable<String>> callables = new HashMap<String, Callable<String>>();
for(int i =0; i < 10; i++)
{
    for(int j =0; j < 10; j++)
    {
        String key = Integer.toString(j);
        if(!callables.containsKey(key))
        {
            callables.put(key, new Callable<String>() {
                public String call() throws Exception {
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    System.out.println("Inside task");
                    return "Task Completed";
                }
            });
            try
            {
                callables.get(key).call();
            }
            catch(Exception ex){
                ex.printStackTrace();
            }
        }
        else
            System.out.println("Task skipped: " + j);
    }
}

Why convert the integer to a string? Why not just use the integer as the key? What are the double loops for? — markspace
– markspace, Commented Mar 9, 2018 at 16:33
I was contriving an example that would help illustrate the problem. My actual key will be a string. The double loop is to demonstrate that the next time through the "key" would not be processed again — Richard Binnington
– Richard Binnington, Commented Mar 9, 2018 at 16:35
You need a thread pool. docs.oracle.com/javase/tutorial/essential/concurrency/… — Yati Sawhney - Yates
– Yati Sawhney - Yates, Commented Mar 9, 2018 at 16:36
You also need to forget about thread and use Executor/ThreadPool, etc — Jean-Baptiste Yunès
– Jean-Baptiste Yunès, Commented Mar 9, 2018 at 16:41

vincrichaud · Accepted Answer · 2018-03-09 16:56:02Z

1

If your problem is to remove the couple (key, Callable) from your Map when the thread has finished its job. Call remove() at the end of its call() function.

HashMap<String, Callable<String>> callables = new HashMap<String, 
Callable<String>>();
for(int i =0; i < 10; i++) {
    for(int j =0; j < 10; j++) {
        String key = Integer.toString(j);
        if(!callables.containsKey(key)) {
            callables.put(key, new Callable<String>() {
                final String mKey = key; //store the current key
                public String call() throws Exception {
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    System.out.println("Inside task");
                    callables.remove(mKey); //Remove the thread from the Map
                    return "Task Completed";
                }
            });
            try {
                callables.get(key).call();
            }
            catch(Exception ex) {
                ex.printStackTrace();
            }
        }
        else
            System.out.println("Task skipped: " + j);
    }
}

edited Mar 9, 2018 at 16:56

answered Mar 9, 2018 at 16:45

vincrichaud

2,20620 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Richard Binnington Over a year ago

interesting. I didn't think you were allowed to access items outside the lambda. I am not seeing how do you get key into mKey?

vincrichaud Over a year ago

@RichardBinnington Edited to add key in mKey

Richard Binnington Over a year ago

Thank you sir! I appreciate it a lot!

OldCurmudgeon · Accepted Answer · 2018-03-09 16:59:57Z

You can wrap Callables inside your own Callable.

HashMap<String, Callable<String>> callables = new HashMap<>();

public void oneOnly(String id, Callable<String> callable) {
    if (!callables.containsKey(id)) {
        // Wrap the callable in my own.
        callables.put(id, new Callable<String>() {
            public String call() throws Exception {
                String result = callable.call();
                // Remove when finished.
                callables.remove(id);
                return result;
            }
        });

    }
}

public void test(String[] args) throws Exception {
    for (int i = 0; i < 10; i++) {
        for (int j = 0; j < 10; j++) {
            String key = Integer.toString(j);
            oneOnly(key, new Callable<String>() {
                public String call() throws Exception {
                    System.out.println("Inside task");
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    return "Task Completed";
                }
            });
            try {
                callables.get(key).call();
            } catch (Exception ex) {
                ex.printStackTrace();
            }
        }
    }
}

Collectives™ on Stack Overflow

Java create unique thread per database row

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related