Parallel execution of a loop that uses async

Question

I'm writing a Windows service and am looking for a way to execute a number of foreach loops in parallel, where each loop makes a call to an asynchronous (TAP) method. I initially tried the following code, which doesn't work because Parallel.ForEach and async/await are not compatible. Does anyone know whether there is an alternate approach that can achieve this?

Parallel.ForEach(messagesByFromNumber, async messageGroup =>
{
    foreach (var message in messageGroup)
    {
        await message.SendAsync();
    }
});

For the sake of clarity, due to the way that SendAsync() operates, each copy of the foreach loop must execute serially; in other words, the foreach loop cannot become concurrent / parallel.

I'm not entirely convinced you would need the parallel at all since the async call is a fire and forget type method. Just do a standard for each and fire the async method should do something very close to what you want it to do — theDarse
– theDarse, Commented Dec 31, 2014 at 19:15
@theDarse The reason I couldn't use a standard foreach as the outer loop is because the SendAsync() method is so slow that I needed the outer loop to have the chance to run on separate threads. Doing two nested standard foreach loops would take too long. — Jeff Glazier
– Jeff Glazier, Commented Dec 31, 2014 at 19:39
Don't mix parallel and async code. Use async when you don't want to burn a thread (usually I/O calls), Parallel is for CPU intensive calculations to execute on all cores in parallel. — thoean
– thoean, Commented Jan 1, 2015 at 14:37

i3arnon · Accepted Answer · 2014-12-31 19:25:15Z

5

There's no need to use Parallel.Foreach if your goal is to run these concurrently. Simply go over all your groups, create a task for each group that does a foreach of SendAsync, get all the tasks, and await them all at once with Task.WhenAll:

var tasks = messagesByFromNumber.Select(async messageGroup =>
{
    foreach (var message in messageGroup)
    {
        await message.SendAsync();
    }
});
await Task.WhenAll(tasks)

edited Dec 31, 2014 at 19:25

answered Dec 31, 2014 at 19:19

i3arnon

117k33 gold badges334 silver badges358 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jeff Glazier Over a year ago

I had no idea that async could be used in the select statement that way. Just tried your solution and the code ran as desired. Thanks so much!

i3arnon Over a year ago

@JeffGlazier sure... any time.

Serge Semenov · Accepted Answer · 2016-08-26 21:20:18Z

0

You can make it more cleat with AsyncEnumerator NuGet Package:

using System.Collections.Async;

await messagesByFromNumber.ParallelForEachAsync(async messageGroup =>
{
    foreach (var message in messageGroup)
    {
        await message.SendAsync();
    }
}, maxDegreeOfParallelism: 10);

answered Aug 26, 2016 at 21:20

Serge Semenov

10.2k4 gold badges25 silver badges24 bronze badges

Collectives™ on Stack Overflow

Parallel execution of a loop that uses async

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related