C# ForEach Loop With ASync Tasks & Dependent Post ASync Tasks

Question

I'm having trouble trying to correctly architect the most efficient way to iterate several async tasks launched from a request object and then performing some other async tasks that depend on both the request object and the result of the first async task. I'm running a C# lambda function in AWS. I've tried a model like this (error handling and such has been omitted for brevity):

public async Task MyAsyncWrapper()
{
  List<Task> Tasks = new List<Task>();
  foreach (var Request in Requests) 
  {
    var Continuation = this.ExecuteAsync(Request).ContinueWith(async x => {
      var KeyValuePair<bool, string> Result = x.Result;
      if (Result.Key == true)
      {
        await this.DoSomethingElseAsync(Request.Id, Request.Name, Result.Value);
        Console.WriteLine("COMPLETED");
      }
    }

    Tasks.Add(Continuation);
  }

  Task.WaitAll(Tasks.ToArray());
}

This approach results in the DoSomethingElseAsync() method not really getting awaited on and in a lot of my Lambda Function calls, I never get the "COMPLETED" output. I've also approached this in this method:

public async Task MyAsyncWrapper()
{
  foreach (var Request in Requests) 
  {
    KeyValuePair<bool, string> Result = await this.ExecuteAsync(Request);

    if (Result.Key == true)
    {
      await this.DoSomethingElseAsync(Request.Id, Request.Name, Result.Value);
      Console.WriteLine("COMPLETED");
    }
  }
}

This works, but I think it's wasteful, since I can only execute one iteration of the loop while waiting on the asnyc's to finish. I also have referenced Interleaved Tasks but the issue is that I basically have two loops, one to populate the tasks, and another to iterate them after they've completed, where I don't have access to the original Request object anymore. So basically this:

List<Task<KeyValuePair<bool, string>>> Tasks = new List<Task<KeyValuePair<bool, string>>>();

foreach (var Request in Requests)
{
  Tasks.Add(ths.ExecuteAsync(Request);
}

foreach (Task<KeyValuePair<bool, string>> ResultTask in Tasks.Interleaved())
{
  KeyValuePair<bool, string> Result = ResultTask.Result;
  //Can't access the original request for this method's parameters
  await this.DoSomethingElseAsync(???, ???, Result.Value);
}

Any ideas on better ways to implement this type of async chaining in a foreach loop? My ideal approach wouldn't be to return the request object back as part of the response from ExecuteAsync(), so I'd like to try and find other options if possible.

Richard Szalay · Accepted Answer · 2017-09-19 23:55:33Z

3

I may be misinterpreting, but why not move your "iteration" into it's own function and then use Task.WhenAll to wait for all iterations in parallel.

public async Task MyAsyncWrapper()
{
  var allTasks = Requests.Select(ProcessRequest);

  await Task.WhenAll(allTasks);
}

private async Task ProcessRequest(Request request)
{
    KeyValuePair<bool, string> Result = await this.ExecuteAsync(request);

    if (Result.Key == true)
    {
      await this.DoSomethingElseAsync(request.Id, request.Name, Result.Value);
      Console.WriteLine("COMPLETED");
    }
}

answered Sep 19, 2017 at 23:55

Richard Szalay

85.1k19 gold badges186 silver badges242 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mhaken Over a year ago

I think this may be really helpful. I'm gonna try it out and see if it works.

mhaken Over a year ago

This solution worked for me and significantly reduced the amount of code I had written to perform these loops on multiple different request types.

Cory Nelson · Accepted Answer · 2017-09-19 22:16:01Z

3

Consider using TPL dataflow:

var a = new TransformBlock<Input, OutputA>(async Input i=>
{
    // do something async.
    return new OutputA();
});

var b = new TransformBlock<OutputA, OutputB>(async OutputA i =>
{
    // do more async.
    return new OutputB();
});

var c = new ActionBlock<OutputB>(async OutputB i =>
{
    // do some final async.
});

a.LinkTo(b, new DataflowLinkOptions { PropogateCompletion = true });
b.LinkTo(c, new DataflowLinkOptions { PropogateCompletion = true });

// push all of the items into the dataflow.
a.Post(new Input());
a.Complete();

// wait for it all to complete.
await c.Completion;

answered Sep 19, 2017 at 22:16

Cory Nelson

30.3k5 gold badges77 silver badges114 bronze badges

2 Comments

mhaken Over a year ago

So I see how this how this can work for a single chain of actions, but will this work as expected in a foreach loop where Input is a Request so that I can ensure that each c is complete before exiting the method?

Cory Nelson Over a year ago

This would replace a foreach loop: instead of having your code inside the loop, your loop would merely Post data into the dataflow.

Collectives™ on Stack Overflow

C# ForEach Loop With ASync Tasks & Dependent Post ASync Tasks

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related