3

The following code is a program.

  • The payloadList contains json-objects like {"id": 1, "foo": "one" }.
  • Each object of payloadList should be sent to a server with httpClient.SendAsync()
  • The response for each request should be stored in responseList

The code below does partly work. But i do not understand why some parts are not working. I assume that the responses are not completed when responseList.Add(foo) is executed.

This request shall be send for each json-object {"id": 1, "foo": "one" }

public static async Task<string> DoRequest(HttpClient client, string payload)
{   
    var request = new HttpRequestMessage(HttpMethod.Post, 
                           "http://httpbin.org/anything");
    request.Content = new StringContent(payload
                           , Encoding.UTF8, "application/json");        
    var response = await client.SendAsync(request); 
    string responseContent = await response.Content.ReadAsStringAsync(); 
    return responseContent;
}

The DoRequest()-method wraps the request and can be used inside main like this

static async Task Main()
{
    var responseList = new List<string>();  
    var payloadList = new List<string>{"{ 'id': 1, 'bar': 'One'}",
                                       "{ 'id': 2, 'bar': 'Two'}",
                                       "{ 'id': 3, 'bar': 'Three'}"};
        
    var client = new HttpClient();
    
    payloadList.ForEach(async (payload) => {
        var responseFoo = await DoRequest(client, payload);
        responseFoo.Dump("response"); // contains responseFoo
        responseList.Add(responseFoo);  // adding responseFoo to list fails
    });                     
    responseList.Dump();    // is empty
}

The responseList is empty.

  • Expected responseList.Dump() contains all responses responseFoo.
  • Actual responseList is empty.

Linqpad-demo

Questions

  • How can each response for await client.SendAsync(request) be added to a responseList?
  • Why is responseList empty despite that foo.Dump() works?
  • How to confirm or check if every client.SendAsync is finished?
  • Would you write the code above different - why?
Improve this question
7
  • ask one question at a time please. See also How to Ask Commented Dec 8, 2022 at 14:14
  • responseList.Add(foo); // this not ...in what way does it "not work"? Is there an error? Commented Dec 8, 2022 at 14:14
  • The responseList is empty. Expected responseList.Dump() contains all responses foo. Actual responseList is empty. Commented Dec 8, 2022 at 14:16
  • P.S. putting responseList.Dump(); outside the foreach loop obviously won't work because that'll run before all the async stuff is complete. Commented Dec 8, 2022 at 14:16
  • stackoverflow.com/questions/25009437/… may be helpful Commented Dec 8, 2022 at 14:17

1 Answer 1

Reset to default
4

List.ForEach is not Task-aware and will execute everything in parallel without waiting for the results (i.e. it will create tasks for all items in payloadList and will continue to the next statement responseList.Dump(); without awaiting them).

  1. In newer versions of .NET you can use Parallel.ForEachAsync(for example as in this answer) combined with use of appropriate collection from System.Collections.Concurrent, for example ConcurrentBag<T>. List is not thread-safe and modifying it concurrently can lead to a lot of problems.

    Code can look like the following:

    var responseList = new ConcurrentBag<string>();

await Parallel.ForEachAsync(payloadList, async (payload, ct) =>
{
    var foo = await DoRequest(client, payload, ct);
    responseList.Add(foo);     
});

static async Task<string> DoRequest(HttpClient client, string payload, CancellationToken ct)
{   
    var request = new HttpRequestMessage(HttpMethod.Post, "http://httpbin.org/anything");
    request.Content = new StringContent(payload, Encoding.UTF8, "application/json");        
    var response = await client.SendAsync(request, ct); 
    string responseContent = await response.Content.ReadAsStringAsync(ct); 
    return responseContent;
}

  2. If you are fine with all requests running in parallel - just create a enumerable of tasks and use Task.WhenAll<T>():

    var tsks = payloadList
    .Select(payload => DoRequest(client, payload));

string[] result = await Task.WhenAll(tsks);

  3. If you want to execute requests one after another - just switch to ordinary foreach:

    var responseList = new List<string>(); // no need for concurrent collection in this case 

foreach (var payload in payloadList)
{
    var foo = await DoRequest(client, payload);
    responseList.Add(foo);  
}

If you want to dive deeper here some links:

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Make sure to await Parallel.ForEachAsync
Could you add some code how to use it?
Thanks for the code. One more thing. I would expect that responseList.Add(foo) should work. But it stays empty. Why is this the case?
@surfmuggle Potentially you app finishes before the requests are completed. And responseList.Dump(); is executed before any request is completed with very high chance. I.e. async-await inside List.ForEach has no effect.
@surfmuggle added some links which can be useful if you will want to go a bit deeper into the topic.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.