0

I have a method that iterates a list of objects and for each item in the list fetches data from an external api.

Sometimes this can be very slow (naturally) and I'd like to add all my items to a Taks list instead to be able to run multiple threads at the same time. Is this possible without rewriting it all to be async? Today I'm using WebClient and fetches everything synchronously.

I tried something like this at first:

public Main() 
{
    List<Task<Model>> taskList = new List<Task<Model>>();
    foreach (var aThing in myThings)
    {
        taskList.Add(GetStuffAsync(aThing));
    }

    List<Model> list = Task.WhenAll(taskList.ToArray()).Result.ToList();
}

public async Task<Model> GetStuffAsync(object aThing)
{
    // Stuff that is all synchronous
    // ...
    return anObject;
}
4
  • 1
    I can't see any major problems with this, are you getting an issue, if so can you be specific about it? Commented Nov 11, 2015 at 8:39
  • Your code won't compile as it is. Show us actual code Commented Nov 11, 2015 at 8:40
  • This might help: msdn.microsoft.com/en-IN/library/hh696703.aspx Commented Nov 11, 2015 at 8:43
  • 1
    You can do this by simply wrapping everything with Task.Run. But it's a shame to waste those thread-pool threads to be blocked on IO, especially if your app is resource intensive. Commented Nov 11, 2015 at 8:43

2 Answers 2

5

Rather than using async here, you can just make GetStuff a normal synchronous method and use Task.Run to create new tasks (which will normally be run on multiple threads) so that your fetches occur in parallel. You could also consider using Parallel.ForEach and the like, which is effectively similar to your current Main code.

Your async approach will not do what you want at the moment, because async methods run synchronously at least as far as the first await expression... in your case you don't have any await expressions, so by the time GetStuffAsync returns, it's already done all the actual work - you have no parallelism.

Of course, an alternative is to use your current code but actually do make GetStuffAsync asynchronous instead.

It's also worth bearing in mind that the HTTP stack in .NET has per-host connection pool limits - so if you're fetching multiple URLs from the same host, you may want to increase those as well, or you won't get much parallelism.

Sign up to request clarification or add additional context in comments.

Comments

-1

Try this concept:

foreach(var aThing in myThings)
{
    Thread myTask = new Thread(new ParameterizedThreadStart(TaskMethod));
    myTask.Start(aThing);
}

and there must be a method called TaskMethod that is void TaskMethod(object obj)

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.