0

I am looking to make a skeleton that will process concurrent downloads. I am not sure if I get that for free using Task.Run or not? My code below has a foreach loop, but what I really want is not to go one at a time but take 5 concurrently on their own thread and then continue processing whenever one gets done move it through more processing. Am I doing this now?

  public async Task ProcessBatch()
        {
            CreateConnection();

            List<string> items=new List<string>();

            foreach (var item in items)
            { 
                //do do stuff that possibly takes awhile
                var result = await Task.Run(() => DownloadItems());

                //do more processing on whatever item is done now
            }
        }
Improve this question

2 Answers 2

Reset to default
4

Am I Creating Multithreading Using Task.Run

I am looking to make a skeleton that will process concurrent downloads.

It's important to distinguish parallelism from concurrency. Parallelism is about multiple threads. Concurrency can be accomplished by parallelism, but it can also be accomplished by asynchronous code.

In particular, threads are a great fit if you have CPU-bound work to do, whereas asynchrony excels at I/O. Since "downloads" strongly implies I/O, I'd recommend an asynchronous approach:

public async Task ProcessBatchAsync()
{
  CreateConnection();

  List<string> items = new List<string>();
  var tasks = items.Select(item => ProcessItemAsync(item));
  await Task.WhenAll(tasks);
}

private async Task ProcessItemAsync(string item)
{
  var result = await DownloadItemAsync(item);
  //do more processing
}

However, if DownloadItem isn't already asynchronous and you don't want to invest the time to make it so, then a multithreading approach (Parallel / Parallel LINQ, or Task.Run if you must) is an acceptable compromise for a UI application.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

3

You're causing a different thread to be used by Task.Run(), but then you're causing your for loop to await that thread's return before it moves on to the next item, so you're not getting any concurrency out of this.

One alternative would be to use Task.Run() to produce a list of tasks first, and then await them all after they've been created.

var tasks = new List<Task<DownloadResult>>();
        foreach (var item in items)
        { 
            //do do stuff that possibly takes awhile
            tasks.Add(Task.Run(() => DownloadItems()));
        }
        foreach (var task in tasks)
        { 
            var result = await task;
            //do more processing on whatever item is done now
        }

However, for something this simple, I'd recommend using the .AsParallel() extension method that's part of the TPL.

items.AsParallel()
    .Select(item => DownloadItems())
    .ForAll(result => AdditionalProcessing(result));
Improve this answer

10 Comments

how do I make the concurrency happen?
Can you show me the Task.Run structure? For the AsParallel how would I act on an item that completes? So when item 5 completes which could be before item 1 I want to continue processing it. Also what if there is 5000 items in the list
In your first example how can I tell it to only do 5 at a time? For the case I have 5000 items to process. I cant make 5000 threads can I?
Im so lost what I a "DownloadResult" object?
@npl77: See my update for your first couple of questions. In the AsParallel example you can use AsOrdered() to ensure the second step is processed in order, even though the first step may return values out of order. You can use .WithMaxDegreeOfParallelism() to control how many threads are used, although the system is usually fairly smart about choosing an effective maximum by default. Using Task.Run() and await would be much more difficult to control parallelism.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.