C# WinSCP .NET assembly: How to upload multiple files asynchronously

Question

I am using WinSCP .NET assembly for uploading files. I want to upload multiple files asynchronous way. I have created a method but it works as single upload.

public class UploadingData {

    private SessionOptions _sessionOptions;
    private Session _session;

    //connection etc

    private void OnConnected() {

        foreach (var fileInfo in localFilesList)
        {
            var task = Task.Factory.StartNew(() => UploadFilesAsync(fileInfo));
        }
    }

    private async Task UploadFilesAsync(string file) {

        string remoteFilePath = _session.TranslateLocalPathToRemote(file, @"/", "/test_data_upload");
        var uploading = _session.PutFiles(file, remoteFilePath, false);

        //When Done

        await Task.Run(() => Thread.Sleep(1000));
    }
}

Please suggest me correct way. Thanks

Martin Prikryl · Accepted Answer · 2021-02-04 11:58:38Z

3

The API of the Session class can be used from a single thread only. If you use it from multiple threads, it will block.

So if you need parallel transfers, you have to create a separate Session instance for each thread.

private async Task UploadFilesAsync(string file)
{
    using (Session session = new Session())
    {
        session.Open(_sessionOptions);
        string remoteFilePath =
            RemotePath.TranslateLocalPathToRemote(file, @"/", "/test_data_upload");
        session.PutFiles(file, remoteFilePath, false).Check();
    }

    ...
}

See also WinSCP article Automating transfers in parallel connections over SFTP/FTP protocol with code examples for C# and PowerShell.

Note that you should not use more than few sessions/threads (about 4). With more sessions, you will hardly get better throughput.

edited Feb 4, 2021 at 11:58

answered Nov 10, 2016 at 10:09

Martin Prikryl

205k64 gold badges560 silver badges1.1k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Tamerlan Rustambayli · Accepted Answer · 2016-11-10 10:03:37Z

1

You can use BackgroundWorker class like this:

        string[] images = new string[50];
        foreach (var path in images)
        {
            BackgroundWorker w = new BackgroundWorker();
            //50 independently async Threads
            w.DoWork += delegate (object s, DoWorkEventArgs args) 
            {
                Upload(args.Argument.ToString());
            };
            w.RunWorkerAsync(path);
        }

answered Nov 10, 2016 at 10:03

Tamerlan Rustambayli

559 bronze badges

3 Comments

Shoaib Ijaz Over a year ago

If need I to upload thousands of files then there will be thousands of background workers. is it fine for application and CPU?

Tamerlan Rustambayli Over a year ago

Using thousands of theads in one application is not good practise. So that, the more your application has threads, the less quota time each threads executing in. You can seperate thousands of files by 50-50 and upload each parts sequentially with 50 async thread, for example.

Shoaib Ijaz Over a year ago

As @Martin said Session class will put all the files in queue.

Collectives™ on Stack Overflow

C# WinSCP .NET assembly: How to upload multiple files asynchronously

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related