SSH.Net Async file download

Question

I am trying to download files asynchronously from an SFTP-server using SSH.NET. If I do it synchronously, it works fine but when I do it async, I get empty files. This is my code:

var port = 22;
string host = "localhost";
string username = "user";
string password = "password";
string localPath = @"C:\temp";

using (var client = new SftpClient(host, port, username, password))
{
    client.Connect();
    var files = client.ListDirectory("");

    var tasks = new List<Task>();

    foreach (var file in files)
    {                        
        using (var saveFile = File.OpenWrite(localPath + "\\" + file.Name))
        {
            //sftp.DownloadFile(file.FullName,saveFile); <-- This works fine
            tasks.Add(Task.Factory.FromAsync(client.BeginDownloadFile(file.FullName, saveFile), client.EndDownloadFile));
        }                        
    }

    await Task.WhenAll(tasks);
    client.Disconnect();

}

Thomas Levesque · Accepted Answer · 2016-02-26 00:19:48Z

17

Because saveFile is declared in a using block, it is closed right after you start the task, so the download can't complete. Actually, I'm surprised you're not getting an exception.

You could extract the code to download to a separate method like this:

var port = 22;
string host = "localhost";
string username = "user";
string password = "password";
string localPath = @"C:\temp";

using (var client = new SftpClient(host, port, username, password))
{
    client.Connect();
    var files = client.ListDirectory("");

    var tasks = new List<Task>();

    foreach (var file in files)
    {                        
        tasks.Add(DownloadFileAsync(file.FullName, localPath + "\\" + file.Name));
    }

    await Task.WhenAll(tasks);
    client.Disconnect();

}

...

async Task DownloadFileAsync(string source, string destination)
{
    using (var saveFile = File.OpenWrite(destination))
    {
        var task = Task.Factory.FromAsync(client.BeginDownloadFile(source, saveFile), client.EndDownloadFile);
        await task;
    }
}

This way, the file isn't closed before you finish downloading the file.

Looking at the SSH.NET source code, it looks like the async version of DownloadFile isn't using "real" async IO (using IO completion port), but instead just executes the download in a new thread. So there's no real advantage in using BeginDownloadFile/EndDownloadFile; you might as well use DownloadFile in a thread that you create yourself:

Task DownloadFileAsync(string source, string destination)
{
    return Task.Run(() =>
    {
        using (var saveFile = File.OpenWrite(destination))
        {
            client.DownloadFile(source, saveFile);
        }
    }
}

edited Feb 26, 2016 at 0:19

answered Dec 2, 2015 at 10:05

Thomas Levesque

294k73 gold badges639 silver badges769 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

spersson Over a year ago

Thank you for the answer, however, I still get the same empty files when I try this. No exception either.

Thomas Levesque Over a year ago

@spersson, I updated my answer. Looks like there is no advantage in using BeginDownloadFile, so you might as well use the synchronous version.

spersson Over a year ago

Thanks for taking the time, I guess I'll use the synchronous version then :)

Ronnie Overby Over a year ago

Though it is a bummer that they don't use IO completion ports in their implementation, the advantage to using the Begin* async overloads over creating your own thread is that some of their IAsyncResult implementations expose a cancellation mechanism, should you need to halt transfers midway through. I wrapped BeginUploadFile in an extension method to make things feel more modern. Check it out: gist.github.com/ronnieoverby/438034b19531e6272f98

HackSlash · Accepted Answer · 2023-12-29 18:00:18Z

6

They finally updated the methods a while back but didn't add explicit DownloadFileAsync as was originally requested. You can see the discussion here:

https://github.com/sshnet/SSH.NET/pull/819

Where the solution provided is:

/// <summary>
/// Add functionality to <see cref="SftpClient"/>
/// </summary>
public static class SftpClientExtensions
{
    private const int BufferSize = 81920;

    public static async Task DownloadFileAsync(this SftpClient sftpClient, string path, Stream output, CancellationToken cancellationToken)
    {
        await using Stream remoteStream = await sftpClient.OpenAsync(path, FileMode.Open, FileAccess.Read, cancellationToken).ConfigureAwait(false);
        await remoteStream.CopyToAsync(output, BufferSize, cancellationToken).ConfigureAwait(false);
    }

    public static async Task UploadFileAsync(this SftpClient sftpClient, Stream input, string path, FileMode createMode, CancellationToken cancellationToken)
    {
        await using Stream remoteStream = await sftpClient.OpenAsync(path, createMode, FileAccess.Write, cancellationToken).ConfigureAwait(false);
        await input.CopyToAsync(remoteStream, BufferSize, cancellationToken).ConfigureAwait(false);
    }
}

answered Dec 29, 2023 at 18:00

HackSlash

6,0272 gold badges28 silver badges58 bronze badges

2 Comments

Rayyyz Over a year ago

Should be the right answer tbh (at least the most up to date one). Hope every other people could accept this one for other people struggling doing an async download with this package.

Rayyyz May 13 at 9:09

I upvoted this answer, but i have to say, there is a big performance gap between this async implementation and the synchronous "DownloadFile" in my case (x5 faster while staying with the synchronous way)

baltermia · Accepted Answer · 2024-06-27 07:56:59Z

0

I found that this solution (mentioned in another answer) did work, but it was extremely slow. I've tried switching the BufferSize a bunch of times but the performance was nothing compared to the default DownloadFile method.

I've found another solution which allows the download to be run asynchronously without the bottleneck:

public static class SftpClientExtensions
{
    public static async Task DownloadFileAsync(this SftpClient client, string source, string destination, CancellationToken cancellationToken)
    {
        TaskCompletionSource<bool> tcs = new();

        using (FileStream saveFile = File.OpenWrite(destination))
        // cancel the tcs when the token gets cancelled
        using (cancellationToken.Register(() => tcs.TrySetCanceled()))
        {
            // begin the asynchronous operation
            client.BeginDownloadFile(source, saveFile, result =>
            {
                try
                {
                    // Try to complete the operation
                    client.EndDownloadFile(result);

                    // indicate success
                    tcs.TrySetResult(true);
                }
                catch (OperationCanceledException)
                { 
                    // properly handle cancellation
                    tcs.TrySetCanceled();
                }
                catch (Exception ex)
                {
                    // handle any other exceptions
                    tcs.TrySetException(ex);
                }
            }, null);

            // await the task to complete or be cancelled
            await tcs.Task;

            // since we catch the cancellation exepction in the task, we need to throw it again
            cancellationToken.ThrowIfCancellationRequested();
        }
    }
}

answered Jun 27, 2024 at 7:56

baltermia

1,4271 gold badge17 silver badges33 bronze badges

2 Comments

Rayyyz May 13 at 8:42

1) I don't know why you were downvoted here because you're absolutely right on the performance, there is a big performance gap between DownloadFile and the async way. 2) You should use "Task.Factory.FromAsync" to handle this async pattern, TaskCompletionSource add complexity here (in my opinion)

baltermia May 14 at 6:29

Hi @Rayyyz, thank you for suggesting Task.Factory.FromAsync. I wasn't even aware this existed; it seems to greatly reduce the complexity, as you mentioned. I will have to look into its details, but will update the code accordingly as soon as I find time.

Collectives™ on Stack Overflow

SSH.Net Async file download

3 Answers 3

4 Comments

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related