I'm designing a file upload API that needs to work with large files. I want to stay away from passing around byte arrays. The endpoint storage for the file will be a third party such as Azure or Rackspace file storage.
I have the following project structure, which is following DDD:
- Web API (Netcore - accepts the uploaded file)
- Business Service (calls to Azure to save the file and saves a record to the database)
- Domain (Domain models for EF)
- Persistence (Repositories EFCore - saves the database changes)
I would like to have methods in each that can start passing through the uploaded filestream as soon as the upload starts. I'm unsure if this is possible?
Previously we've used byte[] to pass the files through the layers, but for large files this seems to require lots of memory to do so and has caused us issues.
Is it possible to optimize the upload of files through a ntier application, so you don't have to copy around large byte arrays, and if so, how can it be done?
In order to clarify, the code structure would be something like the following. Repository stuff has been excluded:
namespace Project.Controllers
{
[Produces("application/json")]
[Route("api/{versionMNumber}/")]
public class DocumentController : Controller
{
private readonly IAddDocumentCommand addDocumentCommand;
public DocumentController(IAddDocumentCommand addDocumentCommand)
{
this.addDocumentCommand = addDocumentCommand;
}
[Microsoft.AspNetCore.Mvc.HttpPost("application/{applicationId}/documents", Name = "PostDocument")]
public IActionResult UploadDocument([FromRoute] string applicationId)
{
var addDocumentRequest = new AddDocumentRequest();
addDocumentRequest.ApplicationId = applicationId;
addDocumentRequest.FileStream = this.Request.Body;
var result = new UploadDocumentResponse { DocumentId = this.addDocumentCommand.Execute(addDocumentRequest).DocumentId };
return this.Ok(result);
}
}
}
namespace Project.BusinessProcess
{
public interface IAddDocumentCommand
{
AddDocumentResponse Execute(AddDocumentRequest request);
}
public class AddDocumentRequest
{
public string ApplicationId { get; set; }
public Stream FileStream { get; set; }
}
public class AddDocumentResponse
{
public Guid DocumentId { get; set; }
}
public class AddDocumentCommand : IAddDocumentCommand
{
private readonly IDocuentRepository documentRepository;
private readonly IMessageBus bus;
public AddDocumentCommand(IDocumentRepository documentRepository, IMessageBus bus)
{
this.documentRepository = documentRepository;
this.bus = bus;
}
public AddDocumentResponse Execute(AddDocumentRequest request)
{
/// We need the file to be streamed off somewhere else, fileshare, Azure, Rackspace etc
/// We need to save a record to the db that the file has been saved successfully
/// We need to trigger the background workers to process the uploaded file
var fileUri = AzureStorageProvider.Save(request.FileStream);
var documentId = documentRepository.Add(new Document { FileUri = fileUri });
bus.AddMessage(new DocumentProcessingRequest { documentId = documentId, fileUri = fileUri });
return new AddDocumentResponse { DocumentId = documentId };
}
}
}
Streambetween layers in your example, just like you want to. It doesn't work as you expect or what?