C# Split a file into two byte arrays

Question

Because the maximum value of a byte array is 2GB, lets say i have a larger file and i need to convert it to a byte array. Since i can't hold the whole file, how should i convert it into two?

I tried:

long length = new System.IO.FileInfo(@"c:\a.mp4").Length;
int chunkSize = Convert.ToInt32(length / 2); 


byte[] part2;
FileStream fileStream = new FileStream(filepath, FileMode.Open, FileAccess.Read);
try
{
    part2 = new byte[chunkSize];            // create buffer
    fileStream.Read(part2, 0, chunkSize);
}
finally
{
    fileStream.Close();
}

byte[] part3;
fileStream = new FileStream(filepath, FileMode.Open, FileAccess.Read);
try
{
    part3 = new byte[chunkSize];            // create buffer
    fileStream.Read(part3, 5, (int)(length - (long)chunkSize));
}
finally
{
    fileStream.Close();
}

but it's not working.

Any ideas?

When you need to do something with a large file you should find an approach that doesn't try to read the entire file into memory if you can help it. — juharr
– juharr, Commented Mar 29, 2018 at 15:26
@AaYy Well what do you plain to do with those byte arrays after you have them in memory? There might be an approach that doesn't require the entire file in memory. Chunking it up like that would break any valid reason anyway. This is basically a XY Problem — juharr
– juharr, Commented Mar 29, 2018 at 15:29

FortyTwo · Accepted Answer · 2018-03-29 15:35:32Z

3

You can use a StreamReader to read in file too large to read into a byte array

const int max = 1024*1024;

public void ReadALargeFile(string file, int start = 0)
{
    FileStream fileStream = new FileStream(file, FileMode.Open,FileAccess.Read);
    using (fileStream)
    {
        byte[] buffer = new byte[max];
        fileStream.Seek(start, SeekOrigin.Begin);
        int bytesRead = fileStream.Read(buffer, start, max);
        while(bytesRead > 0)
        {
            DoSomething(buffer, bytesRead);
            bytesRead = fileStream.Read(buffer, start, max);
        }
    }
}

edited Mar 29, 2018 at 15:35

answered Mar 29, 2018 at 15:28

FortyTwo

2,6593 gold badges25 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Aa Yy Over a year ago

So what exactly is the array that holds my data? is it buffer?

FortyTwo Over a year ago

buffer holds data between start and start + max - 1. Documentation for FileStream.Read

Ron Beyer · Accepted Answer · 2018-03-29 16:00:39Z

2

If you are working with extremely large files, you should use MemoryMappedFile, which maps a physical file to a memory space:

using (var mmf = MemoryMappedFile.CreateFromFile(@"c:\path\to\big.file"))
{
    using (var accessor = mmf.CreateViewAccessor())
    {
        byte myValue = accessor.ReadByte(someOffset);
        accessor.Write((byte)someValue);
    }
}

Comments

FortyTwo · Accepted Answer · 2018-04-02 16:53:35Z

0

This was my solution:

byte[] part1;
byte[] part2;
bool odd = false;
int chunkSize = Convert.ToInt32(length/2);


if (length % 2 == 0)
{
    part1 = new byte[chunkSize];
    part2 = new byte[chunkSize];
}
else
{
    part1 = new byte[chunkSize];
    part2 = new byte[chunkSize + 1];
    odd = true;
}

FileStream fileStream = new FileStream(filepath, FileMode.Open, FileAccess.Read);
using (fileStream)
{
    fileStream.Seek(0, SeekOrigin.Begin);
    int bytesRead = fileStream.Read(part1, 0, chunkSize);
    if (odd)
    {
        bytesRead = fileStream.Read(part2, 0, chunkSize + 1);
    }
    else
    {
        bytesRead = fileStream.Read(part2, 0, chunkSize);
    }
}

edited Apr 2, 2018 at 16:53

FortyTwo

2,6593 gold badges25 silver badges35 bronze badges

answered Apr 2, 2018 at 11:29

Aa Yy

1,7426 gold badges21 silver badges34 bronze badges

1 Comment

Sanjna Malpani Over a year ago

Why is your offset always 0? FortyTwo has a more generic and better solution. Re-usability and abstraction should be kept in mind while programming with an OOP language.

Collectives™ on Stack Overflow

C# Split a file into two byte arrays

3 Answers 3

2 Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related