Conversion from VB6 string data to .NET byte array

One possible way I see is to:

1. Read the data back as a System.Char[], which is Unicode just like VB BSTRs.
2. Convert it to an ASCII byte array via Encoding.ASCII.GetBytes(). Effectively this removes all the interleaved 0s.
3. Copy this ASCII byte array to your final float array.

Something like this:

char[] destinationAsChars = new char[BitConverter.ToInt32(source, 0)* BitConverter.ToInt32(source, 4)];
byte[] asciiBytes = Encoding.ASCII.GetBytes(destinationAsChars);
float[] destination = new float[notSureHowLarge];
Buffer.BlockCopy(asciiBytes, 0, destination, 0, asciiBytes.Length);

Now destination should contain the original floats. CAVEAT: am not sure if the internal format of VB6 Singles is binary-compatible with the internal format of System.Float. If not, all bets are off.

This is the solution I derived from the answer above.

Reading the file in as a unicode char[], and then re-encoding to my default system encoding produced readable files.

internal void FixBytes()
{
    //Convert the bytes from VB6 style BSTR to standard byte[].

    char[] destinationAsChars = 
    System.Text.Encoding.Unicode.GetString(File).ToCharArray();

    byte[] asciiBytes =  Encoding.Default.GetBytes(destinationAsChars);
    byte[] newFile = new byte[asciiBytes.Length];
    Buffer.BlockCopy(asciiBytes,0, newFile, 0, asciiBytes.Length);
    File = newFile;
}

As you probably know, that's very bad coding on the VB6 end. What it's trying to do is to cast the Single data -- which is the same as float in C# -- as a String. But while there are better ways to do that, it's a really bad idea to begin with.

The main reason is that reading the binary data into a VB6 BSTR will convert the data from 8-bit bytes to 16-bit characters, using on the current code page. So this can produce different results in the DB depending on what locale it's running in. (!)

So when you read it back from the DB, unless you specify the same code page used when writing, you'll get different floats, possibly even invalid ones.

It would help to see examples of data both in binary (single) and DB (string) form, in hex, to verify that this is what's happening.

From a later post:

Actually that is not "bad" VB6 code.

It is, because it takes binary data into the string domain, which violates a prime rule of modern VB coding. It's why the Byte data type exists. If you ignore this, you may well wind up with undecipherable data when a DB you create crosses locale boundaries.

What he is doing is storing the array in a compact binary format and saving it as a "chunk" into the database. There are lots of valid reasons to do this.

Of course he has a valid reason for wanting this (although your definition of 'compact' is different from the conventional one). The ends are fine: the means chosen are not.

To the OP:

You probably can't change what you're given as input data, so the above is mostly academic. If there's still time to change the method used to create the blobs, let us suggest methods that don't involve strings.

In applying any provided solution, do your best to avoid strings, and if you can't, decode them using the specific code page that matches the one that created them.

Can you clarify what the contents of the file are (i.e. an example)? Either as binary (perhaps hex) or characters? If the data is a VB6 string, then you'll have to use float.Parse() to read it. .NET strings are also 2-bytes per character, but when loading from a file you can control this using the Encoding.

Actually that is not "bad" VB6 code. What he is doing is storing the array in a compact binary format and saving it as a "chunk" into the database. There are lots of valid reasons to do this.

The reason for the VB6 code saving it to disk and reading it back is because VB6 doesn't have native support for reading and writing files in memory only. This is the common algorithm if you want to create a chunk of binary data and stuff it somewhere else like a database field.

It is not an issues dealing with this in .NET. The code I have is in VB.NET so you will have to convert it to C#.

Modified to handle bytes and the unicode problem.

Public Function DataArrayFromDatabase(ByVal dbData As byte()) As Single(,)
    Dim bData(Ubound(dbData)/2) As Byte
    Dim I As Long
    Dim J As Long

    J=0
    For I = 1 To Ubound(dbData) step 2
        bData(J) = dbData(I)
        J=1
    Next I

    Dim sM As New IO.MemoryStream(bData)
    Dim bR As IO.BinaryReader = New IO.BinaryReader(sM)
    Dim Dim1 As Integer = bR.ReadInt32
    Dim Dim2 As Integer = bR.ReadInt32
    Dim newData(Dim1, Dim2) As Single

    For I = 0 To Dim2
        For J = 0 To Dim1
            newData(J, I) = bR.ReadSingle
        Next
    Next

    bR.Close()
    sM.Close()
    Return newData
End Function

The key trick is to read in the data just like if you were in VB6. We have the ability to use MemoryStreams in .NET so this is fairly easy.

First we skip every other byte to eliminate the Unicode padding.

Then we create a memorystream from the array of bytes. Then a BinaryReader initialized with the MemoryStream.

We read in the first dimension of the array a VB6 Long or .NET Int32 We read in the second dimension of the array a VB6 Long or .NET Int32

The read loops are constructed in reverse order of the array's dimension. Dim2 is the outer loop and Dim1 is the inner. The reason for this is that this is how VB6 store arrays in binary format.

Return newData and you have successfully restored the original array that was created in VB6!

Now you could try to use some math trick. The two dimension are 4 bytes/characters and each array element is 4 bytes/characters. But for long term maintainability I find using byte manipulation with memorystreams a lot more explicit. It take a little more code but a lot more clear when you revisit it 5 years from now.

First we skip every other byte to eliminate the Unicode padding.

Hmmm... if that were a valid strategy, then every other column in the DB string dump would consist of nothing but zeros. But a quick scan down the first one shows that this isn't the case. In fact there are a lot of non-zero bytes in those columns. Can we afford to just discard them?

What this shows is that the conversion to Unicode caused by the use of Strings does not simply add 'padding', but changes the character of the data. What you call padding is a coincidence of the fact that the ASCII range (00-7F binary) is mapped onto the same Unicode range. But this is not true of binary 80-FF.

Take a look at the first stored value, which has an original byte value of 94 9A 27 3A. When converted to Unicode, these DO NOT become 94 00 97 00 27 00 3A 00. They become 1D 20 61 01 27 00 3A 00.

Discarding every other byte gives you 1D 61 27 3A -- not the original 94 9A 27 3A.