2

I have some data containing jagged arrays of uints.
The data must be logged fast, so i was thinking of using stream.Write 

using (var stream = File.Open(Folder + @"\"+Filename, FileMode.OpenOrCreate))
 {                 
   // uint[][] RawData = new Uint [9000][]
  foreach(uint[] LogRow in RawData)
   {
     stream.Write(LogRow, 0, 300); 
   }           
  }
}

However Stream.write accepts only bytes[ ]
Might it be possible, to simply get the pointer offset of LogRow
Then since a a uint is 32bits use a count of 300*4
And do a write the uInt[ ] Logrow as a Byte buffer ?.

Maybe width unsafe conversion ?

 int * ByteOffset = &LogRow;
//... ?? its a start 
// but it doesnt give the LogRow as Byte[] 
// not sure the best way to continou  
//

---Update note ---
This code needs to work with video data buffers, so it has to be as fast as possible with no overhead of conversions, or extra threads, or safety checks, boundary checks. In the old days people used to think more about speed as we had less candy in our coding shop. There are a few areas left in today's coding needs, where speed is still "the" major requirement. This is such a question. Don't assume that I get more speed by improving other areas of my code as you've not seen the rest, neither its a constructive discussion to this topic.

Basically what i want is having uint LogRow[] , be available as byte[] without conversions. Think of it as computer memory that i don't want to alter, just take the uint[] as if it was a byte[], should be possible to calculate the offset and length and assign it to a byte[] array,.. something that would go easy in c++ or assembler, the extra difficultly is its jagged array of uint[]. As a reminder a major difference width VB is that C# does have unsafe options, though using them is an Art.

The one liner I wrote below is a cast conversion (The C# compiler will do boundary check on casting), but I write it down just showing that i'm aware of what conversion is, and that i'm not looking for conversion answers, who add extra instructions. 

byte[] objectCast = (byte[])(object)LogRow;

How to do that?

Improve this question
7
  • "data must be logged fast" about which dimension are you talking here? I wonder whether it is necessary to dive into unsafe code. Commented Feb 16, 2018 at 10:49
  • I'm diving into milliseconds ;) (a raw video buffer must be quickly saved before the next buffer wants to be saved) Commented Feb 16, 2018 at 10:57
  • Conversion takes copy time, essentially i just got allready memory space that can be dumped directly. ( c++ wouldnt mind that, but this needs to be done in C# ). As the buffers do arrive fast so therefore this speed requirement. Commented Feb 16, 2018 at 11:06
  • 1
    Writing costs milliseconds, even on a SSD. The conversion takes nanoseconds. You would benefit much more from a good async implementation than trying to eliminate this tiny step. Commented Feb 16, 2018 at 11:43
  • I will suggest to offload buffet writing to disk to another dedicated thread. Put your buffer to let’s say blocking collection and have other thread monitoring it. Commented Feb 16, 2018 at 12:51

3 Answers 3

Reset to default
2

If you want to use the filestream, try using BinaryWriter which will automatically write each type as bytes.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Each primitive type.
0

Well I believe this does the trick without conversions, it essentially uses a struct to tell the compiler by field offset that both variables point to the same offset, and so a Ushort[ ] becomes a standard byte[ ] variable.

Although I think this is the solution to it, i wait to mark it as answer, maybe I post it on code review. As i'm curious of what other people think of it, if there are better faster solutions i'm still all ears. But as I have no dropping of frames, so i think this is it.

using System.Runtime.InteropServices   //must be on top of cs file

[StructLayout(LayoutKind.Explicit)]
 struct Converter
        {
            [FieldOffset(0)]
            public byte[] Bytes;     //warning bytes are 8 bit ! dont use
            [FieldOffset(0)]
            public ushort[] Ushorts;  //16bit
            [FieldOffset(0)]
            public char[] Chars;      //16bit
        }
 static byte[] ArrayUshort2Byte(ushort[] input)
        {

            Converter translate = new Converter();
            translate.Ushorts = input;
            return translate.Bytes;
        }

 static char[] ArrayUshort2Char(ushort[] input)
        {
            //without any buffer copying i convert types 
            //not even convert.ushort overhead no overhead here..

            Converter translate = new Converter();
            translate.Ushorts = input;
            return translate.Chars;
        }

Small note, In the original question i wanted to go from ushort to byte representation, i was thinking in 'raw bytes' as of memory bytes. However a Ushort is 2 bytes and not one, dealing width streamwriter / streamreader, its therefore better to do such a conversion to char (which is also 16bits like ushort)

Its funny though because its easily extendible to convert between ushort /short / char, just add them to the struct width the same offset. And do a similair convert as i did width byte.

Improve this answer

Comments

0

I'm not sure if this is what you want. This will return byte[] aligned the same way as they are with uint[], but here you have access to each individual byte:

public unsafe static void Main()
{
    var data = new uint[1][];
    data[0] = new uint[100000];
    data[0][0] = 1;
    data[0][1] = 2;
    data[0][2] = 3;

    Stopwatch sw = new Stopwatch();
    sw.Start();

    foreach (uint[] LogRow in data)
    {
        fixed (uint* start = &LogRow[0])
        {
            var dataLength = LogRow.Length * sizeof(uint);
            byte[] result = new byte[dataLength];

            Marshal.Copy((IntPtr)start, result, 0, dataLength);
            // This will return 100020003000...00000......
            // Do something with result
        }
    }

    sw.Stop();

    Console.WriteLine(sw.ElapsedTicks);

    Console.ReadKey();
}

It takes about ~700 ticks for 100 000 elements to finish this on my machine.

Update:

Since C# 7.2 you can use Span<T> to avoid unsafe code:

var jaggedData = new uint[1][];
jaggedData[0] = new uint[100000];
for (var index = 0; index < jaggedData[0].Length; index++)
{
    jaggedData[0][index] = (uint)index;
}

foreach (uint[] logRow in jaggedData)
{
    Span<byte> data = Span<uint>.DangerousCreate(logRow, ref logRow[0], logRow.Length).AsBytes();
    // Do stuff...
}

Note that Span<T> is allocated on the stack, so if you have a lot of data, then you might get StackOverflowException.

Improve this answer

2 Comments

well it requires a copy, my answer doesnt copy (see below, i've not marked it as answer yet)
@user3800527 you can compare speed and let me know if there is any difference :) that byte* can be optimized

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.