72

How can I quickly shift all the items in an array one to the left, padding the end with null?

For example, [0,1,2,3,4,5,6] would become [1,2,3,4,5,6,null]

Edit: I said quickly but I guess I meant efficiently. I need to do this without creating a List or some other data structure. This is something I need to do several hundred thousand times in as short amount of time as possible.

11
  • 3
    Your example above is not a "shift" - a shift implies removing the first element in the array, you are wanting to remove from any index. Commented Mar 4, 2010 at 17:20
  • 8
    If you need to do it several hundred thousand times, you might be using the wrong data structure. Commented Mar 4, 2010 at 17:23
  • 11
    A queue would, in fact, be faster than an array, compiler instructions notwithstanding. A queue maintains start and end pointers, so it can remove the first element (or the last element) in O(1) time (vs. O(n) for the array). If values are to be removed from the middle, then a linked-list would be better, but then that changes the complexity of other operations. Commented Mar 4, 2010 at 17:31
  • 2
    Dested: Heh. "What could possibly be faster than an array?!"... Well, a linked list in this case... However, I'm going to have to play devil's advocate here and ask "If there's only 100 items in the array, how fast does it really need to be? Is this your application's bottleneck?". Unless you're making something performance critical, stop caring. And if you are making something performance critical, then let's see some profiler numbers please? Commented Mar 4, 2010 at 17:34
  • 3
    On your example you ask to shift the elements of an array. The chosen answer, however, treats your array as immutable and creates a new one... which is slower because of the necessary call to the allocator. Commented Mar 4, 2010 at 17:40

22 Answers 22

123

Here's my test harness...

var source = Enumerable.Range(1, 100).Cast<int?>().ToArray();
var destination = new int?[source.Length];

var s = new Stopwatch();
s.Start();
for (int i = 0; i < 1000000;i++)
{
    Array.Copy(source, 1, destination, 0, source.Length - 1);
}
s.Stop();
Console.WriteLine(s.Elapsed);

Here are the performance results for 1 million iterations of each solution (8 Core Intel Xeon E5450 @ 3.00GHz)

                       100 elements  10000 elements
For Loop                     0.390s         31.839s 
Array.Copy()                 0.177s         12.496s
Aaron 1                      3.789s         84.082s
Array.ConstrainedCopy()      0.197s         17.658s

Make the choice for yourself :)

Sign up to request clarification or add additional context in comments.

11 Comments

Nope. I just tried it myself, and got the same results. I guess Array.Copy wins. Dang, that's one fast method.
Array.Copy is probably faster because it can do things you can't do in C#. It could very well be just playing with the page table and setting the flags to force copy on write.
I'd give you +100 if I could. There are only two ways to approach a performance "problem" like this one. A) Profile and pick the best, or B) Decide that you don't really care about a difference of a few hundred ms under realistic conditions. Write your code for readability first, then profile if and only if you need to!
I suggest also testing Buffer.BlockCopy. And @RobertDavis: page table manipulation would provide only a fraction of the capability that Array.Copy exposes. For example, it requires alignment, having nothing else in the same page, etc. No, Array.Copy is just a copy in native code with bounds-checking hoisted outside the loop. Buffer.BlockCopy is supposed to be a more optimized transfer, that may use tricks like SSE loads and stores to move multiple elements at once.
Actually according to the code, the system runs the C function memmove to do most of the heavy lifting. For all those wanting to know more, look in clr/src/vm/comsystem.cpp, and look for SystemNative::ArrayCopy. Which is called from referencesource.microsoft.com/#mscorlib/system/…
|
72

The quickest way to do this is to use Array.Copy, which in the final implementation uses a bulk memory transfer operation (similar to memcpy):

var oldArray = new int?[] { 1, 2, 3, 4, 5, 6 };
var newArray = new int?[oldArray.Length];
Array.Copy(oldArray, 1, newArray, 0, oldArray.Length - 1);
// newArray is now { 2, 3, 4, 5, 6, null }

Edited: according to the documentation:

If sourceArray and destinationArray overlap, this method behaves as if the original values of sourceArray were preserved in a temporary location before destinationArray is overwritten.

So if you don't want to allocate a new array, you can pass in the original array for both source and destination--although I imagine the tradeoff will be a somewhat slower performance since the values go through a temporary holding position.

I suppose, as in any investigation of this kind, you should do some quick benchmarking.

8 Comments

the two arrays in the Copy method may overlap, so you don't need to create another array. Array.Copy(oldArray, 1, oldArray, 0, oldArray.Length - 1) together with setting the last element to null will also work.
I havent ran the tests yet but I think the for loop approach will end up being faster int eh log run. Thank you for the response though.
If you haven't "ran the tests", obviously you don't care that much about performance! You might as well use ToList, Remove, ToArray.
@Dested: unless the for loop compiles down into the same lean loop that a traditional memcpy uses, that won't be the case. See, for example, waldev.blogspot.com/2008/05/…
@Ben M - optimizers can be pretty amazing sometimes. The only way to know for sure is to benchmark it, but the for loop stands a good chance of compiling to the same lean loop, and it can avoid all the checks and planning that the Array.Copy method must do on each invocation. Copy is, after all, an extremely flexible method.
|
14

Here is my solution, similar to Task's in that it is a simple Array wrapper and that it takes O(1) time to shift the array to the left.

public class ShiftyArray<T>
{
    private readonly T[] array;
    private int front;

    public ShiftyArray(T[] array)
    {
        this.array = array;
        front = 0;
    }

    public void ShiftLeft()
    {
        array[front++] = default(T);
        if(front > array.Length - 1)
        {
            front = 0;
        }
    }

    public void ShiftLeft(int count)
    {
        for(int i = 0; i < count; i++)
        {
            ShiftLeft();
        }
    }

    public T this[int index]
    {
        get
        {
            if(index > array.Length - 1)
            {
                throw new IndexOutOfRangeException();
            }

            return array[(front + index) % array.Length];
        }
    }

    public int Length { get { return array.Length; } }
}

Running it through Jason Punyon's test code...

int?[] intData = Enumerable.Range(1, 100).Cast<int?>().ToArray();
ShiftyArray<int?> array = new ShiftyArray<int?>(intData);

Stopwatch watch = new Stopwatch();
watch.Start();

for(int i = 0; i < 1000000; i++)
{
    array.ShiftLeft();
}

watch.Stop();

Console.WriteLine(watch.ElapsedMilliseconds);

Takes ~29ms, regardless of the array size.

2 Comments

"~29ms" is not a usable number in isolation - you need at least one other implementation's timing on the same system and data to compare it to.
shifting is fastest possible yes, but iterating the array is slower
10

Couldn't you use a System.Collections.Generic.Queue instead of an array ?

I feel like you need to perform actions on your value then discard it, thus using a queue seems to be more appropriate :

// dummy initialization
        System.Collections.Generic.Queue<int> queue = new Queue<int>();
        for (int i = 0; i < 7; ++i ) { queue.Enqueue(i); }// add each element at the end of the container

        // working thread
        if (queue.Count > 0)
            doSomething(queue.Dequeue());// removes the last element of the container and calls doSomething on it

Comments

9

Use the Array.Copy() method as in

int?[] myArray = new int?[]{0,1,2,3,4};
Array.Copy(myArray, 1, myArray, 0, myArray.Length - 1);
myArray[myArray.Length - 1] = null

The Array.Copy is probably the way, Microsoft wanted us to copy array elements...

1 Comment

The Array.Copy is probably the way, Microsoft wanted us to copy array elements - File this answer under "Falling into the Pit of Success." If it's obvious and it's C#, well you know the rest.
9

For any pour soul finding this thread and about to implement one of the highly rated answers. All of them are trash, I'm not sure why that is. Maybe Dested asked for a new array implementation at first or something that has now been removed from the question. Well if you simply want to shift the array and don't need a new one, see an answer like tdaines's answer. And read up on things like the Circular Buffer / Ring Buffer : http://en.wikipedia.org/wiki/Circular_buffer. No moving of the actual data is necessary. The performance of shifting an array should not be tied to the size of the array.

1 Comment

Ring buffer is faster because it avoids copies, but it doesn't provide the same API. You can't pass ring buffers to functions that require contiguous storage, for example, because the data is stored in two chunks.
5

If it absolutely has to be in an array, then I would recommend the most obvious code possible.

for (int index = startIndex; index + 1 < values.Length; index++)
     values[index] = values[index + 1];
values[values.Length - 1] = null;

This gives the optimizer the most opportunities to find the best way on whatever target platform the program is installed on.

EDIT:

I just borrowed Jason Punyon's test code, and I'm afraid he's right. Array.Copy wins!

    var source = Enumerable.Range(1, 100).Cast<int?>().ToArray();
    int indexToRemove = 4;

    var s = new Stopwatch();
    s.Start();
    for (int i = 0; i < 1000000; i++)
    {
        Array.Copy(source, indexToRemove + 1, source, indexToRemove, source.Length - indexToRemove - 1);
        //for (int index = indexToRemove; index + 1 < source.Length; index++)
        //    source[index] = source[index + 1]; 
    }
    s.Stop();
    Console.WriteLine(s.Elapsed);

Array.Copy takes between 103 and 150 ms on my machine.

for loop takes between 269 and 338 ms on my machine.

1 Comment

it does depend on the size of the array, on my machine: if its under 54, then the for loop is actually faster
5

Can't you

  • allocate the array with an extra 1000 elements

  • have an integer variable int base = 0

  • instead of accessing a[i] access a[base+i]

  • to do your shift, just say base++

Then after you've done this 1000 times, copy it down and start over.
That way, you only do the copy once per 1000 shifts.


Old joke:
Q: How many IBM 360s does it take to shift a register by 1 bit?
A: 33. 32 to hold the bits in place, and 1 to move the register. (or some such...)

2 Comments

I actually ended up doing something like this.
That's basically the "use a queue with start and end pointers" suggested by several others. But good clear explanation.
3

You can use the same array as source and destination for fast in-place copy:

static void Main(string[] args)
        {
            int[] array = {0, 1, 2, 3, 4, 5, 6, 7};
            Array.ConstrainedCopy(array, 1, array, 0, array.Length - 1);
            array[array.Length - 1] = 0;
        }

Comments

3

The best and most efficient method I believe is using Buffer.BlockCopy function. You will set both source and destination to your array, the offset of the source is 1. Depending on your array type (I assume it is int), 1 int = 4 bytes, so you must pass in 4 as the second parameter of this function. Note that the offset is byte offset.

So it looks like this:

int bytes2copy = yourArray.length - 4;
Buffer.BlockCopy(yourArray, 4, yourArray, 0, bytes2copy);
yourArray[yourArray.length-1] = null;

Comments

2

You might do it like this:

var items = new int?[] { 0, 1, 2, 3, 4, 5, 6 };  // Your array
var itemList = new List<int?>(items);  // Put the items in a List<>
itemList.RemoveAt(1); // Remove the item at index 1
itemList.Add(null); // Add a null to the end of the list
items = itemList.ToArray(); // Turn the list back into an array

Of course, it would be more efficient to get rid of the array entirely and just use a List<>. You could then forget the first line and last line and do it like this:

var itemList = new List<int?> { 0, 1, 2, 3, 4, 5, 6 };
itemList.RemoveAt(1); // Remove the item at index 1
itemList.Add(null); // Add a null to the end of the list

2 Comments

hey! did managed programming completely stole ability to think about performance?
You're absolutely right about this not being the most efficient way. If this operation is going to happen a lot, then this isn't the right answer. And like I said, going from array to List and back to array isn't efficient, so I would skip the array and just use a List to begin with. If this is an operation that only happens on occasion, then this will work just fine and it will be easy to understand. Plus this way can handle changing List lengths.
1

Try this! using Linq. No need of second Array.

        var i_array = new int?[] {0, 1, 2, 3, 4, 5, 6 };

        i_array = i_array.Select((v, k) => new { v = v, k = k }).
            Where(i => i.k > 0).Select(i => i.v).ToArray();

        Array.Resize(ref i_array, i_array.Length + 1);

Output: [0,1,2,3,4,5,6] would become [1,2,3,4,5,6,null]

Comments

1

If you own the memory you could consider using Unsafe Code and good old fashioned pointers.

Make yourself a memory stream and lock it down or use Marshal.AllocHGlobal Construct all your arrays in it with a little bit of padding at the beginning and end. increment or decrement all of the array pointers at once. You'll still need to loop back and set your nulls.

If you need to selectively increment or decrement the arrays you would have to add padding between them.

Arrays are incredibly low level data structures, if you treat them in a low level way you can get huge performance out of them.

A baytrail doing this could outperform Jason's with all its copying 8 Core Intel Xeon E5450 @ 3.00GHz

1 Comment

Even with unsafe code, C# doesn't give you access to the SSE instructions that are needed to maximize memory throughput.
1

Not tested this code, but it should shifts all the values to right by one. Note that the last three lines of code is all you require to efficiently shift the array.

public class Shift : MonoBehaviour {
     //Initialize Array
     public int[] queue;

     void Start () {
          //Create Array Rows
          queue = new int[5];

          //Set Values to 1,2,3,4,5
          for (int i=0; i<5;i++)
          {
               queue[i] = i + 1;
          }

          //Get the integer at the first index
          int prev = queue[0];

          //Copy the array to the new array.
          System.Array.Copy(queue, 1, queue, 0, queue.Length - 1);

          //Set the last shifted value to the previously first value.
          queue[queue.Length - 1] = prev;

1 Comment

Interesting solution!
1

Implementation with Extension methods passing shifting direction as Enum.

"for" statements and indexers only (don't use Array.Copy method).

using System;

namespace ShiftArrayElements
{
    public static class EnumShifter
    {
        public static int[] Shift(int[] source, Direction[] directions)
        {
            for (var i = 0; i < directions.Length; i++)
            {
                var direction = directions[i];
                if (direction == Direction.Left)
                {
                    source.LeftShift();
                }
                else if (direction == Direction.Right)
                {
                    source.RightShift();
                }
                else
                {
                    throw new InvalidOperationException("Direction is invalid");
                }
            }

            return source;
        }

        public static void LeftShift(this int[] source)
        {
            var lastIndex = source?.Length - 1 ?? 0;
            var temp = source[0];

            for (int j = 0; j + 1 < source.Length; j++)
            {
                source[j] = source[j + 1];
            }

            source[lastIndex] = temp;
        }        
        
        public static void RightShift(this int[] source)
        {
            var lastIndex = source?.Length - 1 ?? 0;
            var temp = source[lastIndex];

            for (int j = lastIndex; j > 0; j--)
            {
                source[j] = source[j - 1];
            }

            source[0] = temp;
        }
    }
}

Comments

0

Array copying is an O(n) operation and creates a new array. While array copying can certainly be done quickly and efficiently, the problem you've stated can actually be solved in an entirely different way without (as you've requested) creating a new array/data structure and only creating one small wrapping object instance per array:

using System;
using System.Text;

public class ArrayReindexer
{
    private Array reindexed;
    private int location, offset;

    public ArrayReindexer( Array source )
    {
        reindexed = source;
    }

    public object this[int index]
    {
        get
        {
            if (offset > 0 && index >= location)
            {
                int adjustedIndex = index + offset;
                return adjustedIndex >= reindexed.Length ? "null" : reindexed.GetValue( adjustedIndex );
            }

            return reindexed.GetValue( index );
        }
    }

    public void Reindex( int position, int shiftAmount )
    {
        location = position;
        offset = shiftAmount;
    }

    public override string ToString()
    {
        StringBuilder output = new StringBuilder( "[ " );
        for (int i = 0; i < reindexed.Length; ++i)
        {
            output.Append( this[i] );
            if (i == reindexed.Length - 1)
            {
                output.Append( " ]" );
            }
            else
            {
                output.Append( ", " );
            }
        }

        return output.ToString();
    }
}

By wrapping and controlling access to the array in this manner, we can now demonstrate how the problem was solved with an O(1) method call...

ArrayReindexer original = new ArrayReindexer( SourceArray );
Console.WriteLine( "   Base array: {0}", original.ToString() );
ArrayReindexer reindexed = new ArrayReindexer( SourceArray );
reindexed.Reindex( 1, 1 );
Console.WriteLine( "Shifted array: {0}", reindexed.ToString() );

Will produce the output:

Base array: [ 0, 1, 2, 3, 4, 5, 6 ]
Shifted array: [ 0, 2, 3, 4, 5, 6, null ]

I'm willing to bet that there will be a reason that such a solution won't work for you, but I believe this does match your initial stated requirements. 8 )

It's often helpful to think about all the different kinds of solutions to a problem before implementing a specific one, and perhaps that might be the most important thing that this example can demonstrate.

Hope this helps!

Comments

0

Incorrect and slightly amusing answer (thanks, i'll be here all night !)

int?[] test = new int?[] {0,1,2,3,4,5,6 };

        int?[] t = new int?[test.Length];
        t = test.Skip(1).ToArray();
        t[t.Length - 1] = null; 

In the spirit of still using Skip (dont ask me, i know worst usage of LINQ extension methods ever), the only way I thought of rewriting it would be

        int?[] test = new int?[] { 0, 1, 2, 3, 4, 5, 6 };

        int?[] t = new int?[test.Length];
        Array.Copy(test.Skip(1).ToArray(), t, t.Length - 1);

But it's in NO WAY faster than the other options.

9 Comments

This actually turns the last element you want to keep into a null...so I'm not profiling it in my answer.
@Jason, what exactly are you talking about? The result i get is [1,2,3,4,5,6,null]
@Stan R.: The code posted results in [1,2,3,4,5,null]. You can immediately see that length isn't preserved. test.Skip(1).ToArray() will have a length less than test.
@Jason, ahhh i see..ToArray actually creates a whole new array. Wow, def need to be more careful with this kind of stuff. Thanks for pointing that out. I will rewrite my answer, then you can profile it :)
@Jason, yes, i know..my mistake.
|
0

I know this is an old question but coming from Google there was no simple example so thanks to this is the easiest way to reorder a list, and you don't have to supply the type it will work it out at runtime,

   private static List<T> reorderList<T>(List<T> list){
       List<T> newList = new List<T>();

       list.ForEach(delegate(T item)
       {
           newList.Add(item);
       });

       return newList;
   }

Comments

0
using System;
using System.Threading;

namespace ShiftMatrix
{
    class Program
    {
        static void Main(string[] args)
        {

            MatrixOperation objMatrixOperation = new MatrixOperation();

            //Create a matrix
            int[,] mat = new int[,]
        {
        {1, 2},
        {3,4 },
        {5, 6},
        {7,8},
        {8,9},
        };

            int type = 2;
            int counter = 0;
            if (type == 1)
            {
                counter = mat.GetLength(0);
            }
            else
            {
                counter = mat.GetLength(1);
            }
            while (true)
            {
                for (int i = 0; i < counter; i++)
                {
                    ShowMatrix(objMatrixOperation.ShiftMatrix(mat, i, type));
                    Thread.Sleep(TimeSpan.FromSeconds(2));
                }
            }
        }
        public static void ShowMatrix(int[,] matrix)
        {
            int rows = matrix.GetLength(0);
            int columns = matrix.GetLength(1);
            for (int k = 0; k < rows; k++)
            {
                for (int l = 0; l < columns; l++)
                {
                    Console.Write(matrix[k, l] + " ");
                }
                Console.WriteLine();
            }
        }
    }
    class MatrixOperation
    {
        public int[,] ShiftMatrix(int[,] origanalMatrix, int shift, int type)
        {
            int rows = origanalMatrix.GetLength(0);
            int cols = origanalMatrix.GetLength(1);

            int[,] _tmpMatrix = new int[rows, cols];
            if (type == 2)
            {
                for (int x1 = 0; x1 < rows; x1++)
                {
                    int y2 = 0;
                    for (int y1 = shift; y2 < cols - shift; y1++, y2++)
                    {
                        _tmpMatrix[x1, y2] = origanalMatrix[x1, y1];
                    }
                    y2--;
                    for (int y1 = 0; y1 < shift; y1++, y2++)
                    {
                        _tmpMatrix[x1, y2] = origanalMatrix[x1, y1];
                    }
                }
            }
            else
            {
                int x2 = 0;
                for (int x1 = shift; x2 < rows - shift; x1++, x2++)
                {
                    for (int y1 = 0; y1 < cols; y1++)
                    {
                        _tmpMatrix[x2, y1] = origanalMatrix[x1, y1];
                    }
                }
                x2--;
                for (int x1 = 0; x1 < shift; x1++, x2++)
                {
                    for (int y1 = 0; y1 < cols; y1++)
                    {
                        _tmpMatrix[x2, y1] = origanalMatrix[x1, y1];
                    }
                }

            }
            return _tmpMatrix;
        }
    }

}

Comments

0

See C# code below to remove space from string. That shift character in array. Performance is O(n). No other array is used. So no extra memory either.

    static void Main(string[] args)
    {
        string strIn = System.Console.ReadLine();

        char[] chraryIn = strIn.ToCharArray();

        int iShift = 0;
        char chrTemp;
        for (int i = 0; i < chraryIn.Length; ++i)
        {
            if (i > 0)
            {
                chrTemp = chraryIn[i];
                chraryIn[i - iShift] = chrTemp;
                chraryIn[i] = chraryIn[i - iShift];
            }
            if (chraryIn[i] == ' ') iShift++;
            if (i >= chraryIn.Length - 1 - iShift) chraryIn[i] = ' ';
        }
       System.Console.WriteLine(new string(chraryIn));
       System.Console.Read();
    }

3 Comments

Why -1 ? Can some one please explain ?
because you are creating an extra char Array and string in the process … and the question was about efficient shifting of elements in array so it was off topic
I appreciate you, Gerard. Yes, I am converting the string to a char array. I should have been more clear.
0

a is array of ints & d is number of times array has to shift left.

static int[] rotLeft(int[] a, int d) 
{
     var innerLoop = a.Length - 1;
     for(var loop=0; loop < d; loop++)
     {
         var res = a[innerLoop];
         for (var i= innerLoop; i>=0; i--)
         {
            var tempI = i-1;
            if (tempI < 0)
            {
                tempI = innerLoop;
            }        
            var yolo = a[tempI];
            a[tempI] = res;
            res = yolo;
         }
     }
     return a;
}

Comments

0

Simple way to do it when you need to resize the same array.

            var nLength = args.Length - 1;
            Array.Copy(args, 1, args, 0, nLength);
            Array.Resize(ref args, nLength);

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.