12

I currently want a data structure that acts like a Deque with indexing. So, it should have O(1) addition and removal of elements at the front and back, as well as O(1) access of elements based on the index. It is not that hard to imagine a setup that would work for this.

It seems like ArrayDeque would be a natural choice for this. However, ArrayDeque does not implement List. Since the underlying data structure is an array, is there a good reason that it does not allow indexing?

Also, on a more practical note, does anyone know of any library doing the thing I want. Apache Commons does not have one as far as I can see.

6
  • 1
    I don't know the answer but arrays certainly don't have O(1) addition properties in both front and back due to their fixed size. Linked lists do, but they don't have O(1) access by index. Commented May 18, 2017 at 22:56
  • thecodingforums.com/threads/… Commented May 18, 2017 at 22:58
  • When you fake it in an array you add some gapping there. So you have an array that is 100 items in memory but is stored in the array from positions 45-57 or so. Then you can add items on the front and back in order 1 time. Until you run out of space. Then you ditch that and make a new array. It sounds a bit odd but memory space is so efficient with arrays that it'll basically always outperform a linked-list. In fact, most things will out perform a linked list even things linked lists would be perfect for, simply because memory loves arrays (caching, extra loading of data etc). Commented May 18, 2017 at 23:26
  • In fact, you can implement queues and stacks in that way, and just loop back to zero or .length when needed, and then you only rebuild the queue when the size exceeds the size of the array you are using. Since memory loves arrays, you get much better performance out of this than you would linked list which would constantly miss cache lookups. Commented May 18, 2017 at 23:29
  • 4
    There is an OpenJDK Jira ticket for this, as well: (coll) retrofit ArrayDeque to implement List Commented May 18, 2017 at 23:36

1 Answer 1

8

(edit, original answer was mostly wrong)

There's no good reason why this class doesn't have an indexing. I checked the source code. It runs exactly like I suggested above. Some of the other items might be harder to add within the List interface. But, simple get wouldn't be one of them.

You take an array of a given size depending on your usage. Then set the start position halfway into it. Then you track a variable for the head and the tail. As you iterate items you move the head back and tail forward expanding outward. If you reach need a value outside the range you modulo that value so .length is equal to 0, and -1 is equal to (.length -1) and you keep adding values until ((tail - head) > .length) at which point you build another array copying all the pieces to be coherent usually about twice the size. Then to index it, you take head +index to get the real index.

http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/ArrayDeque.java

Pretty sure the get is trivially akin to:

    @Override
    public E get(int index) {
        int i = (head + index) & (elements.length - 1);
        return elements[i];
    }

Which seems like they should almost certainly have that. (See source code link for licensing info for the following).

import java.io.*;
import java.util.AbstractCollection;
import java.util.Arrays;
import java.util.Collection;
import java.util.ConcurrentModificationException;
import java.util.Deque;
import java.util.Iterator;
import java.util.List;
import java.util.ListIterator;
import java.util.NoSuchElementException;

public class ArrayDequeList<E>  extends AbstractCollection<E>  implements Deque<E>, Cloneable, Serializable, List<E>
{
    private transient E[] elements;
    private transient int head;
    private transient int tail;
    private static final int MIN_INITIAL_CAPACITY = 8;

    private void allocateElements(int numElements) {
        int initialCapacity = MIN_INITIAL_CAPACITY;
        // Find the best power of two to hold elements.
        // Tests "<=" because arrays aren't kept full.
        if (numElements >= initialCapacity) {
            initialCapacity = numElements;
            initialCapacity |= (initialCapacity >>>  1);
            initialCapacity |= (initialCapacity >>>  2);
            initialCapacity |= (initialCapacity >>>  4);
            initialCapacity |= (initialCapacity >>>  8);
            initialCapacity |= (initialCapacity >>> 16);
            initialCapacity++;

            if (initialCapacity < 0)   // Too many elements, must back off
                initialCapacity >>>= 1;// Good luck allocating 2 ^ 30 elements
        }
        elements = (E[]) new Object[initialCapacity];
    }

    @Override
    public boolean addAll(int index, @NonNull Collection<? extends E> c) {
        return false;
    }

    @Override
    public E get(int index) {
        int i = (head + index) & (elements.length - 1);
        return elements[i];
    }

    @Override
    public E set(int index, E element) {
        int i = (head + index) & (elements.length - 1);
        E old = elements[i];
        elements[i] = element;
        return old;
    }

    @Override
    public void add(int index, E element) {
        throw new IllegalStateException("This one is hard to do.");
    }

    @Override
    public E remove(int index) {
        throw new IllegalStateException("This one is hard to do.");
    }

    @Override
    public int indexOf(Object o) {
        throw new IllegalStateException("This one's not that hard but pass..");
    }

    @Override
    public int lastIndexOf(Object o) {
        throw new IllegalStateException("This one's not that hard but pass..");
    }

    @Override
    public ListIterator<E> listIterator() {
        throw new IllegalStateException("Needs to write a new iterator..");
    }

    @NonNull
    @Override
    public ListIterator<E> listIterator(int index) {
        throw new IllegalStateException("Needs to write a new iterator..");
    }

    @NonNull
    @Override
    public List<E> subList(int fromIndex, int toIndex) {
        throw new IllegalStateException("Hm, not sure how this would work.");
    }

    private void doubleCapacity() {
        assert head == tail;
        int p = head;
        int n = elements.length;
        int r = n - p; // number of elements to the right of p
        int newCapacity = n << 1;
        if (newCapacity < 0)
            throw new IllegalStateException("Sorry, deque too big");
        Object[] a = new Object[newCapacity];
        System.arraycopy(elements, p, a, 0, r);
        System.arraycopy(elements, 0, a, r, p);
        elements = (E[])a;
        head = 0;
        tail = n;
    }

    private <T> T[] copyElements(T[] a) {
        if (head < tail) {
            System.arraycopy(elements, head, a, 0, size());
        } else if (head > tail) {
            int headPortionLen = elements.length - head;
            System.arraycopy(elements, head, a, 0, headPortionLen);
            System.arraycopy(elements, 0, a, headPortionLen, tail);
        }
        return a;
    }

    public ArrayDequeList() {
        elements = (E[]) new Object[16];
    }

    public ArrayDequeList(int numElements) {
        allocateElements(numElements);
    }

    public ArrayDequeList(Collection<? extends E> c) {
        allocateElements(c.size());
        addAll(c);
    }


    public void addFirst(E e) {
        if (e == null)
            throw new NullPointerException();
        elements[head = (head - 1) & (elements.length - 1)] = e;
        if (head == tail)
            doubleCapacity();
    }

    public void addLast(E e) {
        if (e == null)
            throw new NullPointerException();
        elements[tail] = e;
        if ( (tail = (tail + 1) & (elements.length - 1)) == head)
            doubleCapacity();
    }

    public boolean offerFirst(E e) {
        addFirst(e);
        return true;
    }

    public boolean offerLast(E e) {
        addLast(e);
        return true;
    }

    public E removeFirst() {
        E x = pollFirst();
        if (x == null)
            throw new NoSuchElementException();
        return x;
    }

    public E removeLast() {
        E x = pollLast();
        if (x == null)
            throw new NoSuchElementException();
        return x;
    }

    public E pollFirst() {
        int h = head;
        E result = elements[h]; // Element is null if deque empty
        if (result == null)
            return null;
        elements[h] = null;     // Must null out slot
        head = (h + 1) & (elements.length - 1);
        return result;
    }

    public E pollLast() {
        int t = (tail - 1) & (elements.length - 1);
        E result = elements[t];
        if (result == null)
            return null;
        elements[t] = null;
        tail = t;
        return result;
    }

    public E getFirst() {
        E x = elements[head];
        if (x == null)
            throw new NoSuchElementException();
        return x;
    }

    public E getLast() {
        E x = elements[(tail - 1) & (elements.length - 1)];
        if (x == null)
            throw new NoSuchElementException();
        return x;
    }

    public E peekFirst() {
        return elements[head]; // elements[head] is null if deque empty
    }

    public E peekLast() {
        return elements[(tail - 1) & (elements.length - 1)];
    }

    public boolean removeFirstOccurrence(Object o) {
        if (o == null)
            return false;
        int mask = elements.length - 1;
        int i = head;
        E x;
        while ( (x = elements[i]) != null) {
            if (o.equals(x)) {
                delete(i);
                return true;
            }
            i = (i + 1) & mask;
        }
        return false;
    }

    public boolean removeLastOccurrence(Object o) {
        if (o == null)
            return false;
        int mask = elements.length - 1;
        int i = (tail - 1) & mask;
        E x;
        while ( (x = elements[i]) != null) {
            if (o.equals(x)) {
                delete(i);
                return true;
            }
            i = (i - 1) & mask;
        }
        return false;
    }


    public boolean add(E e) {
        addLast(e);
        return true;
    }


    public boolean offer(E e) {
        return offerLast(e);
    }

    public E remove() {
        return removeFirst();
    }


    public E poll() {
        return pollFirst();
    }

    public E element() {
        return getFirst();
    }


    public E peek() {
        return peekFirst();
    }

    public void push(E e) {
        addFirst(e);
    }

    public E pop() {
        return removeFirst();
    }

    private void checkInvariants() {
        assert elements[tail] == null;
        assert head == tail ? elements[head] == null :
                (elements[head] != null &&
                        elements[(tail - 1) & (elements.length - 1)] != null);
        assert elements[(head - 1) & (elements.length - 1)] == null;
    }

    private boolean delete(int i) {
        checkInvariants();
        final E[] elements = this.elements;
        final int mask = elements.length - 1;
        final int h = head;
        final int t = tail;
        final int front = (i - h) & mask;
        final int back  = (t - i) & mask;

        // Invariant: head <= i < tail mod circularity
        if (front >= ((t - h) & mask))
            throw new ConcurrentModificationException();

        // Optimize for least element motion
        if (front < back) {
            if (h <= i) {
                System.arraycopy(elements, h, elements, h + 1, front);
            } else { // Wrap around
                System.arraycopy(elements, 0, elements, 1, i);
                elements[0] = elements[mask];
                System.arraycopy(elements, h, elements, h + 1, mask - h);
            }
            elements[h] = null;
            head = (h + 1) & mask;
            return false;
        } else {
            if (i < t) { // Copy the null tail as well
                System.arraycopy(elements, i + 1, elements, i, back);
                tail = t - 1;
            } else { // Wrap around
                System.arraycopy(elements, i + 1, elements, i, mask - i);
                elements[mask] = elements[0];
                System.arraycopy(elements, 1, elements, 0, t);
                tail = (t - 1) & mask;
            }
            return true;
        }
    }

    public int size() {
        return (tail - head) & (elements.length - 1);
    }

    public boolean isEmpty() {
        return head == tail;
    }

    public Iterator<E> iterator() {
        return new DeqIterator();
    }

    public Iterator<E> descendingIterator() {
        return new DescendingIterator();
    }

    private class DeqIterator implements Iterator<E> {

        private int cursor = head;

        private int fence = tail;

        private int lastRet = -1;

        public boolean hasNext() {
            return cursor != fence;
        }

        public E next() {
            if (cursor == fence)
                throw new NoSuchElementException();
            E result = elements[cursor];
            // This check doesn't catch all possible comodifications,
            // but does catch the ones that corrupt traversal
            if (tail != fence || result == null)
                throw new ConcurrentModificationException();
            lastRet = cursor;
            cursor = (cursor + 1) & (elements.length - 1);
            return result;
        }

        public void remove() {
            if (lastRet < 0)
                throw new IllegalStateException();
            if (delete(lastRet)) { // if left-shifted, undo increment in next()
                cursor = (cursor - 1) & (elements.length - 1);
                fence = tail;
            }
            lastRet = -1;
        }
    }

    private class DescendingIterator implements Iterator<E> {

        private int cursor = tail;
        private int fence = head;
        private int lastRet = -1;

        public boolean hasNext() {
            return cursor != fence;
        }

        public E next() {
            if (cursor == fence)
                throw new NoSuchElementException();
            cursor = (cursor - 1) & (elements.length - 1);
            E result = elements[cursor];
            if (head != fence || result == null)
                throw new ConcurrentModificationException();
            lastRet = cursor;
            return result;
        }

        public void remove() {
            if (lastRet < 0)
                throw new IllegalStateException();
            if (!delete(lastRet)) {
                cursor = (cursor + 1) & (elements.length - 1);
                fence = head;
            }
            lastRet = -1;
        }
    }

    public boolean contains(Object o) {
        if (o == null)
            return false;
        int mask = elements.length - 1;
        int i = head;
        E x;
        while ( (x = elements[i]) != null) {
            if (o.equals(x))
                return true;
            i = (i + 1) & mask;
        }
        return false;
    }

    public boolean remove(Object o) {
        return removeFirstOccurrence(o);
    }

    public void clear() {
        int h = head;
        int t = tail;
        if (h != t) { // clear all cells
            head = tail = 0;
            int i = h;
            int mask = elements.length - 1;
            do {
                elements[i] = null;
                i = (i + 1) & mask;
            } while (i != t);
        }
    }


    public Object[] toArray() {
        return copyElements(new Object[size()]);
    }

    public <T> T[] toArray(T[] a) {
        int size = size();
        if (a.length < size)
            a = (T[])java.lang.reflect.Array.newInstance(
                    a.getClass().getComponentType(), size);
        copyElements(a);
        if (a.length > size)
            a[size] = null;
        return a;
    }


    public ArrayDequeList<E> clone() {
        try {
            ArrayDequeList<E> result = (ArrayDequeList<E>) super.clone();
            result.elements = Arrays.copyOf(elements, elements.length);
            return result;

        } catch (CloneNotSupportedException e) {
            throw new AssertionError();
        }
    }

    private static final long serialVersionUID = 2340785798034038923L;

    private void writeObject(ObjectOutputStream s) throws IOException {
        s.defaultWriteObject();

        // Write out size
        s.writeInt(size());

        // Write out elements in order.
        int mask = elements.length - 1;
        for (int i = head; i != tail; i = (i + 1) & mask)
            s.writeObject(elements[i]);
    }

    private void readObject(ObjectInputStream s)
            throws IOException, ClassNotFoundException {
        s.defaultReadObject();

        // Read in size and allocate array
        int size = s.readInt();
        allocateElements(size);
        head = 0;
        tail = size;

        // Read in all elements in the proper order.
        for (int i = 0; i < size; i++)
            elements[i] = (E)s.readObject();
    }
}
Sign up to request clarification or add additional context in comments.

8 Comments

Maybe I'm missing something but I don't get that last argument. Finding the storage location of the i'th item should take no more than adding tail to it and ANDing it with the length of the backing array minus one, no big deal and certainly not a linear time algorithm
Sometimes if you allow in place insertion you leave some gaps there. So it's like a dictionary insert. Where you push one or two books down the line. You can reasonable insert items without copying all the data.
But ArrayDeque isn't sparse, it's just a plain old power-of-two sized circular array
Would ArrayDeque function any differently than ArrayList with insertion?
listIterator is the trickiest, because it's not just a simple iterator, it's a bidirectional walker with add, set and remove methods.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.