cython insert element in array.array

Question

I am trying to convert some python code into cython. In the python code I use data of type array.array('i', [...]) and use the method array.insert to insert an element at a specific index. in cython however, when I try to insert an element using the same method I get this error: BufferError: cannot resize an array that is exporting buffers

basically:

from cpython cimport array
cdef array.array[int] a = array.array('i', [1,2,3,3])
a.insert(1,5) # insert 5 in the index 1 -> throws error

I have been looking at cyappend3 of this answer but I am using libcpp and not sure I understand the magic written there. Any idea how to insert an element at a specific index in an array.array?

Either define another pointer to pint to you array then use indexing like : cdef int[:] ca = a or simply a.data.as_int returns pointer as int and you can use that to index as_int[1] … — amirhm
– amirhm, Commented Nov 1, 2022 at 23:03
@amirhm can you please share a piece of the code for both options? I am not sure I follow here — sos
– sos, Commented Nov 1, 2022 at 23:06
@hpaulj yes, the initial python code uses array.array and I want to convert it to cython using array.array of the cpyton — sos
– sos, Commented Nov 1, 2022 at 23:42

DavidW · Accepted Answer · 2022-11-02 07:16:54Z

2

Partial answer:

BufferError: cannot resize an array that is exporting buffers

This is telling you that you have a memoryview (or similar) of the array somewhere. It isn't possible to resize it because that memoryview is looking directly into that array's data and resizing the array will require reallocating the data. You can replicate this error in Python too if you do view = memoryview(arr) before you try to insert.

In your case:

cdef array.array[int] a = array.array('i', [1,2,3,3])

cdef array.array[int] a is defining an array with a fast buffer to elements of the array, and it's this buffer that prevents you from resizing it. If you just do cdef array.array a it works fine. Obviously you lose the fast buffer access to individual elements, but that's because you're trying to change the data out from under the buffer.

I strongly recommend you don't resize arrays though. Not only does it involve the O(n) copy of every element of the array. Also, unlike Python list, array doesn't over-allocate so even append causes a complete reallocation and copy every time (i.e. is O(n) rather than amortized O(1)).

Instead I'd suggest keeping the data as a Python list (or maybe something else) until you've finalized the length and only then converting to array.

edited Nov 2, 2022 at 7:16

answered Nov 2, 2022 at 6:54

DavidW

31.2k7 gold badges64 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sos Over a year ago

Thanks! for append I think array has a method called extend_buffer that the documentation calls efficient. not sure how it will perform though against append of list.

amirhm · Accepted Answer · 2022-11-02 10:20:13Z

2

what has been answered here in this post is correct, (https://stackoverflow.com/a/74285371/4529589), and I have the same recommendation.

However I want to add this point that if you want to use the insert but as well if you want to use the insert and still define as the c buffer, you could use the std::vector. This will be faster.

from libcpp.vector cimport vector

cdef vector[int] vect = array.array('i', [1,2,3,3])    
vect.insert(vect.begin() + 1 ,5)

and as well I recomend if you want to use this solution just drop the array and from the begining just use the vector initialization.

answered Nov 2, 2022 at 10:20

amirhm

1,23911 silver badges15 bronze badges

Collectives™ on Stack Overflow

cython insert element in array.array

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related