Why does Cython needs more Python calls if elements of NumPy array are defined during array creation?

Question

I don't understand why Cython needs more Python calls to compile my .pyx file if I define elements of array during array's creation (#-1-).

For the elements pos1 and pos2, PyFloat_FromDouble is called four times, twice for each variable, but this function is not used if I create an empty array or array of zeros and change the elements afterwards (#-2-).

import cython
import numpy as np
cimport numpy as np
from libc.math cimport sin
from libc.math cimport cos

@cython.boundcheck(False)
@cython.binding(False)
@cython.wraparound(False)
cpdef np.ndarray[np.float64_t, ndim = 2] mat (double alfa):
    cdef double pos1 = cos(alfa * 0.01745)
    cdef double pos2 = sin(alfa * 0.01745)
    cdef np.ndarray[np.float64_t, ndim = 2] mat_ret

    #-1-
    mat_ret = np.array([[pos1, pos2, 0.0],
                        [pos1, pos2, 0.0],
                        [ 0.0,  0.0, 0.0]], dtype = np.float64)

    #-2-
    mat_ret = np.zeros((3,3), dtype = np.float64)
    mat_ret[0,0] = pos1
    mat_ret[0,1] = pos2
    mat_ret[1,0] = pos1
    mat_ret[1,1] = pos2

    return mat_ret

I'm using Python 2.7.13, NumPy 1.13.1 and Cython 0.25.2

MSeifert · Accepted Answer · 2017-08-12 14:29:20Z

2

That shouldn't be surprising. Your pos1 and pos2 are C doubles, but if you insert them in a list (the [pos1, pos2, 0.0] in your code) they need to be Python objects because lists store PyObject pointers, thus the call to PyFloat_FromDouble. And you really create 3 lists containing 2 pos1 and 2 pos2 so you will end up creating 4 lists - the outer one and the inner 3 - and calling PyFloat_FromDouble 4 times. And it's going to call np.array where these values will be converted to doubles again!

On the other hand if you create a np.zeros array with the "appropriate" dtype it can just insert the doubles. Because it doesn't have to insert them in an intermediate Python object it doesn't need to box them (PyFloat_FromDouble) at all.

edited Aug 12, 2017 at 14:29

answered Aug 12, 2017 at 14:23

MSeifert

154k41 gold badges356 silver badges377 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Nemanja Stojanovic Over a year ago

Thank. So, is there a way to create numpy array without this overhead? numpy.array function needs lists as its arguments. I should probably create C arrays for arguments, but I'm amazed that Cython won't do that for me. And going this far, by making 2D matrices in C actually makes this solution non pythonic at all

MSeifert Over a year ago

As far as I know there's no better way (if one wants to avoid lists) than your approach #2 (np.zeros and then setting them manually). Maybe it would be easier if one uses memoryviews but not sure about that.

Collectives™ on Stack Overflow

Why does Cython needs more Python calls if elements of NumPy array are defined during array creation?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related