Simplest way to share an array when embedding Python

Question

I'm writing a simulation in C++ and decided to delegate the initialization of some arrays to Python by embedding it into my application and use some Python functions for the generation of the initial data.

Since my 2D/3D arrays are allocated and managed by the C++ part, I need to pass them to Python. The best way is probably the buffer protocol. I could write a Python class in C++, which implements the buffer protocol and creates a buffer object to the storage I have in C++.

Now, I would like to avoid over-engineering this procedure. In facts, I just need to pass a pointer, a tuple of sizes and strides to a Python function.

The question is thus: What is the simplest way to pass this information to a Python routine which can then use it, possibly with Numpy, to initialize the pointed storage?

Just to give an example of what I need to do, the Python function I want to call from C++ is:

def init_field(field, X, Y):
    field[:, :] = np.sqrt(X) + np.tanh(Y)

where X and Y are numpy arrays which are created by Python and borrowed by the C++ application and field is exactly this 2D array which is managed by C++. I could of course substitute field with one or more parameters carrying information about the pointer, the sizes and the strides and construct the numpy array. How do I do that?

There exists a boost library for Python. Have you looked into that? (Note that I have not, I'm just curious if you have, since you're looking for a simple integration.) — steinar
– steinar, Commented Dec 28, 2015 at 9:30
Yes, I'm aware of that library, but it does not seem to directly help in my case. The hard thing is not really interfacing with Python, but to make it understand the pointer and the data structure I have. There is a boost::python version of the numpy C interface, but I don't know how well maintained and how future-proof it is. I would like to avoid too many dependencies because the systems where I will deploy this code are not very straighforward. — Spiros
– Spiros, Commented Dec 28, 2015 at 14:43

Klamer Schutte · Accepted Answer · 2015-12-29 16:37:54Z

2

Something not too hard is to use PyArray_SimpleNewFromData. Using that might lead to code like:

void use_array(float* ptr, int width, int height)
{
    PyObject *pFunc = pyFunction("python_function_name");
    if (pFunc == 0)
        return;
    PyObject *pArgs = PyTuple_New(1);
    npy_intp dims[2];
    dims[1] = width;
    dims[0] = height;
    PyTuple_SetItem(pArgs, 0, PyArray_SimpleNewFromData(2, dims, NPY_FLOAT, ptr));
    PyObject *pValue = PyObject_CallObject(pFunc, pArgs);
    Py_DECREF(pArgs);
    if (pValue != NULL) {
        Py_DECREF(pValue);
    }
    else 
    {
        PyErr_Print();
        fprintf(stderr,"Call failed\n");
        return;
    }
    Py_DECREF(pFunc);
}

When setting up the embedded Python environment you should call import_array() as described in http://docs.scipy.org/doc/numpy-1.10.0/reference/c-api.array.html#miscellaneous

edited Dec 29, 2015 at 16:37

answered Dec 28, 2015 at 12:20

Klamer Schutte

1,0939 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Klamer Schutte Over a year ago

Spiros, definition of PyArray_SimpleNewFromData in .../Python27/Lib/site-packages/numpy/core/include/numpy/ndarrayobject.h is

#define PyArray_SimpleNewFromData(nd, dims, typenum, data)          PyArray_New(&PyArray_Type, nd, dims, typenum, NULL,                      data, 0, NPY_ARRAY_CARRAY, NULL)

. Here the NULL before data can have the pointer to the strides.

Jaime Over a year ago

Here's a link to the relevant NumPy's C-API docs. One of those functions is sure to fit your needs.

Jaime Over a year ago

Segfaults with no obvious reason when using NumPy's C API are often due to not having called import_array(). It's hard to tell without seeing your full code, but this link should be helpful.

Klamer Schutte Over a year ago

My code runs more or less unaltered within a larger framework what is hard to reproduce here. Call to import_array as indicated above by Jaime is essential.

Spiros Over a year ago

Thanks for your clarifications, now everything works the way I wanted. I wasn't sure about the inport_array function call because I was embedding numpy, not extending it, but I see I need it anyway.

|

Collectives™ on Stack Overflow

Simplest way to share an array when embedding Python

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related