Return a C String Array from a C Extension

Question

I have a C function that returns an array of strings. How can I call it in the form of a Python C extension which will return the array back to a calling Python function? (I am new to Python C extensions and have minimal experience with the extensions)

This is the definition I tried:

static PyObject* _get_array(PyObject* self, PyObject* args)
{
    int64_t value;
    int init_level;
    int final_level;

    if(!PyArg_ParseTuple(args, "Lii", &value, &init_level, &final_level))
        return NULL;

    // returning the array as a Python object by o
    return Py_BuildValue("o", _get_array(value, init_level, final_level));
}

and the method def:

static PyMethodDef array_methods[] = {
    { "get_array", _get_array, METH_VARARGS, "Returns a string array"},
    { NULL, NULL, 0, NULL }
};

Update

get_array function:

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <Python.h>

char **get_array(int64_t value, int init_level, int final_level) {

  int SHIFTS []= {44, 40, 36, 32, 28, 24, 20, 16, 12, 8, 4, 0};

  long count =  1 << (4* (final_level - init_level));
  char** t_array;
  t_array = malloc(sizeof(char*)*count);

  int shift_coff = 11 -(final_level-init_level);
  int64_t base = (value << SHIFTS[shift_coff]);

  for (long i=0; i < count; i++){
    t_array[i] = malloc((4+final_level)*sizeof(char));
    sprintf(t_array[i], "%llX", (base + i));
  }

  return t_array;
}

Where's the Python code showing how you want to call it? Have you been through any tutorials, such as this one: Python - Extension Programming With C? — lurker
– lurker, Commented Apr 24, 2019 at 22:23
Yes I did. I have done extensions that return simple values, like ints or floats, but couldn't find a way to return an array — c00der
– c00der, Commented Apr 25, 2019 at 3:53
I think you need to show the C function you want to call (to show how the return type is allocated, etc). At the moment you just have a nonsensical recursive call to _get_array. It's basically pretty easy - you create a Python list and iterate through it to fill it with Python strings - but the details do depend on code you don't show — DavidW
– DavidW, Commented Apr 25, 2019 at 9:39
@DavidW, sorry for the delay. Just added. My concern is, there can be a lot of items in the array that is returned and don't want to use a Python list because of the overhead. I would much prefer using a pure C array instead. Somewhere I read you can use Numpy arrays but I am confused whether I can return a pure C array or need to use Python or Numpy specfic array implementation — c00der
– c00der, Commented Apr 25, 2019 at 14:49
You definitely can't return a pure C array. Numpy string arrays would work fine, but the strings must be of fixed length (which it looks like these are). Python lists are better than you think. I'll post a proper answer if/when I can (and if no one else does it first) — DavidW
– DavidW, Commented Apr 25, 2019 at 15:18

DavidW · Accepted Answer · 2019-04-26 08:07:06Z

You can't return your char** directly Python since Python only understands objects of type PyObject* (since this contains the information needed to handle reference counting and identifying the type). You therefore have to create a suitable Python object. The simplest option would be a list of strings. The next simplest you be a numpy array using the string type (you can do this easily because all your strings are the same length). Neither of these have a direct Py_BuildValue conversion so you have to write loops yourself.

For a list of strings you simply create the list with PyList_New then go through element by element with PyList_SetItem:

char** array = get_array(value, init_level, final_level);
PyObject* list = PyList_New(1 << (4* (final_level - init_level)));
if (!list) return NULL;

for (int i=0; i<(1 << (4* (final_level - init_level))); ++i) {
    PyObject* item = PyBytes_FromStringAndSize(array[i],(4+final_level));
    if (!item) goto failed;

    if (PyList_SetItem(list,i,item) != 0) {
        Py_DECREF(item);
        goto failed;
    }

    free(array[i]); // deallocate array as we go
}
free(array);

// returning the array as a Python object by o
return list;

failed:
Py_DECREF(list);
// also deallocate the rest of array?
return NULL;

Note that I haven't finalised memory management of failure so you'll leak array.

For the numpy array you allocate an array with the correct string type, and then copy the data into it

char** array = get_array(value, init_level, final_level);

// create an "Sx" dtype, where x is a suitable number
PyArray_Descr *desc = PyArray_DescrNewFromType(NPY_STRING);
desc->elsize = (4+final_level);

npy_intp array_length[] = {1 << (4* (final_level - init_level))};
PyObject* nparray = PyArray_SimpleNewFromDescr(1,array_length,desc);
if (!nparray) return NULL; // clean up array too

for (int i=0; i<(1 << (4* (final_level - init_level))); ++i) {
    char* data = PyArray_GETPTR1((PyArrayObject*)nparray,i);

    // copy data
    for (int j=0; j<(4+final_level); ++j) {
        data[j] = array[i][j];
    }

    free(array[i]); // deallocate array as we go
}
free(array);

// returning the array as a Python object by o
return nparray;

Again, not all the error handling is perfect. For this example to work you must call import_array() in your module init function.

In both cases you might be better not allocating memory in get_array but instead writing directly into your Python objects.

Collectives™ on Stack Overflow

Return a C String Array from a C Extension

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related