Array allocation of SWIG C++ type from Python

Question

I'm writing a python script for a program that has exposed its C++ API using SWIG. A SWIG exposed function has an interface like this:

void writePixelsRect(JoxColor* colors, int left, int top, int width, int height);

JoxColor is a POD struct looking like this:

struct JoxColor {
    float r, g, b, a;
};

I can easily create a single JoxColor in Python and invoke a call to writePixelsRect like this:

c = JoxApi.JoxColor()
c.r = r
c.g = g
c.b = b
c.a = a
JoxApi.writePixelsRect(c, x, y, 1, 1)

Repeatedly calling writePixelsRect with a 1x1 pixel rectangle is very slow so I want to create an array of JoxColor from python so I can write bigger rectangles at the time. Is this possible with SWIG types?

Note that I don't have access to the source code for the C++ library exposing JoxColor and writePixelsRect so I can't add a help function for this. I also don't want to introduce new C++ code in the system since it would force the users of my python script to compile the C++ code on whatever platform they are running. I do have access to ctypes in the python environment so if I could somehow typecast a float array created in ctypes to the type of JoxColor* for SWIG it would work for me.

Do you have access to *.swig file? Could you check if it includes any functionalities from SWIG libraries like cpointer.i, carrays.i or cmalloc.i - swig.org/Doc1.3/Library.html. Maybe someone that prepared this SWIG wrapper already included there functionality of handling array of JoxColor? You could also check JoxAPImodule.py file for list of all available function and classes. — Zuljin
– Zuljin, Commented May 5, 2011 at 15:19
I don't have access to the swig files and I'm not able to modify the SWIG wrapping part. — Laserallan
– Laserallan, Commented May 5, 2011 at 23:31

gsteff · Accepted Answer · 2011-05-05 13:16:39Z

2

+50

This is kinda tricky, but could you, at least for this part of the code, use a pure-ctypes solution? Basically manually look at the symbols exported by the shared libary file to find the name that the writePixelsRect function was exported as. C++ does name mangling, so while it might just be writePixelsRect if the library authors chose to make it extern "C", it might be something much messier, like _Z15writePixelsRectP8JoxColoriiii (that's how it was exported in a dummy C++ library I just created on my system).

On Linux, this command should tell you the symbol name:

nm libjox.so | grep writePixel | cut -d " " -f 3

Then, save that string and insert it into Python code kinda like this:

from ctypes import *

LIBRARY_NAME = 'libjox.so'
c = cdll.LoadLibrary(LIBRARY_NAME)

WIDTH = 20
HEIGHT = 20

class JoxColor(Structure):
    _fields_ = [("r", c_float), ("g", c_float), ("b", c_float), ("a", c_float)]

ColorBlock = JoxColor * (WIDTH * HEIGHT)

data_array = ColorBlock()

color = JoxColor(0, 0, 1, 0)
for i in range(WIDTH * HEIGHT):
    data_array[i] = color

c._Z15writePixelsRectP8JoxColoriiii(data_array, 0, 0, WIDTH, HEIGHT)

Assuming that _Z15writePixelsRectP8JoxColoriiii is the symbol that the function is accessible as in the shared library. Running this code just worked on my system with a dummy library like this:

#include <stdio.h>

struct JoxColor {
    float r, g, b, a;
};

void writePixelsRect(JoxColor *colors, int left, int top, int width, int height) {
    int p = 0;
    printf("size: %i, %i\n", width, height);
    for (int i = 0; i < width; i++) {
        for (int j = 0; j < height; j++) {
            JoxColor color = colors[p];
            printf("pixel: %f, %f, %f, %f\n", color.r, color.g, color.b, color.a);
        }
    }
}

So I'm hopeful that it's not too far from working code in your environment.

edited May 5, 2011 at 13:16

answered May 5, 2011 at 3:34

gsteff

4,9621 gold badge24 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

gsteff Over a year ago

Also, if you want to run this test, the above library code should treated as C++, not C, to replicate the name mangling (I just detest writing C++). In other words, save in a ".cpp" file and compile with g++.

Zuljin Over a year ago

I'm not sure how it is on Linux but on Windows I think that module doesn't export all functions/methods that it provide to Python. It export only one function called PyInit_modulename or initmodulename and through this function it calls everything else. I think that you cannot import writePixelsRect directly using ctypes, because it isn't there.

Laserallan Over a year ago

Thanks. I do indeed have access to the original c++ api functions but I didn't consider them because of the mangling. However I can keep the swig based code as a fallback for platforms where I don't manage to figure out the mangled symbol name :)

Adam · Accepted Answer · 2011-05-10 23:51:38Z

0

Barring special typemaps, this SWIG prototype

void writePixelsRect(JoxColor* colors, int left, int top, int width, int height);

means that colors is a single object of type JoxColor, not an array. The fact that your call with just a single object works (albeit slowly) suggests that that's true. So passing an array is likely to just give you a type mismatch error from the SWIG wrapper code.

But honestly, this looks like a function that writes an arbitrarily-large rectangle. So if you want to draw a larger rectangle (of one color) just pass in a larger width and/or height:

JoxApi.writePixelsRect(c, x, y, 10, 20)

Edit: I didn't realize you were writing the SWIG wrapper, I thought that was provided to you. In that case you can write a typemap that will convert a Python list (or tuple, or whatever you want) into JoxColor*. The SWIG docs show an example for how turn a Python list-of-strings into char**: http://www.swig.org/Doc1.3/Python.html#Python_nn59 The typemap uses Python C APIs to do the conversion, you can use anything the Python docs say. Essentially you allocate a JoxColor array then iterate over the Python list object and use PyList_GetItem to get each individual object. That will return a SWIG-wrapped PyObject, you can use SWIG_ConvertPtr(list_item_py_object, (void**)&joxcolor_ptr, $descriptor(JoxColor *), 0) to convert that into a pointer to your actual JoxColor element. Then you can copy that into your array.

Remember that a typemap for JoxColor* will apply everywhere JoxColor* appears, you can say JoxColor* colors to specialize it to just this case.

FYI, by default SWIG wraps JoxColor*, JoxColor&, JoxColor, and JoxColor[] in exactly the same way, as a single object. Python only has objects, it doesn't know of pointers/references/arrays (Python lists are also objects). http://www.swig.org/Doc1.3/Python.html#Python_nn22

edited May 10, 2011 at 23:51

answered May 10, 2011 at 5:00

Adam

17.4k8 gold badges55 silver badges103 bronze badges

3 Comments

Laserallan Over a year ago

It's one way of interpreting the API, but not what it does. It does take a pointer to a set of pixels and copies them into the target image in the rectangle specified. The most likely reason it wasn't properly mapped to SWIG is that it's a huge API and much of the SWIG:ing was done automatically and this function seems to need more attention to work all the way.

Adam Over a year ago

I didn't realize you were writing the SWIG wrapper yourself. See my edit.

Laserallan Over a year ago

I don't write my own SWIG wrapper. It's a big C++ API that has been more or less automatically been exposed as for python programmers using SWIG. I do have access to the C++ dll's but not not the SWIG files and it's not an option to change the actual wrapping code.

Collectives™ on Stack Overflow

Array allocation of SWIG C++ type from Python

2 Answers 2

3 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related