Use std::vector member variable as list in swig generated python interface

Question

I am using SWIG to create python wrappers for a C++ library. The library has a few public std::vector member variables that I would like to use as lists in python. However, I haven't been able to found a solution that works. Below is a simplified example which illustrates my current solution:

example.h

#pragma once

#include <vector>

#if defined _WIN32
    #define EXPORT __declspec(dllexport)
#else
    #define EXPORT
#endif

namespace Example {
    class EXPORT MyClass {
        public:
        std::vector<int> my_data{1, 2, 3};
        std::vector<int> get_vec();
    };
}

example.cpp

#include "example.h"

namespace Example {
    std::vector<int> MyClass::get_vec() {
        std::vector<int> v{1, 3, 5};
        return v;
    }
}

example.i

%module example

%include "stdint.i"
%include "std_vector.i"

%naturalvar;

%{
#include <example.h>
%}

%template(int_vector) std::vector<int>;
%include <example.h>

I'am also attaching the CMakeLists.txt file I use, in case anyone wants to build the project.

cmake_minimum_required(VERSION 3.15)
project(CMakeSwigExample LANGUAGES CXX)

include_directories(${CMAKE_CURRENT_SOURCE_DIR})
add_library(example_cpp SHARED example.h example.cpp)
target_compile_features(example_cpp PUBLIC cxx_std_17)

if (UNIX)
set_target_properties(example_cpp PROPERTIES INSTALL_RPATH "$ORIGIN")
endif()

FIND_PACKAGE(SWIG REQUIRED)
INCLUDE(${SWIG_USE_FILE})

FIND_PACKAGE(PythonLibs)

SET(CMAKE_SWIG_FLAGS "")

SET_SOURCE_FILES_PROPERTIES(example.i PROPERTIES CPLUSPLUS ON)
set_property(SOURCE itnfileio.i PROPERTY SWIG_MODULE_NAME example)

SWIG_ADD_LIBRARY(example 
    TYPE SHARED
    LANGUAGE python
    SOURCES example.i)

target_include_directories(example
    PRIVATE 
    ${PYTHON_INCLUDE_DIRS})

target_link_libraries(example PRIVATE example_cpp ${PYTHON_LIBRARIES})

Here is an example of how the generated wrapper can be used in python

>>> from example import MyClass
>>> m = MyClass()
>>> m.get_vec()
(1, 3, 5)
>>> m.my_data
(1, 2, 3)
>>> m.my_data = [9, 8, 7]
>>> m.my_data.append(6)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute 'append'

This is close to what I want, but since std_vector.i converts std::vector to a tuple, it is not possible to append elements to the my_data variable. If I remove the %naturalvar from example.i I can use list methods like append on my_data. However, I then get an error message if I try to assign the variable to a python list (since the type of my_data then is a proxy of a swig object).

I have tried adding a typemap to example.i file. Then the get_vec method returned a python list, but the type of my_data did not change. This was the typemap tried to add to example.i

%typemap(out) std::vector<int> (PyObject* tmp) %{

    tmp = PyList_New($1.size());
    for(int i = 0; i < $1.size(); ++i)
        PyList_SET_ITEM(tmp,i,PyLong_FromLong($1[i]));
    $result = SWIG_Python_AppendOutput($result,tmp);
%}

What do I need to do to be able to use my_data as a normal list in python? Is typemaps the way to go, and if so, what would it look like?

Mark Tolonen · Accepted Answer · 2021-04-18 20:44:39Z

1

Without %naturalvar, this may be acceptable to you:

>>> import example as e
>>> x=e.MyClass()
>>> x.my_data
<example.int_vector; proxy of <Swig Object of type 'std::vector< int > *' at 0x0000029A09E95510> >
>>> x.my_data.push_back(5)
>>> list(x.my_data)
[1, 2, 3, 5]

If you need a more robust solution, you can use %pythoncode to implement a more natural wrapper around the SWIG wrapper:

%module test

%{
#include <vector>
#include <sstream>
#include <string>

class MyClass {
public:
    std::vector<int> my_data{1, 2, 3};
    std::vector<int> get_vec() {
        std::vector<int> v{1, 3, 5};
        return v;
    }
};
%}

%include <std_vector.i>
%include <std_string.i>
%template(vint) std::vector<int>;

%rename(_MyClass) MyClass;  # Rename the original class

class MyClass {
public:
    std::vector<int> my_data;
    std::vector<int> get_vec();
};

%pythoncode %{

# Wrap the my_data member so append/pop work on the actual member.
class MyClassMyData:

    def __init__(self,myclass):
        self._myclass = myclass

    def append(self,value):
        self._myclass.my_data.push_back(value)

    def pop(self):
        return self._myclass.my_data.pop()

    def __repr__(self):
        return repr(list(self._myclass.my_data))

# Expose the get_vec() and my_data the way you want
class MyClass:

    def __init__(self):
        self._myclass = _MyClass()

    def get_vec(self):
        return list(self._myclass.get_vec())  # return a list instead of tuple

    @property
    def my_data(self):
        return MyClassMyData(self._myclass)  # return the wrapper

    @my_data.setter
    def my_data(self,value):
        self._myclass.my_data = vint(value)  # allows list assignment
%}

Output:

>>> import test
>>> t=test.MyClass()
>>> x=t.get_vec()     # returns a list now
>>> x
[1, 3, 5]
>>> x.append(4)       # it's a Python list, so append/pop work
>>> x
[1, 3, 5, 4]
>>> x.pop()
4
>>> t.my_data             # actually a MyClassMyData proxy but displays as list
[1, 2, 3]
>>> t.my_data.append(4)   # using proxy append
>>> t.my_data
[1, 2, 3, 4]
>>> t.my_data = [1]       # proxy property setter works
>>> t.my_data
[1]
>>> t.my_data.pop()       # as does proxy pop
1
>>> t.my_data
[]

edited Apr 18, 2021 at 20:44

answered Apr 14, 2021 at 21:42

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

larsjr Over a year ago

If I can't get anything else to work, I might have to settle on this solution. What I dislike about it is that you get an error message if you try to assign a list to my_data, for example x.my_data = [5, 6, 7]. I know you can work around this by writing x.my_data = e.int_vector([5, 6, 7]), but then users of the library need to be aware of this 'nonstandard' way of assigning.

Mark Tolonen Over a year ago

@larsjr You can always wrap the result in more Python code to give it a more Pythonic interface. See update.

Collectives™ on Stack Overflow

Use std::vector member variable as list in swig generated python interface

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related