How to wrap C++17 std::variant with SWIG for use in python?

Question

I am trying to wrap in Python with SWIG some C++17 code that uses std::variant.

I've found this answer (https://stackoverflow.com/a/58513139) about wrapping boost::variant and I managed to adapt the code so that it works with std::variant instead. However, according to the specification of the answer, the code should work so that 'anywhere a C++ function takes a boost::variant we should transparently accept any of the types the variant can hold for that function argument'. This requirement seems to only be satisfied when a const reference to a std::variant is used. For example if in my c++ file dummy.cpp I have

void foo(std::variant<int, double> value)
{
  std::cout << "foo" << std::endl;
}

void bar(const std::variant<int, double>& value)
{
  std::cout << "bar" << std::endl;
}

then in my dummy.i

%include "variant.i" # My customization of boost_variant.i
%std_variant(DummyVariant, int, double); # Creating the variant type for SWIG/python

when I try to use it from a Python script the following works

from dummy import bar
bar(5)

this, however, does not:

from dummy import foo
foo(5)

I get: TypeError: in method 'foo', argument 1 of type 'std::variant< int,double >'.

Does anybody have an idea of what is missing in my variant.i file to make this works as expected? This is the file I have:

%{
#include <variant>

static PyObject *this_module = NULL;
%}

%init %{
  // We need to "borrow" a reference to this for our typemaps to be able to look up the right functions
  this_module = m; // borrow should be fine since we can only get called when our module is loaded right?
  // Wouldn't it be nice if $module worked *anywhere*
%}

#define FE_0(...)
#define FE_1(action,a1) action(0,a1)
#define FE_2(action,a1,a2) action(0,a1); action(1,a2)
#define FE_3(action,a1,a2,a3) action(0,a1); action(1,a2); action(2,a3)
#define FE_4(action,a1,a2,a3,a4) action(0,a1); action(1,a2); action(2,a3); action(3,a4)
#define FE_5(action,a1,a2,a3,a4,a5) action(0,a1); action(1,a2); action(2,a3); action(3,a4); action(4,a5)

#define GET_MACRO(_1,_2,_3,_4,_5,NAME,...) NAME
%define FOR_EACH(action,...)
  GET_MACRO(__VA_ARGS__, FE_5, FE_4, FE_3, FE_2, FE_1, FE_0)(action,__VA_ARGS__)
%enddef

#define in_helper(num,type) const type & convert_type ## num () { return std::get<type>(*$self); }
#define constructor_helper(num,type) variant(const type&)

%define %std_variant(Name, ...)

%rename(Name) std::variant<__VA_ARGS__>;

namespace std {
  struct variant<__VA_ARGS__> {
    variant();
    variant(const std::variant<__VA_ARGS__>&);
    FOR_EACH(constructor_helper, __VA_ARGS__);
    int index();

    %extend {
      FOR_EACH(in_helper, __VA_ARGS__);
    }
  };
}

%typemap(out) std::variant<__VA_ARGS__> {
  // Make our function output into a PyObject
  PyObject *tmp = SWIG_NewPointerObj(&$1, $&1_descriptor, 0); // Python does not own this object...

  // Pass that temporary PyObject into the helper function and get another PyObject back in exchange
  const std::string func_name = "convert_type" + std::to_string($1.index());
  $result = PyObject_CallMethod(tmp, func_name.c_str(),  "");
  Py_DECREF(tmp);
}

%typemap(in) const std::variant<__VA_ARGS__>& (PyObject *tmp=NULL) {
  // I don't much like having to "guess" the name of the make_variant we want to use here like this...
  // But it's hard to support both -builtin and regular modes and generically find the right code.
  PyObject *helper_func = PyObject_GetAttrString(this_module, "new_" #Name );
  assert(helper_func);
  // TODO: is O right, or should it be N?
  tmp = PyObject_CallFunction(helper_func, "O", $input);
  Py_DECREF(helper_func);
  if (!tmp) SWIG_fail; // An exception is already pending

  // TODO: if we cared, we chould short-circuit things a lot for the case where our input really was a variant object
  const int res = SWIG_ConvertPtr(tmp, (void**)&$1, $1_descriptor, 0);
  if (!SWIG_IsOK(res)) {
    SWIG_exception_fail(SWIG_ArgError(res), "Variant typemap failed, not sure if this can actually happen");
  }
}

%typemap(freearg) const std::variant<__VA_ARGS__>& %{
  Py_DECREF(tmp$argnum);
%}

%enddef

which is taken almost verbatim from Flexo's answer, save a few changes to make it work with std::variant instead of boost::variant.

It works fine here when the argument is const std::variant<int, double>&. You need to add more cases in variant.i to make it work for a by-value case — Jens Munk
– Jens Munk, Commented Jan 29, 2021 at 0:07
My best advice is to use proper ABI without templates and never wrap std::variant. If you insist, duplicate the macros in variant.i for a signature without const. — Jens Munk
– Jens Munk, Commented Jan 29, 2021 at 12:49
I don't understand what your best advice mean, sorry. You need to be more explicit. How do I not wrap std::variant if the C++ code requires std::variant as input? As for the second, I tried, but calling foo(5) I get a TypeError: Variant typemap failed, which means that the line const int res = SWIG_ConvertPtr(tmp, (void**)&$1, $1_descriptor, 0); fails. — A.Manfreda
– A.Manfreda, Commented Jan 29, 2021 at 15:58

Jens Munk · Accepted Answer · 2021-01-29 20:28:39Z

0

The easiest approach to avoid the use of templates on your interface and create an application binary interface (ABI). In your case this corresponds to creating an interface dummy.i

%module dummy

%{
#include "dummy.h"

void mywrap_foo_int(int i) {
  std::variant<int, double> tmp;
  tmp = i;
  foo(tmp);
}
void mywrap_foo_double(double d) {
  std::variant<int, double> tmp;
  tmp = d;
  foo(tmp);
}
%}

%include "variant.i"
%std_variant(DummyVariant, int, double);
%include "dummy.h"

Soon, you will encounter templates and nested templates. This can be done in a generic way like you have already done, which handles on the signature for bar. The task of wrapping basically corresponds to this.

answered Jan 29, 2021 at 20:28

Jens Munk

4,7751 gold badge27 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

A.Manfreda Over a year ago

Thanks for the answer. That is not transparent on the Python side, right? I have to explicitly call mywrap_foo_int or mywrap_foo_double depending on the type?

Jens Munk Over a year ago

No, you can just add python code to the wrapper and here check for the type.

Collectives™ on Stack Overflow

How to wrap C++17 std::variant with SWIG for use in python?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related