Failed to convert UTF-16 string buffer created in C++ into python using SWIG

Question

I have a C++ function I want to use in Python:

struct DataWrap
{
    wchar_t* data;
    int size;
}

extern int fun1( const char* szInBuffer, unsigned int inSize,DataWrap* buff);

fun1 takes an array of chars given in szInBuffer with size of inSize and returns a newly allocated UTF-16 string in dataWrap->data

I would like to use this function fun1 from Python.

%typemap(argout) DataWrap* buff{
                int byteorder = -1;
                $input = PyUnicode_DecodeUTF16( (const char*)($1->data), ($1->size)*sizeof(wchar_t) , NULL, &byteorder);
}

this does work, but there is a memory leak here as the dataWrap->data allocated by fun1 is never released.

I tried to fix that like this:

%typemap(argout) DataWrap* buff{
            int byteorder = -1;
            $input = PyUnicode_DecodeUTF16( (const char*)($1->data), ($1->size)*sizeof(wchar_t) , NULL, &byteorder);
            delete $1;
        }

and now the code crashes.

What am I doing wrong? Is that the correct why to work with SWIG in order to get a UTF-16 string into Python?

Mark Tolonen · Accepted Answer · 2012-04-22 18:50:18Z

Try the freearg typemap:

%typemap(freearg) DataWrap *buff {
    delete $1->data;
}

Also, you had delete $1;. Should it be delete $1->data?

Edit

I discovered another issue when playing with this problem that may be related to your issue. If you are on Windows and your implementation of fun1 is in a different DLL than the SWIG wrapper code, make sure to make both DLLs link to the same C Runtime DLL (via the /MD compiler switch in Microsoft VC++, for example), since the memory would then be allocated (with new I assume) in one DLL and freed in another.

Here's the code I was playing with. It may help you. It also includes a typemap to eliminate sending the string and size from Python and a typemap to generate a temporary DataWrap object so it doesn't have to be passed as well. In the implementation file x.cpp I faked an implementation of converting the string to UTF16 that only works with simple ASCII strings for something to work with.

Also note I got it to work without using the freearg typemap, although using the typemap worked for me as well.

makefile

_x.pyd: x_wrap.cxx x.dll
    cl /MD /nologo /LD /Zi /EHsc /W4 x_wrap.cxx /Id:\dev\python27\include -link /LIBPATH:d:\dev\python27\libs /OUT:_x.pyd x.lib

x.dll: x.cpp x.h
    cl /MD /nologo /Zi /LD /EHsc /W4 x.cpp

x_wrap.cxx: x.h x.i
    swig -python -c++ x.i

x.i

%module x

%begin %{
#pragma warning(disable:4127 4211 4706)
%}

%{
    #include "x.h"
%}

%include <windows.i>

%typemap(in) (const char *szInBuffer,unsigned int inSize) {
   if (!PyString_Check($input)) {
       PyErr_SetString(PyExc_ValueError, "Expecting a string");
       return NULL;
   }
   $1 = PyString_AsString($input);
   $2 = PyString_Size($input);
}

%typemap(in, numinputs=0) DataWrap* buff (DataWrap temp) {
   $1 = &temp;
}

%typemap(argout) DataWrap* buff {
    int byteorder = -1;
    $result = PyUnicode_DecodeUTF16((const char*)($1->data), ($1->size)*sizeof(wchar_t), NULL, &byteorder);
    delete [] $1->data;
}

%include "x.h"

x.h

#ifdef API_EXPORTS
#   define API __declspec(dllexport)
#else
#   define API __declspec(dllimport)
#endif

struct DataWrap
{
    wchar_t* data;
    int size;
};

extern "C" API void fun1(const char* szInBuffer, unsigned int inSize, DataWrap* buff);

x.cpp

#include <stdlib.h>

#define API_EXPORTS
#include "x.h"

API void fun1(const char* szInBuffer, unsigned int inSize, DataWrap* buff)
{
    unsigned int i;
    buff->size = inSize;
    buff->data = new wchar_t[inSize];
    for(i = 0; i < inSize; i++)
        buff->data[i] = szInBuffer[i];
}

Output

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import x
>>> x.fun1('abc')
u'abc'

Thanks mate , I'll check it. do i need the conversion $1 = PyString_AsString($input); doesn't swig do it ? i need to return two values the return value of fun1 and the string in buff so returning buff to the $result is not an option
The manual conversion lets you send just the string instead of the string and it's length, but you don't have to do it. If you want to return two values you have to create a tule of the result instead. I'll update later if you need to see how to do it.

Collectives™ on Stack Overflow

Failed to convert UTF-16 string buffer created in C++ into python using SWIG

1 Answer 1

makefile

x.i

x.h

x.cpp

Output

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

makefile

x.i

x.h

x.cpp

Output

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related