16

What is the correct way to pass a numpy 2d - array to a c function using ctypes ? My current approach so far (leads to a segfault):

C code :

void test(double **in_array, int N) {
    int i, j;
    for(i = 0; i<N; i++) {
        for(j = 0; j<N; j++) {
            printf("%e \t", in_array[i][j]);
        }
        printf("\n");
    }
}

Python code:

from ctypes import *
import numpy.ctypeslib as npct

array_2d_double = npct.ndpointer(dtype=np.double,ndim=2, flags='CONTIGUOUS')
liblr = npct.load_library('libtest.so', './src')

liblr.test.restype = None
liblr.test.argtypes = [array_2d_double, c_int]

x = np.arange(100).reshape((10,10)).astype(np.double)
liblr.test(x, 10)
8
  • 2
    You do know that double ** and double [N][N] are not interchangeable in C, don't you? Commented Mar 15, 2014 at 15:48
  • my problem is that I work with double ** in my c- code is there some solution to this ? Commented Mar 15, 2014 at 15:49
  • Well, I don't know python nor numpy, but if it's an NxN array, you should declare in_array as double (*in_array)[N], where N is the size of the second dimension. Commented Mar 15, 2014 at 15:53
  • How does this work if N is not fixed at runtime ? Commented Mar 15, 2014 at 16:09
  • I think you mean if it's not fixed at compile time. In that case, if you have a C99 compiler supporting VLAs, you could declare test as void test(int width, double in_array[][width]), but I doubt that this will work with numpy. Maybe you'd better wait for someone who is familiar with it, I really have no idea if you can do it with numpy. Commented Mar 15, 2014 at 16:23

4 Answers 4

28

This is probably a late answer, but I finally got it working. All credit goes to Sturla Molden at this link.

The key is, note that double** is an array of type np.uintp. Therefore, we have

xpp = (x.ctypes.data + np.arange(x.shape[0]) * x.strides[0]).astype(np.uintp)
doublepp = np.ctypeslib.ndpointer(dtype=np.uintp)

And then use doublepp as the type, pass xpp in. See full code attached.

The C code:

// dummy.c 
#include <stdlib.h> 

__declspec(dllexport) void foobar(const int m, const int n, const 
double **x, double **y) 
{ 
    size_t i, j; 
    for(i=0; i<m; i++) 
        for(j=0; j<n; j++) 
            y[i][j] = x[i][j]; 
} 

The Python code:

# test.py 
import numpy as np 
from numpy.ctypeslib import ndpointer 
import ctypes 

_doublepp = ndpointer(dtype=np.uintp, ndim=1, flags='C') 

_dll = ctypes.CDLL('dummy.dll') 

_foobar = _dll.foobar 
_foobar.argtypes = [ctypes.c_int, ctypes.c_int, _doublepp, _doublepp] 
_foobar.restype = None 

def foobar(x): 
    y = np.zeros_like(x) 
    xpp = (x.__array_interface__['data'][0] 
      + np.arange(x.shape[0])*x.strides[0]).astype(np.uintp) 
    ypp = (y.__array_interface__['data'][0] 
      + np.arange(y.shape[0])*y.strides[0]).astype(np.uintp) 
    m = ctypes.c_int(x.shape[0]) 
    n = ctypes.c_int(x.shape[1]) 
    _foobar(m, n, xpp, ypp) 
    return y 

if __name__ == '__main__': 
    x = np.arange(9.).reshape((3, 3)) 
    y = foobar(x) 

Hope it helps,

Shawn

Sign up to request clarification or add additional context in comments.

4 Comments

Thank you for this. However I don't really understand what is happening here. How does this extend to 3d and beyond?
@navjotk I'd really recommend Cython for more complicated cases for easier maintenance :)
WARNING: check that x.flags['C_CONTIGUOUS']==True. If not, do x = np.ascontiguousarray(x, dtype= x.dtype) before. Otherwise, if x was transposed before being passed to foobar, xpp will be wrong
That link does not seem to be working any longer.
2
#include <stdio.h>

void test(double (*in_array)[3], int N){
    int i, j;

    for(i = 0; i < N; i++){
        for(j = 0; j < N; j++){
            printf("%e \t", in_array[i][j]);
        }
        printf("\n");
    }
}

int main(void)
{
    double a[][3] = {
        {1., 2., 3.},
        {4., 5., 6.},
        {7., 8., 9.},
    };

    test(a, 3);
    return 0;
}

if you want to use a double ** in your function, you must pass an array of pointer to double (not a 2d array):

#include <stdio.h>

void test(double **in_array, int N){
    int i, j;

    for(i = 0; i < N; i++){
        for(j = 0; j< N; j++){
            printf("%e \t", in_array[i][j]);
        }
        printf("\n");
    }
}

int main(void)
{
    double a[][3] = {
        {1., 2., 3.},
        {4., 5., 6.},
        {7., 8., 9.},
    };
    double *p[] = {a[0], a[1], a[2]};

    test(p, 3);
    return 0;
}

Another (as suggested by @eryksun): pass a single pointer and do some arithmetic to get the index:

#include <stdio.h>

void test(double *in_array, int N){
    int i, j;

    for(i = 0; i < N; i++){
        for(j = 0; j< N; j++){
            printf("%e \t", in_array[i * N + j]);
        }
        printf("\n");
    }
}

int main(void)
{
    double a[][3] = {
        {1., 2., 3.},
        {4., 5., 6.},
        {7., 8., 9.},
    };

    test(a[0], 3);
    return 0;
}

3 Comments

@DavidHeffernan, I don't know about the ctypes code part, I'm trying to explain why double ** isn't working when a 2d array is passed
The problem is that N is not fixed at runtime
@jrsm: if you really want the double ** approach, you can create the array of pointers like this: c_double_p = POINTER(c_double); in_array_ptrs = (c_double_p * len(in_array))(*(r.ctypes.data_as(c_double_p) for r in in_array)). I don't recommend this since it's inefficient.
2

While the reply might be rather late, I hope it could help other people with the same problem.

As numpy arrays are internally saved as 1d arrays, one can simply rebuild 2d shape in C. Here is a small MWE:

// libtest2d.c
#include <stdlib.h> // for malloc and free
#include <stdio.h>  // for printf

// create a 2d array from the 1d one
double ** convert2d(unsigned long len1, unsigned long len2, double * arr) {
    double ** ret_arr;

    // allocate the additional memory for the additional pointers
    ret_arr = (double **)malloc(sizeof(double*)*len1);

    // set the pointers to the correct address within the array
    for (int i = 0; i < len1; i++) {
        ret_arr[i] = &arr[i*len2];
    }

    // return the 2d-array
    return ret_arr;
}

// print the 2d array
void print_2d_list(unsigned long len1,
    unsigned long len2,
    double * list) {

    // call the 1d-to-2d-conversion function
    double ** list2d = convert2d(len1, len2, list);

    // print the array just to show it works
    for (unsigned long index1 = 0; index1 < len1; index1++) {
        for (unsigned long index2 = 0; index2 < len2; index2++) {
            printf("%1.1f ", list2d[index1][index2]);
        }
        printf("\n");
    }

    // free the pointers (only)
    free(list2d);
}

and

# test2d.py

import ctypes as ct
import numpy as np

libtest2d = ct.cdll.LoadLibrary("./libtest2d.so")
libtest2d.print_2d_list.argtypes = (ct.c_ulong, ct.c_ulong,
        np.ctypeslib.ndpointer(dtype=np.float64,
            ndim=2,
            flags='C_CONTIGUOUS'
            )
        )
libtest2d.print_2d_list.restype = None

arr2d = np.meshgrid(np.linspace(0, 1, 6), np.linspace(0, 1, 11))[0]

libtest2d.print_2d_list(arr2d.shape[0], arr2d.shape[1], arr2d)

If you compile the code with gcc -shared -fPIC libtest2d.c -o libtest2d.so and then run python test2d.py it should print the array.

I hope the example is more or less self-explaining. The idea is, that the shape is also given to the C-Code which then creates a double ** pointer for which the space for the additional pointers is reserved. And these then are then set to point to the correct part of the original array.

PS: I am rather a beginner in C so please comment if there are reasons not to do this.

Comments

0

Here i hava pass two 2d numpy array and print value of one array for the reference

you can use and write your own logic in cpp

cpp_function.cpp

compile it using : g++ -shared -fPIC cpp_function.cpp -o cpp_function.so

#include <iostream>
extern "C" {
void mult_matrix(double *a1, double *a2, size_t a1_h, size_t a1_w, 
                  size_t a2_h, size_t a2_w, int size)
{
    //std::cout << "a1_h & a1_w" << a1_h << a1_w << std::endl; 
    //std::cout << "a2_h & a2_w" << a2_h << a2_w << std::endl; 
    for (size_t i = 0; i < a1_h; i++) {
        for (size_t j = 0; j < a1_w; j++) {
            printf("%f ", a1[i * a1_h + j]);
        }
        printf("\n");
    }
    printf("\n");
  }

}

Python File main.py

import ctypes
import numpy
from time import time

libmatmult = ctypes.CDLL("./cpp_function.so")
ND_POINTER_1 = numpy.ctypeslib.ndpointer(dtype=numpy.float64, 
                                      ndim=2,
                                      flags="C")
ND_POINTER_2 = numpy.ctypeslib.ndpointer(dtype=numpy.float64, 
                                    ndim=2,
                                    flags="C")
libmatmult.mult_matrix.argtypes = [ND_POINTER_1, ND_POINTER_2, ctypes.c_size_t, ctypes.c_size_t]
# print("-->", ctypes.c_size_t)

def mult_matrix_cpp(a,b):
    shape = a.shape[0] * a.shape[1]
    libmatmult.mult_matrix.restype = None
    libmatmult.mult_matrix(a, b, *a.shape, *b.shape , a.shape[0] * a.shape[1])

size_a = (300,300)
size_b = size_a

a = numpy.random.uniform(low=1, high=255, size=size_a)
b = numpy.random.uniform(low=1, high=255, size=size_b)

t2 = time()
out_cpp = mult_matrix_cpp(a,b)
print("cpp time taken:{:.2f} ms".format((time() - t2) * 1000))
out_cpp = numpy.array(out_cpp).reshape(size_a[0], size_a[1])

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.