How to creating ctypes variables using python variables instead of hard-coded values?

Question

When creating ctypes variables, can one not pass values using python variables?

I have some code where I am calling a shared C library. If I pass the parameters to this C library using Method 1 (see below) things work well. But if I use Method 2, I get garbage. There are other parts to the code. But I have confirmed that when I replace Method 2 with Method 1, things work well. So something is wrong here.

If what I am doing in Method 2 is not valid, what is the alternative if I want to automate the process of running the code for different values of a given variable(s)?

Method 1 (This works well)

import ctypes as C


c_thresholds = (C.c_double * 4)()
for idx, value in enumerate(thresholds):
    c_thresholds[idx] = value

goodH = Good(C.c_char('H'), C.c_double(0.5), C.c_int(100), C.c_int(20))
goodL = Good(C.c_char('L'), C.c_double(0.5), C.c_int(75), C.c_int(20))

c_parameters = Params(
            var1 = C.c_int(100),
            var2 = C.c_int(4),
            var3 = C.c_int(5),
            var4 = C.c_int(5000),
            var5 = C.c_char_p("modelname"),
            var6 = C.c_double(0.5),
            var7 = C.c_double(90),
            var8 = c_thresholds,
            var9 = C.c_int(2),
            H = goodH,
            L = goodL
)

runsimulation(c_parameters)

Method 2 (This does not work, outputs garbage)

import ctypes as C

def create_cparams(var1, var2, var3, var4, var5, var6, var7, var8, var9):

    c_thresholds = (C.c_double * 4)()
    for idx, value in enumerate(var8):
        c_thresholds[idx] = value

    goodH = Good(C.c_char('H'), C.c_double(0.5), C.c_int(100), C.c_int(20))
    goodL = Good(C.c_char('L'), C.c_double(0.5), C.c_int(75), C.c_int(20))

    c_parameters = Params(
                var1 = C.c_int(var1),
                var2 = C.c_int(var2),
                var3 = C.c_int(var3),
                var4 = C.c_int(var4),
                var5 = C.c_char_p(var5),
                var6 = C.c_double(var6),
                var7 = C.c_double(var7),
                var8 = c_thresholds,
                var9 = C.c_int(var9),
                H = goodH,
                L = goodL
    )

    return c_parameters

# These are python variables
var1 = 100
var2 = 4
var3 = 5
var4 = 5000
var5 = "modelname"
var6 = 0.5
var7 = 90
var8 = [1, 0.9, 0.8, 0.7]
var9 = 2

# Calling the create_cparams function defined above
c_parameters = create_cparams(var1, var2, var3, var4, var5, var6, var7, var8, var9)
runsimulation(c_parameters)

In case it is helpful the Params class is given by (does not change across the two methods):

class Params(C.Structure):
    _fields_ = [
           ("var1", C.c_int),
           ("var2", C.c_int),
           ("var3", C.c_int),
           ("var4", C.c_int),
           ("var5", C.c_char_p ),
           ("var6", C.c_double),
           ("var7", C.c_double),
           ("var8", (C.c_double * 4) ),
           ("var9", C.c_int),
           ("H", Good),
           ("L", Good)
    ]

C function prototype

// runsimulation() function above calls this C function

void run_multiple_reps (struct params parameters, struct repdata *data,
                        int len_timepdsarr, int *timepdsarr);

// params struct on C side, which Params class duplicates

struct params
{
    int var1;
    int var2;
    int var3;
    int var4;
    char *var5;
    double var6;
    double var7;
    double var8[4];
    int var9;
    struct good H;
    struct good L;
};

The difference is that in Method 1 I am assigning var1 = C.c_int(100) as an argument to Params(), that is value 100 is hard-coded. In Method 2, I define var1 = 100 as a regular Python variable and then in the Params arguments I do, var1 = C.c_int(var1). That is, there are no hard-coded values in the argument to Params(). I will post the C function prototype and the struct definitions. Thank you for looking into this. — Curious2learn
– Curious2learn, Commented Sep 9, 2013 at 11:09
Sorry, I meant that methods 1 and 2 should produce the same result. Did you test bytearray(c_parameters1) == bytearray(c_parameters2) for the structs from the respective methods? — Eryk Sun
– Eryk Sun, Commented Sep 9, 2013 at 11:20
@eryksun Sorry for wasting your time. I realized that when I was passing parameters as variables (as opposed to hard-coding the values) to Params() I was using different values, which was causing the problem. That is, one of the variables had a different value and that was causing the problem. Anyhow, the good thing that came out of this is that I learnt that I don't need to explicitly do the conversion since Python does it for me. For learning that I am going to accept your answer. Also, I will look into the bytearray. I don't know what that is. — Curious2learn
– Curious2learn, Commented Sep 9, 2013 at 11:46
bytearray is a mutable byte string type -- not related to ctypes. It initializes using the buffer protocol, which ctypes data objects support. It's just a quick way to grab a copy of the buffer to compare it as a byte string. — Eryk Sun
– Eryk Sun, Commented Sep 9, 2013 at 11:58

Eryk Sun · Accepted Answer · 2013-09-09 04:25:26Z

1

The field attributes of a Structure are CField descriptor objects. A descriptor is like a Python property or like a __slots__ attribute, if you're familiar with either of those. A CField knows the data type of the field and its offset into the buffer. Each C data type has a associated get/set function that converts to and from Python objects. So generally you can assign a Python object directly to the field. For example:

thresholds = [1, 0.9, 0.8, 0.7]    

c_parameters = Params(
    var1 = 100,
    var2 = 4,
    var3 = 5,
    var4 = 5000,
    var5 = "modelname",
    var6 = 0.5,
    var7 = 90,
    var8 = (C.c_double * 4)(*thresholds),
    var9 = 2,
    H = Good('H', 0.5, 100, 20),
    L = Good('L', 0.5, 75, 20),
)

If ctypes needs to hold a reference to a Python object to keep it alive, the reference is stored in the _objects dict of the Structure. In this case the array in var8, for example, is simply copied into the buffer, so c_parameters doesn't need to hold a reference to the original.

answered Sep 9, 2013 at 4:25

Eryk Sun

34.5k7 gold badges97 silver badges113 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Curious2learn Over a year ago

Thanks eryksun. However, you are assigning hard-coded values (100, 4,...) to the variables. I am instead looking for a way where I can do var1=100, var2=20 where var1 and var2 are like regular Python variables, and then create c_parameters by including arguments such as var1 = var1, var2 = var2, where var1 and var2 are already defined. Although, I will try without converting the variables to ctypes myself, since (I think) you are saying it is unnecessary for me to do it manually.

Eryk Sun Over a year ago

Values in CPython are reference-counted objects. Assigning var1 = 100 just bumps the ob_refcnt on the 100 object. If the count goes to zero the object gets deallocated. A name/attribute is just a tag.

Collectives™ on Stack Overflow

How to creating ctypes variables using python variables instead of hard-coded values?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related