Class property using Python C-API

Question

What is the best way to create class properties (as here and here) using the Python C-API? Static properties would also work in my case.

Follow up:

I tried to implement yak's suggestion. I defined a class P with get and set functions in its tp_descr_get and tp_descr_set slots. Then I added an instance of P to the dictionary of the type object for the class X under the key p. Then

x1 = X()
x2 = X()
x1.p = 10
print x1.p
print x2.p
print X.p
x2.p = 11
print x1.p
print x2.p
print X.p

works (first 10 is printed three times, then 11 is printed three times), but

X.p = 12

fails with the error message

TypeError: can't set attributes of built-in/extension type 'X'

How do I fix that?

Follow up 2:

If I allocate the type object with PyMem_Malloc and set the Py_TPFLAGS_HEAPTYPE flag, then everything works ; I can do X.p = 12 with the expected result.

Things also work if I keep the type object in a static variable and set the Py_TPFLAGS_HEAPTYPE flag, but that is obviously not a good idea. (But why does it matter whether the type object is in static or dynamic memory? I never let its reference count drop to 0 anyway.)

The restriction that you can only set attributes on dynamic types seems very strange. What is the rationale behind this?

Follow up 3:

No, it does not work. If I make the type X dynamic, then X.p = 12 does not set the property X.p to twelve; it actually binds the object 12 to the name X.p. In other words, afterwards, X.p is not an integer-valued property but an integer.

Follow up 4:

Here is the C++ code for the extension:

#include <python.h>
#include <exception>

class ErrorAlreadySet : public std::exception {};

// P type ------------------------------------------------------------------

struct P : PyObject
{
    PyObject* value;
};

PyObject* P_get(P* self, PyObject* /*obj*/, PyObject* /*type*/)
{
    Py_XINCREF(self->value);
    return self->value;
}

int P_set(P* self, PyObject* /*obj*/, PyObject* value)
{
    Py_XDECREF(self->value);
    self->value = value;
    Py_XINCREF(self->value);
    return 0;
}

struct P_Type : PyTypeObject
{
    P_Type()
    {
        memset(this, 0, sizeof(*this));
        ob_refcnt = 1;
        tp_name = "P";
        tp_basicsize = sizeof(P);
        tp_descr_get = (descrgetfunc)P_get;
        tp_descr_set = (descrsetfunc)P_set;
        tp_flags = Py_TPFLAGS_DEFAULT;

        if(PyType_Ready(this)) throw ErrorAlreadySet();
    }
};

PyTypeObject* P_type()
{
    static P_Type typeObj;
    return &typeObj;
}


// P singleton instance ----------------------------------------------------

P* createP()
{
    P* p_ = PyObject_New(P, P_type());
    p_->value = Py_None;
    Py_INCREF(p_->value);
    return p_;
}

P* p()
{
    static P* p_ = createP();
    Py_INCREF(p_);
    return p_;
}

PyObject* p_value()
{
    PyObject* p_ = p();
    PyObject* value = p()->value;
    Py_DECREF(p_);
    Py_INCREF(value);
    return value;
}


// X type ------------------------------------------------------------------

struct X : PyObject {};

void X_dealloc(PyObject* self)
{
    self->ob_type->tp_free(self);
}

struct X_Type : PyTypeObject
{
    X_Type()
    {
        memset(this, 0, sizeof(*this));
        ob_refcnt = 1;
        tp_name = "M.X";
        tp_basicsize = sizeof(X);
        tp_dealloc = (destructor)X_dealloc;
        tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HEAPTYPE;

        tp_dict = PyDict_New();
        PyObject* key = PyString_FromString("p");
        PyObject* value = p();
        PyDict_SetItem(tp_dict, key, value);
        Py_DECREF(key);
        Py_DECREF(value);

        if(PyType_Ready(this)) throw ErrorAlreadySet();
    }

    void* operator new(size_t n) { return PyMem_Malloc(n); }
    void operator delete(void* p) { PyMem_Free(p); }
};

PyTypeObject* X_type()
{
    static PyTypeObject* typeObj = new X_Type;
    return typeObj;
}

// module M ----------------------------------------------------------------

PyMethodDef methods[] = 
{
    {"p_value", (PyCFunction)p_value, METH_NOARGS, 0},
    {0, 0, 0, 0}
};

PyMODINIT_FUNC
initM(void)
{
    try {
        PyObject* m = Py_InitModule3("M", methods, 0);
        if(!m) return;
        PyModule_AddObject(m, "X", (PyObject*)X_type());
    }
    catch(const ErrorAlreadySet&) {}
}

This code defines a module M with a class X with a class property p as described before. I have also added a function p_value() that lets you directly inspect the object that implements the property.

Here is the script I have used to test the extension:

from M import X, p_value

x1 = X()
x2 = X()

x1.p = 1
print x1.p
print x2.p
print X.p
print p_value()
print

x2.p = 2
print x1.p
print x2.p
print X.p
print p_value()
print

X.p = 3
print x1.p
print x2.p
print X.p
print p_value()     # prints 2
print

x1.p = 4       # AttributeError: 'M.X' object attribute 'p' is read-only

Check this for an explanation by Guido: mail.python.org/pipermail/python-dev/2008-February/077167.html — yak
– yak, Commented Apr 16, 2012 at 13:09

Johan Råde · Accepted Answer · 2012-04-16 11:15:56Z

6

Similar to these Python solutions, you will have to create a classproperty type in C and implement its tp_descr_get function (which corresponds to __get__ in Python).

Then, if you want to use that in a C type, you would have to create an instance of your classproperty type and insert it into dictionary of your type (tp_dict slot of your type).

Follow up:

It would seem that it's impossible to set an attribute of a C type. The tp_setattro function of the metaclass (PyType_Type) raises a "can't set attributes of built-in/extension type" exception for all non-heap types (types with no Py_TPFLAGS_HEAPTYPE flag). This flag is set for dynamic types. You could make your type dynamic but it might be more work then it's worth.

This means that the solution I gave initially allows you to create a property (as in: computed attribute) on a C type object with the limitation that it's read only. For setting you could use a class/static-method (METH_CLASS/METH_STATIC flag on a method in tp_methods).

edited Apr 16, 2012 at 11:15

Johan Råde

21.7k22 gold badges76 silver badges112 bronze badges

answered Apr 16, 2012 at 0:06

yak

9,1012 gold badges34 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Johan Råde Over a year ago

Thanks! I tried it and it almost works; see the edited question.

Johan Råde Over a year ago

Thanks! I tried using dynamic types and it works; see the edited question.

yak Over a year ago

Can you post what you have so far?

yak Over a year ago

Turns out that tp_descr_set isn't called when a type attribute is set, it works only on instances. A workaround is to move the descriptor to metatype. Your type will be an instance of the metatype so descriptor will work. You would set the metatype as ob_type of X. All this applies to Python as well, see this (python.6.n6.nabble.com/…) for example.

yak Over a year ago

If getting/setting p on an instance of X rather than on the type itself is an option, then things become much simpler because you can use tp_getset.

|

Demitri · Accepted Answer · 2019-12-16 16:44:46Z

4

I'll try to convey the essence of what I've discovered about using class static properties.

My (edited) code is as follows:

// Prepare your type object, which you will no doubt complete 
// by calling PyType_Ready, as here.
if (PyType_Ready(typeObj) < 0)
{
  return;
}

Py_INCREF(typeObj);
PyModule_AddObject(module, typeName, (PyObject*) typeObj);

// Now we add static members directly to the type's tp_dict, but 
// only *after* we've registered the type (above)
PyObject* dict = typeObj->tp_dict;

// You'll have your own wrapper / C values in the line below. This is just
// a pseudo-version of what I am (successfully) using.
PyObject* tmp = MyCreateWrapper(myStaticValueInC);

// Py_INCREF(tmp); // You may need this, depending on how line above works.

PyDict_SetItemString(dict, staticPropertyName, tmp);

Py_DECREF(tmp);

I believe all the essentials are here in terms of what order to construct your code in order to implement a class property.

edited Dec 16, 2019 at 16:44

Demitri

14.2k4 gold badges44 silver badges43 bronze badges

answered Sep 14, 2017 at 12:30

user1738833

3393 silver badges11 bronze badges

1 Comment

Josh Haberman Over a year ago

Modifying tp_dict in this way appears to be prohibited: "Warning It is not safe to use PyDict_SetItem() on or otherwise modify tp_dict with the dictionary C-API."

André Anjos · Accepted Answer · 2014-04-23 16:37:48Z

1

If that is an acceptable solution, you may create a method on the module holding the declaration of X, that simply sets the class variable as you wish. For example:

PyObject* set_p_value(PyObject*, PyObject* o) {
  if(PyDict_SetItemString(X_type()->tp_dict, "p", o) == -1) return 0;
  Py_RETURN_NONE; 
}

PyMethodDef methods[] = 
{
    ...
    {"set_p_value", (PyCFunction)set_p_value, METH_O, 0},
    {0, 0, 0, 0}
};

Once that is there, than:

from M import X, set_p_value
set_p_value(3)
print X.p #should print '3'

Should work as expected. A draw-back is that, unfortunately, this functionality is unrelated to the type object itself. This could be partially circumvented if you provided a class method that sets the class variable as you wish.

answered Apr 23, 2014 at 16:37

André Anjos

5,4692 gold badges30 silver badges36 bronze badges

1 Comment

Josh Haberman Over a year ago

Modifying tp_dict in this way appears to be prohibited: "Warning It is not safe to use PyDict_SetItem() on or otherwise modify tp_dict with the dictionary C-API."

Forty-Bot · Accepted Answer · 2022-01-14 07:04:59Z

0

You can add items to tp_dict as long as it happens before you call PyType_Ready:

int err;
PyObject *p;

if (!X_Type.tp_dict) {
    X_Type.tp_dict = PyDict_New();
    if (!X_Type.tp_dict)
        return NULL;
}

p = PyLong_FromLong(12);
if (!p)
    return NULL;

err = PyDict_SetItemString(X_Type.tp_dict, "p", p)
Py_DECREF(p);
if (err)
    return NULL;

m = PyModule_Create(&M_module);
if (!m)
    return NULL;

if (PyModule_AddType(m, &X_type));
    return NULL;

return m;

edited Jan 14, 2022 at 7:04

answered Jan 12, 2022 at 23:26

Forty-Bot

1342 silver badges8 bronze badges

Collectives™ on Stack Overflow

Class property using Python C-API

4 Answers 4

6 Comments

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

6 Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related