Exception: access violation reading when calling a C function from Python

Question

I wanted to play around a bit with the Python C Api. But ran into an error.

OSError: exception: access violation writing 0x0000000000000020

The error occurs on the lines PyObject_RichCompare(first, second, Py_LT)

There are no errors in the first and second variables. If you remove the line PyObject_RichCompare(first, second, Py_LT) everything works.

Tried building "DLL" both on Linux in GCC and on Windows in Visual Studio. Everywhere I get this error

C code

#include "Python.h"

extern "C" {
    __declspec(dllexport)
    long test(PyObject* list) {
        PyObject* first, * second;
        int t = PyList_Size(list);
        first = PyList_GetItem(list, 0);
        second = PyList_GetItem(list, 1);
        PyObject* result = PyObject_RichCompare(first, second, Py_LT);
        return PyLong_AsLong(first) + PyLong_AsLong(second);
    }
}

And Python Code

import ctypes

lib = ctypes.CDLL('please.dll')
lib.test.restype = ctypes.c_long
lib.test.argtypes = [ctypes.py_object]

py_values = [1, 645, 546, 8646, 45646, 6545688, 5465]
a = lib.test(py_values)

print(a)

When you are calling code that uses the Python C API, you must load it as a PyDLL, not CDLL. As a CDLL it releases the GIL, which makes basically any interaction with any Python level object inherently unsafe. Probably not the cause of your problem, but things will eventually explode if a thread is introduced into your program (even if this library is only used by a single thread, reference counts can get out of sync due to the GIL holding thread and the thread calling this library racing). — ShadowRanger
– ShadowRanger, Commented Sep 15, 2022 at 23:35
Why is the broken line there at all? PyObject* result = PyObject_RichCompare(first, second, Py_LT); leaks a reference to a bool and don't even use the result... — ShadowRanger
– ShadowRanger, Commented Sep 15, 2022 at 23:37

Mark Tolonen · Accepted Answer · 2022-09-15 23:35:49Z

2

The GIL must be held to use Python C APIs. Use PyDLL for that:

lib = ctypes.PyDLL('please.dll')

answered Sep 15, 2022 at 23:35

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

ShadowRanger Over a year ago

While this is true, I don't think it will change anything in a single-threaded program, the problem is likely not related to it (though they should absolutely switch to using PyDLL as well).

Mark Tolonen Over a year ago

@ShadowRanger I tested the change. The result prints correctly (646, the addition of the first two elements of the list).

ShadowRanger Over a year ago

Awesome, that's enough for me. I knew it was bad form, but I guess the mechanics of going through all the rich comparison code paths eventually do stuff that trigger bugs when they're hit without the GIL.

Mark Tolonen Over a year ago

@ShadowRanger And just for completeness I'll note that I reproduced the crash with CDLL. Undefined behavior is undefined behavior after all 😉

Collectives™ on Stack Overflow

Exception: access violation reading when calling a C function from Python

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related