Incref of params to PyEval_EvalCode depending on script

Question

In the following MCVE two different scripts get executed. Both don't do anything special, the first one has an empty function, the second script does literally nothing. But depending on the script PyEval_EvalCode increfs the passed global/local dictionary or not. The documentation doesn't say anything. But depending on the script I have a dangling dictionary after it got executed. When do I have to decref them afterwards or when not? Or what am I missing here? The following C snippet outputs the refcount of the passed dictionaries.

I tried this on Windows with the standard Python-2.7 interpreter.

#pragma comment(lib, "C:\\Python27\\libs\\python27.lib")
#include "C:\\Python27\\include\\Python.h"

static int execute(const char* script)
{
    PyObject* globals = PyDict_New();
    PyDict_SetItemString(globals, "__builtins__", PyEval_GetBuiltins());

    PyObject* code = Py_CompileString(script, "test", Py_file_input);
    if (!code)
    {
        PyErr_Print();
        return 0;
    }

    PyObject* result = PyEval_EvalCode((PyCodeObject*)code, globals, nullptr);
    Py_DECREF(code);

    if (!result)
    {
        PyErr_Print();
        return 0;
    }

    Py_DECREF(result);
    printf("Refcount of globals: %zd\n", globals->ob_refcnt);
    Py_DECREF(globals); // missing decref spotted by user2357112

    return 0;
}

int main()
{
    const char* script = nullptr;

    Py_Initialize();

    // First script contains a function
    script = 
        "def main():\n" \
        "    pass\n" \
        "main()\n";
    execute(script);

    // second script does nothing
    script = "12345";
    execute(script);

    Py_Finalize();
    return 0;
}

user2357112 · Accepted Answer · 2018-02-06 23:05:06Z

2

The extra reference is the function's own reference to its global variable dict. This reference is not your concern, and you do not need to decref that reference.

You only need to perform decrefs to account for clearing references you own.

answered Feb 6, 2018 at 23:05

user2357112

286k32 gold badges490 silver badges570 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

HelloWorld Over a year ago

Thanks, great confirmation, I just saw the assignment in PyFunction_New(). But when do they get dereferenced? My problem, all objects in the global scope are still alive, although the scope reached the end of it's lifetime.

user2357112 Over a year ago

@HelloWorld: Python doesn't promise it'll happen at any particular time. For this example, it'll probably happen during Py_Finalize.

HelloWorld Over a year ago

I just tried it out with a data breakpoint, even in Py_Finalize the reference count doesn't get touched anymore

user2357112 Over a year ago

@HelloWorld: You never decref'd your own reference to the dict!

HelloWorld Over a year ago

Thanks for spotting, deleted it by accident while creating the MCVE

HelloWorld · Accepted Answer · 2018-02-07 01:46:57Z

1

After tracing everything down I realized there is a circular reference. The function gets a reference to the globals dictionary, and the dictionary keeps the function alive. Calling the garbage collector right after the script got executed fixes the problem and the circular reference gets resolved.

execute(script);
PyGC_Collect();

P.S. Credit goes also to user2357112 who spotted a missing decref in my MCVE.

answered Feb 7, 2018 at 1:46

HelloWorld

2,0144 gold badges40 silver badges91 bronze badges

Collectives™ on Stack Overflow

Incref of params to PyEval_EvalCode depending on script

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related