Embedding Python to C++ Segmentation fault

Question

I am trying to track the execution of python scripts with C++ Threads (If anyone knows a better approach, feel free to mention it)

This is the code I have so far.

#define PY_SSIZE_T_CLEAN
#include </usr/include/python3.8/Python.h>
#include <iostream>
#include <thread>

void launchScript(const char *filename){
    Py_Initialize();
    FILE *fd = fopen(filename, "r");
    PyRun_SimpleFile(fd, filename);
    PyErr_Print();
    Py_Finalize();
 

}
int main(int argc, char const *argv[])
{
    Py_Initialize();
    PyRun_SimpleString("import sys");
    PyRun_SimpleString("sys.path.append(\".\")");
    std::thread first (launchScript,"script.py");
    std::cout << "Thread 1 is running with thread ID: " << first.get_id() << std::endl;
    std::thread second (launchScript,"script2.py");
    std::cout << "Thread 2 is running with thread ID: " << second.get_id() << std::endl;
   
    first.join();
    second.join();
    Py_Finalize();
    return 0; 
   
}

Script.py just has a print statement that prints "Hello World" Script2.py has a print statement that prints "Goodbye World"

I build the application with the following commands

g++ -pthread -I/usr/include/python3.8/ main.cpp -L/usr/lib/python3.8/config-3.8-x86_64 linux-gnu -lpython3.8 -o output

When I run ./output, I receive the following on my terminal

Thread 1 is running with thread ID: 140594340370176
Thread 2 is running with thread ID: 140594331977472
GoodBye World
./build.sh: line 2:  7864 Segmentation fault      (core dumped) ./output

I am wondering why I am getting Segmentation Fault. I have tried to debug with PyErr_Print(); but that has not given me any clues.

Any feed back is appreciated.

You're calling Py_Initialize and Py_Finalizemultiple times — perivesta
– perivesta, Commented May 24, 2022 at 13:25
I was under the impression that the Python engine had global state, which isn't thread safe, nor even allows more than one Python "context" to run concurrently. In contrast to the Lua engine which has the engine state stored in a "context" variable, so there can be multiple Lua scripts running concurrently or interleaved. (I may be mistaken. Or things may have changed since I had picked up that bit of information.) — Eljay
– Eljay, Commented May 24, 2022 at 13:25
I removed Py_Initialize and Py_Finalize from the function, but It is still inconsistent, sometimes it gives me the segmentation fault error, sometimes both scripts run successfully with no segmentation fault, sometimes only one script runs successfully. — scriptKiddie123
– scriptKiddie123, Commented May 24, 2022 at 13:30
@Eljay I don't believe that things have changed. I remember having read somewhere things might change with Python 4. However, calling the Python interpreter from distinct threads is a bad idea for sure. — Scheff's Cat
– Scheff's Cat, Commented May 24, 2022 at 13:38
FYI: From Python doc. - threading: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). — Scheff's Cat
– Scheff's Cat, Commented May 24, 2022 at 13:44

Richard · Accepted Answer · 2022-05-24 14:59:04Z

After testing and debugging the program for about 20 minutes I found that the problem is caused because in your example you've created the second std::thread named second before calling join() on the first thread.

Thus, to solve this just make sure that you've used first.join() before creating the second thread as shown below:

int main(int argc, char const *argv[])
{
    Py_Initialize();
    PyRun_SimpleString("import sys");
    PyRun_SimpleString("sys.path.append(\".\")");
    std::thread first (launchScript,"script.py");
    std::cout << "Thread 1 is running with thread ID: " << first.get_id() << std::endl;
//--vvvvvvvvvvvvv-------->call join on first thread before creating the second std::thread
    first.join();
    
    std::thread second (launchScript,"script2.py");
    std::cout << "Thread 2 is running with thread ID: " << second.get_id() << std::endl;
   
    second.join();
 
    Py_Finalize();
    return 0; 
   
}

Collectives™ on Stack Overflow

Embedding Python to C++ Segmentation fault

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related