Call python script from c++ without loading the file each time

Question

I have a python script that I need to call lots of times (around 160000 times) and while I can do this in less that a second with hard C++ code, if I try to load and call a python script to do this, it will probably take hours! I think that if I loaded the script once, then run the loaded script over and over, it will be significantly faster. Unfortunately, I don't know how to do this. I don't think I can load the file with ifstream, then use PyRun_SimpleString on all the lines of the string. In case it isn't faster however, is it possible to return a 2D array in python, then convert that Array into a std::vector?

How are the inputs to your script determined? Your suggestion of using PyRun_SimpleString implies that you are not passing any arguments to the script from your C++ code. If that's not the case, what is changing between invocations of your script? — Nasser Al-Shawwa
– Nasser Al-Shawwa, Commented Jul 8, 2015 at 12:27
@Nasser The script does have inputs, which are parameters into a function. I call the function with PyObject_CallFunction. The only reason I mentioned PyRun_SimpleString was so people didn't assume I've done no research. — Ben Hollier
– Ben Hollier, Commented Jul 8, 2015 at 12:31

Nasser Al-Shawwa · Accepted Answer · 2015-07-08 12:59:12Z

5

Consider the following function, in a file called multiply.py

#!/usr/bin/python3

def do_multiply(a, b):
    c = 0
    for i in range(0, a):
        c += b
    return c

In your C++ file:

// Load the function 
PyObject *pName     = PyUnicode_FromString("multiply");
PyObject *pModule   = PyImport_Import(pName);
PyObject *pFunction = PyObject_GetAttrString(pModule, "do_multiply")

Let's say you want to call this function 3 times, as follows:

struct NumberPair 
{ 
    int x;
    int y;
};

std::vector<NumberPair> numberPairs { {1, 2}, {5, 6}, {10, 12} };

Then, you can simply call PyObject_CallFunction several times, while the module is loaded:

for(const auto &numberPair : numberPairs)
{
    PyObject *product = PyObject_CallFunction(pFunction, "ii", numberPair.x, numberPair.y);
    if(product != NULL)
    {
        std::cout << "Product is " << PyLong_AsLong(product) << '\n';
        Py_DECREF(product);
    }
}

There is no need to close the module between calls to PyObject_CallFunction, so that shouldn't be a problem.

edited Jul 8, 2015 at 12:59

answered Jul 8, 2015 at 12:51

Nasser Al-Shawwa

3,69321 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ben Hollier Over a year ago

I imagine this answer was very obvious, and I look like an idiot. It wouldn't be the first time I over thought something to do with programming, thank you for the answer!

Nasser Al-Shawwa Over a year ago

Anytime! Please note the edit I made for de-allocating memory where necessary in the loop.

Collectives™ on Stack Overflow

Call python script from c++ without loading the file each time

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related