Importing tensorflow when embedding python in c++ returns null

Question

I've a question regarding embedding python into a C++ application. The setup is as follows: I have a large C++ application which generates some data (renders images in real-time) and displays them. I have also trained a neural network in python using tensorflow which will accept these images.

My idea was to embed python and send data as a numpy array, predict using the neural network and get back another processed numpy array to display (in C++). I've made some basic tests without tensorflow on python side to get a feel for embedding python in c and such and it seems to work.

However, once i place "import tensorflow" into any python script that I want to import, I'll get a NULL from PyImport_ImportModule in c++ part.

e.g.

import numpy as np
def foo(img):
    return np.clip(img * 2.0, 0, 255).astype(np.uint8)

works fine. But the following doesn't:

import numpy as np
import tensorflow as tf #this causes the fail

def foo(img):
    return np.clip(img * 2.0, 0, 255).astype(np.uint8)

In the second case, I still get the message in stdout from tensorflow that it has found cuda etc, but then the module import fails.

My setup is on Windows 10 x64, Anaconda Python 3.5, tensorflow-0.12 and CUDA 8. Has anyone ran across a similar problem? Other modules that I've tested (numpy, pil, scipy) seem to load fine.

If it looks like it cannot be resolved, I'll resort to IPC of some sort between the c++ part and python.

Nemanja Bartolovic · Accepted Answer · 2017-02-12 01:43:01Z

4

I've resolved the issue. I needed to set argc and argv with PySys_SetArgv. I've uncovered this using PyErr_Occurred() and PyErr_Print() right after the failed import to see the problem.

answered Feb 12, 2017 at 1:43

Nemanja Bartolovic

1014 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Zhenjie Zhao Over a year ago

Sorry that I resolved it. I have posted the whole code for other people may find it useful.

Zhenjie Zhao · Accepted Answer · 2017-03-25 09:30:21Z

ref: https://docs.python.org/3.5/extending/embedding.html

main.cpp

#include <Python.h>
#include <iostream>
#include <QString>
#include <QDir>
#include <cstring>

using namespace std;

int main(int argc, char *argv[])
{
    PyObject *pName, *pModule, *pDict, *pFunc;
    PyObject *pArgs, *pValue;
    int i;

    if (argc < 3) {
        fprintf(stderr,"Usage: call pythonfile funcname [args]\n");
        return 1;
    }

    Py_SetProgramName((wchar_t*)L"test");

    Py_Initialize();

    PySys_SetArgv(argc, (wchar_t**)argv);
    PyRun_SimpleString("import tensorflow as tf\n"
                       "print(tf.__version__)\n");

    PyRun_SimpleString("import cv2\n"
                       "print(cv2.__version__)\n");

    QString qs = QDir::currentPath();
    std::wstring ws = qs.toStdWString();
    PySys_SetPath(ws.data());
    pName = PyUnicode_DecodeFSDefault(argv[1]);
    /* Error checking of pName left out */

    pModule = PyImport_Import(pName);
    Py_DECREF(pName);

    if (pModule != NULL) {
        pFunc = PyObject_GetAttrString(pModule, argv[2]);
        /* pFunc is a new reference */

        if (pFunc && PyCallable_Check(pFunc)) {
            pArgs = PyTuple_New(argc - 3);
            for (i = 0; i < argc - 3; ++i) {
                pValue = PyLong_FromLong(atoi(argv[i + 3]));
                if (!pValue) {
                    Py_DECREF(pArgs);
                    Py_DECREF(pModule);
                    fprintf(stderr, "Cannot convert argument\n");
                    return 1;
                }
                /* pValue reference stolen here: */
                PyTuple_SetItem(pArgs, i, pValue);
            }
            pValue = PyObject_CallObject(pFunc, pArgs);
            Py_DECREF(pArgs);
            if (pValue != NULL) {
                printf("Result of call: %ld\n", PyLong_AsLong(pValue));
                Py_DECREF(pValue);
            }
            else {
                Py_DECREF(pFunc);
                Py_DECREF(pModule);
                PyErr_Print();
                fprintf(stderr,"Call failed\n");
                return 1;
            }
        }
        else {
            if (PyErr_Occurred())
                PyErr_Print();
            fprintf(stderr, "Cannot find function \"%s\"\n", argv[2]);
        }
        Py_XDECREF(pFunc);
        Py_DECREF(pModule);
    }
    else {
        PyErr_Print();
        fprintf(stderr, "Failed to load \"%s\"\n", argv[1]);
        return 1;
    }
    Py_Finalize();
    return 0;
}

multiply.py

import tensorflow as tf
import cv2

def multiply(a,b):
    print(tf.__version__) 
    print(cv2.__version__) 
    print("Will compute", a, "times", b)
    c = 0
    for i in range(0, a):
        c = c + b
    return c

Collectives™ on Stack Overflow

Importing tensorflow when embedding python in c++ returns null

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related