Calling Python 2 script from Python 3

Calling different python versions from each other can be done very elegantly using execnet. The following function does the charm:

import execnet

def call_python_version(Version, Module, Function, ArgumentList):
    gw      = execnet.makegateway("popen//python=python%s" % Version)
    channel = gw.remote_exec("""
        from %s import %s as the_function
        channel.send(the_function(*channel.receive()))
    """ % (Module, Function))
    channel.send(ArgumentList)
    return channel.receive()

Example: A my_module.py written in Python 2.7:

def my_function(X, Y): 
    return "Hello %s %s!" % (X, Y)

Then the following function calls

result = call_python_version("2.7", "my_module", "my_function",  
                             ["Mr", "Bear"]) 
print(result) 
result = call_python_version("2.7", "my_module", "my_function",  
                             ["Mrs", "Wolf"]) 
print(result)

result in

Hello Mr Bear!
Hello Mrs Wolf!

What happened is that a 'gateway' was instantiated waiting for an argument list with channel.receive(). Once it came in, it as been translated and passed to my_function. my_function returned the string it generated and channel.send(...) sent the string back. On other side of the gateway channel.receive() catches that result and returns it to the caller. The caller finally prints the string as produced by my_function in the python 3 module.

You could run python2 using subprocess (python module) doing the following:

From python 3:

#!/usr/bin/env python3
import subprocess

python3_command = "py2file.py arg1 arg2"  # launch your python2 script

process = subprocess.Popen(python3_command.split(), stdout=subprocess.PIPE)
output, error = process.communicate()  # receive output from the python2 script

Where output stores whatever python 2 returned

Maybe to late, but there is one more simple option for call python2.7 scripts:

script = ["python2.7", "script.py", "arg1"]    
process = subprocess.Popen(" ".join(script),
                                        shell=True,  
                                        env={"PYTHONPATH": "."})

I am running my python code with python 3, but I need a tool (ocropus) that is written with python 2.7. I spent a long time trying all these options with subprocess, and kept having errors, and the script would not complete. From the command line, it runs just fine. So I finally tried something simple that worked, but that I had not found in my searches online. I put the ocropus command inside a bash script:

#!/bin/bash

/usr/local/bin/ocropus-gpageseg $1

I call the bash script with subprocess.

command = [ocropus_gpageseg_path,  current_path]
process = subprocess.Popen(command,shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
output, error = process.communicate()
print('output',output,'error',error)

This really gives the ocropus script its own little world, which it seems to need. I am posting this in the hope that it will save someone else some time.

It works for me if I call the python 2 executable directly from a python 3 environment.

python2_command = 'C:\Python27\python.exe python2_script.py arg1'
process = subprocess.Popen(python2_command.split(), stdout=subprocess.PIPE)
output, error = process.communicate()

python3_command = 'python python3_script.py arg1'
process = subprocess.Popen(python3_command.split(), stdout=subprocess.PIPE)
output, error = process.communicate()

I ended up creating a new function in the python3 script, which wraps the python2.7 code. It correctly formats error messages created by the python2.7 code and is extending mikelsr's answer and using run() as recommended by subprocess docs.

in bar.py (python2.7 code):

def foo27(input):
    return input * 2

in your python3 file:

import ast
import subprocess

def foo3(parameter):
    try:
        return ast.literal_eval(subprocess.run(
            [
                "C:/path/to/python2.7/python.exe", "-c", # run python2.7 in command mode
                "from bar import foo27;"+
                "print(foo27({}))".format(parameter) # print the output 
            ],
            capture_output=True,
            check=True
        ).stdout.decode("utf-8")) # evaluate the printed output
    except subprocess.CalledProcessError as e:
        print(e.stdout)
        raise Exception("foo27 errored with message below:\n\n{}"
                                .format(e.stderr.decode("utf-8")))

print(foo3(21))
# 42

This works when passing in simple python objects, like dicts, as the parameter but does not work for objects created by classes, eg. numpy arrays. These have to be serialized and re-instantiated on the other side of the barrier.

Note: This was happening when running my python 2.x s/w in the liclipse IDE. When I ran it from a bash script on the command line it didn't have the problem. Here is a problem & solution I had when mixing python 2.x & 3.x scripts.

I am running a python 2.6 process & needed to call/execute a python 3.6 script. The environment variable PYTHONPATH was set to point to 2.6 python s/w, so it was choking on the followng:

File "/usr/lib64/python2.6/encodings/__init__.py", line 123
raise CodecRegistryError,\

This caused the 3.6 python script to fail. So instead of calling the 3.6 program directly I created a bash script which nuked the PYTHONPATH environment variable.

#!/bin/bash
export PYTHONPATH=
## Now call the 3.6 python scrtipt
./36psrc/rpiapi/RPiAPI.py $1

Instead of calling them in python 3, you could run them in conda env batch by creating a batch file as below:

call C:\ProgramData\AnacondaNew\Scripts\activate.bat

C:\Python27\python.exe "script27.py" C:\ProgramData\AnacondaNew\python.exe "script3.py"

call conda deactivate

pause

I recommend to convert the Python2 files to Python3:

https://pythonconverter.com/