Run a dynamic multi-line string of code in python at runtime

Question

How can I run a multi-line string of code from inside python, additionally the string is generated at runtime and stored in either a string or an array.

Background

I have a function which dynamically generates random code snippets in high volume.
I wish to pass this code to a variable/string and evaluate syntax/return code.
Code can not be imported from file/api request etc as focus is on high throughput and speed.

Example

# my function returns random code snippets
myCode = myRandomCodeGenerator()

# I wish to perform some in fight validation (ideally RC or syntax)
someTestingFunction(myCode)

My attempts so far

I have seen solutions such as the one below, however since my code is generated dynamically I have issues formatting. I have tried generating \n and \r or adding """ in the string to compensate with no luck.

code = """
def multiply(x,y):
    return x*y

print('Multiply of 2 and 3 is: ',multiply(2,3))
"""

exec(code)

I have also tried using the call function on a string yields similar formatting issues.

So far the best I can do is to perform a line by line syntax check and feed it a line at a time as shown below.

import codeop
def is_valid_code(line):
    try:
        codeop.compile_command(line)
    except SyntaxError:
        return False
    else:
        return True

I suspect there may be some syntax tricks I could apply to preserve formatting of indentation and returns. I am also aware of the risk of generating dynamic code in runtime, and have a filter of allowed terms in the function.

how do you expect it to do anything if you cant get your spacing formatted correctly ... ? spacing is very important to python... i think this is an XY problem... where you are doing X to solve Y, but there is a much better path to Y... also whatever safeties you think you have in place... if this is user input data you are gonna get owned if someone really wants to — Joran Beasley
– Joran Beasley, Commented May 23, 2020 at 22:22
I agree, is there a way to preserve spacing/indentation when generated dynamically? The exec example i listed works fine when input manually. — Murchie85
– Murchie85, Commented May 23, 2020 at 22:25
exec("def multiply(x,y):\n return x*y\nprint('Multiply of 2 and 3 is: ',multiply(2,3))\n") is the equivelent of the triple quoted string above, without triple quotes — Joran Beasley
– Joran Beasley, Commented May 23, 2020 at 22:34
Thank you Joran, I see where my code was failing; an automatic space was added after every \n as a safety margin. Even in the example above it fails if the print statement looks like this return x*y\n print('Multiply of 2 and 3 is: Much appreciated and I see both the reason and that this is now possible - thanks again. — Murchie85
– Murchie85, Commented May 23, 2020 at 22:42
I did not see the obvious uin the above comments - or in the answer - but you are aware that using the \n special sequence in a code string will generate a newline character, don't you? Adding a \n to the end of a line is all that is needed for a multiline script. — jsbueno
– jsbueno, Commented May 23, 2020 at 22:50

oak · Accepted Answer · 2020-10-01 06:58:13Z

Another way instead of eval and exec is to compile the code before and then to exec it:

Example:

import contextlib,sys
from io import StringIO 
@contextlib.contextmanager
def stdoutIO(stdout=None):
    old = sys.stdout
    if stdout is None:
        stdout = StringIO()
    sys.stdout = stdout
    yield stdout
    sys.stdout = old


def run_code(override_kale_blocks):
    compiled123 = []
    for b123 in override_kale_blocks:
        compiled123.append(compile(b123,"<string>","exec"))
    
    with stdoutIO() as s:
        for c123 in compiled123:
            exec(c123)
    return s.getvalue()
    


block0='''
import time
a=5
b=6
b=a+b
'''
block1='''
b=a+b
'''
block2="print(b)"
blocksleep='''
print('startsleep')
time.sleep(1)
print('donesleep')
'''
pc  = (block0,blocksleep,block1,block2)
cb = []
print('before')
output= run_code(pc)
print(output)
print('after')
print("Hello World!\n")

source: https://stackoverflow.com/a/3906390/1211174

Murchie85 · Accepted Answer · 2020-05-27 18:38:04Z

0

Question was answered by Joran Beasley.

Essentially, my issue was I was adding a space after \n as other compilers get confused, python however takes this into account. Thus if you add a space after a return carriage in a def statement, it will produce syntax error.

I have kept this incase others encounter similar issues.

edited May 27, 2020 at 18:38

answered May 24, 2020 at 1:00

Murchie85

8552 gold badges15 silver badges28 bronze badges

Comments

Chems Eddine · Accepted Answer · 2020-05-23 22:35:48Z

-1

Try using '\t' instead of placing tabs and spaces for the indentation, that may preserve your formatting.

code = """
def multiply(x,y):
\t return x*y

print('Multiply of 2 and 3 is: ',multiply(2,3))
"""
exec(code)

answered May 23, 2020 at 22:35

Chems Eddine

1516 bronze badges

1 Comment

jsbueno Over a year ago

\t simply expands to a tab character - even in generated code it should be avoided, as there are several pitfalls.

Collectives™ on Stack Overflow

Run a dynamic multi-line string of code in python at runtime

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related