I want to hack around with the Python interpreter and try creating a small DSL . Is there any module where I can do something like this theoretical code (similar to LINQ expression trees)?
expression_tree = Function(
Print(
String('Hello world!')
)
)
compile_to_bytecode(expression_tree)
Or would it just be easier to just generate Python source code? Could this be made easier by using C or SWIG or Cython?
Working via ast and compiling the tree into bytecode, as another answer suggests, is probably simplest; generating sources and compiling those, almost as good.
However, to explore lower-level approaches, check out the links from this page; I've found byteplay especially useful (currently doesn't work on 2.6 nor 3.*, only 2.4 or 2.5, but I think fixing it for 2.6 should be easy, as currently discussed in its tracker). I have not used Phil Eby's similarly-featured BytecodeAssembler, but given the author's reputation I'm sure it's worth checking it out!
In Python 2.X, you would typically approach this with the compiler module and its ast sub-module (but note this module is deprecated since version 2.6). In Python 3.X, you would use just ast.
Both offer a compile() function that will go from source/AST to "a code object that can be executed by the exec statement or eval()."
It's easier to generate Python code and run it. If you do that you can also debug it more easily since there is actual source for the debugger to display. See also Malte Borchs article in the July issue of Python magazine, where he talks about this amongst other things.
Fernando Meyer recently wrote a blog post explaining how to use the # coding directive to specify your own extensions to Python. Example (the actual format definition is in pyspec.py and tokenizer.py):
# coding: pyspec
class Bow:
def shot(self):
print "got shot"
def score(self):
return 5
describe Bowling:
it "should score 0 for gutter game":
bowling = Bow()
bowling.shot()
assert that bowling.score.should_be(5)
Updating for Python3 - there's also very interesting assembler zachariahreed/byteasm.
Actually the only one working for me in Py3. It has very nice & clean API:
>>> import byteasm, dis
>>> b = byteasm.FunctionBuilder()
>>> b.add_positional_arg('x')
>>> b.emit_load_const('Hello!')
>>> b.emit_load_fast('x')
>>> b.emit_build_tuple(2)
>>> b.emit_return_value()
>>> f = b.make('f')
>>> f
<function f at 0xb7012a4c>
>>> dis.dis(f)
1 0 LOAD_CONST 0 ('Hello!')
3 LOAD_FAST 0 (x)
6 BUILD_TUPLE 2
9 RETURN_VALUE
>>> f(108)
('Hello!', 108)
Check out the disassembler module found here:
