How to parse a Python module from file

Question

I saw this SO question and tried using it by creating a .py file with 2 methods and trying to read it.
The file:

def f1(a):
    print "hello", a
    return 1

def f2(a,b):
    print "hello",a,", hello",b

Trying to read it:

>>> r = open('ToParse.py','r')
>>> t = ast.parse(r.read)

Exception thrown:

Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
  File "C:\Python26\lib\ast.py", line 37, in parse
    return compile(expr, filename, mode, PyCF_ONLY_AST)
TypeError: expected a readable buffer object

What am I doing wrong?
My goal is to get a python module and be able to parse it using Python - expose its classes and methods.

Python has better introspection mechanisms than most, dir(), inspect, __dir__, etc. Looks like you are reinventing the wheel. — cdarke
– cdarke, Commented Mar 20, 2013 at 8:53
I'm not trying to invent the wheel, I'm looking for the best existing wheel :) — Noich
– Noich, Commented Mar 20, 2013 at 9:38
@BurhanKhalid, @ cdarke Sometimes we can't import a module, because the dependencies aren't there. But we still want to analyse its structure. — Evgeni Sergeev
– Evgeni Sergeev, Commented Mar 21, 2018 at 23:42

pradyunsg · Accepted Answer · 2013-03-20 08:59:03Z

13

You need to call read. So your line

t = ast.parse(r.read)

Should be

t = ast.parse(r.read())

See here for info on files and here for info on ast.parse

answered Mar 20, 2013 at 8:59

pradyunsg

19.9k11 gold badges48 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

n1ckolas · Accepted Answer · 2013-03-20 10:06:40Z

6

If you want to expose your classes and methods dynamically, then you probably need to use eval along with compile.

In this case you may do it like the following.

Create a file:

#test.py
def hello():
    print "hello"

And you can call it like this:

#main.py
testContent = open("test.py").read()
#evaluate a content
eval(compile(testContent, "<string>", 'exec'))
#call function
hello() #prints hello

EDIT: there is another way to evaluate file:

#main.py
#evaluate a content
eval(compile("import test", "<string>", 'exec')) #test.py
#check list of methods
dir(test) # ['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'hello']
#call function
hello() #prints hello

I do realize, that eval may be not that good choice, but I don't know other way. I'd glad to see other solution

edited Mar 20, 2013 at 10:06

answered Mar 20, 2013 at 9:00

n1ckolas

4,4503 gold badges39 silver badges43 bronze badges

3 Comments

n1ckolas Over a year ago

agree. but in some particular cases its usage is suitable

Noich Over a year ago

I just tried it and PythonWin didn't recognize my methods. Any chance I can get a list out of compile(), of modules and functions (I also need to show them, not just be able to call them)?

Francesco Frassinelli Over a year ago

compile and ast.parse are similar in this case. Try with ast.parse directly. Otherwise you can always import the code directly and use dir().

Francesco Frassinelli · Accepted Answer · 2013-03-20 08:52:31Z

4

Use:

t = ast.parse(r.read()) # () is needed

Source: http://docs.python.org/2/tutorial/inputoutput.html#methods-of-file-objects

answered Mar 20, 2013 at 8:52

Francesco Frassinelli

3,4152 gold badges34 silver badges44 bronze badges

Comments

davidcorne · Accepted Answer · 2013-03-20 08:58:19Z

2

You are trying to parse the function read on the file.

You want

t = ast.parse(r.read())

or (to more closely follow the example)

text = r.read()
ast.parse(text)

not

t = ast.parse(r.read)

answered Mar 20, 2013 at 8:58

davidcorne

3632 silver badges10 bronze badges

Collectives™ on Stack Overflow

How to parse a Python module from file

4 Answers 4

Comments

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related