4

I was reading up about the Python interpreter because I couldn't understand why some things had the python compiled objects (.pyc), but others didn't.

I got the answer to my question, but now I'm confused. So okay, the interpreter compiles a script to a module...which is 'sort of' like an object in C if I'm understanding this correctly (C programmer here, new to Python) - or I guess more like a .class in Java since it's compiled bytecode, not native instructions...anyway it does this when either you import a script, OR if you explicitly call it to be compiled (which is for some reason less favorable).

So under that understanding, is there any runtime difference between compiled bytecode and not? Assuming there's only one interpreter (a bytecode interpreter), it would mean that if the module isn't already compiled, it has to do the grammar/lexing/parsing (compiling) right before it does the interpretation. Won't that lead to a higher execution time?

So if you take the above to be true, then it's obviously best if the modules are compiled into .pyc, not ran as a standard .py script on the fly.

Would that mean it's best to have as minimal execution in your main run as possible?

I would think, if your entry point has any hardcore logic (ie. mine has a couple tree traversals, and other heavy compares), shouldn't that entry point in and of itself be wrapped so it's compiled?

That is, instead of:

# file.py:
def main():
    <stuff goes here - setup, whatever shared resources different modules need, etc.>

main()

Would it be better to do:

# wrapper.py:
from file.py import *
main()

Hope I explained what I'm asking, well enough. It's quite possible that I got the wrong understanding of how the interpreter/compiler is used in Python and this question isn't even reasonable to ask - I'm quite new to Python.

TIA

Improve this question

2 Answers 2

Reset to default
7

You are correct:

1) A .pyc file is a cached copy of the compilation of Python source to bytecode

2) A .pyc file is created when a module is imported

3) The main program that you name on the python command line is not compiled to a .pyc

4) Therefore the main program is compiled every time it is run.

But:

The main program is compiled to bytecode, and then the bytecode is interpreted, so there's only a small overhead (the compilation), there isn't an ongoing overhead to all execution in the main program. The extra time for compiling is proportional to the number of lines in the file, not the number of time those lines are executed.

tl;dr: don't worry about it.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

While I understand what you're saying, Ned, if your driver file (your 'main' if you will) has a good amount of logic to drive the script/module interface and the order of execution of different modules, then that's still extra complexity that must be compiled at runtime - every time. So I have to regress to my question and ask if it's best if the initially called script simply imports what was your main method, and simply has one line: "main()" - as that would be the most efficient? It seems the answer is kind of what I thought: Yes.
Yes, it will be most efficient to keep you main file small. But the more logic you have in your main file, the more the compilation time will be dwarfed by the running time anyway. Compiling one Python file won't take long even for the most complex file. In a large production system it just won't matter. Write your code so that it works, then worry next about readability and maintainability.
To me, whether there's a one line main function call in a separate wrapper file doesn't hinder readability nor maintainable, but has the ability to speed up execution time. I ran some numbers today. about .75s with the direct call, so compiling the main driver script on the fly. Approx .5s on the other hand, if I called it with one line in a separate file, and ran that. Results on python 3.2 on a I7 2600k - Win7 64 & 8GB @ 1333. I was quite surprised, even more-so when the numbers persisted.
0

This is for beginners,

Python automatically compiles your script to compiled code, so called byte code, before running it.

Running a script is not considered an import and no .pyc will be created.

For example, if you have a script file abc.py that imports another module xyz.py, when you run abc.py, xyz.pyc will be created since xyz is imported, but no abc.pyc file will be created since abc.py isn’t being imported.

If you need to create a .pyc file for a module that is not imported, you can use the py_compile and compileall modules.

The py_compile module can manually compile any module. One way is to use the py_compile.compile function in that module interactively:

import py_compile
py_compile.compile('abc.py')

This will write the .pyc to the same location as abc.py (you can override that with the optional parameter cfile).

For more: http://effbot.org/pyfaq/how-do-i-create-a-pyc-file.htm

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.