Does the Python compiler recognize constants defined within a function, such that it only computes their values once regardless of how many times the function is subsequently called in the code?
For example,
def f():
x = [ 1, 2, 3, 4 ]
# stuff
for i in range( 100 ):
f()
Would x be recalculated the 100 times f() is called?
It is not always possible to define constants outside the function that uses them, and I'm curious if Python has got your back in these situations.
(Note that this applies to CPython, and may be different in other implementations)
Python code is parsed and compiled into bytecode. You can see the instructions used with the dis module.
>>> def f(x):
... x = [1, 2, 3, 4]
>>> dis.dis(f)
2 0 LOAD_CONST 1 (1)
2 LOAD_CONST 2 (2)
4 LOAD_CONST 3 (3)
6 LOAD_CONST 4 (4)
8 BUILD_LIST 4
10 STORE_FAST 0 (x)
12 LOAD_CONST 0 (None)
14 RETURN_VALUE
>>> print(dis.Bytecode(f).info())
Name: f
Filename: <stdin>
Argument count: 1
Kw-only arguments: 0
Number of locals: 1
Stack size: 4
Flags: OPTIMIZED, NEWLOCALS, NOFREE
Constants:
0: None
1: 1
2: 2
3: 3
4: 4
Variable names:
0: x
As you can see, integer literals are constants, but lists have to be built everytime.
This is a relatively fast operation (Probably even quicker than looking up a global, but the time is still negligible)
If you had a function g that used a tuple instead, it is loaded as a constant:
>>> def g(x):
... x = (1, 2, 3, 4)
>>> dis.dis(g)
2 0 LOAD_CONST 5 ((1, 2, 3, 4))
2 STORE_FAST 0 (x)
4 LOAD_CONST 0 (None)
6 RETURN_VALUE
>>> print(dis.Bytecode(g).info())
Name: g
Filename: <stdin>
Argument count: 1
Kw-only arguments: 0
Number of locals: 1
Stack size: 4
Flags: OPTIMIZED, NEWLOCALS, NOFREE
Constants:
0: None
1: 1
2: 2
3: 3
4: 4
5: (1, 2, 3, 4)
Variable names:
0: x
But this seems like a case of premature optimisation.
The constants stored for a function can be found as function.__code__.co_consts.
>>> g.__code__.co_consts
(None, 1, 2, 3, 4, (1, 2, 3, 4))
The reason a new list has to be built every time is so that if the list is changed, it won't affect a list that is loaded everytime.
And the tuple optimisation goes away if it isn't a list of constants.
>>> def h(x):
... x = (1, 2, 3, x)
>>> dis.dis(h)
2 0 LOAD_CONST 1 (1)
2 LOAD_CONST 2 (2)
4 LOAD_CONST 3 (3)
6 LOAD_FAST 0 (x)
8 BUILD_TUPLE 4
10 STORE_FAST 0 (x)
12 LOAD_CONST 0 (None)
14 RETURN_VALUE
>>> print(dis.Bytecode(h).info())
Name: h
Filename: <stdin>
Argument count: 1
Kw-only arguments: 0
Number of locals: 1
Stack size: 4
Flags: OPTIMIZED, NEWLOCALS, NOFREE
Constants:
0: None
1: 1
2: 2
3: 3
Variable names:
0: x
lists of constants that are used in a manner that precludes mutation (directly iterated or containment checked without assigning to a named variable, e.g. for x in [1,2,3]: or if x in [1,2,3]:) are compile-time constants; the peephole optimizer replaces them with tuple constants (since, when it's impossible to mutate them and they're used in a few known patterns, a list and tuple behave identically, but the tuple is slightly more efficient to store and constant, making it safe).Short answer: for lists, it does not.
If we check the intermediate code after compilation with dis, we see:
>>> dis.dis(f)
2 0 LOAD_CONST 1 (1)
3 LOAD_CONST 2 (2)
6 LOAD_CONST 3 (3)
9 LOAD_CONST 4 (4)
12 BUILD_LIST 4
15 STORE_FAST 0 (x)
18 LOAD_CONST 0 (None)
21 RETURN_VALUESo as you can see the program first loads constants 1 to 4 and pushes these on the stack, and the constructs a list with these constants, so that means it constructs a list each time.
In case the list is not mutated, I propose to define the constant outside the function:
some_constant = [1, 2, 3, 4]
def f():
# use some_constant
# ...
pass
list specific behaviors (e.g. equality comparison with another list), using a tuple of constant literals within the function would be faster and avoid scoping issues (avoids some other function/module messing with it by accident), since it can be stored fully constructed in the function's constant table. Each use becomes a simple LOAD_CONST (using the literal directly) or LOAD_FAST (if in a named local) which boil down to C array lookups, rather than a LOAD_GLOBAL, which is (at least) one dict lookup.isinstance(x, list), then for a list, it will fail. So we can not assume that it will have completely identical behavior.list, so the global wouldn't be safe. You'd need to copy the global on entering the function (x = some_constant[:]) to avoid that risk. Fun alternative: Python 3.5's unpacking generalizations let you have your cake and eat it too. You could do x = [*(1, 2, 3, 4)], which constructs the list by loading the constant tuple, then performing a single byte code BUILD_LIST_UNPACK.tuple->list is no real improvement on the list literal, but for larger sequences unpacking is faster. They're both faster than slicing a global list to get a shallow copy for any reasonable size list (the slicing approach takes over 33% longer for four elements; slices are weirdly expensive here).
xas a constant in python? How would the compiler know thatxisn't going to change (in your#stuffsection for example)?