In my code, I have a line like:
result = len([x for x in otherList if func(x) is not None and func(x).weekday == 3])
Written in a for loop, I would just define a variable like fx = func(x) and then use it twice. But I rather like the way my code is currently written, I am just concerned that since func(x) can be expensive, it will be evaluated twice. Is there some reasonably easy way to know whether the compiler would optimize such a statement and only evaluate the function once?
Is there some reasonably easy way to know whether the compiler would optimize such a statement and only evaluate the function once?
Yes, there is, you can look at the generated bytecode:
import dis
otherList = []
def func(x):
pass
def main():
result = len([x for x in otherList if func(x) is not None and func(x).weekday == 3])
print(dis.dis(main))
Output:
Disassembly of <code object <listcomp> at 0x000002AAC3C743A0, file "test.py", line 8>:
8 0 BUILD_LIST 0
2 LOAD_FAST 0 (.0)
>> 4 FOR_ITER 34 (to 40)
6 STORE_FAST 1 (x)
8 LOAD_GLOBAL 0 (func)
10 LOAD_FAST 1 (x)
12 CALL_FUNCTION 1
14 LOAD_CONST 0 (None)
16 IS_OP 1
18 POP_JUMP_IF_FALSE 4
20 LOAD_GLOBAL 0 (func)
22 LOAD_FAST 1 (x)
24 CALL_FUNCTION 1
26 LOAD_ATTR 1 (weekday)
28 LOAD_CONST 1 (3)
30 COMPARE_OP 2 (==)
32 POP_JUMP_IF_FALSE 4
34 LOAD_FAST 1 (x)
36 LIST_APPEND 2
38 JUMP_ABSOLUTE 4
>> 40 RETURN_VALUE
None
As you can see, the function is called twice.
How should the interpreter know that the result will not change based on some global variable?
If the result of your function never changes for the same arguments, you could consider to cache it, e.g. by using lru_cache
As @sahasrara62 pointed out, you could also use “the walrus operator” to store the result of a function call within your expression.
You can use an assignment expression for that:
result = len([x for x in otherList if (out:=func(x)) is not None and out.weekday == 3])
Note that using sum might be more efficient if you only want the number of matches:
result = sum(1 for x in otherList if (out:=func(x)) is not None and out.weekday == 3)
len([x for x in range(1000000)]), sum(1 for x in range(1000000)), and len([1 for x in range(1000000)]), the third one is the fastest. Summing also has a cost ;)
and