str performance in python

Question

While profiling a piece of python code (python 2.6 up to 3.2), I discovered that the str method to convert an object (in my case an integer) to a string is almost an order of magnitude slower than using string formatting.

Here is the benchmark

>>> from timeit import Timer
>>> Timer('str(100000)').timeit()
0.3145311339386332
>>> Timer('"%s"%100000').timeit()
0.03803517023435887

Does anyone know why this is the case? Am I missing something?

And what about '{}'.format(100000)

wim
– wim

2012-05-10 13:09:16 +00:00
Commented May 10, 2012 at 13:09 — wim
– wim, Commented May 10, 2012 at 13:09
That is the slowest but also the most flexible.

Luca Sbardella
– Luca Sbardella

2012-05-10 14:02:28 +00:00
Commented May 10, 2012 at 14:02 — Luca Sbardella
– Luca Sbardella, Commented May 10, 2012 at 14:02

Chris Morgan · Accepted Answer · 2012-05-10 08:59:18Z

109

'%s' % 100000 is evaluated by the compiler and is equivalent to a constant at run-time.

>>> import dis
>>> dis.dis(lambda: str(100000))
  8           0 LOAD_GLOBAL              0 (str)
              3 LOAD_CONST               1 (100000)
              6 CALL_FUNCTION            1
              9 RETURN_VALUE        
>>> dis.dis(lambda: '%s' % 100000)
  9           0 LOAD_CONST               3 ('100000')
              3 RETURN_VALUE

% with a run-time expression is not (significantly) faster than str:

>>> Timer('str(x)', 'x=100').timeit()
0.25641703605651855
>>> Timer('"%s" % x', 'x=100').timeit()
0.2169809341430664

Do note that str is still slightly slower, as @DietrichEpp said, this is because str involves lookup and function call operations, while % compiles to a single immediate bytecode:

>>> dis.dis(lambda x: str(x))
  9           0 LOAD_GLOBAL              0 (str)
              3 LOAD_FAST                0 (x)
              6 CALL_FUNCTION            1
              9 RETURN_VALUE        
>>> dis.dis(lambda x: '%s' % x)
 10           0 LOAD_CONST               1 ('%s')
              3 LOAD_FAST                0 (x)
              6 BINARY_MODULO       
              7 RETURN_VALUE

Of course the above is true for the system I tested on (CPython 2.7); other implementations may differ.

edited May 10, 2012 at 8:59

Chris Morgan

91.6k28 gold badges217 silver badges220 bronze badges

answered May 10, 2012 at 8:43

georg

216k57 gold badges324 silver badges401 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Luca Sbardella Over a year ago

Indeed this looks like the reason, I just tried myself and string formatting is just about 5% faster than str. Thanks for answer. No reason to change code everywhere :-)

Karl Knechtel Over a year ago

To elaborate further: str is a name that can be rebound to something other than the string type, but string formatting - i.e. the str.__mod__ method - cannot be replaced, which permits the compiler to do the optimization. The compiler doesn't do very much in the way of optimization, but it does more than you might think :)

UncleZeiv Over a year ago

...and the lesson to learn here is: never use literals in tests like these!

Aaron Newton Over a year ago

This particular blog entry may interest you: skymind.com/~ocrow/python_string. It contains a chart of benchmarks for various string concatenation methods similar to what you have provided above.

Dietrich Epp · Accepted Answer · 2012-05-10 08:41:42Z

16

One reason that comes to mind is the fact that str(100000) involves a global lookup, but "%s"%100000 does not. The str global has to be looked up in the global scope. This does not account for the entire difference:

>>> Timer('str(100000)').timeit()
0.2941889762878418
>>> Timer('x(100000)', 'x=str').timeit()
0.24904918670654297

As noted by thg435,

>>> Timer('"%s"%100000',).timeit()
0.034214019775390625
>>> Timer('"%s"%x','x=100000').timeit()
0.2940788269042969

answered May 10, 2012 at 8:41

Dietrich Epp

216k39 gold badges366 silver badges426 bronze badges

Collectives™ on Stack Overflow

str performance in python

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related