3

So according to the optimization tips at http://wiki.python.org/moin/PythonSpeed/PerformanceTips , joining strings should be done with

out = "<html>%(head)s%(prologue)s%(query)s%(tail)s</html>" % locals()
and not
out = "<html>" + head + prologue + query + tail + "</html>" My question is, is this the same for if I wanted to print, instead of store the value? Also, would putting consecutive print statements all on one line be faster? Like would it be better to use

print "Some word"
print "Another line"
print "something else"

or 

print '''Some word
Another line
something else'''

Thanks in advance!

Improve this question
5
  • 1
    by default print would add newline at the end of the string, so your first example isn't equivalent to the second. Commented Nov 8, 2010 at 21:54
  • Thanks, Andrew, I missed that, I'll fix it Commented Nov 8, 2010 at 21:57
  • 1
    See Back to Basics by Joel Spolsky for a discussion of this issue. Although the article is about C, the issue it discusses also applies to Python (and many other languages). Commented Nov 8, 2010 at 21:58
  • If you don't need to support older then python2.6, it is preferred to use str.format to do the interpolation. eg. print "My name is {name}".format(name='Parker') Commented Nov 8, 2010 at 22:15
  • I would expect that the time required for console output would dwarf the differences in how the string is constructed. Once you're printing, everything is slow. Commented Nov 8, 2010 at 23:17

3 Answers 3

Reset to default
6

String concatenation has been improved for the (fairly common) case where there is only one reference to the string. See PyString_ConcatAndDel in stringobject.c

So usually concatenation in a loop is linear since there is only ever one reference of the string

Here is a simple experiment to demonstrate the behaviour. When there is no room to extend the string the id() changes

>>> s = ""
>>> prev_id = None
>>> for i in range(1000):
...  s += "*"
...  if prev_id != id(s):
...   print id(s), len(s)
...   prev_id = id(s)
... 
3077352864 1
3077437728 2
3077434328 9
3077428384 17
3077379928 25
3077291808 33
3077712448 41
3077358800 49
3077394728 57
3077667680 65
3077515120 73
3077354176 81
3077576488 89
3077559200 97
3077414248 105
3077670336 113
3077612160 121
3077707040 129
3077526040 137
3077571472 145
3077694944 153
3077595936 161
3077661904 169
3077552608 177
3077715680 185
3077583776 193
3077244304 201
3077604560 209
3077510392 217
3077334304 225
144468768 233
144787416 245
144890104 389
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Cool stuff, I gotta stick with katrielalex's answer, but you get a +1 for explaining this for me! I think you've answered my questions before here too, so as always, thanks!
I find it interesting the ramp-up rate python uses for this optimization, and the fact that both of us end up with different output. I'm accustomed to the rather boring algorithm of just increasing the capacity by a constant factor (usually of 2), but apparently python does something different. Either way, this makes appending a linear operation.
5

Your question isn't really about the most efficient way to print strings, it's about formatting them for output, for which you should use format in any case because it does more than simple concatenation. However, here are some notes on concatenation.

Edit: rewritten to include some details

The printing is irrelevant. The important point is that due to the way that some languages handle string concatenation, joining lots of strings may be of quadratic order. The (very naive and basic) reasoning is that to concatenate two strings you have to walk down all the characters of the first string and then append all the characters of the second. So if you are concatenating ten strings, you first walk the first and append the second, then you walk the first+second and append the third, then you walk the first+second+third and append the fourth, and so on.

A naive implementation of concatenation will thus cause you to do much more work than you need to. Indeed, in early versions of Python this was an issue. However, @gnibbler has pointed out in the comments, later versions now generally optimise this, thus mooting the point entirely.

The Python idiom to join strings is "".join(...). This completely bypasses any possible issue and is the standard idiom anyway. If you want the ability to construct a string by appending, have a look at a StringIO:

>>> from io import StringIO
>>> foo = StringIO()
>>> for letter in map(chr, range(128)):
...     foo.write(letter)
...
>>> foo.seek(0)
0
>>> foo.read()
'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\
x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./0123456789:;<=>?@ABC
DEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f'
Improve this answer

9 Comments

As an aside, this concatenation issue is not specific to Python, and applies to a many languages. Also, it's a very popular interview question.
Okay, thanks!So just to clarify, the number of print statements doesn't matter, but when I do print, I shouldn't use '+' ?
Note that in Python versions 2.5 and later "joining lots of strings with + is in fact of quadratic order." is no longer true (but I still avoid it in favor of other idioms anyways).
@jimbob, concatenation isn't always quadratic - see stackoverflow.com/questions/4128494
It's worth noting that the actual action of concatenation is much faster than any of the alternatives for small amounts of strings. Also, join is almost entirely besides the point here as the question asks about interpolation where the format method is faster and join is useless.
|
2

For printing there is no need to concatenate:

print "<html>", head, prologue, query, tail, "</html>"

This works the same (comma at the end prevents \n):

print "<html>",
print head,
...
print "</html>"

I think the answer is NO, do not concatenate just for printing, it will make things slower. But you really should not take my word for it, just write a few tests and profile with timeit.

Improve this answer

1 Comment

This works ok in Python3 with the print function too, and you can specify your own separator. eg. print("foo", "bar", sep="-")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.