Python readline() from a string?

Question

In python, is there a built-in way to do a readline() on string? I have a large chunk of data and want to strip off just the first couple lines w/o doing split() on the whole string.

Hypothetical example:

def handleMessage(msg):
   headerTo  = msg.readline()
   headerFrom= msg.readline()
   sendMessage(headerTo,headerFrom,msg)

msg = "Bob Smith\nJane Doe\nJane,\nPlease order more widgets\nThanks,\nBob\n"
handleMessage(msg)

I want this to result in: sendMessage("Bob Smith", "Jane Doe", "Jane,\nPlease order...")

I know it would be fairly easy to write a class that does this, but I'm looking for something built-in if possible.

EDIT: Python v2.7

jterrace · Accepted Answer · 2021-02-18 07:03:01Z

100

In Python 3, you can use io.StringIO:

>>> msg = "Bob Smith\nJane Doe\nJane,\nPlease order more widgets\nThanks,\nBob\n"
>>> msg
'Bob Smith\nJane Doe\nJane,\nPlease order more widgets\nThanks,\nBob\n'
>>>
>>> import io
>>> buf = io.StringIO(msg)
>>> buf.readline()
'Bob Smith\n'
>>> buf.readline()
'Jane Doe\n'
>>> len(buf.read())
44

In Python 2, you can use StringIO (or cStringIO if performance is important):

>>> import StringIO
>>> buf = StringIO.StringIO(msg)
>>> buf.readline()
'Bob Smith\n'
>>> buf.readline()
'Jane Doe\n'

edited Feb 18, 2021 at 7:03

answered Sep 19, 2011 at 14:55

jterrace

67.5k24 gold badges164 silver badges208 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Brian McFarland Over a year ago

Perfect. buf.read() what's left of the buffer after reading a certain number of lines.

Glaslos Over a year ago

Why allocating extra memory for buf?

Mario Wanka · Accepted Answer · 2018-03-27 14:11:26Z

41

The easiest way for both python 2 and 3 is using string's method splitlines(). This returns a list of lines.

>>> "some\nmultilene\nstring\n".splitlines()

['some', 'multilene', 'string']

answered Mar 27, 2018 at 14:11

Mario Wanka

5114 silver badges4 bronze badges

4 Comments

C.W. Over a year ago

And str.splitlines() is cross-platform, regardless the style of line endings like \r\n or \n.

Alex Metsai Over a year ago

I think that this is more convenient than the accepted answer.

Ratul Hasan Over a year ago

the fun fact is that it removes all the \n or \r\n, so if you need to keep them as is, but split the lines, better go for the StringIO or put keepend = True arg in the splitlines()

Otzen Over a year ago

@RatulHasan from doc: "str.splitlines(keepends=False)", set the keepends to True if you want to keep them.

Amber · Accepted Answer · 2011-09-19 15:03:33Z

22

Why not just only do as many splits as you need? Since you're using all of the resulting parts (including the rest of the string), loading it into some other buffer object and then reading it back out again is probably going to be slower, not faster (plus the overhead of function calls).

If you want the first N lines separated out, just do .split("\n", N).

>>> foo = "ABC\nDEF\nGHI\nJKL"
>>> foo.split("\n", 1)
['ABC', 'DEF\nGHI\nJKL']
>>> foo.split("\n", 2)
['ABC', 'DEF', 'GHI\nJKL']

So for your function:

def handleMessage(msg):
   headerTo, headerFrom, msg = msg.split("\n", 2)
   sendMessage(headerTo,headerFrom,msg)

or if you really wanted to get fancy:

# either...
def handleMessage(msg):
   sendMessage(*msg.split("\n", 2))

# or just...
handleMessage = lambda msg: sendMessage(*msg.split("\n", 2))

edited Sep 19, 2011 at 15:03

answered Sep 19, 2011 at 14:58

Amber

531k89 gold badges643 silver badges558 bronze badges

2 Comments

Brian McFarland Over a year ago

This works if you don't mind dealing with exceptions and/or checking array lengths that come with split AND you know how many lines to read in advance. I don't know how many header lines there are in advance. Perhaps my example was excessively trivial.

Amber Over a year ago

If you're inspecting things as you go, then yes, StringIO is probably your best bet. (No worries re: examples - constructing the appropriate example can often be a difficult balance between simplifying and not losing context.)

Glaslos · Accepted Answer · 2011-09-19 15:22:55Z

6

Do it like StringIO does it:

i = self.buf.find('\n', self.pos)

So this means:

pos = msg.find("\n")
first_line = msg[:pos]
...

Seems more elegant than using the whole StringIO...

edited Sep 19, 2011 at 15:22

answered Sep 19, 2011 at 15:01

Glaslos

2,9221 gold badge25 silver badges33 bronze badges

Comments

Sérgio · Accepted Answer · 2019-03-19 04:49:33Z

4

in Python string have method splitlines

msg = "Bob Smith\nJane Doe\nJane,\nPlease order more widgets\nThanks,\nBob\n"
msg_splitlines = msg.splitlines()
headerTo = msg_splitlines[0]
headerFrom= msg_splitlines[1]
sendMessage(headerTo,headerFrom,msg)

answered Mar 19, 2019 at 4:49

Sérgio

7,3671 gold badge51 silver badges58 bronze badges

Collectives™ on Stack Overflow

Python readline() from a string?

5 Answers 5

2 Comments

4 Comments

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

2 Comments

4 Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related