How to use the regex module in python to split a string of text into the words only?

Question

Here's what I'm working with…

string1 = "Dog,cat,mouse,bird. Human."

def string_count(text):
    text = re.split('\W+', text)
    count = 0
    for x in text:
        count += 1
        print count
        print x

return text

print string_count(string1)

…and here's the output…

1
Dog
2
cat
3
mouse
4
bird
5
Human
6

['Dog', 'cat', 'mouse', 'bird', 'Human', '']

Why am I getting a 6 even though there are only 5 words? I can't seem to get rid of the '' (empty string)! It's driving me insane.

In the question editor, drag-select your code and press the {} button located above the text editor. — Robᵩ
– Robᵩ, Commented Aug 26, 2014 at 1:26

Avinash Raj · Accepted Answer · 2014-08-26 01:41:48Z

1

Because while it splits based on the last dot, it gives the last empty part also.

You splitted the input string based on \W+ which means split the input string based on one or more non-word character. So your regex matches the last dot also and splits the input based on the last dot also. Because of no string present after to the last dot, it returns an empty string after splitting.

edited Aug 26, 2014 at 1:41

answered Aug 26, 2014 at 1:27

Avinash Raj

175k32 gold badges247 silver badges289 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mark Reed Over a year ago

This is the correct answer. The string ends with a character that matches \W+ - namely, a period (.). That means there is an extra, empty field at the end of the string. If the string began with a non-word character, there would be an empty field at the beginning, too.

Robᵩ Over a year ago

Note the first example here: docs.python.org/2/library/re.html#re.split demonstrates precisely this behavior.

Adam Smith · Accepted Answer · 2014-08-26 02:37:52Z

1

Avinash Raj correctly stated WHY it's doing that. Here's how to fix it:

string1 = "Dog,cat,mouse,bird. Human."
the_list = [word for word in re.split('\W+', string1) if word]
# include the word in the list if it's not the empty string

Or alternatively (and this is better...)

string1 = "Dog,cat,mouse,bird. Human."
the_list = re.findall('\w+', string1)
# find all words in string1

answered Aug 26, 2014 at 2:37

Adam Smith

54.6k13 gold badges84 silver badges120 bronze badges

Collectives™ on Stack Overflow

How to use the regex module in python to split a string of text into the words only?

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related