Correctly splitting strings after reading from file in Python

Question

I'm attempting to average the lengths of each word after reading from a file. The text inside the file, however, is not formatted with normal sentence structure. Sometimes there is an extra space between words and new line breaks mid sentence.

Current code

def average(filename):
    with open(filename, "r") as f:
        for line in f:
            words = line.split()
            average = sum(len(words) for words in words)/len(words)
            return average

>>>4.3076923076923075

Expected
>>>4.352941176470588

File

Here are some words   there is no punctuation but there are words what
is the average length

please provide full code. return can only be used inside a function. — marmeladze
– marmeladze, Commented Oct 10, 2015 at 20:39
you say you attempting to calculate the average length of each string, do you mean words or number of words per line? Also, you might want to edit the double meaning of the name words in the code — Pynchia
– Pynchia, Commented Oct 10, 2015 at 20:47

MattDMo · Accepted Answer · 2015-10-10 20:54:20Z

2

When you open the file as f, then run

for x in f:

x will be each line in the file, ended by a newline. The answer you are getting is perfectly correct for the first line of text. If you want the second line to be included in the first one, you'll need to process the text file as a whole, and not line by line.

Assuming you want to get the average of all the words in the file, the following should work a little better:

def average(filename):
    with open(filename, "r") as f:
        lines = [line for line in f]
        words = " ".join(lines).split()
        average = sum(len(word) for word in words)/len(words)
    return average

edited Oct 10, 2015 at 20:54

answered Oct 10, 2015 at 20:45

MattDMo

103k21 gold badges251 silver badges239 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

23k Over a year ago

would you care to provide an example? I'm fairly new to Python coming from Java

MattDMo Over a year ago

@23k are there multiple entries in the file, or is this it? If this is the only entry, just " ".join() each line

23k Over a year ago

this is the only entry.

23k Over a year ago

Awesome! Thanks for the help :)

Collectives™ on Stack Overflow

Correctly splitting strings after reading from file in Python

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related