For-loop only iterates last element in Python

Question

I'm looping through a XML document and matching usernames from a txt-file.

The txt looks like:

DPL bot
Nick Number
White whirlwind
Polisci
Flannel

And the program looks like:

    import xmltodict, json

with open('testarticles.xml', encoding='latin-1') as xml_file:
    dic_xml = xmltodict.parse(xml_file.read())
    for page in dic_xml['mediawiki']['page']:
        for rev in  page['revision']:
            for user in open("usernames.txt", "r"):
                print(user)

                if 'username' in rev['contributor'] and rev['contributor']['username'] == user:
                    print(user)
                    print(rev['timestamp'])
                    timestamp = rev['timestamp'];

                    try:
                        print(rev['comment'])
                        comment = rev['comment'];
                    except:
                        print("no comment")
                        comment = ''

                    print('\n')
                    with open("User data/" + user + ".json", "a") as outfile:
                        json.dump({"timestamp": timestamp, "comment": comment}, outfile)
                        outfile.write('\n')

The problem is that the program only goes through the if-statement for the last line in the text file. It prints all the users' names before the if-statement. All users have matching posts in the XML-file and by changing to another user at the end line, that user's data is extracted into the json file.

add an else clause and print rev['contributor'] to see what's going on when it fails? Try if 'username' in rev['contributor'] and rev['contributor']['username'] == user.strip(): — jDo
– jDo, Commented Apr 1, 2016 at 9:18
Out of curiosity, does for line in open(...): automatically assume with context and thus close the file when the loop is done? — Torxed
– Torxed, Commented Apr 1, 2016 at 9:20
@Torxed Good question. On one hand, I wouldn't think so since open() doesn't work like that normally but, then again, it's open() plus a for loop which is a rather "ephemeral" construct that doesn't persist beyond it's scope. Also, there's nothing to call file.close() on since open() has no handle when used in a for loop. It's all speculation though hehe. — jDo
– jDo, Commented Apr 1, 2016 at 9:32
Repeatedly rereading the user file inside the loop seems like a spectacularly inefficient design, unless you expect the file to change between iterations (and even then, there is probably a better way to do it). Just read it once into a dictionary before opening the XML file. — tripleee
– tripleee, Commented Apr 1, 2016 at 9:40

jDo · Accepted Answer · 2016-04-01 09:52:08Z

1

Maybe all lines except the last have a newline at the end...

Try this:

for user in open("usernames.txt", "r"):
    user = user.strip()
    if 'username' in rev['contributor'] and rev...

or use this construct so we don't get a headache debating whether or not your code works like a with statement or not :P

with open("usernames.txt", "r") as f:
    for line in f:
        user = line.strip()
        if 'username' in rev['contributor'] and rev...

The main thing is user = user.strip() or user = line.strip()

When in doubt, look at the binary. That goes for all encoding issues as well since encoding is just a way of transforming ones and zeros to characters according to some translation table/code page.

"\n".encode("hex") == "0a" # True
# so if
user.encode("hex") 
# has "0a" at the end, there is definitely a newline after "user"

edited Apr 1, 2016 at 9:52

answered Apr 1, 2016 at 9:21

jDo

4,0301 gold badge13 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Knokkelgeddon Over a year ago

Perfect, works now! But what about the "newline" at the end?

jDo Over a year ago

Cool :) What about the newline? Do you need it later on? In that case, don't redefine user or line by doing user = user.strip() or line = line.strip() but make a separate variable that's only used in the comparison - or simply add the newline back in when you write to a file again (which you're already doing here: outfile.write('\n')). Alternatively, add all usernames to a dictionary instead and only write and dump the whole thing as json once - after the loop (I’d probably do that instead of appending to the file once per user/line)

Collectives™ on Stack Overflow

For-loop only iterates last element in Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related