How to create a list of JSON objects in Python 2.7

Question

I am converting an XML file to a JSON file. I do this by opening the xml, use the xmltodict module and then use the .get method to traverse the tree to the level I want. This level is the parents to the leaves. I then check on a certain condition that some of the leaves for each of these task is true and if it is then I use json.dumps() and write it to the file. The issue is (I think this is where it is stemming from) that when I only append one JSON object to the file, it doesn't append a comma to the end of the object because it thinks it is the only object. I tried combating this by appending a ',' at the end of each JSON object but then when I try to use the json.loads() method it gives me an error saying "No JSON object could be decoded". However when I manually append the '[' and ']' to the file it doesn't give me an error. My code is below and I'd appreciate any help/suggestions you have.

def getTasks(filename):
    f = open(filename, 'r')
    a = open('tasksJSON', 'w')
    a.write('[')

    d = xmltodict.parse(f)
    l = d.get('Project').get('Tasks').get('Task')
    for task in l:
        if (task['Name'] == 'dinner'):  #criteria for desirable tasks
            j = json.dumps(task)
            a.write (str(j))   
            a.write(',')         

    a.write(']')
    f.close()
    a.close()

This works and puts everything in tasksJSON but like I said, when I call

my_file = open('tasksJSON', 'r')
data = json.load(my_file)           # LINE THAT GIVES ME ERROR

I get an error saying

ValueError: No JSON object could be decoded

and the output file contains:

[{"UID": "4", "ID": "14", "Name": "Design"},{"UID": "5", "ID": "15", "Name": "Basic Skeleton"}]
                                           ^
                                 this is the comma I manually inserted

That isn't your output file. If it were, it would work, as you can see in this online interpreter. The actual output generated by your code will have an extra comma on the end, which will give you an error as you can see here. — abarnert
– abarnert, Commented Sep 15, 2014 at 20:18

Harsh Bhikadia · Accepted Answer · 2014-09-15 20:25:50Z

2

make it this way:

def getTasks(filename):
    f = open(filename, 'r')
    a = open('tasksJSON', 'w')
    x = []
    d = xmltodict.parse(f)
    l = d.get('Project').get('Tasks').get('Task')
    for task in l:
        if (task['Name'] == 'dinner'):  #criteria for desirable tasks
            #j = json.dumps(task)
            x.append(task)
            #a.write (str(j))   
            #a.write(',')         

    a.write(json.dumps(x))
    f.close()
    a.close()

answered Sep 15, 2014 at 20:25

Harsh Bhikadia

11k10 gold badges53 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

abarnert · Accepted Answer · 2014-09-15 20:05:39Z

1

JSON doesn't allow extra commas at the end of an array or object. But your code adds such an extra comma. If you look at the official grammar here, you can only have a , before another value. And Python's json library conforms to that grammar, so:

>>> json.loads('[1, 2, 3, ]')
ValueError: Expecting value: line 1 column 8 (char 7)

To fix this, you could do something like this:

first = True
for task in l:
    if (task['Name'] == 'dinner'):  #criteria for desirable tasks
        if first:
            first = False
        else:
            a.write(',')
        j = json.dumps(task)
        a.write(str(j))

On the other hand, if memory isn't an issue, it might be simpler—and certainly cleaner—to just add all of the objects to a list and then json.dumps that list:

output = []
for task in l:
    if (task['Name'] == 'dinner'):  #criteria for desirable tasks
        output.append(task)
a.write(json.dumps(output))

Or, more simply:

json.dump([task for task in l if task['Name'] == 'dinner'], a)

(In fact, even if memory is an issue, you can extend JSONEncoder, as shown in the docs, to handle iterators by converting them lazily into JSON arrays, but this is a bit tricky, so I won't show the details unless someone needs them.)

edited Sep 15, 2014 at 20:05

answered Sep 15, 2014 at 20:00

abarnert

368k54 gold badges626 silver badges691 bronze badges

2 Comments

sbru Over a year ago

this makes sense, thanks @abarnet. However I'm still a bit confused. I will show you what the output file looks like from this code. update above

abarnert Over a year ago

@bagelboy: The output file you posted is not your real output file, and doesn't contain the same problem, so of course it works. If you add the same problem as your real output file, you get the same error. See my comment on the question for more details.

Sergius · Accepted Answer · 2014-09-15 19:58:44Z

0

It seems, that you put into a file several json objects and add your own square brackets. Hence, it can not load as single obj

answered Sep 15, 2014 at 19:58

Sergius

9869 silver badges21 bronze badges

1 Comment

abarnert Over a year ago

JSON is just strings, and a bunch of valid JSON objects separated by commas and surrounded by brackets makes a valid JSON array.

Collectives™ on Stack Overflow

How to create a list of JSON objects in Python 2.7

3 Answers 3

Comments

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related