Parse list of json string in python

Question

Using an incoming string like list:

[{"FECHA":"2019-01-28 13:15:42","SERIAL":2,"LONGITUD":-4.2958984375,"LATITUD":50.4469470596,"ID":1,"VALOR":193,"JOURNEYID":1},
{"FECHA":"2019-01-28 13:15:44","SERIAL":3,"LONGITUD":-4.2958984375,"LATITUD":40.4469470596,"ID":1,"VALOR":34,"JOURNEYID":1},
{"FECHA":"2019-01-28 13:15:46","SERIAL":6,"LONGITUD":-3.2958984375,"LATITUD":50.4469470596,"ID":1,"VALOR":14,"JOURNEYID":1},<..>]

With a lenght N and each element with the following format:

{"FECHA":"2019-01-28 13:15:42","SERIAL":2,"LONGITUD":-4.2958984375,"LATITUD":50.4469470596,"ID":1,"VALOR":193,"JOURNEYID":1}

What I want?

Iterate the list and process each JSON elem individually

first output: {"FECHA":"2019-01-28 13:15:42","SERIAL":2,"LONGITUD":-4.2958984375,"LATITUD":50.4469470596,"ID":1,"VALOR":193,"JOURNEYID":1}

second output: {"FECHA":"2019-01-28 13:15:44","SERIAL":3,"LONGITUD":-4.2958984375,"LATITUD":40.4469470596,"ID":1,"VALOR":34,"JOURNEYID":1}

<...>

How could I split to take each JSON element?

What I have tried:

Option 1:

def ParseIncomingDataAzure(message):
    print ("incoming data: {}".format(message))
    x = ast.literal_eval(message)
    for frame in x:
        print("x: {}".format(frame))

Output:

x: {'LATITUD': 50.4469470596, 'FECHA': '2019-01-28 13:15:42', 'JOURNEYID': 1, 'VALOR': 193, 'SERIAL': 2, 'ID': 1, 'LONGITUD': -4.2958984375}
x: {'LATITUD': 40.4469470596, 'FECHA': '2019-01-28 13:15:44', 'JOURNEYID': 1, 'VALOR': 34, 'SERIAL': 3, 'ID': 1, 'LONGITUD': -4.2958984375}
x: {'LATITUD': 50.4469470596, 'FECHA': '2019-01-28 13:15:46', 'JOURNEYID': 1, 'VALOR': 14, 'SERIAL': 6, 'ID': 1, 'LONGITUD': -3.2958984375}

Option 2:

def ParseIncomingDataAzure(message):
    messages = message.split(",")
    for frame in messages:
        print("x: {}".format(frame))

Output:

x: [{"FECHA":"2019-01-28 13:15:42"
x: "SERIAL":2
x: "LONGITUD":-4.2958984375
x: "LATITUD":50.4469470596
x: "ID":1
x: "VALOR":193
x: "JOURNEYID":1}
x: {"FECHA":"2019-01-28 13:15:44"
x: "SERIAL":3
x: "LONGITUD":-4.2958984375
x: "LATITUD":40.4469470596
x: "ID":1
x: "VALOR":34
x: "JOURNEYID":1}
x: {"FECHA":"2019-01-28 13:15:46"
x: "SERIAL":6
x: "LONGITUD":-3.2958984375
x: "LATITUD":50.4469470596
x: "ID":1
x: "VALOR":14
x: "JOURNEYID":1}]

Option 3:

Simple for to iterate over the list

def ParseIncomingDataAzure(message):
    for frame in message:
         print("x: {}".format(frame))

Output:

x: [
x: {
x: "
x: F
x: E
x: C
x: H
x: A
<....>

Possible solution:

def ParseIncomingDataAzure(message):
    print ("incoming data: {}".format(message))
    x = ast.literal_eval(message)
    for frame in x:
        print("x: {}".format(json.dumps(frame)))

Is this the correct way?

What you started with and what you are after is quite unclear. You likely want this: docs.python.org/3/library/json.html — Stephen Rauch
– Stephen Rauch ♦, Commented Jan 28, 2019 at 15:28
Sorry, I don't understand. As far as I can tell, you have a list of valid dictionaries. What's wrong with a simple for loop to iterate over them? — Valentino
– Valentino, Commented Jan 28, 2019 at 15:43
@Valentino that was I thought and the begging but I'm not sure why that way doesn't work for my example (option 3). — IoT user
– IoT user, Commented Jan 28, 2019 at 15:51
From where does your list come? If is not coded in your script but you read it from a file or something similar, then you just have a sequence of character, and you need to parse it with json as other people have said. — Valentino
– Valentino, Commented Jan 28, 2019 at 16:06
Are streaming data, so I finally added a solution to the main post just waiting to know if that is optimal according to performance. — IoT user
– IoT user, Commented Jan 28, 2019 at 16:10

Ivaylo Strandjev · Accepted Answer · 2019-01-28 15:29:07Z

2

Consider using the json package for this:

In [2]: import json                                                                                                                                                                                                                                                                       
In [3]: s = '{"FECHA":"2019-01-28 13:15:42","SERIAL":2,"LONGITUD":-4.2958984375,"LATITUD":50.4469470596,"ID":1,"VALOR":193,"JOURNEYID":1}'                                                                                                                                                
In [4]: json.loads(s)                                                                                                                                                                                                                                                                     
Out[4]: 
{'FECHA': '2019-01-28 13:15:42',
 'SERIAL': 2,
 'LONGITUD': -4.2958984375,
 'LATITUD': 50.4469470596,
 'ID': 1,
 'VALOR': 193,
 'JOURNEYID': 1}

answered Jan 28, 2019 at 15:29

Ivaylo Strandjev

71.4k18 gold badges130 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

IoT user Over a year ago

Thank you for your answer. I also tried before post the questions with json, but i'm not able to iterate over my list and return each element in json format. I supposed a simple loop over the list should be enough but I can not find how to do it

Ivaylo Strandjev Over a year ago

@IoTuser you can directly parse the whole list as json and iterate the resulting python list

IoT user · Accepted Answer · 2019-01-29 11:38:10Z

0

This both functions solved my use case:

def ParseIncomingDataAzure(message):
    print ("incoming data: {}".format(message))
    x = ast.literal_eval(message)
    for frame in x:
        print("x: {}".format(json.dumps(frame)))
        <...>

The inconvenient of the following function is that you will need to know the position where the "," is separating each element of the incoming list, so for real data where the number of fields of the json could be different in each element of the list you can not use:

def ParseIncomingDataAzure(message):
    n = 7
    frames = message.split(",")
    while frames:
        y= ','.join(frames[:n])
        frames = frames[7:]
        print (y)

edited Jan 29, 2019 at 11:38

answered Jan 29, 2019 at 8:05

IoT user

1,3104 gold badges25 silver badges51 bronze badges

4 Comments

Dan D. Over a year ago

If the message is JSON, you shouldn't use ast.literal_eval, you should use json.loads.

IoT user Over a year ago

well, the incoming message is a list of string with json format and if I try to use json.load I have an error: ´AttributeError: 'str' object has no attribute 'read'´

Dan D. Over a year ago

Note the s in loads that means it takes a string. The function load without the s takes a file.

IoT user Over a year ago

My mistake @DanD. I forgot the ´s´. The problem using ´json.loads´is that the output is string(u') and I also need to send as JSON. It should be enough to iterate over the incoming list which contains JSON elements but I'm not able to do it with a simple loop

Collectives™ on Stack Overflow

Parse list of json string in python

2 Answers 2

2 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related