How to read object and convert to json in AWS lambda python

Question

My csv is below. I need to convert into json using AWS lambda using python

value,Name,flag
ABC,[email protected]|[email protected],Y
ABC,[email protected],Y
CDE,[email protected],N
CDE,[email protected],N

Code is below

import boto3
import csv
s3_read = boto3.client('s3')
obj = s3_read.get_object(Bucket=bucket, Key=key)
CSVcontent = obj['Body'].read().decode('utf-8').split('\n')
CSVfile = csv.reader(CSVcontent)
print('CSVfile', list(CSVfile))

My current out is

[['value', 'Name', 'flag'], ['ABC', '[email protected]|[email protected]', 'Y'], ['ABC', '[email protected]', 'Y'], ['CDE', '[email protected]', 'N'], ['CDE', '[email protected]', 'N']]

I need to merge all the same keys and same flag
I need to change all the | to different string

Expected out is below

[{'value': 'ABC', 'name': ['[email protected]','[email protected]','[email protected]'], 'flag': 'Y'},
 {'value': 'CDE', 'name': ['[email protected]','[email protected]'], 'flag': 'N'}]

There's nothing AWS or Lambda specific about this. It's just a basic Python coding question. — Mark B
– Mark B, Commented Mar 24, 2021 at 12:41
@MarkB , I worte code which is working fine in local system, but when i use obj = s3_read.get_object(Bucket=bucket, Key=key) my code is not working — sim
– sim, Commented Mar 24, 2021 at 12:50
What should happen if two rows with value=ABC actually have different flag values? — jarmod
– jarmod, Commented Mar 24, 2021 at 12:51
@sim that detail should have been included in your question then. — Mark B
– Mark B, Commented Mar 24, 2021 at 13:35

balderman · Accepted Answer · 2021-03-24 13:21:14Z

1

Starting with your current data (list(CSVfile)) and creating a list of dicts

current = [['value', 'Name', 'flag'], ['ABC', '[email protected]|[email protected]', 'Y'], ['ABC', '[email protected]', 'Y'],
           ['CDE', '[email protected]', 'N'], ['CDE', '[email protected]', 'N']]


def handle(k, v):
    if k == 'Name':
        return v.split('|')
    else:
        return v


data = []
for entry in current[1:]:
    data.append({k: handle(k,entry[idx]) for idx, k in enumerate(current[0])})
print(data)

output

[{'value': 'ABC', 'Name': ['[email protected]', '[email protected]'], 'flag': 'Y'}, {'value': 'ABC', 'Name': ['[email protected]'], 'flag': 'Y'}, {'value': 'CDE', 'Name': ['[email protected]'], 'flag': 'N'}, {'value': 'CDE', 'Name': ['[email protected]'], 'flag': 'N'}]

edited Mar 24, 2021 at 13:21

answered Mar 24, 2021 at 12:53

balderman

24k8 gold badges39 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

sim Over a year ago

[{'value': 'ABC', 'name': ['[email protected]','[email protected]','[email protected]'], 'flag': 'Y'}, {'value': 'CDE', 'name': ['[email protected]','[email protected]'], 'flag': 'N'}]

balderman Over a year ago

@sim I do not understand your comment. Can you please explain?

balderman Over a year ago

I see .. the emails... OK. Code was changed

sim Over a year ago

no its not correct still , please check this one

sim Over a year ago

'value': 'ABC', 'name': ['[email protected]','[email protected]','[email protected]'], 'flag': 'Y

Collectives™ on Stack Overflow

How to read object and convert to json in AWS lambda python

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related