Insert multiple lines of json in DynamoDB

Question

I have the following json file

{"columnwithoutname":"structureet","nofinesset":810001792,"nofinessej":810001784}
{"columnwithoutname":"structureet","nofinesset":670797117,"nofinessej":670010339}

I want to insert it in DynamoDB using Lambda. This is what I did:

def lambda_handler(event, context):
    bucket=event['b']
    file_key=event['c']
    table=event['t']
    recList=[]
    s3 = boto3.client('s3')
    dynamodb = boto3.client('dynamodb')
    obj= s3.get_object(Bucket=bucket, Key=file_key)
    recList=obj['Body'].read().split('\n')
    for row in recList:
        response = dynamodb.put_item(TableName='test-abe', Item=row)

But I have this error:

 "errorMessage": "Parameter validation failed:\nInvalid type for parameter Item

Apparently I also need to precise the type of each column so it could be accepted. Anyway to do it automatically? I want all columns to be strings. thank you

alexis-donoghue · Accepted Answer · 2020-03-10 13:28:53Z

1

DynamoDB client expects Item parameter to be a dict (per https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Client.put_item)

When you do recList = obj['Body'].read().split('\n') what you get is a list of str, so passing str to a param expecting dict will obviously fail.

One more thing to consider is that DynamoDB client expects item in a very specific format, with explicitly specified attribute datatypes. If you want to read JSON and simply write it, I suggest using DynamoDB resource, something like this:

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(table_name)
table.put_item(item)

Table.put_item() accepts simple dict with no need to specify data type for every attribute, so you can simply read from file, convert it to dict and send it away (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.put_item).

answered Mar 10, 2020 at 13:28

alexis-donoghue

3,42713 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Haha Over a year ago

Thank you. Can I just convert my json rows to dict and insert them using table.put_item?

alexis-donoghue Over a year ago

@Haha generally that should be enough. You can use json.loads(row) to load json into dict and test it.

Horatiu Jeflea · Accepted Answer · 2020-03-10 13:25:56Z

1

You need to manipulate each row in order to have the 'S' format:

import json

for row in recList:
    row_dict = json.loads(row)
    ddb_row_dict = {k:{"S": v} for (k,v) in row_dict.items()}
    response = dynamodb.put_item(TableName='test-abe', Item=ddb_row_dict)

answered Mar 10, 2020 at 13:25

Horatiu Jeflea

7,4747 gold badges42 silver badges69 bronze badges

Collectives™ on Stack Overflow

Insert multiple lines of json in DynamoDB

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related