batch upload json to dynamodb using boto3

Question

I would like to batch upload a json file to dynamodb. At the moment I can successfully manually put items in a python file (as below) and upload to a table, however how can I amend the script to read an external json file (containing 200 items) and batch upload all 200 items to the table.

import boto3

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('exampletable')

with table.batch_writer() as batch:
    batch.put_item(
        Item={
            'ID': '2',
            'DateTime': '21/12/2017 13:16',
            'SourceDevice': '10',
            'DestinationDevice': '20',
            'DataType': 'full',
            'Activity': 'unusual'

    }
)
    batch.put_item(
        Item={
            'ID': '3',
            'DateTime': '21/12/2017 13:40',
            'SourceDevice': '10',
            'DestinationDevice': '20',
            'DataType': 'full',
            'Activity': 'unusual'

    }
)

json file contents as below

[{
"ID": "1",
"DateTime": "21/12/2017 13:16",
"SourceDevice": "10",
"DestinationDevice": "20",
"DataType": "part",
"Activity": "normal"
}, {
"ID": "1",
"DateTime": "21/12/2017 13:16",
"SourceDevice": "40",
"DestinationDevice": "25",
"DataType": "full",
"Activity": "unusual"
}]

Underrated question.

ababuji
– ababuji

2020-03-11 23:53:49 +00:00
Commented Mar 11, 2020 at 23:53 — ababuji
– ababuji, Commented Mar 11, 2020 at 23:53

Mark B · Accepted Answer · 2019-01-12 16:22:09Z

5

You would simply break that down into two tasks:

Reading the JSON from a file
Looping through the JSON, calling batch.put_item each time.

There are lots of results when you do a search for the first task. And the second task is literally just writing a loop.

A full solution would look something like this:

import json
import boto3

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('exampletable')

# Read the JSON file
with open('items.json') as json_data:
    items = json.load(json_data)

    with table.batch_writer() as batch:

        # Loop through the JSON objects
        for item in items:
            batch.put_item(Item=item)

answered Jan 12, 2019 at 16:22

Mark B

203k27 gold badges339 silver badges332 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Bat Over a year ago

thanks I was way overthinking this using boto3 and getting myself confused. Your answer worked perfectly

Collectives™ on Stack Overflow

batch upload json to dynamodb using boto3

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related