Saving a complex object to dynamoDb using python

Question

I am fairly new to python, I am trying to save a complex object to dynamoDb table, I am using boto3 as my interface with dynamodb

This is what the table I am trying to insert the object looks like :

email(primaryKey)   |  id    | lastlogin(unix timestamp) 
--------------------------------------------------------
[email protected] | someId | 923123122
--------------------------------------------------------

Now I am trying to add an object which looks like this in json :

       {
        "uri":"http://www.edamam.com/ontologies/edamam.owl#recipe_f6fa738e5eee260004693a9fae610bb2",
        "label":"Sous Vide Chicken Breast Recipe",
        "image":"https://www.edamam.com/web-img/e3d/e3d9d9a141a06d0c6adfb4e49b1224a5.jpg",
        "source":"Serious Eats","url":"http://www.seriouseats.com/recipes/2015/07/sous-vide-chicken-breast-recipe.html",
        "shareAs":"http://www.edamam.com/recipe/sous-vide-chicken-breast-recipe-f6fa738e5eee260004693a9fae610bb2/breast",
        "dietLabels":["Low-Carb"],
        "healthLabels":["Sugar-Conscious","Peanut-Free","Tree-Nut-Free","Alcohol-Free"],
        "cautions":[],
        "ingredientLines":["2 bone-in, skin-on chicken breast halves","Kosher salt and freshly ground black pepper","4 sprigs thyme or rosemary (optional)"],"ingredients":[{"text":"2 bone-in, skin-on chicken breast halves","weight":174},{"text":"Kosher salt and freshly ground black pepper","weight":0},{"text":"Kosher salt and freshly ground black pepper","weight":0.522}],
        "calories":300.59022,
        "totalWeight":174.522,
        "totalTime":165,
        "totalNutrients":
        {"ENERC_KCAL":{"label":"Energy","quantity":300.59022,"unit":"kcal"},
        "FAT":{"label":"Fat","quantity":16.1120172,"unit":"g"},
        "FASAT":{"label":"Saturated","quantity":4.63566624,"unit":"g"},
        "FATRN":{"label":"Trans","quantity":0.1827,"unit":"g"},"FAMS":{"label":"Monounsaturated","quantity":6.65065758,"unit":"g"},"FAPU":{"label":"Polyunsaturated","quantity":3.41560956,"unit":"g"},"CHOCDF":{"label":"Carbs","quantity":0.33381900000000003,"unit":"g"},"FIBTG":{"label":"Fiber","quantity":0.13206600000000002,"unit":"g"},"SUGAR":{"label":"Sugars","quantity":0.0033408000000000005,"unit":"g"},"PROCNT":{"label":"Protein","quantity":36.333235800000004,"unit":"g"},"CHOLE":{"label":"Cholesterol","quantity":111.36,"unit":"mg"},"NA":{"label":"Sodium","quantity":109.7244,"unit":"mg"},"CA":{"label":"Calcium","quantity":21.452460000000002,"unit":"mg"},"MG":{"label":"Magnesium","quantity":44.39262,"unit":"mg"},"K":{"label":"Potassium","quantity":389.73738000000003,"unit":"mg"},"FE":{"label":"Iron","quantity":1.3382862,"unit":"mg"},"ZN":{"label":"Zinc","quantity":1.3982118000000001,"unit":"mg"},"P":{"label":"Phosphorus","quantity":303.58476,"unit":"mg"},"VITA_RAE":{"label":"Vitamin A","quantity":41.90094,"unit":"µg"},"THIA":{"label":"Thiamin (B1)","quantity":0.11018375999999999,"unit":"mg"},"RIBF":{"label":"Riboflavin (B2)","quantity":0.14883960000000002,"unit":"mg"},"NIA":{"label":"Niacin (B3)","quantity":17.245886459999998,"unit":"mg"},"VITB6A":{"label":"Vitamin B6","quantity":0.9237190200000001,"unit":"mg"},"FOLDFE":{"label":"Folate equivalent (total)","quantity":7.04874,"unit":"µg"},"FOLFD":{"label":"Folate (food)","quantity":7.04874,"unit":"µg"},"VITB12":{"label":"Vitamin B12","quantity":0.5916,"unit":"µg"},"VITD":{"label":"Vitamin D","quantity":27.84,"unit":"IU"},"TOCPHA":{"label":"Vitamin E","quantity":0.47522880000000006,"unit":"mg"},"VITK1":{"label":"Vitamin K","quantity":0.8545140000000001,"unit":"µg"},"WATER":{"label":"Water","quantity":120.92544119999998,"unit":"g"}},"totalDaily":{"ENERC_KCAL":{"label":"Energy","quantity":15.029511,"unit":"%"},"FAT":{"label":"Fat","quantity":24.787718769230768,"unit":"%"},"FASAT":{"label":"Saturated","quantity":23.1783312,"unit":"%"},"CHOCDF":{"label":"Carbs","quantity":0.11127300000000001,"unit":"%"},"FIBTG":{"label":"Fiber","quantity":0.5282640000000001,"unit":"%"},"PROCNT":{"label":"Protein","quantity":72.66647160000001,"unit":"%"},"CHOLE":{"label":"Cholesterol","quantity":37.12,"unit":"%"},"NA":{"label":"Sodium","quantity":4.57185,"unit":"%"},"CA":{"label":"Calcium","quantity":2.145246,"unit":"%"},"MG":{"label":"Magnesium","quantity":10.569671428571429,"unit":"%"},"K":{"label":"Potassium","quantity":8.292284680851065,"unit":"%"},"FE":{"label":"Iron","quantity":7.434923333333334,"unit":"%"},"ZN":{"label":"Zinc","quantity":12.711016363636363,"unit":"%"},"P":{"label":"Phosphorus","quantity":43.36925142857143,"unit":"%"},"VITA_RAE":{"label":"Vitamin A","quantity":4.65566,"unit":"%"},"THIA":{"label":"Thiamin (B1)","quantity":9.181980000000001,"unit":"%"},"RIBF":{"label":"Riboflavin (B2)","quantity":11.449200000000001,"unit":"%"},"NIA":{"label":"Niacin (B3)","quantity":107.78679037499998,"unit":"%"},"VITB6A":{"label":"Vitamin B6","quantity":71.05530923076924,"unit":"%"},"FOLDFE":{"label":"Folate equivalent (total)","quantity":1.7621849999999997,"unit":"%"},"VITB12":{"label":"Vitamin B12","quantity":24.650000000000002,"unit":"%"},"VITD":{"label":"Vitamin D","quantity":185.6,"unit":"%"},"TOCPHA":{"label":"Vitamin E","quantity":3.1681920000000003,"unit":"%"},"VITK1":{"label":"Vitamin K","quantity":0.712095,"unit":"%"}}
    }

The way I am trying to add it to dynamodb right now is converting it from json to a namedtuple and then adding it to the table using boto3 api:

        rawRecipe = '{"uri":"http://www.edamam.com/ontologies/edamam.owl#recipe_f6fa738e5eee260004693a9fae610bb2","label":"Sous Vide Chicken Breast Recipe","image":"https://www.edamam.com/web-img/e3d/e3d9d9a141a06d0c6adfb4e49b1224a5.jpg","source":"Serious Eats","url":"http://www.seriouseats.com/recipes/2015/07/sous-vide-chicken-breast-recipe.html","shareAs":"http://www.edamam.com/recipe/sous-vide-chicken-breast-recipe-f6fa738e5eee260004693a9fae610bb2/breast","dietLabels":["Low-Carb"],"healthLabels":["Sugar-Conscious","Peanut-Free","Tree-Nut-Free","Alcohol-Free"],"cautions":[],"ingredientLines":["2 bone-in, skin-on chicken breast halves","Kosher salt and freshly ground black pepper","4 sprigs thyme or rosemary (optional)"],"ingredients":[{"text":"2 bone-in, skin-on chicken breast halves","weight":174},{"text":"Kosher salt and freshly ground black pepper","weight":0},{"text":"Kosher salt and freshly ground black pepper","weight":0.522}],"calories":300.59022,"totalWeight":174.522,"totalTime":165,"totalNutrients":{"ENERC_KCAL":{"label":"Energy","quantity":300.59022,"unit":"kcal"},"FAT":{"label":"Fat","quantity":16.1120172,"unit":"g"},"FASAT":{"label":"Saturated","quantity":4.63566624,"unit":"g"},"FATRN":{"label":"Trans","quantity":0.1827,"unit":"g"},"FAMS":{"label":"Monounsaturated","quantity":6.65065758,"unit":"g"},"FAPU":{"label":"Polyunsaturated","quantity":3.41560956,"unit":"g"},"CHOCDF":{"label":"Carbs","quantity":0.33381900000000003,"unit":"g"},"FIBTG":{"label":"Fiber","quantity":0.13206600000000002,"unit":"g"},"SUGAR":{"label":"Sugars","quantity":0.0033408000000000005,"unit":"g"},"PROCNT":{"label":"Protein","quantity":36.333235800000004,"unit":"g"},"CHOLE":{"label":"Cholesterol","quantity":111.36,"unit":"mg"},"NA":{"label":"Sodium","quantity":109.7244,"unit":"mg"},"CA":{"label":"Calcium","quantity":21.452460000000002,"unit":"mg"},"MG":{"label":"Magnesium","quantity":44.39262,"unit":"mg"},"K":{"label":"Potassium","quantity":389.73738000000003,"unit":"mg"},"FE":{"label":"Iron","quantity":1.3382862,"unit":"mg"},"ZN":{"label":"Zinc","quantity":1.3982118000000001,"unit":"mg"},"P":{"label":"Phosphorus","quantity":303.58476,"unit":"mg"},"VITA_RAE":{"label":"Vitamin A","quantity":41.90094,"unit":"µg"},"THIA":{"label":"Thiamin (B1)","quantity":0.11018375999999999,"unit":"mg"},"RIBF":{"label":"Riboflavin (B2)","quantity":0.14883960000000002,"unit":"mg"},"NIA":{"label":"Niacin (B3)","quantity":17.245886459999998,"unit":"mg"},"VITB6A":{"label":"Vitamin B6","quantity":0.9237190200000001,"unit":"mg"},"FOLDFE":{"label":"Folate equivalent (total)","quantity":7.04874,"unit":"µg"},"FOLFD":{"label":"Folate (food)","quantity":7.04874,"unit":"µg"},"VITB12":{"label":"Vitamin B12","quantity":0.5916,"unit":"µg"},"VITD":{"label":"Vitamin D","quantity":27.84,"unit":"IU"},"TOCPHA":{"label":"Vitamin E","quantity":0.47522880000000006,"unit":"mg"},"VITK1":{"label":"Vitamin K","quantity":0.8545140000000001,"unit":"µg"},"WATER":{"label":"Water","quantity":120.92544119999998,"unit":"g"}},"totalDaily":{"ENERC_KCAL":{"label":"Energy","quantity":15.029511,"unit":"%"},"FAT":{"label":"Fat","quantity":24.787718769230768,"unit":"%"},"FASAT":{"label":"Saturated","quantity":23.1783312,"unit":"%"},"CHOCDF":{"label":"Carbs","quantity":0.11127300000000001,"unit":"%"},"FIBTG":{"label":"Fiber","quantity":0.5282640000000001,"unit":"%"},"PROCNT":{"label":"Protein","quantity":72.66647160000001,"unit":"%"},"CHOLE":{"label":"Cholesterol","quantity":37.12,"unit":"%"},"NA":{"label":"Sodium","quantity":4.57185,"unit":"%"},"CA":{"label":"Calcium","quantity":2.145246,"unit":"%"},"MG":{"label":"Magnesium","quantity":10.569671428571429,"unit":"%"},"K":{"label":"Potassium","quantity":8.292284680851065,"unit":"%"},"FE":{"label":"Iron","quantity":7.434923333333334,"unit":"%"},"ZN":{"label":"Zinc","quantity":12.711016363636363,"unit":"%"},"P":{"label":"Phosphorus","quantity":43.36925142857143,"unit":"%"},"VITA_RAE":{"label":"Vitamin A","quantity":4.65566,"unit":"%"},"THIA":{"label":"Thiamin (B1)","quantity":9.181980000000001,"unit":"%"},"RIBF":{"label":"Riboflavin (B2)","quantity":11.449200000000001,"unit":"%"},"NIA":{"label":"Niacin (B3)","quantity":107.78679037499998,"unit":"%"},"VITB6A":{"label":"Vitamin B6","quantity":71.05530923076924,"unit":"%"},"FOLDFE":{"label":"Folate equivalent (total)","quantity":1.7621849999999997,"unit":"%"},"VITB12":{"label":"Vitamin B12","quantity":24.650000000000002,"unit":"%"},"VITD":{"label":"Vitamin D","quantity":185.6,"unit":"%"},"TOCPHA":{"label":"Vitamin E","quantity":3.1681920000000003,"unit":"%"},"VITK1":{"label":"Vitamin K","quantity":0.712095,"unit":"%"}}}'

        def _json_object_hook(d): return namedtuple('Recipe', d.keys())(*d.values())
        def json2obj(data): return json.loads(data, object_hook=_json_object_hook)

        recipe = json2obj(rawRecipe) #Convert to namedtuple

        userTable = dynamodb.Table('foodNutrition_users_table') # Add to table
        userTable.update_item(
            Key = {
                'email' : userEmail
            },
            UpdateExpression = 'Set liked_recipes = :i',
            ExpressionAttributeValues={
                ':i': [recipe],
            }
        )

But when I run this on AWS lambda I get this error :

{
  "errorMessage": "Unsupported type \"<class 'likeRecipe.Recipe'>\" for value \"Recipe(uri='http://www....

I am not sure if I am doing something wrong or if this can't be achieved by boto3 as I wasn't able to find much to help me in their documentation :https://boto3.amazonaws.com/v1/documentation/api/latest/index.html

E.J. Brennan · Accepted Answer · 2020-03-01 16:44:39Z

4

Not sure if I am not understanding your requirements, but it seems like you might be over-complicating it - this is an example of what I use:

import uuid
import boto3
import json

def lambda_handler(event, context):

    dynamodb = boto3.resource('dynamodb')

    table = dynamodb.Table('<myTableName>')

    body = json.loads(body)

    print("--- body ---")
    print(body) 

    if not "Id" in body:
        body['Id'] = str(uuid.uuid4())

    response = table.put_item(Item=body)

IN my case, the primary-key is 'Id', and if it doesn't come in with one, I assign one - pretty simple, straight json saved to DynamoDb - perhaps you can build on this to make it work for you.

Edit: and I should ad, this works with complex json objects no problem - that's how I use it.

answered Mar 1, 2020 at 16:44

E.J. Brennan

46.9k8 gold badges94 silver badges120 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Snedden27 Over a year ago

Thanks, this seems to work just something to point out ,I was trying to update an item as this seems to overwrite entire row so I used put_item() instead, also had to parse decimals to floats as dynamoDb didn't like the floats: recipe = json.loads(rawRecipe, parse_float=Decimal)

Collectives™ on Stack Overflow

Saving a complex object to dynamoDb using python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related