AWS dynamo stream processing using Go lambda

Question

I'm using an AWS Lambda function written in Go with Dynamo stream but I don't see any way where I can marshall the old & new image to my struct because it's returning the image as map[string]DynamoDBAttributeValue.
I can check individual key and then assign it to my struct one by one, but is there an direct way to Marshall directly?

func HandleRequest(ctx context.Context, event events.DynamoDBEvent) {
    for _, record := range event.Records {

        var newStruct models.MyStruct // want to marshall newImage in this struct

        logger.Debugf("New: %#v", record.Change.NewImage)
    }
}

UPDATE: Here is the custom DynamoDBEvent that I'm using now:

type DynamoDBEvent struct {
    Records []DynamoDBEventRecord `json:"Records"`
}

type DynamoDBEventRecord struct {
    AWSRegion      string                       `json:"awsRegion"`
    Change         DynamoDBStreamRecord         `json:"dynamodb"`
    EventID        string                       `json:"eventID"`
    EventName      string                       `json:"eventName"`
    EventSource    string                       `json:"eventSource"`
    EventVersion   string                       `json:"eventVersion"`
    EventSourceArn string                       `json:"eventSourceARN"`
    UserIdentity   *events.DynamoDBUserIdentity `json:"userIdentity,omitempty"`
}

type DynamoDBStreamRecord struct {
    ApproximateCreationDateTime events.SecondsEpochTime             `json:"ApproximateCreationDateTime,omitempty"`
    Keys                        map[string]*dynamodb.AttributeValue `json:"Keys,omitempty"`
    NewImage                    map[string]*dynamodb.AttributeValue `json:"NewImage,omitempty"`
    OldImage                    map[string]*dynamodb.AttributeValue `json:"OldImage,omitempty"`
    SequenceNumber              string                              `json:"SequenceNumber"`
    SizeBytes                   int64                               `json:"SizeBytes"`
    StreamViewType              string                              `json:"StreamViewType"`
}

Can our share some code as an answer to demonstrate how you used DynamoDBEvent to solve the problem? That might prove useful for future readers. — Jeremy Caney
– Jeremy Caney, Commented Mar 29, 2021 at 18:12

cn0047 · Accepted Answer · 2021-04-14 12:49:30Z

2

Unfortunately there is no elegant way yet, sot you have to check each key/value pairs and assign them to your struct.

PS: As small sugar you can use func FromDynamoDBMap from this package and work with map[string]interface{} not with map[string]events.DynamoDBAttributeValue which is way easier.

answered Apr 14, 2021 at 12:49

cn0047

17.2k7 gold badges61 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Gnanakeethan Balasubramaniam · Accepted Answer · 2022-05-16 13:37:05Z

I have created a solution for the problem. This was crucial to my implementation since my implementation uses custom keys.

Unfortunately, the Number type has to be cast to Float64 to make sure that it is not lost.

This will result in a completely usable struct against the changeset. You may need a map of your models and a known field

Usage

newResult, _ := UnmarshalStreamImage(record.Change.NewImage)
newMarshal, _ := attributevalue.MarshalMap(newResult)
model := SomeModel{}
err := attributevalue.UnmarshalMap(newMarshal, &model)

Function Code




// UnmarshalStreamImage converts events.DynamoDBAttributeValue to struct
func UnmarshalStreamImage(attribute map[string]events.DynamoDBAttributeValue) (map[string]interface{}, error) {
    baseAttrMap := make(map[string]interface{})
    for k, v := range attribute {
        baseAttrMap[k] = extractVal(v)
    }
    return baseAttrMap, nil
}

func extractVal(v events.DynamoDBAttributeValue) interface{} {
    var val interface{}
    switch v.DataType() {
    case events.DataTypeString:
        val = v.String()
    case events.DataTypeNumber:
        val, _ = v.Float()
    case events.DataTypeBinary:
        val = v.Binary()
    case events.DataTypeBoolean:
        val = v.Boolean()
    case events.DataTypeNull:
        val = nil
    case events.DataTypeList:
        list := make([]interface{}, len(v.List()))
        for _, item := range v.List() {
            list = append(list, extractVal(item))
        }
        val = list
    case events.DataTypeMap:
        mapAttr := make(map[string]interface{}, len(v.Map()))
        for k, v := range v.Map() {
            mapAttr[k] = extractVal(v)
        }
        val = mapAttr
    case events.DataTypeBinarySet:
        set := make([][]byte, len(v.BinarySet()))
        for _, item := range v.BinarySet() {
            set = append(set, item)
        }
        val = set
    case events.DataTypeNumberSet:
        set := make([]string, len(v.NumberSet()))
        for _, item := range v.NumberSet() {
            set = append(set, item)
        }
        val = set
    case events.DataTypeStringSet:
        set := make([]string, len(v.StringSet()))
        for _, item := range v.StringSet() {
            set = append(set, item)
        }
        val = set
    }
    return val
}```

Collectives™ on Stack Overflow

AWS dynamo stream processing using Go lambda

2 Answers 2

Comments

Usage

Function Code

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Usage

Function Code

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related