how to write aws dynamodb scan filter

Question

I have one dynamodb table and one record is like belwo:

{
    "Items": [
        {
            "toObjectId": {
                "S": "5678"
            },         
            "keyValue": {
                "S": "7890"
            },
            "aTypeId": {
                "S": "test"
            },
            "aws:rep:deleting": {
                "BOOL": false
            },
            "currentAt": {
                "N": "1582476260000"
            },
            "keyTypeId": {
                "S": "test2"
            },
            "aId": {
                "S": "1234"
            },
            "aTypeId_keyTypeId_keyValue_currentAt": {
                "S": "test_test2_7890_1582476260000"
            },
            "fromObjectId": {
                "S": "12345"
            },

        }
    ],
    "Count": 2562,
    "ScannedCount": 2562,
    "ConsumedCapacity": null

}

How can I write one aws dynamodb scan/query filter with aws cli to just get aTypeId and aId when aTypeId is "test"? And

Primary partition key is aId (String)
Primary sort key is aTypeId_keyTypeId_keyValue_currentAt (String)

I have tried below but no lucky with it

aws dynamodb query \
    --table-name test \
    --key-condition-expression "aTypeId = :aTypeId" \
    --expression-attribute-values '{
        ":aTypeId": { "S": "test" }     
    }'

What is the Primary Key for the table? And does the table has Sort Key too? — Popoi Menenet
– Popoi Menenet, Commented Mar 14, 2020 at 14:16
Thanks a lot @PopoiMenenet, I have updated the question with the sort key and primary key. — Anne Liu
– Anne Liu, Commented Mar 19, 2020 at 8:53

hoangdv · Accepted Answer · 2020-03-19 09:10:12Z

5

You field is not in a key or GSI (Global secondary index), then I think you have to use scan method to get object by aTypeId,

The query will like:

aws dynamodb scan \
     --table-name test \
     --filter-expression "aTypeId = :typeValue" \
     --projection-expression "aTypeId, aId" \
     --expression-attribute-values '{":typeValue":{"S":"test"}}'

If you get back result with LastEvaluatedKey value, this mean you need take one or more query to get all data:

aws dynamodb scan \
     --table-name test \
     --filter-expression "aTypeId = :typeValue" \
     --projection-expression "aTypeId, aId" \
     --starting-token VALUE_NEXT_TOKEN_OF_LAST_QUERY \
     --expression-attribute-values '{":typeValue":{"S":"test"}}'

But, I recommended that create a GSI with aTypeId is hash key will be better.

answered Mar 19, 2020 at 9:10

hoangdv

16.3k4 gold badges31 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Popoi Menenet · Accepted Answer · 2020-03-19 09:19:00Z

Since aId is a Primary Key and you are trying to search for it, you can only scan through the table (which is might be very expensive operation). Try below command to scan the item

aws dynamodb scan \
 --table-name test \
 --projection-expression "aTypeId, aId" \
 --filter-expression "aTypeId = :aTypeId" \
 --expression-attribute-values '{":aTypeId":{"S":"test"}}'

As a precaution, a single Scan request will only return up to 1MB size of data. If there is anymore data to scan, the response from the request will append LastEvaluatedKey and that tells you to run Scan request again to find the item, but this time with --starting-token and the value is exactly LastEvaluatedKey

If this type of operation is a routine in your case, it will be better to use Global Secondary Index and use aTypeId as Primary Key

Collectives™ on Stack Overflow

how to write aws dynamodb scan filter

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related