22

I have an array of Job ID's.

[ '01', '02', '03', '04' ]

Currently I am looping through the array and executing a Query for each item in the array to get the job details.

Is there a way to use a single query to get all the jobs whose Partition Key is in the array ?

6
  • One thing I am trying is to change to Global Secondary Index to include ALL_KEYS Commented Mar 19, 2017 at 23:04
  • My above comment is wrong it should say: One thing I am trying is to change to Global Secondary Index ProjectionType to ALL. This seems to work. So when I get items by my secondary index I get all the details, so querying multiple ID's from an array is no longer necessary Commented Mar 19, 2017 at 23:10
  • 1
    GSI projecttype ALL is to include all attributes on main table onto GSI. I am not sure how it resolved the above problem. Commented Mar 20, 2017 at 7:54
  • It doesn't answer my question, no. But it eliminates the need to query multiple partition keys, because I already have the data I require by including all attributes on the GSI. But by all means if you have a solution to the above question feel free to post it @notionquest Commented Mar 21, 2017 at 17:18
  • 1
    You can use batch get item API .. docs.aws.amazon.com/amazondynamodb/latest/APIReference/… Commented Mar 21, 2017 at 17:37

3 Answers 3

12

There are a few options, each with some pros/cons:

  1. BatchGetItem as @notionquest pointed out. This can fetch up to 100 items or 16MB of data in a single call, but you need to provide all of the key values (both partition and sort for each item, if your table uses a composite key schema).

  2. TransactGetItems - this can retrieve up to 25 items in a single call and as the name implies, is transactional, so the entire operation will fail if there is another pending operation on any of the items being queries. This can be good or bad depending on your use case. Similar to BatchGetItem, you need to provide all key attributes.

  3. Query, which you are already using. Query has high performance but only supports 1 key per request (partition key required, sort key optional). Will return up to 1MB of data at a time, and supports paginated results.

If your jobs table has a composite key (partition + sort), Query is the best option in terms of performance and no constraint on specifying the sort key values. If the table only has a partition key, then BatchGetItems is probably the best bet (assuming the each job item is relatively small in size, and you expect less than 100 total jobs to be returned). If either of those assumptions is incorrect, multiple Querys would be the best option.

Sign up to request clarification or add additional context in comments.

Comments

10

You can use Batch Get Item API to get multiple items from DynamoDB table based on key attributes (I.e. partition key and sort key if available).

1 Comment

Batch get item requires both partition and sort keys for all items, so it only works if the table has only a partition key. If the table has a composite key (partition + sort), then there's no way to do this query.
8

You can use partiQL for this use case:

SELECT * 
FROM <TABLE_NAME> 
WHERE "Id" IN [ARRAY]

But do note that partiQL has length constraints: Minimum length of 1. Maximum length of 8192.

let statement = {
    "Statement": "SELECT * \nFROM <TABLE_NAME> \nWHERE \"Id\" IN [ARRAY]"
}

let result = await dynamoDbClient.executeStatement(statement).promise();

1 Comment

I'm curious - how many RCUs does this cost? Under the hood, is this just doing multiple Query operations - one for each element in the array?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.