DynamoDB; Fetch all data in a table - PHP

Question

I found some help in how to get this done via the answer linked here. But it doesn't fit my use case. My goal is to fetch all records in a DynamoDB Table (usernames and emails). Looking through doc I have to use LastEvaluatedKey or ExclusiveStartKey to implement pagination. Any guidance will be appreciated.

Thanks

halfer · Accepted Answer · 2018-07-25 06:58:35Z

1

This query will able to get all records from DynamoDB Table.

function scanAllData($table,$limit){

  $result = $this->getClientdb()->scan(array(
        'TableName' => $table,
        'Limit' => $limit,
        'Select' => 'ALL_ATTRIBUTES'                
     ),
    array('limit' => $limit),
  );
    return $result['Items'];
}

You can call this function like this. for example you have 'users' table and columns are usernames and emails

$getobj = $this->scanAllData('users','10');

foreach($getobj as $cols){

   echo $cols['usernames']['S'];
   echo $cols['emails']['S'];    

}

edited Jul 25, 2018 at 6:58

halfer

20.2k20 gold badges110 silver badges207 bronze badges

answered Sep 22, 2015 at 10:40

Hassan Raza

6791 gold badge10 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

moh_abk Over a year ago

What if the records fetched are more than 1MB? Do you not need to implement pagination?

Hassan Raza Over a year ago

for pagination use jquery datatable and manage the pagination. Its responsive and advance feature.

moh_abk Over a year ago

When i say pagination I don't mean website pagination. Based on your code if my table contains 1 million records will the code echo all 1 million?

Hassan Raza Over a year ago

Yes good question. Now check i update my answer that will utilize limit on the dynamodb record.

moh_abk Over a year ago

I don't want to implement limit. I want to fetch all. Will your original answer fetch all?

|

mkobit · Accepted Answer · 2015-09-23 23:52:33Z

I don't know PHP, but here is a Java 8 example ran against DynamoDB Local with a dependency on compile group: 'com.amazonaws', name: 'aws-java-sdk', version: '1.10.20'. This shows how Limit (mostly, it doesn't show how filters effect the returned items), LastEvaluatedKey, and ExclusiveStartKey work. You can see how the parameters are set, and compare it to the PHP example:

import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.ScalarAttributeType;
import com.amazonaws.services.dynamodbv2.model.ScanRequest;
import com.amazonaws.services.dynamodbv2.model.ScanResult;
import com.amazonaws.services.dynamodbv2.util.Tables;
import com.google.common.collect.ImmutableList;

import java.util.Map;

public class ScanExample {

    private static final String TABLE_NAME = "test_table";
    private static final String HASH_ATTR_NAME = "hash";

    public static void main(String[] args) throws InterruptedException {
        AWSCredentials awsCredentials = new BasicAWSCredentials("key", "secret");
        AmazonDynamoDB dynamoDBClient = new AmazonDynamoDBClient(awsCredentials);
        dynamoDBClient.setEndpoint("http://localhost:4000");

        if (Tables.doesTableExist(dynamoDBClient, TABLE_NAME)) {
            dynamoDBClient.deleteTable(TABLE_NAME);
        }

        CreateTableRequest createTableRequest = new CreateTableRequest();
        createTableRequest.setTableName(TABLE_NAME);
        createTableRequest.setProvisionedThroughput(new ProvisionedThroughput(50l, 50l));

        createTableRequest.withKeySchema(
                ImmutableList.of(new KeySchemaElement(HASH_ATTR_NAME, KeyType.HASH)))
                .withAttributeDefinitions(ImmutableList.of(
                                new AttributeDefinition(HASH_ATTR_NAME, ScalarAttributeType.N))
                );

        dynamoDBClient.createTable(createTableRequest);
        Tables.awaitTableToBecomeActive(dynamoDBClient, TABLE_NAME);

        final Table table = new DynamoDB(dynamoDBClient).getTable(TABLE_NAME);
        createItems(table, 8);

        final int limit = 3;
        performScan(dynamoDBClient, limit);
    }

    private static void performScan(final AmazonDynamoDB client, final int limit) {
        ScanRequest scanRequest = new ScanRequest(TABLE_NAME)
                .withLimit(limit);

        Map<String, AttributeValue> exclusiveStartKey = null;
        do {
            final ScanResult scanResult = client.scan(scanRequest);
            System.out.println("With exclusiveStartKey=" + exclusiveStartKey);
            scanResult.getItems().forEach(System.out::println);
            exclusiveStartKey = scanResult.getLastEvaluatedKey();
            System.out.println("Result lastEvaluatedKey=" + exclusiveStartKey);
            // Reusing same request object, just setting the start key
            scanRequest.setExclusiveStartKey(exclusiveStartKey);
            System.out.println();
        } while(exclusiveStartKey != null);
    }

    private static void createItems(final Table table, final int n) {
        for (int i = 0; i < n; i++) {
            table.putItem(new Item().withNumber(HASH_ATTR_NAME, i));
        }
    }
}

Example output:

With exclusiveStartKey=null
{hash={N: 2,}}
{hash={N: 1,}}
{hash={N: 3,}}
Result lastEvaluatedKey={hash={N: 3,}}

With exclusiveStartKey={hash={N: 3,}}
{hash={N: 5,}}
{hash={N: 7,}}
{hash={N: 0,}}
Result lastEvaluatedKey={hash={N: 0,}}

With exclusiveStartKey={hash={N: 0,}}
{hash={N: 6,}}
{hash={N: 4,}}
Result lastEvaluatedKey=null

Collectives™ on Stack Overflow

DynamoDB; Fetch all data in a table - PHP

2 Answers 2

10 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

10 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related