how to get entire table data using dynamodb scan operation in node.js

Question

Table name: users no. of rows : 100000

How can I get all records using scan operation with the help of node.js.

var params = {
    TableName:"users"};

docClient.scan(params, (error, result) => {
    if (error) {
      console.log('error', error);
    } else {
      console.log(result.Items); // x items
    }
});

but its shows part of data only. not entire table data.

cementblocks · Accepted Answer · 2018-09-10 12:54:06Z

9

You can use the eachPage function on the request.

docClient.scan(params).eachPage((err, data, done) => { ... })

You can also use the lastEvaluatedKey you get from the results of one scan as the exclusiveStartKey for the next. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Pagination

answered Sep 10, 2018 at 12:54

cementblocks

4,64520 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

samson Over a year ago

I tried docClient.scan(params).eachPage((err, data, done) => {console.log(data_;} //// it show 2892 rows and lastEvaluatedKey Count: 2892, ScannedCount: 2892, LastEvaluatedKey: { device_id: 1451 }

cementblocks Over a year ago

You have to call done when you are done processing that page. Or you can omit the done parameter from the function e.g. (err, data) => {...} and api calls will be made immediately.

samson Over a year ago

docClient.scan(params).eachPage((err, data, done) => {console.log(done); } //output: function (result) { if (result === false) return; if (response.hasNextPage()) { response.nextPage().on('complete', wrappedCallback).send(); } else { callback.call(response, null, null, AWS.util.fn.noop); } }// where and how can I use lastEvaluatedKey

cementblocks Over a year ago

try docClient.scan(params).eachPage((err, data, done) => {console.log(data); done(); }

samson Over a year ago

Its working.. thank you I try,

var count = 0; docClient.scan(params).eachPage((err, data, done) => {     if (data != null) {         for (let index = 0; index < data.Items.length; index++) {             const element = data.Items[index];             count++;             console.log("TOTAL::> " + count + " ITEM::> " + index + " DATA:: " + JSON.stringify(element));         }     }     done(); });

|

samson · Accepted Answer · 2018-09-12 07:27:09Z

4

var docClient = new AWS.DynamoDB.DocumentClient();
var params = {
    TableName: "tableName"
};
var count = 0;
docClient.scan(params).eachPage((err, data, done) => {
    if (data != null) {
        for (let index = 0; index < data.Items.length; index++) {
            const element = data.Items[index];
            count++;
            console.log("TOTAL::> " + count + " ITEM::> " + index + " DATA:: " + JSON.stringify(element));
        }
    }
    done();
});

answered Sep 12, 2018 at 7:27

samson

1,6473 gold badges15 silver badges20 bronze badges

Collectives™ on Stack Overflow

how to get entire table data using dynamodb scan operation in node.js

2 Answers 2

7 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related