DynamoDB scan function returns correct results, but with DynamoDBResponseError

Question

I'm trying to retrieve any 10 results from a table in my database using the scan function as follows:

  conn = boto.connect_dynamodb(
         aws_access_key_id=config.AWS_KEY,
         aws_secret_access_key=config.AWS_SECRET)

  table = conn.get_table('tablename')
  results = table.scan(attributes_to_get={'id'},
            max_results=10)

  for item in results:
    print item

  ./dbtest.py
   {'id': 'SkAJWDUZPSNrwepf7gdnFhExXPFABmqLjk1ADDRJuoo'}
   {'id': 'RjAVvd4SAmjtUbXEYmzBaIIDuruL5UZWEQPdcpj4XRc'}
   ...

But then I get this error at the end (after the correct results are returned):

Traceback (most recent call last):
  File "./mediatest.py", line 23, in <module>
    for item in results:
  File "/usr/lib/python2.7/dist-packages/boto/dynamodb/layer2.py", line 767, in scan
    object_hook=item_object_hook)
  File "/usr/lib/python2.7/dist-packages/boto/dynamodb/layer1.py", line 521, in scan
    return self.make_request('Scan', json_input, object_hook=object_hook)
  File "/usr/lib/python2.7/dist-packages/boto/dynamodb/layer1.py", line 121, in make_request
    retry_handler=self._retry_handler)
  File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 746, in _mexe
    status = retry_handler(response, i, next_sleep)
  File "/usr/lib/python2.7/dist-packages/boto/dynamodb/layer1.py", line 148, in _retry_handler
    json_response)
boto.exception.DynamoDBResponseError: DynamoDBResponseError: 400 Bad Request
{'Message': 'Expected null', '__type': 'com.amazon.coral.service#SerializationException'}

Am I not handling something that needs to be handled?

I'm running 2.5.2. I get this error even when I remove the attributes_to_get variable — ensnare
– ensnare, Commented Aug 29, 2012 at 15:19
@jtlebi This error seems to occur when I'm requesting less items than the table has. So in this case I'm requesting 10 items from a table of 15 items. If I request all 15 items or more, I don't get the error. I'm unsure how to handle this — ensnare
– ensnare, Commented Aug 29, 2012 at 16:22
I think you should report a bug to the Boto team. I tried 10 elements on a table with 100+ and it worked fine for me. — yadutaf
– yadutaf, Commented Aug 29, 2012 at 16:25

yadutaf · Accepted Answer · 2012-08-29 15:18:57Z

2

There is a mistake on the scan line:

results = table.scan(attributes_to_get={'id'},
        max_results=10)

instead of

results = table.scan(attributes_to_get=['id'],
        max_results=10)

EDIT:

This works with Boto 2.5.2

import boto

db = boto.connect_dynamodb()

table = db.get_table('MyTable')
res = table.scan(attributes_to_get=['id'], max_results=10)

for i in res:
    print i

edited Aug 29, 2012 at 15:18

answered Aug 29, 2012 at 14:57

yadutaf

7,2201 gold badge41 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

ensnare Over a year ago

When I make that change, I get a different error and no results: TypeError: set(['id']) is not JSON serializable

ensnare Over a year ago

I think the problem is somewhere in the loop that displays the results. When I take that part out, I don't get an error anymore.

ensnare Over a year ago

out of curiousity, how would I adjust your sample query to loop through 10 results at a time until there are no more results?

yadutaf Over a year ago

You would not use max_results=10 but request_limit=10 instead. This instructs DynamoDB to send results by pages of, at most, 10 results. Just loop over the results and Boto will do the magic for you. For lower-level control, see layer1.scan instead.

ensnare Over a year ago

I was able to get this to work by using db.layer1.scan() instead of the layer2 function. Thank you for walking me through this. I've submitted a bug report.

Collectives™ on Stack Overflow

DynamoDB scan function returns correct results, but with DynamoDBResponseError

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related