AWS DynamoDB using QueryAsync instead of ScanAsync

Question

Using .net core api (c#) with dynamodb here. I have my dbmanager class as:

public class DbManager<T> : DynamoDBContext, IDynamoDbManager<T> where T : class
{
    private DynamoDBOperationConfig _config;

    public DbManager(IAmazonDynamoDB client, string tableName) : base(client)
    {
        _config = new DynamoDBOperationConfig()
        {
            OverrideTableName = tableName
        };
    }

    public Task<List<T>> GetAsync(IEnumerable<ScanCondition> conditions)
    {
        return ScanAsync<T>(conditions, _config).GetRemainingAsync();
    }       
}

public interface IDbManager<T> : IDisposable where T : class
{
    Task<List<T>> GetAsync(IEnumerable<ScanCondition> conditions);
}

And in my controller:

  public class ValuesController : Controller
  {
      private readonly IDbManager<MyData> _dbManager;
      public ValuesController(IDbManager<MyData> dbManager)
      {
         _dbManager = dbManager;
      }

     [HttpGet()]
     [Route("sets/filter")]
     public async Task<IActionResult> GetAllData(string id, string name)
     {
        List<ScanCondition> conditions = new List<ScanCondition>();
        conditions.Add(new ScanCondition("Id", ScanOperator.Equal, id));
        conditions.Add(new ScanCondition("Name", ScanOperator.Equal, name));

        var response = await _dynamoDbManager.GetAsync(conditions);
        return Ok(response.ToList());           
     }    
  }

My above code works fine, its just that when the data in the table gets too large the operation becomes very slow. I was told by someone to look into QueryAsync method instead of ScanAsync as Scan method scans the entire table and is slow.

I looked at QueryAsync method and checked that it takes in hashkey as one of the parameter.

My Question is how can I make use of QueryAsync instead of ScanAsync with my code above. I am calling the above endpoint from my UI and it passes different params to scanconditions and filters there. How can I pass these conditions to the QueryAsync method.

Would appreciate inputs

Brian Winant · Accepted Answer · 2019-01-12 01:28:20Z

6

You can't. Dynamo queries require at least the hash key of the table or index you want to query. You cannot query on arbitrary attributes. If you need to support that your options are:

Use a Dynamo scan and accept that the performance and monetary cost will likely be less than acceptable
Use Dynamo as a database of record but copy data into something like AWS ElasticSearch to enable random querying
Ditch Dynamo and just use a regular relational database. RDS supports many different database types
Perform an analysis of the common queries you need to do and model your data in Dynamo appropriately. This might require adding extra indexes to the table and duplicating data multiple times with different keys to support different query types.

answered Jan 12, 2019 at 1:28

Brian Winant

3,03517 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

AWS DynamoDB using QueryAsync instead of ScanAsync

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related