Implementing Azure Table's ITableEntity interface to support nested objects and custom property types

Question

Azure Table Storage supports only basic data types like (int, string, Guid, DateTime) etc. This makes it hard to store objects which themselves contain objects inside them i.e. nested objects. It is also not supported to directly store List<> and other such collection data types.

For this, I created a CustomEntity class which inherits from ITableEntity and implemented the ReadEntity and WriteEntity methods using the newly introduced TableEntity.Flatten(...) and TableEntity.ConvertBack<TResult>(...) methods. I am for now only focusing on storing nested objects.

public class CustomEntity : ITableEntity
{
    // Partition Key, Row Key, Timestamp and ETag here

    public void ReadEntity(IDictionary<string, EntityProperty> properties, OperationContext operationContext)
    {
        CustomEntity customEntity = TableEntity.ConvertBack<CustomEntity>(properties, operationContext);
        // Do the memberwise clone for this object from the returned CustomEntity object
        CloneThisObject(this, customEntity);
    }

    public IDictionary<string, EntityProperty> WriteEntity(OperationContext operationContext)
    {
        IDictionary<string, EntityProperty> flattenedProperties = TableEntity.Flatten(this, operationContext);
        flattenedProperties.Remove("PartitionKey");
        flattenedProperties.Remove("RowKey");
        flattenedProperties.Remove("Timestamp");
        flattenedProperties.Remove("ETag");
        return flattenedProperties;
    }
}

I used the above code in my model and was able to successfully insert a nested object in the Azure tables. The issue comes when I want to retrieve the same object. The code gets stuck in an infinite loop at the table.Execute(TableOperation.Retrieve(partitionKey, rowKey)) command.

Also, while the WriteEntity code was hit during insert operation of the CustomEntity object, the ReadEntity code isn't hit while retrieving the entity. What am I missing here? Any help would be greatly appreciated.

Why are you trying to create a kind of document store in an engine that is designed to be a key-value store? Why not use CosmosDB for that matter? Or just store a json document in a Table Storage column? — Peter Bons
– Peter Bons, Commented Jun 25, 2017 at 15:18
@PeterBons there are many reasons to want to use Table storage and hierarchical objects.. price, performance, query patterns, batching, etc — Igorek
– Igorek, Commented Jun 25, 2017 at 20:06

Amor · Accepted Answer · 2017-06-26 09:17:31Z

1

the ReadEntity code isn't hit while retrieving the entity.

table.Execute(TableOperation.Retrieve(partitionKey, rowKey)) method will return a DynamicTableEntity. It will not invoke ReadEntity method which you defined. If you execute a query like following, the ReadEntity method will be hit.

var  entity = table.Execute(TableOperation.Retrieve<CustomEntity>(partitionKey, rowKey)).Result;

I created a general method which could help you get the entities which have nested objects, code below is for your reference.

public static List<T> GetEntities<T>(CloudTable table, string filter) where T : ITableEntity, new()
{
    List<T> entities = new List<T>();

    DynamicTableEntity dynamicTableEntity = new DynamicTableEntity();

    dynamicTableEntity.Properties = TableEntity.Flatten(new T(), new OperationContext());

    TableQuery<DynamicTableEntity> projectionQuery = new TableQuery<DynamicTableEntity>().Where(filter);
    foreach (var dEntity in table.ExecuteQuery(projectionQuery))
    {
        T entity = EntityPropertyConverter.ConvertBack<T>(dEntity.Properties, new OperationContext());

        entity.PartitionKey = dEntity.PartitionKey;
        entity.RowKey = dEntity.RowKey;
        entity.ETag = dEntity.ETag;
        entity.Timestamp = dEntity.Timestamp;

        entities.Add(entity);
    }
    return entities;
}

Here are test code.

Nested object definition. It inherits from CustomEntity defined in your post.

public class EmployeeEntity : CustomEntity
{
    public EmployeeEntity(string lastName, string firstName)
    {
        this.PartitionKey = lastName;
        this.RowKey = firstName;
    }

    public EmployeeEntity() { }

    public string Email { get; set; }

    public Address ContactAddress { get; set; }
}

public class Address
{
    public string Province { get; set; }

    public string City { get; set; }
}

Get employee entities from table.

CloudTable table = tableClient.GetTableReference("tableName");
var entities = CustomEntity.GetEntities<EmployeeEntity>(table, "PartitionKey eq 'pk'");

answered Jun 26, 2017 at 9:17

Amor

8,5392 gold badges21 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Amor Over a year ago

Any updates? Have you tried my suggestions? If you have further questions, please feel free to let me know.

Tapish Raniwal Over a year ago

Thanks for your help. I tried your suggestion of using Retrieve<T> and I was able to successfully write and read an entity from Azure tables. However, considering that even with Flatten and ConvertBack methods, I can't use collection objects, I ended up writing my own serialization, deserialization logic.

Dogu Arslan Over a year ago

This should not be needed

DynamicTableEntity dynamicTableEntity = new DynamicTableEntity();      dynamicTableEntity.Properties = TableEntity.Flatten(new T(), new OperationContext());

Also ConvertBack should take care of PK, RK, ETag and Timestamp already as far as I remember no need for explicit assignment.

Dogu Arslan · Accepted Answer · 2018-05-18 20:26:10Z

0

It is probably a bit late but wanted to update the answer as this would have fixed your problem. I updated Object Flattener Recomposer api to support IEnumerable/ICollection type properties as well. Basically you can write pretty much anything to table storage using v2.0 of Object Flattener Recomposer api: https://www.nuget.org/packages/ObjectFlattenerRecomposer/

The api handles all transformations/serializations and deserializations transparently so no need to write your own logic.

answered May 18, 2018 at 20:26

Dogu Arslan

3,40326 silver badges46 bronze badges

Collectives™ on Stack Overflow

Implementing Azure Table's ITableEntity interface to support nested objects and custom property types

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related