.NET Core Web API and MongoDB Driver using Generic JSON Objects

Question

I'm creating an ASP.NET Web API to perform CRUD operations in a MongoDB database. I've been able to create a simple application based on the following Microsoft tutorial: Create a web API with ASP.NET Core and MongoDB.

This tutorial, like others I found, all use defined data models (in the above tutorial it's the Book model). In my case i need to perform CRUD operations with generic JSON objects. For example, the JSON object might be any of the following examples:

Example #1:

{_id: 1, name: 'Jon Snow', address: 'Castle Black', hobbies: 'Killing White Walkers'}

Example #2:

{_id: 2, name: 'Daenerys Targaryen', family: 'House Targaryen', titles: ['Queen of Meereen', 'Khaleesi of the Great Grass Sea', 'Mother of Dragons', 'The Unburnt', 'Breaker of Chains', 'Queen of the Andals and the First Men', 'Protector of the Seven Kingdoms', 'Lady of Dragonstone']}

The reason why I'm using a NoSQL database (MongoDB) is mainly because of the undefined data structure, and the ability to preform CRUD operations with just JSON.

As a trial and error attempt, I replaced the 'Book' model with 'object' and 'dynamic' but I get all sorts of errors regarding cast types and unknown properties:

public class BookService
{
    private readonly IMongoCollection<object> _books;

    public BookService(IBookstoreDatabaseSettings settings)
    {
        var client = new MongoClient(settings.ConnectionString);
        var database = client.GetDatabase(settings.DatabaseName);

        _books = database.GetCollection<object>(settings.BooksCollectionName);
    }

    public List<object> Get() => _books.Find(book => true).ToList();

    //public object Get(string id) => _books.Find<object>(book => book.Id == id).FirstOrDefault();

    //public object Create(object book)
    //{
    //    _books.InsertOne(book);
    //    return book;
    //}

    //public void Update(string id, object bookIn) => _books.ReplaceOne(book => book.Id == id, bookIn);

    //public void Remove(object bookIn) => _books.DeleteOne(book => book.Id == bookIn.Id);

    //public void Remove(string id) => _books.DeleteOne(book => book.Id == id);
}

Errors:

'object' does not contain a definition for 'Id' and no accessible extension method 'Id' accepting a first argument of type 'object' could be found (are you missing a using directive or an assembly reference?)

InvalidCastException: Unable to cast object of type 'd__51' to type 'System.Collections.IDictionaryEnumerator'.

So, my question is, how can I use generic JSON data types with ASP.NET Core Web API and the MongoDB Driver?

UPDATE: Based on @pete-garafano suggestion, I've decided to proceed with a POCO model.

I found an article in MongoDB's Github page explaining how to use static and dynamic data with the ASP.NET Core Driver. So I made the following changes to the Book model:

public class Book
{
    [BsonId]
    [BsonRepresentation(BsonType.ObjectId)]
    public string Id { get; set; }

    public string Name { get; set; }

    public decimal Price { get; set; }

    public string Category { get; set; }

    public string Author { get; set; }

    [BsonExtraElements]
    public BsonDocument Metadata { get; set; } //new property
}

Now I'm facing other issues, if my data is formatted exactly as the model, i'm able to list the data and create new entries in the database. But, if i try to create a new entry with the bellow format, i get an error:

{
    "Name": "test 5",
    "Price": 19,
    "Category": "Computers",
    "Author": "Ricky",
    "Metadata": {"Key": "Value"} //not working with this new field
}

System.InvalidCastException: Unable to cast object of type 'MongoDB.Bson.BsonElement' to type 'MongoDB.Bson.BsonDocument'.

Also, if i change the data format of one entry in Mongo and then try to list all results, i get the same error:

System.InvalidCastException: Unable to cast object of type 'MongoDB.Bson.BsonDocument' to type 'MongoDB.Bson.BsonBoolean'.

Based on the Mongo documents, the BsonExtraElements should allow generic/dynamic data to be attached to the model. What am I doing wrong in the new approach?

UPDATE #2: Added detailed stack trace of the error

System.InvalidCastException: Unable to cast object of type 'MongoDB.Bson.BsonDocument' to type 'MongoDB.Bson.BsonBoolean'. at get_AsBoolean(Object ) at System.Text.Json.JsonPropertyInfoNotNullable`4.OnWrite(WriteStackFrame& current, Utf8JsonWriter writer) at System.Text.Json.JsonPropertyInfo.Write(WriteStack& state, Utf8JsonWriter writer) at System.Text.Json.JsonSerializer.HandleObject(JsonPropertyInfo jsonPropertyInfo, JsonSerializerOptions options, Utf8JsonWriter writer, WriteStack& state) at System.Text.Json.JsonSerializer.WriteObject(JsonSerializerOptions options, Utf8JsonWriter writer, WriteStack& state) at System.Text.Json.JsonSerializer.Write(Utf8JsonWriter writer, Int32 originalWriterDepth, Int32 flushThreshold, JsonSerializerOptions options, WriteStack& state) at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type inputType, JsonSerializerOptions options, CancellationToken cancellationToken) at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding) at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|29_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker) at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

UPDATE #3: Added the Book service and controller code files, database Book collection and exception launched in get() result.

BookServices.cs:

public class BookService
{
    private readonly IMongoCollection<Book> _books;

    public BookService(IBookstoreDatabaseSettings settings)
    {
        var client = new MongoClient(settings.ConnectionString);
        var database = client.GetDatabase(settings.DatabaseName);

        _books = database.GetCollection<Book>(settings.BooksCollectionName);
    }

    public List<Book> Get() => _books.Find(book => true).ToList();


    public Book Get(string id) => _books.Find<Book>(book => book.Id == id).FirstOrDefault();

    public Book Create(Book book)
    {
        _books.InsertOne(book);
        return book;
    }

    public void Update(string id, Book bookIn) => _books.ReplaceOne(book => book.Id == id, bookIn);

    public void Remove(Book bookIn) => _books.DeleteOne(book => book.Id == bookIn.Id);

    public void Remove(string id) => _books.DeleteOne(book => book.Id == id);
}

BooksController.cs:

[Route("api/[controller]")]
[ApiController]
public class BooksController : ControllerBase
{
    private readonly BookService _bookService;

    public BooksController(BookService bookService)
    {
        _bookService = bookService;
    }

    [HttpGet]
    public ActionResult<List<Book>> Get() => _bookService.Get(); // error happens when executing Get()

    [HttpGet("{id:length(24)}", Name = "GetBook")]
    public ActionResult<Book> Get(string id)
    {
        var book = _bookService.Get(id);

        if (book == null)
        {
            return NotFound();
        }

        return book;
    }

    [HttpPost]
    public ActionResult<Book> Create([FromBody] Book book)
    {
        _bookService.Create(book);

        return CreatedAtRoute("GetBook", new { id = book.Id.ToString() }, book);
    }

    [HttpPut("{id:length(24)}")]
    public IActionResult Update(string id, Book bookIn)
    {
        var book = _bookService.Get(id);

        if (book == null)
        {
            return NotFound();
        }

        _bookService.Update(id, bookIn);

        return NoContent();
    }

    [HttpDelete("{id:length(24)}")]
    public IActionResult Delete(string id)
    {
        var book = _bookService.Get(id);

        if (book == null)
        {
            return NotFound();
        }

        _bookService.Remove(book.Id);

        return NoContent();
    }
}

BookstoreDb.Books:

//non-pretty
{ "_id" : ObjectId("5df2b193405b7e9c1efa286f"), "Name" : "Design Patterns", "Price" : 54.93, "Category" : "Computers", "Author" : "Ralph Johnson" }
{ "_id" : ObjectId("5df2b193405b7e9c1efa2870"), "Name" : "Clean Code", "Price" : 43.15, "Category" : "Computers", "Author" : "Robert C. Martin" }
{ "_id" : ObjectId("5df2b1c9fe91da06078d9fbb"), "Name" : "A New Test", "Price" : 43.15, "Category" : "Computers", "Author" : "Ricky", "Metadata" : { "Key" : "Value" } }

Detailed result from Mongo Driver:

[/0]:{Api.Models.Book} Author [string]:"Ralph Johnson" Category [string]:"Computers" Id [string]:"5df2b193405b7e9c1efa286f" Metadata [BsonDocument]:null Name [string]:"Design Patterns" Price [decimal]:54.93

[/1]:{Api.Models.Book} Author [string]:"Robert C. Martin" Category [string]:"Computers" Id [string]:"5df2b193405b7e9c1efa2870" Metadata [BsonDocument]:null Name [string]:"Clean Code" Price [decimal]:43.15

[/2]:{Api.Models.Book} Author [string]:"Ricky" Category [string]:"Computers" Id [string]:"5df2b1c9fe91da06078d9fbb" Metadata [BsonDocument]:{{ "Metadata" : { "Key" : "Value" } }} AllowDuplicateNames [bool]:false AsBoolean [bool]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBoolean' threw an exception of type 'System.InvalidCastException' AsBsonArray [BsonArray]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonArray' threw an exception of type 'System.InvalidCastException' AsBsonBinaryData [BsonBinaryData]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonBinaryData' threw an exception of type 'System.InvalidCastException' AsBsonDateTime [BsonDateTime]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonDateTime' threw an exception of type 'System.InvalidCastException' AsBsonDocument [BsonDocument]:{{ "Metadata" : { "Key" : "Value" } }} AsBsonJavaScript [BsonJavaScript]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonJavaScript' threw an exception of type 'System.InvalidCastException' AsBsonJavaScriptWithScope [BsonJavaScriptWithScope]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonJavaScriptWithScope' threw an exception of type 'System.InvalidCastException' AsBsonMaxKey [BsonMaxKey]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonMaxKey' threw an exception of type 'System.InvalidCastException' AsBsonMinKey [BsonMinKey]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonMinKey' threw an exception of type 'System.InvalidCastException' AsBsonNull [BsonNull]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonNull' threw an exception of type 'System.InvalidCastException' AsBsonRegularExpression [BsonRegularExpression]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonRegularExpression' threw an exception of type 'System.InvalidCastException' AsBsonSymbol [BsonSymbol]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonSymbol' threw an exception of type 'System.InvalidCastException' AsBsonTimestamp [BsonTimestamp]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonTimestamp' threw an exception of type 'System.InvalidCastException' AsBsonUndefined [BsonUndefined]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsBsonUndefined' threw an exception of type 'System.InvalidCastException' AsBsonValue [BsonValue]:{{ "Metadata" : { "Key" : "Value" } }} AsByteArray [byte[]]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsByteArray' threw an exception of type 'System.InvalidCastException' AsDateTime [DateTime]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsDateTime' threw an exception of type 'System.InvalidCastException' AsDecimal [decimal]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsDecimal' threw an exception of type 'System.InvalidCastException' AsDecimal128 [Decimal128]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsDecimal128' threw an exception of type 'System.InvalidCastException' AsDouble [double]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsDouble' threw an exception of type 'System.InvalidCastException' AsGuid [Guid]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsGuid' threw an exception of type 'System.InvalidCastException' AsInt32 [int]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsInt32' threw an exception of type 'System.InvalidCastException' AsInt64 [long]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsInt64' threw an exception of type 'System.InvalidCastException' AsLocalTime [DateTime]:'(new System.Collections.Generic.ICollectionDebugView(test).Items2).Metadata.AsLocalTime' threw an exception of type 'System.InvalidCastException' [More] Name [string]:"A New Test" Price [decimal]:43.15

Community · Accepted Answer · 2020-06-20 09:12:55Z

5

You should use BsonDocument to work with untyped data in C# with MongoDB.

private readonly IMongoCollection<BsonDocument> _books;

It isn't ideal, as C# would prefer strongly typed field names. I would recommend trying to build POCO models for the data to simplify query/update operations. If you cannot do this, you won't be able to use syntax like

_books.DeleteOne(book => book.Id == id);

You will need to instead use a dictionary type accesor syntax, like:

_books.DeleteOne(book => book["_id"] == id);

Please note that the _id field is special in MongoDB in that it must be present in every document, and unique in the collection. In the example you linked to, they provide an entity model. The Id field in this model has 2 decorators

[BsonId]
[BsonRepresentation(BsonType.ObjectId)]

These tell the driver that the Id field should be used as the _id, and that field, while a string in C# should be treated as an ObjectId by MongoDB.

If you are using a completely untyped model, you will need to be aware of the difference between _id and id and be sure to either map the fields properly or create an index on id (the former probably being what you want).

I wrote a post a while back that may be helpful to you. It covers much of the same material as the Microsoft post, but may provide you some more insight.

While the example data you provided does vary somewhat, it would still be possible to create a POCO model that would allow you to use type information in the queries. I would suggest that you investigate the feasibility of this to simplify your development. As I explained above, this isn't a requirement, but it will definitely improve the query experience.

Update To Address Extra Questions

The BsonExtraElements attribute is meant as a place for the driver to deserialize fields not in your model. For example, if you rename the Metadata field to, lets say, Foo, and rerun it. The Metadata field from the database should now actually be contained in the Foo field.

System.InvalidCastException: Unable to cast object of type 'MongoDB.Bson.BsonDocument' to type 'MongoDB.Bson.BsonBoolean'.

This exception seems to indicate something is a BsonDocument in the database, but the driver is trying to assign it to a bool value. I am unable to reproduce the error on my end. I created a document in the database just as you provided above.

I then queried using LINQPad and a simple program.

Can you perhaps provide the rest of the stack trace? It may provide us more information on which field is causing the problem. You can also try removing BsonExtraElements from the Metadata in your POCO and creating a new field just for BsonExtraElements.

Update 3

Thank you for providing the full stack trace. This lead me to an "a ha!" moment. The error is not coming from the MongoDB Driver, per se. The error is actually coming from the JSON Serializer as it accesses all the fields on the BsonDocument type.

A BsonDocument is a kind of lazy type. It doesn't "know" what it contains until you try to access it. This is handled by providing a getter for a number of different fields, all named by the type that it could contain. You can see them here.

The JSON Serializer in ASP is dutifully iterating over each field (AsBoolean, AsBsonArray, AsBsonBinaryData, etc) attempting to retrieve the value to serialize to JSON. Unfortunately, most of these are going to fail as the value in Metadata cannot cast to most (or any) of them.

I think you're going to need to either tell the JSON Serializer to ignore the Metadata field, or write a JSON serializer for BsonDocument.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Dec 3, 2019 at 22:35

Pete Garafano

4,9331 gold badge25 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Ricky Over a year ago

Thanks for the suggestion! I'm going to test this today and get back to you with feedback.

Ricky Over a year ago

Hi again! I've followed your suggestion and used a POCO model for my example. I also found an article in MongoDB's Github page showing how to mix static and dynamic data using the APS.NET Core Driver. But i'm facing new issues. I've updated my question with more details.

Ricky Over a year ago

thanks for the update. I'm going to recheck my code and get back to you with feedback.

Ricky Over a year ago

i've rechecked my code and i'm still getting the same error. If i add a new Metadata object property in a document, it throws the 'System.InvalidCastException' error. I've updated my question with the detailed stack trace.

Pete Garafano Over a year ago

@Ricky, I've provided a follow-up

|

Art · Accepted Answer · 2020-09-02 15:22:22Z

1

If anyone else happens upon this article because they received the System.InvalidCastException: Unable to cast object of type 'MongoDB.Bson.BsonDocument' to type 'MongoDB.Bson.BsonBoolean' when trying to use the [BsonExtraElements] property, the quickest and easiest fix will be to make your property an `IDictionary<string, object>' element. Once I changed my ExtraElements property to the following, everything worked perfectly:

[BsonExtraElements]
public Dictionary<string, object> ExtraElements
{
    get; set;
}

The ExtraElements property will contain the Metadata element.

answered Sep 2, 2020 at 15:22

Art

212 bronze badges

1 Comment

Matthew Spencer Over a year ago

This is a fantastic answer. I browsed for hours looking for an elegant solution. Thank you!

Collectives™ on Stack Overflow

.NET Core Web API and MongoDB Driver using Generic JSON Objects

2 Answers 2

Update To Address Extra Questions

Update 3

8 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Update To Address Extra Questions

Update 3

8 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related