Indexing parallel arrays in Mongodb

Question

I am starting to use MongoDb C#, but have run into a slight issue.

So I have a document with 2 embedded collections(of distinct types). I want to search on fields of both of these collections however I have discovered that if I try to index the searchable fields on the 2 collections I get "cannot index parallel arrays". Reading the Mongodb documentations on multikey indexes I discovered that this is indeed a limitation.

My question is what is the normal work around regarding this issue? I cant really combine these collections since they are pretty distinct? What pattern should I follow?

public class Capture
{
        [BsonId]
        public Guid Id { get; set; }
        ...Some other fields
        public IList<CustomerInformation> CustomerInformations { get; set; }
        public IList<VehicleLicenseDisk> VehicleLicenseDisks { get; set; }
}

wdberkeley · Accepted Answer · 2014-07-31 19:11:29Z

4

Before talking about possible workarounds, I just want to highlight why MongoDB has chosen to enforce this restriction on indexing parallel arrays. When you index an array in MongoDB, it creates a multikey index with one key per array element. Therefore, if you create a compound index on two arrays, one with M distinct values and one with N distinct values, the index essentially has MN keys. This is very bad- it's nonlinear in the number of distinct array elements. Consider the amount of work it takes to maintain an index like this when you add or remove array elements.

OK, justification aside, to work around this restriction it will be helpful to use the current MongoDB version (2.6), which supports index intersection. One can create an index on CustomerInformations and VehicleLicenseDisks and then MongoDB can use both indices and intersect them to serve queries that have restrictions on both.

If you are, for whatever reason, stuck with MongoDB < 2.6, then your options are either to consider redesigning the schema or to depend on indexes that use at most one of the array fields.

answered Jul 31, 2014 at 19:11

wdberkeley

11.7k1 gold badge30 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Murdock Over a year ago

Thanks. I am using >2.6, but I am not sure how I would go about creating separate indexes on embedded tables (as well as the properties of these collections). How would it differ from x.CreateIndex("CustomerInformations.FieldA", "VehicleLicenseDisks .FieldB"); which currently gives that error?

wdberkeley Over a year ago

Create an index on each individually: x.CreateIndex("CustomerInformations.FieldA") and x.CreateIndex("VehicleLicenseDisks .FieldB"). The query planner will be aware of both and will choose to use the intersection if it determines that is the optimal way.

Ehsan Sarshar Over a year ago

@wdberkeley have you test it ?

alernerdev · Accepted Answer · 2014-07-31 14:07:20Z

0

It will help if you think about MongoDB schema concerns in terms of MongoDB -- not in terms of programming language objects. Do the arrays really need to be arrays? Can they be replaced with concrete field names? In your case, why is CustomerInformation an array? What does Capture object really represent? You might have to split out, as an example, CustomerInformation into a separate collection where each record contains a link/reference back to the Capture document it belongs to. I dont know the details of what you are trying to model, but whatever it is, forget about object oriented programming and put on a MongoDB hat -- objects will come later.

answered Jul 31, 2014 at 14:07

alernerdev

2,0643 gold badges20 silver badges37 bronze badges

2 Comments

Murdock Over a year ago

I have kindof found a work around for my issue however still having some issues with performance stackoverflow.com/questions/25061040/mongodb-search-performance

Murdock Over a year ago

I feel that this is the right representation since a capture is the only unit that will ever really be returned from the store. The other detail is specific to the capture and will never exist without it. I have read lots of material on embedding vs referencing and I feel confortable that this is the correct decision However not being able to search on the fields lead me to the above solution and a new set of issues

Collectives™ on Stack Overflow

Indexing parallel arrays in Mongodb

2 Answers 2

3 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related