How to search dynamic elements of array in ElasticSearch using C# NEST?

Question

I have these data in my elasticsearch with this structure.

How do I search firstName, middleName and surname from this array? Note: The NameDetails array length is dynamic. Person A could have just one element of NameDetails. Person B could have 3 elements of NameDetails.

At the moment, I could only search by gender. I am using NEST nuget C#. This is my query.

var response = await _elasticClient.SearchAsync<Person>(s => s
          .Query(q => q
              .Bool(b => b.Must(
                mu => mu
                .Match(m => m
                 .Field(f => f.Gender)
                 .Query(Gender)
                )

                )
              )
          )
        );

In NEST, I tried with this code but return no result.

        var response = _elasticClient.Search <Model.Entities.Split.Person.Person> (s => s
            .Index("person")
            .Query(q => q
                .Match(m => m
                    .Field(f => f.NameDetails.Name[0].NameValue.FirstName)
                    .Query("Fawsu")
                )
            )
        );

But if I directly run the DSL query at ElasticSearch with query below, it returns result.

GET /person/_search
{
  "query": {
    "match": {
          "nameDetails.nameValue.firstName": {
            "query": "Fawsu"
          }
        }
    }
  }
}

or

GET /person/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "fuzzy": {
            "nameDetails.nameValue.surname": {
              "value": "Pibba",
              "fuzziness": "AUTO"
            }
          }
        },
        
        {
          "fuzzy": {
            "nameDetails.nameValue.firstName": {
              "value": "Fawsu",
              "fuzziness": "AUTO"
            }
          }
        }
      ]
    }
  }
}

Russ Cam · Accepted Answer · 2021-03-30 00:43:26Z

3

It's useful to see how this is mapped to a POCO

public class Person
{
    public string Gender { get; set; }
    public string ActiveStatus { get; set; }
    public string Deceased { get; set; }
    public List<Name> NameDetails { get; set; }
}

public class Name
{
    public List<NameValue> NameValue { get; set; }
    public string NameType { get; set; }
}

public class NameValue
{
    public string FirstName { get; set; }
    public string MiddleName { get; set; }
    public string LastName { get; set; }
}

(You might map some of these fields as types other than string, I'll leave that as an exercise for the reader).

To search on first name

var client = new ElasticClient();


var response = client.Search<Person>(s => s
    .Index("people")
    .Query(q => q
        .Match(m => m
            .Field(f => f.NameDetails[0].NameValue[0].FirstName)
            .Query("Fawsu")
        )
    )
);

which yields the query

POST http://localhost:9200/people/_search?pretty=true&typed_keys=true 
{
  "query": {
    "match": {
      "nameDetails.nameValue.firstName": {
        "query": "Fawsu"
      }
    }
  }
}

The expression in .Field(f => f...) is an expression to build the path to "nameDetails.nameValue.firstName", which will look for a match in any first name of name value and name details. The fact it uses indexers does not mean that it is targeting the first name value of the first name details, but are simply a way to traverse the object graph to build the expression.

To build a composite query to target multiple values of the same name value, both Name and NameValue would need to be mapped as nested data types, and then nested queries would be used.

For a single field

var response = client.Search<Person>(s => s
    .Index("people")
    .Query(q => q
        .Nested(n => n
            .Path(f => f.NameDetails)
            .Query(nq => nq
                .Nested(nn => nn
                    .Path(f => f.NameDetails[0].NameValue)
                    .Query(nnq => nnq
                        .Match(m => m
                            .Field(f => f.NameDetails[0].NameValue[0].FirstName)
                            .Query("Fawsu")
                        )
                    )
                )
            )
        )
    )
);

For multiple fields

var response = client.Search<Person>(s => s
    .Index("people")
    .Query(q => q
        .Nested(n => n
            .Path(f => f.NameDetails)
            .Query(nq => nq
                .Nested(nn => nn
                    .Path(f => f.NameDetails[0].NameValue)
                    .Query(nnq => nnq
                        .Bool(b => b
                            .Must(m => m
                                .Match(m => m
                                    .Field(f => f.NameDetails[0].NameValue[0].FirstName)
                                    .Query("Fawsu")
                                ), m => m
                                .Match(m => m
                                    .Field(f => f.NameDetails[0].NameValue[0].LastName)
                                    .Query("Pibba")
                                )
                            )
                        )
                    )
                )
            )
        )
    )
);

the latter of which results in the query

POST http://localhost:9200/people/_search?pretty=true&typed_keys=true 
{
  "query": {
    "nested": {
      "path": "nameDetails",
      "query": {
        "nested": {
          "path": "nameDetails.nameValue",
          "query": {
            "bool": {
              "must": [
                {
                  "match": {
                    "nameDetails.nameValue.firstName": {
                      "query": "Fawsu"
                    }
                  }
                },
                {
                  "match": {
                    "nameDetails.nameValue.lastName": {
                      "query": "Pibba"
                    }
                  }
                }
              ]
            }
          }
        }
      }
    }
  }
}

answered Mar 30, 2021 at 0:43

Russ Cam

126k34 gold badges211 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Steve Over a year ago

The solutions looks good. Just wondering, how do I check whether it is mapped as nested data types?

Russ Cam Over a year ago

You can get the index mapping and check the type for the field: elastic.co/guide/en/elasticsearch/reference/current/…. Here's the get mapping API test for NEST: github.com/elastic/elasticsearch-net/blob/…. It uses a visitor over fields to assert mapping type counts, but you can traverse to the properties directly on the response and assert the type

Russ Cam Over a year ago

Please copy and paste code/text rather than screenshots - it's easier to read, work with, and is indexable/searchable. It looks like both nameDetails and nameValue are mapped as "object", which is OK if you want to search for single fields in isolation but will not work if you want to search across fields of the same nameDetail or nameValue.

Russ Cam Over a year ago

The expression .Field(f => f.NameDetails[0].NameValue[0].FirstName) will search through all first names of all name values and all name details

Russ Cam Over a year ago

which query? do the examples here help?

|

Hooman Bahreini · Accepted Answer · 2021-03-30 00:49:07Z

1

If you want to search something like:

firstName == "John" AND middleName == "Ben" AND lastName == "smith"

Then you can use TermQuery to search firstName, middleName and lastName:

I am more familiar with Object Initializer Syntax (which requires using static Nest.Infer)

using static Nest.Infer;

var query = new TermQuery
{
    Name = "my firstName query",
    Field = Field<Person>(p => p.firstName),
    Value = "John"
};

You would need to write 2 more queries for middleName and lastName and then AND all of these queries.

If you want to search something like:

firstName == "John" OR middleName == "John" OR lastName == "John"

Then you can use Multi-match Query:

using static Nest.Infer;

var query = new MultiMatchQuery
{
    Name = "My Multi-match query",
    Fields = Field<Person>(p => p.firstName, 1.1) // 1.1 is boost (default is 1)
        .And<AdDocument>(p => p.middleName, 1)
        .And<AdDocument>(p => p.lastName, 1)
    Query = "John"
    //Fuzziness = Fuzziness.Auto
};

edited Mar 30, 2021 at 0:49

answered Mar 29, 2021 at 23:27

Hooman Bahreini

15.7k11 gold badges87 silver badges158 bronze badges

1 Comment

Steve Over a year ago

But firstname is nested in nameDetails. Basically, one person could have many set of names. Example, the person I posted has 2 set of names, firstName: Fawsu and surName: Pibba AND firstName: Fausu and surName: Pibba.

Collectives™ on Stack Overflow

How to search dynamic elements of array in ElasticSearch using C# NEST?

2 Answers 2

8 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

8 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related