ElasticSearch query using NEST 2.x with scroll is not returning result

Question

I'm trying to retrieve all data from the elasticsearch based on a message occurrence, i figured that if i used Scroll i could loop until the document search end but the following query returns Documents = 0 but Total = 1954:

var response = client.Search<Log4Net>(s => s
                                            .Query(q => q.QueryString(qs => qs
                                             .DefaultField(m => m.Message).Query("\"" + message + "\"")))
                                             .SearchType(SearchType.Scan)
                                             .Scroll("60s"));
        while (response.Documents.Any())
        {
            var request = new BulkRequest();
            request.Refresh = true;
            request.Consistency = Consistency.One;
            request.Operations = new List<IBulkOperation>();
            foreach (var item in response.Documents)
            {
                request.Operations.Add(new BulkIndexOperation<Log4Net>(item));
            }

            var result = client.Bulk(request);

            response = client.Scroll<Log4Net>("60s", response.ScrollId);
        }

the response.Document is coming empty if i use the scroll, if i remove and get the first 1000 messages i can get the data, is anything wrong with how i'm using the Scroll?

Russ Cam · Accepted Answer · 2016-07-21 03:24:33Z

3

If you specify .SearchType(SearchType.Scan), the first response doesn't contain any documents; It will give you the total documents in the .Total property that will be returned by scrolling using the .ScrollId on the response in a scroll request.

If you don't specify .SearchType(SearchType.Scan), the first response will contain the first set of documents.

This is a difference in Elasticsearch and not NEST. SearchType.Scan is actually deprecated in 2.1.0, but is still in NEST 2.x as it supports all minor versions of Elasticsearch 2.x.

answered Jul 21, 2016 at 3:24

Russ Cam

126k34 gold badges211 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ElasticSearch query using NEST 2.x with scroll is not returning result

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related