Elasticsearch.net - NEST Scroll keeps returning the same results

Question

I'm trying to understand why an elasticsearch.net NEST scroll call keeps returning the same results. I have an outer loop in my C# application that tracks the current page and it's passed in along with the batch size. I've simplified the code:

List<int> ids = GetIds();
int count = _batchSize;
int currentPage = 0;

while (count == _batchSize)
{
    var results = Execute(client => client.Search<Invoice>(s => s
                .Index(indexName)
                .Query(q => q
                .Terms(n => n
                .Field(f => f.Items.FirstOrDefault().MyInformation.FirstOrDefault().ItemID)
                .Terms(ids)))
                .Size(batchSize)
                .From(currentPage * batchSize)
                .Scroll("1m")
           ));

    DoSomethingWithResults();
    count = results.Count();
    currentPage++;
}

In the above call the list of ids are ids of nested elements in a one to many relationship with the objects they're contained within. That's why I want to use scroll as I don't know how many Invoice objects will be returned. Every time this is called currentPage is incremented by 1. I had assumed that the next batch from the scroll would be returned.

I think I'm doing something wrong as I'm looking at this as more of a paging flow.

Community · Accepted Answer · 2017-05-23 12:08:51Z

4

This isn't quite how the Scroll API works.

First request is to .Search<T>() (or SearchAsync<T>()), specifying .Scroll() time. the response will contain the first set of documents
Subsequent requests are to .Scroll<T> (or ScrollAsync<T>()), passing in the scroll id from the previous request, and a scroll time to keep the scroll open on Elasticsearch. Loop sending scroll requests until a response does not contain any documents.

Take a look at this answer for an example

edited May 23, 2017 at 12:08

CommunityBot

11 silver badge

answered Feb 8, 2017 at 1:07

Russ Cam

126k34 gold badges211 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

CorribView Over a year ago

Thanks again Russ! I hadn't noticed that the first and subsequent retrievals were using different call types. Everything is working working as planed now, I'm excited!

Collectives™ on Stack Overflow

Elasticsearch.net - NEST Scroll keeps returning the same results

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related