Elasticsearch: Sorted scroll in python inconsistent

Question

I am a little confused with the results. I have a simple query to get the latest document added (based on sorted created date or timestamp):

query = {
            "query": {"match_all": {}},
            "sort": [
                {"created_date":  "desc"}
            ],
            "size": 1
        }

When I use helpers.scan() abstraction over Scroll() API. I get a hit which is different each time (inconsistent). My Elastic cluster is static (no new data points are being added) and the inconsistency in response is strange as I have sorted all entries and asked to return the the first hit (size 1) in my query. What am I missing here ?

Segmented · Accepted Answer · 2017-10-28 10:18:47Z

4

For future references to people who stumble upon this. The documentation on the ElasticSearch homepage may not clarify doubts here but the python driver has a very good documentation. As per helpers.scan():

By default scan does not return results in any pre-determined order. To have a standard order in the returned documents (either by score or explicit sort definition) when scrolling, use preserve_order=True. This may be an expensive operation and will negate the performance benefits of using scan

So, for use cases like this, it is better to use search() than scan()

answered Oct 28, 2017 at 10:18

Segmented

2,0542 gold badges27 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elasticsearch: Sorted scroll in python inconsistent

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related