Elasticsearch query not returning _scroll_id for scroll query

Question

We have an Elasticsearch cluster which all seems to be working fine except that scrolling does not work. When I do a query with a ?scroll=1m querystring no _scroll_id is returned in the results.

To check if it was anything to do with the existing Indexes I created a new Index:

PUT scroll_test

POST scroll_test/1
{
    "foo": "bar"
}

POST scroll_test/2
{
    "foo": "baz"
}

POST /scroll_test/_search?scroll=1m
{
    "size": 1, 
    "query": {
        "match_all": {}
    }
}

returns

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 3,
      "max_score": 1,
      "hits": [
         {
            "_index": "scroll_test",
            "_type": "1",
            "_id": "AV0N_R0jl33mdjPtW4uQ",
            "_score": 1,
            "_source": {
               "foo": "bar"
            }
         }
      ]
   }
}

We have just done a rolling upgrade from v5.2 to v5.4.3 (cluster health is now green). Scrolling still does not work after upgrading to v5.4.3.

I am able to execute scroll based queries on a local Elasticsearch v5.4.2 instance.

That's strange. I executed the same requests as you on ES 5.4.3 and I get a scroll_id. What are you using to send the search/scroll request? Because this looks like the ?scroll=1m parameter is cut/ignored. — Slomo
– Slomo, Commented Jul 4, 2017 at 15:25
I'm using the Sense Chrome plugin. I'm not getting _scroll_id returned using the .NET library either — Phil Peace
– Phil Peace, Commented Jul 4, 2017 at 15:26
I don't really have an idea right now. Maybe you could compare the configurations/plugins of both ES instances. Especially the modules and plugins you get listed via GET /_nodes. If there is no difference, then I'm out of ideas unfortunately. — Slomo
– Slomo, Commented Jul 5, 2017 at 11:53

nmz787 · Accepted Answer · 2023-02-08 00:04:51Z

0

After reading a lot of other questions, I took away these main ideas:

Aggregation can't scroll
- the query I copied from the Kibana "Discover" page "inspect" button had this, but I don't know what it was doing, and I was able to remove it with seemingly fine results.
Don't use scroll, and just use search_after:
- docs state: We no longer recommend using the scroll API for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).

I don't know if aggregations also miss out on search_after but I am playing it safe by not using them for now.

answered Feb 8, 2023 at 0:04

nmz787

2,2191 gold badge26 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elasticsearch query not returning _scroll_id for scroll query

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related