How to use pagination with scan search type in elasticsearch?

Question

I use this config to scan my index

$client = ClientBuilder::create()->build();
$params = [
    "search_type" => "scan",    // use search_type=scan
    "scroll" => "30s",          // how long between scroll requests. should be small!
    "size" => 50,               // how many results *per shard* you want back
    "index" => "my_index",
    "body" => [
        "query" => [
            "match_all" => []
        ]
    ]
];

$docs = $client->search($params);   // Execute the search
$scroll_id = $docs['_scroll_id'];   // The response will contain no results, just a _scroll_id

// Now we loop until the scroll "cursors" are exhausted
while (\true) {

    // Execute a Scroll request
    $response = $client->scroll([
            "scroll_id" => $scroll_id,  //...using our previously obtained _scroll_id
            "scroll" => "30s"           // and the same timeout window
        ]
    );

    // Check to see if we got any search hits from the scroll
    if (count($response['hits']['hits']) > 0) {
        // If yes, Do Work Here

        // Get new scroll_id
        // Must always refresh your _scroll_id!  It can change sometimes
        $scroll_id = $response['_scroll_id'];
    } else {
        // No results, scroll cursor is empty.  You've exported all the data
        break;
    }
}

but i want order this scan by id and select result from 10.000 to 11.000 (my index have 33.000 result). How can do it?

I use from and size in this search body but it don't work.

Can you explain the real use case behind this? What are you trying to achieve? — Val
– Val, Commented Apr 23, 2016 at 4:59

Community · Accepted Answer · 2017-05-23 11:45:22Z

0

When using scan and scroll ,pagination works but data repeats in every page.so only use from to

click the link Here is the data for pagination

It retrieves only 10000 records in a index with pagination.After 10,000 documents its shows errror.so also search data inbetween two dates for better results.

example:

$params['body']['query']['filtered']['filter']['bool']['must'][]['range']['meta.datetime']['gte'] = $startdate;
$params['body']['query']['filtered']['filter']['bool']['must'][]['range']['meta.datetime']['lte'] = $enddate;

edited May 23, 2017 at 11:45

CommunityBot

11 silver badge

answered Apr 25, 2016 at 11:47

Veerendra Borra

1,28615 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to use pagination with scan search type in elasticsearch?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related