0

I'm trying to create a full text search using Elasticsearch for my react web app but it is not returning all the results on query. I think I'm doing some mistake in my bulk method but I'm unable to understand what I'm doing wrong. My code is:

// Declare variable to contain body of JSON data for loading to ElasticSearch

                    let br = [];

// Function to create body for loading to ElasticSearch
                    function create_bulk (bulk_request) {
                        let obj;

                        for (let i = 0; i < res.length; i++) {
                            obj = res[i];
                            // Insert header of record
                            bulk_request.push({index: {_index: 'tweet', _type: 'tweet', _id: i+1}});
                            bulk_request.push(obj);
                        }
                        return bulk_request;
                    }

// Call function to get body for loading
                    create_bulk(br);

// Standard function of ElasticSearch to use bulk command
                    client.bulk(
                        {
                            body : br
                        }, function (err, resp) {
                            console.log(err);
                        });

                    client.search({
                        index: 'tweet',
                        type: 'tweet',
                        body: {
                            query: {
                                match: {"text" : "new york" }
                            },
                        }
                    },function (error, response,status) {
                        if (error){
                            console.log("search error: "+error)
                        }
                        else {
                            console.log("--- Response ---");
                            console.log(response);
                            console.log("--- Hits ---");
                            response.hits.hits.forEach(function(hit){
                                console.log(hit);
                            })
                        }
                    });

Data Sample which is an array of JSON data:

{date: "2014-06-04 11:30:07", text: "New York Today: Honoring our Civil Servants t.co/Lhz9fgt2FW", user_id: "nytimes "}
6
  • Between your bulk call and your search call, you need to call refresh to make sure all indexed data is available for search. Commented Jan 18, 2018 at 4:46
  • How can I do that? Commented Jan 18, 2018 at 4:52
  • @Val It got indexed fine as it is returning 10 results which is the limit I guess. Can you please tell me how to change that limit if you have any idea about it from node? I tried it from localhost and after changing the size it is returning all the results. Commented Jan 18, 2018 at 5:14
  • Yes, simply add size: 100 in your search call Commented Jan 18, 2018 at 5:15
  • Thanks, is there any way to include all search? If not then I'll use some huge number. Commented Jan 18, 2018 at 5:17

1 Answer 1

1

Between your bulk call and your search call, you need to call refresh to make sure all indexed data is available for search.

                client.bulk(
                    {
                        refresh: true,            <--- add this
                        body : br
                    }, function (err, resp) {
                        console.log(err);

                        client.search({...})      <--- and make your search call only when the bulk returns

                    });
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.