How to integrate Elasticsearch in my search

Question

We have an ad search website and all the searches are being done through entity framework directly querying the sql server database.

It was working very well when the database had around 1000 ads, but now it is reaching 300k and lots of users searching. The searches now are very slow (using raw sql didn't help much) and I was instructed to consider Elasticsearch.

I've been some tutorials and I get the idea of how it works now, but what I don't know is:

Should I stop using sql server to store the ads and start using Elasticsearch instead? What about all the other related data? Is Elasticsearch an alternative to sql server?
Each Ad has some related data stored in different tables, how would I load it to Elasticsearch? As a single json element?
I read a lot of "billions of data" handled by Elasticsearch, so I don't think I would have performance problems with 300k rows in it, correct?

Would anybody explain me better these questions?

Josué Zatarain · Accepted Answer · 2016-07-12 13:58:09Z

1

1- You could still use it; you don't want to search over the complete database, rigth? Just over the ads. It works with a no-sql format, so it is very scalable. It also works with json's so you have an easy form to access it.

2- When indexing data, you should try to add the complete necessary data in the same document(sql row), which is a single json, but in a limited way. Storage is cheap, but computing time isn't.

To index your data, you could either use filebeat, a program a bit similar to logstash, or create your own solution like, making a program that reads data from your db, and then passes it to elasticsearch in bulks.

3- Correct, 300k rows is a small quantity, but it also depends on the memory from where you are hosting elasticsearch.

Hope this helps.

answered Jul 12, 2016 at 13:58

Josué Zatarain

8571 gold badge7 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user3900456 Over a year ago

Thanks Josué, so Elasticsearch would be a read-only database that gets updated every X time through this filebeat or something else you mentioned?

Josué Zatarain Over a year ago

@user3900456 not read-only. You can still perform CRUD operations through json calls.

Collectives™ on Stack Overflow

How to integrate Elasticsearch in my search

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related