1

I need to implement search on a small database < 500 rows and i just learnt about elasticsearch and lucene . ElasticSearch takes a hell lot of RAM . So what should i do , go with traditional sql queries or use lucene or ES . Only thing is dataset is small ( < 500 ) , but i really want to learn ES. Any suggestions ?

NOTE - i am using mysql

Improve this question
4
  • when you want to learn, you should go for it, sooner or later you may need it right? Commented Mar 25, 2018 at 4:03
  • yes , but the issue is of RAM , i dont have the liberty to use separate RAM for ES Commented Mar 25, 2018 at 4:05
  • What do you mean by "hell lot of RAM"? You can assign as little or as much RAM as you need for the node to be operational. Commented Mar 25, 2018 at 7:33
  • It depends from your needs. ES performs better with fulltext search and regex search that mysql DB. You need this kind of query? Or you need only literal search on your fields? What kind of operations do you have to do on the db? Commented Mar 26, 2018 at 8:52

2 Answers 2

Reset to default
2

Primarily I think ES is an overkill here for 500 records (unless your each record is 100MB size!), the answer remains in various other questions

1) These 500 records changes very frequently? If these records are static or do not change frequently, you should cache this data in your application using plain simple Lucene and run your query against cached data.

2) Do you really need free-text search capability? (e.g. fuzzy matches, relevancy sorting and so on.) If not, running MqSQL queries with 500 records table will be extremely fast too considering Full table scan.

3) Also what is your expected "search" rate on this data? are you firing search on these 500 records 100 million times a day or 100 times a day? If you are searching at extremely high rate (>100000 times a day, please don't fire MySQL queries, use cached version of data as suggested in #1)

There are various points also to be considered as 1) SLA expectation in searching. 2) If 500 records are updated very frequently, how sooner your "search" client is expecting data to be available for search after update?

I would go for ES here only if your each record is larger data (10s of MBs) and your data updates very frequently.

Happy to discuss more on this.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

You should figure out what are your text search needs. MySQL has basic text search with limited options. For example, with current version of MySQL you can not select a ranking function for your text search whereas Lucene offers a wide variety of them.

About the RAM problem, it really depends on how you implement your system and index. With Lucene, you can build an in-memory index which will use a lot of RAM or you can store the index on the disk and let Lucene take care of the rest.

And finally, if you want to learn it, you should go for it :)

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.