24

I'm creating a microservice to handle the contacts that are created in the software. I'll need to create contacts and also search if a contact exists based on some information (name, last name, email, phone number). The idea is the following: A customer calls, if it doesn't exist we create the contact asking all his personal information. The second time he calls, we will search coincidences by name, last name, email, to detect that the contact already exists in our DB. What I thought is to use a MongoDB as primary storage and use ElasticSearch to perform the query, but I don't know if there is really a big difference between this and querying in a common relational database.

EDIT: Imagine a call center that is getting calls all the time from mostly different people, and we want to search fast (by name, email, last name) if that person it's in our DB, wouldn't ElasticSearch be good for this?

Improve this question

4 Answers 4

Reset to default
34

A relational database can store data and also index it.

A search engine can index data but also store it.

Relational databases are better in read-what-was-just-written performance. Search engines are better at really quick search with additional tricks like all kinds of normalization: lowercase, ä->a or ae, prefix matches, ngram matches (if indexed respectively). Whether its 1 million or 10 million entries in the store is not the big deal nowadays, but what is your query load? Well, there are only this many service center workers, so your query load is likely far less than 1qps. No problem for a relational DB at all. The search engine would start to make sense if you want some normalization, as described above, or you start indexing free text comments, descriptions of customers.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Yes, agree. The type of search capabilities between ES and a normal RDBMS is different. In ES searches are really fast, but have also a lot search strategies that a RDMS doesn't support. That could also be interesting for smaller amounts of data.
There is no significant difference for small amounts of data. ElasticSearch is best for document stores. To manage structured, relational data as users on a call center, use relational databases, not MongoDB, nor ElasticSearch. Any major RDBMS already support inverted indices (the technology behind ElasticSearch)
20

If you don't have a problem with performance, then keep it simple and use 1 single datastore (maybe with some caching in your application).

Elasticsearch is not meant to be a primary datastore so my advice is to use a simple relational database like Postgres and use simple SQL queries / a ORM mapper. If the dataset is not really large it should be fast enough.

When you have performance issues on searches you can use a combination of relation db and Elasticsearch. You can use Elasticsearch feeders to update ES with your data in you relational db.

Improve this answer

7 Comments

but what if I have a db that handles maybe a million contacts, querying by text shouldn't be slower in a regular database than elasticsearch? About caching, it's not so common that the same contact calls twice in a short period of time (imagine a call center). I don't know if caching will increase the response time and I don't know if looking for email, name and last name will be fast
yes, of course caching have also a small price (in performance and complexity) and you only should use it if you really win a lot with it (reuse same data frequently). Everything depends on the situation what's the best solution but I like to keep it as simple as possible. You can always do extra things when you run into performance problems. If you have a representative dataset you do a proof of concept and measure performance. That can help to decide if you need ES or not. Don't forget that the maintenance becomes more complex with ES and the required sync. system.
So I'll need to do a proof of concept. I'll probably use ES phonetic matching plugin (as it's intended for a call center) and will check if it's way faster than PostgreSQL
Keep also in mind that you are able to analyse data with Kibana when using Elasticsearch. That could be also interesting.
@emilly Elasticsearch is build for fast searches and is really good in that specific task, but ES isn't fully ACID compliant. You can use a RDBMS as primary store and synchronize the data to Elasticsearch for fast searches (using feeders(rivers) for example). For big data there are better solutions on the market like Cassandra, MongoDB, Neo4J, ...
|
3

Indexed RDBMS works well for search

If your data is structured i.e. columns are clearly defined, searching 1 million records will also not be a problem in RDBMS.

When to use Elastic

  • Text Search: Searching words across multiple properties (e.g. description, name etc.)
  • JSON Store and search: If data being stored is in json format and later needs to be searched
  • Auto Suggestions: Elastic is better at providing autocomplete suggestions

Elastic as an application data provider

Elastic should not be seen as data store, even if you storing data in it. It is about how you perceive elastic. Elastic should be used to store and setup data for the application. It is the application which decides how and when to use elastic (search and suggestions). Elastic is not a nosql storage alternative if compared to RDBMS, you should use a nosql database instead.

This perception puts elastic in line with redis and kafka. These tools are key components of an application design and they are used to serve as events stores, search engines and cache etc. to the applications.

Database with Elastic

Your design should use both. For storing the contacts use the database, index the contacts for querying. Also make the data available in elastic for searching, autocomplete and related matches.

Improve this answer

Comments

2

As always, it depends on your specific use case. You briefly described it, but how are you acually going to use the data?

If it's just something simple like checking if a customer exists and then creating a new customer, then use the RDMS option. Moreover, if you don't expect a large dataset, so that scaling isn't an issue (hence the designation that Elasticsearch is for BigData), but you have transactions and data integrity is important, then a RDMS will be the right fit. Some examples could be for tax, leasing, or financial reporting systems.

However, if you have a large dataset, you need a wide range of query capabilities, such as a fuzzy search or searches where the user can select multiple filters on the data or you want to do some predictive analysis on the data, then Elasticsearch is the clear choice.

For example, I worked on an web based app with a large customer base: 11 million, with 200+ hits per second at peak time for a find a doctor application. The customer could check some checkboxes to determine, specialty, spoken languages, ratings, hospitals, etc. all sorted by the distance from the users location with a 2 second or less response time. It would be very difficult for a RDMS to match that.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.