Elasticsearch and SQL Server for full text search?

Question

I'm working on a project that requires documents to be ocr-ed and the text that's returned stored and be searchable. The biggest obstacle is the performance of full-text searching of the scraped text.

My idea is to use a combination of SQL Server for data persistence and Elasticsearch for performant searching. When a document has been scraped it would be inserted into the database and then if that was successful it would be indexed by Elasticsearch.

Can anyone see any caveats with this setup or offer any insight as to how it could be done better?

Your approach seems valid. Once you determine the code has passed OCR, you can either save it to RDBMS and then have ES index it. Or you can directly index it in ES. IMO I would skip the RDBMS, since you will not make searches on it, but on ES. — Amiga500
– Amiga500, Commented Jun 1, 2017 at 13:10
I should also store the document in the rdbms or other persistent storage so you can easily recreate the es index with different mapping settings. — broersa
– broersa, Commented Jun 1, 2017 at 13:26

Ohlsen1980 · Accepted Answer · 2017-06-01 13:16:18Z

1

I developed a pretty same project using sql server for the complete erp storage. Written a windows service, which permanently syncs the data i want to search into an elasticsearch cluster. It runs perfectly, on the one side the database with all data and on the other side the es cluster for fast searching.

answered Jun 1, 2017 at 13:16

Ohlsen1980

3245 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jacob Mason Over a year ago

Sounds very similar! How did you achieve 'permanent synchronization' between the rdbs and es?

Ohlsen1980 Over a year ago

I wrote a windows service using NEST api to sync my data to es. The service runs in defined cycles and queries the database for changes (with LINQ). Then it indexes the data in es cluster.

Collectives™ on Stack Overflow

Elasticsearch and SQL Server for full text search?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related