Elasticsearch store user input as JSON document

Question

I have a following architectural question - my application back-end is written at Java and client side at AngularJS. Right now I need to store the user input on the page in order to be able to share and bookmark my application urls and restore the state by this url.

I'm going to implement the following approach - every time the user interacts with my application via selecting the data and conditions at the page I'll collect all of his input at a complex JSON document and store this document at Elasticsearch. Key of this document from ES I'll send back to client application(AngularJS) and based on this key I'll update the page url. For example the original url looks like:

http://example.com/some-page

based on a key from server I'll update this url to following:

http://example.com/some-page/analysis/234532453455

where 234532453455 is a key of a document in ES.

Every time the users will try to access the following url - http://example.com/some-page/analysis/234532453455 AngularJS application will try to get a saved state by key (234532453455) via Java backend REST endpoint.

Will it work ?

Also, I'm in doubt right now how to prevent the duplication of the documents in ES. Right now I have no experience with ES so don't know what approach from ES can be used out of the box for this purpose.

For example is it a good idea to calculate some hash code of each JSON document and store this hash code as a key of a document.. so before storing the new document I can check the old document by hash code. Also performance is very important to me so please take this into account also.

Vova Bilyachat · Accepted Answer · 2017-02-12 23:11:28Z

1

For me it sounds you try to implement cache.

Yes you can do this but if you will use ES only for this solution then I think you should better look to redis or memcached.

I can't say that ES is bad solution for this but ES has some tricks which you must remember for instance its near realtime search. After you index data they are not immediately available to search it takes few seconds depends on configuration(But you can also call _refresh but I am not sure about performance if you index data very often).

Hash: I dont see reasons to use has I'd better create right id. So if you have types of reports per user than id could be "reporttype_{userid}", because if you will use hash as an ID then each new object will have new id and instead of rewriting you will end up with having many copies of old data for that user. if you go with pattern reporttype_{userid} then each time when user regenerate report with new data you will just override it.

As an option you could add to that option fields userid and expireat for future cleanup, for instance you could have job which will cleanup expired reports, but this is valid only if you go with ES, since in redis and memcached there is option to set expiration when you save data

edited Feb 12, 2017 at 23:11

answered Feb 12, 2017 at 22:56

Vova Bilyachat

19.5k4 gold badges60 silver badges88 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

alexanoid Over a year ago

Thanks for your detailed answer. I was thinking about ES in order to avoid "technological zoo" .. because I'm also going to use ES as a search engine for my project. Taking into account ES "near realtime search" feature I think Redis will be more appropriate place for such kind of cache..

alexanoid Over a year ago

Also, due to data and project nature I can't really to "report type" or "user id" as a data key. I'll user CRC32(calculated for JSON string) as a key of my document.

alexanoid Over a year ago

And the main criterion right now - I need extremely fast storage for set/get operations by own(provided by me) ID. I hope Redis should be a good choice for this purpose.

Vova Bilyachat Over a year ago

@alexanoid Then Redis would be best choise

alexanoid Over a year ago

I have one more criterion - the data in this cache should not be lost.. In other words - the data should be persisted to disk and should be available after restart. Is Redis still a good choice for this ?

|

Collectives™ on Stack Overflow

Elasticsearch store user input as JSON document

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related