8

The following diagram is a basic representation of a web application I am creating.

Node.js Web Application Schematic

The basic operation of the application is as follows:

  1. Client sends request for data from node.js server
  2. Server receives request.
  3. Server fetches data from database.
  4. Server sends data back to client as JSON string.
  5. Client receives data stores it as a JSON object and binds it to a table.

This all works fine. The problem I am having is best represented by the following situation.

  1. Clients 1 to 4 all complete the steps above (i.e. all have an table of data binded to a JSON object).
  2. Client 1 now sends an update request to the server.
  3. Server receives request.
  4. Server updates database.
  5. Server sends response to client 1 indicating successful operation and updates JSON object binded to table.
  6. THE PROBLEM JSON data shown on clients 2 to 4 is now no longer in sync with the database.

So my question is how do I keep the JSON data on all 4 (or more) of my clients in real-time sync with the database on my Node.js server?

Improve this question
8
  • 1
    There have been entire books written and many PHDs done on all the types of synchronization strategies. What to use depends upon a whole bunch of things (use cases, number of data changes, how often clients are offline, conflicts strategy, size of data set, etc...) Are clients 1,2,3,4 always online? Or do they come online and then have to update their synchronization? Also, do you really have to store all data locally. One advantage of cloud services is to maintain a single central repository and just have clients query that as needed. Commented Feb 22, 2017 at 3:05
  • @jfriend00 Can you recommend any books that are specific to a Node.js server? Clients are always online. At this stage the data is to be stored locally on our in-house server, which will also be used as the single central repository of data. Commented Feb 22, 2017 at 3:08
  • 1
    Briefly, what you need is to notify clients when something has changed. Try looking for websockets, there are lots of good tutorial on the web. Basicaly, a websocket is a communication channel between server and clients., what you should do is to send a notification to the client about what has changed, then each client should determine if it has to update something (and request the information) or not. Commented Feb 22, 2017 at 3:11
  • 2
    Your problem really isn't specific to the node.js server at all. It's specific to your data store and all the other issues I listed in my prior comment. If your clients are always online, you can create a webSocket connection from each client to the node.js server and then every time any client makes a change to the data store, you send that delta to all the other clients. But, if any client goes offline for awhile, then you have to invent a catch up strategy when it comes back online. If you google "data synchronization strategies" there's lots to read. Commented Feb 22, 2017 at 3:14
  • 1
    I've edited my answer in order to provide one of the best libs for WebSockets. You will find examples for client, server and demos. I strongly recommend you not to include from the scratch to your project, but also build a runneable example to understand what WebSockets are, how they work and what exactly you need. Once you are done, go ahead and apply it to your project. Commented Feb 22, 2017 at 11:54

3 Answers 3

Reset to default
4

Briefly, what you need is to notify clients when something has changed. Try looking for websockets, there are lots of good tutorial on the web. Basicaly, a websocket is a communication channel between server and clients., what you should do is to send a notification to the client about what has changed, then each client should determine if it has to update something (and request the information) or not.

Take a look a this lib:https://www.npmjs.com/package/websocket

I think it is one of the best (if it is not the best) and you will even find examples and demos.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Guy asks for sync and you talk about communication. GL HF implementing async-multithreaded-read-on-write objects bro.
Would it be better to send the changes directly to the clients instead of notifying the clients and then the clients fetch the changes again making new requests?
2

I think the best way to do this in 2021 is with server-sent events. Your backend can emit a server-sent event to all other clients every time it receives data. SSEs are basically downstream unidirectional events, and browsers can listen for them the same way they can listen to a websocket.

The clients will then only be out of sync with respect to network latency, which is unavoidable.

The only problem with SSEs in my opinion is that they have lower browser compatibility, so it'll break IE 11 support.

Improve this answer

Comments

0

If you are open to taking on a full-stack framework, Meteor does everything you need to keep all your client views in sync with changes.

Meteor is a huge dependency to take on - it has opinions about your technology choices at every layer in your stack - so its not going to be for everyone, but if you want quick results it is hard to beat.

Improve this answer

5 Comments

Sounds great but I was reading that the stack comes with a pre-installed database. Is it able to be swapped out with our already pre-configured MySQL database?
Is Meteor actual data store sync or just keep a client view up to date? The OP asked for data store synchronization.
It uses a pub/sub mechanism over Comet to keep all connected clients views in sync with changes made by any client.
@josh Meteor is strongly based around MongoDB, but there may be third party support for SQL. I would suggest sticking to Mongo if you are going to try Meteor.
@saille Great thanks! Your help is much appreciated.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.