1

If I have system which writes data to a storage.

In cluster setup if the data is replicated synchronously, that is when I get request to change data that is updated across all the nodes and then the response is returned back.

In this scenario is it possible to add node dynamically?

That is, say there are 2 nodes and some transactions are executed on that cluster. Can I add a 3rd node with cluster still running?

If I add a new node like this, does not it break the strong consistency of the cluster?

2 Answers 2

0

I believe what you are looking for is Consistent hashing (Wikipedia) it allows you to distribute you memory in a way so that the data is properly replicated between multiple nodes and adding or deleting a node has less overhead associated with it, I suggest you go through the link The simple magic of consistent hashing

Sign up to request clarification or add additional context in comments.

Comments

0

A simplistic and schematic example (just one of many possible variants):

  1. A node is added in recording mode. The cluster behaves as if the new node wasn't there, but also duplicates updates to objects that will need to reside on it to this node.
  2. It is determined which of the existing objects will need to reside on the new node. Snapshots are created on each of the existing nodes and sent to the new node. The new node processes them (inserts into its storage), possibly in batch mode.
  3. Updates that have been accumulated during the recording phase are replayed on the new node, while new updates are still being recorded. If some updates were already accounted as part of the snapshot, they either should be skipped or they should lead to no change.
  4. Once all updates have been replayed and if there were no errors in the process the cluster switches to the new configuration (with the new node fully started). Updates to the objects that should reside on the new node stop being sent to their old destinations. Data is cleared from the old destinations.
  5. If an error happens at any point during the process, it is aborted and possibly repeated anew.

Possible tweaks:

  • Query both the old and the new nodes for some time prior to switching to the new configuration and compare results returned. In case of any difference abort and investigate.
  • Conduct the full comparison (perhaps checksumming) before switching to the new configuration.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.