box: update synchro quorum in on_commit trigger instead of on_replace
Currently, we update the synchronous replication quorum from the `on_replace` trigger of the `_cluster` space when registering a new replica. However, during the join process, the replica cannot ack its own insertion into the `_cluster` space. In the scope of #9723, we are going to enable synchronous replication for most of the system spaces, including the `_cluster` space. There are several problems with this: 1. Joining a replica to a 1-member cluster without manual changing of quorum won't work: it is impossible to commit the insertion into the `_cluster` space with only 1 node, since the quorum will equal to 2 right after the insertion. 2. Joining a replica to a 3-member cluster may fail: the quorum will become equal to 3 right after the insertion, the newly joined replica cannot ACK its own insertion into the `_cluster` space — if one out of original 3 nodes fails, then reconfiguration will fail. Generally speaking, it will be impossible to join a new replica to the cluster, if a quorum, which includes the newly added replica (which cannot ACK), cannot be gathered. To solve these problems, let's update the quorum in the `on_commit` trigger. This way we’ll be able to insert a node regardless of the current configuration. This somewhat contradicts with the Raft specification, which requires application of all configuration changes in the `on_replace` trigger (i.e., as soon as they are persisted in the WAL, without quorum confirmation), but still forbids several reconfigurations at the same time. Closes #10087 NO_DOC=<no special documentation page devoted to cluster reconfiguration> (cherry picked from commit 29d1c0fa)
Showing
- changelogs/unreleased/gh-10087-update-quorum-in-on_commit-trigger.md 7 additions, 0 deletions...unreleased/gh-10087-update-quorum-in-on_commit-trigger.md
- src/box/alter.cc 18 additions, 0 deletionssrc/box/alter.cc
- src/box/replication.cc 0 additions, 1 deletionsrc/box/replication.cc
- test/replication-luatest/gh_10087_update_quorum_in_on_commit_trigger_test.lua 94 additions, 0 deletions...test/gh_10087_update_quorum_in_on_commit_trigger_test.lua
Loading
Please register or sign in to comment