Added a `wait_index` parameter to `proc_replication_demote` to improve
synchronization during demotion operations.

Diana Tikhonova authored 3 months ago and Diana Tikhonova committed 2 months ago

Key updates include:
Adding the term field to relevant RPC requests (ConfigureReplicationRequest, ReplicationSyncRequest, and DemoteRequest).
Utilizing node.status().check_term(req.term) in critical sections of the replication logic.

The bug was introduced when I changed the behaviour of conf_change in
regard to instances with target_state Expelled. As a result we would
sometimes arbitrarily demote healthy voters in presence of degenerate
ones.

For example we could have this situation:

instance i1: raft_id=1, target_state=Online, raft_configuration=voter
instance i2: raft_id=2, target_state=Expelled, raft_configuration=voter (!)
instance i3: raft_id=3, target_state=Offline, raft_configuration=voter (!)
instance i4: raft_id=4, target_state=Online, raft_configuration=learner (!)
instance i5: raft_id=5, target_state=Online, raft_configuration=learner (!)

This patch changes when we update current_master_name during consistent
master switchover. Now it is updated as soon as we determine that the
new master has synchronized it's vclock with the old master.

As a result we can now block expel of a non-last replica in cases where
other replicas are offline, which is crucial because instances can
easily become offline due to temporary network issues.

Now we assume instance may not respond if
- target state is Offline (non-graceful assumed)
- current state is Expelled

We pass a timeout into the connection pool call which should be enough

godzie44 authored 6 months ago and Georgy Moshkin committed 6 months ago

Closes #664

Georgy Moshkin authored 6 months ago and Yaroslav Dynnikov committed 6 months ago

Strictness!

Move vshard related fields from _pico_property to _pico_tier
and adapt it in governor algorithm

This reverts commit d7e93cfc.