fix: expel is blocked by master switchover even if replicas are offline
Summary
- fix: don't update current_master_id until it synchronizes
This patches changes when we update current_master_id during consistent master switchover. Now it is updated as soon as we determine that the new master has synchronized it's vclock with the old master.
As a result we can now block expel of a non-last replica in cases where other replicas are offline, which is crucial because instances can easily become offline due to temporary network issues.
-
fix: expel is blocked my master switchover even if replicas are offline
-
fix: change definition of Instance::may_respond
Now we assume instance may not respond if
- target state is Offline (non-graceful assumed)
- current state is Expelled
-
test: add a failing test for bucket rebalancing before instance expel
-
test: don't panic libtestplug.so in case of unauthorized RPC
-
refactor: cleanup code in resolve_rpc_target
- Cherry-pick to: none
- Closes: #186 (closed), #1055 (closed)
- Docs follow-up: docs#395 (closed)
- changed return value of proc_replication
- new RPC proc_replication_sync
Edited by Georgy Moshkin
Merge request reports
Activity
requested review from @rosik
added 26 commits
-
c5858ebd...d5b17c71 - 20 commits from branch
gmoshkin/update_instance_cleanup - 949131f7 - refactor: cleanup code in resolve_rpc_target
- dc43a576 - test: don't panic libtestplug.so in case of unauthorized RPC
- ece5ce53 - test: add a failing test for bucket rebalancing before instance expel
- 24193a73 - fix: change definition of Instance::may_respond
- 7178ed79 - fix: expel is blocked my master switchover even if replicas are offline
- 488a00ce - fix: don't update current_master_id until it synchronizes
Toggle commit list-
c5858ebd...d5b17c71 - 20 commits from branch
mentioned in merge request !1343
added 6 commits
- 14317c03 - chore: clippy warning
- 0e7e82c8 - test: add a failing test for bucket rebalancing before instance expel
- ac13a05f - fix: change definition of Instance::may_respond
- 95deb154 - fix: expel is blocked my master switchover even if replicas are offline
- ac1104a7 - fix: don't update current_master_id until it synchronizes
- 3eebb2e4 - fix: previous commit broke transferring of raft leadership
Toggle commit listadded 12 commits
-
3eebb2e4...4a8d4a19 - 4 commits from branch
gmoshkin/update_instance_cleanup - a07122c4 - refactor: cleanup code in resolve_rpc_target
- 4d7092e2 - test: don't panic libtestplug.so in case of unauthorized RPC
- 9a51163f - chore: clippy warning
- 21a7440a - test: add a failing test for bucket rebalancing before instance expel
- 7e09f4d8 - fix: change definition of Instance::may_respond
- b4f01e75 - fix: expel is blocked my master switchover even if replicas are offline
- ceabf4f9 - fix: don't update current_master_id until it synchronizes
- cb38fe86 - fix: previous commit broke transferring of raft leadership
Toggle commit list-
3eebb2e4...4a8d4a19 - 4 commits from branch
added 12 commits
-
cb38fe86...14ac78e8 - 4 commits from branch
gmoshkin/update_instance_cleanup - c59ca442 - refactor: cleanup code in resolve_rpc_target
- 88683dcc - test: don't panic libtestplug.so in case of unauthorized RPC
- ba144da8 - chore: clippy warning
- 5dc0b83c - test: add a failing test for bucket rebalancing before instance expel
- 5588e338 - fix: change definition of Instance::may_respond
- d38765a6 - fix: expel is blocked my master switchover even if replicas are offline
- 6d5fcf73 - fix: don't update current_master_id until it synchronizes
- 78ae3145 - fix: previous commit broke transferring of raft leadership
Toggle commit list-
cb38fe86...14ac78e8 - 4 commits from branch
added 29 commits
-
78ae3145...cca2c02f - 22 commits from branch
gmoshkin/update_instance_cleanup - 92d6b1f5 - refactor: cleanup code in resolve_rpc_target
- 1b84f171 - test: don't panic libtestplug.so in case of unauthorized RPC
- f2d3e9d5 - chore: clippy warning
- ae252975 - test: add a failing test for bucket rebalancing before instance expel
- 1c437767 - fix: change definition of Instance::may_respond
- 979c4b60 - fix: expel is blocked my master switchover even if replicas are offline
- 954d5788 - fix: previous commit broke transferring of raft leadership
Toggle commit list-
78ae3145...cca2c02f - 22 commits from branch
mentioned in issue #787
added 1 commit
- cae33e0a - test: make test_vshard_updates_on_master_change less flaky
added 1 commit
- a0d1c9dc - test: make test_vshard_updates_on_master_change less flaky
added 18 commits
-
a0d1c9dc...179c780f - 9 commits from branch
master - 31fc7714 - refactor: cleanup code in resolve_rpc_target
- 79e58750 - test: don't panic libtestplug.so in case of unauthorized RPC
- fd6f2438 - chore: clippy warning
- c2065909 - test: add a failing test for bucket rebalancing before instance expel
- 825f0063 - fix: change definition of Instance::may_respond
- 464a61b6 - fix: expel is blocked my master switchover even if replicas are offline
- 268952b9 - fix: previous commit broke transferring of raft leadership
- 1bff6f6e - fix: don't update current_master_id until it synchronizes
- b6f94e0a - test: make test_vshard_updates_on_master_change less flaky
Toggle commit list-
a0d1c9dc...179c780f - 9 commits from branch
added 21 commits
-
b6f94e0a...e29b4d07 - 12 commits from branch
master - 2b3ff532 - refactor: cleanup code in resolve_rpc_target
- 96dbc9e5 - test: don't panic libtestplug.so in case of unauthorized RPC
- c34e91da - chore: clippy warning
- 237d52da - test: add a failing test for bucket rebalancing before instance expel
- 8dac4444 - fix: change definition of Instance::may_respond
- eddab61c - fix: expel is blocked my master switchover even if replicas are offline
- 13f7198c - fix: previous commit broke transferring of raft leadership
- bcbad30c - fix: don't update current_master_id until it synchronizes
- 00581341 - test: make test_vshard_updates_on_master_change less flaky
Toggle commit list-
b6f94e0a...e29b4d07 - 12 commits from branch
Please register or sign in to reply