- Now we explicitly store the current & target vshard configuration in
  global storage

Georgy Moshkin authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This used to cause flakyness of test_bucket_rebalancing_respects_replication_factor
like this:

- governor proposes entry at index N with weight updates,
  and waits till it's applied

- commit index is not updated yet, but governor calls rpc::sharding to
  update the weights in the vshard subsystem on instances and passes the
  lagging commit index for syncing

- peers sync up to the lagging commit index and then reconfigure vshard
  to the outdated weights

- governor thinks everything is in order and waits for updates

- the test fails because buckets don't get rebalanced to the storages
  with weights set to 0

NOTE: there's still a bug, because we set each new replicaset's weight
to 1 (even ones which don't satisfy replication factor)
before the bucket distribution is bootstrapped. But there must be at
least one replicaset with non zero weight in order for vshard.*.cfg to
work.

A potential solution would be to only configure vshard once a replicaset
is filled up.

Georgy Moshkin authored 2 years ago and Georgy Moshkin committed 2 years ago

Currently stores:
- replicset_id
- replicaset_uuid
- sharding weight
- replication master id

Yaroslav Dynnikov authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

Do it only when the governor tells to. When `_G.vshard == nil`, it's
more alike `traft::node::global()` which reurns either `Ok` or
`Err(Uninitialized)`.