godzie44 authored 11 months ago and Yaroslav Dynnikov committed 11 months ago

Introduce a plugin system for picodata cluster.
Plugin system contains:
- `picoplugin` crate with plugin interfaces.
This crate exposes plugin interfaces for user implementation and API
that used by picodata to interact with plugin as with a code from dynamic library.
- Support for plugin work cycle in picodata instance work cycle.
Picodata instance interacts with running plugin by `PluginManager`
component.
Typically, plugins react to events that occur in instance or whole cluster.
- New lua API with plugin CRUD actions.
    - pico.create_plugin - load a new plugin into picodata cluster
    - pico.remove_plugin - remove plugin from picodata cluster
    - pico.update_plugin_config - update plugin service configuration

Here's the order of plan stages for memories:

```
$ grep "return Ok" src/governor/plan.rs
return Ok(ConfChange { conf_change }.into());
return Ok(TransferLeadership { to: new_leader }.into());
return Ok(Downgrade { req }.into());
return Ok(CreateReplicaset { master_id, replicaset_id, rpc, op }.into());
return Ok(UpdateTargetReplicasetMaster { op }.into());
return Ok(Replication { targets, master_id, replicaset_peers, req }.into());
return Ok(UpdateCurrentReplicasetMaster { old_master_id, demote, new_master_id, sync_and_promote, replicaset_id, op }.into());
return Ok(ProposeReplicasetStateChanges { op }.into());
return Ok(UpdateTargetVshardConfig { dml }.into());
return Ok(UpdateCurrentVshardConfig { targets, rpc, dml }.into());
return Ok(ShardingBoot { target, rpc, op }.into());
return Ok(ToOnline { target, rpc, req }.into());
return Ok(ApplySchemaChange { rpc, targets }.into());
```

- remove logical clock from raft entry context, (term, index) is used
  for synchronization instead
- this allows us to remove the Notify/Notifier boilerplate
- NodeImpl::notifications -> NodeImpl::read_state_wakers
- this and the recently refactored proc_join finally allows us to remove
  OpResult
- ContextCoercion & CoercionError are also removed for the sake of
  simplification

Yaroslav Dynnikov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Use `std::ops::ControlFlow` instead

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This patch adds initiator field to most of the events. Initiator is
added to members of Op enum which do not contain it already (e g owner
in CreateUser serves as initiator). To me it is reasonable to remove
as_user from CaS/ACL checks because it matches initiator. But I decided
not to do this just yet.

Additionally I've mande small cleanup to UserMetadata to avoid need for
intermediate step in decoding when we first fetch tuple and then decode
it into model struct.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/433

- Now we explicitly store the current & target vshard configuration in
  global storage

Transferred replication factor from Properties table to new Tier table

Instance and Replicaset belongs to tier via new field tier in _pico_instance and _pico_replicaset

New cli option init-cfg with path to configuration file in yaml format. For now contains only info about tiers.

Still ignore expelled ones though.
Also this introduces yet another hardcoded timeout value, which should
be unhardcoded at some point (related to #64).

Georgy Moshkin authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

They're being replaced with ddl create_space, etc.

Georgy Moshkin authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This used to cause flakyness of test_bucket_rebalancing_respects_replication_factor
like this:

- governor proposes entry at index N with weight updates,
  and waits till it's applied

- commit index is not updated yet, but governor calls rpc::sharding to
  update the weights in the vshard subsystem on instances and passes the
  lagging commit index for syncing

- peers sync up to the lagging commit index and then reconfigure vshard
  to the outdated weights

- governor thinks everything is in order and waits for updates

- the test fails because buckets don't get rebalanced to the storages
  with weights set to 0

Make it more alike `find_farthest`:

```rust
sum_distance(all, voters, candidate);
find_farthest(all, voters, candidates);
```

1. Besides `raft_conf` local state introduce `promotable` set of
   instances that are suitable for promotion.

2. Remove an instance from `promotable` set when it's actually promoted.