feature(plugin migration): use md5 from migration files meta for to confirm consistency of migrations between versions

Closes #652

- add `migration_up` and `migration_down` lua API functions
- now the migration mechanism in plugins takes into account the plugin version and, accordingly, previous applied migrations
- add _pico_plugin_migration system table

Closes #683

- use RouteBuilder to register rpc endpoints (input and ouptut is raw
  bytes for now)
- use RequestBuilder to send RPC requests to other instances which are
  running a specified plugin.service (guarded by version)
    - supported routing by
        - explicit instance_id
        - replicaset_id
        - bucket_id
        - any available instance with given service
- use context::Context to extract information about request or add
  custom fields at runtime (middleware not yet implemented)

1. make it non-nullable
2. fix tests

godzie44 authored 11 months ago and Georgy Moshkin committed 10 months ago

- add description field to `_pico_plugin` and `_pico_service`
- plugin loading: filter files by its extensions
- plugin loading: check possible service collisions
- plugin loading: now plugin searching at {plugin_dir}/{plugin_name} path
- now all errors between plugin and picodata wrapping with `BoxError` (this protects plugin caller from UAF)

godzie44 authored 1 year ago and Yaroslav Dynnikov committed 11 months ago

Now for installation and enable plugin, there are two steps instead of one
(`pico.load_plugin` previously).
1. Install plugin (`pico.install_plugin`):
validate plugin, create system tables.
2. Enable plugin (`pico.enable_plugin`):
loads plugin into instances memory, call `on_start` callback.

Same for a plugin removing process (`pico.remove_plugin` previously):
1. Disable plugin (`pico.disable_plugin`): call `on_stop` callbacks, drop
plugin from instances.
2. Remove plugin (`pico.remove_plugin`):
remove plugin meta from system tables.

BREAKING CHANGE: "name" index in the _pico_index has changed:
previously it guarantees uniqueness for a pair of (space_id,
index_name). Now index name must be unique among the cluster.
So, all the global indexes for _pico tables were renamed.

BREAKING CHANGE: _pico_index structure has changed.

The reason for refactoring is the request for the cluster wide
indexes with all possible options. The old structure didn't allow
us to support them all.

godzie44 authored 1 year ago and Yaroslav Dynnikov committed 11 months ago

Introduce a plugin system for picodata cluster.
Plugin system contains:
- `picoplugin` crate with plugin interfaces.
This crate exposes plugin interfaces for user implementation and API
that used by picodata to interact with plugin as with a code from dynamic library.
- Support for plugin work cycle in picodata instance work cycle.
Picodata instance interacts with running plugin by `PluginManager`
component.
Typically, plugins react to events that occur in instance or whole cluster.
- New lua API with plugin CRUD actions.
    - pico.create_plugin - load a new plugin into picodata cluster
    - pico.remove_plugin - remove plugin from picodata cluster
    - pico.update_plugin_config - update plugin service configuration

Make field 'auth' nullable

- Update sbroad submodule to commit with
grant/revoke procedure support
- Use existing mechanisms for grant/revoke
ACL opcodes for grant/revoke procedure
implementation
- Add integration tests
- Valid combinations:

all procedures: grant/revoke create/execute/drop
specific procedure: grant/revoke execute/drop

- Syntax:

grant create procedure to alice option(timeout=3)
grant execute procedure on "spam"(int) to "alex"
grant drop procedure on foo to bob
revoke drop procedure on foo from bob

- add new ddl opcode for renaming procedure
- update sbroad submodule to get sql support
- syntax:

alter procedure "foo" rename to "bar" option(timeout=3)
alter procedure "foo"(int, int) rename to "bar"

Maksim Kaitmazian authored 1 year ago and Denis Smirnov committed 1 year ago

Without this commit creation of a function for non-admin users fail with
`AccessDenied: Read access to space \'_pico_routine\' is denied` error.

Current commit introduces
- _pico_routine replicated table
- procedure creation with SQL API

Egor Ivkov authored 1 year ago and Егор Ивков committed 1 year ago

Panic happened in `on_access_denied` trigger. There was a switch to admin and back to user with `su`. Switch back to user panicked in SuGuard drop as user did not have session privilege.