Note that after this merge request, we no longer validate whether the format
of the created object matches with the schema. Consequently, we do not
suppress errors if formats match anymore.

Before:
```sql
CREATE USER u WITH PASSWORD 'Passw0rd'
1
CREATE USER u WITH PASSWORD 'Passw0rd'
0
CREATE USER u WITH PASSWORD 'DifferentPassw0rd'
error: user already exists with different auth method
```

Now:
```sql
CREATE USER u WITH PASSWORD 'Passw0rd'
1
CREATE USER u WITH PASSWORD 'Passw0rd'
error: user u already exists
CREATE USER u WITH PASSWORD 'DifferentPassw0rd'
error: user u already exists
CREATE USER IF NOT EXISTS u WITH PASSWORD 'DifferentPassw0rd'
0
```

CAS predicates should always contain the current raft term, because this
is what's explicitly checked in proc_cas.

Note that this will sometimes result in a EntryTermMismatch error
because the latest applied entry may have a different term in case the
election has started but not finished yet.

We could add this check to all the client cas precondition checks, but
it's not a big deal, because we handle the situation correctly anyways.

- Simplify code
- Check plugin exists on client side

- Move all preconditions checking to the client
- Return errors if preconditions fail instead of silently ignoring them

Dmitry Rodionov authored 7 months ago and Yaroslav Dynnikov committed 7 months ago

Previously function was ~700 loc. The patch splits out the logic of
conversion from ir node to operation. Previously there were two stages,
initial validation that constructed intermediate structure and then
intermediate structure was transformed into Op potentially several times
in case of retries. The patch removes intermediate conversion by
direct mapping from ir node to op. It makes reading more convenient
since there is no need to jump back and forth between initial checks and
actual Op construction. Another benefit is that some checks need to be
repeated on each retry since retries originate from conflicts and
conflicts change system state. So in newer state operation may no longer
make sense. By coupling both operations together there is no way a check
can be missed in the retry phase.

We still do some cloning here and there. We can probably improve on that
by constructing Op once and then adjusting it in place for conflicts but
since retries should be rare in practice it shouldnt be a problem.

Additionally we do a lot of conversion between SmolStr and String. We
need to use SmolStr in more placese inside picodata so we dont need to
do that.

godzie44 authored 8 months ago and Maksim Kaitmazian committed 7 months ago

Closes #651

godzie44 authored 8 months ago and Maksim Kaitmazian committed 7 months ago

Closes #584

godzie44 authored 8 months ago and Maksim Kaitmazian committed 7 months ago

Closes #682

feature(plugin migration): use md5 from migration files meta for to confirm consistency of migrations between versions

Closes #652

- add `migration_up` and `migration_down` lua API functions
- now the migration mechanism in plugins takes into account the plugin version and, accordingly, previous applied migrations
- add _pico_plugin_migration system table

Closes #683

- use RouteBuilder to register rpc endpoints (input and ouptut is raw
  bytes for now)
- use RequestBuilder to send RPC requests to other instances which are
  running a specified plugin.service (guarded by version)
    - supported routing by
        - explicit instance_id
        - replicaset_id
        - bucket_id
        - any available instance with given service
- use context::Context to extract information about request or add
  custom fields at runtime (middleware not yet implemented)

1. make it non-nullable
2. fix tests

godzie44 authored 11 months ago and Georgy Moshkin committed 10 months ago

- add description field to `_pico_plugin` and `_pico_service`
- plugin loading: filter files by its extensions
- plugin loading: check possible service collisions
- plugin loading: now plugin searching at {plugin_dir}/{plugin_name} path
- now all errors between plugin and picodata wrapping with `BoxError` (this protects plugin caller from UAF)

godzie44 authored 11 months ago and Yaroslav Dynnikov committed 11 months ago

Now for installation and enable plugin, there are two steps instead of one
(`pico.load_plugin` previously).
1. Install plugin (`pico.install_plugin`):
validate plugin, create system tables.
2. Enable plugin (`pico.enable_plugin`):
loads plugin into instances memory, call `on_start` callback.

Same for a plugin removing process (`pico.remove_plugin` previously):
1. Disable plugin (`pico.disable_plugin`): call `on_stop` callbacks, drop
plugin from instances.
2. Remove plugin (`pico.remove_plugin`):
remove plugin meta from system tables.