Note: this changes return values from all .proc_* stored procedures,
making them more usable, but completely breaking the compatibility with
older versions.

Closes #500

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Cases when verdict was missing:
- unknown user
- user was blocked

Close: https://git.picodata.io/picodata/picodata/picodata/-/issues/438

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

close: https://git.picodata.io/picodata/picodata/picodata/-/issues/449

Having two different fixtures `binary_path` and
`path_to_binary_with_webui` results in weird failures in CI:

```
FAILED test/int/test_couple.py::test_exit_after_apply - conftest.TarantoolError:
    ('ER_NO_SUCH_PROC', "Procedure 'pico._inject_error' is not defined")
```

The root cause is that one `cargo build` overrides another with
different features enabled. To resolve the issue this patch
significantly refactors the test flow:

1. Web UI tests are skipped by default. One can enable it explicitly
   with `pytest --with-webui`

2. New fixture `cargo_build` behavior depends on environment:
   - In local dev environment it rebuilds the project with required features
   - In CI it skips rebuilding and runs tests against the existing build

3. The `path_to_binary_with_webui` fixture is removed.

See also:

- https://stackoverflow.com/a/55769818/1182787
- https://docs.pytest.org/en/7.4.x/how-to/writing_hook_functions.html
- https://docs.pytest.org/en/7.4.x/how-to/mark.html
- https://docs.pytest.org/en/7.4.x/example/simple.html#control-skipping-of-tests-according-to-command-line-option

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This patch adds initiator field to most of the events. Initiator is
added to members of Op enum which do not contain it already (e g owner
in CreateUser serves as initiator). To me it is reasonable to remove
as_user from CaS/ACL checks because it matches initiator. But I decided
not to do this just yet.

Additionally I've mande small cleanup to UserMetadata to avoid need for
intermediate step in decoding when we first fetch tuple and then decode
it into model struct.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/433

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Frequently when we want to execute commands from different users we need
to pass username and password separatly to each invokation. With this
patch this approach can be simplified and connection can be used
directlry in form of context manager.

Additionally this patch fixes handling of NOLOG env var. When it is
supplied logs are disabled which simplifies debugging of a test nbecause
with continuous output from multiple instances it is impossible to use
debugger efficiently.

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

At the later stage I discovered that sbroad applies the same validation
rules during parsing of grant/revoke statements and even its own
Privilege enum that is a ~copy of our PrivilegeType. Unfortunately
at the moment there is no way to share code between picodata and sbroad
efficiently (now only tarantool-module is shared and it is not suitable
for such kind of things) so it still makes sense to have this in
picodata because this is the point where all APIs converge to the single
point (CaS). In the future all other ways of validation should be
removed. Aside from sbroad similar kind of validation is performed
independently on lua API side.

Note that in prior commit 45ba7392 we've
removed all privileges from role super. This patch removes privileges
from admin that do not match the model: namely all privileges on
universe except session and usage. With this patch it is no longer possible
to grant or revoke such privileges.

The patch adds remaining access checks for cluster wide management
operations involving tables users and roles. Added checks closely follow
ones in vanilla tarantool. Basic scenario now works. See newly added
`test_create_space_smoke`.

There are some follow ups I'd like to address separately, namely:
- more tests for new ifs around box_access_check_ddl calls
  https://git.picodata.io/picodata/picodata/picodata/-/issues/420
- add tests for ownership semantics (these can be mostly ported from tarantool-module) blocked by
  https://git.picodata.io/picodata/picodata/picodata/-/issues/408
- port access.test and role.test from core (at some point later)
  https://git.picodata.io/picodata/picodata/picodata/-/issues/421
- Detect and prohibit circular role grants
  https://git.picodata.io/picodata/picodata/picodata/-/issues/415

Close #339

Transferred replication factor from Properties table to new Tier table

Instance and Replicaset belongs to tier via new field tier in _pico_instance and _pico_replicaset

New cli option init-cfg with path to configuration file in yaml format. For now contains only info about tiers.

BREAKING CHANGE!:
- pico.trace() function was removed. Use pico.sql() for tracing;
- opentelemetry tables __SBROAD_STAT and __SBROAD_QUERY were renamed
  into _SQL_STAT and _SQL_QUERY tables.

Introduce enrypoints for extended PG protocol:

- pg_bind: bind parameters to portal (IR) by descriptor;
- pg_close: remove portal from the storage by descriptor;
- pg_describe: get the portal metadata and type by descriptor;
- pg_execute: execute portal by descriptor;
- pg_parse: parse an SQL pattern into IR (reusing LRU),
  save IR into portal storage and return the descriptor;
- pg_portals: get a list of portal descriptors visible
  to the current user;

Co-authored-by: Kaitmazian Maksim <m.kaitmazian@picodata.io>

Denis Smirnov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Check that picodata SQL uses `key_def` module from Tarantool to
calculate the tuple hash.

and minor rpc fixes