Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Cases when verdict was missing:
- unknown user
- user was blocked

Close: https://git.picodata.io/picodata/picodata/picodata/-/issues/438

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Close: https://git.picodata.io/picodata/picodata/picodata/-/issues/436

Arseniy Volynets authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

- Use retries for flaky sql tests
with global subqueries instead of
commenting them out

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Close: https://git.picodata.io/picodata/picodata/picodata/-/issues/434

Anatoly Popov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

1. http-server:

    1. Get info about tiers from _pico_tier
    2. Get info about instance addresses from:
        - http: lua api
        - binary:  _pico_peer_addresses
    3. Switch from arrays to map where it makes sense
    4. Compose all the data to `cluster_state` object
    5. Get version information for lua api.

2. Tests:

    1. Add DeepDiff package to better diff for json
    2. Amend tests to new specification
    3. Get picodata version via lua api and use it in asserts

3. Chore:

    1. Add .envrc and .direnv to .gitingore (direnv specific files)
    2. Add .vscode to .gitignore.

Signed-off-by: Anatoly Popov <a.popov@picodata.io>

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

close: https://git.picodata.io/picodata/picodata/picodata/-/issues/449

Having two different fixtures `binary_path` and
`path_to_binary_with_webui` results in weird failures in CI:

```
FAILED test/int/test_couple.py::test_exit_after_apply - conftest.TarantoolError:
    ('ER_NO_SUCH_PROC', "Procedure 'pico._inject_error' is not defined")
```

The root cause is that one `cargo build` overrides another with
different features enabled. To resolve the issue this patch
significantly refactors the test flow:

1. Web UI tests are skipped by default. One can enable it explicitly
   with `pytest --with-webui`

2. New fixture `cargo_build` behavior depends on environment:
   - In local dev environment it rebuilds the project with required features
   - In CI it skips rebuilding and runs tests against the existing build

3. The `path_to_binary_with_webui` fixture is removed.

See also:

- https://stackoverflow.com/a/55769818/1182787
- https://docs.pytest.org/en/7.4.x/how-to/writing_hook_functions.html
- https://docs.pytest.org/en/7.4.x/how-to/mark.html
- https://docs.pytest.org/en/7.4.x/example/simple.html#control-skipping-of-tests-according-to-command-line-option

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This patch adds initiator field to most of the events. Initiator is
added to members of Op enum which do not contain it already (e g owner
in CreateUser serves as initiator). To me it is reasonable to remove
as_user from CaS/ACL checks because it matches initiator. But I decided
not to do this just yet.

Additionally I've mande small cleanup to UserMetadata to avoid need for
intermediate step in decoding when we first fetch tuple and then decode
it into model struct.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/433

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Frequently when we want to execute commands from different users we need
to pass username and password separatly to each invokation. With this
patch this approach can be simplified and connection can be used
directlry in form of context manager.

Additionally this patch fixes handling of NOLOG env var. When it is
supplied logs are disabled which simplifies debugging of a test nbecause
with continuous output from multiple instances it is impossible to use
debugger efficiently.

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

At the later stage I discovered that sbroad applies the same validation
rules during parsing of grant/revoke statements and even its own
Privilege enum that is a ~copy of our PrivilegeType. Unfortunately
at the moment there is no way to share code between picodata and sbroad
efficiently (now only tarantool-module is shared and it is not suitable
for such kind of things) so it still makes sense to have this in
picodata because this is the point where all APIs converge to the single
point (CaS). In the future all other ways of validation should be
removed. Aside from sbroad similar kind of validation is performed
independently on lua API side.

Note that in prior commit 45ba7392 we've
removed all privileges from role super. This patch removes privileges
from admin that do not match the model: namely all privileges on
universe except session and usage. With this patch it is no longer possible
to grant or revoke such privileges.

Yaroslav Dynnikov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Tested scenarios:

1. Alice is granted a privilege to create global/sharded table
2. Alice can create, read, and write it
3. Bob can't read it
4. Alice grants Bob a privilege to read it
5. Now Bob can read it, but he still can't write

This patch reintroduces instance-related events (e.g. `join_instance`)
which were accidentally disabled in a bugfix of the DML handler (see
commit 52ed9237 for more information) due to the lack of tests.

Furthermore, this patch adds the following events:

* expel_instance
* change_config, which tracks changes in `_pico_property`

This was the original intended change of the patch, but the apparent
breakage of the preexisting events called for action.

This change allows for creators of corresponding objects to consequently have all privileges on them.

Arseniy Volynets authored 1 year ago and Georgy Moshkin committed 1 year ago

- Test fails because of timeout on socket.
Current vshard multi storage call works
in 2 stages:
1. Ref needed storages
2. Execute stored procedure on the storages
When creating Ref on storage, vshard waits for all
buckets to become writable, if it happens that
rebalancer will start to move buckets, the query
execution will freeze until rebalancing is
done, which happens for this test sometimes.
- Comment out the flaky test until we switch
to new vshard multicall algorithm. Another
solution would be to increase timeout, but
there is no guarantee that new timeout is
big enough: I caught this error with 5s
timeout.

This patch fixes a possible DOS by checking whether the user exists
before creating a record for its unsuccessful auth attempt. This is
due to the fact that `box.session.on_auth` will be called even for
unknown users (this helps vanilla users log such events).

The patch adds remaining access checks for cluster wide management
operations involving tables users and roles. Added checks closely follow
ones in vanilla tarantool. Basic scenario now works. See newly added
`test_create_space_smoke`.

There are some follow ups I'd like to address separately, namely:
- more tests for new ifs around box_access_check_ddl calls
  https://git.picodata.io/picodata/picodata/picodata/-/issues/420
- add tests for ownership semantics (these can be mostly ported from tarantool-module) blocked by
  https://git.picodata.io/picodata/picodata/picodata/-/issues/408
- port access.test and role.test from core (at some point later)
  https://git.picodata.io/picodata/picodata/picodata/-/issues/421
- Detect and prohibit circular role grants
  https://git.picodata.io/picodata/picodata/picodata/-/issues/415

Close #339

Arseniy Volynets authored 1 year ago and Denis Smirnov committed 1 year ago

- update sbroad submodule
- add integration tests with aggregates
on global tables
- groupby, having and aggregates are now
supported for global tbls