Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Cases when verdict was missing:
- unknown user
- user was blocked

Close: https://git.picodata.io/picodata/picodata/picodata/-/issues/438

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Close: https://git.picodata.io/picodata/picodata/picodata/-/issues/434

Web UI feature is already enabled in CI `test` stage. This patch adds it
to the `pack` stage.

To do that, a path to a webui buldle can be passed explicitly, and it
does in CI which now builds it in a separate stage.

In default Rust features `webui` remains disabled by default to keep
minimal development environment truly minimal.

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Before this patch we've had ADMIN_ID in schema.rs and ADMIN_USER_ID
in lib.rs. This patch removes ADMIN_USER_ID in favor of ADMIN_ID

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This patch adds initiator field to most of the events. Initiator is
added to members of Op enum which do not contain it already (e g owner
in CreateUser serves as initiator). To me it is reasonable to remove
as_user from CaS/ACL checks because it matches initiator. But I decided
not to do this just yet.

Additionally I've mande small cleanup to UserMetadata to avoid need for
intermediate step in decoding when we first fetch tuple and then decode
it into model struct.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/433

This patch fixes a possible DOS by checking whether the user exists
before creating a record for its unsuccessful auth attempt. This is
due to the fact that `box.session.on_auth` will be called even for
unknown users (this helps vanilla users log such events).

This patch adds two new events:
* auth_ok
* auth_fail, which may contain "verdict" describing user suspension.

Examples:

```json
{
  "id": "1.1.19",
  "message": "successfully authenticated user `guest`",
  "severity": "low",
  "time": "2023-11-30T19:02:10.708+0300",
  "title": "auth_ok",
  "user": "guest"
}
```

```json
{
  "id": "1.0.11",
  "message": "failed to authenticate user `borat`",
  "severity": "high",
  "time": "2023-11-30T18:58:48.635+0300",
  "title": "auth_failed",
  "user": "borat"
}
```

Logically, the record id consists of 3 fields:
* `raft_id` -- raft id of the current node (or `null` if uninitialized);
* `count` -- in-memory id which is auto incremented for each new record;
* `gen` -- persistent id which is incremented each time node is started.

However, in practice they are stored inside a global `LogicalClock`.

This is what the record looks like now:

```json
{
  "id": "1.0.3",
  "instance_id": "i1",
  "message": "a new instance `i1` joined the cluster",
  "raft_id": "1",
  "severity": "low",
  "time": "2023-11-29T17:07:36.524+0300",
  "title": "join_instance"
}
```

Note how those fields are concatenated using `.`; the resulting
unique id is represented by a json string type.

Furthermore, since audit record logger now depends on the
raft machine's state, we decided to move its initialization
further into the init sequence (postjoin).

The patch adds remaining access checks for cluster wide management
operations involving tables users and roles. Added checks closely follow
ones in vanilla tarantool. Basic scenario now works. See newly added
`test_create_space_smoke`.

There are some follow ups I'd like to address separately, namely:
- more tests for new ifs around box_access_check_ddl calls
  https://git.picodata.io/picodata/picodata/picodata/-/issues/420
- add tests for ownership semantics (these can be mostly ported from tarantool-module) blocked by
  https://git.picodata.io/picodata/picodata/picodata/-/issues/408
- port access.test and role.test from core (at some point later)
  https://git.picodata.io/picodata/picodata/picodata/-/issues/421
- Detect and prohibit circular role grants
  https://git.picodata.io/picodata/picodata/picodata/-/issues/415

Close #339

sbroad needs to be updated to the version that supports newer
tarantool-module

sbroad support for new module version was merged here:
https://git.picodata.io/picodata/picodata/sbroad/-/merge_requests/355

This is useful to avoid sorting during snapshot preparation and in
upcoming access control patch it simplifies the check whether a
particular space id belongs to system space id range.

This patch implements an `--audit` flag (disabled by default) which
gives user the ability to enable and configure the audit log. The
flag takes a string containing the mode and parameters needed to
initialize tarantool's logging machinery.

See the flag's documentation for more info.

Example:

```
picodata run --interactive --audit=/tmp/audit.log
```

Dmitry Ivanov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Implemented:

- features:
    * FFI wrappers for `say.{h,c}`
    * safe logger object on top of FFI definitions
    * unified tlog/audit definitions for formatting
    * fixed a needless allocation in tlog (new root per each line)

- events:
    * create_user
    * drop_user
    * create_role
    * drop_role
    * change_password
    * grant_privilege
    * revoke_privilege
    * create_table
    * drop_table
    * grant_role
    * revoke_role

Todo:

- features:
    * unique id generation
    * security subject (who executed the action)
    * audit configuration (via `pico.audit` function)

- well-defined events:
    * create_database (universe)
    * change_config
    * change_current_grade

- special events (local to each node):
    * auth_ok / auth_fail
    * access_denied (still not sure if we absolutely need that)
    * audit_rotate

- Unavailable (haven't been defined yet):
    * rename_user
    * create_procedure
    * drop_procedure

Example:

```
$ cat /tmp/audit.log
2023-11-10 12:07:19.457 [15735] main/103/interactive I> audit log is ready, title: initialize_audit
2023-11-10 12:07:19.457 [15735] main/103/interactive I> database is starting, title: startup
2023-11-10 12:07:19.818 [15741] main/103/interactive I> audit log is ready, title: initialize_audit
2023-11-10 12:07:19.818 [15741] main/103/interactive I> database is starting, title: startup
2023-11-10 12:07:34.910 [15741] main/111/raft_main_loop storage.rs:2809 W> created user `KEK`, auth_type: chap-sha1, title: create_user
2023-11-10 12:07:52.869 [15741] main/111/raft_main_loop storage.rs:2861 W> created role `mgr`, title: create_role
2023-11-10 12:07:55.602 [15741] main/111/raft_main_loop storage.rs:2861 W> created role `kek`, title: create_role
2023-11-10 12:07:57.594 [15741] main/111/raft_main_loop storage.rs:2906 W> granted role `mgr` to role `kek`, title: grant_role
2023-11-10 12:08:08.482 [15741] main/111/raft_main_loop storage.rs:2913 W> granted privilege execute on universe `` to role `kek`, title: grant_privilege
```

Kurdakov Alexander authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

pico.whoami() returns new field tier

Transferred replication factor from Properties table to new Tier table

Instance and Replicaset belongs to tier via new field tier in _pico_instance and _pico_replicaset

New cli option init-cfg with path to configuration file in yaml format. For now contains only info about tiers.

BREAKING CHANGE!:
- pico.trace() function was removed. Use pico.sql() for tracing;
- opentelemetry tables __SBROAD_STAT and __SBROAD_QUERY were renamed
  into _SQL_STAT and _SQL_QUERY tables.

Introduce enrypoints for extended PG protocol:

- pg_bind: bind parameters to portal (IR) by descriptor;
- pg_close: remove portal from the storage by descriptor;
- pg_describe: get the portal metadata and type by descriptor;
- pg_execute: execute portal by descriptor;
- pg_parse: parse an SQL pattern into IR (reusing LRU),
  save IR into portal storage and return the descriptor;
- pg_portals: get a list of portal descriptors visible
  to the current user;

Co-authored-by: Kaitmazian Maksim <m.kaitmazian@picodata.io>

main.rs has grown a bit, adding new stuff to it doesnt feel natural.
Smaller files are easier to digest. For me it is easier to find stuff
when there are less files on the top level and some hierarchy shows.
where to go. A while back I wanted to move sql command to separate
file but there is already sql.rs on the top level.

This patch solves this issue.