Egor Ivkov authored 11 months ago and Yaroslav Dynnikov committed 11 months ago

(cherry picked from commit 46dfe8c1)

Egor Ivkov authored 11 months ago and Yaroslav Dynnikov committed 11 months ago

(cherry picked from commit 466cecc5)

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Previously it didnt contain title and severity because the event is
emitted on tarantool side. As a dirty hack match the message and insert
relevant fields.

This patch adds a proper support for auxiliary fields containing `Option<T>`.
As of now, KV pairs containing `Some(x)` will be printed as just `x`,
while the ones with `None` won't be printed at all.

Example:

```rust
crate::audit!(
    message: "demo audit event",
    title: "example",
    severity: Low,
    test: Some(0u32),
);
```

```json
{
  "id": "1.0.1",
  "message": "demo audit event",
  "severity": "low",
  "time": "2023-12-06T17:16:51.085+0300",
  "title": "example",
  "test": "0"
}
```

```rust
crate::audit!(
    message: "demo audit event",
    title: "example",
    severity: Low,
    test: None::<i32>,
);
```

```json
{
  "id": "1.0.1",
  "message": "demo audit event",
  "severity": "low",
  "time": "2023-12-06T17:16:51.085+0300",
  "title": "example"
}
```

Logically, the record id consists of 3 fields:
* `raft_id` -- raft id of the current node (or `null` if uninitialized);
* `count` -- in-memory id which is auto incremented for each new record;
* `gen` -- persistent id which is incremented each time node is started.

However, in practice they are stored inside a global `LogicalClock`.

This is what the record looks like now:

```json
{
  "id": "1.0.3",
  "instance_id": "i1",
  "message": "a new instance `i1` joined the cluster",
  "raft_id": "1",
  "severity": "low",
  "time": "2023-11-29T17:07:36.524+0300",
  "title": "join_instance"
}
```

Note how those fields are concatenated using `.`; the resulting
unique id is represented by a json string type.

Furthermore, since audit record logger now depends on the
raft machine's state, we decided to move its initialization
further into the init sequence (postjoin).

Now the records look like this:

```
{
  "message": "instance is starting",
  "severity": "low",
  "time": "2023-11-22T22:10:24.211+0300",
  "title": "local_startup"
}
{
  "message": "target grade of instance `i1` changed to Online(7)",
  "severity": "low",
  "time": "2023-11-22T22:10:24.334+0300",
  "title": "change_target_grade"
}
{
  "message": "current grade of instance `i1` changed to Replicated(7)",
  "severity": "medium",
  "time": "2023-11-22T22:10:24.436+0300",
  "title": "change_current_grade"
}
```

This patch changes the way audit records are formatted. Previously
we would format them as plain strings (we still do that in tlog),
now we use json to store all KV pairs including the message.

TODO: optimize string allocations using a stream formatter.

Example:

```
{
  "time": "2023-11-16T22:48:08.297+0300",
  "level": "WARN",
  "auth_type": "chap-sha1",
  "message": "created user `idris`",
  "title": "create_user",
  "pid": 66625,
  "cord_name": "main",
  "fiber_id": 111,
  "fiber_name": "raft_main_loop",
  "file": "src/storage.rs",
  "line": 2633
}
```

This patch implements an `--audit` flag (disabled by default) which
gives user the ability to enable and configure the audit log. The
flag takes a string containing the mode and parameters needed to
initialize tarantool's logging machinery.

See the flag's documentation for more info.

Example:

```
picodata run --interactive --audit=/tmp/audit.log
```

Dmitry Ivanov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

Implemented:

- features:
    * FFI wrappers for `say.{h,c}`
    * safe logger object on top of FFI definitions
    * unified tlog/audit definitions for formatting
    * fixed a needless allocation in tlog (new root per each line)

- events:
    * create_user
    * drop_user
    * create_role
    * drop_role
    * change_password
    * grant_privilege
    * revoke_privilege
    * create_table
    * drop_table
    * grant_role
    * revoke_role

Todo:

- features:
    * unique id generation
    * security subject (who executed the action)
    * audit configuration (via `pico.audit` function)

- well-defined events:
    * create_database (universe)
    * change_config
    * change_current_grade

- special events (local to each node):
    * auth_ok / auth_fail
    * access_denied (still not sure if we absolutely need that)
    * audit_rotate

- Unavailable (haven't been defined yet):
    * rename_user
    * create_procedure
    * drop_procedure

Example:

```
$ cat /tmp/audit.log
2023-11-10 12:07:19.457 [15735] main/103/interactive I> audit log is ready, title: initialize_audit
2023-11-10 12:07:19.457 [15735] main/103/interactive I> database is starting, title: startup
2023-11-10 12:07:19.818 [15741] main/103/interactive I> audit log is ready, title: initialize_audit
2023-11-10 12:07:19.818 [15741] main/103/interactive I> database is starting, title: startup
2023-11-10 12:07:34.910 [15741] main/111/raft_main_loop storage.rs:2809 W> created user `KEK`, auth_type: chap-sha1, title: create_user
2023-11-10 12:07:52.869 [15741] main/111/raft_main_loop storage.rs:2861 W> created role `mgr`, title: create_role
2023-11-10 12:07:55.602 [15741] main/111/raft_main_loop storage.rs:2861 W> created role `kek`, title: create_role
2023-11-10 12:07:57.594 [15741] main/111/raft_main_loop storage.rs:2906 W> granted role `mgr` to role `kek`, title: grant_role
2023-11-10 12:08:08.482 [15741] main/111/raft_main_loop storage.rs:2913 W> granted privilege execute on universe `` to role `kek`, title: grant_privilege
```