This could result in a broken instance if it terminated in between the
two storage writes. Now we always persist the commit index before
applying the persisted entries and do so in a transaction with unstable
entries or snapshot if they are present.

This patch fixes a possible DOS by checking whether the user exists
before creating a record for its unsuccessful auth attempt. This is
due to the fact that `box.session.on_auth` will be called even for
unknown users (this helps vanilla users log such events).

This patch adds two new events:
* auth_ok
* auth_fail, which may contain "verdict" describing user suspension.

Examples:

```json
{
  "id": "1.1.19",
  "message": "successfully authenticated user `guest`",
  "severity": "low",
  "time": "2023-11-30T19:02:10.708+0300",
  "title": "auth_ok",
  "user": "guest"
}
```

```json
{
  "id": "1.0.11",
  "message": "failed to authenticate user `borat`",
  "severity": "high",
  "time": "2023-11-30T18:58:48.635+0300",
  "title": "auth_failed",
  "user": "borat"
}
```

Logically, the record id consists of 3 fields:
* `raft_id` -- raft id of the current node (or `null` if uninitialized);
* `count` -- in-memory id which is auto incremented for each new record;
* `gen` -- persistent id which is incremented each time node is started.

However, in practice they are stored inside a global `LogicalClock`.

This is what the record looks like now:

```json
{
  "id": "1.0.3",
  "instance_id": "i1",
  "message": "a new instance `i1` joined the cluster",
  "raft_id": "1",
  "severity": "low",
  "time": "2023-11-29T17:07:36.524+0300",
  "title": "join_instance"
}
```

Note how those fields are concatenated using `.`; the resulting
unique id is represented by a json string type.

Furthermore, since audit record logger now depends on the
raft machine's state, we decided to move its initialization
further into the init sequence (postjoin).

The patch adds remaining access checks for cluster wide management
operations involving tables users and roles. Added checks closely follow
ones in vanilla tarantool. Basic scenario now works. See newly added
`test_create_space_smoke`.

There are some follow ups I'd like to address separately, namely:
- more tests for new ifs around box_access_check_ddl calls
  https://git.picodata.io/picodata/picodata/picodata/-/issues/420
- add tests for ownership semantics (these can be mostly ported from tarantool-module) blocked by
  https://git.picodata.io/picodata/picodata/picodata/-/issues/408
- port access.test and role.test from core (at some point later)
  https://git.picodata.io/picodata/picodata/picodata/-/issues/421
- Detect and prohibit circular role grants
  https://git.picodata.io/picodata/picodata/picodata/-/issues/415

Close #339

Arseniy Volynets authored 1 year ago and Denis Smirnov committed 1 year ago

- update sbroad submodule
- add integration tests with aggregates
on global tables
- groupby, having and aggregates are now
supported for global tbls

test: mark test_script_failure as xfail, because it's too flaky and doesn't check for anything we care about

Arseniy Volynets authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

- Update sbroad submodule to commit adding support
for subqueries
- Add integration tests for sql queries with
subqueries and global tables

Supported queries:
- Queries that reference global tables in subqueries:
```sql
select a from segment_t where
b in (select c + d from global_t)
```
- Queries that read from global tables and use subquery:
```sql
select c from global_t
where d in (select a from segment_t)
```

Note: the support of subqueries applies only to types
of queries currently supported for global tables.
Joins, aggregates, except, union all are currently not
supported for global tables.

Rename space object type to table

Now the records look like this:

```
{
  "message": "instance is starting",
  "severity": "low",
  "time": "2023-11-22T22:10:24.211+0300",
  "title": "local_startup"
}
{
  "message": "target grade of instance `i1` changed to Online(7)",
  "severity": "low",
  "time": "2023-11-22T22:10:24.334+0300",
  "title": "change_target_grade"
}
{
  "message": "current grade of instance `i1` changed to Replicated(7)",
  "severity": "medium",
  "time": "2023-11-22T22:10:24.436+0300",
  "title": "change_current_grade"
}
```

This patch adds the following events:
* create_database
* change_target_grade
* change_current_grade