In the initial credentials schema, the hashes were supposed to give a
way to have passwords out of plain-text config file.
Later, it was decided to remove this feature, because this way of
authorisation is inferior to the one with auth service and tokens,
but the latter is out of scope for current config development.
This patch removes `credentials.password.{sha1,sha256}` and moves
plain password from `credentials.password.plain` to `credentials.password`.

Part of #8967

NO_DOC=tarantool/doc#3544 links the most actual schema,
       no need to update the issue.
NO_CHANGELOG=removed feature was not released yet

Before this patch, credentials applier used to just grant all privileges
and permissions with {if_not_exists = true}. It didn't allow removing a
permission, nor setting only new permissions.

Now credentials applier converts box configuration and desired config to
an intermediate representation, calculates diff for them and only after
that applies the diff.

Part of #8967

NO_DOC=yet

Sometimes it is useful to have one function to execute command,
read and compare response.
This patch introduces such function - `roundtrip()` to
interactive_tarantool helper.

NO_CHANGELOG=test helper change
NO_TEST=test helper change
NO_DOC=test helper change

test/unit/guard.cc calls stack_break_f() recursively until the stack
overflows and a signal is fired, however it relies on undefined behavior
when compares pointers to local variables. Fixed by comparing
__builtin_frame_address() instead.

One of the examples of this UB is when ASAN allocates local variables on
fake stacks, in that case the test completes without the stack overflow.

Also this patch disables ASAN for stack_break_f() to keep the array on the
fiber stack (see the corresponding comment) and marks it as volatile to
avoid optimizing it away by the compiler.

Closes tarantool/tarantool-qa#323

NO_DOC=test fix
NO_CHANGELOG=test fix

Closes #8803

@TarantoolBot document
Title: Document `lua_eval`, `lua_call`, and `sql` grant object types

In Tarantool 3.0 we introduced the new `lua_eval`, `lua_call`, and `sql`
object types for `box.schema.user.grant` to control access to code
execution over the network protocol (IPROTO).

1. Granting the 'execute' privilege on `lua_eval` permits the user to
   execute arbitrary Lua code with the `IPROTO_EVAL` request.

   Example:

   ```Lua
   box.cfg({listen = 3301})
   box.schema.user.create('alice', {password = 'secret'})
   conn = require('net.box').connect(
       box.cfg.listen, {user = 'alice', password = 'secret'})
   conn:eval('return true') -- access denied
   box.schema.user.grant('alice', 'execute', 'lua_eval')
   conn:eval('return true') -- ok
   ```

2. Granting the 'execute' privilege on `lua_call` permits the user to
   call any global (accessible via the `_G` Lua table) user-defined
   Lua function with the `IPROTO_CALL` request. It does **not** permit
   the user to call built-in Lua functions, such as `loadstring` or
   `box.session.su`. It does **not** permit the user to call functions
   registered in the `_func` system space with `box.schema.func.create`
   (access to those functions is still controlled by privileges granted
   on `function`).

   Example:

   ```Lua
   function my_func() end
   box.cfg({listen = 3301})
   box.schema.user.create('alice', {password = 'secret'})
   conn = require('net.box').connect(
       box.cfg.listen, {user = 'alice', password = 'secret'})
   conn:call('my_func') -- access denied
   box.schema.user.grant('alice', 'execute', 'lua_call')
   conn:call('my_func') -- ok
   conn:call('box.session.su', {'admin'}) -- access denied
   ```

3. Granting the 'execute' privilege on `sql` permits the user to
   execute an arbitrary SQL expression with the `IPROTO_PREPARE`
   and `IPROTO_EXECUTE` requests. Without this privilege or the
   'execute' privilege granted on `universe`, the user is **not**
   permitted to execute SQL expressions over IPROTO anymore.
   Note that before Tarantool 3.0 any user (even guest) could execute
   SQL expressions over IPROTO. It is possible to revert to the old
   behavior by toggling the `sql_priv` compat option. Please add
   a description to https://tarantool.io/compat/sql_priv

   Example:

   ```Lua
   box.cfg({listen = 3301})
   box.schema.user.create('alice', {password = 'secret'})
   conn = require('net.box').connect(
       box.cfg.listen, {user = 'alice', password = 'secret'})
   conn:execute('SELECT 1') -- access denied
   box.schema.user.grant('alice', 'execute', 'sql')
   conn:execute('SELECT 1') -- ok
   ```

Granting the execute privilege on the 'lua_eval' object enables
evaluaing any Lua expression via IPROTO_EVAL.

Granting the execute privilege on the 'lua_call' object enables calling
any global Lua function via IPROTO_CALL except:
 - Functions from the _func system space (see box.schema.func.create).
   Access to them is still governed only by the 'function' object type.
 - Built-in functions. We assume that all functions that were added to
   _G before loading user modules are built-in.

Note, after this change access_check_universe_object becoms unused so
it's removed (merged with access_check_universe).

Part of #8803

NO_DOC=later

We are planning to add access checks for EXECUTE and PREPARE requests.
(Currently, everyone, even guest, may execute these requests.)
Checking access in tx_process_sql(), which is defined in IPROTO code,
would violate encapsulation and look inconsistent with other request
handlers. Let's move the code that actually processes an SQL request
to the new function box_process_sql() taking sql_request and returning
the result in a port object.

To unify handling of all SQL requests in box_process_sql(), we add a new
format for port_sql - UNPREPARE. The format works only for dumping port
content to MsgPack buffer - it encodes an empty map then. This way, we
don't need to return the is_unprepare flag from box_process_sql().

Needed for #8803

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

SC_ENTITY_FOO is used instead of SC_FOO when a privilege is granted to
an entire object class, not an individual object (object id is set to ''
in the _priv system space). Introduction of this new concept made the
access checking code rather confusing, especially the part converting
entity types to object types and back, and complicated addition of new
schema object types.

Actually, there's no point in maintaining separate schema object types
for entities. Instead, we can simply add a flag to the priv_def struct
saying that the object id stored in the struct is meaningless and the
privilege should be applied to an entire object class. This simplifies
the code quite a bit and makes introduction of new schema object types
must easier.

Needed for #8803

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

access_check_ddl() uses access_find() to look up the access structure of
the altered object by id, but the caller usually already knows what
object is altered so the second look up is redundant. Let's pass an
access struct to access_check_ddl() instead of an object id and make
access_find() private to src/box/user.cc, as it used to be.

Note that in priv_def_check(), we first call access_check_ddl() and only
then look up the target object to do some extra checks but we can
reverse the order since it doesn't change anything per se. This allows
us to get rid of schema_find_name(), which performed yet another extra
object lookup.

The goal of this cleanup is to simplify addition of new schema object
types, which is necessary to implement new privileges for SQL and Lua
code execution.

Needed for #8803

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Currently the `exclude_null` field of `index_object.parts` is inconsistent
between local box and net.box connection. It returns the following values:
- locally: true / null
- net.box: true / false

This mismatch makes it difficult to compare schemas on the storage and via
the net.box connection on the router (see tarantool/crud#361).

Closes #8649

NO_DOC=minor

This reverts commit ace88542.

That commit disabled repl in tarantool for debugger console session
because debugger console wasn't compatible with Tarantool console,
i.e. this code hang in terminal:

```lua
tarantool> dbg = require 'luadebug'
tarantool> dbg()
```

With the previous patch in the patchset, full readline support and
console compatibility was introduced. Thus, no need to disable repl.

Part of #7738

NO_TEST=internal
NO_DOC=internal, revert unpublished
NO_CHANGELOG=internal

Implemented readline history and autocomplete by reusing
readline facilities of Tarantool console. They used to be
being hidden once Lua 'console' module is loaded. With c432e9e9
(lua: don't use public module name as internal one), now they
are available as 'console.lib'.

Closes #7738

NO_TEST=covered by refactored console_debugger_session_test.lua

@TarantoolBot document
Title: proper readline support in readline

Similar to tarantool interactive console, tdbg now uses readline
for its shell. It enables handier input editing, command history
and so on.

`console_debugger_session_test.lua` can be simplified and made more
stable with interactive_tarantool helper. With the new
`read_untill_prompt()` helper function, dbgr prompt can be used as
end-of-output marker instead of '...' in yaml-outputting main console.
This way, popen results are unambiguous and no retries are required.
At the same time, dbgr prompt is now expected with every result, thus
there is no need for an extra check. But `continue` command usually
reaches the end of the script and exits debugger, thus
'<END_OF_EXECUTION>' marker was introduced.
Empty stderr and header check were moved to interactive_tarantool helper.
This patch breaks the test without debugger readline support patch,
that is next in the patchset.

Part of #7738

NO_CHANGELOG=test refactoring
NO_DOC=test refactoring

The following changes were applied to `interactive_tarantool` helper
to adapt it for debugger tests:
* compared commands are now stripped from tabs and color codes too
* now each command independent of control symbols is being stripped
  before comparison in `execute_command()`
* created internal new function that is called from both `new()` and
  `new_debugger()`
* now user defined prompt can be set up from `new` and `new_debugger()`
* now there are several less typos in comments

Part of #7738

NO_CHANGELOG=test helper update
NO_DOC=test helper update
NO_TEST=test helper update

Runtime tuple formats are reusable, which means that a tuple format
returned by runtime_tuple_format_new may not be brand new, but actually
be used by a Lua object. As a result, if we call any function that may
trigger Lua GC between runtime_tuple_format_new and tuple_format_ref,
the tuple format may be deleted, leading to a use-after-free bug. This
is what happens in lbox_tuple_format_new. Fix this issue by moving the
runtime_tuple_format_new call after the Lua object allocation.

Closes #8889

NO_DOC=bug fix
NO_TEST=difficult to reproduce, found by ASAN

Closes #8801

@TarantoolBot document
Title: Document `box.session.new`

The new function creates a new session given a table of options:

 - `type`: string, optional. Default: "binary". [Type][session-type] of
   the new session. Currently, only "binary" is supported, which creates
   a new IPROTO session.

 - `fd`: number, mandatory. File descriptor number (fd) to be used for
   the new connection input-output. The fd must refer to a [socket] and
   be switched to the [non-blocking mode][socket-nonblock] but this
   isn't enforced, i.e. the user may pass an invalid fd, in which case
   the connection won't work as expected.

 - `user`: string, optional. Default: "guest". Name of the user to
   authenticate the new session as. Note, this doesn't prevent the other
   end to perform explicit [authentication].

 - `storage`: table, optional. Default: empty table. Initial value of
   the [session-local storage][session-storage].

On success, `box.session.new` takes ownership of the `fd` and returns
nothing. On failure, an error is raised.

Possible errors:
 - Invalid option value type.
 - `fd` isn't specified or has an invalid value.
 - `box.cfg` wasn't called.
 - `user` doesn't exist.

Example:

The code below creates a TCP server that accepts all incoming
IPROTO connections on port 3301, authenticates them as 'admin'
and sets the session-local storage to `{foo = 'bar'}`.

```lua
box.cfg()
require('socket').tcp_server('localhost', 3301, function(s)
    box.session.new({
        type = 'binary',
        fd = s:fd(),
        user = 'admin',
        storage = {foo = 'bar'},
    })
    s:detach()
end)
```

Notes:
- `box.cfg` must be called before using `box.session.new` to start
  IPROTO threads. Setting [`box.cfg.listen`][box-cfg-listen] isn't
  required though.
- The socket object must be detached after passing its fd to
  `box.session.new`, otherwise the fd would be closed on Lua garbage
  collection.

[authentication]: https://www.tarantool.io/en/doc/latest/dev_guide/internals/iproto/authentication/
[box-cfg-listen]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-basic-listen
[session-storage]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/storage/
[session-type]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/type/
[socket]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/
[socket-nonblock]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/#socket-nonblock

Sometimes we need to submit a remote call over cbus without waiting for
the result or even yielding. The cbus_call_timeout function isn't apt
for that because it yields even if timeout is 0 and sets diag. Let's
introduce cbus_call_async that exits immediately after pushing the call
message to the remote thread.

NO_DOC=internal
NO_CHANGELOG=internal

Part of #8862

NO_DOC=this API is not marked as public, at least now

A set of testing helpers was added in commit 06ca83c9 ("test/config:
add several application script tests"). In particular, they allow to run
a tarantool instance with some default config file and write only
necessary config options in a test case.

In fact, if no options were set in the test, the config file was not
written. It is fixed in this commit and used in the next commit.

Part of #8862

NO_DOC=testing helper change
NO_CHANGELOG=see NO_DOC

Closes #8928

@TarantoolBot document
Title: Document `from_fd` net.box module function

The function takes a file descriptor number (fd) as its first argument
and a table of connection options as its second, optional argument. It
creates and returns a new connection object.

The fd should point to a socket and be switched to the non-blocking
mode, but this isn't enforced. If the fd is invalid, the connection may
not work as expected.

The functions takes the same options as [`connect`][net-box-connect].

The function takes the ownership of the fd, i.e. the fd must not be used
or closed after this function successfully returns.

Example:

The code below connects to a Tarantool instance at port 3301 using
the [`socket`][socket] module, then wraps the socket fd in a `net.box`
connection object.

```Lua
local net = require('net.box')
local socket = require('socket')

-- Connect a socket then wrap it in a net.box connection object.
local s = socket.tcp_connect('localhost', 3301)
assert(s ~= nil)
local conn = net.from_fd(s:fd(), {fetch_schema = false})
s:detach()

conn:call('box.schema.user.info')
conn:close()
```

[net-box-connect]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/#net-box-connect
[socket]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/

Those are really nice helpers. Let's move them to the utils module so
that we can use them everywhere. Also, let's add some unit tests.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

The check_param and check_param_table Lua helpers are defined in
box/lua/schema.lua but used across the whole code base. The problem is
we can't use them in files that are loaded before box/lua/schema.lua,
like box/lua/session.lua. Let's move them to a separate source file
lua/utils.lua to overcome this limitation. Also, let's add some tests.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

* ci: support coveralls
* cmake: add code coverage support
* test: run flake8 static analysis via CMake
* test: fix E741 errors by pycodestyle
* test: fix E722 errors by pycodestyle
* test: fix E711 errors by pycodestyle
* test: fix E502 errors by pycodestyle
* test: fix E501 errors by pycodestyle
* test: fix E305 errors by pycodestyle
* test: fix E303 errors by pycodestyle
* test: fix E302 errors by pycodestyle
* test: fix E301 errors by pycodestyle
* test: fix E275 errors by pycodestyle
* test: fix E251 errors by pycodestyle
* test: fix E231 errors by pycodestyle
* test: fix E203 errors by pycodestyle
* test: fix E201 and E202 errors by pycodestyle
* test: suppress E131 errors by pycodestyle
* test: fix E128 errors by pycodestyle
* test: fix E122 errors by pycodestyle
* gdb: fix Python <assert> statement usage

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=LuaJIT submodule bump

NO_DOC=seed corpus
NO_CHANGELOG=seed corpus
NO_TEST=seed corpus

NO_DOC=testing
NO_CHANGELOG=testing

NO_DOC=seed corpus
NO_CHANGELOG=seed corpus
NO_TEST=seed corpus

Examples of IPROTO decoding issues: #3900, #1928, #6781.
Patch adds a number of fuzzing tests that covers IPROTO decoding:

- xrow_decode_auth
- xrow_decode_begin
- xrow_decode_call
- xrow_decode_dml
- xrow_decode_id
- xrow_decode_raft
- xrow_decode_sql
- xrow_decode_watch
- xrow_greeting_decode

NO_DOC=testing
NO_CHANGELOG=testing

Now the default value of a tuple field can be computed as a result of a
function call.

Closes #8609

@TarantoolBot document
Title: Document functional default field values
Product: Tarantool
Since: 3.0
Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/format/
Depends on: tarantool/doc#3520

The format clause contains, for each field, a definition within braces:
`{name='...',type='...'[,default=...][,default_func=...]}`,
where:

* (Optional) The `default_func` string value specifies the name of a
  function, that is used to generate a default value for the field. If
  `default_func` is set, the `default` value is used as the function
  argument. See [field default functions](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-default-functions).

---

Root document: https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#constraint-functions

**Field default functions**

Stored Lua function can be used to generate a default value for the tuple
field. It can take one optional parameter, and must return exactly one
value. To create a field default function, use func.create with function
body. The function must not yield.

Example:

```lua
box.schema.func.create('random_point', {
    language = 'Lua',
    body = 'function(param) return math.random(param.min, param.max) end'
})

box.schema.space.create('test')
box.space.test:create_index('pk')
box.space.test:format({
    {name = 'id', type = 'integer'},
    {name = 'latitude', type = 'number',
     default_func = 'random_point', default = {min = -90, max = 90}},
    {name = 'longitude', type = 'number',
     default_func = 'random_point', default = {min = -180, max = 180}}
})
```

```lua
tarantool> math.randomseed(os.time())
---
...

tarantool> box.space.test:insert{1}
---
- [1, 56, 38]
...
```

This structure will be extended in the next commit.

Part of #8609

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This call is redundant and does nothing. Constraints are destroyed in
space_cleanup_constraints(), which is called from space_create() and
space_delete(). Standalone tuples can't have initialized constraints.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

User id instead of function id was erroneously used there.

NO_DOC=bugfix
NO_CHANGELOG=minor

The checks in box.schema.index.create() and box.schema.index.alter()
were case sensitive, also it was possible to insert incorrect index
options directly into `box.space._index`. Fixed by adding checks
to memtx_space_check_index_def() and vinyl_space_check_index_def().

Closes #8937

NO_DOC=bugfix

And remove unused index_def_check_xc().

As index_def_dup() never returns NULL anymore, change index_create() and
index_read_view_create() return type to `void` and update their callers.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The `__sanitizer_start_switch_fiber()` function takes a pointer as the
first argument to store the current fake stack if there is one (it is
necessary when stack-use-after-return detection is enabled). When leaving a
fiber definitely, NULL must be passed so that the fake stack is destroyed.

Before this patch, NULL was passed for dead fibers, however this is wrong
for dead fibers that are recycled and resumed. In such cases ASAN destroys
the fake stack, and the fiber crashes trying to use it in `fiber_yield()`
upon return from `coro_transfer()`.

Closes tarantool/tarantool-qa#321

NO_DOC=bugfix
NO_TEST=tested by test-release-asan workflow

_cluster on-replace trigger parses old and new tuples into
replica_def objects. One path handles the case of new_def != NULL.
The other assumes that old_def != NULL if new_def was NULL. This
is correct, because replace wouldn't happen if both tuples are
NULL. It would mean nothing changed.

Nonetheless coverity complained here that the old tuple could be
NULL even if the new one was NULL. The patch silences this warning
by adding an assert.

NO_DOC=refactor
NO_CHANGELOG=refactor
NO_TEST=not testable

box_set_instance_name() has a scoped guard to try to restore the
replication back to normal if the connection to master was
recreated to make the applier request a new name.

The guard calls box_restart_replication() which might throw
exceptions. But in this case it apparently can only happen in
theory with bootstrap strategy legacy + high connection quorum +
failing to gather this quorum. Also iostream creation might fail.

Iostream creation failure is not testable, and the legacy
bootstrap strategy is being deprecated at some point. So the
patch simply adds try-catch without a test as it is too much
effort for too little gain to bother. This should also calm down
the coverity reporter.

NO_DOC=bugfix/refactor
NO_CHANGELOG=hardly possible to observe this bug
NO_TEST=too hard to repro

Fuzzing test for LuaJIT generates random Lua programs and executes them.
We want to build a fuzzing test that will produce Lua programs that will
not contain semantic errors and will trigger as much as possible
components in LuaJIT.

This proposed patch introduces metrics that gathered after running the
test. LuaJIT metrics gathered using LuaJIT getmetrics module [1]. All
gathered metrics test will output after running with a finite number of
runs or finite duration of time (options `-runs` and `-max_total_time`)
or after sending SIGUSR1 to a test process.

```
$ ./build/test/fuzz/luaL_loadbuffer/luaL_loadbuffer_fuzzer -runs=1000

<snipped>

Done 1000 runs in 1 second(s)
Total number of samples: 1000
Total number of samples with errors: 438 (43%)
Total number of samples with recorded traces: 87 (8%)
Total number of samples with snap restores: 30 (3%)
Total number of samples with abort traces: 55 (5%)
```

1. https://www.tarantool.io/en/doc/latest/reference/tooling/luajit_getmetrics/#getmetrics-c-api

NO_CHANGELOG=testing
NO_DOC=testing

The function can't be called on an unconfigured instance because it
needs IPROTO threads up and running. Let's raise an error to avoid
a crash.

Since we have two other places where we need to raise the same error
(box.session.su and box.__index), let's introduce the new code
ER_UNCONFIGURED for this error.

Closes #8975

NO_DOC=bug fix

The `popen_new()` function may be called with inherit_fds = NULL,
nr_inherit_fds = 0. The lua/popen.c code actually does this.

It leads to the following memcpy() call.

```
memcpy(dst, NULL, 0);
```

According to the C11 standard (n1256, 7.21.1), the pointer argument
should have a valid value, which means pointing to some area in the
program's address space, not NULL. The standard doesn't make an
exception for a zero size array.

Personally I doubt that any memcpy() implementation attempts to
dereference the source pointer in case of a zero size, but it is my
assumption, while the standard is the standard.

The problem is found by Coverity.

Follows up #8926

NO_DOC=it is a bug
NO_CHANGELOG=this code is not released yet
NO_TEST=verified by existing test cases, which call popen.new() without
        the inherit_fds option

Closes #8802

@TarantoolBot document
Title: Deprecate box.session.push

The `box.session.push` Lua function and `box_session_push` C API
function are deprecated starting from Tarantool 3.0. Calling any
of these functions for the first time results in printing a warning
message to the log.

The new compat module option `box_session_push_deprecation` was
introduced to control whether the functions are still available.
With the old behavior, which is the default in Tarantool 3.0,
`box.session.push` is still available. With the new behavior,
any attempt to use it raises an exception.

(Please create https://tarantool.io/compat/box_session_push_deprecation)

We are planning to switch the compat option to the new behavior
starting from Tarantool 4.0 with the ability to revert to the
old behavior. Starting from Tarantool 5.0 we are planning to
drop `box.session.push` completely.