It is convenient for development environments, when the configuration
file and the application sources reside in the same directory.

The same logic was recently implemented for the main script, see #8182.
The same problems appears in context of startup from a configuration
file, so it seems meaningful to adjust module search paths in this case
too.

Part of #8862

NO_DOC=This change is too minor to describe in the documentation issue
       https://github.com/tarantool/doc/issues/3544. I'll work with the
       documentation team regarding details of startup/reload flow and
       we'll determine what should go to the user documentation and what
       shouldn't.

The new default directory layout is the following.

```
+ var/
  + lib/
    + instance-001/
      - *.xlog
      - *.snap
      - *.vylog
  + log/
    + instance-001/
      - tarantool.log
  + run/
    + instance-001/
      - tarantool.control
      - tarantool.pid
```

Our guess is that it should be convenient for development environments,
when the application consists of several instances. The idea is borrowed
from the `cartridge-cli` and `tt` tools.

We plan to synchronize these defaults with the `tt` tool, to simplify
cases, when pure tarantool (without `tt`) should be run in the
directories layout created by `tt`. It should simplify debugging using
`gdb`, `strace` and other tools.

Also, it should reduce possible confusion for users of `cartridge-cli`
and `tt`.

Part of #8862

NO_DOC=https://github.com/tarantool/doc/issues/3544 already points to
       the actual instance config schema

The current working directory of a tarantool process is changed during
startup to `process.work_dir`.

The mkdir applier works before and after this point, so it should take
into account both situations.

Before first box.cfg() call (in the box_cfg applier) it should prepend
directories with `process.work_dir`. However, it shouldn't do that after
the current wordking directory change.

Part of #8862

NO_DOC=It is bugfix.

The `process.work_dir` option can be set in the configuration. All the
other directories/files has the before-box-cfg creation logic and it
seems meaningful to support it for `process.work_dir` too.

Part of #8862

NO_DOC=This change is too minor to describe in the documentation issue
       https://github.com/tarantool/doc/issues/3544. I'll work with the
       documentation team regarding details of startup/reload flow and
       we'll determine what should go to the user documentation and what
       shouldn't.

For example, if we want the logs to be in a separate `var/log`
directory, it is convenient to just configure it as `var/run/{{
instance_name }}.log` and let tarantool create the parent directory.

Part of #8862

NO_DOC=This change is too minor to describe in the documentation issue
       https://github.com/tarantool/doc/issues/3544. I'll work with the
       documentation team regarding details of startup/reload flow and
       we'll determine what should go to the user documentation and what
       shouldn't.

For example, if we want the console socket to be in a separate `var/run`
directory, it is convenient to just configure it as `var/run/{{
instance_name }}.control` and let tarantool create the parent directory.

Part of #8862

NO_DOC=This change is too minor to describe in the documentation issue
       https://github.com/tarantool/doc/issues/3544. I'll work with the
       documentation team regarding details of startup/reload flow and
       we'll determine what should go to the user documentation and what
       shouldn't.

Before this patch an attempt to set, say, `wal.dir` option to a
non-existent directory `foo` succeeds, while the same with `foo/bar`
directory fails.

The patch removes a race condition check, because `fio.mktree()`
performs the check on its own. See #4660 for details.

Part of #8862

NO_DOC=It is a bugfix.

We agreed that it is up to a user how to draw a line between one set of
instances that is considered as a separate cluster and another set of
instances that is assumed as a different cluster. This line is virtual
and there is no sense to impose restrictions until we add some certain
(and consistent) semantic to word 'cluster' in context of tarantool.

Part of #8862

NO_DOC=There is an existing documentation request
       https://github.com/tarantool/doc/issues/3544 and it doesn't
       describe such details. I'll work with the documentation team on
       demand on all the necessary details anyway.

Before this patch, user password could be set or updated only for
auth_type == 'chap-sha1'. Now password can be set, updated or removed
for any auth_type. Note that the password is changed only if necessary
to minimise db writes.

Part of #8967

NO_DOC=tarantool/doc#3544 links the most actual schema,
       no need to update the issue.

In the initial credentials schema, the hashes were supposed to give a
way to have passwords out of plain-text config file.
Later, it was decided to remove this feature, because this way of
authorisation is inferior to the one with auth service and tokens,
but the latter is out of scope for current config development.
This patch removes `credentials.password.{sha1,sha256}` and moves
plain password from `credentials.password.plain` to `credentials.password`.

Part of #8967

NO_DOC=tarantool/doc#3544 links the most actual schema,
       no need to update the issue.
NO_CHANGELOG=removed feature was not released yet

Before this patch, credentials applier used to just grant all privileges
and permissions with {if_not_exists = true}. It didn't allow removing a
permission, nor setting only new permissions.

Now credentials applier converts box configuration and desired config to
an intermediate representation, calculates diff for them and only after
that applies the diff.

Part of #8967

NO_DOC=yet

Sometimes it is useful to have one function to execute command,
read and compare response.
This patch introduces such function - `roundtrip()` to
interactive_tarantool helper.

NO_CHANGELOG=test helper change
NO_TEST=test helper change
NO_DOC=test helper change

test/unit/guard.cc calls stack_break_f() recursively until the stack
overflows and a signal is fired, however it relies on undefined behavior
when compares pointers to local variables. Fixed by comparing
__builtin_frame_address() instead.

One of the examples of this UB is when ASAN allocates local variables on
fake stacks, in that case the test completes without the stack overflow.

Also this patch disables ASAN for stack_break_f() to keep the array on the
fiber stack (see the corresponding comment) and marks it as volatile to
avoid optimizing it away by the compiler.

Closes tarantool/tarantool-qa#323

NO_DOC=test fix
NO_CHANGELOG=test fix

Closes #8803

@TarantoolBot document
Title: Document `lua_eval`, `lua_call`, and `sql` grant object types

In Tarantool 3.0 we introduced the new `lua_eval`, `lua_call`, and `sql`
object types for `box.schema.user.grant` to control access to code
execution over the network protocol (IPROTO).

1. Granting the 'execute' privilege on `lua_eval` permits the user to
   execute arbitrary Lua code with the `IPROTO_EVAL` request.

   Example:

   ```Lua
   box.cfg({listen = 3301})
   box.schema.user.create('alice', {password = 'secret'})
   conn = require('net.box').connect(
       box.cfg.listen, {user = 'alice', password = 'secret'})
   conn:eval('return true') -- access denied
   box.schema.user.grant('alice', 'execute', 'lua_eval')
   conn:eval('return true') -- ok
   ```

2. Granting the 'execute' privilege on `lua_call` permits the user to
   call any global (accessible via the `_G` Lua table) user-defined
   Lua function with the `IPROTO_CALL` request. It does **not** permit
   the user to call built-in Lua functions, such as `loadstring` or
   `box.session.su`. It does **not** permit the user to call functions
   registered in the `_func` system space with `box.schema.func.create`
   (access to those functions is still controlled by privileges granted
   on `function`).

   Example:

   ```Lua
   function my_func() end
   box.cfg({listen = 3301})
   box.schema.user.create('alice', {password = 'secret'})
   conn = require('net.box').connect(
       box.cfg.listen, {user = 'alice', password = 'secret'})
   conn:call('my_func') -- access denied
   box.schema.user.grant('alice', 'execute', 'lua_call')
   conn:call('my_func') -- ok
   conn:call('box.session.su', {'admin'}) -- access denied
   ```

3. Granting the 'execute' privilege on `sql` permits the user to
   execute an arbitrary SQL expression with the `IPROTO_PREPARE`
   and `IPROTO_EXECUTE` requests. Without this privilege or the
   'execute' privilege granted on `universe`, the user is **not**
   permitted to execute SQL expressions over IPROTO anymore.
   Note that before Tarantool 3.0 any user (even guest) could execute
   SQL expressions over IPROTO. It is possible to revert to the old
   behavior by toggling the `sql_priv` compat option. Please add
   a description to https://tarantool.io/compat/sql_priv

   Example:

   ```Lua
   box.cfg({listen = 3301})
   box.schema.user.create('alice', {password = 'secret'})
   conn = require('net.box').connect(
       box.cfg.listen, {user = 'alice', password = 'secret'})
   conn:execute('SELECT 1') -- access denied
   box.schema.user.grant('alice', 'execute', 'sql')
   conn:execute('SELECT 1') -- ok
   ```

Granting the execute privilege on the 'lua_eval' object enables
evaluaing any Lua expression via IPROTO_EVAL.

Granting the execute privilege on the 'lua_call' object enables calling
any global Lua function via IPROTO_CALL except:
 - Functions from the _func system space (see box.schema.func.create).
   Access to them is still governed only by the 'function' object type.
 - Built-in functions. We assume that all functions that were added to
   _G before loading user modules are built-in.

Note, after this change access_check_universe_object becoms unused so
it's removed (merged with access_check_universe).

Part of #8803

NO_DOC=later

We are planning to add access checks for EXECUTE and PREPARE requests.
(Currently, everyone, even guest, may execute these requests.)
Checking access in tx_process_sql(), which is defined in IPROTO code,
would violate encapsulation and look inconsistent with other request
handlers. Let's move the code that actually processes an SQL request
to the new function box_process_sql() taking sql_request and returning
the result in a port object.

To unify handling of all SQL requests in box_process_sql(), we add a new
format for port_sql - UNPREPARE. The format works only for dumping port
content to MsgPack buffer - it encodes an empty map then. This way, we
don't need to return the is_unprepare flag from box_process_sql().

Needed for #8803

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

SC_ENTITY_FOO is used instead of SC_FOO when a privilege is granted to
an entire object class, not an individual object (object id is set to ''
in the _priv system space). Introduction of this new concept made the
access checking code rather confusing, especially the part converting
entity types to object types and back, and complicated addition of new
schema object types.

Actually, there's no point in maintaining separate schema object types
for entities. Instead, we can simply add a flag to the priv_def struct
saying that the object id stored in the struct is meaningless and the
privilege should be applied to an entire object class. This simplifies
the code quite a bit and makes introduction of new schema object types
must easier.

Needed for #8803

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

access_check_ddl() uses access_find() to look up the access structure of
the altered object by id, but the caller usually already knows what
object is altered so the second look up is redundant. Let's pass an
access struct to access_check_ddl() instead of an object id and make
access_find() private to src/box/user.cc, as it used to be.

Note that in priv_def_check(), we first call access_check_ddl() and only
then look up the target object to do some extra checks but we can
reverse the order since it doesn't change anything per se. This allows
us to get rid of schema_find_name(), which performed yet another extra
object lookup.

The goal of this cleanup is to simplify addition of new schema object
types, which is necessary to implement new privileges for SQL and Lua
code execution.

Needed for #8803

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Currently the `exclude_null` field of `index_object.parts` is inconsistent
between local box and net.box connection. It returns the following values:
- locally: true / null
- net.box: true / false

This mismatch makes it difficult to compare schemas on the storage and via
the net.box connection on the router (see tarantool/crud#361).

Closes #8649

NO_DOC=minor

This reverts commit ace88542.

That commit disabled repl in tarantool for debugger console session
because debugger console wasn't compatible with Tarantool console,
i.e. this code hang in terminal:

```lua
tarantool> dbg = require 'luadebug'
tarantool> dbg()
```

With the previous patch in the patchset, full readline support and
console compatibility was introduced. Thus, no need to disable repl.

Part of #7738

NO_TEST=internal
NO_DOC=internal, revert unpublished
NO_CHANGELOG=internal

Implemented readline history and autocomplete by reusing
readline facilities of Tarantool console. They used to be
being hidden once Lua 'console' module is loaded. With c432e9e9
(lua: don't use public module name as internal one), now they
are available as 'console.lib'.

Closes #7738

NO_TEST=covered by refactored console_debugger_session_test.lua

@TarantoolBot document
Title: proper readline support in readline

Similar to tarantool interactive console, tdbg now uses readline
for its shell. It enables handier input editing, command history
and so on.

`console_debugger_session_test.lua` can be simplified and made more
stable with interactive_tarantool helper. With the new
`read_untill_prompt()` helper function, dbgr prompt can be used as
end-of-output marker instead of '...' in yaml-outputting main console.
This way, popen results are unambiguous and no retries are required.
At the same time, dbgr prompt is now expected with every result, thus
there is no need for an extra check. But `continue` command usually
reaches the end of the script and exits debugger, thus
'<END_OF_EXECUTION>' marker was introduced.
Empty stderr and header check were moved to interactive_tarantool helper.
This patch breaks the test without debugger readline support patch,
that is next in the patchset.

Part of #7738

NO_CHANGELOG=test refactoring
NO_DOC=test refactoring

The following changes were applied to `interactive_tarantool` helper
to adapt it for debugger tests:
* compared commands are now stripped from tabs and color codes too
* now each command independent of control symbols is being stripped
  before comparison in `execute_command()`
* created internal new function that is called from both `new()` and
  `new_debugger()`
* now user defined prompt can be set up from `new` and `new_debugger()`
* now there are several less typos in comments

Part of #7738

NO_CHANGELOG=test helper update
NO_DOC=test helper update
NO_TEST=test helper update

Runtime tuple formats are reusable, which means that a tuple format
returned by runtime_tuple_format_new may not be brand new, but actually
be used by a Lua object. As a result, if we call any function that may
trigger Lua GC between runtime_tuple_format_new and tuple_format_ref,
the tuple format may be deleted, leading to a use-after-free bug. This
is what happens in lbox_tuple_format_new. Fix this issue by moving the
runtime_tuple_format_new call after the Lua object allocation.

Closes #8889

NO_DOC=bug fix
NO_TEST=difficult to reproduce, found by ASAN

Closes #8801

@TarantoolBot document
Title: Document `box.session.new`

The new function creates a new session given a table of options:

 - `type`: string, optional. Default: "binary". [Type][session-type] of
   the new session. Currently, only "binary" is supported, which creates
   a new IPROTO session.

 - `fd`: number, mandatory. File descriptor number (fd) to be used for
   the new connection input-output. The fd must refer to a [socket] and
   be switched to the [non-blocking mode][socket-nonblock] but this
   isn't enforced, i.e. the user may pass an invalid fd, in which case
   the connection won't work as expected.

 - `user`: string, optional. Default: "guest". Name of the user to
   authenticate the new session as. Note, this doesn't prevent the other
   end to perform explicit [authentication].

 - `storage`: table, optional. Default: empty table. Initial value of
   the [session-local storage][session-storage].

On success, `box.session.new` takes ownership of the `fd` and returns
nothing. On failure, an error is raised.

Possible errors:
 - Invalid option value type.
 - `fd` isn't specified or has an invalid value.
 - `box.cfg` wasn't called.
 - `user` doesn't exist.

Example:

The code below creates a TCP server that accepts all incoming
IPROTO connections on port 3301, authenticates them as 'admin'
and sets the session-local storage to `{foo = 'bar'}`.

```lua
box.cfg()
require('socket').tcp_server('localhost', 3301, function(s)
    box.session.new({
        type = 'binary',
        fd = s:fd(),
        user = 'admin',
        storage = {foo = 'bar'},
    })
    s:detach()
end)
```

Notes:
- `box.cfg` must be called before using `box.session.new` to start
  IPROTO threads. Setting [`box.cfg.listen`][box-cfg-listen] isn't
  required though.
- The socket object must be detached after passing its fd to
  `box.session.new`, otherwise the fd would be closed on Lua garbage
  collection.

[authentication]: https://www.tarantool.io/en/doc/latest/dev_guide/internals/iproto/authentication/
[box-cfg-listen]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-basic-listen
[session-storage]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/storage/
[session-type]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/type/
[socket]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/
[socket-nonblock]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/#socket-nonblock

Sometimes we need to submit a remote call over cbus without waiting for
the result or even yielding. The cbus_call_timeout function isn't apt
for that because it yields even if timeout is 0 and sets diag. Let's
introduce cbus_call_async that exits immediately after pushing the call
message to the remote thread.

NO_DOC=internal
NO_CHANGELOG=internal

Part of #8862

NO_DOC=this API is not marked as public, at least now

A set of testing helpers was added in commit 06ca83c9 ("test/config:
add several application script tests"). In particular, they allow to run
a tarantool instance with some default config file and write only
necessary config options in a test case.

In fact, if no options were set in the test, the config file was not
written. It is fixed in this commit and used in the next commit.

Part of #8862

NO_DOC=testing helper change
NO_CHANGELOG=see NO_DOC

Closes #8928

@TarantoolBot document
Title: Document `from_fd` net.box module function

The function takes a file descriptor number (fd) as its first argument
and a table of connection options as its second, optional argument. It
creates and returns a new connection object.

The fd should point to a socket and be switched to the non-blocking
mode, but this isn't enforced. If the fd is invalid, the connection may
not work as expected.

The functions takes the same options as [`connect`][net-box-connect].

The function takes the ownership of the fd, i.e. the fd must not be used
or closed after this function successfully returns.

Example:

The code below connects to a Tarantool instance at port 3301 using
the [`socket`][socket] module, then wraps the socket fd in a `net.box`
connection object.

```Lua
local net = require('net.box')
local socket = require('socket')

-- Connect a socket then wrap it in a net.box connection object.
local s = socket.tcp_connect('localhost', 3301)
assert(s ~= nil)
local conn = net.from_fd(s:fd(), {fetch_schema = false})
s:detach()

conn:call('box.schema.user.info')
conn:close()
```

[net-box-connect]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/#net-box-connect
[socket]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/

Those are really nice helpers. Let's move them to the utils module so
that we can use them everywhere. Also, let's add some unit tests.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

The check_param and check_param_table Lua helpers are defined in
box/lua/schema.lua but used across the whole code base. The problem is
we can't use them in files that are loaded before box/lua/schema.lua,
like box/lua/session.lua. Let's move them to a separate source file
lua/utils.lua to overcome this limitation. Also, let's add some tests.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

* ci: support coveralls
* cmake: add code coverage support
* test: run flake8 static analysis via CMake
* test: fix E741 errors by pycodestyle
* test: fix E722 errors by pycodestyle
* test: fix E711 errors by pycodestyle
* test: fix E502 errors by pycodestyle
* test: fix E501 errors by pycodestyle
* test: fix E305 errors by pycodestyle
* test: fix E303 errors by pycodestyle
* test: fix E302 errors by pycodestyle
* test: fix E301 errors by pycodestyle
* test: fix E275 errors by pycodestyle
* test: fix E251 errors by pycodestyle
* test: fix E231 errors by pycodestyle
* test: fix E203 errors by pycodestyle
* test: fix E201 and E202 errors by pycodestyle
* test: suppress E131 errors by pycodestyle
* test: fix E128 errors by pycodestyle
* test: fix E122 errors by pycodestyle
* gdb: fix Python <assert> statement usage

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=LuaJIT submodule bump

NO_DOC=seed corpus
NO_CHANGELOG=seed corpus
NO_TEST=seed corpus

NO_DOC=testing
NO_CHANGELOG=testing

NO_DOC=seed corpus
NO_CHANGELOG=seed corpus
NO_TEST=seed corpus

Examples of IPROTO decoding issues: #3900, #1928, #6781.
Patch adds a number of fuzzing tests that covers IPROTO decoding:

- xrow_decode_auth
- xrow_decode_begin
- xrow_decode_call
- xrow_decode_dml
- xrow_decode_id
- xrow_decode_raft
- xrow_decode_sql
- xrow_decode_watch
- xrow_greeting_decode

NO_DOC=testing
NO_CHANGELOG=testing

Now the default value of a tuple field can be computed as a result of a
function call.

Closes #8609

@TarantoolBot document
Title: Document functional default field values
Product: Tarantool
Since: 3.0
Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/format/
Depends on: tarantool/doc#3520

The format clause contains, for each field, a definition within braces:
`{name='...',type='...'[,default=...][,default_func=...]}`,
where:

* (Optional) The `default_func` string value specifies the name of a
  function, that is used to generate a default value for the field. If
  `default_func` is set, the `default` value is used as the function
  argument. See [field default functions](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-default-functions).

---

Root document: https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#constraint-functions

**Field default functions**

Stored Lua function can be used to generate a default value for the tuple
field. It can take one optional parameter, and must return exactly one
value. To create a field default function, use func.create with function
body. The function must not yield.

Example:

```lua
box.schema.func.create('random_point', {
    language = 'Lua',
    body = 'function(param) return math.random(param.min, param.max) end'
})

box.schema.space.create('test')
box.space.test:create_index('pk')
box.space.test:format({
    {name = 'id', type = 'integer'},
    {name = 'latitude', type = 'number',
     default_func = 'random_point', default = {min = -90, max = 90}},
    {name = 'longitude', type = 'number',
     default_func = 'random_point', default = {min = -180, max = 180}}
})
```

```lua
tarantool> math.randomseed(os.time())
---
...

tarantool> box.space.test:insert{1}
---
- [1, 56, 38]
...
```

This structure will be extended in the next commit.

Part of #8609

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This call is redundant and does nothing. Constraints are destroyed in
space_cleanup_constraints(), which is called from space_create() and
space_delete(). Standalone tuples can't have initialized constraints.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

User id instead of function id was erroneously used there.

NO_DOC=bugfix
NO_CHANGELOG=minor