This commit finishes implementation of the logic for overriding built-in
modules. It adds the loader, which looks into a file system for
`override.foo` module when `foo` is required. It works as for
`require()` in an application/module code as well as for `require()`
called from tarantool during its initialization.

Added replacements into the treegen testing helper.

Disabled the override loader in the LuaJIT submodule test suites. It is
done using `debug.getupvalue()`, which is a fragile way. Don't worry:
the next commit reimplements this logic.

Fixes #7774

@TarantoolBot document
Title: Built-in modules overriding

Now it is possible to override a built-in module. Everything is simple
here, in fact: if `override.foo` module exists on default search paths,
it'll replace built-in `foo` module.

Sometimes it is necessary to run tarantool with particular arguments and
verify its output. `luatest.server` provides a supervisor like
interface: an instance is started, calls box.cfg() and we can
communicate with it using net.box. Another helper,
`test.interactive_tarantool`, aims to solve all the problems around
readline console and also provides ability to communicate with the
instance interactively.

However, there is nothing like 'just run tarantool with given args and
give me its output'.

This commit adds a helper for running tarantool executable with given
parameters (cwd, env, args), catch its output and parse the output as
JSON lines.

It is used in preload_test.lua and I want to use it in a future
overload_test.lua.

The code is just moved. The comment for the function is a bit expanded.

Part of #7774

NO_CHANGELOG=a testing helper is added
NO_DOC=see NO_CHANGELOG

This commit adds a generator of Lua file tree, which generates scripts
using provided templates and filenames. This approach is used in
preload_test.lua and I'm going to use it in a future overload_test.lua.

The original code from preload_test.lua remain almost same. It is
arranged to be a module and adopted for convenient use from a luatest
based test. Also, it gains ability to provide several templates for Lua
files instead of just one for all the files.

Part of #7774

NO_CHANGELOG=a testing helper is added
NO_DOC=see NO_CHANGELOG

We're going to provide an ability to override a built-in module. The
code of an override module will work in a bit unusual circumstances: a
part of tarantool's Lua runtime is initialized, but a part isn't.

However, nothing prevents us from providing the `arg` global variable at
this stage. It used to be accessible anywhere in a user defined code.
Let's keep this property by making it accessible from an override
module.

Part of #7774

NO_TEST=It will be tested as part of the override feature.
NO_CHANGELOG=It is not possible to execute any user provided code at
             this loading stage until modules overriding will be
             implemented. So this commit doesn't change any behavior
             that a user might observe before this commit.
NO_DOC=It spreads exiting runtime guarantee to a new stage, where a user
       defined code may appear. Nothing new is introduced.

It is necessary to implement overloading of a built-in module by an
external one. `require('foo')` will work this way: check for
`override.foo` module on the file system and then load built-in `foo`.

If `foo` is already assigned to `package.loaded`, the loaders will not
be called at all, so we can't check the file system.

This commit changes a built-in module registration process: the modules
are assigned into another table called `loaders.builtin`. A special
loader is added to `package.loaders` to look into this table.

Comments on the luajit-test-init.lua changes:

* The LuaJIT tests use its own tap module, so the script unloads the
  built-in tap module. However, now it is not enough to remove it from
  `package.loaded` (it'll be loaded again at next require()). It should
  also be removed from `loaders.builtin`.

  Maybe it would be better to move this external tap module to
  <...>/override/tap.lua in a future.
* `package.loaded` may miss `internal.print` and `internal.pairs` if
  they were not required. The right way to obtain the modules is to
  require() them.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

Further commits will change how built-in modules are registered, so a
direct assignment to `_LOADED` (`package.loaded`) is not suitable
anymore.

Precisely, it would work, but would forbid to implement an override
module.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

There are two flags in the LuaJIT useful for debugging and runtime
configuration purposes: `-j` and `-b`. However, if you want to
check the same Lua code from the Tarantool, you will need to make
some adjustments in the script itself, as those flags are not
present in the Tarantool's CLI.

This patch introduces those flags to the Tarantool, so debugging is
much more convenient now. Flags are working the same as they do in
the LuaJIT.

Closes #5541

NO_DOC=http://luajit.org/running.html#opt_b
NO_DOC=http://luajit.org/running.html#opt_j

Check allocation and deallocation of different sizes of
of functional index function result.

Follow-up #6786

NO_DOC=test
NO_CHANGELOG=test

Let's use xregion_alloc instead of region_alloc, because memory
allocations from fiber region shouldn't normally fail, see #3534.

Closes tarantool/security#97

NO_DOC=bug fix
NO_TEST=shouldn't normally happen
NO_CHANGELOG=see NO_TEST

This patch introduces new sql_seq_scan_default compat option. This
compat option allows to set default value for sql_seq_scan session
setting. Note that sql_seq_scan_default compat option only affects
sessions during initialization. This means that you should set
sql_seq_scan_default before running box.cfg{} or creating a new session.

Closes #8096

NO_DOC=Already exists

This patch removes unused ck_constraint.c module.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This patch fixes potential read of uninitialized variable
in history_truncate_file()

Fixed in upstream:
https://git.savannah.gnu.org/cgit/readline.git/commit/?h=devel&id=b4ebdc06601fb54297435d2e286d901cba1cd6c6

Closes tarantool/security#95

NO_DOC=security
NO_TEST=security
NO_CHANGELOG=security

We plan to provide compat options information via the website. It was
requested by Kirill Yukhin instead of the wiki pages (and I don't mind
it).

NO_DOC=notified the documentation team in
       https://github.com/tarantool/doc/issues/3259
NO_TEST=it's just help messages
NO_CHANGELOG=those compat options were not released yet

n_ssl3_mac(): Fix possible divide by zero.
Fixed in openssl3:
https://github.com/openssl/openssl/commit/624efd2ba6f1dabdcdecf17c77bd206c421efdaf

Closes tarantool/security#90

NO_DOC=security
NO_TEST=security
NO_CHANGELOG=security

Sometimes we need to disable system triggers to perform a tricky change
in system spaces. Later we enable them back. But in case of error in
between we miss the enabling. With system triggers off the application
behaviour will be quite surpising. Let's fix it.

Follow-up #7718

NO_DOC=minor
NO_TEST=minor
NO_CHANGELOG=minor

We have a mechanism for detecting runtime memory leaks, see commit
19abfd2a ("misc: get rid of fiber_gc") so there's no need to test
it manually. The test is inherently flaky, because the size of runtime
memory depends on test case execution order so let's drop it.

Follow-up commit ec1a71ff ("box: introduce pagination to memtx_tree
and tuple position methods").

NO_DOC=test
NO_CHANGELOG=test

Now it is accessible using `require('compat')` instead of
`require('tarantool').compat`.

It follows our usual way to expose built-in modules.

It was not done initially due to doubts about projects, which have its
own `myproject/compat.lua` and have modified `package.loaded` to obtain
the project's compat as `require('compat')`.

It is not recommended and should be avoided. Project's modules should be
imported using `require('myproject.<...>')`. Otherwise it may clash with
a future built-in module or any external one.

Follows up #7000

NO_DOC=will be added to https://github.com/tarantool/doc/issues/3259

The patch extends feedback_daemon functionality - now it collects
default metrics if module metrics of required version (>= 0.16.0)
is installed. The metrics are sent as a part of feedback. Feedback
version is bumped.

Closes #8192

@TarantoolBot document
Title: Feedback metrics

Now tarantool collects default metrics and sends it with feedback if
module metrics of required version (>= 0.16.0) is installed. The process
can be tuned with following options:

- `feedback_send_metric` - boolean, collect and send metrics if true.
  Is set to true by default.

- `feedback_metrics_collect_interval` - number, period of metrics
  collection, in seconds. Is set to 60 by default.

- 'feedback_metrics_limit' - number, memory limit for metrics. Is set to
  1024 * 1024 (1 MB) by default.

If required metrics module is not installed or collect always returns nil,
metrics will not present in feedback.

New version of feedback = 8.

Fixed a bug with syslog priority for OS with non-standart `LOG_MAKEPRI`
macro.

Affected all versions for OS: Alpine (including official docker images),
OpenBSD and maybe others.

Fixes #8269

NO_DOC=bugfix
NO_TEST=exists

To support read view listing, we need to add name, id, system flag,
timestamp, vclock, and signature to struct read_view. (Previously they
were stored in Lua read view object implemented in Tarantool EE.)
Also, we have to maintain a registry of all active read views in C.
The registry pursues two goals:

 1. It's used for pushing read view objects (which may be created
    entirely in C, circumventing Lua code) to Lua.
 2. We look up a read view in the registry by id to query the read view
    status ('open' or 'closed') from Lua. This is required so that a
    read view object returned by box.read_view.list() and cached by the
    caller reports the up-to-date status. If a read view isn't found in
    the registry, then it must be closed.

Apart from the C registry of active read views, we also maintain a Lua
registry of all read views that are used in Lua. We add read view
objects returned by box.read_view.list() to this registry so that the
next call would return the same objects. The Lua registry is backed by
a weak table so that it doesn't pin a closed read view object when the
caller drops the last reference to it.

We also intend to move all read view listing machinery from the EE code
to CE (Lua registry, metatables). The EE code will need to override two
methods box.read_view.open() and box.internal.read_view_close(), which
are stubbed out in the CE code. To set the metatable for a read view
object and add it to the registry, EE version of box.read_view.open()
will use box.internal.read_view_register().

Closes #8260

@TarantoolBot document
Title: Document box.read_view.list

The new function returns an array of all active database read views.
It includes both read views created for system purposes (e.g. to make a
checkpoint or join a new replica) and read views created by application
code (this feature is available only in Tarantool Enterprise Edition,
see https://github.com/tarantool/enterprise_doc/issues/194).

Each read view is represented by a table with the following fields:
 - `id` - unique read view identifier.
 - `name` - read view name.
 - `is_system` - true if the read view is used for system purposes.
 - `timestamp` - `fiber.clock()` when the read view was opened.
 - `vclock` - `box.info.vclock` when the read view was opened.
 - `signature` - `box.info.signature` when the read view was opened.
 - `status` - 'open' or 'closed'.

Read views created by application code also have the 'space' field,
which lists all spaces available in the read view, and may be used just
like a read view object returned by `box.read_view.open()`.

The array is sorted by read view id. Since read view ids grow
monotonically, this means that the most recent read view goes last.

Example:

```
tarantool> box.read_view.list()
---
- - timestamp: 4057333.4969064
    signature: 3
    is_system: true
    status: open
    vclock: {1: 3}
    name: join
    id: 3
  - timestamp: 4057357.0874192
    signature: 6
    is_system: true
    status: open
    vclock: {1: 6}
    name: checkpoint
    id: 4
...
```

Move lbox_pushvclock from box/lua/info.c to lua/utils.h so that we can
reuse it. While we are at it, cleanup the header list ln lua/utils.c.

Needed for #8260

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This patch adds a type check of the first argument of the
tuple_field_by_path() function.

Closes tarantool/security#82

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

If `dynamic_cast` fails, then NULL is returned. Even thought
assertion is set, we cannot rely on it, as we don't use debug
version of icu. Let's check if `rbnf` variable is not NULL
explicitly.

If it somehow turned out to be NULL, then memory allocation
error will be thrown.

Closes tarantool/security#61

NO_CHANGELOG=<security fix>
NO_DOC=<security fix>
NO_TEST=<third-party security fix>

According to the business logic and assertions `idx` and `data32`
variables cannot be equal to NULL at the same time. However, we
cannot rely on assertions.

Let's check that explicitly. If this situation occurs somehow
the function exits as we cannot recover from this situation: we
don't have sources, from which values for enumeration can be taken.
Moreover, continuing of the code execution is such situation may
lead to accessing NULL if `c<limit`.

Closes tarantool/security#59

NO_CHANGELOG=<security fix>
NO_DOC=<security fix>
NO_TEST=<third-party security fix>

Closes #8263

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

Closes #8312

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

Bump the small submodule and use small_getpagesize(), which is a
wrapper over sysconf(_SC_PAGESIZE) with a proper error checking.

Closes tarantool/security#78

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

obuf_alloc(&log->zbuf, XLOG_FIXHEADER_SIZE) can potentially fail,
because there is no obuf_reserve() prior to it.

Closes tarantool/security#74

NO_DOC=bugfix
NO_CHANGELOG=bugfix
NO_TEST=no test harness for checking OOM

Commit 3fb0f7f1 ("fix gh-362 and lots of error messages fixes")
introduces an option "dont_check" that disables checks for a certain
parameter. This option is not documented anywhere and looks unusable.
This commit removes it.

Follows up #362

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

New implementation works correctly when MsgPack and Tuple expressions in
the mentioned commands contain spaces

Closes #8299

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

Closes #8265

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

See documentation below for details.

By the way fix `grant_rw_access_on__session_settings_to_role_public` of
upgrade to be idempotent. The changes made by the function are not reverted
on downgrade thus we need this fix to be able to downgrade and then
upgrade back.

Part of #7718

@TarantoolBot document
Title: Document box.schema.downgrade etc

Downgrade makes possible to run database on older versions on Tarantool.
Typical usage is next (running on 2.11.0 in the example below):

tarantool> box.schema.downgrade('2.10.0')
tarantool> box.snapshot()
tarantool> os.exit()

After this command you can run database on Tarantool version 2.10.0
or later. `box.schema.downgrade` only takes version listed in
`box.schema.downgrade_versions()` which is a list of all releases
since 2.8.2.

`downgrade` will fail if downgrading is not possible without losing
data in system spaces. This can be the case if you used functionality
introduced in newer version. For example if you used tuple constraints
intoroduced in version 2.10.0:

NO_WRAP
tarantool> box.schema.downgrade('2.8.4')
---
- error: 'builtin/box/upgrade.lua:1860: Tuple constraint is found in space ''pos_in_box''.
    It is supported starting from version 2.10.0. There are more downgrade issues.
    To list them all call box.schema.downgrade_issues.'
...
NO_WRAP

In the example above there are more then one issues with downgrade and
only first encountered is reported. To see all issues preventing
downgrade use `box.schema.downgrade_issues`:

NO_WRAP
tarantool> box.schema.downgrade_issues('2.8.4')
---
- - Tuple constraint is found in space 'pos_in_box'. It is supported starting from
    version 2.10.0.
  - Tuple constraint is found in space 'pos_in_circle'. It is supported starting from
    version 2.10.0.
...
NO_WRAP

The YAML serializer fails to detect aliases in objects returned by
the __serialize method:

tarantool> x = {}
---
...

tarantool> {a = x, b = x}
---
- a: &0 []
  b: *0
...

tarantool> setmetatable({}, {
         >     __serialize = function() return {a = x, b = x} end,
         > })
---
- a: []
  b: []
...

Fix this by scanning the object returned by the __serialize method
(called by luaL_checkfield) for references.

Closes #8240

NO_DOC=bug fix

The YAML format supports aliasing - if the same object is referenced
more than once, it will be encoded in one places with other places
being turned to references:

  tarantool> x = {}
  ---
  ...

  tarantool> {a = x, b = x}
  ---
  - a: &0 []
    b: *0
  ...

This feature is useful for dumping a space list (e.g. box.space)
to the console, because each space is referenced by name and id.

However, it doesn't work if the referenced object implements
the __serialize method:

  tarantool> x = setmetatable({}, {
           >     __serialize = function() return {} end,
           > })
  ---
  ...

  tarantool> {a = x, b = x}
  ---
  - a: []
    b: []
  ...

This happens because we check for aliases in dump_array and dump_table
(with get_yaml_anchor), after calling the __serialize method via
luaL_checkfield. Since the __serialize method may (and usually does)
return a different object on each invocation, aliases aren't detected.

Let's fix it by calling alias detection (get_yaml_anchor) before
luaL_checkfield and passing the anchor to dump_table/dump_array.

Needed for https://github.com/tarantool/tarantool-ee/issues/221
Part of #8240

NO_DOC=bug fix
NO_CHANGELOG=next commit

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-27-g42d4e24 [1]

[1] tarantool/test-run@ea517ac

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

It was possible to execute arbitrary Lua code outside of the setfenv()
environment. Example:

NO_WRAP
```lua
tarantool> box.cfg{replication_synchro_quorum = [=[N / 2 + 1]] _G.test = true --[[]=]}
tarantool> test
---
- true
...
```
NO_WRAP

How it works:

```lua
local expr = [[%s]]
```

Let's assume that `%s` is replaced by `]]<..code..>--[[`. The result is the
following (newlines are added for readability):

```lua
local expr = [[]]
<..code..>
--[[]]
```

This code is executed outside of the setfenv() protected function.

The fix is to pass the expression as an argument instead of using
`snprintf()`.

Fixes https://github.com/tarantool/security/issues/20
Fixes GHSA-74jr-2fq7-vp42

NO_DOC=bugfix

tarantool/checks [1] is a lua module (distributed as a separate rock)
for function input validation. After this patch, it will a part of
the tarantool binary.

1. https://github.com/tarantool/checks

Closes #7726
Needed for #7725

@TarantoolBot document
Title: embedded checks

Now tarantool has checks module on its board. checks is a lua module
previously distributed as a separate rock which is widely used by
many other tarantool lua modules (like cartridge, metrics and crud) and
tarantool applications. checks has its own repo with README covering its
API usage: https://github.com/tarantool/checks/blob/master/README.md .

Sometimes the return value of cfg_gets() is checked for NULL, and sometimes
not. Actually this is intended, although a bit confusing. If an option can
have a nil value, it must be checked for NULL, but if it can't be nil,
there is no sense in it. The nil value can be assigned only by default, it
cannot be set via box.cfg{}.

This patch removes the NULL checks for cfg_gets("election_mode") and
cfg_gets("election_fencing_mode") because they are not nil by default.
All other non-nil options (e.g. cfg_gets("bootstrap_strategy")) are
already implemented without the NULL checks.

Follow-up tarantool/security#75

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

`mpstream_encode_double`, apparently, has a typo: the result of
`mpstream_reserve` is checked after encoding the double into the result
buffer — fix it.

Closes tarantool/security#63

NO_DOC=bug fix
NO_CHANGELOG=see NO_TEST
NO_TEST=unlikely to happen because malloc shouldn't normally fail, and
	we don't test other mpstream methods for OOM either

 - Use tabs instead of spaces as we usually do.
 - Drop pointless coversion of (void *) to (char *).
 - Add missing comments to struct mpstream members.
 - Cleanup header list.
 - Use short licence.

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup