This patch fixes potential read of uninitialized variable
in history_truncate_file()

Fixed in upstream:
https://git.savannah.gnu.org/cgit/readline.git/commit/?h=devel&id=b4ebdc06601fb54297435d2e286d901cba1cd6c6

Closes tarantool/security#95

NO_DOC=security
NO_TEST=security
NO_CHANGELOG=security

We plan to provide compat options information via the website. It was
requested by Kirill Yukhin instead of the wiki pages (and I don't mind
it).

NO_DOC=notified the documentation team in
       https://github.com/tarantool/doc/issues/3259
NO_TEST=it's just help messages
NO_CHANGELOG=those compat options were not released yet

n_ssl3_mac(): Fix possible divide by zero.
Fixed in openssl3:
https://github.com/openssl/openssl/commit/624efd2ba6f1dabdcdecf17c77bd206c421efdaf

Closes tarantool/security#90

NO_DOC=security
NO_TEST=security
NO_CHANGELOG=security

Sometimes we need to disable system triggers to perform a tricky change
in system spaces. Later we enable them back. But in case of error in
between we miss the enabling. With system triggers off the application
behaviour will be quite surpising. Let's fix it.

Follow-up #7718

NO_DOC=minor
NO_TEST=minor
NO_CHANGELOG=minor

We have a mechanism for detecting runtime memory leaks, see commit
19abfd2a ("misc: get rid of fiber_gc") so there's no need to test
it manually. The test is inherently flaky, because the size of runtime
memory depends on test case execution order so let's drop it.

Follow-up commit ec1a71ff ("box: introduce pagination to memtx_tree
and tuple position methods").

NO_DOC=test
NO_CHANGELOG=test

Now it is accessible using `require('compat')` instead of
`require('tarantool').compat`.

It follows our usual way to expose built-in modules.

It was not done initially due to doubts about projects, which have its
own `myproject/compat.lua` and have modified `package.loaded` to obtain
the project's compat as `require('compat')`.

It is not recommended and should be avoided. Project's modules should be
imported using `require('myproject.<...>')`. Otherwise it may clash with
a future built-in module or any external one.

Follows up #7000

NO_DOC=will be added to https://github.com/tarantool/doc/issues/3259

The patch extends feedback_daemon functionality - now it collects
default metrics if module metrics of required version (>= 0.16.0)
is installed. The metrics are sent as a part of feedback. Feedback
version is bumped.

Closes #8192

@TarantoolBot document
Title: Feedback metrics

Now tarantool collects default metrics and sends it with feedback if
module metrics of required version (>= 0.16.0) is installed. The process
can be tuned with following options:

- `feedback_send_metric` - boolean, collect and send metrics if true.
  Is set to true by default.

- `feedback_metrics_collect_interval` - number, period of metrics
  collection, in seconds. Is set to 60 by default.

- 'feedback_metrics_limit' - number, memory limit for metrics. Is set to
  1024 * 1024 (1 MB) by default.

If required metrics module is not installed or collect always returns nil,
metrics will not present in feedback.

New version of feedback = 8.

Fixed a bug with syslog priority for OS with non-standart `LOG_MAKEPRI`
macro.

Affected all versions for OS: Alpine (including official docker images),
OpenBSD and maybe others.

Fixes #8269

NO_DOC=bugfix
NO_TEST=exists

To support read view listing, we need to add name, id, system flag,
timestamp, vclock, and signature to struct read_view. (Previously they
were stored in Lua read view object implemented in Tarantool EE.)
Also, we have to maintain a registry of all active read views in C.
The registry pursues two goals:

 1. It's used for pushing read view objects (which may be created
    entirely in C, circumventing Lua code) to Lua.
 2. We look up a read view in the registry by id to query the read view
    status ('open' or 'closed') from Lua. This is required so that a
    read view object returned by box.read_view.list() and cached by the
    caller reports the up-to-date status. If a read view isn't found in
    the registry, then it must be closed.

Apart from the C registry of active read views, we also maintain a Lua
registry of all read views that are used in Lua. We add read view
objects returned by box.read_view.list() to this registry so that the
next call would return the same objects. The Lua registry is backed by
a weak table so that it doesn't pin a closed read view object when the
caller drops the last reference to it.

We also intend to move all read view listing machinery from the EE code
to CE (Lua registry, metatables). The EE code will need to override two
methods box.read_view.open() and box.internal.read_view_close(), which
are stubbed out in the CE code. To set the metatable for a read view
object and add it to the registry, EE version of box.read_view.open()
will use box.internal.read_view_register().

Closes #8260

@TarantoolBot document
Title: Document box.read_view.list

The new function returns an array of all active database read views.
It includes both read views created for system purposes (e.g. to make a
checkpoint or join a new replica) and read views created by application
code (this feature is available only in Tarantool Enterprise Edition,
see https://github.com/tarantool/enterprise_doc/issues/194).

Each read view is represented by a table with the following fields:
 - `id` - unique read view identifier.
 - `name` - read view name.
 - `is_system` - true if the read view is used for system purposes.
 - `timestamp` - `fiber.clock()` when the read view was opened.
 - `vclock` - `box.info.vclock` when the read view was opened.
 - `signature` - `box.info.signature` when the read view was opened.
 - `status` - 'open' or 'closed'.

Read views created by application code also have the 'space' field,
which lists all spaces available in the read view, and may be used just
like a read view object returned by `box.read_view.open()`.

The array is sorted by read view id. Since read view ids grow
monotonically, this means that the most recent read view goes last.

Example:

```
tarantool> box.read_view.list()
---
- - timestamp: 4057333.4969064
    signature: 3
    is_system: true
    status: open
    vclock: {1: 3}
    name: join
    id: 3
  - timestamp: 4057357.0874192
    signature: 6
    is_system: true
    status: open
    vclock: {1: 6}
    name: checkpoint
    id: 4
...
```

Move lbox_pushvclock from box/lua/info.c to lua/utils.h so that we can
reuse it. While we are at it, cleanup the header list ln lua/utils.c.

Needed for #8260

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This patch adds a type check of the first argument of the
tuple_field_by_path() function.

Closes tarantool/security#82

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

If `dynamic_cast` fails, then NULL is returned. Even thought
assertion is set, we cannot rely on it, as we don't use debug
version of icu. Let's check if `rbnf` variable is not NULL
explicitly.

If it somehow turned out to be NULL, then memory allocation
error will be thrown.

Closes tarantool/security#61

NO_CHANGELOG=<security fix>
NO_DOC=<security fix>
NO_TEST=<third-party security fix>

According to the business logic and assertions `idx` and `data32`
variables cannot be equal to NULL at the same time. However, we
cannot rely on assertions.

Let's check that explicitly. If this situation occurs somehow
the function exits as we cannot recover from this situation: we
don't have sources, from which values for enumeration can be taken.
Moreover, continuing of the code execution is such situation may
lead to accessing NULL if `c<limit`.

Closes tarantool/security#59

NO_CHANGELOG=<security fix>
NO_DOC=<security fix>
NO_TEST=<third-party security fix>

Closes #8263

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

Closes #8312

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

Bump the small submodule and use small_getpagesize(), which is a
wrapper over sysconf(_SC_PAGESIZE) with a proper error checking.

Closes tarantool/security#78

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

obuf_alloc(&log->zbuf, XLOG_FIXHEADER_SIZE) can potentially fail,
because there is no obuf_reserve() prior to it.

Closes tarantool/security#74

NO_DOC=bugfix
NO_CHANGELOG=bugfix
NO_TEST=no test harness for checking OOM

Commit 3fb0f7f1 ("fix gh-362 and lots of error messages fixes")
introduces an option "dont_check" that disables checks for a certain
parameter. This option is not documented anywhere and looks unusable.
This commit removes it.

Follows up #362

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

New implementation works correctly when MsgPack and Tuple expressions in
the mentioned commands contain spaces

Closes #8299

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

Closes #8265

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

See documentation below for details.

By the way fix `grant_rw_access_on__session_settings_to_role_public` of
upgrade to be idempotent. The changes made by the function are not reverted
on downgrade thus we need this fix to be able to downgrade and then
upgrade back.

Part of #7718

@TarantoolBot document
Title: Document box.schema.downgrade etc

Downgrade makes possible to run database on older versions on Tarantool.
Typical usage is next (running on 2.11.0 in the example below):

tarantool> box.schema.downgrade('2.10.0')
tarantool> box.snapshot()
tarantool> os.exit()

After this command you can run database on Tarantool version 2.10.0
or later. `box.schema.downgrade` only takes version listed in
`box.schema.downgrade_versions()` which is a list of all releases
since 2.8.2.

`downgrade` will fail if downgrading is not possible without losing
data in system spaces. This can be the case if you used functionality
introduced in newer version. For example if you used tuple constraints
intoroduced in version 2.10.0:

NO_WRAP
tarantool> box.schema.downgrade('2.8.4')
---
- error: 'builtin/box/upgrade.lua:1860: Tuple constraint is found in space ''pos_in_box''.
    It is supported starting from version 2.10.0. There are more downgrade issues.
    To list them all call box.schema.downgrade_issues.'
...
NO_WRAP

In the example above there are more then one issues with downgrade and
only first encountered is reported. To see all issues preventing
downgrade use `box.schema.downgrade_issues`:

NO_WRAP
tarantool> box.schema.downgrade_issues('2.8.4')
---
- - Tuple constraint is found in space 'pos_in_box'. It is supported starting from
    version 2.10.0.
  - Tuple constraint is found in space 'pos_in_circle'. It is supported starting from
    version 2.10.0.
...
NO_WRAP

The YAML serializer fails to detect aliases in objects returned by
the __serialize method:

tarantool> x = {}
---
...

tarantool> {a = x, b = x}
---
- a: &0 []
  b: *0
...

tarantool> setmetatable({}, {
         >     __serialize = function() return {a = x, b = x} end,
         > })
---
- a: []
  b: []
...

Fix this by scanning the object returned by the __serialize method
(called by luaL_checkfield) for references.

Closes #8240

NO_DOC=bug fix

The YAML format supports aliasing - if the same object is referenced
more than once, it will be encoded in one places with other places
being turned to references:

  tarantool> x = {}
  ---
  ...

  tarantool> {a = x, b = x}
  ---
  - a: &0 []
    b: *0
  ...

This feature is useful for dumping a space list (e.g. box.space)
to the console, because each space is referenced by name and id.

However, it doesn't work if the referenced object implements
the __serialize method:

  tarantool> x = setmetatable({}, {
           >     __serialize = function() return {} end,
           > })
  ---
  ...

  tarantool> {a = x, b = x}
  ---
  - a: []
    b: []
  ...

This happens because we check for aliases in dump_array and dump_table
(with get_yaml_anchor), after calling the __serialize method via
luaL_checkfield. Since the __serialize method may (and usually does)
return a different object on each invocation, aliases aren't detected.

Let's fix it by calling alias detection (get_yaml_anchor) before
luaL_checkfield and passing the anchor to dump_table/dump_array.

Needed for https://github.com/tarantool/tarantool-ee/issues/221
Part of #8240

NO_DOC=bug fix
NO_CHANGELOG=next commit

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-27-g42d4e24 [1]

[1] tarantool/test-run@ea517ac

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

It was possible to execute arbitrary Lua code outside of the setfenv()
environment. Example:

NO_WRAP
```lua
tarantool> box.cfg{replication_synchro_quorum = [=[N / 2 + 1]] _G.test = true --[[]=]}
tarantool> test
---
- true
...
```
NO_WRAP

How it works:

```lua
local expr = [[%s]]
```

Let's assume that `%s` is replaced by `]]<..code..>--[[`. The result is the
following (newlines are added for readability):

```lua
local expr = [[]]
<..code..>
--[[]]
```

This code is executed outside of the setfenv() protected function.

The fix is to pass the expression as an argument instead of using
`snprintf()`.

Fixes https://github.com/tarantool/security/issues/20
Fixes GHSA-74jr-2fq7-vp42

NO_DOC=bugfix

tarantool/checks [1] is a lua module (distributed as a separate rock)
for function input validation. After this patch, it will a part of
the tarantool binary.

1. https://github.com/tarantool/checks

Closes #7726
Needed for #7725

@TarantoolBot document
Title: embedded checks

Now tarantool has checks module on its board. checks is a lua module
previously distributed as a separate rock which is widely used by
many other tarantool lua modules (like cartridge, metrics and crud) and
tarantool applications. checks has its own repo with README covering its
API usage: https://github.com/tarantool/checks/blob/master/README.md .

Sometimes the return value of cfg_gets() is checked for NULL, and sometimes
not. Actually this is intended, although a bit confusing. If an option can
have a nil value, it must be checked for NULL, but if it can't be nil,
there is no sense in it. The nil value can be assigned only by default, it
cannot be set via box.cfg{}.

This patch removes the NULL checks for cfg_gets("election_mode") and
cfg_gets("election_fencing_mode") because they are not nil by default.
All other non-nil options (e.g. cfg_gets("bootstrap_strategy")) are
already implemented without the NULL checks.

Follow-up tarantool/security#75

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

`mpstream_encode_double`, apparently, has a typo: the result of
`mpstream_reserve` is checked after encoding the double into the result
buffer — fix it.

Closes tarantool/security#63

NO_DOC=bug fix
NO_CHANGELOG=see NO_TEST
NO_TEST=unlikely to happen because malloc shouldn't normally fail, and
	we don't test other mpstream methods for OOM either

 - Use tabs instead of spaces as we usually do.
 - Drop pointless coversion of (void *) to (char *).
 - Add missing comments to struct mpstream members.
 - Cleanup header list.
 - Use short licence.

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

`fiber_new_system` can potentially fail — its return value for the watcher
fiber must be checked and an exception must be raised in case it does fail.

Closes tarantool/security#87

NO_CHANGELOG=<security fix>
NO_DOC=<security fix>
NO_TEST=<no test harness for checking OOM>

This patch fixes an issue with checking the result of sql_get_coll_seq()
in sql_expr_coll(). This fix only changes the error if the collation
combination is invalid because sql_get_coll_seq() sets the is_aborted
flag and error will be thrown in any case.

Closes tarantool/security#80

NO_DOC=change of returned error in rare case
NO_CHANGELOG=change of returned error in rare case

`set_client_ciphersuite` can potentially dereference NULL if the session's
cipher is not set — add a check for this condition.

Closes tarantool/security#27

NO_CHANGELOG=<security fix>
NO_DOC=<security fix>
NO_TEST=<third-party security fix>

The main cord's event loop is initialized by fiber_init(), but for some
reason successful initialization is only checked in main() after other
initialization code might try to use the event loop already.

For example, some of the loop users are coio_enable(), signal_init(),
tarantooL_lua_init(), and they are all run before we actually check that
loop is not NULL.

Closes tarantool/security#28

NO_DOC=code health
NO_TEST=code health
NO_CHANGELOG=code health

This patch replaces malloc() with xmalloc() in key_def_dup() to avoid
the possibility of skipping the malloc() return value check.

Closes tarantool/security#81

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The problem is if cat fails, because a patch file doesn't exist
PATH_COMMAND written like this won't detect it, because the last
command (patch) will complete successfully (apply existing patches
found by cat):

  cat XXX.patch YYY.patch | patch -p1

The proper way is to use PATCH_COMMAND continuation:

  PATCH_COMMAND patch -p1 -i XXX.patch
  COMMAND       patch -p1 -i YYY.patch

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

The main motivation to introduce the function is to abstract out the
built-in module registration process from certain actions like 'assign a
`package.loaded` field'. The future built-in module overriding
implementation will diverge from assigning to `package.loaded` on the
loading stage.

There is luaT_newmodule(), which creates a module table from an array of
functions written on C. The new luaT_setmodule() is convenient, when a
table of the module is created in another way: say, by loading a Lua
code from a string.

The luaT_setmodule() function is different from luaT_newmodule() in
several ways:

- accepts a module table, doesn't create it
- allows to register the same value with the same name twice
- pops the table from the Lua stack

The second point is useful, when several basic functions are written on
C, but the rest is written on Lua. So we just call luaT_newmodule() for
the C part and then luaT_setmodule() for the Lua part. If there is no
mistake and the values are the same, the second call is no-op. `fio` is
example of such module.

Unlike a simple assignment to `package.loaded` the function performs
several checks in the Debug build type, which are useful for debugging
problems during development.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

There are several reasons to do so:

1. Direct `package.loaded` assignments contradicts with future
   implementation of built-in module overriding.
2. It is common for external Lua modules, so a Lua developer used to
   follow this convention.
3. src/lua/*.lua files already return module tables instead of setting
   them to `package.loaded` directly.

This change follows the idea of the previous commit, see it for details.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

There are built-in Lua modules written in Lua in src/lua and src/box/lua
directories. Surprisingly, they are loaded in a slightly different ways.

The src/lua/*.lua modules return a module table, which is written into
`package.loaded` by the loading mechanism. This approach is usual for
external modules as well. The src/box/lua/*.lua modules write themselves
into `package.loaded` on its own.

This commit modifies the box modules loading function to store the
return values from box's built-in modules in `package.loaded` if the
module name is provided in the `lua_sources` registry.

The next commit will replace `package.loaded` assignments with
returning module tables.

The motivation behind this change is not only to make the modules
structure more natural for a Lua developer, but also concentrate
`package.loaded` assignments in a few common places. Those places will
be changes to implement modules overriding in future commits. Direct
`package.loaded` assignments would prevent ability to override a
built-in module by an external one.

The change also makes the ways to load src/lua and src/box/lua modules
closer. Now it is easier to coalesce corresponding code. It is not done
in this patchset, but can be done later to simplify the code.

This commit is pure refactoring change and it doesn't change how the
`getsources()` debugging function works. It is to be fixed separately.
See the comment in the code.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

The built-in modules overriding functionality will be implemented as a
Lua loader. It must be in effect, when built-in modules are loading, so
setup the loaders earlier.

This commit doesn't change any user visible behavior, but it marks a
minor problem with a filename assigned to the loaded Lua code (seen in
error messages and `debug.getinfo()`) to fix it later.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

The `minifio` module is created specifically to use in code that needs
file operations, but works before the `fio` module is initialized.

The loaders module will be loaded at early loading stage to make the
override loader working from very start and allow to override most of
the built-in modules.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST