To achieve that, we bypass the read-only check for the _truncate system
space in box_process1() and perform it in the on_replace system trigger
instead, when we know which space is truncated.

Note, we have to move the check for insertion of a new record into the
_truncate system space before the read-only check in the on_replace
trigger callback; this is needed for initial recovery with a non-empty
_truncate space to work. While we are at it, let's use recovery_state to
make the check explicit.

Closes #5616

@TarantoolBot document
Title: Mention that temp and local spaces can be truncated in ro mode

DML operations on temporary and local spaces can be performed even if
the instance is in the read-only mode, but DDL operations (such as
`alter`) are forbidden in this case[^1]. Technically, `truncate` is
a DDL operation so initially it was forbidden as well. However, it
should be safe to perform this operation on a temporary or local space
because logically it only modifies the data stored in the space (like
DML) and it isn't replicated (see tarantool/tarantool#4263). So starting
from Tarantool 2.11.1 we allow users to truncate temporary spaces in the
read-only mode.

[^1]: https://www.tarantool.io/en/doc/latest/concepts/replication/repl_architecture/#replication-local

A read source iterator stores statements in a vy_history object using
vy_history_append_stmt(). If a statement can be referenced, it's
reference counter is incremented. If it can't, i.e. it belongs to a
memory source, it's stored in a vy_history object without referencing.

This works fine because memory sources are append-only. A problem arises
only when we get to scanning disk sources. Since we yield while reading
disk, a dump task may complete concurrently dropping the memory sources
and possibly invalidating statements stored in the iterator history.
Although we drop the history accumulated so far and restart the
iteration from scratch in this case, there's still an issue that can
result in a use-after-free bug in vy_read_iterator_next().

The problem is that we access the current candidate for the next
statement while evaluating a disk source after a disk read. If 'next'
refers to a referenced statement, it's fine, but if it refers to a
statement from a memory source, it may cause use-after-free because
the memory source may be dropped during a disk read.

To fix this issue, let's make vy_history_append_stmt() copy statements
coming from memory sources. This should be fine performance-wise because
we copied memory statements eventually in vy_history_apply() anyway,
before returning them to the user.

Note that we also have to update vy_read_iterator_restore_mem() because
it implicitly relied on the fact that 'next' coming from a memory source
can't be freed by vy_mem_iterator_restore(), which cleans up the memory
source history. Now, it isn't true anymore so we have to temporarily
take a reference to 'next' explicitly.

Closes #8852

NO_DOC=bug fix
NO_TEST=tested by ASAN

This refactoring will:

1. Move macros from a header to the source file.
Macros should be used in header only with undef to avoid redefinitions.
Undef directive is not useful since we want to use these macros in the
source file.

2. Remove `using namespace lua_grammar` from header.
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-using-directive

3. Moving serializer entry point and constant parameters into
luajit_fuzzer namespace.
It's a common practice in C++ to avoid name collisions.

4. Move serializer functions into anonymous namespace.
These functions are not a part of the interface so should have
static linkage.
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-unnamed2

5. Fix ConvertToStringDefault function.
It was logically wrong so it would generate an identifier `123` from
`*123`.

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

LuaJIT fuzzer used to stop due to timeout caused by infinite cycles and
recursions. Counters were introduced for every cycle and function to
address LuaJIT fuzzer timeouts.

The idea is to add unique counters for every cycle and function to
ensure finite code execution, if it wasn't already. For while, repeat,
for cycles, local and global named, anonymous functions, counters will
be initialized before the code generated from protobuf, and checked
in the first body statement. An entry point for the serializer was
created to count cycles and functions for counter initialization.

The idea was taken from a paper "Program Reconditioning: Avoiding
Undefined Behaviour When Finding and Reducing Compiler Bugs" [1].

Here is an example of a change in serialized code made by this commit.

Before:
```lua
while (true) do
    foo = 'bar'
end
function bar()
    bar()
end
```

After:
```lua
counter_0 = 0
counter_1 = 0
while (true) do
    if counter_0 > 5 then
        break
    end
    counter_0 = counter_0 + 1
    foo = 'bar'
end
function bar()
    if counter_1 > 5 then
        return
    end
    counter_1 = counter_1 + 1
    bar()
end
```
Protobuf structures that reproduce the timeout problem were added to
the LuaJIT fuzzer corpus.

[1] https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2023/PLDI.pdf

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

Cases in two switches had no breaks, so they were falling
through. Breaks were added to solve the problem. Code
generated by the LuaJIT fuzzer became more various.

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

In scope of the commit 82f4b4a3 ("lib/core/fiber: Increase default
stack size") the default value of fiber stack size is increased up to
512 Kb (you can find the reasons in the aforementioned commit message
and in https://github.com/tarantool/tarantool/issues/3418 description).

Some of the tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT repo
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in
https://github.com/tarantool/tarantool/issues/5782, but the crashes
still occurs while running LuaJIT tests with ASan support enabled.

To make the testing routine more convenient, FIBER_STACK_SIZE option is
introduced to Tarantool CMake machinery. One can provide the size either
by raw digits (i.e. in bytes) or using Kb/Mb suffixes for convenience.

A couple of important nits:
* If the given value is not a multiple of 4Kb, CMake machinery adjusts
  it up to the nearest one greater than this value.
* If the adjusted value is less than 512Kb, configuration fails with the
  corresponding CMake fatal error.

Follows up #3418
Relates to #5782

@TarantoolBot document
Title: introduce FIBER_STACK_SIZE configuration option

To make managing of the default fiber stack size more convenient, the
corresponding CMake option is added.

**NB**: The stack size can't be less than 512Kb and if the given value
is not a multiple of 4Kb, CMake machinery adjusts it up to the nearest
one greater than this value.

In the original commit 5f6d367c ("compat: add is_new and
is_old to options") `compat.<option_name>:is_new()` and `:is_old()`
were introduced, but by mistake they contained different usage
messages. This patch updates `:is_new()` usage msg to more
informative one from `:is_old()`.

Follows up #8807

NO_CHANGELOG=changelog from 5f6d367c is valid
NO_DOC=doc from 5f6d367c is valid

There was a little mess in exports file regarding LuaJIT symbols to be
visible from Tarantool binary, so some functions (e.g. <lua_tonumberx>
and <lua_tointegerx>) were hidden unintentionally. As a result of the
patch almost all public LuaJIT symbols are exported from Tarantool
binary; there are still several functions reasonably hidden (you can
find the rationale around the corresponding cases in the test file).

Closes #3680

@TarantoolBot document
Title: clean up LuaJIT exported functions

The list of the LuaJIT-related functions being exported (i.e. public)
from Tarantool should be updated. The actual list of the exported
symbols can be found within changeset (either extra/exports file or
related Lua test chunk).

It used to be somewhat complicated to check the effective value
of a compat option, because `<option_name>.current` could contain
'default' state.
This patch introduces helper functions that take care of that.

The following alternatives were considered:
* `compat.<option_name>.effective` - it is excessive in the presence
  if `current` and `default`, and is visible in serialization
* `compat.<option_name>.get()` - while it is a function, it does only
  half of the work required, user still has to compare result to 'new'

Closes #8807

@TarantoolBot document
Title: Add `:is_new/old()` helpers to tarantool.compat options

`compat.<option_name>.current` can be 'new', 'old' or 'default',
thus when it is default there must be an additional check if
`compat.<option_name>.default` is 'new'. It is handier to have a
helper to deal with that instead of complicated `if`:
* check if effective value is 'new' before the patch:
  ```lua
  if compat.<option_name>.current == 'new' or
          (compat.<option_name>.current == 'default' and
           compat.<option_name>.default == 'new') then
      ...
  end
  ```
* after the patch:
  ```lua
  if compat.<option_name>:is_new() then
      ...
  end
  ```

Please update [tutorial on using compat], maybe add an example to
[Listing options details].

[tutorial on using compat]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/
[Listing options details]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/#listing-options-details

This patch introduces all remaining vinyl options that have not been
introduced before.

Part of #8861

NO_DOC=Was already described before.

Prior to this patch, if a cluster configuration was not present or an
instance was not found in the configuration during a reload, the error
would begin with 'Startup failure'. Now it starts with 'Reload failure'
in these cases.

Part of #8862

NO_DOC=fix of error message

All the tests are independent and nothing prevents running them in
parallel. The option to enable the parallel running was just forgotten.

Part of #8862

NO_DOC=testing change
NO_CHANGELOG=see NO_DOC

Test uses a popen module that starts tarantool process in background
mode. Tarantool process started in background mode forks a new process
and closes a parent, after that popen loses a PID of the started process
and `ph:kill()` and `ph:terminate()` doesn't work anymore. It leads to
non-terminated tarantool processes after running the test.

Patch fixes that by running `kill` using os.execute with a PID of
tarantool process written to a pid file.

Follows up #6128

NO_CHANGELOG=fix test
NO_DOC=fix test

Fixed the implementation of the box console.
Before this fix, result of `\set language` is shared between clients
via `console.connect`, despite the fact that clients have different
`box.session.id`. Now the parameter of the selected language is stored
by each client in his own `box.session.storage`.

Fixes #8817

NO_DOC=bugfix

This patch replaces region_*() functions with xregion_*() functions.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

1. Make double-formatted fields accept integer and float values.
2. Make indexes compare the values as double if the field key type
   is FIELD_TYPE_DOUBLE.
3. Make hashers cast double key field to double before hashing, so
   we are able to insert and select any int, uint, float or double
   if their value casted to double is equal (for double keys).

Notes about tuple_compare.cc:

Since now `mp_compare_double` casts any value placed in field to
double it was renamed to `mp_compare_as_double` to not semantically
conflict with existing `mp_compare_double_*` functions.

Notes about tuple_hash.cc:

The hashee cast result is encoded in MP_DOUBLE and hashed for
backward compatibility reasons.

Since now the field hashing function (tuple_hash_field) requires
field type to hash the field correctly, a new parameter has been
introduced.

By the way added assertions to the generic `field_hash` to prevent
invalid hashing for new precompiled hashers and made
`key_hash_slowpath` static cause it's only used in this file.

Closes #7483
Closes #5933
Unblocks tarantool/crud#298

@TarantoolBot document
Title: It's not required to ffi-cast integral floating point to
double anymore.

The page describing tarantool data model states that:

> In Lua, fields of the double type can only contain non-integer
> numeric values...

If the patch is merged this isn't the case anymore, so this
statement and the code snippet below it should be updated.

Link to the document: [Data storage](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-type-details).

Affected segments:

> double. The double field type exists mainly to be equivalent
> to Tarantool/SQL’s DOUBLE data type. In msgpuck.h (Tarantool’s
> interface to MsgPack), the storage type is MP_DOUBLE and the
> size of the encoded value is always 9 bytes. In Lua, fields of
> the double type can only contain non-integer numeric values and
> cdata values with double floating-point numbers. Examples: 1.234,
> -44, 1.447e+44.
>
> To avoid using the wrong kind of values inadvertently, use
> ffi.cast() when searching or changing double fields. For example,
> instead of space_object:insert{value} use ffi = require('ffi')
> ... space_object:insert({ffi.cast('double',value)}). Example:
>
> ```
> s = box.schema.space.create('s', {format = {{'d', 'double'}}})
> s:create_index('ii')
> s:insert({1.1})
> ffi = require('ffi')
> s:insert({ffi.cast('double', 1)})
> s:insert({ffi.cast('double', tonumber('123'))})
> s:select(1.1)
> s:select({ffi.cast('double', 1)})
> ```

This update pulls the following commits:
* Add mp_read_double_lossy without direct convertibility checks
* Fix mp_read_double_lossy tests freebsd build

These commits introduce a function required to compare and hash
msgpack values of double key fields as double.

Need for #7483, #5933

NO_DOC=see the next commit
NO_CHANGELOG=see the next commit

NO_DOC=no code changes
NO_TEST=no code changes
NO_CHANGELOG=no code changes

Follows up #8724.

NO_DOC=just add a changelog entry
NO_TEST=see NO_DOC

Tarantool 3.0.0 is not going to be distributed as a set of packages for
specific distros. Instead, it is planned to have a few `deb` and `rpm`
packages with a statically compiled Tarantool binary inside, which will
work on almost any distro. Also, we drop building packages with GC32.
Now only GC64 packages will be shipped.

Static build packaging was added at #8771.

NO_DOC=ci
NO_TEST=ci

* test: fix flaky <unit-jit-parse.test.lua>
* Fix use-def analysis for vararg functions.
* Fix use-def analysis for BC_VARG.
* Fix TNEW load forwarding with instable types.
* Fix memory probing allocator to check for valid end address, too.
* Another fix for lua_yield() from C hook.
* Fix lua_yield() from C hook.
* Fix saved bytecode encapsulated in ELF objects.
* x64: Fix 64 bit shift code generation.
* Fix canonicalization of +-0.0 keys for IR_NEWREF.
* test: add utility for parsing `jit.dump`
* test: split utils.lua into several modules
* test: rewrite lj-49-bad-lightuserdata test in C
* test: rewrite misclib-sysprof-capi test in C
* test: rewrite misclib-getmetrics-capi test in C
* test: introduce utils.h helper for C tests
* test: introduce module for C tests
* test: fix setting of {DY}LD_LIBRARY_PATH variables
* build: fix build with LUAJIT_USE_GDBJIT enabled

Closes #8718
Part of #7900
Part of #8516

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

The following syntax variants are fobidden now:

* user@
* user:pass@
* user@host:port
* user:pass@host:port

Only host:port is allowed (inet URI or unix socket URI).

The idea is that we shouldn't distribute login/password information
using the configuration, so it is better to explicitly forbid such
usage of the option.

Closes #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

Before this commit all the peers of the given replicaset were obligated
to have suitable URI to connect -- either iproto.advertise.peer or one
of iproto.listen URIs.

Now replication.peers is successfully constructed if at least one
suitable URI is constructed (except URI of the given instance itself).

However, if an unknown user is used in the iproto.advertise.peer option
of any peer, it is considered as an error (startup failure), just like
before.

The testing code for unsuccessful cases of replication.peers
autobuilding is rewritten to reflect that just one bad peer doesn't
cause a startup failure anymore. Now it also needs stripping of log
messages, because the peers autobuilding tells about unsuitable URIs.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

This failover mode enables automatic leader election on a replicaset.
Assigning a leader manually (`leader` option) and assigning RO/RW mode
(`database.mode` option) are forbidden in this failover mode.

Configuration example:

```yaml
replication:
  failover: election         # !!

groups:
  group-001:
    replicasets:
      replicaset-001:
        instances:
          instance-001: {}
          instance-002: {}
          instance-003: {}
```

All the replicaset instances are so called candidates by default: they
can vote for a leader, they can be elected as a leader. However, it is
possible to set `replicaset.election_mode` to 'voter', 'off' or
'manual'. See more detailed description in the box_cfg applier comments
in the code and in the box.cfg() options documentation [1].

Note: Unlike box.cfg()'s option 'election_mode', the
replication.election_mode = "off" forces the instance to be read-only
(if failover = "election" is enabled). box.cfg() call has no cluster
configuration and it doesn't know, whether the given replicaset is
managed by the built-in election algorithm or an external coordinator.

The election failover may be used to improve cluster's availability.

[1]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-replication-election-mode

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

This failover mode allows to set a leader for a replicaset by its
instance name instead of per-instance database.mode option. For example:

```yaml
replication:
  failover: manual           # !!

groups:
  group-001:
    replicasets:
      replicaset-001:
        leader: instance-001 # !!
        instances:
          instance-001: {}
          instance-002: {}
          instance-003: {}
```

The "manual" failover mode doesn't allow several leaders in a
replicaset. If it is desired, use the "off" failover mode.

The "manual" mode doesn't perform a proper leader switching at the
moment. An administrator should resign the old leader (by unsetting the
leader option or setting it to `null`), wait till all the data arrives
to the upcoming leader and then configure it as the new leader.

The proper leader switching is subject of a future work. Also, the next
commit will add replicaset.failover = "election", which handles such
problems.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

It has the only one value for now: 'off'. Two other ones are planned:
'manual' and 'election'.

The 'off' value means exactly what is implemented now: the database.mode
option controls whether a particular instance should serve write
requests.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

If the given instance has no existing snapshot, it means that it is not
registered in the replicaset and will attempt to join as a new replica.
It has no chance to succeed if all the instances in the replicaset are
in the read-only mode: neither of them can register the new instance.

Report an error and fail the startup in the case to make the situation
clear.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

If the startup may take a long time (reads of an existing snapshot or
fetches it from a remote master), then

* If there are other instances in the replicaset, force the read-only
  mode on the database startup[^1].
* After the database startup, re-read the configuration and apply the
  fresh one (now without any RO/RW mangling).

If the startup shouldn't take a long time[^2], the configuration is
read and applied once.

[^1]: The read-only mode is not forced if there is no existing snapshot.
      It may lead to missing bootstrap leader in the replicaset. See
      details in the comment in the box_cfg applier code.
[^2]: There is the only case that we consider as such: startup of an
      instance without an existing snapshot without other instances in
      the same replicaset.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC
NO_TEST=it is not trivial to verify automatically

Enabling read-write by default is unsafe for an instance in a replicaset
with more than one instance. The only meaningful default here is
read-only.

On the other hand, a signleton instance (the only one in its replicaset)
almost always started in the read-write mode in practice.

Let's use these values as defaults for these situations.

The name of the option is changed from `rw` (boolean) to `mode` (enum
with allowed values `'ro'` and `'rw'`). We agreed on the enum after a
long controversy whether it should be `ro` or `rw`.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

In brief:

* client -- for external clients
* peer -- for connections within the cluster, in particular for replicas
* sharding -- for routers and a rebalancer

See the instance_config.lua file for the details.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

The following new syntax variants are introduced for `iproto.advertise`.

* `user@` -- use the given user, a password from the `credentials`
  section and `host:port` from `iproto.listen`
* `user:pass@` -- use the given user and password, use `host:port` from
  `iproto.listen`
* `user@host:port` -- use the given user, host and port, use a password
  from the `credentials` section

It allows to don't repeat the same information in different places of
the config.

The `test_no_advertise_*` test cases are generalized: now the same
boilerplate code is used for these and new cases.

The `test.luatest_helpers.server` util gains ability to parse all the
new `iproto.advertise` variants.

The replicaset and etcd configuration examples are updated to use the
`user@` syntax.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

There are a few nuances in using the `iproto.listen` option as a peer
URI at replicaset building (constructing a list of peer URIs).

* `iproto.listen` is a comma separated list of URIs: the single URI is
  just a special case.
* `iproto.listen` may contain `0.0.0.0` or `::` host and/or zero port.
  It is perfectly valid for listening, but doesn't give enough
  information to connect.

Let's handle `iproto.listen` appropriately: split it the URI list, walk
over the URIs and check whether the URI is suitable to connect. The
first suitable URI is used as the peer URI.

If no suitable URI is found, raise an error (it means a startup failure
if occurs at a startup).

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

Aside of a fast feedback for a user that feeds an incorrect
configuration, this validation simplifies further box_cfg applier code:
we can assume that the URI list is valid.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

The following values shouldn't be accepted as an advertise URI:

* Not an URI.
* Comma separated list of URIs.
* An URI with IPv4/IPv6 INADDR_ANY host (`0.0.0.0` or `::`).
* An URI with zero TCP port.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

We can't just pass it over and start the instance:

1. If it is an initial bootstrap (there is no data for the given
   instance yet), the instance will form its own replicaset instead of
   joining into the existing one.
2. If it is a startup of an existing instance, the local data may be
   outdated and serving requests is undesirable.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

In scope of #4693, we now create runtime tuple formats from format data, so
we need to adapt space read views accordingly: they use a field definition
array for creating tuple formats and for accessing field names and types —
for the latter case we will also allocate a field definition array
separately.

Follows-up #4693

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix
flakiness in gh_6128_background_mode_test") used a constant buffer size
in check_err_msg function. Tarantool 2.10 has a bit bigger log before a
desired message that other versions of Tarantool and it leads to a this
resulted in a truncated message ("entering the even" instead of
"entering the event loop"). Patch replaces check_err_msg()
implementation to grep_log used in luatest, it reads the whole log.

Also patch renames check_err_msg to check_msg, because "entering the
event loop" is not an error message.

Follows up #6128

NO_CHANGELOG=fix test
NO_DOC=fix test

This patch reworks the way the meta is populated. This is done to
incrementally populate the metadata, instead of set the metadata at the
end of reading data from the source. This allows to get the correct meta
in cases where getting data from the source failed.

Follow-up #8789

NO_DOC=feature not yet released
NO_TEST=tested in EE
NO_CHANGELOG=feature not yet released

The bug was fixed in the libyaml repository. This commit just updates
the submodule and adds a test.

Closes #8782

NO_DOC=bug fix

Function execution permissions should only be checked on constraint
creation.

So when the function is used to check a tuple access rights don't
have to be checked on each call for the current user.

Closes #7873

NO_DOC=bugfix