Follows up #8194

NO_CHANGELOG=ci
NO_DOC=ci
NO_TEST=ci

(cherry picked from commit 07d686dc)

After this patch, the workaround for LuaJIT profiling tests to
avoid runner's shutdown due to no space left on the disk is going
to be used in all relevant workflows. Previously, we applied it
only for the coverage test, but we noticed that other jobs may be
affected as well after tarantool/tarantool#8737 is merged.

Follows up tarantool/tarantool#7472

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit b41fc204)

Corrupted snap files that are used in the test were generated manually
using a now old Tarantool version that has an outdated system schema.
In the scope of #7149 DDL was forbidden until the system schema is
upgraded. The problem is luatest tries to grant super privileges to
the guest user (which is a DDL operation) after starting a test instance
unless they are already granted. Since the snap files don't store the
required privileges, luatest fails.

To fix this issue, let's generate corrupted snap files right in the test
using error injection.

Closes #8702

NO_DOC=test
NO_CHANGELOG=test

(cherry picked from commit 67598073)

The patch fixes two mistakes that were made while writing the test.
Firstly, we should replace metrics module with an empty table before
every test case - otherwise, metrics can be collected before the test.
Secondly, all attempts to check if a required amount of metrics was
collected is pointless, even with margin - some environments are
incredibly slow. So let's check if some metrics were collected, and check
if there are not too many of them.

Also, the patch increases some timeouts to minimize the probability of
fail due to slow environment.

Follows up #8192
Closes tarantool/tarantool-qa#312

NO_CHANGELOG=test
NO_DOC=test

(cherry picked from commit d187b142)

Bump the metrics submodule to 1.0.0-2-gea83227 version.

NO_DOC=metrics submodule bump
NO_TEST=metrics submodule bump
NO_CHANGELOG=metrics submodule bump

(cherry picked from commit 2780e8e0)

Some replication tests (linearizable_test.lua and
bootstrap_strategy_test.lua) used default test-runner to test box.cfg{}
calls which are expected to fail.

Since box.cfg{} is going to be prohibited on default test runner, let's
move such test cases into properly initialized servers.

NO_DOC=test
NO_CHANGELOG=test

(cherry picked from commit 8b10902d)

Direct call and configuration of the runner instance is prohibited. Now
if you need to test something with specific configuration use a server
instance please (see luatest.Server module).

In-scope-of tarantool/luatest#245

NO_DOC=ban calling box.cfg
NO_TEST=ban calling box.cfg
NO_CHANGELOG=ban calling box.cfg

(cherry picked from commit fc3426d8)

Drop the IPROTO_FEATURE_NAMES table and use box.iproto.feature in
iproto_feature_resolve so that we don't have to update it manually every
time we add a new feature.

Follow-up #8443
Follow-up commit b3fb883b ("iproto: export IPROTO constants to Lua
automatically")

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 606e50c4)

There are no substantial gaps in the remaining IPROTO constant enums so
there's no need in iproto_constant struct. Instead we can generate
string arrays, as we usually do. This is more flexible because it allows
us to look up a name by code. It's also consistent with iproto_type and
iproto_key names.

The only tricky part here is the iproto_flag enum because it contains
bit masks. To generate names for the flags, we add the auxiliary enum
iproto_flag_bit that contains bit numbers.

Follow-up #8443
Follow-up commit b3fb883b ("iproto: export IPROTO constants to Lua
automatically")

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 75632133)

Currently, we fill iproto_type_strs only for command codes exported to
box.stat while for the rest of command codes we have a switch-case in
the iproto_type_name function. This is ugly and error-prone because we
can easily forget to update iproto_type_name when we add a new command
code. Let's generate iproto_type_strs automatically just like we
generate iproto_key_strs.

There are a few things that should be noted here:
 - We don't generate strings for IPROTO_TYPE_ERROR and IPROTO_UNKNOWN
   because the former has a big code while the latter has a negative
   code. The only place where we need the strings is exporting IPROTO
   constants to Lua so now we just export these special codes explicitly
   there.
 - We don't generate strings for IPROTO codes reserved for vinyl because
   they aren't exported to Lua and use a different naming convention.
   As before, we have a switch-case in iproto_type_name for them.
 - We remove IPROTO_RESERVED_TYPE_STAT_MAX because it isn't a reserved
   code. Instead we define IPROTO_TYPE_STAT_MAX explicitly in the
   iproto_type enum as IPROTO_ROLLBACK + 1. This allows us to remove
   the condition that skips "RESERVED" constants from the code that
   exports IPROTO constants to Lua.
 - Before this change iproto_type_strs didn't have names for OK,
   CALL_16, and NOP, because they aren't shown in box.stat. After this
   change the names are present so we have to filter out the stat items
   explicitly in the rmean_foreach callback.

Generating iproto_type_strs makes iproto_type_constants useless so we
drop it in the scope of this patch and start using iproto_type_strs
to populate box.iproto.type.

Follow-up #8443
Follow-up commit b3fb883b ("iproto: export IPROTO constants to Lua
automatically")

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 42dc000e)

Currently, we fill vy_page_info_key_strs, vy_run_info_key_strs, and
vy_row_index_key_strs manually, which is inconvenient and error-prone.
Let's generate them automatically from enum member names, like we do
for IPROTO keys.

Note, we have to rename VY_RUN_INFO_BLOOM and VY_RUN_INFO_BLOOM_LEGACY
to VY_RUN_INFO_BLOOM_FILTER and VY_RUN_INFO_BLOOM_FILTER_LEGACY to
preserve the xlog reader output.

Still, the result isn't exactly the same:
 - An underscore is used instead of a space.
 - Strings are upper case now, not lower case, as they used to be.
 - VY_ROW_INDEX_DATA is now translated to "data", not "row index".

The key names are used for two purposes:
 - For reporting ER_INVALID_INDEX_FILE error in vy_run.c. The changes
   enumerated above don't really matter there.
 - In the xlog reader. We replace spaces with underscores anyway there
   and convert the names to the lower case so the only problem is that
   "row_index" is replaced with "data" in xlog reader output. This
   should be fine though because (a) from the context it's clear that
   the data belong to a row index section, (b) reading vinyl index files
   is only useful for debugging and introspection, and (c) the field is
   a part of vinyl internals and was never documented properly.

After this change we can remove the code replacing spaces with
underscores from the xlog reader because all IPROTO constant names
now use underscores.

Follow-up #8443
Follow-up commit b3fb883b ("iproto: export IPROTO constants to Lua
automatically")

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 96599a5b)

Currently, we fill iproto_key_strs manually, which is inconvenient and
error-prone. Let's generate it automatically from enum member names.

The result isn't exactly the same:
 - An underscore is used instead of a space.
 - Strings are upper case now, not lower case, as they used to be.
 - IPROTO_REQUEST_TYPE is now translated to  "REQUEST_TYPE", not "type".
 - IPROTO_OPS is now translated to "OPS" not "operations".

The key names are used for two purposes:
 - For reporting ER_MISSING_REQUEST_FIELD error while decoding a packet
   in xrow.c. The changes enumerated above don't really matter there.
 - In the xlog reader. Here we do need some workarounds. First, we have
   to convert the names to the lower case. Second, we have to use "type"
   and "operations" instead of generated names for IPROTO_REQUEST_TYPE
   and IPROTO_OPS. Spaces are already translated to underscores so we
   don't need to do anything about it.

Generating iproto_key_strs makes iproto_key_constants useless so we
drop it in the scope of this patch and start using iproto_key_strs to
populate box.iproto.key.

Follow-up #8443
Follow-up commit b3fb883b ("iproto: export IPROTO constants to Lua
automatically")

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 99e8abe2)

Let's merge the key value type information into IPROTO_KEYS to keep them
close together.

Follow-up #8443
Follow-up commit b3fb883b ("iproto: export IPROTO constants to Lua
automatically")

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 26b9cc86)

There's no need to add prefixes to generated iproto constant strings
(like IPROTO_, IPROTO_FEATURE_, etc) because we strip them anyway when
exporting constants to Lua. Let's drop the prefixes to cleanup the code.
Note that enum constants themselves still have the prefixes to avoid
name clashes.

Follow-up #8443
Follow-up commit b3fb883b ("iproto: export IPROTO constants to Lua
automatically")

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 37dd09af)

All the code outside of relay.cc judges about relay's liveliness looking
only at relay state. When relay->state is RELAY_FOLLOW, the relay is
considered operational.

This is not always true: for example, both relay_push_raft() and
relay_trigger_vclock_sync() are only possible after relay thread pairs
with tx via the cbus. This happens **after** the relay enters
RELAY_FOLLOW state.

Fix the possible access to uninitialized cpipe by
relay_trigger_vclock_sync(): make it a nop until the relay is paired
with tx.

Closes #7991

NO_DOC=bugfix
NO_TEST=covered by replication-luatest/linearizable_test.lua

(cherry picked from commit 0ef5e3b2)

Relay had a is_raft_enabled member with mixed meaning: firstly, it was set
to true only when relay was ready to accept messages via cbus, and
secondly, it was set to true only for replicas which need raft updates
(newer than Tarantool 2.6.0 and not anonymous).

Let's better use the flag only as an indication that the relay is ready
to accept cbus pushes, and check whether the relay needs raft updates
separately.

The flag will be reused in the following commit, which will make tx
check that relay is connected prior to sending a message to it.

Prerequisite #7991

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit b787f328)

Due to a typo raft candidate counted a vote for another node as a vote
for self in its split-vote detector. This could lead to spurious
split-vote detection in cases when another node wins elections with a bare
minimum of votes for it (exactly a quorum of votes).

Closes #8698

NO_DOC=bugfix

(cherry picked from commit 2afde5b1)

box.ctl.promote() was implemented as follows: an instance bumps the
term and marks itself a candidate, but doesn't vote for self
immediately. Instead it relies on the machinery which makes a candidate
vote for self as soon as it persists a new term.

This differs from a normal election start due to leader timeout: there
term and vote are bumped at once.

Besides, this increases probability of box.ctl.promote() resulting in
other node getting elected: if a node first broadcasts a term without a
vote, it is not considered a candidate, so other candidates might start
elections and vote for themselves.

Let's bring promote into line with automatic elections.

Closes #8497

NO_DOC=bugfix

(cherry picked from commit 17371215)

Commit c9155ac8 ("raft: persist new term and vote separately") made
the nodes persist new term and vote separately, using 2 WAL writes.
Writing the term first is needed to flush all the ongoing transactions,
so that the node's vclock is updated and can be checked against the
candidate's vclock. Otherwise it could happen that the node persists a
vote for some candidate only to find that it's vclock would actually
become incomparable with the candidate's.

Actually, this guard is not needed when checking a vote for self,
because a node can always vote for self. Besides, splitting term bump
and vote can lead to increased probability of split-vote. It may happen
that a candidate bumps and broadcasts the new term without a vote,
making other nodes vote for self. Let's go back to writing term and vote
together for self votes.

This change makes raft candidate persist term bump and vote for self in
one WAL write instead of two, so all the tests which count WAL writes or
expect 2 separate state updates for term and vote are rewritten.

Prerequisite #8497

NO_DOC=not user-visible
NO_CHANGELOG=not user-visible

(cherry picked from commit 8a124e50)

NO_DOC=changelog update
NO_TEST=changelog update

(cherry picked from commit e64568b2)

Bump the metrics submodule to 1.0.0 version.

NO_DOC=submodule bump
NO_TEST=submodule bump
NO_CHANGELOG=submodule bump

(cherry picked from commit 8bbc73ce)

Bump test-run to new version with the following improvements:

- lib: propagate test status 'skip' [1]
- Show overall progress while running [2]
- Follow test timeout for luatest [3]
- Run luatest test by pattern [4]
- Refactor command to run luatest test [5]
- Bump luatest to 0.5.7-39-g89da427 [6]
- consistent mode: fix worker's vardir calculation [7]

[1] tarantool/test-run@6fbb7fd
[2] tarantool/test-run@c5fa909
[3] tarantool/test-run@f67d523
[4] tarantool/test-run@264af05
[5] tarantool/test-run@e19bb11
[6] tarantool/test-run@3e74192
[7] tarantool/test-run@aac77f5

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 252865ce)

Currently on tuple encoding failure we raise Lua error. In many placess
the error is not handled in Lua C code and we get misc leaks. Let's
instead pass error as return value.

Note that generally speaking encoding code can raise an error on OOM.
Which will lead to leak again. Hopefully application will be killed by
OOM killer instead. Other then that we expect no more errors in the
code. If code calls a user defined callback then pcall is used (see
lua_field_inspect_ucdata for example). So the turn from raising errors
to returning error code seems the right direction.

Closes #7939

NO_DOC=bugfix

(cherry picked from commit 9f9142d6)

This will bring new ibuf_truncate method.

Part of #7939

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 45c9a096)

This commit pulls matras statistics.

Needed for https://github.com/tarantool/tarantool-ee/issues/143

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 1ea236e7)

* LJ_GC64: Make ASMREF_L references 64 bit.
* lldb: introduce luajit-lldb
* x64/LJ_GC64: Fix emit_rma().
* Limit path length passed to C library loader.

Part of #4808
Part of #8069
Part of #8516

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Currently is_nullable property of a functional index part disables the
unique property of the index. The bug is in func_index_compare(), which
compares functional keys first, and if they are equal it compares the
primary keys. This behaviour is correct only when some part of the key
is NULL (and for non-unique indexes), but for now the primary keys are
compared unconditionally. Fix this by checking for NULL key parts.

Closes #8587

NO_DOC=bugfix

(cherry picked from commit 6bcd51f9)

This patch adds a check that sqlXPrintf() does not fail in the built-in
SQL function printf(). There are two possible problems: the result might
get too large, or there might be an integer overflow because internally
int values are converted to size_t.

Closes #tarantool/security#122

NO_DOC=bugfix

(cherry picked from commit 13159230)

This patch fixes problems with INSERT INTO ... SELECT FROM optimization.
These problems appeared after 6b8acd8f, where the check became redundant,
but was not updated. Two problems arose:
1) an assertion or segmentation fault when optimization was used and the
source space does not have an index;
2) optimization can be used even if the indexes are incompatible.

The second problem does not result in changes that are user-visible, so
there is no test.

Closes #8661

NO_DOC=bugfix

(cherry picked from commit 039f714d)

Sometimes we need negative timestamps to work with dates before
1970. But seems such cases were even covered in tests. So there
wasn't any handling of negative timestamps with fraction part.
Such datetime objects had incorrect string representation (e.g.
"1963-11-22T12:30:02.-999"). This patch fixes it.

Closes #8570

NO_DOC=bugfix

(cherry picked from commit 8e7514b9)

Seems that problem code part was ported from Lua as is. But there
is some difference between modulo operator in C and in Lua. Lua
always returns positive value but in C result could be negavive.
This difference led to the case when after subtraction nsec part
of datetime object become negative that yielded weird result on
attempt to get string representation (e.g."2008-02-03T03:36:43.-100Z").
This patch fixes it.

Part of #8570

NO_DOC=bugfix
NO_CHANGELOG=see next commit

(cherry picked from commit a9c7639a)

Currently, to export IPROTO constants to Lua, we define a translation
table in C manually. As a result, some constants are forgotten and some
are exported with mistakes.

Fortunately, we have a mechanism to generate enums and its stringified
names in compile-time. Let's rewrite iproto constants using such mechanism
and generate translation table automatically.

Closes #8443
Closes #8574
Closes #8577

NO_DOC=bugfix

(cherry picked from commit b3fb883b)

We are going to generate iproto_key enum automatically to generate iproto
constants for Lua as well. This mechanism generates *_MAX constant for enum
using its name, which is in lower case. So let's convert IPROTO_KEY_MAX to
form which is appropriate for enum generator.

Part of #8443

NO_TEST=rename constant
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 82474b46)

This patch fixes a case when timestamp is passed to datetime.set
function at the same time with nsec, usec or msec.
It works fine for datetime.new but some logic was missed for set
function. Here we fix that and introduce a test.

Closes #8583

NO_DOC=bugfix

(cherry picked from commit e0855097)

With MVCC off (box.cfg.memtx_use_mvcc_engine = false), a memtx space
read view may include a dirty (not committed to WAL) record. To prevent
such records from being written to a snapshot, we sync WAL after
creating a read view for a snapshot. The problem is that it doesn't work
for long (yielding) DDL operations, such as building a new index,
because such operations yield before waiting on WAL. As a result,
a dirty DDL record may make it to a snapshot even though it may fail
eventually. To fix that, let's keep track of all yielding DDL statements
and exclude them from a read view using the memtx snapshot cleaner.

Closes #8530

NO_DOC=bug fix

(cherry picked from commit a532e375)

We don't use this functionality in net.box (sync number is always 0 for
all watch/event packets), but other clients may actually use it.

Closes #8393

@TarantoolBot document
Title: Document that `IPROTO_EVENT` has sync number

Initially the sync number sent by a client in an `IPROTO_WATCH` request
was ignored and `IPROTO_EVENT` packet didn't have a sync number. There
were complaints about it from users so we consider this to be a bug.
Now the server sends the same sync number in an `IPROTO_EVENT` packet
as the one sent by the client in the last corresponding `IPROTO_WATCH`
request.

(cherry picked from commit 99389ac6)

Add necessary wait for replication to appear on the replica, enforce
correct txn isolation to avoid memtx/vinyl discrepancy.
Remove the test from the fragile list.

Closes tarantool/tarantool-qa#292

NO_DOC=test fix
NO_CHANGELOG=test fix

(cherry picked from commit 3a220dad)

According to libFuzzer documentation [1] backslash should be escaped.

1. https://llvm.org/docs/LibFuzzer.html#dictionaries

```
$ swim_proto_meta_fuzzer -dict=swim_proto_meta_fuzzer.dict
ParseDictionaryFile: error in line 1
                "\001\000\000\004"
$ swim_proto_member_fuzzer -dict=swim_proto_member_fuzzer.dict
ParseDictionaryFile: error in line 1
                "\022\000\000\000\000\000\000\000"
```

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

(cherry picked from commit 62d03f15)

All box.cfg options values used to be plain values or arrays. Now some
options contain key-value or nested tables.

This patch allows all such options to be set through environment
variables too.

Closes #8494
Closes #8051

@TarantoolBot document
Title: Set box.cfg table value via env vars

This patch implements a way to set tables as box.cfg options value in
two different ways:
* as plain key-value table:
```
bash> export TT_LOG_MODULES=aaa=info,bbb=error && ./tarantool
Tarantool 2.10.0-beta1-1977-g2970bd57a
type 'help' for interactive help
tarantool> box.cfg{}
...
2023-04-11 07:22:10.951 [219020] main/103/interactive/box.load_cfg I> set \
'log_modules' configuration option to {"aaa":"info","bbb":"error"}
---
...

tarantool> box.cfg.log_modules.aaa
---
- info
...

tarantool> box.cfg.log_modules.bbb
---
- error
...
```

* as a table in json encoding (important: don't forget to put env var
  value into single quotes):
NO_WRAP
```
bash> export TT_METRICS='{"labels":{"alias":"mystorage"},"include":"all","exclude":["vinyl"]}' && ./tarantool
Tarantool 2.10.0-beta1-1977-g2970bd57a
type 'help' for interactive help
tarantool> box.cfg{}
...
2023-04-11 07:26:03.635 [219288] main/103/interactive/box.load_cfg I> set \
'metrics' configuration option to {"exclude":["vinyl"],"include":"all",\
"labels":{"alias":"mystorage"}}
---
...

tarantool> box.cfg.metrics.include
---
- all
...
```
NO_WRAP

(cherry picked from commit cd58c321)

An autoconf-generated configure script doesn't enable compiler
optimization flags if CFLAGS / CXXFLAGS options are set explicitly.
We started setting CFLAGS / CXXFLAGS in commit e6abe1c9
("cmake: add extra security compiler options"). As a result, users
started experiencing performance degradation issues, like the one
described in tarantool/tarantool-ee#440.

Let's set -O2 in CFLAGS / CXXFLAGS explicitly to fix that.

Closes #8606
Needed for tarantool/tarantool-ee#440

NO_DOC=build
NO_TEST=build

(cherry picked from commit 52f6ed4d)